What do lambda function closures capture?

Question

Recently I started playing around with Python and I came around something peculiar in the way closures work. Consider the following code:

adders=[None, None, None, None]

for i in [0,1,2,3]:
   adders[i]=lambda a: i+a

print adders[1](3)

It builds a simple array of functions that take a single input and return that input added by a number. The functions are constructed in for loop where the iterator i runs from 0 to 3. For each of these numbers a lambda function is created which captures i and adds it to the function's input. The last line calls the second lambda function with 3 as a parameter. To my surprise the output was 6.

I expected a 4. My reasoning was: in Python everything is an object and thus every variable is essential a pointer to it. When creating the lambda closures for i, I expected it to store a pointer to the integer object currently pointed to by i. That means that when i assigned a new integer object it shouldn't effect the previously created closures. Sadly, inspecting the adders array within a debugger shows that it does. All lambda functions refer to the last value of i, 3, which results in adders[1](3) returning 6.

Which make me wonder about the following:

• What do the closures capture exactly?
• What is the most elegant way to convince the lambda functions to capture the current value of i in a way that will not be affected when i changes its value?

Answer 1

you may force the capture of a variable using an argument with a default value:

>>> for i in [0,1,2,3]:
...    adders[i]=lambda a,i=i: i+a  # note the dummy parameter with a default value
...
>>> print( adders[1](3) )
4

the idea is to declare a parameter (cleverly named i) and give it a default value of the variable you want to capture (the value of i)

Answer 2

For completeness another answer to your second question: You could use partial in the functools module.

With importing add from operator as Chris Lutz proposed the example becomes:

from functools import partial
from operator import add   # add(a, b) -- Same as a + b.

adders = [0,1,2,3]
for i in [0,1,2,3]:
    # store callable object with first argument given as (current) i
    adders[i] = partial(add, i) 

print adders[1](3)

Answer 3

Consider the following code:

x = "foo"

def print_x():
    print x

x = "bar"

print_x() # Outputs "bar"

I think most people won't find this confusing at all. It is the expected behaviour.

So, why do people think it would be different when it is done in a loop? I know I did that mistake myself, but I don't know why. It is the loop? Or perhaps the lambda?

After all, the loop is just a shorter version of:

adders= [0,1,2,3]
i = 0
adders[i] = lambda a: i+a
i = 1
adders[i] = lambda a: i+a
i = 2
adders[i] = lambda a: i+a
i = 3
adders[i] = lambda a: i+a

Answer 4

Here's a new example that highlights the data structure and contents of a closure, to help clarify when the enclosing context is "saved."

def make_funcs():
    i = 42
    my_str = "hi"

    f_one = lambda: i

    i += 1
    f_two = lambda: i+1

    f_three = lambda: my_str
    return f_one, f_two, f_three

f_1, f_2, f_3 = make_funcs()

What is in a closure?

>>> print f_1.func_closure, f_1.func_closure[0].cell_contents
(<cell at 0x106a99a28: int object at 0x7fbb20c11170>,) 43 

Notably, my_str is not in f1's closure.

What's in f2's closure?

>>> print f_2.func_closure, f_2.func_closure[0].cell_contents
(<cell at 0x106a99a28: int object at 0x7fbb20c11170>,) 43

Notice (from the memory addresses) that both closures contain the same objects. So, you can start to think of the lambda function as having a reference to the scope. However, my_str is not in the closure for f_1 or f_2, and i is not in the closure for f_3 (not shown), which suggests the closure objects themselves are distinct objects.

Are the closure objects themselves the same object?

>>> print f_1.func_closure is f_2.func_closure
False

Answer 5

In answer to your second question, the most elegant way to do this would be to use a function that takes two parameters instead of an array:

add = lambda a, b: a + b
add(1, 3)

However, using lambda here is a bit silly. Python gives us the operator module, which provides a functional interface to the basic operators. The lambda above has unnecessary overhead just to call the addition operator:

from operator import add
add(1, 3)

I understand that you're playing around, trying to explore the language, but I can't imagine a situation I would use an array of functions where Python's scoping weirdness would get in the way.

If you wanted, you could write a small class that uses your array-indexing syntax:

class Adders(object):
    def __getitem__(self, item):
        return lambda a: a + item

adders = Adders()
adders[1](3)

Answer 6

One way to sort out the scope of i is to generate the lambda in another scope (a closure function), handing over the necessary parameters for it to make the lambda:

def get_funky(i):
    return lambda a: i+a

adders=[None, None, None, None]

for i in [0,1,2,3]:
   adders[i]=get_funky(i)

print(*(ar(5) for ar in adders))

giving 5 6 7 8 of course.