CentOS Question posted in
The official documentation can be found here.

Argument binding of python 3 lambda appears to be broken – CentOS

I’m using Python 3.6.10 in a CentOS 7 environment. I am attempting to create a list of commands to execute based on a structured specification. It seems natural and pythonic to think of this as a list of lambdas. I build the lambda list by traversing the specification. To my surprise, when I execute the result, I find that every lambda is the same because it doesn’t capture its argument at the time the lambda is created. I think that’s a bug.

Here is sample code that illustrates the behavior:

specification = {
    'labelOne': ['labelOne.one', 'labelOne.two', 'labelOne.three', 'labelOne.four', 'labelOne.five'],
    'labelTwo': ['labelTwo.one', 'labelTwo.two', 'labelTwo.three', 'labelTwo.four', 'labelTwo.five'],
    'labelThree': ['labelThree.one', 'labelThree.two', 'labelThree.three', 'labelThree.four', 'labelThree.five'],
    'labelFour': ['labelFour.one', 'labelFour.two', 'labelFour.three', 'labelFour.four', 'labelFour.five'],
    'labelFive': ['labelFive.one', 'labelFive.two', 'labelFive.three', 'labelFive.four', 'labelFive.five'],
    }

lambdas = []
for label, labelStrings in specification.items():
    for labelString in labelStrings:
        lambdaString = f"""Label: "{label}" with labelString: "{labelString}""""
        oneArgLambda = lambda someArg: print(someArg, lambdaString)
        lambdas.append(oneArgLambda)

for each in lambdas:
    each('Show: ')

I expected to see this:

Show:  Label: "labelOne" with labelString: "labelOne.one"
Show:  Label: "labelOne" with labelString: "labelOne.two"
Show:  Label: "labelOne" with labelString: "labelOne.three"
Show:  Label: "labelOne" with labelString: "labelOne.four"
Show:  Label: "labelOne" with labelString: "labelOne.five"
Show:  Label: "labelTwo" with labelString: "labelTwo.one"
Show:  Label: "labelTwo" with labelString: "labelTwo.two"
Show:  Label: "labelTwo" with labelString: "labelTwo.three"
Show:  Label: "labelTwo" with labelString: "labelTwo.four"
Show:  Label: "labelTwo" with labelString: "labelTwo.five"
Show:  Label: "labelThree" with labelString: "labelThree.one"
Show:  Label: "labelThree" with labelString: "labelThree.two"
Show:  Label: "labelThree" with labelString: "labelThree.three"
Show:  Label: "labelThree" with labelString: "labelThree.four"
Show:  Label: "labelThree" with labelString: "labelThree.five"
Show:  Label: "labelFour" with labelString: "labelFour.one"
Show:  Label: "labelFour" with labelString: "labelFour.two"
Show:  Label: "labelFour" with labelString: "labelFour.three"
Show:  Label: "labelFour" with labelString: "labelFour.four"
Show:  Label: "labelFour" with labelString: "labelFour.five"
Show:  Label: "labelFive" with labelString: "labelFive.one"
Show:  Label: "labelFive" with labelString: "labelFive.two"
Show:  Label: "labelFive" with labelString: "labelFive.three"
Show:  Label: "labelFive" with labelString: "labelFive.four"
Show:  Label: "labelFive" with labelString: "labelFive.five"

Instead, I see this:

Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"
Show:  Label: "labelFive" with labelString: "labelFive.five"

The argument binding of lambda is happening when the lambda is executed, rather than when the lambda is created. That is at least unexpected and I think arguably wrong.

I think lambda, as limited as it is, is supposed to create a CLOSURE — its entire purpose in life is to capture the state its arguments at the time it is created, so that they can be used later when the lambda is evaluated. That’s why it is called a "closure", because it closes over the value of its arguments at creation time.

What am I misunderstanding?

Tags:

3

Answers


  1. 0

    This is an alternative

    lambdas = []
for label, labelStrings in specification.items():
    for labelString in labelStrings:
        lambdaString = f"""Label: "{label}" with labelString: "{labelString}""""
        oneArgLambda = lambda someArg, lambdaString: print(someArg, lambdaString)
        lambdas.append((oneArgLambda, lambdaString))

for f, lambdaString in lambdas:
    f('Show: ', lambdaString)
    Login or Signup to reply.
  2. 0

    As you said it create CLOSURE, and closure is using stated variable in upper scope of lambdaString, but the trick is that all of your lambdas uses the same ref to lambdaString and since you change it every time it remembers the last one. For instance:

    c = ['one', 'two']
res = []

for i in range(2):
    for y in c:
        def la(x):
            print(x, y)
        res.append(la)

for la in res:
    la('Show: ')
# Show:  two
# Show:  two
# Show:  two
# Show:  two

    You just need another closure to prevent this

    c = ['one', 'two']
res = []

for i in range(2):
    for y in c:
        def closure_y(y):
            def la(x):
                print(x, y)
            return la
            
        res.append(closure_y(y))

for la in res:
    la('Show: ')
# Show:  one
# Show:  two
# Show:  one 
# Show:  two

    Full code might be

    specification = {
    'labelOne': ['labelOne.one', 'labelOne.two', 'labelOne.three', 'labelOne.four', 'labelOne.five'],
    'labelTwo': ['labelTwo.one', 'labelTwo.two', 'labelTwo.three', 'labelTwo.four', 'labelTwo.five'],
    'labelThree': ['labelThree.one', 'labelThree.two', 'labelThree.three', 'labelThree.four', 'labelThree.five'],
    'labelFour': ['labelFour.one', 'labelFour.two', 'labelFour.three', 'labelFour.four', 'labelFour.five'],
    'labelFive': ['labelFive.one', 'labelFive.two', 'labelFive.three', 'labelFive.four', 'labelFive.five'],
    }

lambdas = []
for label, labelStrings in specification.items():
    for labelString in labelStrings:
        lambdaString = f"""Label: "{label}" with labelString: "{labelString}""""

        def clousure(lambdaString):
            oneArgLambda = lambda someArg: print(someArg, lambdaString)
            return oneArgLambda

        lambdas.append(clousure(lambdaString))

for each in lambdas:
    each('Show: ')
    Login or Signup to reply.
  3. 0

    You have some other comments and answers explaining what’s going on, and your
    question is "interesting" is the sense that it forces the reader to puzzle over
    the code and the various binding issues.

    But if I saw your code from a colleague during a review, I would ask for a
    rewrite – not because I would immediately know there was a bug, but because it
    requires too much head-scratching over whether the bindings from the surrounding
    (and changing) scope will behave exactly as hoped.

    Instead, insist on stricter discipline in your programs, and thus lighten the
    cognitive load on your reader (who is you most of the time). Specifically, move
    the function creation to a truly isolated scope, and pass all of the varying
    inputs to that function-creator. That approach is solid because it will either
    work on the first try or fail entirely (if you neglect to pass all of the
    needed arguments to the function-creator).

    One way to do it:

    # A function to create another function, with non-surprising argument binding.
# We expect nothing from the surrounding scope. All business can be done locally.
def get_func(label, x):
    return lambda prefix: print(f'''{prefix} => {label}: {x}''')

# Some input data.
specification = {
    label : [label + str(n) for n in range(3)]
    for label in ('A', 'B', 'C')
}

# Use that data to create some functions.
funcs = [
    get_func(label, x)
    for label, xs in specification.items()
    for x in xs
]

# Run 'em.
for f in funcs:
    f('Show')
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search