Found a python bug, can someone explain? [duplicate]

Question

I wrote a function to find the different keys in two nested dictionaries. I was heavily inspired by this answer.

    def find_diff_keys(d1: dict, d2: dict, not_included_keys:list = [], path=""):
    for k in d1:
        if k in d2:
            if type(d1[k]) is dict:
                find_diff_keys(d1[k],d2[k],not_included_keys, "%s.%s" % (path, k) if path else k)
        else:
            if type(d1[k]) is dict:
                not_included_keys.append("%s" % path if path else k) # For Root Valuess
                for sub_k in d1[k]:
                    not_included_keys.append("%s.%s" % (k, sub_k))
                continue
            not_included_keys.append("%s.%s" % ("%s" % path if path else "", k))
    return not_included_keys

Now lets assume we have two nested dictionaries : a and b. If I wanted to get the diff of these two dictionaries I would run the following. Note that I did not specify an empty list as the not_included_keys parameter...

print("The following keys are included in A but not in B:")
diff_a_to_b = find_diff_keys(ini_dict_a, ini_dict_b)
pprint(diff_a_to_b)
print("")
print("The following keys are included in B but not in A:")
diff_b_to_a = find_diff_keys(ini_dict_b, ini_dict_a)
pprint(diff_b_to_a)

The output of this would be :

The following keys are included in A but not in B:
['FAIL.WEAREDOINGSOMETHING',
 'FAIL.ehh',
 'FAIL.dsa',
 'FAIL.fds',
 'FAIL.gd',
 'FAIL.ewq',
 'TESTVALUE',
 'TESTVALUE.Name',
 'TESTSUBSUB.TestValues2',
 'TESTSUBSUB.TestValues2ltaHz']

The following keys are included in B but not in A:
['FAIL.WEAREDOINGSOMETHING',
 'FAIL.ehh',
 'FAIL.dsa',
 'FAIL.fds',
 'FAIL.gd',
 'FAIL.ewq',
 'TESTVALUE',
 'TESTVALUE.Name',
 'TESTSUBSUB.TestValues2',
 'TESTSUBSUB.TestValues2ltaHz']

Now if I run the function with the empty list as parameter :

diff_a_to_b = find_diff_keys(ini_dict_a, ini_dict_b, [])
diff_b_to_a = find_diff_keys(ini_dict_b, ini_dict_a, [])

The output will be :

The following keys are included in A but not in B:
['FAIL.WEAREDOINGSOMETHING',
 'FAIL.ehh',
 'FAIL.dsa',
 'FAIL.fds',
 'FAIL.gd',
 'FAIL.ewq',
 'TESTVALUE',
 'TESTVALUE.Name',
 'TESTSUBSUB.TestValues2',
 'TESTSUBSUB.TestValues2ltaHz',
 'TESTSUBSUB.TestValues2eltaHz']

The following keys are included in B but not in A:
['PointsConfig.130324',
 'ResetAlarm.NotEnabledValue',
 'ResetAlarm.SomeValue',
 'PointsConfig.13032402.Mains.Some_small_difference']

Can someone understand why in the first approach python just copies the old list whilst in the second approach the function does what it is supposed to ?

Answer 1

Don't use mutable objects as default arguments.(lists, dicts, etc)

Do this instead.

def append_to(element, to=None):
    if to is None:
        to = []
    to.append(element)
    return to

Why? Because a list or dict for example is mutable it means you're editing the same reference throughout the code. And python creates the list when the function is defined, not when it's called, because functions are first class citizens in python.

For more info: https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments