Why doesn't JSONEncoder work for namedtuples?

Question

I am unable to to dump collections.namedtuple as correct JSON.

First, consider the official example for using custom JSON serializer:

import json

class ComplexEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, complex):
            return [obj.real, obj.imag]
        # Let the base class default method raise the TypeError
        return json.JSONEncoder.default(self, obj)

json.dumps(2 + 1j, cls=ComplexEncoder)   # works great, without a doubt

Second, now consider the following example which tells Python how to JSONize a Friend object:

import json

class Friend():
    """ struct-like, for storing state details of a friend """
    def __init__(self, _id, f_name, l_name):
        self._id = _id
        self.f_name = f_name
        self.l_name = l_name

t = Friend(21, 'Steve', 'Rogerson')

class FriendEncoder(json.JSONEncoder):
    """ take a Friend object and make it truly json """
    def default(self, aFriend):
        if isinstance(aFriend, Friend):
            return {
                "id": aFriend._id,
                "f_name": aFriend.f_name,
                "l_name": aFriend.l_name,
            }
        return super(FriendEncoder, self).default(aFriend)

json.dumps(t, cls=FriendEncoder) # returns correctly JSONized string

Finally when we try to implement the same thing using namedtuples, json.dumps(t, cls=FriendEncoder) doesn't give any errors but gives the wrong output. Take a look:

import pdb
import json
from collections import namedtuple

Friend = namedtuple("Friend", ["id", 'f_name', 'l_name'])

t = Friend(21, 'Steve', 'Rogerson')

print(t)

class FriendEncoder(json.JSONEncoder):
    """ take a Friend collections.namedtuple object and make it truly json """
    def default(self, obj):
        if True:    # if isinstance(obj, Friend):
            ans = dict(obj._asdict())
            pdb.set_trace()     # WOW!! even after commenting out the if and hardcoding True, debugger doesn't get called
            return ans
        return json.JSONEncoder.default(self, obj)

json.dumps(t, cls=FriendEncoder)

The output I get is not a dict-like but rather just a list of values i.e. [21, 'Steve', 'Rogerson']

Why?

Is the default behavior such that information is lost?
Does json.dumps ignore the explicitly passed encoder?

---

Edit: by correctly jsonized namedtuple I mean that json.dumps should return data like exactly dict(nt._asdict()), where nt is a pre defined namedtuple

Answer 1

As I said in a comment, the json.JSONEncoder only calls default when it encounters an object type it doesn't already know how to serialize itself. There's a table of them in the json documentation. Here's a screenshot of it for easy reference:

Note that tuple is on the list, and since namedtuple is a subclasses of tuple, it applies to them, too. (i.e. because isinstance(friend_instance, tuple) ? True).

This is why your code for handling instances of the Friend class never gets called.

Below is one workaround — namely by creating a simple Wrapper class whose instances won't be a type that the json.JSONEncoder thinks it already knows how to handle, and then specifying a default= keyword argument function that's to be called whenever an object is encountered that it doesn't already know how to do.

Here's what I mean:

import json
from collections import namedtuple

class Wrapper(object):
    """ Container class for objects with an _asdict() method. """
    def __init__(self, obj):
        assert hasattr(obj, '_asdict'), 'Cannot wrap object with no _asdict method'
        self.obj = obj


if __name__ == '__main__':

    Friend = namedtuple("Friend", ["id", 'f_name', 'l_name'])
    t = Friend(21, 'Steve', 'Rogerson')
    print(t)
    print(json.dumps(t))
    print(json.dumps(Wrapper(t), default=lambda wrapped: wrapped.obj._asdict()))

Output:

Friend(id=21, f_name='Steve', l_name='Rogerson')
[21, "Steve", "Rogerson"]
{"id": 21, "f_name": "Steve", "l_name": "Rogerson"}

For some additional information and insights, also check out my answer to the related question Making object JSON serializable with regular encoder.