'When a ModelSerializer is nested, it loses its custom defined fields

I am using DRF's ModelSerializer and Nested relationships.

Problem:

The ModelSerializer, ProcedureSerializer when displayed shows all the custom fields defined on it correctly. However, when ProcedureSerializer is nested into customerSerializer, the fields of ProcedureSerializer shown are only those original fields defined in its model, and it loses the custom defined fields.

The issue is that in nested modelserializer, some of the fields defined on, and visible on listing the nested serializer do not show up when listing the parent.

Models:

class customer(models.Model):
    cstid = models.AutoField(primary_key=True, unique=True)
    name = models.CharField(max_length=35, blank=False)
    ...SNIPPED

    class Meta:
        unique_together = ["name", "mobile", "linkedclinic"]
        ordering = ['name']

    def __str__(self):
        return self.name    
    
class Procedure(models.Model):
    procid = models.AutoField(primary_key=True, unique=True)
    timestr = models.DateTimeField(default=timezone.now)
    template = models.ForeignKey(
        ProcedureTemplate, on_delete=models.CASCADE, blank=True, null=True)
    clinic = models.ForeignKey(Clinic, on_delete=models.CASCADE)
    doctor = models.ForeignKey(
        doctor, on_delete=models.SET_NULL, blank=True, null=True)
    customer = models.ForeignKey(
        customer, on_delete=models.CASCADE, null=False, related_name='procedures')

    def __str__(self):
        return f'{self.template} for {self.customer} on {self.timestr}'

    def eventtime(self):
        class_date = timezone.localtime(self.timestr)
        return class_date.strftime("%d-%m-%Y %I:%M %p")    

Serializers:

class FindingSerializer(serializers.ModelSerializer):
    class Meta:
        model = Finding
        depth = 1
        fields = [
            'fid',
            'fieldheading',
            'value',
            'linkedprocedure',
        ]
        
class ProcedureSerializer(serializers.ModelSerializer):
    finding_proc = FindingSerializer(many=True, read_only=True)
    
    class Meta:
        model = Procedure
        depth = 2
        fields = [
            'procid',
            'timestr',
            'template',
            'clinic',
            'doctor',
            'customer',
            'eventtime',
            'finding_proc',
        ]
        
class customerSerializer(ConvertNoneToStringSerializerMixin, serializers.ModelSerializer):
    def get_unique_together_validators(self):
        """Overriding method to disable unique together checks"""
        return []

    class Meta:
        model = customer
        depth = 3
        biovar_data = Biovariable_dataSerializer(
            read_only=True, many=True)  # many=True is required
        procedures = ProcedureSerializer(
            read_only=True, many=True)  # many=True is required            
        fields = [
            'cstid',
            'date_of_registration',
            'insurance_number',
            'name',
            'age',
            'ageyrs',
            'agemnths',
            'dob',
            'gender',
            'maritalstatus',
            'mobile',
            'alternate',
            'email',
            'address',
            'city',
            'occupation',
            'bloodgroup',
            'linkedclinic',
            'allergies_data',
            'biovar_data',
            'procedures'
        ]
        validators = []
        none_to_str_fields = ('insurance_number', )

If I display a procedure instance: GET /api/procedures/209/, All the required fields show up:

HTTP 200 OK
Allow: GET, PUT, PATCH, DELETE, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "procid": 209,
    "timestr": "2022-02-02T17:23:55.048232+05:30",
    "template": 1,
    "clinic": 21,
    "doctor": 4,
    "customer": 5596,
    "eventtime": "02-02-2022 05:23 PM",
    "finding_proc": [
        {
            "fid": 1423,
            "fieldheading": {
                "procid": 5,
                "name": "Nasal mucosa",
                "default": "Nasal mucosa appears normal.",
                "sortorder": 1,
                "fieldtype": "heading1",
                "template": 1
            },
            "value": "",
            "linkedprocedure": {
                "procid": 209,
                "timestr": "2022-02-02T17:23:55.048232+05:30",
                "template": 1,
                "clinic": 21,
                "doctor": 4,
                "customer": 5596
            }
        },
        {
            "fid": 1424,
            "fieldheading": {
                "procid": 6,
                "name": "Turbinates",
                "default": "Bilateral inferior, middle and superior turbinates appear normal.",
                "sortorder": 15,
                "fieldtype": "heading1",
                "template": 1
            },
            "value": "Left MT bulky, suggestive of concha bullosa",
            "linkedprocedure": {
                "procid": 209,
                "timestr": "2022-02-02T17:23:55.048232+05:30",
                "template": 1,
                "clinic": 21,
                "doctor": 4,
                "customer": 5596
            }
        },
        {
            "fid": 1425,
            "fieldheading": {
                "procid": 7,
                "name": "Septum",
                "default": "No significant nasal septal deviation.",
                "sortorder": 5,
                "fieldtype": "heading1",
                "template": 1
            },
            "value": "",
            "linkedprocedure": {
                "procid": 209,
                "timestr": "2022-02-02T17:23:55.048232+05:30",
                "template": 1,
                "clinic": 21,
                "doctor": 4,
                "customer": 5596
            }
        },
        {
            "fid": 1426,
            "fieldheading": {
                "procid": 8,
                "name": "Middle meatus",
                "default": "Bilateral middle meatus normal. No discharge, or polyps.",
                "sortorder": 50,
                "fieldtype": "heading1",
                "template": 1
            },
            "value": "",
            "linkedprocedure": {
                "procid": 209,
                "timestr": "2022-02-02T17:23:55.048232+05:30",
                "template": 1,
                "clinic": 21,
                "doctor": 4,
                "customer": 5596
            }
        },
        {
            "fid": 1427,
            "fieldheading": {
                "procid": 9,
                "name": "Inferior Meatus",
                "default": "Inferior Meatus is normal. Opening of Hasner's valve normal.",
                "sortorder": 30,
                "fieldtype": "heading1",
                "template": 1
            },
            "value": "Nasopharynx shows adenoid enlargement, occupying less than 2/3rd of the posterior choana.",
            "linkedprocedure": {
                "procid": 209,
                "timestr": "2022-02-02T17:23:55.048232+05:30",
                "template": 1,
                "clinic": 21,
                "doctor": 4,
                "customer": 5596
            }
        },
        {
            "fid": 1428,
            "fieldheading": {
                "procid": 10,
                "name": "Sphenoethmoidal recess",
                "default": "Bilateral sphenoethmoidal recesses normal",
                "sortorder": 60,
                "fieldtype": "heading1",
                "template": 1
            },
            "value": "",
            "linkedprocedure": {
                "procid": 209,
                "timestr": "2022-02-02T17:23:55.048232+05:30",
                "template": 1,
                "clinic": 21,
                "doctor": 4,
                "customer": 5596
            }
        },
        {
            "fid": 1429,
            "fieldheading": {
                "procid": 11,
                "name": "Final Impression",
                "default": "Nasal endoscopy normal",
                "sortorder": 1000,
                "fieldtype": "heading1",
                "template": 1
            },
            "value": "Adenoid hypertrophy noted",
            "linkedprocedure": {
                "procid": 209,
                "timestr": "2022-02-02T17:23:55.048232+05:30",
                "template": 1,
                "clinic": 21,
                "doctor": 4,
                "customer": 5596
            }
        }
    ]
}    

But, if I try to view the serializer customerSerializer which nests ProcedureSerializer, all the fields of ProcedureSerializer do not show up:

`GET /api/customer/5596/`

{
    "cstid": 5596,
    "name": "Some name",
    "age": "10yr",
    "ageyrs": 9,
    SNIPPED
    "linkedclinic": {},
    "procedures": [
        {
            "procid": 209,
            "timestr": "2022-02-02T17:23:55.048232+05:30",
            "template": {
                "templid": 1,
                "title": "DNE",
                "description": "Diagnostic Nasal Endoscopy",
                "clinic": {
                    "clinicid": 10,
                    "SNIP
                }
            },
            "clinic": {
                "clinicid": 21,
                SNIP
            },
            "doctor": {
                "docid": 4,
                SNIP
            },
            "customer": {
                "cstid": 5596,
                "date_of_registration": "2021-12-25",
                SNIP
                }
            }
        }
    ]
}

Thus, it is noted that two of those fields which are there in ProcedureSerializer, including eventtime (which is actually a model method) and finding_proc (which is a nested serializer) are missing from ProcedureSerializer.

repr(ProcedureSerializer()) gives:

"ProcedureSerializer():\n    procid = IntegerField(read_only=True)\n    timestr = DateTimeField(required=False)\n    template = PrimaryKeyRelatedField(allow_null=True, queryset=ProcedureTemplate.objects.all(), required=False)\n    clinic = PrimaryKeyRelatedField(queryset=Clinic.objects.all())\n    doctor = PrimaryKeyRelatedField(allow_null=True, queryset=doctor.objects.all(), required=False)\n    customer = PrimaryKeyRelatedField(queryset=customer.objects.all())\n    eventtime = ReadOnlyField()\n    finding_proc = FindingSerializer(many=True, read_only=True):\n        fid = IntegerField(read_only=True)\n        fieldheading = NestedSerializer(read_only=True):\n            procid = IntegerField(read_only=True)\n            name = CharField(max_length=200)\n            default = CharField(max_length=1000)\n            sortorder = IntegerField(max_value=2147483647, min_value=-2147483648, required=False)\n            fieldtype = ChoiceField(choices=(('heading1', 'Heading1'), ('heading2', 'Heading2')), required=False)\n            template = PrimaryKeyRelatedField(queryset=ProcedureTemplate.objects.all())\n        value = CharField(allow_blank=True, max_length=5000, required=False)\n        linkedprocedure = NestedSerializer(read_only=True):\n            procid = IntegerField(read_only=True)\n            timestr = DateTimeField(required=False)\n            template = PrimaryKeyRelatedField(allow_null=True, queryset=ProcedureTemplate.objects.all(), required=False)\n            clinic = PrimaryKeyRelatedField(queryset=Clinic.objects.all())\n            doctor = PrimaryKeyRelatedField(allow_null=True, queryset=doctor.objects.all(), required=False)\n            customer = PrimaryKeyRelatedField(queryset=customer.objects.all())"

What is happening here? How can I get all fields in the nested parent serializer?



Solution 1:[1]

The problem occured because I included the reference to the nested serializer within the Meta class instead of in the main class. The problem was pointed out by @Smixi at Github (All credits to him. I am posting the answer so that it may help others who may make the same mistake as me)

The correct code would be:

class customerSerializer(ConvertNoneToStringSerializerMixin, serializers.ModelSerializer):
    procedures = ProcedureSerializer(
            read_only=True, many=True)  # many=True is required
    biovar_data = Biovariable_dataSerializer(
            read_only=True, many=True)  # many=True is required
    
    def get_unique_together_validators(self):
        """Overriding method to disable unique together checks"""
        return []

    class Meta:
        model = customer
        depth = 3
        fields = [
            'cstid',
            'date_of_registration',
            'insurance_number',
            'name',
            'age',
            'ageyrs',
            'agemnths',
            'dob',
            'gender',
            'maritalstatus',
            'mobile',
            'alternate',
            'email',
            'address',
            'city',
            'occupation',
            'bloodgroup',
            'linkedclinic',
            'allergies_data',
            'biovar_data',
            'procedures'
        ]
        validators = []

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Joel G Mathew