'How do I iterate through nested pandas dataframe by column?

Problem: I have a 1 row dataframe dfA whose column length can vary, within each cell contains a json object. All fields in each json object are the same.

I'm trying to write code that loops through each cell, parses the JSON object, and loads ALL data into 1 single dataframe dfB, so if I have 5 columns in dfA then the final output of dfB should have 5 rows

I have the code written to where I can manually enter the column number of dfA and dfB outputs correctly:

dataA = dfA[0].values.tolist()
dfB = pd.json_normalize(dataA)
dfB

In the above example, the code pulls the 1st JSON object of dfA and correctly outputs a 1 row dataframe

I've Tried:

for column in dfA:
  dataA = dfA[column].values.tolist()
  dfB = pd.json_normalize(dataA)

This outputs just the last cell in dfA

I've tried different variations of this and either get the 1st cell, last cell, or the original dfA

Please help, thank you!

EDIT:

dfA:
0 1 2 3 4
0 {'id':'testID434556'... {'id':'testID568912'... {'id':'testID652154'... {'id':'testID756298'... {'id':'testID812362'...
dataA = dfA[0].values.tolist()
dfB = pd.json_normalize(dataA)
dfB
id field2 field3 field4 field5
0 testID434556 field2data field3data field4data field5data

Desired Output:

id field2 field3 field4 field5
0 testID434556 field2data field3data field4data field5data
1 testID568912 field2data field3data field4data field5data
2 testID652154 field2data field3data field4data field5data
3 testID756298 field2data field3data field4data field5data
4 testID812362 field2data field3data field4data field5data


Solution 1:[1]

dfA[0] only gets the first column. You should convert you dataframe first row to list then feed the result to pd.json_normalize()

pd.json_normalize((dfA.iloc[0, :].values))

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Ynjxsjmh