'extra double quotes when writing a pandas dataframe with json column

I have a dataframe like this in pandas

import pandas as pd
df = pd.DataFrame(                                                              
    [                                                                           
        ["Jay", "MS", 23],                                                      
        ["Jay", "Music", 23],                                                   
        ["Dorsey", "Music", 23],                                                
        ["Dorsey", "Piano", 23],                                                
        ["Mark", "MS", 22],                                                     
    ],                                                                          
    columns=["Name", "Course", "Age"],                                          
)                                                                               
                                                                                
new_df = (                                                                      
    df.groupby(["Name", "Age"])                                                 
    .apply(lambda x: x.drop(columns=["Name", "Age"]).to_json(orient="records")) 
    .to_frame()                                                                 
)                                                                               
                                                                                
new_df.columns = ["courses_json"]                                               
new_df = new_df.reset_index()

Output of new_df is

new_df
     Name  Age                             courses_json
0  Dorsey   23  [{"Course":"Music"},{"Course":"Piano"}]
1     Jay   23     [{"Course":"MS"},{"Course":"Music"}]
2    Mark   22                        [{"Course":"MS"}]

But when i write the dataframe to a csv file I get courses_json enclosed in double quotes twice for example the course key.

Name|Age|courses_json                                         
Dorsey|23|"[{""Course"":""Music""},{""Course"":""Piano""}]"   
Jay|23|"[{""Course"":""MS""},{""Course"":""Music""}]"         
Mark|22|"[{""Course"":""MS""}]"

I want only a single double quote around all the keys. Any ideas ?

pandas


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source

Related Questions