Load CSV files in dictionary then make data frame for each csv file Python

Question

I have multiple csv files I was able to load them as data frames into dictionary by using keywords

# reading files into dataframes
csvDict = {}
for index, rows in keywords.iterrows():
    eachKey = rows.Keyword
    csvFile = "SRT21" + eachKey + ".csv"
    csvDict[eachKey] = pd.read_csv(csvFile)

Now I have other functions to apply on every data frame's specific column.

on a single data frame the code would be like this

df['Cat_Frames'] = df['Cat_Frames'].apply(process)

df['Cat_Frames'] = df['Cat_Frames'].apply(cleandata)

df['Cat_Frames'] = df['Cat_Frames'].fillna(' ')

My question is how to loop through every data frame in the dictionary to apply those function?

I have tried

for item in csvDict.items():
    df = pd.DataFrame(item)
    df

and it gives me empty result

any solution or suggestion?

Answer 1

You can chain the applys like this:

for key, df in csvDict.items():
    df['Cat_Frames'] = df['Cat_Frames'].apply(process).apply(cleandata).fillna(' ')

Answer 2

Items returns a tuple of key/value, so you should make your for loop actually say:

for key, value in csvDict.items():
  print(df)

also you need to print the df if you aren't in jupyter

Answer 3

for key, value in csvDict.items():
   df = pd.DataFrame(value)
   df

I think this is how you should traverse the dictionary.

Answer 4

When there is no processing of data from one data set/frame involving another data set, don't collect data sets.
Just process the current one, and proceed to the next.

The conventional name for a variable receiving an "unpacked value" not going to be used is _:

for _, df in csvDict.items():
    df['Cat_Frames'] = df['Cat_Frames'].apply(process).apply(…

- but why ask for keys to ignore them? Iterate the values:

for df in csvDict.values():
    df['Cat_Frames'] = df['Cat_Frames'].apply(process).apply(…