How to automate the process of converting the list of .dat files, with their dictionaries (in seperate .dct format), to pandas data frames?

Question

The following code coverts .dat files into data frames with the use of its dictionary file in .dct format. It works well. But my problem is that I was unable to automate this process, creating a loop that takes the pairs of these files from lists is a little bit tricky, atleast for me. I could really use some help with that.

try:
    from statadict import parse_stata_dict
except ImportError:
    !pip install statadict

import pandas as pd
from statadict import parse_stata_dict

dict_file = '2015_2017_FemPregSetup.dct'
data_file = '2015_2017_FemPregData.dat'

stata_dict = parse_stata_dict(dict_file)
stata_dict

nsfg = pd.read_fwf(data_file, 
                   names=stata_dict.names, 
                   colspecs=stata_dict.colspecs)
# nsfg is now a pandas DataFrame

These are the lists of files that I would like to convert into data frames. Every .dat file has its own dictionary file:

dat_name = ['2002FemResp.dat',
'2002Male.dat'...

dct_name = ['2002FemResp.dct',
'2002Male.dct'...

Answer 1

Assuming both lists have the same length and you will want to save the csv dataframe you could try:

c=0
for dat,dct in zip(dat_name, dct_name):
    c+=1
    stata_dict = parse_stata_dict(dct)
    pd.read_fwf(dat, names=stata_dict.names, colspecs=stata_dict.colspecs).to_csv(r'path_name\file_name_{}.csv'.format(c))
    # don't forget the '.csv'! 

Also consider that if you are not using windows you need to use '/' rather than '\' in your path (or you can use os.path.join() to avoid this issue.