'How to automate filtering files in python?

I have a folder Measurement which contain many files(file1data.csv,file2data.csv......,file10data.csv) with csv format. The different functions i am using now:

1.readfiles(Path) ---read the files from the folder
2.initialization(Data) -- some initailization
3.Data_to_analyze() ---some specific data for the filtering
4.create_subset()  - making subset of the data to analyse
5.filter_file1()  - function which contain filtering conditions for file 1
6.filter_file2()  - function which contain filtering conditions for file 2 
7.filter_file3()  - function which contain filtering conditions for file 3

function 5,6,7 want to be called inside the function 4.Currenlty i am commenting the function 5 when i need to filter file 2
function 3,4 are common for all the files

I want to automate this process:

  1. In the function 1,when it reads file 1 then it need to execute function 2-->3-->4-->5.

  2. same for file2data,need to execute 2-->3-->4-->6 .

How to automate it?

the current program layout is as follows:

def readfiles(Path):
    Filenames = glob.glob(Path + '/*.csv')
    for Filename in Filenames:
       #read file
       initialization(Data)

def initialization(Data):
    Data_to_analyze()

def Data_to_analyze():
    .
    .
    create_subset()

def create_subset():
    .
    .
    . 
    filter_file1()
    #filter_file2()

def filter_file1()
    .
    #filtering conditions
    .
    .
     
   

def filter_file2()
    .
    #filtering conditions
    .
    .
    

I would like to improve the above code.

As i am a beginner,it would be very helpful



Solution 1:[1]

I guess you have 10 filter functions for 10 files. if yes, the answer will be useful to you.

def readfiles(Path):
    Filenames = glob.glob(Path + '/*.csv')
    for count, Filename in enumerate(Filenames, 1):
       #read file
       function_no = count
       initialization(Data, function_no)

def initialization(Data, function_no):
    Data_to_analyze()
    create_subset()
    function_map.get(function_no)()

def Data_to_analyze():
    .
    .
    

def create_subset():
    .
    .
    . 
    
    #filter_file2()

def filter_file1()
    .
    #filtering conditions
    .
    .
       
    final_file2df.to_excel('new_file_path', ignore_index=True)

function_map = {
    1: filter_file1
    2: filter_file2
    3: filter_file3
    # please continue the same for remain...
}

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 NANDHA KUMAR