'How to automate filtering files in python?
I have a folder Measurement which contain many files(file1data.csv,file2data.csv......,file10data.csv) with csv format. The different functions i am using now:
1.readfiles(Path) ---read the files from the folder
2.initialization(Data) -- some initailization
3.Data_to_analyze() ---some specific data for the filtering
4.create_subset() - making subset of the data to analyse
5.filter_file1() - function which contain filtering conditions for file 1
6.filter_file2() - function which contain filtering conditions for file 2
7.filter_file3() - function which contain filtering conditions for file 3
function 5,6,7 want to be called inside the function 4.Currenlty i am commenting the function 5 when i need to filter file 2
function 3,4 are common for all the files
I want to automate this process:
In the function 1,when it reads file 1 then it need to execute function 2-->3-->4-->5.
same for file2data,need to execute 2-->3-->4-->6 .
How to automate it?
the current program layout is as follows:
def readfiles(Path):
Filenames = glob.glob(Path + '/*.csv')
for Filename in Filenames:
#read file
initialization(Data)
def initialization(Data):
Data_to_analyze()
def Data_to_analyze():
.
.
create_subset()
def create_subset():
.
.
.
filter_file1()
#filter_file2()
def filter_file1()
.
#filtering conditions
.
.
def filter_file2()
.
#filtering conditions
.
.
I would like to improve the above code.
As i am a beginner,it would be very helpful
Solution 1:[1]
I guess you have 10 filter functions for 10 files. if yes, the answer will be useful to you.
def readfiles(Path):
Filenames = glob.glob(Path + '/*.csv')
for count, Filename in enumerate(Filenames, 1):
#read file
function_no = count
initialization(Data, function_no)
def initialization(Data, function_no):
Data_to_analyze()
create_subset()
function_map.get(function_no)()
def Data_to_analyze():
.
.
def create_subset():
.
.
.
#filter_file2()
def filter_file1()
.
#filtering conditions
.
.
final_file2df.to_excel('new_file_path', ignore_index=True)
function_map = {
1: filter_file1
2: filter_file2
3: filter_file3
# please continue the same for remain...
}
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | NANDHA KUMAR |
