'Save first few lines from files to cvs that are located in different directories

my problem is that I need to read specific lines from a large number of text files from different folders and save them in one CSV file per author of files, folder structure looks like this:

main---author1--file1
    |         --....
    |         --file1000         
    ---author2--file1
    |         --....
    |         --file1000  
    |--...
    ---author27--file1
               --....
               --file1000 

I managed to read all files from the different directories and get authors names from the folder names, also read 1-3 lines from the file, but I struggle to find a way to save these lines to CSV.

import os
path = '/content/Data2/STN_INV/'
authors = os.listdir('/content/Data2/STN_INV/'); 
for auth in authors:  
    files = os.listdir(path+auth+'/');
    tmpD,tmpA=[],[]
    for file in files:
        f=open(path+auth+'/'+file, 'r')
        data = f.read()[0:3]
        print(path+auth+'/'+file, os.path.exists(path+auth+'/'+file),'size',len(data),auth)
        tmpD.append(data)
        tmpA.append(auth)

Is there an easy way to do so in google colab?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source