'Print lines from multiple archived log files between two given dates

Basically, I have a folder where absolutely huge log files are archived every day. 3 log files are created per day more precisely.

I'm working on a Python script where the user has to enter a date in YYYYMMDD format in order to locate the 3 files that have been created on this date, then he enters a start and end time in hour, minute and seconds and an IP address. And the script will read the content of the 3 .gz files in the given interval and print the lines where the IP address is present.

import re
import os
import glob
import gzip
from datetime import datetime, timedelta

date_entry = raw_input('Give a date in format YEAR, MONTH, DAY \n')
date = datetime.strptime(re.sub("\s+", "", date_entry), "%Y,%m,%d").date()

path = "/applis/tacacs/log/"


list_of_files = [
    file for file in glob.glob(path + '*.gz')
    if date == datetime.fromtimestamp(os.path.getmtime(file)).date()
]

debut = raw_input('Start (Hour:Minute:Second) \n')
date_debut = datetime.strptime(debut, '%H:%M:%S').date()
fin = raw_input('End (Hour:Minute:Second) \n')
date_fin = datetime.strptime(fin, '%H:%M:%S').date()
Adresse_IP = raw_input('IP Address \n')

filedata = {list_of_files: gzip.open(list_of_files, 'r') for list_of_files in list_of_files}
for line in filedata:
        if date_debut < date_fin:
                re.search(Adresse_IP, line)
                print line

But it doesn't work, when I try the script, nothing happens. There is no error or anything. Even though the IP address I gave should be in the range for at least one of the file I have given. This code manages to find the three files associated with the given day. So the first 19 lines are OK I suppose.

Since I don't have any errors I can't figure out if I have no errors because it can't open the 3 files, or if it is the two dates given in hours, minutes, seconds that aren't correct.

I'm a big beginner on Python so if someone could enlighten me it would help me a lot.

EDIT: I did some checking this morning and the hour, minute and second range doesn't seem to be the problem.

Which means this line is wrong as I suspected, but I don't see why exactly.

filedata = {list_of_files: gzip.open(list_of_files, 'r') for list_of_files in list_of_files}

Anyone have an idea how to open files in a list? I can only find results for files in a directory.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source