How do I measure the time it takes a for loop to run to completion?

Question

I'm trying to debug the length of time for i in image_list: takes to complete, there are two possibilities as to how this loop ends. Either true or false. True occurs if it finds a positive image match, and false occurs if it doesn't.

I'm using:

loop_time = time.time()

And:

# debug the loop rate
print('FPS {}'.format(1 / (time.time() - loop_time)))
loop_time = time.time()  

To debug the time. The issue I have is in the logic below. If I leave the #debug code section where it is, I only get the time of the last iteration if match_points is true.

If I move the #debug code section above match_points I get the time of each individual iteration rather than the time it takes for the whole for loop to run.

I want to know, how long it takes for for i in image_list to run, regardless of where it stops running. How do I achieve this?

def detection(image_list):

    # start timer
    loop_time = time.time()

    for i in image_list:  
        needle_img = i[0]
           
        match_points = #someMethod
        match_image = #someMethod

        if match_points:
            match_image = #someMethod
          
            # debug the loop rate
            print('FPS {}'.format(1 / (time.time() - loop_time)))
            loop_time = time.time()     

            return True      
    return False

Answer 1

I would use a boolean as the return value to only have a single return statement. I find it easier to read that way. And use a break statement instead.

def detection(image_list):
    res = False
    # start timer
    loop_time = time.time()

    for i in image_list:  
        needle_img = i[0]
       
        match_points = #someMethod
        match_image = #someMethod

        if match_points:
            match_image = #someMethod
            res = True
            break
    # debug the loop rate
    print('FPS {}'.format(1 / (time.time() - loop_time)))       
    return res

Answer 2

A nice tool is tqdm. It will display a progress bar, along with loops per second, etc. It can be heavily customized as well.

In your code, you would simply change:

from tqdm import tqdm

# ...

for i in tqdm(image_list, unit='F'):
    # ...

At the end, you'd get a nice summary, e.g. (in a Jupyter notebook):

Edit If you don't want to display the progress during the iteration, but only as a summary at the end, you have several options. One of them is:

it = tqdm(image_list, unit='F', delay=1e9)
for i in it:
    # do stuff

# and at the end:
it.display()

Answer 3

You are looking for the standard library cProfile (docs here)

Answer 4

If you are looking for the time it would take if all the images were computed you can estimate that time

def detection(image_list):
    # start timer
    t0 = time.time()
    nbLoop = len(image_list)

    for i, img in enumerate(image_list):
        needle_img = img[0]

        match_points =  # someMethod
        match_image =  # someMethod

        if match_points:
            match_image =  #    someMethod

            # debug the loop rate
            print('FPS {}'.format(1 / (time.time() - loop_time)))
            loop_time = time.time()

            return True

        percent_done = (i+1) / nbLoop
    ts = time.time() - t0
    time_remaining = ts / (percent_done ) -ts
    print('percent_done',percent_done, 'time remaining',time_remaining, 'elapse time',ts )
    return False