How to time a bunch of things in a bash script

Question

I want to wrote a script that runs a script/service (curls from a service) with different parameters. Now, I want to time each query and store it in a file? How can I do this?

#! /bin/bash
input="/home/ubuntu/flowers"

while IFS= read -r line
do
  time myservice 'get?flower="$line"' 
done < "$input"

I also tried :

cat flowers | xargs -I {} time myservice "get?flower={}"  | jq -c '.[] | {flower}' 

My output looks something like

/usr/local/lib/python2.7/dist-packages/gevent/builtins.py:96: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  result = _import(*args, **kwargs)
{"flower":"daffodil"}
{"flower":"daffodil"}
{"flower":"daffodil"}
{"flower":"daffodil"}
0.47user 0.07system 0:10.49elapsed 5%CPU (0avgtext+0avgdata 65432maxresident)k

OR contains stuff like

Myservice 10.#.#.#:7092 returned bad json for get?flower=lilly
Myservice 10.#.#.#:7092 returned bad json for get?flower=lilly
Myservice 10.#.#.#:7092 returned bad json for get?flower=lilly
Failed to connect (or too slow) on 10.#.#.#2:7092 timed out
Timeout Error: ('10.#.#.#', 7092)
Failed to connect (or too slow) on 10.#.#.#:7092 
Timeout Error: ('10.#.#.#', 7092)
Failed to connect (or too slow) on 10.#.#.#:7092 
Timeout Error: ('10.#.#.#', 7092)

which I would like to skip.

I know I can do a clean up later if there isn't a simple way to do this.

I want a file that is something like

lilly 0.91
hibiscus 0.93

Where the number is the time on the userside.

Answer 1

If all you are looking for is the amount of time each query takes, and you don't care about the output from myservice, then you can redirect it to /dev/null and ignore it.

Measuring the time is a little more tricky. You cannot redirect output from the time command to a different place than the command it is running. So it is better to use other approaches. Bash has an internal variable "SECONDS" that can be used to measure elapsed time, but I think you want more granularity than that. So you should use the 'date' command instead.

You will also need to use bc (or similar) to do floating point arithmetic.

Also, if the myservice command handles failures correctly (i.e. returns a non-zero value upon failure), then you can also handle failures cleanly.

#!/bin/bash

input_file="/home/ubuntu/flowers"

while IFS= read -r line; do
  start_time=$(date +%s.%N)  # 
  myservice 'get?flower="$line"' > /dev/null 2>&1
  return_value=$?
  end_time=$(date +%s.%N)

  elapsed_time=$(echo "scale=3; ${end_time} - ${start_time}" | bc)      

  if [ ${return_value} -eq 0 ]; then
    echo "${line}: ${elapsed_time}"
  else
     echo "${line}: Failed" 
  fi
done < "${input_file}"

The %s.%3N format string in the date commands means:

• %s Seconds since 1 Jan 1970
• . The '.' character
• %N Nanoseconds

The scale=3 input to the bc command tells it to output 3 decimal places.