'How to arrive at a number for the maximum number of API calls that can me made concurrently in your machine?

This is a generic question, in the sense that it does not restrict to certain platform or a prgramming language, though I am writing Clojure and planning to use core.async. Here's the scenario

  1. I have a CSV file with 10 million lines to be processed.
  2. As I parse each line, I have to "send a request over the network" using the data in that line.
  3. The response from the network call takes time with a median of 400 ms.
  4. Assume that the actual CPU processing of the data is negligible (we read the line, make an API call and append the result to another file)
  5. My machine has 4 cores.

Using the above data, how do I arrive at the max number of concurrenct Network calls possible on my machine which allows me to efficiently use all resources without blowing up the machine?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source