'lambda duration spike when new cloudwatch log stream

The problem

I have an aws lambda that runs by ~2ms of runtime duration. Each time a new cloudwatch log stream in that lambda's log-group is created, the first run of that lambda takes ~15K ms (x7500 factor).

I'm looking for a way to make the lambda permanently run in ~2ms, and avoid those duration spikes completely.

Investigation details

  1. I thought that it is related to lambda warmup on behind-the-scenes container allocation, so I added a event-bridge trigger by constant rate of every 3 minutes to that lambda. Also, I allocated provisioned-concurrency for that lambda. But it didn't solve the issue.
  2. I have another lambda, both are using the same code but running in a different flow and it doesn't happen to it.
  3. Note that currently there is no external traffic, only the event-bridge triggering which has a controlled rate.
  4. If it's relevant: I'm using python3.7 with NLTK lib in both lambdas but the delayed one is using wordnet's lemmatization (statically defined as recommended here).


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source