How to monitor+report failures of async invoked concurrent lambdas?

Question

I have this lambda in which the first instance invokes itself 30-40 times to process data concurrently. Invocation happens using the async fire and forget Event invocation type. The very first instance obviously dies after invocation completes.

I want the first lambda to stay alive after invocation and report on number of instances triggered and if any lambdas failed through SNS notifications. So I switched to RequestResponse invocation type, but the problem here is now my lambda invokes one instance waits for the response from the instance (which can take minutes) then invokes the next one.

How can I invoke lambdas asynchronously but still get the reporting and tracking from the first instance?

Answer 1

You can report on the number of instances triggered using the first Lambda invocation.

For failures, it is not possible to use the same Lambda instance, as the error may happen much further down the line (as you mentioned, it may take minutes to complete). However, you can configure the parallelly running instances to report on the invocation status.

In fact, there is a built-in feature for this. Refer to this link. That is, you can send the invocation records (request and response information) to SQS. By consuming the queue, you should be able to get a report of success and failed instances.