AWS Cloudwatch Insights how to query using multiple log groups

Question

Reading the documentation seems that is possible in AWS Cloudwatch to run queries with multiple log groups but I can not see any example on how to do it.

I would like to join two log groups using common attributes and select some logs using filter.

Has anyone run into the same problem and found a solution? Thanks in advance.

Answer 1

You can install Athena Cloudwatch Connector. Before that, you will able to query in Athena using each log group like a table.

select s.message, c.message, split_part(c.message, '/',3)
from "lambda:athena-cloudwatch-catalog"."/aws/lambda/<LOG_GROUP_1>".all_log_streams c,
"lambda:athena-cloudwatch-catalog"."/aws/lambda/<LOG_GROUP_2>".all_log_streams s
where 
s.message like split_part(c.message, '/',3) and
c.message like '%S3%';

Answer 2

If you are talking about AWS Console, the option to select multiple log groups is a checkbox and you can select over 20 log groups.

If you refer to the AWS CLI, you can also choose multiple log groups. First create the query (in one or more log groups) (https://docs.aws.amazon.com/cli/latest/reference/logs/start-query.html) and then get the results (https://docs.aws.amazon.com/cli/latest/reference/logs/get-query-results.html)

Example:

Query creation with multiple log groups:

aws logs start-query --log-group-names "/aws/apigateway/welcome" "/aws/lambda/Test01" 
--start-time 1598936400000 --end-time 1611464400000 --query-string "fields @timestamp, 
@message"

Recover result from created query:

aws logs get-query-results --query-id <query_id>