'How to filter an s3 data event by object key suffix on AWS EventBridge

I've created a rule on AWS EventBridge that trigger a Sagemaker Pipeline execution. To do so, I have the following event pattern:

{
  "source": ["aws.s3"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["s3.amazonaws.com"],
    "eventName": ["PutObject", "CopyObject", "CompleteMultipartUpload"],
    "requestParameters": {
      "bucketName": ["my-bucket-name"],
      "key": [{
        "prefix": "folder/inside/my/bucket/"
      }]
    }
  }
}

I have enabled CloudTrail to log my S3 Data Events and the rule are triggering my Sagemaker Pipeline execution correctly.

The problem here is: A pipeline execution are being triggered for all put/copy of any object in my prefix. Then, I would like to trigger my pipeline execution only when a specific object is uploaded in the bucket, by I don't know its entire name.

For instance, possible object name I will have is, where this date is builded dynamically:

my-bucket-name/folder/inside/my/bucket/2021-07-28/_SUCESS

I would like to write an event pattern with something like this:

"prefix": "folder/inside/my/bucket/{current_date}/_SUCCESS"

or

"key": [{
        "prefix": "folder/inside/my/bucket/"
      }, {
        "suffix": "_SUCCESS"
      }]

I think that Event Pattern on AWS do not support suffix filtering. In the documentation, isn't clear the behavior. I have configured a S3 Event Notification using a suffix and sent the filtered notification to a SQS Queue, but now I don't know what to do with this queue in order to invoke my EventBridge rule to trigger a Sagemaker Pipeline execution.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source