Intro
S3 Buckets emit Events which allows you to build serverless and event driven infrastructure. Events in AWS allow us to connect various Services without gluing them together with Code. S3 generated events can be consumed by SQS, SNS und Lambda which provides us great flexibility and a near endless amount of possible infrastructure variations.
In this article we will look in to the configuration of S3 Events with the goal to get notified from CloudWatch Alarms if something happened in our bucket. You can find a complete and deployment ready example here.
Creating CloudWatch Alarms in case of S3 Event
S3 generate a huge list of Events which can be found in their documentation We can send these Event messages to our event bus at EventBridge or consume them from SQS, SNS or directly trigger Lambdas. In our case we will configure an Alert in CloudWatch which we want to trigger if new Objects are created in our defined S3 Bucket. As S3 Events cannot be consumed directly from CloudWatch we have to plug some service in the middle and create alerts based on its metrics.
In our case we will use a SQS which collects the S3 Events. To set this up we configure an aws_s3_bucket_notification in Terraform, which requires the emitting bucket and the target Queue. events requires a list of strings which contain the event type for which we want to be notified, in our example this will be s3:ObjectCreated:.
resource "aws_s3_bucket_notification" "bucket_notification" { bucket = "ourBucket"
queue {
queue_arn = "collectingQueue" events = ["s3:ObjectCreated:*"]
}}
After we have that part configured we can go on and configure our CloudWatch alarm. We will base our alerts on the NumbersOfMessagesSent metric which is a default metric of AWS SQS.
For this example we want to listen for 5 minutes and if during that time there was no new Object created in our Bucket we will switch to the alarming state.
It is very important to also set the dimension in the alert configuration. This will associate that alarm to the metric of our collection Queue.
resource "aws_cloudwatch_metric_alarm" "alarm" {
alarm_name = var.cloud_watch_alarms[count.index].name
comparison_operator = var.cloud_watch_alarms[count.index].comparison_operator
threshold = var.cloud_watch_alarms[count.index].threshold
evaluation_periods = var.cloud_watch_alarms[count.index].evaluation_periods
metric_name = var.cloud_watch_alarms[count.index].metric_name
namespace = var.cloud_watch_alarms[count.index].namespace
period = var.cloud_watch_alarms[count.index].period
statistic = var.cloud_watch_alarms[count.index].statistic
alarm_description = var.cloud_watch_alarms[count.index].description
treat_missing_data = var.cloud_watch_alarms[count.index].treat_missing_data
actions_enabled = false dimensions = var.cloud_watch_alarms[count.index].dimensions == null ? {} : var.cloud_watch_alarms[count.index].dimensions
}
Conclusion
It’s very easy to emit and consume S3 Events with other AWS Services.
Often you don’t need any glue code to get to your goal by using integrations provided by AWS.