Problem of Not Queueing in SSE SQS by CloudWatch Events

Takahiro Iwasa

Apr 5, 2020

3 min read

CloudWatch Events SQS

I noticed that CloudWatch Events did not queue messages in SSE SQS. After some investigation, it seemed that CloudWatch Events did not support SSE SQS as target. This may be caused by an SQS policy. I’ll leave it as a note.

Use the following CloudFormation template. CloudWatch Events will be configured to start every minute using a cron expression (line 16).

docs.aws.amazon.com

Cron expressions reference - Amazon EventBridge

Cron expressions have six required fields, which are separated by white space.

AWSTemplateFormatVersion: "2010-09-09"
Description: Example of CloudWatch Events not queueing to SSE SQS
Resources:
  SQS:
    Type: AWS::SQS::Queue
    Properties:
      KmsDataKeyReusePeriodSeconds: 86400
      KmsMasterKeyId: alias/aws/sqs
      MessageRetentionPeriod: 1209600
      QueueName: sse-sqs-for-cloudwatch-events

  Events:
    Type: AWS::Events::Rule
    Properties:
      Name: cloudwatch-events-test
      ScheduleExpression: 'cron(0/1 * * * ? *)'
      State: ENABLED
      Targets:
        - Arn: !GetAtt SQS.Arn
          Id: cloudwatch-events-test

  QueuePolicy:
    Type: AWS::SQS::QueuePolicy
    Properties:
      PolicyDocument:
        Version: '2012-10-17'
        Id: !Sub
          - ${SqsArn}/SQSDefaultPolicy
          - {SqsArn: !GetAtt SQS.Arn}
        Statement:
          - Sid: !Sub
            - AWSEvents_${SqsName}_Id123456789012
            - {SqsName: !GetAtt SQS.QueueName}
            Effect: Allow
            Principal:
              Service: events.amazonaws.com
            Action: sqs:SendMessage
            Resource: !GetAtt SQS.Arn
            Condition:
              ArnEquals:
                aws:SourceArn: !GetAtt Events.Arn
      Queues:
        - !Ref SQS

Deploy the CloudFormation stack with the following command.

aws cloudformation deploy --template template.yaml --stack-name cloudwatch-events-test

Check the CloudWatch Events rule with the following command.

aws events describe-rule --name cloudwatch-events-test

{
    "Name": "cloudwatch-events-test",
    "Arn": "arn:aws:events:<YOUR_REGION>:<YOUR_ACCOUNT_ID>:rule/cloudwatch-events-test",
    "ScheduleExpression": "cron(0/1 * * * ? *)",
    "State": "ENABLED",
    "EventBusName": "default",
    "CreatedBy": "<YOUR_ACCOUNT_ID>"
}

Confirm the available message count. It should consistently show zero.

aws sqs get-queue-url --queue-name sse-sqs-for-cloudwatch-events

{
    "QueueUrl": "https://sqs.<YOUR_REGION>.amazonaws.com/<YOUR_ACCOUNT_ID>/sse-sqs-for-cloudwatch-events"
}

aws sqs get-queue-attributes \
  --queue-url https://sqs.<YOUR_REGION>.amazonaws.com/<YOUR_ACCOUNT_ID>/sse-sqs-for-cloudwatch-events \
  --attribute-names ApproximateNumberOfMessages

{
    "Attributes": {
        "ApproximateNumberOfMessages": "0"
    }
}

Comment out KmsDataKeyReusePeriodSeconds and KmsMasterKeyId in the CloudFormation template, and then update the stack.

--- 	Sun Oct 10 17:31:16 2021 UTC
+++ 	Sun Oct 10 17:31:16 2021 UTC
@@ -4,8 +4,8 @@
   SQS:
     Type: AWS::SQS::Queue
     Properties:
-      KmsDataKeyReusePeriodSeconds: 86400
-      KmsMasterKeyId: alias/aws/sqs
+#      KmsDataKeyReusePeriodSeconds: 86400
+#      KmsMasterKeyId: alias/aws/sqs
       MessageRetentionPeriod: 1209600
       QueueName: sse-sqs-for-cloudwatch-events

After successfully updating the stack, check the available message count again. It should consistently increase.

aws sqs get-queue-url --queue-name sse-sqs-for-cloudwatch-events

{
    "QueueUrl": "https://sqs.<YOUR_REGION>.amazonaws.com/<YOUR_ACCOUNT_ID>/sse-sqs-for-cloudwatch-events"
}

aws sqs get-queue-attributes \
  --queue-url https://sqs.<YOUR_REGION>.amazonaws.com/<YOUR_ACCOUNT_ID>/sse-sqs-for-cloudwatch-events \
  --attribute-names ApproximateNumberOfMessages

{
    "Attributes": {
        "ApproximateNumberOfMessages": "1"
    }
}

Clean up the provisioned AWS resources with the following command.

aws cloudformation delete-stack --stack-name cloudwatch-events-test

If you intend to configure an SSE SQS as a target of CloudWatch Events, additional settings may be required. Unfortunately, I could not find information on that issue in the AWS official documentation.