Neil Killen

In my last post, I implemented a message queue which decoupled our application and provided a level of redundancy for our messages. But what about messages that are not processed successfully? This could occur for a number of reasons there might be an issue with the consuming application, a network issue or even an invalid message request.

A Deadletter queue, also known as an undelivered-message queue is a holding queue for messages that cannot be delivered to their destination. They provide a mechanism for persisting your failed messages without continually trying to process the same message. The deadletter queue could then be monitored and any failed messages can be examined and appropriate corrective action taken.

If we go back to our booking application, if we wanted to ensure failed messages are persisted then we would implement a Dead-letter queue. This is relatively straightforward to do in AWS SQS.

1. Create a new Queue called BookingDeadletterQueue
2. Now configure the BookingQueue to use the BookingDeadletterQueue under the Dead Letter Queue settings:
   • Use Redrive Policy to enabled – this indicates that messages should be sent into a dead letter queue after exceeding the Maximum Receives.
   • Specify the name of the Deadletter queue: BookingDeadletterQueue
   • Set the Maximum Receives – this is the maximum number of times a message can be received before it is sent to the Dead Letter Queue.
3. Save Changes.

The Redrive policy specifies the source queue, the dead-letter queue, and the conditions under which AWS SQS moves messages to the Deadletter queue if the consumer of the booking queue fails to process a message a specified number of times. When the ReceiveCount for a booking message exceeds the maxReceiveCount for the Booking queue, Amazon SQS will move the message to a dead-letter queue. For example, using the setting specified above if a message fails to process it will remain in the booking queue for a maximum of 3 reads by the consumer application. If after the 3rd receive the message isn’t processed successfully and deleted it will be moved to the BookingDeadletterQueue automatically by SQS.

A few important things to consider

• A deadletter queue in AWS SQS is just another Queue.
• Messages in the deadletter queue are not persisted indefinitely and falls within the same configurable message retention period of 1 to 14 days.
• The deadletter queue of a standard queue must also be a standard queue.
• The deadletter queue of a FIFO queue must also be a FIFO queue.
• Multiple source queues can target the same deadletter queue.
• A source queue can only target a single deadletter queue.
• It provides a defined location for failed messages that need to be investigated.

Managing Messages in the Deadletter queue

So now that we have our failed messages in a discrete location but how are going to manage these messages? They most likely need to be investigated and/or processed. You could give an administrator access to the deadletter queue in AWS to monitor, however that’s not ideal and is somewhat restrictive. A better solution would be to implement a Consumer of the deadletter queue to process the messages in that queue.

Using Sitecore workflow to review/manage failed messages

Sitecore workflow is normally associated with reviewing and publishing content as it ensures items move through a series of predefined states before being published. This ensures content is reviewed before it is published to the live website.

If you are not familiar with Sitecore workflow I seriously recommend you check it out. Dylan Young’s quick Introduction to Workflow in Sitecore.

The main components of a workflow are:

• States – represent the different stages in your item creation process.
• Commands – move items from one state to another.
• Actions – are automatically performed on the items when they are in particular workflow states or when particular workflow commands are executed.

Sitecore Workbox – gives you an overview of all items that are in workflows and enables you to manage these items.

In order to provide a mechanism whereby someone could review each of the failed booking requests and either update and resend the message or delete the message using Sitecore Workflow we can do the following:

1. Create a new Workflow called: Failed Booking Request Workflow
2. This workflow will have 3 states:
   • Failed Message – this will be the initial workflow state when a Failed message request is created this will allow the failed message to be investigated by someone with access to the workflow.
   • Abandon Message – once the failed message has been reviewed and it has been determined that the message is not required and should be abandoned the message will be set to this state.
   • Resend Message – the message may be reviewed and/or correct if necessary and then set to this state which will send the corrected message back to booking queue for processing.
3. We will create two commands:
   • Abandon Message – this command will be responsible for moving the item from the Initial state to the Abandon Message state.
   • Resend Message – this command will be responsible for moving the item from the initial state to the Resend Message state.
4. We will create two custom Actions:
   • AbandonFailedMessageAction – when this action method is executed it will delete the failed message Sitecore item.
   • ResendFailedMessageAction – when this action method is executed it will send the message to the AWS SQS booking queue and delete the Sitecore item.
5. Create a new Sitecore template Failed Booking Request containing all the fields for the booking request and set the standard value for the workflow to our Failed Booking Request workflow and state set to Failed Message.
6. Create a Bucket in Sitecore called Failed Booking Request Bucket.
7. Create a Sitecore Scheduled Tasks to poll the DeadLetter queue and read the failed messages. For each failed message create a new item in Sitecore programmatically in the Failed Booking Request Bucket using our new template – Failed Booking Request and set each of the fields to the values in the deadletter booking message request.

Now I can give designates users access to any failed messages through workbox. They can review each failed message and take the appropriate action to process the failed messages:

• Edit & Resend Message
• Delete Message

They can do this all within Sitecore without having to access AWS.

Additional Info

• Sitecore Workflows and the Workbox
• Introduction to Workflow in Sitecore by Dylan Young
• Review an item in the Workbox
• AWS SQS Dead-Letter queues

In the next post, I will look at how we could simplify this solution without using an external message queue.