CRUD Operations with Azure Queue Storage in an Azure Function – Retrieve

In the previous blog post, we explored how to Enqueue and Dequeue items from Azure Storage. Let us continue our exploration of Queue storage and write code to “peek” from a queue.

Peek/Retrieve From Queue

For retrieve from Queue, you could use the CloudQueue.PeekMessageAsync() method. For example,

[FunctionName("PeekItemFromQueue")]
public static async Task<IActionResult> PeekItemFromQueue(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest request,
[Queue("SampleQueue")] CloudQueue cloudQueue)
{

    var message = await cloudQueue.PeekMessageAsync();
    

    if (message == null)
        return new OkObjectResult("No item in the queue");

    var resultBuilder = new StringBuilder();
    resultBuilder.AppendLine($"Message : `({message.AsString})`");
    resultBuilder.AppendLine($"Pop Reciept : {message.PopReceipt}");
    resultBuilder.AppendLine($"Dequeue Count : {message.DequeueCount}");
    resultBuilder.AppendLine($"Insertion Time: {message.InsertionTime}");
    resultBuilder.AppendLine($"Next Visible Time: {message.NextVisibleTime}");
    return new OkObjectResult(resultBuilder.ToString());
}

The CloudMessageQueue object provides some interesting properties about the retrieved message. For example, the above query could provide following details.

Message : `SampleMessageForTest`
Pop Reciept : 
Dequeue Count : 0
Insertion Time: 17-04-2021 02:38:55 +00:00
Next Visible Time:

The DequeueCount property specifies the number of times the message has been retrieved without being deleted from the queue. Remember, when a message is retrieved from the Azure Queue, it is not removed from the Queue. The DequeueCount property hints at the number of attempts to retrieve the message by client applications.

While the message is not removed from the Queue entirely, it does disappears from the queue for a designated time. The Next Visible Time indicates the time at which the message would appear back once is has disappeared temperarly.

You could also in effect use the CloudQueue.GetMessageAsync method as well, since by default the mesage is not removed from the original queue. However, the difference lies in the fact that the GetMessageAsync method would remove the message from queue temporarly, and increment the DequeueCount. It would also issue a PopReciept. But that’s not the case with PeekMessageAsync, which would not hide the message from the queue nor would be increment the DequeueCount. Following would be the response if we had used GetMessageAsync instead of PeekMessageAsync.

Message : `SampleMessageForTest`
Pop Reciept : cIGyxZkB2QgBAAAA
Dequeue Count : 1
Insertion Time: 17-04-2021 02:38:55 +00:00
Next Visible Time: 17-04-2021 12:10:20 +00:00

A Word about Poison Queue

There could be situations wherein the message in the queue appear as a clog in the system as the message could retried too many times. As a guard against this, the concept was poison queue was introduced. Any message that has been retried over 5 times, is considered as poison message (could harm the application), are moved to different queue called the Poison Queue, usually named as queuename-poison.

Since you know the name of the poison queue, you could still process the messages in the queue. Depending on your application/business needs you can decide on the stratergy to process the poison queue messages. Irrespective of your business needs, one of the important point to consider when processing the poison messages is attempting to understand the reason why the message was moved to poison queue in first place.

That’s all for now. We will continue exploring the Azure Storage with this byte sized tutorial series.

CRUD Operations with Azure Queue Storage in an Azure Function – Create And Delete

An Azure queue is ideal for storing large number of messages, with each message having an upper cap of 64 Kb. This is ideal for providing asynchronous message queueing facility between application clients.

Queue concepts can be broken down into 4 components.

  • Storage Account : Like all other Azure Storage facilities, Queue is also linked with a Storage Account.
  • Queue : As the name suggest, it is a queue containing a set of messages. One key point to note here is that the queue needs to be named in lowercase.
  • URL : The queue is accessed via the URL associated. URL has a special format
https://<storage account>.queue.core.windows.net/<queue>

  • Message : Message could be of any formated, though size of each message is limited to 64Kb. The default Time-To-Live for a message is 7 days, but it could be configured. You could also make a message non-expiring with a configuration value of -1.

Create

Let us straightaway hit some code to create our items in Queue.

[FunctionName("AddMessageToQueue")]
public static async Task<IActionResult> AddMessageToQueue(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest request,
    [Queue("samplequeue")] CloudQueue cloudQueue)
{
    var message = request.Query["item"];

    var queueMessage = new CloudQueueMessage(message);
    await cloudQueue.AddMessageAsync(queueMessage);
    return new OkObjectResult($"Message :(`{message}`) added to tbe queue");
}

As with earlier posts in this series, we will stick with Azure Functions for the demonstrations. As observed from the code above, we are using CloudQueue class to refer to the Queue in question (“samplequeue“).

The insertion process is pretty straightforward here. You need to create an instance of the CloudQueueMessage object and use the CloudQueue.AddMessageAsync to add the message to the queue. That seems to be pretty simple right.

There are some important characterstics of the inserted message that would be quite useful to be aware of. Each of the inserted messages as the following properties.

  • ID – An Unique Guid
  • Message Text – Message itself
  • Insertion Time – Represents the time message was added to the queue
  • Expiration Time – Represents the time when the message is expected to expire
  • Dequeue Count – Number of times the message has been dequeued.
  • Size – Actual Size of the message

You can set the Expiration Time with an overload of CloudQueue.AddMessageAsync method. Another interesting property to note is Dequeue count. Unlike traditional queue, the Azure Queue doesn’t remove the message as soon as one dequeue it. Instead, it becomes invisible for a specified amount of time and then reappear again. This is a fail-safe mechanism build by Microsoft so that if your application fails to process the message due to any reason, it could still retrieve it again. The Dequeue Count property provides the number of times a message has been dequeued.

Here is another interesting aspect – If a message has been retrieved/dequeued 5 or more time without being removed from the queue, the message would be considered a posionous message and moved to a separate queue named “Posion Queue”. We will go in detail of the posion queue in a later post.

So how do one remove an item from the queue ? You delete it of course.

Delete

[FunctionName("PopMessageFromQueue")]
public static async Task<IActionResult> PopMessageFromQueue(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest request,
[Queue("SampleQueue")] CloudQueue cloudQueue)
{
    var message = await cloudQueue.PeekMessageAsync();
    await cloudQueue.DeleteIfExistsAsync();

    return new OkObjectResult(message == null ? "No Message found to be removed":$"Message :(`{message.AsString}`) has been removed");
}

The code above uses the CloudQueue.PeekMessageAsync method to retrieve the first message in the queue read to be processed. It then uses the CloudQueue.DeleteIfExistsAsync to remove the item.

Additional Note

You could read fetch the count of Messages in queue. For the purpose, you could use the CloudQueue.FetchAttributesAsync method.

[FunctionName("NumberOfItemsInQueue")]
public static async Task<IActionResult> NumberOfItemsInQueue(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest request,
    [Queue("SampleQueue")] CloudQueue cloudQueue)
{
    await cloudQueue.FetchAttributesAsync();
    return new OkObjectResult($"Approx. number of Items in Queue :{cloudQueue.ApproximateMessageCount}");
}

The call to CloudQueue.FetchAttributesAsync updates the property Cloud.ApproximateMessageCount with the number of items in the queue.

In the next post, we will look into Retrieve and Update part of the Queue. We will also have a further look at the Poison Queue.