Multiple Binding in Azure Function (both In process and Out of Process)

In this post, we will look into how to support multiple output binding in an Azure Function. We will attempt to explore how to support multiple output both in in process functions as well as out of process functions.

Multiple output binding are useful when you have a scenario where you want to response to an HTTP Trigger with an Http Message and at the same time, add a new Item to the Queue. This is handled in difference ways in the in-process functions and out-of-process functions.

In Process Functions

In the conventional In process Azure functions, the function is a class library that run in the same process as the Host Process. With the in process functions, you could easily support multiple output binding by marking the ouput parameter as out.

For example, to support a Queue output along with the Http Response, the following could be done.

[FunctionName("MultioutputSample")]
public static IActionResult MultiOutputDemo([HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req,
    ILogger log,[Queue("SampleQueue")]out string queueOutput)
{

    queueOutput = "New Activity detected";
    return new OkObjectResult($"Hello, Welcome to Isolated functions demo.");
}

In the above example, a string “Hello, Welcome to Isolated functions demo.” would be send back as Http Response, while another string “New Activity detected” is added to the “SampleQueue” that has been marked with the queueOutput out parameter.

Out Of Process Isolated Functions

The out of process isolated functions works as console application and runs independed from the Azure Function Host. As a developer, you are responsible for hosting your own IHost instance.

The value returned by the isolated function is bound to the output binding. This would mean the out of process isolated function has to use another approach for support multiple output. This is done defining a Custom Type and having a property marked with the QueueOutput attribute. For example,

[Function(nameof(MultiOutputSample))]
        public static async Task<CustomOutputType> MultiOutputSample(
            [HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequestData req,
            FunctionContext executionContext)
{
    // Use the FunctionContext.GetLogger to fetch instance of ILogger
    var logger = executionContext.GetLogger(nameof(StaticFunctions));
    logger.LogInformation("Started execution of function");

    var data = await req.ReadFromJsonAsync<UserDto>();

    // Should use HttpResponseData as response
    var response = req.CreateResponse(HttpStatusCode.OK);
    await response.WriteStringAsync($"Hello {data.Name}, Welcome to Isolated functions demo.");


    return new CustomOutputType
    {
        UserResponse = response,
        UserTask = $"{data.Name} have logged in"
    };
}

Where the CustomOutputType is defined as

public class CustomOutputType
{
    [QueueOutput("SampleQueue")]
    public string UserTask { get; set; }
    public HttpResponseData UserResponse { get; set; }
}

In the above defined CustomOutput, the UserResponse being of type HttpResponseData would return a http message back to the caller, while at the same time, the string in the UserTask is added to the SampleQueue.

As seen, both in process and out of process Azure Functions supports multiple output binding, but uses different approaches for achieving the results. Both approaches are pretty easy to use.

Introduction to Isolated Azure Functions (.Net 5)

One of the bigger constraints of Azure Function Apps was the tight coupling between the function code and the Azure function runtime. The In-Process Azure Functions ran as a class library in the same process as the host service and was tightly coupled. This forced the function code to use the same version of .Net as the host service. This would also mean you cannot use .Net 5 to develop your function apps.

This restriction was overcome with Isolated Function App, which runs out of process in Azure Functions. The Isolated Function App runs as a console application and could now use .Net 5 for implementation. Additionally, it had fewer conflics due to different versions of same assembly as it runs on a separate process. The other benefits also included the ability to control the function app in a better way, including the application start, configuration and support for dependency injection. One of the biggest limitations of the out-of-process isolated Azure functions is the lack of support for Durable Functions.

Project Template

Let us now proceed to create our Isolated Function.

Isolated Function could be created by choosing the Azure Function Template from Visual Studio.

You need to make sure that you select “.Net 5.0” as the framework. This would generate the Isolated Function template for you.The Isolated Function project is a console application and the template comprises of following files.

  • host.json
  • local.setttings.json
  • C# Project File
  • Program.cs

Start Up and Configuration

Since as a developer you have access to the start up of the Isolated functions, you are responsible for creating, configuring and starting the Host instance. You do so using the HostBuilder class. If you look at the default code generated you could find the following code.

public class Program
{
    public static void Main()
    {
        var host = new HostBuilder()
            .ConfigureFunctionsWorkerDefaults()
            .Build();

        host.Run();
    }
}

The code above creates an instance of HostBuilder, uses the ConfigureFunctionWorkerDefaults extension method to create teh default configurations and uses the Build extension to initialize the configuration.

It then uses the IHost.Run() method to start the Host Instance.

As mentioned earlier, since you are building your own host instance, you have ability to configure it with the services and middleware you would prefer.

ConfigureFunctionsWorkerDefaults

The ConfigureFunctionWorkerDefaults method adds the default settings which is required to run the function app as __ out of process_. The default configuration includes

  • Default set of converters
  • Default JsonSerializerOptions
  • Integrate Azure Logging
  • Output binding Middlewareand features
  • Function execution middleware
  • Default gRPC support

You could use an override of the ConfigureFunctionWorkerDefaults to custom configure some of the existing services/configuration. For example, the default JsonSerializerOptions configured by the ConfigureFunctionWorkerDefaults is as following.


// Code from IServiceCollection.AddFunctionsWorkerDefaults extension method
services.Configure<JsonSerializerOptions>(options =>
{
    options.PropertyNameCaseInsensitive = true;
});

This configures the default Json configuration to ignore the casing on property names. You could further configure it to ignore the trailing commas in Json (default is false).

var host = new HostBuilder()
            .ConfigureFunctionsWorkerDefaults(builder=>
            {
                builder.Services.Configure<JsonSerializerOptions>(options =>
                {
                    options.AllowTrailingCommas = true;
                });

            })
            .Build();


Middleware

The isolated functions also allows you to configure your custom middlewares to the invocation pipelines. The ConfigureFunctionsWorkerDefault method again comes in handy for the purpose.

var host = new HostBuilder()
                .ConfigureFunctionsWorkerDefaults(builder=>
                {
                    builder.Services.Configure<JsonSerializerOptions>(options =>
                    {
                        options.AllowTrailingCommas = true;
                    });

                    builder.UseMiddleware<ExceptionMiddleWare>();
                })
                .Build();

The ExceptionMiddleWare in the above is a custom middleware that can inject custom logic into the invocation pipeline.

public class ExceptionMiddleWare : IFunctionsWorkerMiddleware
{
    public async Task Invoke(FunctionContext context, FunctionExecutionDelegate next)
    {
        try
        {
            await next(context);
        }
        catch(Exception ex)
        {
            var logger = context.GetLogger(context.FunctionDefinition.Name);
            logger.LogError("Unexpected Error in {0}: {1}",context.FunctionDefinition.Name,ex.Message);
        }
    }

}

In the above code, the logic injected by the Middleware is called after the function exection, but you could also inject logic before the function is called.

Dependency Injection and Services

You can register your services during the start up using the IHostBuilder.ConfigureServices extension method. For example,

var host = new HostBuilder()
            .ConfigureFunctionsWorkerDefaults(builder=>
            {
                builder.Services.Configure<JsonSerializerOptions>(options =>
                {
                    options.AllowTrailingCommas = true;
                });

                builder.UseMiddleware<ExceptionMiddleWare>();
            })
            .ConfigureServices(serviceCollection =>
            {
                serviceCollection.AddSingleton<IMockDataService>(new MockDataService());
            })
            .Build();

For the sake of this example, I am using a sample Service which emulate by Data Service.

public interface IMockDataService
{
    UserDto CreateUser(string userName);
}

public class MockDataService : IMockDataService
{
    private readonly Random _randomGenerator = new Random();
    public UserDto CreateUser(string userName)
    {
        return new UserDto
        {
            Name = "Anu Viswan",
            Id = _randomGenerator.Next()
        };
    }
}

I could now inject the dependency in my function code.

Functions

Since the Isolated Functions are running out of process, the way we have the Http Request in the Http Trigger functions are difference from the in-process version. Since we do not have access to the original Http Request and response, you would need to use the HttpRequestData and HttpResponseData object instead to access the request and response. While these provides all the Headers,URL, Body of the incoming request, it is not the original request, bur rather a representation of the same. A sample Http Trigger function might look like the following.

[Function(nameof(SayHello))]
public static async Task<HttpResponseData> SayHello(
    [HttpTrigger(AuthorizationLevel.Anonymous,"post")] HttpRequestData req,
    FunctionContext executionContext)
{
    // Use the FunctionContext.GetLogger to fetch instance of ILogger
    var logger = executionContext.GetLogger(nameof(StaticFunctions));
    logger.LogInformation("Started execution of function");

    var data = await req.ReadFromJsonAsync<UserDto>();

    // Should use HttpResponseData as response
    var response = req.CreateResponse(HttpStatusCode.OK);

    await response.WriteStringAsync($"Hello {data.Name}, Welcome to Isolated functions demo.");
    return response;
}

The bindings too are serverly restricted as there is no longer deep integrtion with the Azure function runtime. This would also mean you are restricted from using the ICollection<T> or CloudBlockBlob. Instead your need to relay POCOS or strings (or arrays). The Logger also needs to be fetched using the FunctionContext.GetLogger method.

Earlier,we registered own DataService. We could use Dependency Injection to inject the same and use in our function. For example,

private readonly IMockDataService _mockDataService;
public InstanceFunctions(IMockDataService mockDataService)
{
    _mockDataService = mockDataService;
}

[Function(nameof(CreateUser))]
public async Task<HttpResponseData> CreateUser(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequestData req,
    FunctionContext executionContext)
{
    var queryStrings = System.Web.HttpUtility.ParseQueryString(req.Url.PathAndQuery);
    var userName = queryStrings["user"];

    // Use the FunctionContext.GetLogger to fetch instance of ILogger
    var logger = executionContext.GetLogger(nameof(StaticFunctions));
    logger.LogInformation("Started execution of function");

    var data = _mockDataService.CreateUser(userName);

    // Should use HttpResponseData as response
    var response = req.CreateResponse(HttpStatusCode.OK);
    await response.WriteAsJsonAsync<UserDto>(data);

    return response;
}

The above method uses Constructor Injection to inject our dependency and later uses the same in the CreateUser method.

Summary

In the this post, we have a peak in the world of Isolated Functions and how to create them. We understood some of the differences between the in-process functions and the new out-of-process isolated functions. We also understood how we have more control over the start up process, which in-turns helps use to configure the function app with the desired services and middlewares. One of the key draw backs of the Isolated function at the moment is the lack of support of Durable Functions.

The complete list of differences is outlined here by Microsoft : Difference with .Net Library Functions.

CRUD Operations with Azure Queue Storage in an Azure Function – Retrieve

In the previous blog post, we explored how to Enqueue and Dequeue items from Azure Storage. Let us continue our exploration of Queue storage and write code to “peek” from a queue.

Peek/Retrieve From Queue

For retrieve from Queue, you could use the CloudQueue.PeekMessageAsync() method. For example,

[FunctionName("PeekItemFromQueue")]
public static async Task<IActionResult> PeekItemFromQueue(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest request,
[Queue("SampleQueue")] CloudQueue cloudQueue)
{

    var message = await cloudQueue.PeekMessageAsync();
    

    if (message == null)
        return new OkObjectResult("No item in the queue");

    var resultBuilder = new StringBuilder();
    resultBuilder.AppendLine($"Message : `({message.AsString})`");
    resultBuilder.AppendLine($"Pop Reciept : {message.PopReceipt}");
    resultBuilder.AppendLine($"Dequeue Count : {message.DequeueCount}");
    resultBuilder.AppendLine($"Insertion Time: {message.InsertionTime}");
    resultBuilder.AppendLine($"Next Visible Time: {message.NextVisibleTime}");
    return new OkObjectResult(resultBuilder.ToString());
}

The CloudMessageQueue object provides some interesting properties about the retrieved message. For example, the above query could provide following details.

Message : `SampleMessageForTest`
Pop Reciept : 
Dequeue Count : 0
Insertion Time: 17-04-2021 02:38:55 +00:00
Next Visible Time:

The DequeueCount property specifies the number of times the message has been retrieved without being deleted from the queue. Remember, when a message is retrieved from the Azure Queue, it is not removed from the Queue. The DequeueCount property hints at the number of attempts to retrieve the message by client applications.

While the message is not removed from the Queue entirely, it does disappears from the queue for a designated time. The Next Visible Time indicates the time at which the message would appear back once is has disappeared temperarly.

You could also in effect use the CloudQueue.GetMessageAsync method as well, since by default the mesage is not removed from the original queue. However, the difference lies in the fact that the GetMessageAsync method would remove the message from queue temporarly, and increment the DequeueCount. It would also issue a PopReciept. But that’s not the case with PeekMessageAsync, which would not hide the message from the queue nor would be increment the DequeueCount. Following would be the response if we had used GetMessageAsync instead of PeekMessageAsync.

Message : `SampleMessageForTest`
Pop Reciept : cIGyxZkB2QgBAAAA
Dequeue Count : 1
Insertion Time: 17-04-2021 02:38:55 +00:00
Next Visible Time: 17-04-2021 12:10:20 +00:00

A Word about Poison Queue

There could be situations wherein the message in the queue appear as a clog in the system as the message could retried too many times. As a guard against this, the concept was poison queue was introduced. Any message that has been retried over 5 times, is considered as poison message (could harm the application), are moved to different queue called the Poison Queue, usually named as queuename-poison.

Since you know the name of the poison queue, you could still process the messages in the queue. Depending on your application/business needs you can decide on the stratergy to process the poison queue messages. Irrespective of your business needs, one of the important point to consider when processing the poison messages is attempting to understand the reason why the message was moved to poison queue in first place.

That’s all for now. We will continue exploring the Azure Storage with this byte sized tutorial series.

CRUD Operations with Azure Queue Storage in an Azure Function – Create And Delete

An Azure queue is ideal for storing large number of messages, with each message having an upper cap of 64 Kb. This is ideal for providing asynchronous message queueing facility between application clients.

Queue concepts can be broken down into 4 components.

  • Storage Account : Like all other Azure Storage facilities, Queue is also linked with a Storage Account.
  • Queue : As the name suggest, it is a queue containing a set of messages. One key point to note here is that the queue needs to be named in lowercase.
  • URL : The queue is accessed via the URL associated. URL has a special format
https://<storage account>.queue.core.windows.net/<queue>

  • Message : Message could be of any formated, though size of each message is limited to 64Kb. The default Time-To-Live for a message is 7 days, but it could be configured. You could also make a message non-expiring with a configuration value of -1.

Create

Let us straightaway hit some code to create our items in Queue.

[FunctionName("AddMessageToQueue")]
public static async Task<IActionResult> AddMessageToQueue(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest request,
    [Queue("samplequeue")] CloudQueue cloudQueue)
{
    var message = request.Query["item"];

    var queueMessage = new CloudQueueMessage(message);
    await cloudQueue.AddMessageAsync(queueMessage);
    return new OkObjectResult($"Message :(`{message}`) added to tbe queue");
}

As with earlier posts in this series, we will stick with Azure Functions for the demonstrations. As observed from the code above, we are using CloudQueue class to refer to the Queue in question (“samplequeue“).

The insertion process is pretty straightforward here. You need to create an instance of the CloudQueueMessage object and use the CloudQueue.AddMessageAsync to add the message to the queue. That seems to be pretty simple right.

There are some important characterstics of the inserted message that would be quite useful to be aware of. Each of the inserted messages as the following properties.

  • ID – An Unique Guid
  • Message Text – Message itself
  • Insertion Time – Represents the time message was added to the queue
  • Expiration Time – Represents the time when the message is expected to expire
  • Dequeue Count – Number of times the message has been dequeued.
  • Size – Actual Size of the message

You can set the Expiration Time with an overload of CloudQueue.AddMessageAsync method. Another interesting property to note is Dequeue count. Unlike traditional queue, the Azure Queue doesn’t remove the message as soon as one dequeue it. Instead, it becomes invisible for a specified amount of time and then reappear again. This is a fail-safe mechanism build by Microsoft so that if your application fails to process the message due to any reason, it could still retrieve it again. The Dequeue Count property provides the number of times a message has been dequeued.

Here is another interesting aspect – If a message has been retrieved/dequeued 5 or more time without being removed from the queue, the message would be considered a posionous message and moved to a separate queue named “Posion Queue”. We will go in detail of the posion queue in a later post.

So how do one remove an item from the queue ? You delete it of course.

Delete

[FunctionName("PopMessageFromQueue")]
public static async Task<IActionResult> PopMessageFromQueue(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest request,
[Queue("SampleQueue")] CloudQueue cloudQueue)
{
    var message = await cloudQueue.PeekMessageAsync();
    await cloudQueue.DeleteIfExistsAsync();

    return new OkObjectResult(message == null ? "No Message found to be removed":$"Message :(`{message.AsString}`) has been removed");
}

The code above uses the CloudQueue.PeekMessageAsync method to retrieve the first message in the queue read to be processed. It then uses the CloudQueue.DeleteIfExistsAsync to remove the item.

Additional Note

You could read fetch the count of Messages in queue. For the purpose, you could use the CloudQueue.FetchAttributesAsync method.

[FunctionName("NumberOfItemsInQueue")]
public static async Task<IActionResult> NumberOfItemsInQueue(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest request,
    [Queue("SampleQueue")] CloudQueue cloudQueue)
{
    await cloudQueue.FetchAttributesAsync();
    return new OkObjectResult($"Approx. number of Items in Queue :{cloudQueue.ApproximateMessageCount}");
}

The call to CloudQueue.FetchAttributesAsync updates the property Cloud.ApproximateMessageCount with the number of items in the queue.

In the next post, we will look into Retrieve and Update part of the Queue. We will also have a further look at the Poison Queue.

CRUD Operations with Azure Blob Storage in an Azure Function – Read

In the previous post, we examined how we could add an item to the Azure blob.

In this part, we will use an Azure function to download the file store in Azure Blob Storage.

Block Blob

Let us begin with Block Blobs.

[FunctionName("DownloadBlockBlob")]
public static async Task<IActionResult> DownloadBlockBlob(
    [HttpTrigger(AuthorizationLevel.Anonymous,"get",Route =null)] HttpRequest req,
    [Blob("todos")] CloudBlobContainer blobContainer,
    ILogger log)
{
    var key = req.Query["key"];

    var stream = new MemoryStream();
    var blob = await blobContainer.GetBlobReferenceFromServerAsync(key);
    await blob.DownloadToStreamAsync(stream);
    stream.Position = 0;
    return new FileStreamResult(stream, "application/octet-stream") { FileDownloadName = key };
}

This turns out to be pretty straightforward, isn’t it. The CloudBlobContainer.GetBlobReferenceFromServerAsync gets the reference to the blob we are interested in using the blob name, in this case, passed on as a query parameter. We could then use the CloudBlob.DownloadToStreamAsync method to download the contents of the blob to a stream, which could be later pass back using the FileStreamResult.

One obvious question would be why use a azure function in the above case, when there is no authentication. One could directly use a Http Get Request. Partially, that is true, however, do not forget this is only an example and most often than note, authentication would not be anonymous and there could be other things that you would like your azure function to during the download, like a log or updating/fetching another document.

Of course, you could simplify things using input bindings. For example, the above code could be rewritten using binding as the following.

[FunctionName("DownloadBlockBlobUsingBinding")]
public static async Task<IActionResult> DownloadBlockBlobUsingBinding(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "DownloadBlockBlobUsingBinding/{fileName}")] HttpRequest req,
[Blob("todos/{fileName}",FileAccess.Read)] Stream blob,
string fileName,
ILogger log)
{
    var memoryStream = new MemoryStream();
    await blob.CopyToAsync(memoryStream);
    memoryStream.Position = 0;
    return new FileStreamResult(memoryStream, "application/octet-stream") { FileDownloadName = fileName };
}

Append Blob

To download a file from an Append Blob is no different from what we have seen with Block Blob.

[FunctionName("DownloadAppendBlob")]
public static async Task<IActionResult> DownloadAppendBlob(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = null)] HttpRequest req,
[Blob("sample")] CloudBlobContainer blobContainer,
ILogger log)
{

    var key = req.Query["key"];

    var stream = new MemoryStream();
    var blob = blobContainer.GetAppendBlobReference(key);
    await blob.DownloadToStreamAsync(stream);
    stream.Position = 0;
    return new FileStreamResult(stream, "text/plain") { FileDownloadName = key };
}


The only point of importance while learning would be to remember that Append Blobs are not supported by the Storage emulator. For rest of the code look quite similar to the Block blob, sole difference being we use a different method to get reference to the blob – CloudContainer.GetAppendBlobReference.

As mentioned in the earlier post, we will address the Page blob separately. But for now, go ahead and play around with the Block Blob and Append Blob. You could access the source code explained in this code in my Github

CRUD Operations with Azure Blob Storage in an Azure Function – Create, Part 1

With Azure Blob storage, Microsoft provides an easy to use cloud based storage solution for storing massive amount of data, particularly unstructured data. These are ideal for storing media files which could be served directly to browser, maintaining log files among others.

There are three components of Blob storage one needs to be aware of.

  • Storage Account – The unique namespace for your azure data.
  • Container – Containers are similar to Directory in a file system, and is used to organize your blobs into sub sections.
  • Blobs – Reflects the data to be stored.

The blobs themselves are of 3 types.

  • Block Blobs : Ideal for storing image and other media files for streaming.
  • Append Blobs : Ideal for logging (append to end)
  • Page Blobs : Ideal for frequent read/write operations

That was a short introduction on the Blob Storages, let us now look at some code to create a new item in the blob storage.

We will stick to Azure Functions, and keep the code as simple as possible. As mentioned in the series opener, the more advanced topics including security would be discussed in a seperate posts.

Create Item in Block Blob

We will begin with the block blob first. In the very first example, we will add a text file to the blob, contents of which are passed via the request. I guess that would be the simplest way to begin the journey.

[FunctionName("AddItemBlockBlob")]
public static async Task<IActionResult> AddItemBlockBlob(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req,
    [Blob("todos")] CloudBlobContainer blobContainer,
    ILogger log)
{
    log.LogInformation("Request Recieved");

    await blobContainer.CreateIfNotExistsAsync();

    string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
    var data = JsonConvert.DeserializeObject<TodoDto>(requestBody);

    data.Id = Guid.NewGuid().ToString();

    var serializedData = JsonConvert.SerializeObject(data);

    var blob = blobContainer.GetBlockBlobReference($"{data.Id}.json");
    await blob.UploadTextAsync(serializedData);
    return new OkObjectResult($"Item added to blob with Id = {data.Id}");
}

public class TodoDto
{
    public string Id { get; set; }
    public string Title { get; set; }
    public string Description { get; set; }
    public bool IsCompletd { get; set; }
}


CloudBlobContainer references the container in which our blob resides. If the container does not exists, we could create it using the CloudBlobContainer.CreateIfNotExistsAsync method.

Since we want to store our data in a Block Blob, we use CloudBlockBlob.GetBlockBlobReference method to get reference to the blob and use the CloudBlockBlob.UploadTextAsync method to upload the text data to the blob, referred by the blob name. In our first example, it is a Guid(followed by a string .json – but this is only for bringing more clarity to content of blob, you could choose a different name as well).

That was pretty clean, thanks to the rich set of APIs defined by Microsoft. There is however a behavior one needs to be aware of. If we were to upload text again to the same blob (refered by same blob name), the content would be replaced.

Upload a file to Block Blob

As the second example, let us attempt to upload an image file to the blob.

[FunctionName("UploadFileBlockBlob")]
public static async Task<IActionResult> UploadFileBlockBlob(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req,
    [Blob("todos")] CloudBlobContainer blobContainer,
    ILogger log
    )
{
    string key = req.Query["key"];
    var data = req.Form.Files["file"];
    using (var stream = data.OpenReadStream())
    {
        var blob = blobContainer.GetBlockBlobReference(key);
        await blob.UploadFromStreamAsync(stream);
    }
    return new OkObjectResult($"Item added to blob with Id = {key}");
}

In this example, we are retrieving the contents of the file from the HttpRequest and using the CloudBlockBlob.UploadFromStreamAsync method to upload the file to the blob.

Of course, Bindings are applicable here as well, and you could reduce fair bit of code with help of output binding.

[FunctionName("UploadFileBlockBlobBinding")]
public static async Task<IActionResult> UploadFileBlockBlobBinding(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "UploadFileBlockBlobBinding/{filename}")] HttpRequest req,
    [Blob("todos/{filename}",FileAccess.Write)] Stream fileStream,
    string filename,
    ILogger log)
{
    log.LogInformation("Request Recieved");
    var data = req.Form.Files["file"];
    var inputDataStream = data.OpenReadStream();
    await inputDataStream.CopyToAsync(fileStream);
    return new OkObjectResult($"Item added to blob with Id = {filename}");
}

The parameter fileStream would refer to the blob specified by the todos/{filename} where the filename is passed via the Http request.

The inputDatastream, read from the HttpRequest is being copied over to the fileStream using the CopyToAsync method, ensuring the blob has been updated with the stream of data from the request.

Create a Append Blob

Let us now attempt to create another type of blob – Append Blob. Unfortunately, the Azure Storage Emulator does not support the Append Blob, so do make sure you have your connections string in place before trying the following code.

[FunctionName("AddToAppendBlob")]
public static async Task<IActionResult> AddToAppendBlob(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req,
    [Blob("sample")] CloudBlobContainer blobContainer,
    ILogger log)
{
    log.LogInformation("Request Recieved");

    await blobContainer.CreateIfNotExistsAsync();
    var requestBody = await new StreamReader(req.Body).ReadToEndAsync();
    var data = JsonConvert.DeserializeObject<LogMessage>(requestBody);

    var blob = blobContainer.GetAppendBlobReference("applog.txt");
    if (!blob.Exists())
    {
        blob.CreateOrReplace();
    }
    await blob.AppendTextAsync($"User:{data.User}, Message:{data.Message}");

    return new OkObjectResult($"Item added to blob");
}

We retrieve a reference to the Append Blob using the CloudBlobContainer.GetAppendBlobReference method. We then use the CloudAppendBlob.AppendTextAsync to append the data to the end of the existing blob.

The Page Blobs has a bit more story to tell, which is why we will reserve it for another post. But as you have already seen, Microsoft’s rich API makes it easy to create blobs, as it was the case with Azure Table Storage.

CRUD Operations with Azure Table Storage in an Azure Function – D

In the previous part of this series we briefly described how to Create, Retreive and Update records in an Azure Table Storage using Azure Web Functions. In this part, we will look into final part of quadrant – the Delete operations.

As you would have guessed after going through previous posts in this series, we would be using the CloudTable for deletion of record.

[FunctionName("DeleteRecord")]
public static async Task<IActionResult> DeleteEntity(
    [HttpTrigger(AuthorizationLevel.Anonymous,"POST", Route = "DeleteRecord/{partitionKey}/{rowKey}")] HttpRequest request,
    [Table("todos", "{partitionKey}", "{rowKey}")] TodoTableEntity tableEntity,
    [Table("todos")] CloudTable todoTable,
    ILogger log)
{
    log.LogInformation("Request to delete the record");

    var deleteOperation = TableOperation.Delete(tableEntity);
    var result = await todoTable.ExecuteAsync(deleteOperation);
    return new OkObjectResult(result);
}

In this above example, we are using Bindings to retrieve the record to be deleted. The PartitionKey and RowKey is retrieved from the query string, quite similiar to the method we learned in the Retrieve record post.

A word of caution here. It is tempting to retrieve the record using RowKey alone, but one needs to be aware of the consequences. If one was to attempt retrival/deletion based on RowKey alone, the whole table has to be scanned for the record. Furthermore, it is theoratical to have the same RowKey over different patition (though this could be averted by application logic), which would retrieve multiple entites instead of one.

Deletion Table Binding

Of course, you could achieve the above without Table Binding.

[FunctionName("DeleteWithoutBinding")]
        public static async Task<IActionResult> DeleteWithoutBinding(
        [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest request,
        [Table("todos")] CloudTable todoTable,
         ILogger log)
{
    log.LogInformation("Request to delete Record without binding");

    string partitionKey = request.Query["pkey"];
    string rowKey = request.Query["rkey"];

    var tableQuery = new TableQuery<TodoTableEntity>();

    var filterRowKeyAndPartitionKey = TableQuery.CombineFilters(
        TableQuery.GenerateFilterCondition(nameof(TodoTableEntity.RowKey), QueryComparisons.Equal, rowKey),
        TableOperators.And,
        TableQuery.GenerateFilterCondition(nameof(TodoTableEntity.PartitionKey), QueryComparisons.Equal, partitionKey));

    tableQuery.FilterString = TableQuery.CombineFilters(
        TableQuery.GenerateFilterCondition(nameof(TableEntity.PartitionKey), QueryComparisons.NotEqual, "Key"),
        TableOperators.And,
        filterRowKeyAndPartitionKey);

    var itemToDelete = todoTable.ExecuteQuery(tableQuery).First();

    var deleteOperation = TableOperation.Delete(itemToDelete);
    var deleteResponse = await todoTable.ExecuteAsync(deleteOperation);
    return new OkObjectResult(deleteResponse);
}

But honestly, I feel that is quite a lot of boilerplate code which could be avoided using bindings. I strongly suggest you use binding unless you have very strong reasons of doing otherwise.

Well, so far we have seen CRUD operations using the Azure Table Storage. In the next part of this series, we will explore another medium of storage provided by Azure. Until then, enjoy coding.

Azure Storage with Azure Functions

In this series of Articles, we will explore various Azure Storage options using Azure Functions. We will stick to the basic usage, leaving out the more advanced topics involved with the storage to later blog posts.

CRUD using Azure Table Storage

CRUD using Azure Blob Storage

CRUD using Azure Queue Storage

CRUD Operations with Azure Table Storage in an Azure Function – U

In the earlier posts, we enlightened ourselves with creation and retrieval of records from Azure Table Storage using Azure Web Functions. In this segment, we will attempt to update a record.

Let us once again look at our table before proceeding further.

In the first approach, we will attempt to update the record based on the RowKey and PartitionKey provided by the request. Let us go ahead and write our method now.

[FunctionName("Update")]
public static async Task<IActionResult> Update(
    [HttpTrigger(AuthorizationLevel.Anonymous,"post",Route = null)] HttpRequest request,
    [Table("todos")]CloudTable todoTable,
    ILogger log)
{
    log.LogInformation("Request to update Record");

    string requestBody = await new StreamReader(request.Body).ReadToEndAsync();
    var data = JsonConvert.DeserializeObject<TodoDto>(requestBody);

    var rowKeyToUpdate= request.Query[nameof(TableEntity.RowKey)];
    var partitionKeyToUpdate= request.Query[nameof(TableEntity.PartitionKey)];

    var tableEntity = new TodoTableEntity
    {
        RowKey = rowKeyToUpdate,
        PartitionKey = partitionKeyToUpdate,
        Title = data.Title,
        Description = data.Description,
        IsCompleted = data.IsCompletd,
        ETag = "*"
    };

    var updateOperation = TableOperation.Replace(tableEntity);
    var result = await todoTable.ExecuteAsync(updateOperation);
    return new OkObjectResult(result);
}

I will skip the Bindings here since we have learned about the same in the previous posts. We will first create an instance of record which will have the new value.

var tableEntity = new TodoTableEntity
{
    RowKey = rowKeyToUpdate,
    PartitionKey = partitionKeyToUpdate,
    Title = data.Title,
    Description = data.Description,
    IsCompleted = data.IsCompletd,
    ETag = "*"
};

Notice that we have used the PartitionKey and Rowkey retrieved from our query (we will address how to fetch it from database, if not given in query later). The other important point to notice here is the presence of the ETag. The Update Table Operation will not be allowed to process without the ETag.

As the next step, we will use the TableOperation to update or replace our record. This turns out to be pretty easy, thanks to the rich collection of methods introduced by Microsoft.

var updateOperation = TableOperation.Replace(tableEntity);
var result = await todoTable.ExecuteAsync(updateOperation);

Now you are ready to hit F5 and test your web function. For example, let us consider the following input.

// Request
http://localhost:7071/api/Update?PartitionKey=L&RowKey=1001

//Body
{
    title:'Learn Azure Func',
    description:'Learn Azure Func with Table Storage',
    isCompleted:'True'
}

We want to flip the IsCompleted Flag to True here. Final result would be.

We could, optimize the code a bit with help of the Bindings we have learned in the previous post. For example, the above code could be rewritten as,

[FunctionName("UpdateUsingBinding")]
public static async Task<IActionResult> UpdateUsingBinding(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "UpdateUsingBinding/{partitionKey}/{rowKey}")] HttpRequest request,
[Table("todos", "{partitionKey}", "{rowKey}")] TodoTableEntity tableEntity,
[Table("todos")] CloudTable todoTable,
    ILogger log)
{
    log.LogInformation("Request to update Record");

    string requestBody = await new StreamReader(request.Body).ReadToEndAsync();
    var data = JsonConvert.DeserializeObject<TodoDto>(requestBody);

    tableEntity.Title = data.Title;
    tableEntity.Description = data.Description;
    tableEntity.IsCompleted = data.IsCompletd;

    var updateOperation = TableOperation.Replace(tableEntity);
    var result = await todoTable.ExecuteAsync(updateOperation);
    return new OkObjectResult(result);
}

As you could notice, we have made using of the Table Bindings to retrieve the record here, which is then updated with new values passed on from the Http Request.

In the last approach for the post, we will address the case when PartitionKey and RowKey is not known to the HttpRequest and we need to fetch it using the Title. Well, as you might have guessed it, it would involve retrieval we learned in the previous posts.

[FunctionName("UpdateWithRetrival")]
public static async Task<IActionResult> UpdateWithRetrival(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest request,
[Table("todos")] CloudTable todoTable,
    ILogger log)
{
    log.LogInformation("Request to update Record");

    string requestBody = await new StreamReader(request.Body).ReadToEndAsync();
    var data = JsonConvert.DeserializeObject<TodoDto>(requestBody);

    var tableQuery = new TableQuery<TodoTableEntity>();

    tableQuery.FilterString = TableQuery.CombineFilters(
        TableQuery.GenerateFilterCondition(nameof(TableEntity.PartitionKey), QueryComparisons.NotEqual, "Key"),
        TableOperators.And,
        TableQuery.GenerateFilterCondition(nameof(TodoTableEntity.Title), QueryComparisons.Equal, data.Title));
    var result = todoTable.ExecuteQuery(tableQuery);

    var itemToUpdate = result.First();
    itemToUpdate.Description = data.Description;
    itemToUpdate.IsCompleted = data.IsCompletd;

    var updateOperation = TableOperation.Replace(itemToUpdate);
    var updateResponse = await todoTable.ExecuteAsync(updateOperation);
    return new OkObjectResult(updateResponse);
}

As you can observe, thanks to the rich API provided by Microsoft, the basic operations with Azure Storage are pretty straightforward. We will address Deletion in the next post, until then, have a great day.

CRUD Operations with Azure Table Storage in an Azure Function – R

In an earlier post, we discussed how to insert a new item in the Azure Storage Table. In this article, we will delve into how to retrieve data from Azure Storage Table.

Retrieve a Single Entity

In the previous article, we had partitioned the entities based on their first letter of their Title. Here is how our table looked at end our insert operations.

As mentioned before, we would use CloudTable to retrieve the entity we desire. For the sake of example, let us assume that the RowKey of the desired record would be passed via QueryString in the Http Request.

[FunctionName("TodoGetOne")]
public static async Task<IActionResult> GetOne(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
[Table("todos")] CloudTable todoTable,
ILogger log)
{
    string id = req.Query["id"];

    var tableQuery = new TableQuery<TodoTableEntity>();

    tableQuery.FilterString = TableQuery.CombineFilters(
        TableQuery.GenerateFilterCondition(nameof(TableEntity.PartitionKey), QueryComparisons.NotEqual, "Key"), 
        TableOperators.And, 
        TableQuery.GenerateFilterCondition(nameof(TableEntity.RowKey), QueryComparisons.Equal, id));

    var result = todoTable.ExecuteQuery(tableQuery);
    return new OkObjectResult(result);
}

As you can observe, we have used the TableQuery to filter our desired entity.

var tableQuery = new TableQuery<TodoTableEntity>();

tableQuery.FilterString = TableQuery.CombineFilters(
    TableQuery.GenerateFilterCondition(nameof(TableEntity.PartitionKey), QueryComparisons.NotEqual, "Key"), 
    TableOperators.And, 
    TableQuery.GenerateFilterCondition(nameof(TableEntity.RowKey), QueryComparisons.Equal, id));

The TableQuery.FilterString property enables us to provide Custom filters for the query. This is further facilitated by supporting utility methods in TableQuery, such as CombineFilters, and GeneratedFilterCondition. In our case, as seen in code above, we are filtering the data where the PartitionKey is not equal to text Key (remember, the partition key in our example is a special partition which keeps track of next Primary Id available ) and RowKey equavalent to the Id passed via querystring.

Further, we execute the query using the CloudTable.ExecuteQuery method.

var result = todoTable.ExecuteQuery(tableQuery);

That’s all you would need to fetch the data from Azure Table Storage.

Retrieve One Entity Using Binding

Table Binding also helps us retrieve a single record skipping a lot of boiler plate code, provided we know the parition key and the row key. For example, consider the following code.

[FunctionName("TodoGetOneBinding")]
public static async Task<IActionResult> TodoGetOneBinding1(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = "TodoGetOneBinding/{partition}/{id}")] HttpRequest req,
[Table("todos", "{partition}", "{id}")] TodoTableEntity todo,
ILogger log)
{
    return new OkObjectResult(todo);
}

This effectively does the same as the code in our previous example, but provides a less cluttered code. The {parition} and {id} parameters from the Route is used as parameters for filter the table here.

Retrieve Multiple Entities

I guess there is very little to explain here once we have done the above examples. So let us write the code straightaway for retrieve all the todo items

[FunctionName("TodoGetAll")]
public static async Task<IActionResult> GetAll(
    [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
    [Table("todos")] CloudTable todoTable,
    ILogger log)
{
    var tableQuery = new TableQuery<TodoTableEntity>();
    tableQuery.SelectColumns = new List<string> { nameof(TodoTableEntity.Description) };
    tableQuery.FilterString = TableQuery.GenerateFilterCondition(nameof(TableEntity.PartitionKey), QueryComparisons.NotEqual, "Key");

    var result = todoTable.ExecuteQuery(tableQuery);
    return new OkObjectResult(result);
}

That’s all for now. We will continue our journey exploring the Azure Cloud Storage in upcoming articles.