Allocation free ‘async’ Methods

Task Asynchronous Programming (TAP) model will go down as one of the landmark of C# language revolution. The typical method signature with return type Task/Task<T> has since then made significant appearances in our programming life.

But despite all its glorious functionalities, it needs to be noted that it comes at a certain cost – performance bottleneck, especially when you are working in tight loops in a memory constrained environment. Memory allocations of Task/Task<T> (note both are references types) impacts the performances adversely in critical paths. What developers longed for in such scenarios is a comparatively light-weight value type, which could be returned instead of the references types in the async methods. This was a limitation till C# 7.x happened.

Starting from C# 7, an async method could return any type that has an accessible ‘GetAwaiter’ method, which returns an object that implements INotifyCompletion and ICriticalNotifyCompletion Interfaces. The ValueTask<T> is a "goto" under such circumstances. Being a value type, it doesn’t require additional memory allocations. This is extremely useful when you already have the ‘cached result’ and the operation can be completed synchronously. But like Task, it comes at a cost, which we will discuss later in the article.

To begin with, let’s write some sample code to demonstrate the need of ValueTask<T> or rather, need for replacing Task<T>. Consider the following code.

public interface IWeatherService
{
  Task<double> GetWeather();
}

public class MockWeatherService : IWeatherService 
{
  public async Task<double> GetWeather() => await Task.FromResult(30);
}

The above code demonstrate a Mock Weather Service. In real world scenario, this might be accessing a Web API to fetch current weather details. For the sake of example, only the temperature is fetched, which is hard coded. Consider GetWeather to be invoked quite frequently (say in a loop) by the Client. There is a chance this might turn disastrous if this was executed in a memory constraint environment.

Ideally, you wouldn’t want to fetch from the Web API always. You would prefer to cache the result, and only fetch from the Web API only after specific interval (from the time the last value was cached). Despite doing the operation (fetch from cache) synchronously, you are allocating memory for the reference type Task<T>. One would have wished to avoid this scenario and yet use the async syntax.

This is possible by making use of the ValueTask<T> DataType. Let’s change our syntax a bit.

public interface ICachedWeatherService
{
  ValueTask<double> GetWeather();
}
public class MockCachedWeatherService : ICachedWeatherService
{
  private DateTime _lastAccessedTime = DateTime.MinValue;
  private double _lastAccessedValue = 30;
  public async ValueTask<double> GetWeather()
  {
    if(DateTime.Now - _lastAccessedTime < TimeSpan.FromSeconds(10))
    {
      return _lastAccessedValue;
    }
    _lastAccessedTime = DateTime.Now;
    return await Task.FromResult(30);
   }
}

In the above code, we have changed the signature of the method (return type) to ValueTask<T>. Since it implements all the necessary requirements that is needed for async method (as discussed earlier), we could still use the same signature in client to access the method, just like in Task based syntax.

Using Task Return Type

var _weatherService = new MockWeatherService();
           
for(int i = 0; i < 10; i++)
{
  await Task.Delay(5000);
  var currentWeather = await _weatherService.GetWeather();
  Console.WriteLine($"Weather @ {DateTime.Now.ToString("hh:mm:ss")}={currentWeather}");
}

Using ValueTask Return Type

var _cachedWeatherService = new MockCachedWeatherService();
for (int i = 0; i < 10; i++)
{
  await Task.Delay(5000);
  var currentWeather = await _cachedWeatherService.GetWeather();
  Console.WriteLine($"Weather @ {DateTime.Now.ToString("hh:mm:ss")}={currentWeather}");
}

Check Complete Code Sample

Difference is, when the method needs to return from the cache, it returns ValueTask avoiding the pain of creating the Task Reference Type. However, when it has to go through the Web API, it can still return Task<T>.

Even though ValueTask seem highly useful, you need to maintain caution and should rather use it only if there is considerable performance issues that were measured. The reason for this is that, despite the ValueTask helps you avoid unnecessary allocation, it comes with its own cost. The ValueTask is a two field structure compared to Task<T>, which despite being reference type has single field. More fields would result in more data to copy. It fact Microsoft warns us in the API documentation itself.

Quoting from Documentation

There are tradeoffs to using a ValueTask<TResult> instead of a Task<TResult>. For example, while a ValueTask<TResult> can help avoid an allocation in the case where the successful result is available synchronously, it also contains two fields whereas a Task<TResult> as a reference type is a single field. This means that a method call ends up returning two fields worth of data instead of one, which is more data to copy. It also means that if a method that returns one of these is awaited within an async method, the state machine for that async method will be larger due to needing to store the struct that’s two fields instead of a single reference.

Hence, stick onto Task/Task<T> for almost all your use, until performance analysis pushes you to use ValueTask<T>. Complete code samples described in the example is available in my Github

Advertisements

APM/EAP to TAP

Asynchronous Programming has evolved over the years, right from the APM to the extremely powerful TAP Model. One (or probably the only one) of the problem which developers find themselves in is to use legacy code which were written somewhere in between the evolution. Quite often, developers might be inclined to wrap the existing code to the latest TAP model. How do one do that ? Let’s examine each model and its conversion one by one.

Asynchronous Programming Model 

APM was the very first model introduced by the .Net designers, however it is no longer a recommended pattern. We will begin by writing an example code in APM, before figuring out how to convert it to TAP.

 

string filePath = @"yourfilepath";
byte[]  buffer = new byte[20000];
using(FileStream  fs = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read, 1024, FileOptions.Asynchronous))
{
       Console.WriteLine(ReadFile(fs,buffer,0,buffer.Length));
}

private int ReadFile(Stream fs, byte[] buffer,int offset,int count)
{
  IAsyncResult result =  fs.BeginRead(buffer, 0, count, null, null);
  int numBytes = fs.EndRead(result);
  return numBytes;
}

The Task.Factory provides us an easy way to convert APM patterns to TAP. Let’s rewrite the above code in TAP now.

 

private Task<int> ReadAsync(Stream stream, byte[] buffer, int offset, int count) => Task<int>.Factory.FromAsync(stream.BeginRead, stream.EndRead, buffer, offset, count, null);

Event based Asynchronous Pattern

The second approach that Microsoft adopted was known as EAP. Let’s first examine a sample code for EAP.

Process p = new Process();
p.Exited += (e,args)=>
{
  Console.WriteLine("Executing Exited Event");
  Console.WriteLine($"Ended : {p.ExitTime}");
};
p.StartInfo.FileName = @"cmd";
p.EnableRaisingEvents = true;
p.Start();
Console.WriteLine($"Started : {DateTime.Now}");
Console.WriteLine("Invoke Process completed");

The conversion from EAP to TAP is facilitated using the TaskCompletionSource class.

Console.WriteLine($"Ended : {InvokeProcess().Result}");

Task<string> InvokeProcess()
{
       var tcs = new TaskCompletionSource<string>();
       Process p = new Process();
       p.Exited += (e,args)=>{
       Console.WriteLine("Executing Exited Event");
       tcs.SetResult("Result Data");
       };
       p.StartInfo.FileName = @"cmd";
       p.EnableRaisingEvents = true;
       p.Start();
       Console.WriteLine("Invoke Process completed");
       return tcs.Task;
}

WaitForFirstN Tasks Method

There is a definite gap between TAP Utilities Task.WaitAny and Task.WaitAll. How do we wait for the first ‘N’ tasks from an array of tasks. That is one utility that is missing from TAP’s rich repository. It might not be hard to implement one by yourself though.

public static int[] WaitForFirstN(this System.Threading.Tasks.Task[] tasks,int numberOfTasks)
{
   if (numberOfTasks > tasks.Length) throw new ArgumentOutOfRangeException();

   var _taskDictionary = Enumerable.Range(0, tasks.Length).ToDictionary(x => tasks[x]);
   List<int> completedTaskIndices = new List<int>();

   while (completedTaskIndices.Count < numberOfTasks) 
   { 
      var index = System.Threading.Tasks.Task.WaitAny(tasks);
      completedTaskIndices.Add(_taskDictionary[tasks[index]]); 
      tasks = tasks.Where(x => x.Id != tasks[index].Id).ToArray();
    }
    return completedTaskIndices.ToArray();
}

The above code uses existing Task Collaborators Task.WaitAny to wait for ‘N’ tasks specified by numberOfTasks parameter. Let’s try the following in code.

 
System.Threading.Tasks.Task[] tasks = new System.Threading.Tasks.Task[5];

[TestInitialize]
public void Init()
{
  for (int ctr = 0; ctr <= 4; ctr++)
  {
    int factor = ctr;
    tasks[ctr] = System.Threading.Tasks.Task.Run(() => Thread.Sleep(factor * 250 + 50));
   }
}
[TestMethod]
public void WaitForFirstN_ValidResult_EqualsExpected()
{
   var completedTaskList = tasks.WaitForFirstN(3);
   CollectionAssert.AreEqual(new int[] { 0, 1, 2 }, completedTaskList);
}

Code shown in this sample can also be on my GitHub.

One interesting though. Curious why Framework doesn’t allow us to create Static Extension Methods for types.