C# 8 : Nullable Reference Types, Null Forgiving and Null Coalescing Assignment

There are some properties that ought to be learned together to understand the complete picture. In this blog post, we will address two of such features.

Nullable Reference Types

Let us begin by writing some code.

public class Foo
{
    public Bar Bar { get; set; }
}

public class Bar
{
    public int Id { get; set; }
}

// Client Code
var foo = new Foo();
Console.WriteLine(foo.Bar.Id);

Notice anything wrong ? How often have been guilty of attempting to access something that could potentially be null ? In the above case, we haven’t initialized the Foo.Bar property, but in the client code, we are attempting to access the Bar.Id property.

This would throw a NullReferenceException, as you are dereferencing a null value (Bar hasn’t been assigned yet). We should have checked for null in the first place.

But as the code base increases, there are times when we forget to do so. And unfortunately, prior to C# 8, there was no way compiler could warn us to do the null check even if we might have wondered many a times if it could do so.

Nullable Reference Types in C# 8 brings us pretty much the same. It allows us to mark the Reference Types which can/cannot be null, using the same syntax we used to mark Nullable Value Types – using the ? operator.

In C# 8, with Nullable Reference Types enable, if you were to compile the code, the compiler warns you with the following.

Warning	CS8618	Non-nullable property 'Bar' is uninitialized. Consider declaring the property as nullable.

Depending on your business logic, you have couple of ways to proceed from here.

Case 1 – Bar is Nullable

If your business logic requires that the Bar property could be null, then you could mark it explicitly. For example,

public Bar? Bar { get; set; }

The ? operator tells the compiler that the property could be null and it should warn the developer for possible null checks if he hasn’t done so yet. The compiler duely warns you with the following.

Warning	CS8602	Dereference of a possibly null reference.

Case 2 – Bar is non-nullable

If your application logic requires that the Bar property cannot be null, then you could let the compiler know by using the following syntax (basically omitting the ? operator).

public Bar Bar { get; set; }

The above code tells the compiler that the Bar cannot be null. Interestingly, we still get a warning now, let us first examine what it is before understanding why compiler throws the warning.

Warning	CS8618	Non-nullable property 'Bar' is uninitialized. Consider declaring the property as nullable.

We have declared the property as Non-nullable, yet we have left it uninitialized. We could of course remove this warning by initializing the property with a default value (either in the property declaration or in the constructor), but this gives us an oppurtunity to introduce another feature.

Null Forgiving Operator

You could have a situation where, as per the business logic, you know that a particular property can never be null at runtime. However, you are in no position to assign a default value. In such situations, you could use the Null-forgivingoperator to disable the warnings. Null-Forgiving is denoted by using the postfix operator !. For example,

public Bar Bar { get; set; } = null!;

The above code informs the compiler that it would skip the warning about Bar being uninitialized and it would not be null in runtime. Do note the null-forgiving operator has not runtime influence.

Null Coalescing Assignment

C# 8 also introduces the Null Coalescing assignment via the the assignment operator ??=. For demonstration, let us further develop our code and introduce a CreateBarInstance to assign instances of Bar. For example,

public void CreateBarInstance(Bar instance)
{
    Bar = instance;
}

Due to some business reasons, we have requirement that the method should assign Bar instance, only if it is null. Of course, one could do this an if condition, but that would introduce a lot of boiler plate code. We could eliminate these extraneous code using the null coalescing operator.

public void CreateBarInstance(Bar instance)
{
    Bar ??= instance;
}

The line Bar ??= instance ensures that Bar is assigned only when the it is null. Or in other words, the value of right hand operand is assigned to left hand operand only if the left hand operand is null.

Demonstrating the code in action,

var foo = new Foo();
foo.CreateBarInstance(new Bar(2));
Console.WriteLine(foo.Bar.Id);
foo.CreateBarInstance(new Bar(3));
Console.WriteLine(foo.Bar.Id);

Output of the above code

2
2

That’s all for now, we will continue our exploration of C# 8 features.

C# 8 : Asynchronous Streams and Static Local Functions

Asynchronous Stream
How often have you come across situations when you would want to return streams of data from an asynchronous method ? Plenty of times I would guess. The asynchronous methods were pretty limited when return data in streams, but not any longer. 

Consider the following code.

// Will not compile
public static async Task<IEnumerable<int>> Generate(int max)
{
    for (int i = 0; i < max; i++)
    {
        await Task.Delay(1000);
        yield return i;
    }
}

The above code would not compile. There is no way you could `yield return` values (or in other words, stream values) from an asynchronous method. This would have been a highly useful situation when you want to iterate over results from a query over an extremely large database. There are other countless situation the ability to stream data from asynchronous method could be useful. However, prior to C# 8.0, we were severely handicapped in such a situation.

With C# 8.0, .Net comprises a new Type called the IAsyncEnumerable, which allows us to accomplis this. The above code could be now rewritten as.

public static async IAsyncEnumerable<int> Generate(int max)
{
    for (int i = 0; i < max; i++)
    {
        await Task.Delay(1000);
        yield return i;
    }
}

//client code
await foreach (var item in Generate(10))
{
    Console.WriteLine(item);
};

The placement of `await` keyword in the consumer code needs to be noted here.

Local Static Functions
In previous versions of C#, local methods could capture the enclosing scope. For example, consider the following code.

public void Foo()
{
    int internalValue = 20;

    void Bar()
    {
        internalValue++;
    }

    Bar();
}

This enable usage of variables such as internalValue within the local method. If the usage is accidental, this might lead to huge consequences. One could not declare a static local method in the earlier versions of .Net.

With C# 8.0, .Net removed this limitation. This enables the developers to create *pure local functions* as it does not allow usage of variables from enclosing types within it.

public void Foo()
{
    int internalValue = 20;

    static void Bar()
    {
        Console.WriteLine("Do Something");
        // internalValue++; // This would throw compile error
    }

    Bar();

}

This also removes certain overhead created when using the captured variables. For example, the previous code (C# < 8.0), would be treated by the compiler as

public void Foo()
{
	DemoClass.<>c__DisplayClass0_0 CS$<>8__locals1;
	CS$<>8__locals1.internalValue = 20;
	DemoClass.<Foo>g__Bar|0_0(ref CS$<>8__locals1);
}
internal static void <Foo>g__Bar|0_0(ref DemoClass.<>c__DisplayClass0_0 A_0)
{
	Console.WriteLine("Do Something");
	int internalValue = A_0.internalValue;
	A_0.internalValue = internalValue + 1;
}

Notice how the compiler generates a struct to store the captured variables and pass it using `ref`. Such an overhead would be completely avoided when using the *static local functions*, simply because – the compiler won’t compile or capture the enclosing scope.

We explored two features introduced in C# 8.0. We will explore more in coming posts, possibly in bunches as we have plenty to catch up before exploring C# 9.0.

Managing Props as Component Developer

One of the little things one would have in mind when developing Components using React is that you might wonder if the developer who would consume the compoenent would pass in the required props.

Let us consider a simple component here.

export Navbar extends Component{
    render (<div><h1>{this.props.title}</h1</div>)
}

The above Navbar components expects the props to contain title, however, the Developer who would use it might not be aware of it and could forget supplying the same.

PropTypes

One of the ways to deal with it is using the PropTypes. This allows you to leave warnings for the developer regarding the props needed in the componenet. For example,

static propTypes = {
title: PropTypes.string.isRequired
};

The IsRequired flag raises console warnings for the developer each time he misses passing them when using the component.

DefaultProps

The other, but often less used approach is providing our default props, in case the developers ommits them accidentaly. This could be done using the defaultProps. For example,

static defaultProps = { title : 'Default Finder'}

Now if the developer forgets to add the props, the default ones would be used by the component. In case the developer adds them, the default ones would be overriden.

I guess you would need a bit of both (depending on the component requirement) to bring a balance to the approach.

Asynchronous Code – Behind the Scenes – 004

During this series of deep dive into the asynchronous calls, we have so far looked into

  • [x] General Structure of generated code.
  • [x] Role of Stub/Worker method.
  • [x] Structure of State Machine and role of Fields.
  • [x] Implementation of the SetStateMachine method.
  • [ ] Implementation of the MoveNext method.

It is now time to look at the most important piece of the puzzle – the MoveNext() method.

Before we begin exploring the MoveNext(), let us remind ourself that the method is called when the async method is first invoked and then, each time it is resumed. The Method would be responsible for the following.

  • Ensure the method starts/resumes execution at the right place when it starts for the first time or resumes after a pause.
  • Preserve the state of State Machine when it needs to pause.
  • Schedule a continuation when the awaited expression hasn’t been completed yet.
  • Retrieve values from the awaiter.
  • Propagate the return values or method completion via the Builder.
  • Propagate the exceptions if any via the Builder.

The last 2 points are curious if you were to consider that the MoveNext method has a void return Type. So how does the MoveNextreturn the result or exceptions ? Of course via the Builder instance. It is the role of the Stub method to return the Task to the Caller method.

Without taking any time longer, let us take a peek at the generated code. We will then proceed to split it into parts and find how it works

The Whole Code

private void MoveNext()
{
    int num = <>1__state;
    try
    {
        TaskAwaiter awaiter;
        if (num != 0)
        {
            if (num == 1)
            {
                awaiter = <>u__1;
                <>u__1 = default(TaskAwaiter);
                num = (<>1__state = -1);
                goto IL_00cc;
            }
            awaiter = Task.Delay(delay).GetAwaiter();
            if (!awaiter.IsCompleted)
            {
                num = (<>1__state = 0);
                <>u__1 = awaiter;
                <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
                return;
            }
        }
        else
        {
            awaiter = <>u__1;
            <>u__1 = default(TaskAwaiter);
            num = (<>1__state = -1);
        }
        awaiter.GetResult();
        Console.WriteLine(delay);
        awaiter = Bar().GetAwaiter();
        if (!awaiter.IsCompleted)
        {
            num = (<>1__state = 1);
            <>u__1 = awaiter;
            <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
            return;
        }
        goto IL_00cc;
        IL_00cc:
        awaiter.GetResult();
    }
    catch (Exception exception)
    {
        <>1__state = -2;
        <>t__builder.SetException(exception);
        return;
    }
    <>1__state = -2;
    <>t__builder.SetResult();
}

That does look a bit scary to begin with. But, have no worries. We will break it down and understand it better.

Exception Handling

Now we already know that the associated Task object returned by the async method would contain any exception if any. It also sets the status to faulted. So how does the State Machine help in doing so ? That’s the first part we will explore. Let us have birds-eye view of the MoveNext() method – for time being, we will ignore all code within the try block.

private void MoveNext()
{
    int num = <>1__state;
    try
    {
        // Ignore this code for the moment
    }
    catch (Exception exception)
    {
        <>1__state = -2;
        <>t__builder.SetException(exception);
        return;
    }
    <>1__state = -2;
    <>t__builder.SetResult();
}

As you can observe the entire MoveNext() method has a big try catch wrapping the code within. The interesting part for the moment would be the catch block. If any exceptions occurs in the try block, the MoveNext() method sets the state to -2 to indicate the method has completed (-2 indicates completion, irrespective of success or failure). It then uses the Builder to set the exception using the Builder.SetException method.

Only special exceptions like the ThreadAbortException or the StackOverflowException can cause the MoveNext() method to end with an exception.

High Level Flow of State Machine

At a higher level, one can observe that the MoveNext() method returns if any of the following are true

  • Each time the state machine needs to be pause (for an await statement to complete).
  • Execution reaches the end of the method
  • Exception is thrown, but not caught in the async method.

A High level flow of the State Machine could be summarized as follows.

  1. The Stub Method (Worker Method) initiates the State Machine using the Builder Object (AsyncTaskMethodBuilder).
  2. Jump to the correct place in State Machine based on the State Field.
  3. Execute the State Machine until the code reaches await statement or end of the method (return statement).
  4. Fetch the awaiter.
    • If the awater is completed, go back to the Step 2.
    • If not, attach a continuation to the awaiter.
    • If this is the first awaiter, return the Task.
  5. The Task returned in Step 5, would be returned the caller via the Builder.

The Try Block

The Try blocks starts with a switch/if condition depending on the number of await statements within the method. If it has 3 or more awaits, usually one could notice a switch case, in all other cases, an if statement is used.

Irrespective of the approach, the condition to check resolves around the State of the State Machine. If the state is negative, it indicates the first call to the MoveNext() method. If the value of State is a positive number, then it indicates the State Machine is resuming from a pause.

public int <>1__state;
public AsyncTaskMethodBuilder <>t__builder;
public int delay;
private TaskAwaiter <>u__1;

private void MoveNext()
{
    int num = <>1__state;
    try
    {
        TaskAwaiter awaiter;
        if (num != 0)
        {
            if (num == 1)
            {
                awaiter = <>u__1;
                <>u__1 = default(TaskAwaiter);
                num = (<>1__state = -1);
                goto IL_00cc;
            }
            awaiter = Task.Delay(delay).GetAwaiter();
            if (!awaiter.IsCompleted)
            {
                num = (<>1__state = 0);
                <>u__1 = awaiter;
                <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
                return;
            }
        }
        else
        {
            awaiter = <>u__1;
            <>u__1 = default(TaskAwaiter);
            num = (<>1__state = -1);
        }
        awaiter.GetResult();
        Console.WriteLine(delay);
        awaiter = Bar().GetAwaiter();
        if (!awaiter.IsCompleted)
        {
            num = (<>1__state = 1);
            <>u__1 = awaiter;
            <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
            return;
        }
        goto IL_00cc;
        IL_00cc:
        awaiter.GetResult();
    }
    catch (Exception exception)
    {
        // Not displayed for clarity
    }
    <>1__state = -2;
    <>t__builder.SetResult();
}

One of the first things you notice in the code above is that State is stored in a local variable. I guess this is done for optimization purposes. We could use a dedicated post later for understanding different optimizations techniques used by compiler here, for now let us stick to the task in hand.

As one can observe, when the Method is invoked for the first time, as the state would be -1, the code would proceed and hit the first await statement.

awaiter = Task.Delay(delay).GetAwaiter();
if (!awaiter.IsCompleted)
{
    num = (<>1__state = 0);
    <>u__1 = awaiter;
    <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
    return;
}

It fetches the Awaiter using the GetAwaiter method. If the awaiter is already completed, it would proceed to the next step in the original method. If not, it would set the State to 0 (indicating the first instance where the MoveNext() method had to await – zero based index), store the awaiter in the field and schedules the state machine to proceed to the next action when the specified awaiter completes using the Builder.AwaitUnsafeOnCompleted method.

On resumption after the pause, it moves to else part (remember, the state is having a value 0 now). It restores the awaiter stored in the field and clears the fields so that GC could take care of it. It also sets the State to -1.

else
{
    awaiter = <>u__1;
    <>u__1 = default(TaskAwaiter);
    num = (<>1__state = -1);
}
awaiter.GetResult();
Console.WriteLine(delay);
awaiter = Bar().GetAwaiter();
if (!awaiter.IsCompleted)
{
    num = (<>1__state = 1);
    <>u__1 = awaiter;
    <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
    return;
}

It then proceeds to fetch the Result using the TaskAwaiter.GetResult() method and then executes the remaining steps untill it hits the next await or the method completes. On completion (either finished or faulted), it sets the State to -2 and sets the Result using the Builder.

<>1__state = -2;
<>t__builder.SetResult();

Over the last few posts, we have traced through the generated source code behind the asynchronous methods. We noticed how the method gets translated to a pair of Stub/Working method and a State Machine. We also explored the State Machine in detail and understood how the MoveNext method method navigates the original method while maitaining the states.

The whole process, starting from the moment your code hits the await expression could be summarized as,

  1. Get the awaiter from the awaitable expression using the GetAwaiter() method.
  2. Check if the awaiter has been comepleted
    • If Yes, Go to Step 8. (Fast Path)
    • If No, remember where you have reached using the State Field. (Slow Path)
  3. Store the awaiter in a field.
  4. Schedule a continuation with the awaiter, such that when the continuation is executed, you are back at the right place.
  5. Return from the MoveNext, either to the original caller if it is the first pause, or to whatever has scheduled the continuation.
  6. When the continuation fires, set the State to -1 to indicate running.
  7. Restore the Awaiter from the field and store it back in the Stack. Remember to reset the field so that GC could take care of it.
  8. Fetch the result using GetResult() method.
  9. Continue with rest of the code.

This, was a simple asynchornous method devoid of any controls methodologies like the loops. In the next part of this series, we will use the knowledge we have gained so far to understand more complex scenarios in depth.

Once again, I would like to thank the wonderful Jon Skeets for his brillant book – C# in Depth. You ought to rename it to “C# Bible” Jon !!

Asynchronous Code – Behind the Scenes – 003

Okay, I wasn’t quite realistic in the earlier post when I mentioned we would look at MoveNext in this one. I missed an important clog of the wheel. The SetStateMachine() method.

IAsyncStateMachine.SetStateMachine

We will only breifly visit the SetStateMachine method here, as the complete picture becomes more clear when we look to details of the MoveNext() method.

So how does the SetStateMachine method looks like in the generated code. Interestingly, it has two different implementation depending on whether you are in Release or Debug mode.

// Release Mode
[DebuggerHidden]
private void SetStateMachine(IAsyncStateMachine stateMachine)
{
    <>t__builder.SetStateMachine(stateMachine);
}

void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine stateMachine)
{
    this.SetStateMachine(stateMachine);
}


// Debug Mode
[DebuggerHidden]
private void SetStateMachine(IAsyncStateMachine stateMachine)
{
}

void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine stateMachine)
{
    this.SetStateMachine(stateMachine);
}

As one can observe, in the Debug Mode, the method is empty. Hence the following explanation is more relavant for the Release mode.

Let’s go back a bit and think about our little Stub method.

private static Task Bar()
{
	<Bar>d__2 stateMachine = new <Bar>d__2();
	stateMachine.<>t__builder = AsyncTaskMethodBuilder.Create();
	stateMachine.<>1__state = -1;
	stateMachine.<>t__builder.Start(ref stateMachine);
	return stateMachine.<>t__builder.Task;
}

When the State Machine is started by the Stub Method, it is residing on the Stack as a local variable of the Stub Method.

This is where the whole crazy stuff starts. When the State Machine pauses and resumes again, it needs a lot of information. For this to happen, when it pauses, the state machine has to box itself and store in heap, so that when it resumes, it has all the necessary informations. After it is boxed, the state machine is called on the box value using box value as arguement.

Do note that the boxing happens only once. The State machine also ensures that the builder has a reference to the single boxed version of the state machine.

This can be noticed if you dig a deep into the code of AsyncMethodBuilderCore.SetStateMachine

public void SetStateMachine(IAsyncStateMachine stateMachine)
{
	if (stateMachine == null)
	{
		throw new ArgumentNullException("stateMachine");
	}
	if (m_stateMachine != null)
	{
		throw new InvalidOperationException(Environment.GetResourceString("AsyncMethodBuilder_InstanceNotInitialized"));
	}
	m_stateMachine = stateMachine;
}

We will leave the SetStateMachine here because that’s all it does. Its role would be more visible once we examine the MoveNext method in detail.

Asynchronous Code – Behind the Scenes – 002

In the earlier part of this series, we reviewed the generic structure of decompiled async code, especially the stub method. In this part, we would continue our explore of async code and look into the State Machine. We would not delve deep into the most important MoveNext() method yet, we will first familiar with the different parts of the State Machine first.

State Machine

Let us go back to the ILSpy and see how State Machine looks like.

[StructLayout(LayoutKind.Auto)]
[CompilerGenerated]
private struct <Foo>d__1 : IAsyncStateMachine
{
	public int <>1__state;

	public AsyncTaskMethodBuilder <>t__builder;

	public int delay;

	private TaskAwaiter <>u__1;

	private void MoveNext()
	{
		// To be discussed later
	}

	void IAsyncStateMachine.MoveNext()
	{
		//ILSpy generated this explicit interface implementation from .override directive in MoveNext
		this.MoveNext();
	}

	[DebuggerHidden]
	private void SetStateMachine(IAsyncStateMachine stateMachine)
	{
		<>t__builder.SetStateMachine(stateMachine);
	}

	void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine stateMachine)
	{
		//ILSpy generated this explicit interface implementation from .override directive in SetStateMachine
		this.SetStateMachine(stateMachine);
	}
}

  • IAsyncStateMachine Interface

One of the first things we would notice is the implementation of IAsyncStateMachine interface. The IAsyncStateMachine interface, which is defined under the System.Runtime.CompilerServices namespace, represents the state machine generated for the async method. The interface itself is a simple one, with just two methods in it.

public interface IAsyncStateMachine
{
    /// <summary>Moves the state machine to its next state.</summary>
    void MoveNext();
    /// <summary>Configures the state machine with a heap-allocated replica.</summary>
    /// <param name="stateMachine">The heap-allocated replica.</param>
    void SetStateMachine(IAsyncStateMachine stateMachine);
}

The MoveNext() as explained earlier, represents the heart of asynchronous code. We would, for time being, delay visiting the method for a bit longer. However, the key point to remember at this point of time is that each time the State Machine is starts or resumes (after a pause), the MoveNext() method would be called. The SetStateMachine() method associates the builder with the specific state machine.

The importance of the implementation of the interface and how it binds the state machine with the stub method could be understood by looking at the signature of the AsyncTaskMethodBuilder.Start(). The method accepts a single generic parameter, which has a constraint of having implemented the IAsyncStateMachine.

public void Start<TStateMachine>(ref TStateMachine stateMachine) where TStateMachine : IAsyncStateMachine
{
	if (stateMachine == null)
	{
		throw new ArgumentNullException("stateMachine");
	}
	ExecutionContextSwitcher ecsw = default(ExecutionContextSwitcher);
	RuntimeHelpers.PrepareConstrainedRegions();
	try
	{
		ExecutionContext.EstablishCopyOnWriteScope(ref ecsw);
		stateMachine.MoveNext();
	}
	finally
	{
		ecsw.Undo();
	}
}

We would not go too deep into AsyncTaskMethodBuilder.Start(), but key take away would be

  • The constraint applied to parameter where TStateMachine : IAsyncStateMachine
  • The method is responsible for calling IAsyncStateMachine.MoveNext()

There is another interesting fact to this look at this point. The generated State Machine has a small but significant difference depending on whether your are looking at debug/release mode code. When in release mode, the compiler optimizes the code and creates a stuct based State Machine, while in debug mode, it creates a class. This is supposed to an optimization done to so that the compiler would skip allocating memory when the awaitable has already been completed awaited. The following code displays the State Machine when decompiled in debug mode.

[CompilerGenerated]
private sealed class <Foo>d__1 : IAsyncStateMachine
{
	public int <>1__state;

	public AsyncTaskMethodBuilder <>t__builder;

	public int delay;

	private TaskAwaiter <>u__1;

	private void MoveNext()
	{
		// To be discussed later
	}

	void IAsyncStateMachine.MoveNext()
	{
		//ILSpy generated this explicit interface implementation from .override directive in MoveNext
		this.MoveNext();
	}

	[DebuggerHidden]
	private void SetStateMachine(IAsyncStateMachine stateMachine)
	{
	}

	void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine stateMachine)
	{
		//ILSpy generated this explicit interface implementation from .override directive in SetStateMachine
		this.SetStateMachine(stateMachine);
	}
}


  • Fields

public int <>1__state;
public AsyncTaskMethodBuilder <>t__builder;
public int delay;
private TaskAwaiter <>u__1;

The next thing one would notice with the generated code is the presence of certains fields in the state machine. The fields could be broadly categorized into

  • Current State : As discussed in earlier post, this could have any of the following values
-1 : Not Started
-2 : Completed
Any other Value : Paused

  • Method Builder : Communicates with the async infrastructure and returns the task
  • TaskAwaiter
  • Parameters and local variables
  • Temporary Stack Variables

TaskAwaiter and parameters are used to remember the values when the State Machine resumes after a Pause. If the state machine requires a variable which it doesn’t need to remember after resuming, then it remains as private variable.

Temporary stack variables are used as a part of larger expression, when the compiler needs to remember intermediate results. For example,

int result = x + y + await task;

The most important point to remember about the Fields and Variables is that the compiler ensures it uses minimum fields/variables as possible by reusing them.

If your code have multiple await that was supposed to return

  • Task<int>
  • Task<string>
  • Task

Then the compiler would most likely create just 3 awaiters, one each for the different types involved.

That is it about the general structure of the State Machine. We would now proceed to the most important part, which is of course the MoveNext() method. We will do it in the next post.

TPL And Asynchronous Programming

This series follows my study notes on TPL and asynchronous programming. A special thanks to Jon Skeets and Jeffrey Richter for their wonderful books,which has been the inspirations behind this series.

Asynchronous Code – Behind the Scenes – 001

If you were to ask me what was the biggest milestone in .Net development, then my choice would definetly be .Net 5.0 – especially the introduction of the async/await. The more you learn about the underlying working, you cannot but stop and admire the efforts done by the lang uage developers to make our life easier.

In this mini series on asynchronous programming in .Net, we will delve deeper into the fascinating world of async await and learn more about what happens behind the scenes. I thought i would structure it as a mini-series rather than a single monolythic post as this is a vast topic (at least for an average developer like me).

Setting the Stage

Let us begin by writing a simple async method, which we would then decompile using ILSpy to know what happens beneath. We will keep the base code as simple as possible.

class Program
{
    static void Main(string[] args)
    {
        Foo(10);
    }

    static async Task Foo(int delay)
    {
        await Task.Delay(delay);
        Console.WriteLine(delay);
        await Bar();
    }

    static async Task Bar()
    {
        await Task.Delay(100);
    }
}

For the demonstration purpose, we will use ILSpy for decompiling our code, but please feel free to choose any decompiler you are comfortable with. In case you are using ILSpy, please ensure you have the following settings unchecked.

View->Options->Decompiler->C# 5.0-> Decompile async methods

This would ensure we could view the decompiled async code and the associated state machine. Okay, now let us see what ILSpy has to offer us after decompiling our code. We will read the code in parts, so that it is easier for us to understand the whole structure.

  1. Stub Method
  2. State Machine structure
  3. MoveNext Method method

Stub Method

As you would be already aware, the async methods are implemented with the help of State Machines internally.

Stubs are methods which has the same signature as your original async method and is responsible for creating the state machine. Let us check the decompiled stub method.

[AsyncStateMachine(typeof(<Foo>d__1))]
[DebuggerStepThrough]
private static Task Foo(int delay)
{
	<Foo>d__1 stateMachine = new <Foo>d__1();
	stateMachine.delay = delay;
	stateMachine.<>t__builder = AsyncTaskMethodBuilder.Create();
	stateMachine.<>1__state = -1;
	stateMachine.<>t__builder.Start(ref stateMachine);
	return stateMachine.<>t__builder.Task;
}

[AsyncStateMachine(typeof(<Bar>d__2))]
[DebuggerStepThrough]
private static Task Bar()
{
	<Bar>d__2 stateMachine = new <Bar>d__2();
	stateMachine.<>t__builder = AsyncTaskMethodBuilder.Create();
	stateMachine.<>1__state = -1;
	stateMachine.<>t__builder.Start(ref stateMachine);
	return stateMachine.<>t__builder.Task;
}

Let us skip the attributes for a moment (we will come back to very shortly), and first concentrate on signature of both Stub methods. As mentioned earlier, both shares the same signature with their original async methods.

The Stub Method is responsible for creating/initializing the State Machine and Starting it. The state machines are initialized with following

  • Parameters as Fields

Any parameter in the original async method are added as Fields in the State Machine. For example, if you inspect the following code from Foo Stub Method.

stateMachine.delay = delay;

The Foo method, if you remember, accepted a single parameter of Type int and was called delay. This parameter would be now added a field in the State Machine.

  • Type of Builder

The type of Builder varies depending on the return type of the Method in question.

Return TypeBuilder
TaskAsyncTaskMethodBuilder
Task<TResult>AsyncTaskMethodBuilder<T>
voidAsyncVoidMethodBuilder
Custom Task TypeBuilder specified by AsyncTaskMethodBuilderAttribute

In the above scenario, both methods returns Task. For the same reason, the both uses an AsyncTaskMethodBuilder. Please note that the Custom Task Type was introduced only with C# 7. Prior to C# 7, only the first 3 builders were applicable.

The attribute AsyncStateMachine points to the method’s particular state machine and aids in tooling.

  • State of the State Machine

A async method could be in either the following states – Not Started – Executing – Paused – Completed (Successfully or Faulted)

Out of these, the most important state for the State Machine is the Paused State. While in the Executing State, the async method is pretty much like synchronous code as it passes through each instruction. The CPU would keep track of the currently executing step via Instruction Pointer.

However, the state machine comes into picture immediately as the method pauses when it reaches an await (incomplete) expression. Please note that this is applicable only for async expression that has not been completed. In case the awaited expression is completed, the code works similiar to synchronous code and the state machine would not be brought into picture.

Each time the state has to be paused, the state is recorded so that once the operation awaited is completed, the method could be continued.

As one can expected, the Stub method would like to set the initial state of the state machine to Not Started.

The State Property of the state machine handles the current state with following value codes

ValueDescription
-1Not Started
-2Completed (Successfully/Faulted)
Any other valuePaused at an await Expression

As one can observe, these are the values which the Stub method initializes the State Machine with.

<Foo>d__1 stateMachine = new <Foo>d__1();
stateMachine.delay = delay;
stateMachine.<>t__builder = AsyncTaskMethodBuilder.Create();
stateMachine.<>1__state = -1;
stateMachine.<>t__builder.Start(ref stateMachine);

Additional, it also starts the State Machine using the Builder.Start method, passing a reference to the State Machine created. The important point to note here is that the stateMachine is passed as reference to the method. This is because both StateMachine and AsyncTaskBuilder are mutable value types, and passing the instance by reference ensures no local copy are created. One can notice a lot of optimization done by the compiler.

The final step of the Stub method is of course, to return the Task object. The Task object is created by the Builder, who also ensures the Task’s state is changed accordingly as the method execution progresses.

return stateMachine.<>t__builder.Task;

When the Builder.Start method is invoked, it begins executing the MoveNext method (we will discuss it later) untill the method reaches an incomplete await expression. At this point, the MoveNext would return the Task, following up which, the Start method also returns. The task is then returned to the Caller method.

That’s it for the Stub Method, in the next section, we will look into the State Machine structure.

Revisiting Exception Handling in async methods

It is interesting to observe how the Exceptions behave in async code. While the .Net framework attempts to ensure the exprerience of handling failures in async methods are as similar to the synchronous methods, there are subtle differences which is worth understanding.

Let us examine the following code.

async Task<string> Foo()
{
	var client = new HttpClient();
	try
	{
		return await client.GetStringAsync("http://www.InvalidUrl.com");
	}
	catch(HttpRequestException ex)
	{
		Console.WriteLine($"Exception of Type {ex.GetType()} has been raised");
	}
	catch(AggregateException ex)
	{
		Console.WriteLine($"Exception of Type {ex.GetType()} has been raised");
	}
	catch(Exception ex)
	{
		Console.WriteLine($"Exception of Type {ex.GetType()} has been raised");
	}
	return default;
}

What could be the output of the above code ? Specifically, what type of an exception would be caught assuming the Url is invalid ?

Before we get to the answer, let us examine how the returned Task<string> object indicate the failure.

Property/MethodIndication
StatusFaulted
IsFaultedtrue
ExceptionAggregateException
Wait()Throws AggregateException
ResultThrows AggregateException
await TaskThrows First Exception with AggregateException

The last 3 rows holds importance to our question. Let us reexamnine the method call.

return client.GetStringAsync("http://www.InvalidUrl.com").

In the above case, we would recieve an AggregateException with an HttpRequestException within it.

When Exception causes Status |= Faulted

As mentioned above, the status of Task would be set to Faulted in most cases, except for one particular kind of Exception,the OperationCancelledException.

Let us write some code before we discuss this further.

async void Main()
{
	var task = Foo();
	try
	{
		await task;
	}
	catch(Exception Ex)
	{
		Console.WriteLine($"Task Status:{task.Status}, Exception:{Ex.Message}");
	}
}

async Task Foo()
{
	throw new OperationCanceledException();
}

Examining the output

Task Status:Canceled, Exception:The operation was canceled.

The TPL uses OperationCanceledException when a Token from CancelationTokenSource is canceled by the calling method. If a method like the code above, decides to throw this special exception, then instead of the Status being set to Faulted, the Status is set to Canceled.

Lazy Exceptions

There is another aspect of Exception Handling in async methods that are worth examining. An async task would not directly throw an exception, instead it would return a faulted Task. Significance of the method could be better understood with help of a bit of code.

async void Main()
{
	try
	{
		var fooTask = Foo(-3);
		Console.WriteLine("Task is not awaited yet");
		await fooTask;
		Console.WriteLine("Task Completed");
	}
	catch(ArgumentException ex)
	{
		Console.WriteLine($"{nameof(ArgumentException)} has been raised");
	}
}


public async Task<string> Foo(int value)
{
	Console.WriteLine($"Method {nameof(Foo)} Invoked");
	if(value<0)
	{
		throw new ArgumentException();
	}
	
	Console.WriteLine($"Method {nameof(Foo)} mocking real task via Delay");
	await Task.Delay(1000);
	return default;
}

The Foo() method has a precondition check which validates if the passed arguement is a positive number. Otherwise, it raises an ArgumentException. With the example code, the method invoker is passing a negative value to the method, and should hit the precondition block.

Let us examine the output and discuss further.

Method Foo Invoked
Task is not awaited yet
ArgumentException has been raised

As you can observe, the message “Task is not awaited yet” is displayed before the exception thrown. This is because exceptions would not be raised untill the task is awaited (or completed). This lazy nature of evaluation of exceptions could be useful at most times, but in times such as above, where preconditions needs to be evaluated and the developer would prefer an early evaluation, this would need a slight workaround.

The idea, similar to how we made iterator methods to evalute exceptions early (and as John Skeets mentions in his invaluable book series C# in Depth), lies in introducing a synchronous method which does the arguement validation, and which in-turn calls the original method. If the original method is moved as an internal method of proposed method, the original method can now safely assume that the arguements are validated.

public Task<string> Foo(int value)
{
	Console.WriteLine($"Method {nameof(Foo)} Invoked");
	if(value<0)
	{
		throw new ArgumentException();
	}
	
	async Task<string> FooAsync()
	{
		Console.WriteLine($"Method {nameof(Foo)} mocking real task via Delay");
		await Task.Delay(1000);
		return default;
	}
	return FooAsync();
}

This ensures the validation and subsequent exception is evaluated early. Let’s hit F5 and run our code now.

Method Foo Invoked
ArgumentException has been raised

As observed, the exception has been evaluated early and we get the much desired result.

That’s all for now, see you soon again

Dragging Shapes in Wpf

One of the projects I have in mind as my side-projects needs ability to drag-move shapes on a Canvas. I thought it would idea to explore the possibilities of achieving it via test projects first before starting the real one. So the goal of this article would be
* Create a Rectangle in a Canvas
* Support ability to Drag-move it within the Canvas boundary

Let’s begin by defining the basic UI including our Canvas and Rectangle. We would be using Caliburn Micro for supporting MVVM in the example.

<Canvas Grid.Row="2" Background="AliceBlue" >
<Rectangle Height="100" Width="100" Fill="Red" Canvas.Left="{Binding Left}" Canvas.Top="{Binding Top}" />
</Canvas>

Our ViewModel, at the moment looks like,

public class ShellViewModel:PropertyChangedBase
{
public double Left { get; set; } = 10;
public double Top { get; set; } = 10;
}

The Left and Top are double properties which will have the position of the Rectange with respect to the Canvas.

The next step is to capture the Mouse Position each time the User holds the Left Mouse Button down and drags the object. For this there are few things needs to be done. We need to detect when the Left Mouse Button is pressed down and released. Also, we need to trace the Mouse Position when the User holds the Left Mouse Down within the rectangle.

To detect the mouse position, we will write a Behavior.

public class MouseMoveBehavior:Behavior<Canvas>
{
public double MouseTop
{
get { return (double)GetValue(MouseTopProperty); }
set { SetValue(MouseTopProperty, value); }
}

public static readonly DependencyProperty MouseTopProperty =
DependencyProperty.Register("MouseTop", typeof(double), typeof(MouseMoveBehavior), new PropertyMetadata(0d));

public double MouseLeft
{
get { return (double)GetValue(MouseLeftProperty); }
set { SetValue(MouseLeftProperty, value); }
}

public static readonly DependencyProperty MouseLeftProperty =
DependencyProperty.Register("MouseLeft", typeof(double), typeof(MouseMoveBehavior), new PropertyMetadata(0d));

protected override void OnAttached()
{
AssociatedObject.MouseMove += AssociatedObject_MouseMove;
}

private void AssociatedObject_MouseMove(object sender, System.Windows.Input.MouseEventArgs e)
{
var currentPosition = e.GetPosition(AssociatedObject);
MouseLeft = currentPosition.X;
MouseTop = currentPosition.Y;
}

protected override void OnDetaching()
{
AssociatedObject.MouseMove -= AssociatedObject_MouseMove;
}
}

The Behavior MouseMoveBehavior defines two Dependency Properties named MouseTop and MouseLeft indicating the X-Y cordinates of the mouse position. The behavior intends to capture the Mouse position using the MouseEventArgs.GetPosition and update the dependency properties with the Position.X and Position.Y values.

So, we have our behavior in place. Now is the time to update our Xaml to use our Behavior.

<Canvas Grid.Row="2" Background="AliceBlue" cal:Message.Attach="[Event MouseMove]=[Action MouseMove];
[Event PreviewMouseLeftButtonUp]=[Action MouseUp];[Event MouseLeave]=[Action MouseUp]" >
<i:Interaction.Behaviors>
<behaviors:MouseMoveBehavior MouseLeft="{Binding Left,Mode=TwoWay}" MouseTop="{Binding Top,Mode=TwoWay}"/>
</i:Interaction.Behaviors>
<Rectangle Height="100" Width="100" Fill="Red" Canvas.Left="{Binding Left}" Canvas.Top="{Binding Top}" cal:Message.Attach="[Event PreviewMouseLeftButtonDown]=[Action MouseDown]" />
</Canvas>

As you can observe, we are updating the Left and Top properties by capturing them in Behavior and assigning the the values of current mouse position. This would ensure the the shape would move along the mouse pointer.

But you want to update the Left and Top position only when the Mouse Left Button is pressed and dragged, not when it is freely moved. To help us achieve this, we will introduce three additional properties in our ViewModel. Let us rewrite our ViewModel. This would also explain why we are capturing the Mouse Events in the XAML, which was intensionally not explained till this point.

public class ShellViewModel:PropertyChangedBase
{
public double Left { get; set; } = 10;
public double Top { get; set; } = 10;

public double CurrentMouseX { get; set; }
public double CurrentMouseY { get; set; }

public bool IsShapeCaptured { get; set; }

public void MouseDown()
{
IsShapeCaptured = true;
NotifyOfPropertyChange(nameof(IsShapeCaptured));
}

public void MouseMove()
{
if (!IsShapeCaptured) return;
Left = CurrentMouseX;
Top = CurrentMouseY;
NotifyOfPropertyChange(nameof(Left));
NotifyOfPropertyChange(nameof(Top));
}

public void MouseUp()
{
IsShapeCaptured = false;
NotifyOfPropertyChange(nameof(IsShapeCaptured));
}
}

The IsShapeCaptured is enabled only when the Mouse Left Button is pressed. As soon as the Mouse Button is released, you are resetting the IsShapeCaptured flag. The MouseMove method ensures that the Left & Top properties (which are bound to the position of Shape) are updated only when the IsShapeCaptured is set.

I guesss that bit of code is quite self explanatory. This is one way to achieve the drag/drop functionality, but obviously there could be others. Will provide another solution in coming days.