A2Z.Net : A – Anonymous Types

This article marks the beginning of a new series of posts, which aims to dig deeper into 26 different features of .Net. For the first article, or the A – I have selected Anonymous Types. I would have chosen asynchronous programming, but I have whole dedicated series on asynchronous programming which could followed in this link.

Introduced in .Net 3.x, Anonymous Types has made the life easy for the developers by handling out the ability to work on an encapsulated type, without having to actually define one. I should say encapsulate is a wrong word here (but have used it so that I could highlight why) because you cannot place any behavior with the properties.

Anonymous Types and Syntax

Anonymous Type provides an easy way to encapsulate a set of properties without having to explicitly define a type. Notice that I used the word explicitly here – that is because the compiler creates the type for you under the hood. We will look into what happens behind the scenes a little later, to begin with, let us introduce the syntax of defining an anonymous type.

var user = new
{
    UserName ="Anu Viswan",
    Age = userRecord.Age,
    JoiningDate= DateTime.Now
};

The above code uses object initializer (without having to specify the data type) to create an instance of an anonymous type initialized with 3 properties – namely UserName(string),Age(int), and JoiningDate (DateTime). You could also create an anonymous type.

As I mentioned earlier, the compiler generates a type under the hood and ensure the code is still statically typed.

Since the type of anonymous Type is known statically, they must be stored using variables with var allowing type inference. You could also store them as object, but that would defeat the whole purpose of anonymous types.

In addition to the creation syntax mentioned above, framework also provides another way initializing an anonymous type. This syntax is known as projection initializer, and it allows you to use the property/field names from somewhere else.

var person = new Person
{
    FirstName = "Anu",
    LastName = "Viswan",
    Age = 36
};

var user = new
{
    person.FirstName,
    person.Age,
};

The above code uses a part of type Person to create an anonymous type. In this case, as you can notice, you are not explictly providing names for properties, instead, they uses the same name as the Person. This is of course equavalent of writing as

var user = new
{
    FirstName = person.FirstName,
    Age = person.Age
};

The projection initializer syntax is highly useful in Linq queries when you need to project a subset of a query or combine properties from multiple objects. For Example,

var personList = new List<Person>
{
	new Person {UserName="Anu Viswan",    Age = 36},
	// Other Persons
};

var employeeList = new List<Employee>
{
    new Employee{UserName="Anu Viswan",JoiningDate=new DateTime(2020,2,2)},
    // Other employee records //
};

var user = employeeList.Join(personList,
							e=>e.UserName,
							p=>p.UserName,
							(e,p)=> new
								{
									e.UserName,
									UserDetail = new  // An example of Nested Anonymous Type
									{
									e.JoiningDate,
									p.Age,
									QueryTimeStamp=DateTime.Now
									}
								});

As you can observe the creation of anonymous types are pretty simple. You could also create a type combining multiple objects as seen in above example. The above code also demonstrates that the anonymous types could be nested. UserDetail is an anonymous type within the user.

Behind the Scenes

Alright, enough of syntax, let us now look behind the scenes now. Let us bring back our first example of anonymous type and see how the type generated by the compiler looks like.

var user = new
{
    UserName = "Anu Viswan",
    Age = 36,
    JoiningDate= DateTime.Now
};

The compiler generated (_generated using ILSpy) type would look like

[CompilerGenerated]
[DebuggerDisplay("\\{ UserName = {UserName}, Age = {Age}, JoiningDate = {JoiningDate} }", Type = "<Anonymous Type>")]
internal sealed class <>f__AnonymousType0<<UserName>j__TPar, <Age>j__TPar, <JoiningDate>j__TPar>
{
	[DebuggerBrowsable(DebuggerBrowsableState.Never)]
	private readonly <UserName>j__TPar <UserName>i__Field;

	[DebuggerBrowsable(DebuggerBrowsableState.Never)]
	private readonly <Age>j__TPar <Age>i__Field;

	[DebuggerBrowsable(DebuggerBrowsableState.Never)]
	private readonly <JoiningDate>j__TPar <JoiningDate>i__Field;

	public <UserName>j__TPar UserName => <UserName>i__Field;

	public <Age>j__TPar Age => <Age>i__Field;

	public <JoiningDate>j__TPar JoiningDate => <JoiningDate>i__Field;

	[DebuggerHidden]
	public <>f__AnonymousType0(<UserName>j__TPar UserName, <Age>j__TPar Age, <JoiningDate>j__TPar JoiningDate)
	{
		<UserName>i__Field = UserName;
		<Age>i__Field = Age;
		<JoiningDate>i__Field = JoiningDate;
	}

	[DebuggerHidden]
	public override bool Equals(object value)
	{
		// Hidden : Will be discussed later
	}

	[DebuggerHidden]
	public override int GetHashCode()
	{
		// Hidden : Will be discussed later
	}

	[DebuggerHidden]
	public override string ToString()
	{
		// Hidden : Will be discussed later
	}
}


Characteristics of Generated Code

Let us break down the generated code and examine it in parts.

It is a Generic Class

As you would have already observed, the generated type is a class, name of which is generated by the compiler and cannot be accessed from your source code.

internal sealed class <>f__AnonymousType0<<UserName>j__TPar, <Age>j__TPar, <JoiningDate>j__TPar>
{
    public <>f__AnonymousType0(<UserName>j__TPar UserName, <Age>j__TPar Age, <JoiningDate>j__TPar JoiningDate)
	{
		<UserName>i__Field = UserName;
		<Age>i__Field = Age;
		<JoiningDate>i__Field = JoiningDate;
	}
}

It is also interesting to note that the generated type is a Generic class, with one type parameter for each of the properties. This would mean that if there is a second string, it would still have a different type parameter than the original one. For example,

var user = new
{
    UserName = "Anu Viswan",
    FirstName = "Anu"
};

The above would translate to

internal sealed class <>f__AnonymousType0<<UserName>j__TPar, <FirstName>j__TPar>
{
}

Note that the class is marked internal and sealed. Now both these characterstics aren’t guaranteered as per Jon Skeet’s book C# in Depth. This is actually interesting, as to know when and why the compiler could not guarante a consistent behavior.

The generated type also has a constructor which accepts parameters which are in consistent with all the properties in the type.

Read Only Properties

The other characterstics of the generated type that is quite obvious is that all properties are read-only. One cannot reassign properties of an anonymous type. If the need arise, you would have to recreate the entire object.

var user = new
{
    UserName = "Anu Viswan",
    Age = 36,
    JoiningDate = DateTime.Now,
};

user.Age = 37; // This line would throw error

user = new  // The correct approach would be recreate
{
    UserName = user.UserName,
    Age = 37,
    JoiningDate = user.JoiningDate,
};

Derieved from System.Object & Overrides Equals()/GetHashCode()

The generated class for anonymous types is derieved from System.Object. It cannot be, hence, cast to any other type other than Object. The generated type also overrides the Equals() and GetHashcode() methods.

public override bool Equals(object value)
{
    <>f__AnonymousType0<<UserName>j__TPar, <Age>j__TPar, <JoiningDate>j__TPar> anon = value as <>f__AnonymousType0<<UserName>j__TPar, <Age>j__TPar, <JoiningDate>j__TPar>;

    return anon != null && EqualityComparer<<UserName>j__TPar>.Default.Equals(<UserName>i__Field, anon.<UserName>i__Field) && EqualityComparer<<Age>j__TPar>.Default.Equals(<Age>i__Field, anon.<Age>i__Field) && EqualityComparer<<JoiningDate>j__TPar>.Default.Equals(<JoiningDate>i__Field, anon.<JoiningDate>i__Field);
}

public override int GetHashCode()
{
    return ((459989953 * -1521134295 + EqualityComparer<<UserName>j__TPar>.Default.GetHashCode(<UserName>i__Field)) * -1521134295 + EqualityComparer<<Age>j__TPar>.Default.GetHashCode(<Age>i__Field)) * -1521134295 + EqualityComparer<<JoiningDate>j__TPar>.Default.GetHashCode(<JoiningDate>i__Field);
}

These ensures that two instances of AnonymousTypes are equal only if all the properties are equal. Do note that the exact implementation of GetHashCode() might also vary. For example,

var user = new
{
    UserName = "Anu Viswan",
    Age = 36,
    JoiningDate = new DateTime(2020,10,10),
};

var anotherInstance = new
{
    UserName = "Anu Viswan",
    Age = 36,
    JoiningDate = new DateTime(2020, 10, 10),
};

Console.WriteLine($"user & anotherInstance - Equality :{user.Equals(anotherInstance)}"); // True

Reuse of Anonymous Types

If two anonymoys types are created with same Property Names, in same order and type, then the compiler would treat the two objects as of same type, ie, essentially reuse the generated type.

For example, the preceeding code would reuse the same generated type.

Console.WriteLine($"user & anotherInstance - Type Equality :{user.GetType() == anotherInstance.GetType()}"); // True

This could verified when we check the generated code.

<>f__AnonymousType0<string, int, DateTime> user = new <>f__AnonymousType0<string, int, DateTime>("Anu Viswan", 36, new DateTime(2020, 10, 10));

<>f__AnonymousType0<string, int, DateTime> anotherInstance = new <>f__AnonymousType0<string, int, DateTime>("Anu Viswan", 36, new DateTime(2020, 10, 10));

However, if you change the order of Properties in either of the anonymous types, the compiler would generate another type. Consider the following code.

 var user = new
 {
     UserName = "Anu Viswan",
     Age = 36,
     JoiningDate = new DateTime(2020,10,10),
 };
  var differentInstance = new
 {
     UserName = "Anu Viswan",
     JoiningDate = new DateTime(2020, 10, 10),
     Age = 36,
 };

As you can observe, while the PropertyNames and Types are preserved in the two anonymous types, the order is changed.The generated code would be as following.

<>f__AnonymousType0<string, int, DateTime> user = new <>f__AnonymousType0<string, int, DateTime>("Anu Viswan", 36, new DateTime(2020, 10, 10));

f__AnonymousType1<string, DateTime, int> differentInstance = new <>f__AnonymousType1<string, DateTime, int>("Anu Viswan", new DateTime(2020, 10, 10), 36);

As seen in the generated code, the compiler generates and use different internal types for the two anonymous types.

This holds true only when the two anonymous types are in the same assembly. If the two anonymous types are defined in two different assemblies, compiler would generate two different types for each. We will explore this later in this article.

Points Of Interest

The above section outlined the characterstics of Anonymous Types. There are however, certain points which is worth noting while learning about Anonymous Types.

Pass as Parameter/ Return Type

Since the generated type is not accessible/addressed by the developer in his code, it to an extend limits the transfer an anonymous types from one method to another as you cannot express the type as Parameter. However, you could make use of Generics to pass the anonymous types around. For example,

 public void Print<T>(T data)
 {
     var properties = typeof(T).GetProperties();
     foreach(var property in properties)
     {
         Console.WriteLine($"{property.Name} = { property.GetValue(data)}");
     }
 }

The compiler here treats the generated type as any reference type in the system. This allows us to use generics to pass anonymous types around.

You could also pass the anonymous type as an object or dynamic parameter as well, however, that would take away any benefits associated to anonymous types.

Similiar Anonymous Types in Two Assemblies

In the earlier section we noticed how the compiler would reuse the same generated type if the Property Names, Order and Types are the same. However, there is a caveat. The generated types are localized to the assembly.

If the two types resides in different assembly, then the compiler would generate different Anonymous Types for each of them.

Consider the following scenario.

// Assembly 1
 var user = new
 {
     UserName = "Anu Viswan",
     Age = 36,
     JoiningDate = new DateTime(2020,10,10),
 };
 var bar = new Bar();
 bar.Print(user);


// Assembly 2
public class Bar
{
    public void Print(object data)
    {
        var typeSchema = new { UserName = string.Empty, Age = default(int), JoiningDate = default(DateTime) };
        var castInstance = Cast(typeSchema, data);
        Console.WriteLine(data.GetType() == castInstance.GetType());
    }
    private static T Cast<T>(T typeHolder, Object x) where T : class
    {
        return (T)x;
    }
}

The above casting would raise an InvalidCastException as the two types resides in different assemblies, resulting in compiler generating two different anonymous types.

This would have been a succesful cast if the code resided in the same assembly.

Conclusion

As observed, the implement of Anonymous Types is pretty simple, and so is its usage. The compiler generates a generic class for each unique anonymous type within the assembly. The feature has been extensively used since Linq came along as it allows to create subsets from queries without having to define an explicit type.

The complete code citd in this article is available in my Github.

MemoryCache – AddOrGetExisting Method

MemoryCache provides an easy to use method AddOrGetExisting which provides an easy way to retrive or add data to cache. While it is extremely convenient, it comes off at a cost. Let’s explore it a bit along with alternative.

Note : For the sake of example, let us ignore multi-threading scenarios for the moment.

   public class MemoryCacheRunner
   {
        private MemoryCache _memoryCache;
        private string _uniqueKey = "UniqueKey";
        public MemoryCacheRunner()
        {
            _memoryCache = new MemoryCache("DemoCache");
            _memoryCache.Add(_uniqueKey, new Foo { Name = "Anu Viswan", Age = 36 }, new CacheItemPolicy());
        }
   }

There are two ways we could check if the a particular key exists in the cache and retrieve it (after adding if it is not already present).

Option 1 – Using AddOrGetExisting

public Foo GetFromCacheUsingAddOrGet()
{
    var instance = new Foo { Name = "Anu Viswan", Age = 36 };
    return (Foo)_memoryCache.AddOrGetExisting(_uniqueKey, instance, new CacheItemPolicy());
}

As you can observe, that looks pretty easy and convenient. This is far less code the following approach.

Option 2 – Using Contains/Get

public Foo GetFromCacheUsingContainsAndGet()
{
    if (_memoryCache.Contains(_uniqueKey))
    {
        return (Foo)_memoryCache.Get(_uniqueKey);
    }
    var instance = new Foo { Name = "Anu Viswan", Age = 36 };
    _memoryCache.Add(_uniqueKey, instance, new CacheItemPolicy());
    return instance;
}

That’s definetly a lot more code. But if one was to carefuly observe, Option 2 is infact far more efficient.

In the first approach, you are forced to create an instance of Foo, even if the object exists in the cache. The second approach, however, creates a new instance only if the cache doesn’t contains the specified key.

This could be insignificant when the size of data type is pretty small, but as the size of data increases, one has to be wary of such differences to make your application work optimally.

Stay Open Functionality for Oxyplot Tracker

One of the recent questions on Stackoverflow was about having a Oxyplot tracker that

  • Is evoked only over the Points and NOT line/outside.
  • Stays Open until another point is selected.

The first part could be easily achieved by using a Custom PlotController that Binds the Mouse Down event only the Left Mouse Button and only on Tracks

CustomPlotController = new PlotController();
CustomPlotController.UnbindAll();
CustomPlotController.BindMouseDown(OxyMouseButton.Left, PlotCommands.PointsOnlyTrack);

You could now bind the CustomPlotController with your Xaml as

<oxy:PlotView Model="{Binding MyModel}" Controller="{Binding CustomPlotController}">

The second part needs you to write a Custom TrackerManipulator. The core idea would revolve around override the Completed method in TrackerManipulator, so that the Oxyplot is unable to hide the tracker.

If we were to ensure with the same TrackerManipulator that the trackers is enabled only Points, then the current tracker would be left open until the new tracker is opened at new location by a succeeding click on another point in the graph.

Let us go ahead and write our Custom TrackerManipulator.

public class StaysOpenTrackerManipulator : TrackerManipulator
{
    public StaysOpenTrackerManipulator(IPlotView plotView) : base(plotView)
    {
        Snap = true;
        PointsOnly = true;
    }
    public override void Completed(OxyMouseEventArgs e)
    {
        // Do nothing
    }
}

That’s all you need to achieve the goal. You can find the source code of this post in my Github Repo

C# 9.0 : Top Level Programs and Targeted type ‘new’ Expression

In the previous post, we saw how C# 9.0 introduced the init only properties. In this blog post, we will explore some more of the language features which would be introduced in C# 9.0.

Top Level Programs

One of the annonying quality of any programming language is the baggage of boiler plate code that needs to be written for trying out a one-line. For example, prior to C# 9, even hello word looks like following.

using System;

namespace CSharp9.TopLevelPrograms
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
        }
    }
}

That is plenty of boiler plate code for a simple hello world program. There was hardly any improvements in this regard, with the exception of static directive which was introduced in C# 6.

using static System.Console;

namespace CSharp9.TopLevelPrograms
{
    class Program
    {
        static void Main(string[] args)
        {
            WriteLine("Hello World!");
        }
    }
}

Even with that, there were plenty of boiler plate code. All that comes to an end with the introduction of Top Level Program. Yo could no do away with explicit Main and other boiler plate codes. Starting with C# 9, you could rewrite the above code as

using System;
Console.WriteLine("Hello World");

Or including the static directive

using static System.Console;
WriteLine("Hello World");

If you are trying out a new language feature, this would be highly useful. No longer do you need to all those unncessary boiler plate codes. Behind the scenes, nothing has changed, it is the same old Main which the compiler has generated.

// $Program
using System;

private static void $Main(string[] args)
{
	Console.WriteLine("Hello World");
}

The Top Level Programs are not limited to above, you could also write methods and classes. The methods gets converted to local methods. For example,

using static System.Console;
WriteLine($"Hello {GetName()}");
WriteLine($"Hello {new Foo().GetName()}");

string GetName() => "Anu Viswan";
class Foo
{
    public string GetName() => "Anu Viswan";
}

The above code gets converted to

using System;
using System.Runtime.CompilerServices;

[CompilerGenerated]
internal static class $Program
{
	private static void $Main(string[] args)
	{
		Console.WriteLine("Hello " + GetName());
		Console.WriteLine("Hello " + new Foo().GetName());
		static string GetName()
		{
			return "Anu Viswan";
		}
	}
}
internal class Foo
{
	public string GetName()
	{
		return "Anu Viswan";
	}
}


As you can see, behind the scenes, the compiler does all the magic to create the regular code, but enables the developers to write sample codes within minimum code.

Targeted Type ‘new’ Expression

How often have you wondered _what if, what if the compiler could detect the targeted type, without the developer having the mention it explicitly`. For example, consider the following code

public class Foo
{
    public string FirstName { get; set; }
    public string Get(Bar bar) => bar.UserName;  
}

public class Bar
{
    public string UserName { get; set; }
}

// client code
Dictionary<string, string> dictionary = new Dictionary<string,string>
            {
                ["Name1"] = "Anu Viswan",
                ["Name2"] = "Jia Anu"
            };
Foo foo = new Foo{ FirstName = nameof(Foo.FirstName) };
foo.Get(new Bar{ UserName = nameof(Bar.UserName) });

If you observe the client code, you could be left wondering

  • Why do you need to the Dictionary Type explicitly defined on Right Hand side when the assigned type has been defined.
  • Same is the case with the type Foo on second line of client code.
  • Why not omit the type when you know the method expects the particular type in the last line ? The compiler already knows that the method Get() accepts a parameter type of Bar.

All these are now answered by in C# 9.0 with the Targeted Type new expression. For example, now you could skip the duplicate types and omit the types when the types could be infered.

The above code could be rewritten as

Dictionary<string, string> dictionary = new()
            {
                ["Name1"] = "Anu Viswan",
                ["Name2"] = "Jia Anu"
            };
Foo foo = new (){ FirstName = nameof(Foo.FirstName) };
foo.Get(new() { UserName = nameof(Bar.UserName) });

Now isn’t that better. I am more excited about the last line, where you could skip the Type name if it could be inferred.

We will continue our exploration of the language features in upcoming posts.

Evolution of Properties : C# 1 to C# 9

Properties in .Net has evolved over time, retaining its core functionality while making it super sleek via subtle changes. Let us take a brief look at the evolution before digging in deeper about the features introduced in C# 9.

C# 1.0.

Back when it all started, if you were to declare a property in C#, you needed to write a lot of boiler plate code. For example, if you were to create a class to represent a person, it could be like

public class Person
{
    public string _firstName;
    public string _lastName;

    public string FirstName
    {
        get
        {
            return _firstName;
        }
        set
        {
            _firstName = value;
        }
    }

    public string LastName
    {
        get
        {
            return _lastName;
        }
        set
        {
             _lastName = value;
        }

    }
}

That was a lot of boiler plate code for something so simple with no extranous validation or execution of arbitary code. You had to declare backing up fields and manually declare getters/setters for each of the property.

C# 2.0

C# 2.0 didn’t make the life much easier, but it provided a significant improvement. It allowed us to write different access modifiers for getter and setter. For example,

public string _firstName;
public string FirstName
{
    get
    {
        return _firstName;
    }
    private set
    {
        _firstName = value;
    }
}

C# 3.0.

C# 3.0 brought the first significant improvement in property declation with the introduction of auto implemented properties. Now you no longer needed to declare backing fields(of course you could do it as well if needed). The Person class declared in the preceeding example could be now modified as following.

public class Person
{
    public string FirstName{get;set;}
    public string LastName{get;set;}
}

Compare that with the code we wrote in C# 1.0 and we realize how much cleaner the code looks now. Of course the compiler would generate the backing up fields behind the scenes, but as a developer that was a lot of ‘less’ code to write.

However, there was a major caveat still. The Auto implemented property syntax was restricted to read-write properties. You weren’t allowed to use the auto-implemented properties in the case of readonly properties, at least not yet. For Read-only properties, you were forced to write the backing fields still.

We could of course use the private accessor for the setter, but that wasn’t exactly readonly property. It would only restrict another class from changing the property, but it never constrains the class to change it within itself. For example,

public class Person
{
    public string FirstName{get;private set;}

    public void Foo()
    {
        FirstName = "Hey I can change you here";
    }
}

In other words, if you intend to create a imutable property which could be set only in the constructor, this approach didn’t quite work.

C# 6.0

Another problem with the private setter approach is that the intend is not clearly indicated. However, this got a major boost in C# 6, which allow us to skip the setter in the declaration, and there by making the intend more clear.

public string FirstName {get;}

It also ensured that we could now write the much desired immutable property pretty easily and with more concise code. That isn’t all with the improvements in C# 6 for properties. There was other improvements which go a long way in improving our exprience as developer.

Prior to C# 6, if you wanted to initialize the value of a Auto Implemented Property, you were forced to do it within a constructor (if the constructor was implicit, you are forced to make it explicit). For example,

public class Foo
{
    public string FirstName { get; set; }
    public Foo()
    {
        FirstName = "Initial Value";
    }
}

That’s way too verbose. Why would on want to introduce an explicit constructor, when you could have done it at point of declaration. However, the developers were deprieved of this feature till C# 6.0. With C# 6.0 and above, you could now do,

public class Foo
{
    public string FirstName { get; set; } = "Initial Value";
}

That is a lot more concise code. C# 6.0 also introduced Expression Bodied Members, which could be applied to properties as well. For example, if you want to have a Readonly Age property, which needs to be calculated from DateOfBirth, you could do so,

public class Foo
{

    public DateTime DateOfBirth { get; set; }
    public int Age => DateTime.Now.Year - DateOfBirth.Year;

}

C# 9.0

Dispite the improvements made to the one of the most basic functionality, there was still something lacking. In C# 3.0, Object initializers were introduced, which enabled us a new way to initialize properties at time of creation. For example,

public class Foo
{
    public string FirstName{get;set;}
    public string LastName{get;set;}
}

var foo = new Foo{
    FirstName = "Anu",
    LastName = "Viswan"
};

This freed the author of the class from writing all sorts of constructor, which was an alternative till then. However, there was something amiss here.

If your property is a immutable property, you cannot use object initialier syntax to initialize the value during creation. You would have to depend on a lot of boiler plate constructors. For example,

public class Foo
{
    public string FirstName { get; set; }
    public string LastName { get; }
}

var bar = new Foo
{
    FirstName = "Anu",
    LastName = "Viswan" // This throws an error
};

This problem was resolved in C# 9 with introduction of init only properties. The C# language designers introduced the initaccessor which could be only used with the initializers. For example, the above scenario could be rewritten as

public class Foo
{
    public string FirstName { get; set; }
    public string LastName { get; init; }
}
var foo = new Foo
{
    FirstName = "Anu",
    LastName = "Viswan" // This works !!!
};

init accessor is a variant of set, which works only with object initializers. As you can see, this is a small change, but it fills a void that was irritating for the developers.

That’s it for now.

C# 8 : Nullable Reference Types, Null Forgiving and Null Coalescing Assignment

There are some properties that ought to be learned together to understand the complete picture. In this blog post, we will address two of such features.

Nullable Reference Types

Let us begin by writing some code.

public class Foo
{
    public Bar Bar { get; set; }
}

public class Bar
{
    public int Id { get; set; }
}

// Client Code
var foo = new Foo();
Console.WriteLine(foo.Bar.Id);

Notice anything wrong ? How often have been guilty of attempting to access something that could potentially be null ? In the above case, we haven’t initialized the Foo.Bar property, but in the client code, we are attempting to access the Bar.Id property.

This would throw a NullReferenceException, as you are dereferencing a null value (Bar hasn’t been assigned yet). We should have checked for null in the first place.

But as the code base increases, there are times when we forget to do so. And unfortunately, prior to C# 8, there was no way compiler could warn us to do the null check even if we might have wondered many a times if it could do so.

Nullable Reference Types in C# 8 brings us pretty much the same. It allows us to mark the Reference Types which can/cannot be null, using the same syntax we used to mark Nullable Value Types – using the ? operator.

In C# 8, with Nullable Reference Types enable, if you were to compile the code, the compiler warns you with the following.

Warning	CS8618	Non-nullable property 'Bar' is uninitialized. Consider declaring the property as nullable.

Depending on your business logic, you have couple of ways to proceed from here.

Case 1 – Bar is Nullable

If your business logic requires that the Bar property could be null, then you could mark it explicitly. For example,

public Bar? Bar { get; set; }

The ? operator tells the compiler that the property could be null and it should warn the developer for possible null checks if he hasn’t done so yet. The compiler duely warns you with the following.

Warning	CS8602	Dereference of a possibly null reference.

Case 2 – Bar is non-nullable

If your application logic requires that the Bar property cannot be null, then you could let the compiler know by using the following syntax (basically omitting the ? operator).

public Bar Bar { get; set; }

The above code tells the compiler that the Bar cannot be null. Interestingly, we still get a warning now, let us first examine what it is before understanding why compiler throws the warning.

Warning	CS8618	Non-nullable property 'Bar' is uninitialized. Consider declaring the property as nullable.

We have declared the property as Non-nullable, yet we have left it uninitialized. We could of course remove this warning by initializing the property with a default value (either in the property declaration or in the constructor), but this gives us an oppurtunity to introduce another feature.

Null Forgiving Operator

You could have a situation where, as per the business logic, you know that a particular property can never be null at runtime. However, you are in no position to assign a default value. In such situations, you could use the Null-forgivingoperator to disable the warnings. Null-Forgiving is denoted by using the postfix operator !. For example,

public Bar Bar { get; set; } = null!;

The above code informs the compiler that it would skip the warning about Bar being uninitialized and it would not be null in runtime. Do note the null-forgiving operator has not runtime influence.

Null Coalescing Assignment

C# 8 also introduces the Null Coalescing assignment via the the assignment operator ??=. For demonstration, let us further develop our code and introduce a CreateBarInstance to assign instances of Bar. For example,

public void CreateBarInstance(Bar instance)
{
    Bar = instance;
}

Due to some business reasons, we have requirement that the method should assign Bar instance, only if it is null. Of course, one could do this an if condition, but that would introduce a lot of boiler plate code. We could eliminate these extraneous code using the null coalescing operator.

public void CreateBarInstance(Bar instance)
{
    Bar ??= instance;
}

The line Bar ??= instance ensures that Bar is assigned only when the it is null. Or in other words, the value of right hand operand is assigned to left hand operand only if the left hand operand is null.

Demonstrating the code in action,

var foo = new Foo();
foo.CreateBarInstance(new Bar(2));
Console.WriteLine(foo.Bar.Id);
foo.CreateBarInstance(new Bar(3));
Console.WriteLine(foo.Bar.Id);

Output of the above code

2
2

That’s all for now, we will continue our exploration of C# 8 features.

C# 8 : Asynchronous Streams and Static Local Functions

Asynchronous Stream
How often have you come across situations when you would want to return streams of data from an asynchronous method ? Plenty of times I would guess. The asynchronous methods were pretty limited when return data in streams, but not any longer. 

Consider the following code.

// Will not compile
public static async Task<IEnumerable<int>> Generate(int max)
{
    for (int i = 0; i < max; i++)
    {
        await Task.Delay(1000);
        yield return i;
    }
}

The above code would not compile. There is no way you could `yield return` values (or in other words, stream values) from an asynchronous method. This would have been a highly useful situation when you want to iterate over results from a query over an extremely large database. There are other countless situation the ability to stream data from asynchronous method could be useful. However, prior to C# 8.0, we were severely handicapped in such a situation.

With C# 8.0, .Net comprises a new Type called the IAsyncEnumerable, which allows us to accomplis this. The above code could be now rewritten as.

public static async IAsyncEnumerable<int> Generate(int max)
{
    for (int i = 0; i < max; i++)
    {
        await Task.Delay(1000);
        yield return i;
    }
}

//client code
await foreach (var item in Generate(10))
{
    Console.WriteLine(item);
};

The placement of `await` keyword in the consumer code needs to be noted here.

Local Static Functions
In previous versions of C#, local methods could capture the enclosing scope. For example, consider the following code.

public void Foo()
{
    int internalValue = 20;

    void Bar()
    {
        internalValue++;
    }

    Bar();
}

This enable usage of variables such as internalValue within the local method. If the usage is accidental, this might lead to huge consequences. One could not declare a static local method in the earlier versions of .Net.

With C# 8.0, .Net removed this limitation. This enables the developers to create *pure local functions* as it does not allow usage of variables from enclosing types within it.

public void Foo()
{
    int internalValue = 20;

    static void Bar()
    {
        Console.WriteLine("Do Something");
        // internalValue++; // This would throw compile error
    }

    Bar();

}

This also removes certain overhead created when using the captured variables. For example, the previous code (C# < 8.0), would be treated by the compiler as

public void Foo()
{
	DemoClass.<>c__DisplayClass0_0 CS$<>8__locals1;
	CS$<>8__locals1.internalValue = 20;
	DemoClass.<Foo>g__Bar|0_0(ref CS$<>8__locals1);
}
internal static void <Foo>g__Bar|0_0(ref DemoClass.<>c__DisplayClass0_0 A_0)
{
	Console.WriteLine("Do Something");
	int internalValue = A_0.internalValue;
	A_0.internalValue = internalValue + 1;
}

Notice how the compiler generates a struct to store the captured variables and pass it using `ref`. Such an overhead would be completely avoided when using the *static local functions*, simply because – the compiler won’t compile or capture the enclosing scope.

We explored two features introduced in C# 8.0. We will explore more in coming posts, possibly in bunches as we have plenty to catch up before exploring C# 9.0.

Asynchronous Code – Behind the Scenes – 004

During this series of deep dive into the asynchronous calls, we have so far looked into

  • [x] General Structure of generated code.
  • [x] Role of Stub/Worker method.
  • [x] Structure of State Machine and role of Fields.
  • [x] Implementation of the SetStateMachine method.
  • [ ] Implementation of the MoveNext method.

It is now time to look at the most important piece of the puzzle – the MoveNext() method.

Before we begin exploring the MoveNext(), let us remind ourself that the method is called when the async method is first invoked and then, each time it is resumed. The Method would be responsible for the following.

  • Ensure the method starts/resumes execution at the right place when it starts for the first time or resumes after a pause.
  • Preserve the state of State Machine when it needs to pause.
  • Schedule a continuation when the awaited expression hasn’t been completed yet.
  • Retrieve values from the awaiter.
  • Propagate the return values or method completion via the Builder.
  • Propagate the exceptions if any via the Builder.

The last 2 points are curious if you were to consider that the MoveNext method has a void return Type. So how does the MoveNextreturn the result or exceptions ? Of course via the Builder instance. It is the role of the Stub method to return the Task to the Caller method.

Without taking any time longer, let us take a peek at the generated code. We will then proceed to split it into parts and find how it works

The Whole Code

private void MoveNext()
{
    int num = <>1__state;
    try
    {
        TaskAwaiter awaiter;
        if (num != 0)
        {
            if (num == 1)
            {
                awaiter = <>u__1;
                <>u__1 = default(TaskAwaiter);
                num = (<>1__state = -1);
                goto IL_00cc;
            }
            awaiter = Task.Delay(delay).GetAwaiter();
            if (!awaiter.IsCompleted)
            {
                num = (<>1__state = 0);
                <>u__1 = awaiter;
                <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
                return;
            }
        }
        else
        {
            awaiter = <>u__1;
            <>u__1 = default(TaskAwaiter);
            num = (<>1__state = -1);
        }
        awaiter.GetResult();
        Console.WriteLine(delay);
        awaiter = Bar().GetAwaiter();
        if (!awaiter.IsCompleted)
        {
            num = (<>1__state = 1);
            <>u__1 = awaiter;
            <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
            return;
        }
        goto IL_00cc;
        IL_00cc:
        awaiter.GetResult();
    }
    catch (Exception exception)
    {
        <>1__state = -2;
        <>t__builder.SetException(exception);
        return;
    }
    <>1__state = -2;
    <>t__builder.SetResult();
}

That does look a bit scary to begin with. But, have no worries. We will break it down and understand it better.

Exception Handling

Now we already know that the associated Task object returned by the async method would contain any exception if any. It also sets the status to faulted. So how does the State Machine help in doing so ? That’s the first part we will explore. Let us have birds-eye view of the MoveNext() method – for time being, we will ignore all code within the try block.

private void MoveNext()
{
    int num = <>1__state;
    try
    {
        // Ignore this code for the moment
    }
    catch (Exception exception)
    {
        <>1__state = -2;
        <>t__builder.SetException(exception);
        return;
    }
    <>1__state = -2;
    <>t__builder.SetResult();
}

As you can observe the entire MoveNext() method has a big try catch wrapping the code within. The interesting part for the moment would be the catch block. If any exceptions occurs in the try block, the MoveNext() method sets the state to -2 to indicate the method has completed (-2 indicates completion, irrespective of success or failure). It then uses the Builder to set the exception using the Builder.SetException method.

Only special exceptions like the ThreadAbortException or the StackOverflowException can cause the MoveNext() method to end with an exception.

High Level Flow of State Machine

At a higher level, one can observe that the MoveNext() method returns if any of the following are true

  • Each time the state machine needs to be pause (for an await statement to complete).
  • Execution reaches the end of the method
  • Exception is thrown, but not caught in the async method.

A High level flow of the State Machine could be summarized as follows.

  1. The Stub Method (Worker Method) initiates the State Machine using the Builder Object (AsyncTaskMethodBuilder).
  2. Jump to the correct place in State Machine based on the State Field.
  3. Execute the State Machine until the code reaches await statement or end of the method (return statement).
  4. Fetch the awaiter.
    • If the awater is completed, go back to the Step 2.
    • If not, attach a continuation to the awaiter.
    • If this is the first awaiter, return the Task.
  5. The Task returned in Step 5, would be returned the caller via the Builder.

The Try Block

The Try blocks starts with a switch/if condition depending on the number of await statements within the method. If it has 3 or more awaits, usually one could notice a switch case, in all other cases, an if statement is used.

Irrespective of the approach, the condition to check resolves around the State of the State Machine. If the state is negative, it indicates the first call to the MoveNext() method. If the value of State is a positive number, then it indicates the State Machine is resuming from a pause.

public int <>1__state;
public AsyncTaskMethodBuilder <>t__builder;
public int delay;
private TaskAwaiter <>u__1;

private void MoveNext()
{
    int num = <>1__state;
    try
    {
        TaskAwaiter awaiter;
        if (num != 0)
        {
            if (num == 1)
            {
                awaiter = <>u__1;
                <>u__1 = default(TaskAwaiter);
                num = (<>1__state = -1);
                goto IL_00cc;
            }
            awaiter = Task.Delay(delay).GetAwaiter();
            if (!awaiter.IsCompleted)
            {
                num = (<>1__state = 0);
                <>u__1 = awaiter;
                <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
                return;
            }
        }
        else
        {
            awaiter = <>u__1;
            <>u__1 = default(TaskAwaiter);
            num = (<>1__state = -1);
        }
        awaiter.GetResult();
        Console.WriteLine(delay);
        awaiter = Bar().GetAwaiter();
        if (!awaiter.IsCompleted)
        {
            num = (<>1__state = 1);
            <>u__1 = awaiter;
            <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
            return;
        }
        goto IL_00cc;
        IL_00cc:
        awaiter.GetResult();
    }
    catch (Exception exception)
    {
        // Not displayed for clarity
    }
    <>1__state = -2;
    <>t__builder.SetResult();
}

One of the first things you notice in the code above is that State is stored in a local variable. I guess this is done for optimization purposes. We could use a dedicated post later for understanding different optimizations techniques used by compiler here, for now let us stick to the task in hand.

As one can observe, when the Method is invoked for the first time, as the state would be -1, the code would proceed and hit the first await statement.

awaiter = Task.Delay(delay).GetAwaiter();
if (!awaiter.IsCompleted)
{
    num = (<>1__state = 0);
    <>u__1 = awaiter;
    <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
    return;
}

It fetches the Awaiter using the GetAwaiter method. If the awaiter is already completed, it would proceed to the next step in the original method. If not, it would set the State to 0 (indicating the first instance where the MoveNext() method had to await – zero based index), store the awaiter in the field and schedules the state machine to proceed to the next action when the specified awaiter completes using the Builder.AwaitUnsafeOnCompleted method.

On resumption after the pause, it moves to else part (remember, the state is having a value 0 now). It restores the awaiter stored in the field and clears the fields so that GC could take care of it. It also sets the State to -1.

else
{
    awaiter = <>u__1;
    <>u__1 = default(TaskAwaiter);
    num = (<>1__state = -1);
}
awaiter.GetResult();
Console.WriteLine(delay);
awaiter = Bar().GetAwaiter();
if (!awaiter.IsCompleted)
{
    num = (<>1__state = 1);
    <>u__1 = awaiter;
    <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
    return;
}

It then proceeds to fetch the Result using the TaskAwaiter.GetResult() method and then executes the remaining steps untill it hits the next await or the method completes. On completion (either finished or faulted), it sets the State to -2 and sets the Result using the Builder.

<>1__state = -2;
<>t__builder.SetResult();

Over the last few posts, we have traced through the generated source code behind the asynchronous methods. We noticed how the method gets translated to a pair of Stub/Working method and a State Machine. We also explored the State Machine in detail and understood how the MoveNext method method navigates the original method while maitaining the states.

The whole process, starting from the moment your code hits the await expression could be summarized as,

  1. Get the awaiter from the awaitable expression using the GetAwaiter() method.
  2. Check if the awaiter has been comepleted
    • If Yes, Go to Step 8. (Fast Path)
    • If No, remember where you have reached using the State Field. (Slow Path)
  3. Store the awaiter in a field.
  4. Schedule a continuation with the awaiter, such that when the continuation is executed, you are back at the right place.
  5. Return from the MoveNext, either to the original caller if it is the first pause, or to whatever has scheduled the continuation.
  6. When the continuation fires, set the State to -1 to indicate running.
  7. Restore the Awaiter from the field and store it back in the Stack. Remember to reset the field so that GC could take care of it.
  8. Fetch the result using GetResult() method.
  9. Continue with rest of the code.

This, was a simple asynchornous method devoid of any controls methodologies like the loops. In the next part of this series, we will use the knowledge we have gained so far to understand more complex scenarios in depth.

Once again, I would like to thank the wonderful Jon Skeets for his brillant book – C# in Depth. You ought to rename it to “C# Bible” Jon !!

Asynchronous Code – Behind the Scenes – 003

Okay, I wasn’t quite realistic in the earlier post when I mentioned we would look at MoveNext in this one. I missed an important clog of the wheel. The SetStateMachine() method.

IAsyncStateMachine.SetStateMachine

We will only breifly visit the SetStateMachine method here, as the complete picture becomes more clear when we look to details of the MoveNext() method.

So how does the SetStateMachine method looks like in the generated code. Interestingly, it has two different implementation depending on whether you are in Release or Debug mode.

// Release Mode
[DebuggerHidden]
private void SetStateMachine(IAsyncStateMachine stateMachine)
{
    <>t__builder.SetStateMachine(stateMachine);
}

void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine stateMachine)
{
    this.SetStateMachine(stateMachine);
}


// Debug Mode
[DebuggerHidden]
private void SetStateMachine(IAsyncStateMachine stateMachine)
{
}

void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine stateMachine)
{
    this.SetStateMachine(stateMachine);
}

As one can observe, in the Debug Mode, the method is empty. Hence the following explanation is more relavant for the Release mode.

Let’s go back a bit and think about our little Stub method.

private static Task Bar()
{
	<Bar>d__2 stateMachine = new <Bar>d__2();
	stateMachine.<>t__builder = AsyncTaskMethodBuilder.Create();
	stateMachine.<>1__state = -1;
	stateMachine.<>t__builder.Start(ref stateMachine);
	return stateMachine.<>t__builder.Task;
}

When the State Machine is started by the Stub Method, it is residing on the Stack as a local variable of the Stub Method.

This is where the whole crazy stuff starts. When the State Machine pauses and resumes again, it needs a lot of information. For this to happen, when it pauses, the state machine has to box itself and store in heap, so that when it resumes, it has all the necessary informations. After it is boxed, the state machine is called on the box value using box value as arguement.

Do note that the boxing happens only once. The State machine also ensures that the builder has a reference to the single boxed version of the state machine.

This can be noticed if you dig a deep into the code of AsyncMethodBuilderCore.SetStateMachine

public void SetStateMachine(IAsyncStateMachine stateMachine)
{
	if (stateMachine == null)
	{
		throw new ArgumentNullException("stateMachine");
	}
	if (m_stateMachine != null)
	{
		throw new InvalidOperationException(Environment.GetResourceString("AsyncMethodBuilder_InstanceNotInitialized"));
	}
	m_stateMachine = stateMachine;
}

We will leave the SetStateMachine here because that’s all it does. Its role would be more visible once we examine the MoveNext method in detail.

Asynchronous Code – Behind the Scenes – 002

In the earlier part of this series, we reviewed the generic structure of decompiled async code, especially the stub method. In this part, we would continue our explore of async code and look into the State Machine. We would not delve deep into the most important MoveNext() method yet, we will first familiar with the different parts of the State Machine first.

State Machine

Let us go back to the ILSpy and see how State Machine looks like.

[StructLayout(LayoutKind.Auto)]
[CompilerGenerated]
private struct <Foo>d__1 : IAsyncStateMachine
{
	public int <>1__state;

	public AsyncTaskMethodBuilder <>t__builder;

	public int delay;

	private TaskAwaiter <>u__1;

	private void MoveNext()
	{
		// To be discussed later
	}

	void IAsyncStateMachine.MoveNext()
	{
		//ILSpy generated this explicit interface implementation from .override directive in MoveNext
		this.MoveNext();
	}

	[DebuggerHidden]
	private void SetStateMachine(IAsyncStateMachine stateMachine)
	{
		<>t__builder.SetStateMachine(stateMachine);
	}

	void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine stateMachine)
	{
		//ILSpy generated this explicit interface implementation from .override directive in SetStateMachine
		this.SetStateMachine(stateMachine);
	}
}

  • IAsyncStateMachine Interface

One of the first things we would notice is the implementation of IAsyncStateMachine interface. The IAsyncStateMachine interface, which is defined under the System.Runtime.CompilerServices namespace, represents the state machine generated for the async method. The interface itself is a simple one, with just two methods in it.

public interface IAsyncStateMachine
{
    /// <summary>Moves the state machine to its next state.</summary>
    void MoveNext();
    /// <summary>Configures the state machine with a heap-allocated replica.</summary>
    /// <param name="stateMachine">The heap-allocated replica.</param>
    void SetStateMachine(IAsyncStateMachine stateMachine);
}

The MoveNext() as explained earlier, represents the heart of asynchronous code. We would, for time being, delay visiting the method for a bit longer. However, the key point to remember at this point of time is that each time the State Machine is starts or resumes (after a pause), the MoveNext() method would be called. The SetStateMachine() method associates the builder with the specific state machine.

The importance of the implementation of the interface and how it binds the state machine with the stub method could be understood by looking at the signature of the AsyncTaskMethodBuilder.Start(). The method accepts a single generic parameter, which has a constraint of having implemented the IAsyncStateMachine.

public void Start<TStateMachine>(ref TStateMachine stateMachine) where TStateMachine : IAsyncStateMachine
{
	if (stateMachine == null)
	{
		throw new ArgumentNullException("stateMachine");
	}
	ExecutionContextSwitcher ecsw = default(ExecutionContextSwitcher);
	RuntimeHelpers.PrepareConstrainedRegions();
	try
	{
		ExecutionContext.EstablishCopyOnWriteScope(ref ecsw);
		stateMachine.MoveNext();
	}
	finally
	{
		ecsw.Undo();
	}
}

We would not go too deep into AsyncTaskMethodBuilder.Start(), but key take away would be

  • The constraint applied to parameter where TStateMachine : IAsyncStateMachine
  • The method is responsible for calling IAsyncStateMachine.MoveNext()

There is another interesting fact to this look at this point. The generated State Machine has a small but significant difference depending on whether your are looking at debug/release mode code. When in release mode, the compiler optimizes the code and creates a stuct based State Machine, while in debug mode, it creates a class. This is supposed to an optimization done to so that the compiler would skip allocating memory when the awaitable has already been completed awaited. The following code displays the State Machine when decompiled in debug mode.

[CompilerGenerated]
private sealed class <Foo>d__1 : IAsyncStateMachine
{
	public int <>1__state;

	public AsyncTaskMethodBuilder <>t__builder;

	public int delay;

	private TaskAwaiter <>u__1;

	private void MoveNext()
	{
		// To be discussed later
	}

	void IAsyncStateMachine.MoveNext()
	{
		//ILSpy generated this explicit interface implementation from .override directive in MoveNext
		this.MoveNext();
	}

	[DebuggerHidden]
	private void SetStateMachine(IAsyncStateMachine stateMachine)
	{
	}

	void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine stateMachine)
	{
		//ILSpy generated this explicit interface implementation from .override directive in SetStateMachine
		this.SetStateMachine(stateMachine);
	}
}


  • Fields

public int <>1__state;
public AsyncTaskMethodBuilder <>t__builder;
public int delay;
private TaskAwaiter <>u__1;

The next thing one would notice with the generated code is the presence of certains fields in the state machine. The fields could be broadly categorized into

  • Current State : As discussed in earlier post, this could have any of the following values
-1 : Not Started
-2 : Completed
Any other Value : Paused

  • Method Builder : Communicates with the async infrastructure and returns the task
  • TaskAwaiter
  • Parameters and local variables
  • Temporary Stack Variables

TaskAwaiter and parameters are used to remember the values when the State Machine resumes after a Pause. If the state machine requires a variable which it doesn’t need to remember after resuming, then it remains as private variable.

Temporary stack variables are used as a part of larger expression, when the compiler needs to remember intermediate results. For example,

int result = x + y + await task;

The most important point to remember about the Fields and Variables is that the compiler ensures it uses minimum fields/variables as possible by reusing them.

If your code have multiple await that was supposed to return

  • Task<int>
  • Task<string>
  • Task

Then the compiler would most likely create just 3 awaiters, one each for the different types involved.

That is it about the general structure of the State Machine. We would now proceed to the most important part, which is of course the MoveNext() method. We will do it in the next post.