Benchmark your code with Benchmark.Net #

Hey folks,

Have you ever encountered a situation where you had a task at hand and you had two different implementations in mind? Were you unsure which one would be faster?

Have you ever hesitated when updating a colleague's algorithm and asked yourself that question about memory usage and execution speed?

You are in the right place!!

via GIPHY

In this article, we will cover how to benchmark our code and get all those metrics to analyze and optimize our code and be the smartest guy in the room.

Let’s be sharp and create our first benchmark

What is a benchmark? #

A benchmark is an analysis of a program presenting a set of measurements translating the execution time and how much resources the code consumed during the execution.

In this article, we will cover one of the most used packages in this category

Benchmark.NET #

Benchmark.NET is a cool .NET library that provides a set of metrics on our code such as memory usage, execution time, garbage collector usage…

The only thing you need to do is to install the NuGet package and create a benchmark class.

Benchmark flow #

As we all know, there is always a startup cost when starting a C# application.

If we run our benchmark at T0 of the execution, this startup time may affect our benchmark, and we will not have accurate measurements. To avoid this, Benchmark.NET added an extra step to warm up the application and prepare for a clean benchmark.

Create a new project #

Let’s create a simple .NET 7 console application to test our code. If you don't have .NET 7 installed on your IDE, you can use another version.

I’m using Rider as my main IDE, the same applies to Visual studio.

Benchmark example #

Let’s imagine that we are building some kind of counter and we need to display a value of a counter in the console. Our final result will be:

Counter : 1
Counter : 2

Counter : 3

….

Counter : 10,000

For this benchmark, we will try to compare two different ways of building a string:

String interpolation vs string builder.

Create a benchmark #

using System.Text;
using BenchmarkDotNet.Attributes;

namespace StringsBenchmark;

public class StringBenchmark
{
    private const int Counter = 10000;

    [Benchmark]
    public string StringInterpolation()
    {
        string result = "";
        for (int i = 0; i < Counter; i++)
        {
            result = $"{result} Counter: {i}";
        }
        return result;
    }

    [Benchmark]
    public string StringBuilderBenchmark()
    {
        StringBuilder sb = new StringBuilder();
        for (int i = 0; i < Counter; i++)
        {
            sb.Append(" Counter: ");
            sb.Append(i);
        }
        return sb.ToString();
    }
}

To run this benchmark, you need to call the BenchmarkRunner in your Program.cs file

public class Program
{
    static void Main()
    {
        BenchmarkRunner.Run<StringsBenchmark>();
    }
}

One last thing, the solution needs to run in Release mode and without debugging to have accurate results. If you try to run your benchmark in Debug mode, you will have an error message.

Benchmark results #

When you hit the Green arrow or Ctrl + F5 to run your application, you will have 3 main steps shown in your console.

Warm-up :

Execution:

Summary:

What we get from this benchmark is the execution time for our algorithm.

If we try to read this table:

String interpolation code takes 316,455 Microsecond to run (Mean column) in the other hand String builder is much faster with 390 Microsecond.

Mind-blowing, right?

via GIPHY

With this dummy example, we are talking about execution time measured with microseconds (0.000001 sec). It may seem not important right now, but imagine you have a huge workflow with in your application that parses and builds strings for LED screens or data analysis, for sure you will have seconds instead of microseconds.

Especially if your application is hosted on the cloud, Time + memory usage == Money.

Let’s see how we can get more info about memory usage.

Memory usage #

It’s pretty simple, Benchmark Dotnet is loaded with attributes that you can put into your class or methods to shape your benchmark and have the wanted result at the end.

Add MemoryDiagnoser attribute

Let’s check the result

Now we have 4 extra columns in our result:

Allocated: Allocated memory per single operation

Gen0, Gen1, Gen2: Those columns represent how many times the garbage collection tried to clean up unused objects in the memory. To have a clear vision of what is going on here, you need to understand how GC (garbage collector) works behind the scenes.

Gen0 is the first execution of the GC. In this execution, the garbage collector will try to free all unused memory allocation. If it detects that we still have some objects that are in use or that potentially they can be deallocated later, it will mark them as Gen1.

Basically, what the GC is saying is, “I’ve done my job; I will pass those informations to the next execution. It will make the cleanup in the next execution.”

If you want more details about GC check this doc

In the summary table, we have how many times the GC tried to collect per 1000 operations.

Generally, when numbers are high, it’s not a good sign 😄 Let's explain why we have those numbers and why string interpolation is costing us a lot of memory.

Analyze results #

Let's review our code and check what is going on. String interpolation is taking more time and consuming more memory due to the immutability of the string class. Let's explain more!

An immutable object is an object that its value can’t be changed after its creation, so if we want to update its value, we need to create a new object with the new value, and this is what we are doing in our code.

    public string StringInterpolation()
    {
        string result = "";
        for (int i = 0; i < Counter; i++)
        {
            result = $"{result} Counter: {i}";
        }
        return result;
    }

Each time we update our result object, we create a new object behind the scenes, and we store it in the memory.

In conclusion, benchmarking is an essential tool for any developer who wants to optimize their code and improve its performance. By using a package like Benchmark.NET, developers can easily measure their code's execution time, memory usage, and other important metrics.

In this article, we explored how to create a benchmark using Benchmark.NET, how to warm up the application, and how to analyze the results. We also looked at a simple example that compared string interpolation and string builder methods, and we concluded that string builder is a better option for handling a lot of string manipulation. With benchmarking, developers can make informed decisions about their code and create faster, more efficient applications.