7+ Million HTTP requests per second from a single server
It’s coming up to 2 years since I last posted about the performance of ASP.NET Core;
during its preview, pre version 1.0. As preview 2 of has ASP.NET Core 3.0
has just been to released; it’s time to follow up, and find out how its evolved.
ASP.NET Core 2.2 (Current)
Looking at the latest run from the TechEmpower Benchmarks continuous results
ASP.NET 2.2 is the 3rd fastest webserver (0.046% off the top spot); able to respond to 7 Million HTTP request per second:
I recently saw another post; about a different product, where throughput that was measured monthly;
so if we use that time period, that’s 18.1 trillion HTTP requests per month.
It’s also an extraordinary amount of bandwidth; enough to continuously saturate a 10GBps link.
These results are with the webserver and load tester running inside Docker containers,
on two different physical Linux machines *; connected with a 10GbE network.
All this throughput from a single server! ASP.NET Core is fast on Linux (and on Windows).
How does it compare to other well known servers?
In these “platform” comparisons that’s:
Internet Facing Server
ASP.NET Core’s Kestrel Webserver;
which is used in these benchmarks,
is an edge server so can be used as an
internet facing webserver, as explained in the documentation:
It doesn’t need a second webserver acting as a reverse-proxy server in front and can go full speed.
It also works with reverse-proxies if that fits more with your infrastructure.
Data Access Performance
I often hear the defeatist argument that performance like this doesn’t matter because “my database is slow”; well times are a changing…
Test type 5: Database updates
Exercises the ORM’s persistence of objects and the database driver’s performance at running UPDATE statements or similar.
The spirit of this test is to exercise a variable number of read-then-write style database operations.
HTTP api request => 20 database queries => 20 database updates
ASP.NET Core occupies the 8th, 9th and 10th positions out of 331 entries,
for raw, middleware and mvc (which are “platform”, “micro-framework”, and “full-framework” respectively)
The “platform” level runs 550,120 SQL statements per second (13,753 * (20 SELECT + 20 UPDATE))
Looking that over longer time periods that’s:
Test type 4: Fortunes
This is the most full featured test, most closely mimicking a wider range of activities a web application executes in combination to produce a web page:
Exercises the ORM, database connectivity, dynamic-size collections, sorting, server-side templates, XSS countermeasures, and character encoding.
ASP.NET Core performs well here also with 298,477 requests served per second; coming 7th out of 350 entries.
.NET runs on the Common Language Runtime, which is language neutral, so its also not all about C#; with VB.NET in 14th position.
.NET can work with many different databases; so looking at MySQL
Test type 3: Multiple database queries
Multiple rows are fetched to more dramatically punish the database driver and connection pool.
At the highest queries-per-request tested (20), this test demonstrates all frameworks’ convergence
toward zero requests-per-second as database activity increases.
ASP.NET comes 6th out of 178 MySql entries; performing 419,460 queries per second (20,973 * 20)
Why is Performance Important?
At Illyriad Games we are building a new scale of gaming for Age of Ascent,
an Ultra-MMO with real-time twitch combat at unprecedented scale.
High performance at a business level means we can do more with less – which directly affects our bottom line.
So the choice and performance of a framework is very important to us.
In the words of TechEmpower as to the motivations for setting up their benchmarks:
Application performance can be directly mapped to hosting dollars, and for companies both large and small, hosting costs can be a pain point.
Weak performance can also cause premature and costly scale pain by requiring earlier optimization efforts and increased architectural complexity.
Finally, slow applications yield poor user experience and may suffer penalties levied by search engines.
What if building an application on one framework meant that at the very best your hardware is suitable for one tenth as much load as
it would be had you chosen a different framework?
This has become all the more important with cloud-based hosting; where rather than paying a fixed cost for a server,
you pay for only what you need and the usage of that at per minute granularity.
What are .NET Core and ASP.NET Core?
They are fully open-source frameworks under the .NET Foundation; commonly used with
C#, VB.NET
and F# languages which are all also open source and designed on GitHub.
They run on the open-source “Common Language Runtime” which has the aim:
The goal of the CLR is to make programming easy – from the Book of the Runtime
.NET Core and ASP.NET Core were born in the open-sourcing of .NET and the reimagining of ASP.NET;
which also took them from Windows only to cross-platform;
additionally supporting macOS and flavors of Linux, BSD and adding ARM to the supported chipsets.
The .NET Community has embraced the open-sourcing of .NET;
and its progress has never been so vibrant. A .NET renaissance has begun!
(Graph below from: Matt Warren’s: Open Source .NET – 4 years later)
While grounded in .NET’s original aim of being highly productive to empower developers to achieve more;
along the journey they gained a laser like focus on performance so your servers too can achieve more!
Ready for Production?
Yes, we use .NET Core and ASP.NET Core, Bing.com uses them
and ASP.NET Core’s webserver Kestrel is already the 12th most used Webserver on the Internet
even though it was only first released less than 2.5 years ago.
Jump in, the water’s warm!
ASP.NET Core 3.0 (Preview 2)
Every version of .NET Core has been making huge improvements in performance both to the framework’s core libraries (examples in 2.0
and in 2.1) and introducing new types and concepts to allow higher performance:
Continuing in these improvements, there are many changes in .NET Core 3.0 Preview 2 but
perhaps the biggest to us is the .NET Hardware Intrinsics API
Platform Dependent Intrinsics are coming. Thanks for everybody’s work on this project. https://t.co/oK84LFJeU8
— Fei Peng (@fiigii)
Which allows the Intel SSE, SSE2, AVX2 and Arm/Arm64 SIMD hardware instrinsics to be used directly from managed code
(e.g.
dotnet/coreclr#22127,
dotnet/coreclr#2107, dotnet/coreclr#22187, dotnet/coreclr#22118);
and many aspects of the runtime are moving from the C++ portion of the runtime to C# where the JIT can easily
target exactly the CPU it’s running on and achieve higher performance (e.g. dotnet/coreclr#21729).
Reduced Allocations
One of the big focuses of the performance work in the frameworks has been to reduce allocations; as the less allocations, the less work the Garbage Collector has to do.
The allocations and performance of the .NET’s async statemachine has been dramatically improved:
It’s impressive how much the allocations for the ‘async/await’ state machine were reduced in .NET Core 2.2 (kudos to @STeplyakov for the great tool https://t.co/cACIXg5u6A) pic.twitter.com/mJzoFNBO03
— Matt Warren (@matthewwarren)
As have the allocations in ASP.NET Core:
Snapshot of 1 million requests doing a “Hello World” chunked response on 2.2 vs 3.0 https://t.co/4PmXMQN7SX Core #dotnetcore #aspnetcore #performance pic.twitter.com/2rkPw3MlaO
— David Fowler #BlackLivesMatter (@davidfowl)
Zero Allocations
In fact combined, with the merging of these two Pull Requests (PRs)
aspnet/AspNetCore#4601 and dotnet/coreclr#21159
the steady state allocations for Plaintext on at the platform level has been entirely eliminated and is now zero allocation.
What does Zero allocations look like?
Running a 6 minute, 64 connection test on localhost:
Results in 122,946,688 HTTP requests and responses; and the allocations?
Less than 1MB allocated in total for processing more than 122 Million requests and generating their responses!
Continuous Performance Monitoring
I mentioned earlier “a laser like focus on performance”; but what does that really mean?
All the ASP.NET Core repositories have micro-benchmarks using the great tool BenchmarkDotNet
to verify in isolation changes and how they will affect performance.
However, more importantly there is an entire suite of full system benchmarks of over 100 scenarios,
which are then run on both Linux and Windows, physical servers and cloud for a total of 400+ full system performance tests,
running continuously as part of their Continuous Integration (CI) several times a day to catch any regressions early.
Public Performance Dashboard
As its fully open source and open to contributors; the full 10 pages of all performance KPIs and time series graphs
is completely available and public: aspnet/Benchmarks Results
Looking at the “KPIs – Baselines” page:
We already saw how ASP.NET Core was fast; but looking at the dashboard it shows
ASP.NET Core 3.0 is already 30% faster than ASP.NET Core 2.2 for some scenarios!
You can also see that 3.0 preview is tracked and compared against all prior versions
as are all the current patch and servicing releases; so there are no surprise impacts
from any patching.
It’s not only public; but the dashboard is on continuous display in the Microsoft offices:
Finally got our replacement TV so the #aspnetcore SmurfLab dashboard is back up again! pic.twitter.com/64VbCLoXXr
— Damian Edwards 😷🌈#BlackLivesMatter (@DamianEdwards)
More Than Raw Throughput
As well as throughput; latency; time from cold start to first response, and memory usage is
closely tracked over time; for every scenario.
Memory usage for ASP.NET Core 3.0 is greatly reduced:
Cold start time to first response have improved in .NET 3.0 with features like ReadyToRun (AoT)
and Tiered Compilation;
which when combined give both improved start-up and improved steady-state performance:
Good for serverless scenarios like Azure functions and AWS Lambda; as well as the first spin up of a new container, VM or server when scaling out; or deployment update.
Runtime Performance
The performance is also tracked below ASP.NET Core at lower levels of in the stack.
The .NET Core runtime is continuously monitored also; on Windows, Linux, macOS, ARM;
and the repository dotnet/performance contains benchmarks used for testing the performance of .NET Frameworks.
Regressions are quickly spotted and reported on so they can be addressed:
ASP.NET Core: Very Fast and Getting Even Faster
.NET and ASP.NET have always be highly productive; and the future with
ASP.NET Core and .NET Core is as productive, but also fast; very fast!
Keep up with latest news & developments, we won’t spam you or pass your email on!
This content was originally published here.
