The Hidden Cost of Heap Allocations and GC Pressure
A few years ago, our team rolled out a high-throughput payment processor designed for 25,000 requests per second. On our staging rigs, tests passed with flying colors. But once Black Friday traffic surged in production, our 99th-percentile response times suddenly spiked from 12ms to past 180ms. Thread dumps quickly told the real story: worker threads were routinely stalling for 10 to 45 milliseconds during stop-the-world garbage collection cycles.
The problem wasn’t a deadlocked database or a slow downstream HTTP call. It was death by a thousand paper cuts on the managed heap. Ordinary calls like data.Substring(0, 10) or new byte[1024] in hot loops churned through 1.2 GB of temporary objects every second. The .NET Garbage Collector had to trigger nonstop Generation 0 and Generation 1 sweeps to keep up, burning valuable CPU cycles just to throw garbage away.
Traditional C# APIs allocate brand-new string and array instances whenever you split text, inspect a payload segment, or copy a buffer. If you want to push throughput into tens of thousands of ops per second without blowing your memory budget, you need contiguous memory access without heap allocations. Span<T> and Memory<T> give you exactly that.
Understanding Span<T> vs Memory<T>
Span<T>: Fast Stack-Only Memory Slicing
Span<T> acts as a safe, type-enforced window into any contiguous block of memory. Under the hood, it is defined as a ref struct containing only two fields: an interior reference pointer and an integer length. Because it is a ref struct, the runtime strictly enforces that a Span<T> can only ever reside on the execution stack. It cannot be boxed, stored as a field in a standard class, or passed across thread boundaries.
Because the stack lifetime is predictable, the JIT compiler generates machine code comparable to bare-metal C pointer arithmetic, complete with zero allocation overhead and fast, vectorized bounds checks. You can point a single Span<T> at three completely different backing stores:
- Managed arrays on the heap (
byte[],int[]) - Short-lived stack buffers (
stackalloc byte[128]) - Unmanaged native memory buffers allocated via native interop
Memory<T>: Heap-Safe Slicing for Async Code
The stack-only constraint of Span<T> comes with a major catch: it cannot survive an asynchronous execution boundary. When an async method hits an await, the compiler lifts its local variables into an autogenerated heap state machine. Trying to hold a Span<T> across that suspension point triggers compiler error CS4013.
Enter Memory<T>. Unlike Span<T>, it is a standard readonly struct backed by an array, string, or pooled memory manager. That means it can comfortably sit on the heap, reside inside class fields, and survive across await points. When you are ready to parse or transform the underlying buffer synchronously, simply call the .Span property to get an allocation-free stack slice.
ReadOnlySpan<char> for String Operations
Strings in .NET are immutable heap allocations. Every time you call string.Substring(), the runtime allocates a brand-new string and copies characters into it. By switching to ReadOnlySpan<char> via string.AsSpan(), you slice substrings as virtual windows into existing memory with zero allocations and zero string copying.
Hands-on Practice: From Heavy Allocations to Zero-Allocation
Tip 1: Parse Delimited Text Without Allocating Substrings
Take a routine task like extracting fields from a structured log line formatted as TIMESTAMP|SEVERITY|MESSAGE. The naive implementation creates unnecessary waste:
// Allocates 1 array and 3 string objects per invocation
public (string Level, string Message) ParseLogLegacy(string rawLog)
{
string[] parts = rawLog.Split('|');
return (parts[1], parts[2]);
}When you pump 100,000 log events per second through that helper, you force the GC to clean up 400,000 heap objects every single second. Here is the zero-allocation equivalent using ReadOnlySpan<char>:
// Zero heap allocations, pure pointer slicing
public bool TryParseLog(ReadOnlySpan<char> rawLog, out ReadOnlySpan<char> level, out ReadOnlySpan<char> message)
{
level = default;
message = default;
int firstSeparator = rawLog.IndexOf('|');
if (firstSeparator == -1) return false;
ReadOnlySpan<char> remainder = rawLog.Slice(firstSeparator + 1);
int secondSeparator = remainder.IndexOf('|');
if (secondSeparator == -1) return false;
level = remainder.Slice(0, secondSeparator);
message = remainder.Slice(secondSeparator + 1);
return true;
}In our microbenchmarks, this refactor dropped execution time from 42ns down to 9ns while eliminating 128 bytes of heap allocation per line entirely.
Tip 2: Direct Number Parsing with Utf8Parser
Incoming HTTP headers, JSON payloads, and IoT telemetry streams arrive as raw UTF-8 byte streams. Many developers decode those bytes into a UTF-16 string first, only to feed that string into int.Parse(). Skip the middleman. With System.Buffers.Text.Utf8Parser, you can parse values directly from byte spans:
using System;
using System.Buffers.Text;
ReadOnlySpan<byte> payload = "1048576"u8;
if (Utf8Parser.TryParse(payload, out int parsedValue, out int bytesConsumed))
{
Console.WriteLine($"Parsed integer: {parsedValue} ({bytesConsumed} bytes processed)");
}By consuming the UTF-8 literal directly, you eliminate intermediate string decoding and save both CPU cycles and memory.
Tip 3: Use stackalloc for Small Intermediate Buffers
When calculating checksums, formatting compact framing headers, or generating HMAC tokens, allocating short-lived byte arrays on the heap is pure waste. Instead, borrow memory directly from your thread’s stack with stackalloc:
using System;
using System.Security.Cryptography;
public static void ComputePayloadToken(ReadOnlySpan<byte> data, Span<byte> destinationHash)
{
// 64-byte scratchpad allocated on the CPU stack frame
Span<byte> scratchBuffer = stackalloc byte[64];
int bytesToCopy = Math.Min(data.Length, scratchBuffer.Length);
data.Slice(0, bytesToCopy).CopyTo(scratchBuffer);
SHA256.HashData(scratchBuffer, destinationHash);
}Thread stacks are small—typically 1 MB on Linux and Windows 64-bit systems. Never use stackalloc for dynamic, unbounded user input. Keep allocations under 1 KB to prevent fatal StackOverflowException crashes.
Tip 4: Handle Async Pipelines with Memory<T> and ArrayPool
Sockets and network streams require asynchronous reads, which means stack-only spans are off the table. Combine Memory<T> with ArrayPool<T> to keep your async pipeline allocation-free:
using System.Buffers;
using System.IO;
using System.Threading.Tasks;
public async Task ProcessStreamAsync(Stream networkStream)
{
// Rent a 4KB chunk rather than creating a new array
byte[] rentedArray = ArrayPool<byte>.Shared.Rent(4096);
Memory<byte> memoryBuffer = rentedArray.AsMemory(0, 4096);
try
{
int bytesRead = await networkStream.ReadAsync(memoryBuffer);
// Switch to Span for synchronous parsing
ReadOnlySpan<byte> packetSlice = memoryBuffer.Slice(0, bytesRead).Span;
ProcessPacket(packetSlice);
}
finally
{
// Return array to pool so other callers can reuse it
ArrayPool<byte>.Shared.Return(rentedArray);
}
}
private static void ProcessPacket(ReadOnlySpan<byte> packet)
{
// Parsing logic runs without touching the heap
}When we converted our ingestion pipelines to this pattern, Gen 0 collections plunged from 350 per minute to almost zero, cutting overall CPU load by 32%.
When to Use Each Type
You don’t need to rewrite your entire codebase with spans. Standard business code, admin dashboards, and batch reporting scripts rarely create enough pressure to notice. Focus your attention on hot paths: telemetry ingestors, custom serializers, middleware, and high-volume message consumers.
Here is your cheat sheet:
Span<T>/ReadOnlySpan<T>: Synchronous parsing, CPU-bound string manipulation, and in-place transformations over buffers that never cross anawait.Memory<T>/ReadOnlyMemory<T>: Buffer slicing acrossasyncI/O operations (likeStream.ReadAsync) and data models that must store sliced segments inside heap classes.ArrayPool<T>.Shared: Heavy scratch buffers (1 KB to 64 KB+) that get rented, populated, and quickly returned across repeated requests.
By bringing Span<T> and Memory<T> into your performance-critical paths, you get the zero-copy efficiency of raw pointers without sacrificing the type safety and memory protection that make C# great.
