Testing Caching Strategies in .NET Core

In this article, we’re going to prepare a simple test in .NET and use k6 for load testing to evaluate different caching strategies, including In-Memory Cache, Hybrid Cache, and Fusion Cache. We’ll examine their performance and see how well they address common issues like cache stampede, system overload, and data consistency.

What is Cache Stampede?
Cache stampede occurs when a cached item expires, and multiple requests hit the system at once, all trying to recreate the same data. This can put too much pressure on your database or external services, especially in high-traffic applications, and lead to slower performance or system issues.

The Three Contenders

We’ll compare three different caching implementations:

• Memory Cache: The built-in ASP.NET Core caching solution
• Hybrid Cache: Microsoft’s new preview library combining in-memory and distributed caching
• Fusion Cache: A specialized library with built-in stampede protection

Implementation Details

Let’s look at how each caching strategy is implemented:

You can find the code in this repository:
https://github.com/dev3mike/dotnet-cache-strategies

1. Memory Cache Implementation

var cachedValue = await memoryCache.GetOrCreateAsync("memory-cache", async (entry) =>
{
    await Task.Delay(3000); // Simulated database delay
    return "Data from the database";
}, new MemoryCacheEntryOptions() { SlidingExpiration = TimeSpan.FromSeconds(1) });

2. Hybrid Cache Implementation

var cachedValue = await hybridCache.GetOrCreateAsync("hybrid-cache", async (entry) =>
{
    await Task.Delay(3000); // Simulated database delay
    return "Data from the database";
}, new HybridCacheEntryOptions
{
    LocalCacheExpiration = TimeSpan.FromSeconds(1),
});

3. Fusion Cache Implementation

var cachedValue = await fusionCache.GetOrSetAsync("fusion-cache", async (entry) =>
{
    await Task.Delay(3000); // Simulated database delay
    return "Data from the database";
}, TimeSpan.FromSeconds(1));

Load Testing Setup

To compare these implementations, we created a comprehensive load testing scenario using k6:

• Test Duration: 6 seconds total
• Load Pattern:
• 0–2s: Ramp up to 100 users
• 2–4s: Ramp up to 1000 users
• 4–6s: Ramp down to 0 users
• Performance Thresholds: 95% of requests should complete under 2 seconds
• Failure rate should be less than 10%

Performance Results

Memory Cache Performance

• ✅ 100% success rate (1,272 requests)
• ⚠️ Average request duration: 1.45s
• ⚠️ Maximum request duration: 3.16s
• ❌ Failed p95 threshold requirement (p(95)<2000)
• 📊 Total iterations: 1,272

Hybrid Cache Performance

• ✅ 100% success rate (1,763 requests)
• ✅ Average request duration: 1.52s
• ⚠️ Maximum request duration: 3.01s
• ❌ Failed p95 threshold requirement (p(95)<2000)
• 📊 Total iterations: 1,763

Fusion Cache Performance

• ✅ 100% success rate (1,760 requests)
• ⚠️ Average request duration: 1.59s
• ⚠️ Maximum request duration: 3.14s
• ❌ Failed p95 threshold requirement (p(95)<2000)
• 📊 Total iterations: 1,760

Key Findings and Recommendations

• Hybrid Cache: The Winner
• Best overall performance with 1.52s average response time
• Handled the highest number of requests (1,763)
• Successfully managed high concurrency
• Fusion Cache: The Specialist
• Built-in stampede protection
• Similar performance to Hybrid Cache
• Good for most of the scenarios especially when stampede protection is critical
• More complex implementation than Memory Cache
• Memory Cache: The Basic Solution
• Simple implementation
• Moderate performance
• Suitable for low-traffic applications
• No built-in stampede protection

When to Use Each Strategy

1. Choose Hybrid Cache when:

• You need the best overall performance
• Your application handles high traffic
• You want both in-memory and distributed caching capabilities
• Performance is your top priority

2. Choose Fusion Cache when:

• You need the best overall performance (similar to hybrid approach)
• Your application handles high traffic
• Cache stampede protection is critical
• You need fine-grained control over caching behavior
• You can tolerate slightly higher latency for better protection
• You need built-in failure handling

3. Choose Memory Cache when:

• You have a simple application with low traffic
• You need a straightforward implementation
• You don’t need distributed caching
• Resource usage is a concern

Performance testing shows that Microsoft’s Hybrid Cache and Fusion Cache deliver nearly identical results, both achieving a 100% success rate and comparable average request durations (1.52s for Hybrid Cache and 1.59s for Fusion Cache).

However, since Hybrid Cache is still in its preview phase and not yet production-ready, Fusion Cache stands out as the best available solution for production environments.

But in general when implementing caching in your .NET applications, consider your specific needs around performance, scalability, and protection against cache stampede to choose the most appropriate solution.

Remember that these results are based on specific test scenarios, and your actual performance may vary depending on your use case, infrastructure, and load patterns. Always perform thorough testing in your specific environment before making a final decision.