Mastering Singleflight in Go: Stop Cache Stampedes and Protect Your Upstream Services

Quick start (5 min)

It is 2:00 AM on Black Friday. Your product service is cruising along at 15,000 requests per second, serving a discounted flagship monitor. Redis holds the inventory record with a 5-minute TTL. The moment that key expires, 2,500 requests hit your API in the same 50-millisecond window. Every single one misses Redis and falls straight through to PostgreSQL with an identical query: SELECT * FROM products WHERE id = 9812.

Postgres CPU pegs at 100%. Latency skyrockets from 3ms to 8,500ms. Within four seconds, all 128 connections in the pool are exhausted, triggering cascading 504 Gateway Timeouts across your cluster. This is the thundering herd problem in the wild.

Go offers a clean, direct fix in its official extended library: golang.org/x/sync/singleflight. The package deduplicates in-flight calls. When 500 requests ask for the same resource simultaneously, only one query runs. The other 499 callers simply wait for that result and share it.

Add the package to your module:

go get golang.org/x/sync/singleflight

Here is a complete, runnable reproduction. Five goroutines query the exact same user record concurrently, yet the database fetch executes only once:

package main

import (
	"fmt"
	"sync"
	"sync/atomic"
	"time"

	"golang.org/x/sync/singleflight"
)

var dbQueryCount int32

func fetchUserDataFromDB(userID string) (string, error) {
	atomic.AddInt32(&dbQueryCount, 1)
	time.Sleep(150 * time.Millisecond) // Simulate DB I/O latency
	return fmt.Sprintf("User profile payload for %s", userID), nil
}

func main() {
	var group singleflight.Group
	var wg sync.WaitGroup
	userID := "user-4021"

	for i := 1; i <= 5; i++ {
		wg.Add(1)
		go func(workerID int) {
			defer wg.Done()
			v, err, shared := group.Do(userID, func() (interface{}, error) {
				return fetchUserDataFromDB(userID)
			})

			if err != nil {
				fmt.Printf("Worker %d failed: %v\n", workerID, err)
				return
			}

			fmt.Printf("Worker %d received data: %s (shared = %t)\n", workerID, v.(string), shared)
		}(i)
	}

	wg.Wait()
	fmt.Printf("\nTotal queries executed on DB: %d\n", atomic.LoadInt32(&dbQueryCount))
}

Run the code. Despite five competing callers, fetchUserDataFromDB runs once. The first caller does the heavy lifting, while the remaining four get shared = true with the exact same payload.

How it works under the hood

The entire package relies on a compact state machine. Inside singleflight.Group, you find a standard mutex and an unexported hash map:

type Group struct {
	mu sync.Mutex       // protects m
	m  map[string]*call // lazily initialized
}

Every active query is tracked by a call struct holding synchronization primitives, duplicate counters, and the return payload:

type call struct {
	wg   sync.WaitGroup
	val  interface{}
	err  error
	dups int
	chans []chan<- Result
}

Here is what happens step by step when goroutine A calls group.Do("item-9812", fn):

1. Singleflight locks its mutex and checks whether m["item-9812"] already exists.
2. Finding nothing, it creates a new *call, adds 1 to call.wg, saves the pointer into the map, and releases the mutex.
3. Goroutine A executes your retrieval callback fn().

If goroutines B and C arrive while goroutine A is still awaiting database I/O:

1. Both routines acquire the mutex and find m["item-9812"] already in the map.
2. They increment call.dups, unlock the mutex, and block directly on call.wg.Wait().

When goroutine A finishes, it writes the result and error to the call struct, calls call.wg.Done(), re-locks the mutex to delete the key from the map, and unlocks. Goroutines B and C wake up and pull the shared result. One database query served three consumers.

Production patterns

Real-world production services require more than plain synchronous blocking. You need timeout handling, targeted cache invalidation, and defensive memory safety.

Handling context timeouts with DoChan

The standard group.Do() blocks until the inner function finishes. If an upstream client disconnects or sets a 200ms HTTP timeout, a slow 2-second database query will trap every waiting goroutine. Use group.DoChan() to listen on both the result channel and your request context:

package cache

import (
	"context"
	"errors"
	"fmt"

	"golang.org/x/sync/singleflight"
)

type Service struct {
	sf singleflight.Group
}

func (s *Service) GetArticle(ctx context.Context, id string) (string, error) {
	resultChan := s.sf.DoChan(id, func() (interface{}, error) {
		return fetchFromOrigin(id)
	})

	select {
	case <-ctx.Done():
		// Return early if the HTTP client abandons the request
		return "", ctx.Err()
	case res := <-resultChan:
		if res.Err != nil {
			return "", res.Err
		}
		val, ok := res.Val.(string)
		if !ok {
			return "", errors.New("unexpected type from singleflight result")
		}
		return val, nil
	}
}

func fetchFromOrigin(id string) (string, error) {
	return fmt.Sprintf("Data for item %s", id), nil
}

Keep in mind that if the first caller’s context expires, the background callback continues executing. Singleflight does not cancel the shared task automatically, ensuring other waiting callers still receive their data.

Forgetting slow or failed keys with group.Forget

Normally, singleflight removes a key only after the callback returns. If a third-party payment gateway hangs for 30 seconds, every incoming request during that window joins the stalled queue. Calling s.sf.Forget(id) tells the group to immediately delete the key from its internal map.

The original in-flight worker continues in the background, but the very next caller starts a fresh query instead of queueing behind the hung operation. This is especially useful for failing endpoints where fast retries matter.

Avoid returning mutable pointers

Concurrency bugs often creep in through shared pointers. Because every waiting goroutine receives the exact same interface{}, returning a pointer like *UserProfile lets multiple threads read and mutate the exact same memory address at once. Data races follow.

Always return value types, deep clones, or raw serialized bytes (such as []byte). Let each caller deserialize its own local copy safely.

Hard-won operational rules

Having run Singleflight across multiple high-throughput production clusters, we rely on four concrete practices to keep systems stable:

• Singleflight gates the DB; it does not replace your cache: Always inspect Redis or an in-memory LRU cache first. If you miss, pass through singleflight.Group to fetch from PostgreSQL, write the fresh record back to Redis, and return.
• Namespace every single key: Singleflight uses plain string keys. If user queries and invoice queries both look up ID "4501", they will block each other and return corrupt payloads. Use strict prefixes like fmt.Sprintf("user:v1:%s", userID).
• Scope groups per domain: Do not declare one global singleflight.Group for your entire microservice. Scope instances inside their respective repository structs. This isolates map locks and eliminates mutex contention between unrelated domains.
• Always pass an isolated context timeout to the worker: If your origin query lacks a deadline, a deadlocked database connection locks up all subscribers. Always set an explicit query timeout (for instance, context.WithTimeout(context.Background(), 3*time.Second)) inside your retrieval function.

Adding Singleflight in front of read-heavy SQL tables and rate-limited third-party APIs reliably flattens sudden traffic spikes. Instead of your database drowning in duplicated work, it processes each request exactly once.