The Double-Spend Problem in Multi-Node Setups
Picture a midnight flash sale with 10,000 active shoppers. Two customers click “Buy Now” at the exact same millisecond for the very last PlayStation 5 in stock. Upstream, an NGINX load balancer routes Customer A to Node 1 and Customer B to Node 2.
Both nodes query the database: stock level is 1. Both proceed to deduct inventory and charge payment cards. Within milliseconds, you sell two consoles when you only had one in the warehouse. A standard Java synchronized block or Node.js mutex only guards threads sharing memory on a single machine. The moment your app scales across multiple instances, in-process synchronization offers zero protection.
Databases like MySQL or PostgreSQL handle transactions well, but using them for high-frequency application coordination has a steep cost. When dozens of nodes compete for the same row locks during high traffic spikes, connection pools exhaust within seconds and database latency shoots through the roof.
Evaluating Your Options for Distributed Concurrency
Once you run more than one application node, three common strategies emerge to serialize operations.
1. Database-Level Locking (Pessimistic vs. Optimistic)
Relational databases let you lock rows explicitly using SELECT ... FOR UPDATE or track versions with optimistic locking numbers.
-- Pessimistic row locking
BEGIN;
SELECT stock FROM products WHERE id = 42 FOR UPDATE;
-- Validate stock and decrement
UPDATE products SET stock = stock - 1 WHERE id = 42;
COMMIT;2. Custom Redis Locks with SETNX
Redis operates on a single-threaded event loop, completing in-memory operations in sub-millisecond times. Early lock patterns relied on the atomic SETNX command:
# Acquire lock atomically with a 10-second TTL
SET lock:product:42 "client-uuid-123" NX PX 100003. Production Distributed Locking via Redisson
Redisson bridges raw Redis commands into familiar Java concurrency primitives like RLock. Under the hood, it executes battle-tested Lua scripts and handles automatic lease renewal through a watchdog mechanism.
RLock lock = redissonClient.getLock("lock:product:42");
lock.lock();
try {
// Serialized across all nodes in the cluster
inventoryService.deductStock(42);
} finally {
lock.unlock();
}Trade-offs: Choosing the Right Tool
Database Row Locking
- Pros: Requires no extra infrastructure; delivers strict ACID guarantees out of the box.
- Cons: Locks tie up scarce database connections. Under peak loads of 2,000+ requests per second, row contention causes transaction queuing and cascading connection pool timeouts.
Custom Redis SETNX
- Pros: Extremely lightweight; delivers sub-millisecond lock acquisition.
- Cons: Deceptively tricky to write safely. If a garbage collection pause or slow third-party API delays execution past the TTL, Redis drops the lock early and another worker grabs it. Safely releasing the lock requires checking ownership inside a custom Lua script to avoid deleting another process’s token.
Redisson Distributed Lock
- Pros: Fully reentrant; supports non-blocking retry backoffs; includes an automatic lease watchdog that prevents locks from expiring during unexpected GC pauses or network blips.
- Cons: Adds another client library; standard Redis master-replica architectures can still suffer from split-brain lock loss if the primary fails before replicating writes to replicas.
Recommended Architecture: The Fast-Path Pattern
In high-throughput microservices, keeping coordination workloads away from your core database yields the lowest latencies. A proven pattern separates duties cleanly:
- Redis Cluster / Sentinel: Serves as the central coordination layer, holding short-lived synchronization tokens in RAM.
- Redisson: Manages thread acquisition, retries, and automatic lease extensions.
- Relational DB (PostgreSQL / MySQL): Executes business transactions safely once Redisson admits exactly one node at a time.
Implementation: Setting Up Redisson in Spring Boot
Step 1: Start Redis
Start a local Redis container using Docker:
docker run -d --name redis-lock -p 6379:6379 redis:7-alpineStep 2: Add Maven Dependency
Add the Redisson starter to your pom.xml:
<dependency>
<groupId>org.redisson</groupId>
<artifactId>redisson-spring-boot-starter</artifactId>
<version>3.34.0</version>
</dependency>Step 3: Configure the Redisson Client Bean
Define a configuration class to tune connection pool thresholds:
import org.redisson.Redisson;
import org.redisson.api.RedissonClient;
import org.redisson.config.Config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class RedissonConfig {
@Bean
public RedissonClient redissonClient() {
Config config = new Config();
config.useSingleServer()
.setAddress("redis://127.0.0.1:6379")
.setConnectionPoolSize(64)
.setConnectionMinimumIdleSize(24);
return Redisson.create(config);
}
}Step 4: Protect Critical Business Logic
Use tryLock with an explicit wait timeout so caller threads fail fast instead of hanging forever under heavy contention:
import org.redisson.api.RLock;
import org.redisson.api.RedissonClient;
import org.springframework.stereotype.Service;
import java.util.concurrent.TimeUnit;
@Service
public class OrderProcessingService {
private final RedissonClient redissonClient;
public OrderProcessingService(RedissonClient redissonClient) {
this.redissonClient = redissonClient;
}
public boolean processOrder(Long productId, int quantity) {
String lockKey = "lock:order:product:" + productId;
RLock lock = redissonClient.getLock(lockKey);
try {
// Wait up to 3 seconds; pass -1 to let the watchdog extend the lease dynamically
boolean acquired = lock.tryLock(3, -1, TimeUnit.SECONDS);
if (!acquired) {
// Another worker holds the lock; reject or queue the request
return false;
}
try {
return executeOrderDeduction(productId, quantity);
} finally {
if (lock.isHeldByCurrentThread()) {
lock.unlock();
}
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return false;
}
}
private boolean executeOrderDeduction(Long productId, int quantity) {
// Database inventory checks and order creation happen here
return true;
}
}How the Watchdog Prevents Premature Expirations
Setting the lease time to -1 turns on Redisson’s internal watchdog. Every 10 seconds—one-third of the default 30-second watchdog timeout—a background task checks whether the thread is still executing. If a slow payment gateway takes 15 seconds to reply, the watchdog automatically refreshes the TTL back to 30 seconds.
What happens if your node crashes mid-operation? The watchdog dies along with the JVM. The lock in Redis then expires naturally within 30 seconds, freeing the key so healthy nodes can resume processing without causing permanent deadlocks.
