Distributed Rate Limiting with Redis: Why an In-Memory Limiter Fails Behind a Load Balancer

An in-memory rate limiter is usually the first thing people reach for when an endpoint needs protection from abuse, and it works fine as long as the API runs in a single process. The trouble starts the moment that same API scales horizontally behind a load balancer: each replica keeps its own count, none of them know what the others are counting, and a limit designed as "one hundred requests per minute per client" quietly turns into "one hundred requests per minute per client, per replica that happens to handle the request." We saw this in production before we understood it: abuse alerts that never fired, followed by load spikes that did make it all the way to the database.

How the local token bucket breaks

A typical in-memory token bucket implementation keeps its state in a Map inside the Node.js process itself:

const buckets = new Map();

function takeToken(clientId, capacity = 100, windowMs = 60_000) {
  const now = Date.now();
  const state = buckets.get(clientId) ?? { tokens: capacity, lastRefill: now };

  const elapsed = now - state.lastRefill;
  const refill = (elapsed / windowMs) * capacity;
  state.tokens = Math.min(capacity, state.tokens + refill);
  state.lastRefill = now;

  if (state.tokens < 1) {
    buckets.set(clientId, state);
    return false;
  }

  state.tokens -= 1;
  buckets.set(clientId, state);
  return true;
}

This is correct and cheap as long as a single process serves the traffic. The Map lives in that process's memory, so there is no coordination to solve: every request reads and writes the same state without competing against anything else. As soon as a second replica goes live behind a round-robin load balancer, each one starts with its own empty Map, and a client sending a hundred requests split across two replicas never hits the limit, because from each process's point of view only fifty requests arrived. The limit written in the code is still one hundred, but the limit the system actually enforces is one hundred multiplied by however many replicas happen to be running at that moment — a number that changes every time autoscaling adds or removes an instance.

INCR and TTL: a shared, atomic counter

The fix isn't to synchronize each process's Map with the others — that just reintroduces the distributed-coordination problem we were trying to avoid in the first place. It's to move the count out of the process entirely and into a store every replica can read and write equally. Redis is a good fit here because INCR is atomic on the server side: two replicas running INCR on the same key at the same time never step on each other's increment, since Redis processes commands serially internally.

const redis = require('ioredis').createClient();

async function allowRequest(clientId, limit = 100, windowSec = 60) {
  const key = `ratelimit:${clientId}:${Math.floor(Date.now() / 1000 / windowSec)}`;

  const count = await redis.incr(key);
  if (count === 1) {
    await redis.expire(key, windowSec);
  }

  return count <= limit;
}

The key embeds a truncated time window, so every minute produces a fresh key without anything needing to clean up manually: the EXPIRE call, which only runs the first time the key is created, tells Redis to remove it on its own once the window ends. Any replica that receives a request from that client during that minute increments the same counter, regardless of which one actually handled the request, because the state no longer lives in the process — it lives on a shared server.

This fixed-window scheme has a known edge effect: a client can exhaust its limit in the last second of one window and exhaust it again in the first second of the next, effectively getting double the nominal limit within a two-second span around the window boundary. For most abuse-protection use cases this is acceptable, since the goal is to keep a client from sustaining load against the backend, not to enforce a mathematically exact limit over any arbitrary sliding window.

A sliding window with sorted sets when the boundary matters

When the boundary effect does matter — say, a limit protecting a third-party API billed per request, where doubling the nominal limit has a direct cost — a Redis sorted set lets you implement a real sliding window by storing each request's timestamp as its score:

async function allowRequestSliding(clientId, limit = 100, windowMs = 60_000) {
  const key = `ratelimit:sliding:${clientId}`;
  const now = Date.now();
  const lowerBound = now - windowMs;

  const multi = redis.multi();
  multi.zremrangebyscore(key, 0, lowerBound);
  multi.zadd(key, now, `${now}-${Math.random()}`);
  multi.zcard(key);
  multi.pexpire(key, windowMs);
  const results = await multi.exec();

  const totalInWindow = results[2][1];
  return totalInWindow <= limit;
}

Every request is recorded as a sorted-set member with its timestamp as score. Before counting, members whose score falls outside the current sliding window get removed, so ZCARD always reflects exactly how many of that client's requests fell within the last windowMs milliseconds, no matter where the clock happens to sit relative to the window. This costs more than a plain INCR — it stores one member per request instead of a single counter, and expired members need periodic cleanup — so it's worth reserving for limits where the fixed-window edge genuinely matters, not applying it by default to every endpoint.

What didn't work: separate GET and SET from the client

Before switching to INCR, the first version we tried read the counter with GET, incremented it in the Node.js code itself, and wrote it back with SET:

// Version with a race condition
async function allowRequestBroken(clientId, limit = 100) {
  const key = `ratelimit:${clientId}`;
  const currentValue = await redis.get(key);
  const count = (currentValue ? parseInt(currentValue, 10) : 0) + 1;
  await redis.set(key, count, 'EX', 60);
  return count <= limit;
}

This looked equivalent to INCR, but it had a real race condition: there are two separate network round trips between the GET and the SET, and if two replicas run this block for the same client at nearly the same time, both can read the same value before either has written its own, and both end up writing the same incremented count — losing an increment in the process. Under light traffic this almost never showed up, which made it worse: it passed manual testing without issue and only started letting extra requests through once concurrent traffic per client rose enough that the GET/SET windows from different replicas began overlapping regularly. The fix wasn't to add retries or optimistic locking around the GET/SET pair — that just moves the problem elsewhere — it was to replace the pattern entirely with INCR, which resolves the read, the increment, and the write as a single atomic operation on the Redis server, leaving no window where another replica can slip in between steps.

When the round trip to Redis is worth it

Adding a network call to Redis in the critical path of every request has a cost that an in-memory Map doesn't, and for low-risk internal services running a single replica that cost isn't justified. The question we use to decide is direct: can this endpoint receive traffic split across more than one process at the same time? If the answer is yes — and for any API with autoscaling, or with more than one fixed instance behind a load balancer, the answer almost always is — an in-memory counter isn't reliably limiting anything. It's only limiting what each replica happens to see on its own, which is a much weaker guarantee than the one the limit was supposed to provide on paper.