Rate Limiter

Rate limiting is the system-design question almost everyone can recite and almost nobody can actually implement. We all know the lines — *token bucket allows bursts, sliding window log is exact but expensive* — and then, asked to write one, produce a Map<key, count> with a setTimeout and call it a day.

The trouble is that the five canonical algorithms differ in ways you cannot feel from a paragraph. The token bucket and the leaking bucket are duals — one tolerates bursts, one smooths them — and that sentence means nothing until you watch a burst hit both. The fixed window counter's famous flaw, the **2× boundary burst**, is a footnote in every blog post and a real outage in production. The sliding window counter is an *approximation* that most production systems actually use, and reading its formula tells you nothing about whether the approximation is any good.

So: implement all five properly — thread-safe, cost-weighted, per-key, on a monotonic clock — in C++, put them behind a real API, and then build a simulator where you can *watch* the mechanism. Tokens refilling from a drip pipe. Water leaking from a tank. A window dial counting down and flashing as it resets. Packets flying at a gate and veering off into a rejected bin.

And do it honestly: the hot path is compiled C++ running inside the process, not JavaScript pretending to be one.

Solo Engineer — the C++ library, the N-API bridge, the Node API, the simulator, the tests and the CI.

**C++ library (header-only, C++17):** five algorithms, each in its own header behind a common RateLimiter interface returning a Decision { allowed, remaining, retry_after, limit, used }. Every limiter guards its per-key state with a std::mutex, all timing uses std::chrono::steady_clock, every request carries a cost, and constructors validate their arguments and throw. Zero dependencies beyond the standard library.

**The bridge:** an N-API native addon (node-addon-api, built with node-gyp) exposing a Manager registry that owns one limiter instance per algorithm *plus the parameters it was built with* — so a single limiter can be rebuilt live. The registry has its own mutex.

**Node API (Express):** /api/meta, /api/check, /api/config, /api/reset, /api/source, with real semantics — allowed returns **200**, denied returns **429** with Retry-After, and every response carries X-RateLimit-* headers.

**Frontend (Next.js 15, App Router):** an animated <canvas> simulator driven by a requestAnimationFrame loop — request packets travelling Client → gate → Server or Rejected, with a distinct mechanism animating per algorithm. Live parameter sliders that reconfigure the running engine, an auto-traffic generator, traffic presets (Steady / Bursty / DDoS-spike), a decision feed, a live throughput chart, a built-in benchmark, a C++ source viewer, and a *served by C++ in X ms* latency badge.

**Ops:** one-command dev (npm run dev builds the addon, installs deps, and runs both servers with prefixed logs and tied lifetimes), Docker Compose, an assertion-based C++ test suite with no framework, and GitHub Actions CI that runs the C++ tests, builds the addon, smoke-tests the API and builds the frontend.

1$      Next.js :3000                 Node / Express :8080
2$  ┌──────────────────┐          ┌──────────────────────────┐
3$  │ canvas simulator │          │  /api/* routes           │
4$  │                  │          │          │ require()     │
5$  │ /api/* ─rewrite──┼─────────▶│  ┌───────▼────────────┐  │
6$  └──────────────────┘  no CORS │  │ N-API addon (C++)  │  │
7$                                │  │ Manager (mutex)    │  │
8$                                │  └───────┬────────────┘  │
9$                                │  ┌───────▼────────────┐  │
10$                                │  │ rate_limiter lib   │  │
11$                                │  │ 5 algorithms       │  │
12$                                │  │ per-key · mutex    │  │
13$                                │  └────────────────────┘  │
14$                                └──────────────────────────┘

The browser only ever talks to one origin: Next.js rewrites /api/* to the Node backend, so there is no CORS layer to configure — or to get subtly wrong — and the frontend never hard-codes a backend host.

**The library.** Every algorithm implements one call — Decision allow(const std::string& key, int cost = 1) — and keeps an unordered_map<key, state>, so keys are completely independent. allow() takes the lock, reads the monotonic clock, and runs the algorithm's check() while still holding it, which makes every read-modify-write atomic with respect to other threads.

**The five, and what actually separates them:**

- **Token bucket** — tokens refill at a steady rate up to a capacity; a request spends cost. Allows bursts up to the capacity while bounding the long-run average. O(1) memory. The sensible default. - **Leaking bucket** — requests fill a bucket that leaks at a constant rate; anything that would overflow is rejected. The dual of the token bucket: it *smooths* bursts into a constant outflow. O(1). For a fragile downstream that wants a steady rate. - **Fixed window counter** — a count per window, reset on the boundary. O(1) and trivial — and it permits up to **2× the limit** across a boundary, which is the entire reason the sliding variants exist. - **Sliding window log** — one timestamp per accepted request, evicted as it slides out of the window. **Exact**, no boundary burst — at O(limit) memory per key. - **Sliding window counter** — approximates the log with two counters, weighting the previous window by its live overlap: estimated = current + previous × (1 − elapsed / window). Near-exact accuracy at **O(1)** memory. The practical production default.

**The registry.** Inside the addon, a Manager owns one limiter per algorithm and the params it was constructed with. /api/config rebuilds a single limiter with new parameters (clearing only that algorithm's state); /api/reset rebuilds them all. That is what lets the UI's sliders reconfigure the *running* engine rather than a copy of it.

Rate Limiter is a **teaching instrument that happens to be production-shaped**.

**Who it is for:** engineers preparing for system-design interviews, where *design a rate limiter* is close to a guaranteed question — and where the gap between reciting the five algorithms and having actually **built** them is immediately audible. And anyone who wants a dependency-free C++ limiter they can drop straight into a service.

**The hook is the simulator.** Set the traffic to a steady 10/s against a limit of 10 per 5s and watch the five diverge: the buckets let a trickle through forever, the fixed window allows a chunk and then hard-blocks until it resets, and the sliding variants throttle evenly. That is something you can *see* in twenty seconds and could not have got from twenty minutes of reading. The DDoS-spike preset is the same lesson with a bigger hammer.

**Go-to-market:** live at rate-limiter-xi.vercel.app → GitHub with the full algorithm write-up, pseudocode and a side-by-side comparison table → system-design prep communities. MIT-licensed, and the C++ headers are usable standalone.

**What it signals:** most portfolios prove you can *call* an API. This one proves you can write the thing that API is protected **by** — in C++, thread-safe — and then bridge it into Node without pretending the boundary does not exist.

Live at rate-limiter-xi.vercel.app with source at github.com/subhm2004/Rate_Limiter.

**Shipped (engine):** five algorithms in header-only C++17 — token bucket, leaking bucket, fixed window counter, sliding window log, sliding window counter — behind one RateLimiter interface returning { allowed, remaining, retry_after, limit, used } · **thread-safe** (a std::mutex per limiter, another on the addon registry) · **cost-weighted** requests · **per-key isolation** · **monotonic** steady_clock timing · validating constructors · zero dependencies beyond the standard library.

**Shipped (bridge + API):** N-API native addon (node-addon-api / node-gyp) with a Manager registry owning one limiter per algorithm and its parameters · Express API — /api/meta, /api/check, /api/config, /api/reset, /api/source · **real semantics**: 200 on allow, **429 with Retry-After** on deny, X-RateLimit-* headers on every response · live reconfiguration driven by the UI's sliders.

**Shipped (simulator):** animated <canvas> — packets fly Client → gate → Server / Rejected, with a distinct mechanism per algorithm (tokens refilling from a drip pipe, water leaking from a tank, a window dial counting down and flashing on reset, log ticks sliding out, the two weighted counters of the sliding counter) · live parameter sliders · auto-traffic generator · **traffic presets** (Steady / Bursty / DDoS spike) · decision feed · **live throughput chart** (allowed vs denied per second) · **built-in benchmark** comparing allow-rate, throughput and p99 latency across all five · **C++ source viewer** serving the real header from the repo · a **"served by C++ in X ms"** latency badge · no CORS, thanks to the Next.js rewrite.

**Shipped (ops):** one-command dev workflow · Docker Compose for a toolchain-free run · an assertion-based C++ test suite covering burst limits, time-based recovery, window resets, sliding-log boundary accuracy, key independence, and a **200-thread concurrency stress** asserting the limit is never exceeded · GitHub Actions CI running the C++ tests, the addon build, an API smoke test and the frontend build on every push.

The honest limitation is that **state lives in memory inside a single process** — two backend instances would not share counters, so this is not a distributed limiter. The fix is well understood (back the state with Redis and make each check an atomic Lua script) and, notably, the *algorithm logic would not change at all* — only where the state lives would.

Related: the per-key unordered_map grows as new keys appear and nothing ever evicts them, so a long-lived process with a large key space slowly leaks memory. TTL or LRU eviction of idle keys is the obvious next commit.

The API is also unauthenticated and un-TLS'd, which is fine for a local teaching tool and would be indefensible anywhere else. And the genuinely useful next feature is a real **middleware** wrapper — the limiter guarding actual endpoints rather than a /check route — because that is how anyone would really use it.