API Rate Limiting Strategies: Algorithms, Limits and Retries

The best rate limiting strategy for most APIs is a token bucket (or a sliding window counter) keyed by authenticated client, with a tighter limit on expensive endpoints, a clear 429 Too Many Requests response with a Retry-After header, and a shared store such as Redis once you run more than one instance. Everything else is tuning. This guide explains how to choose each piece, where the tradeoffs sit, and how the client side should behave when it hits a limit.

TL;DR

• Rate limiting caps how many requests a client can make in a time window. It protects against overload, unfair usage, abuse and runaway costs.
• Pick the algorithm by traffic shape: token bucket for legitimate bursts, leaky bucket to smooth traffic into fragile systems, sliding window when boundary bursts matter, fixed window when simplicity wins.
• Limit by the identity that matches your risk: API key or user for paying clients, IP for anonymous traffic, tenant for multi-tenant fairness, and endpoint for expensive operations.
• Return 429 with Retry-After, publish your limits, and expect clients to retry with exponential backoff and jitter.
• Across several instances, decide how exact the global limit must be. Exactness costs latency and adds a shared dependency.

What rate limiting does and why you need it

Rate limiting regulates how many requests a user can make in a specific time. Microsoft’s documentation lists the benefits as preventing overload, ensuring fair usage, increasing security and managing costs. Those four reasons map to real decisions you will make later, so it helps to keep them separate.

Overload. A single misbehaving script, a bug in a client retry loop or a viral launch can send more traffic than your database can serve. A limit keeps the damage contained to the caller that caused it.

Fairness. Without limits, one heavy customer can monopolize shared capacity. Akamai describes the goal as giving eligible users similar service quality and preventing resource monopolization, while tracking usage to flag problems.

Security. Login, password reset and token endpoints attract brute-force attempts. A strict limit on those routes is one of the cheapest defenses you can add.

Cost. If a request triggers heavy compute, large database reads or a paid third-party call, every extra request is money. Limits turn an open-ended bill into a bounded one. This matters most when your API sits in front of other APIs: shipping carriers, payment providers and AI models all enforce their own limits, and your own service has to respect them. If you work with carrier APIs, this overview of shipping API integration pitfalls shows how quickly those constraints shape a project.

The design also has a human side. The API Patterns catalog says a rate limit design should weigh economics, performance, reliability, abuse severity and client awareness. Clear limits are part of the API contract, so your clients can pace themselves, batch work and cache responses.

Rate limiting vs throttling

People use the two terms interchangeably, and vendors define them slightly differently. The most useful working distinction is about what happens to the excess request.

• Rate limiting sets a hard ceiling, for example 100 requests per minute, and rejects what goes beyond it. Gravitee’s explanation frames it this way.
• Throttling shapes traffic. Depending on load and policy it may delay, queue or reject requests. Zuplo puts it simply: a limiter rejects, a throttle queues requests to run when capacity is available.

Not everyone draws the line in the same place. Rokka, a Swiss image service, describes its throttling as delaying requests before rejecting excess calls with a 429. In practice, treat throttling as a family of behaviors and rate limiting as one of them.

There is a handy way to tell them apart from the client side. According to Requestly, rising latency with successful 2xx responses may point to throttling, while immediate 429 responses mean a hard limit was reached.

A related term is quota. Rate limits usually cover short intervals such as seconds, minutes or hours, while quotas cover longer ones such as days, weeks or months. Most commercial APIs use both: a per-minute limit to protect the system and a monthly quota tied to the plan.

The main rate limiting algorithms

Four algorithms cover almost every production case. Vercel’s comparison names fixed window, sliding window (log or counter), token bucket and leaky bucket.

Algorithm	How it works	Strength	Weakness	Fits best
Fixed window	Counts requests in fixed intervals, such as each minute	Cheap and simple	Bursts at window boundaries	Internal tools, coarse protection
Sliding window log	Stores a timestamp per request	Exact	Costly in memory	Low volume, high precision
Sliding window counter	Weights the current and previous window counts	Fixed memory	Approximate	General public APIs
Token bucket	Tokens refill at a steady rate, each request spends one	Allows controlled bursts	Needs refill computation	Developer APIs with legitimate bursts
Leaky bucket	Requests enter a queue that drains at a constant rate	Smooth output	Bounded by queue size, adds delay	Protecting fragile downstream systems

The tradeoffs are consistent across sources. Upstash’s documentation describes fixed windows as cheap but vulnerable to bursts at window boundaries, sliding windows as reducing that problem at the cost of more storage and computation, and token buckets as permitting controlled bursts while requiring ongoing refill computation.

The boundary burst problem

With a fixed window of 100 requests per minute, a client can send 100 requests at 12:00:59 and another 100 at 12:01:00. That is 200 requests in two seconds, while every window technically stays within its limit. If your backend cannot absorb double the nominal rate, move to a sliding window or a token bucket.

Token bucket in plain terms

Imagine a bucket holding 20 tokens that refills at 5 tokens per second. A client that was idle can send 20 requests at once, then settles at 5 per second. That matches how real integrations behave: a page load fires several calls together, then goes quiet. Vercel’s guidance is that token buckets suit developer APIs with legitimate bursts, while leaky buckets suit smoothing traffic into fragile downstream dependencies.

A practical default

If you have no strong reason to choose otherwise, start with a token bucket or a sliding window counter. Both avoid the boundary problem without the memory cost of a full log. Use a leaky bucket only when something behind your API truly needs a constant request rate.

What to limit by

The algorithm decides how you count. The key decides who gets counted. MDN notes that limits may be server-wide or per resource, and are typically based on a client’s IP address or, for authenticated requests, on users or authorized applications.

Key	Use it when	Watch out for
IP address	Anonymous traffic, login pages, public endpoints	Many users behind one office or mobile network address
API key or user	Authenticated APIs and paid plans	Leaked keys, free accounts created in bulk
Tenant or organization	Multi-tenant products where fairness between customers matters	One tenant’s many users sharing a single budget
Endpoint	Expensive or sensitive routes	Too many separate rules to maintain

Most mature APIs combine them. A common pattern is a generous per-key limit across the whole API, plus much stricter per-endpoint limits on a few routes. Salesforce, for example, describes using per-tenant, per-endpoint limits in its coordinated limiter.

Setting the numbers

No formula gives you the right number. A workable process looks like this:

1. Measure real traffic. Find the typical and peak request rates of your well-behaved clients per key.
2. Find your capacity. Load-test the slowest dependency (usually the database) to learn how many requests it can serve before latency degrades.
3. Set the limit above normal peaks but well below capacity divided by the number of clients you expect to be active together.
4. Tier by value. Give paying plans higher limits, and give expensive endpoints their own lower ones.
5. Start in observe mode. Log what would have been blocked before you enforce anything.

The last step matters. Akamai warns that aggressively low or inconsistent limits can cause slower responses, denial errors and intermittent access for legitimate users. A limit that blocks a good customer during a normal busy hour is a bug, not protection.

Making limits work across several instances

A limiter that counts in a single process is easy. The trouble starts when your API runs on several instances or in several regions, because each one sees only part of the traffic.

Tyk documents the core problem: if each data plane enforces limits on its own, the effective global limit can multiply. A limit of 100 per minute across five instances can quietly become 500.

You have three broad options:

• Local counters per instance. Fastest and most isolated, but the global limit is only approximate.
• A shared Redis store. Exact coordination, but each request pays a network round trip and the store becomes a dependency whose failure affects every request. Tyk spells out both costs.
• Replicated Redis. Better availability, but eventually consistent, so limits can be overshot during replication lag.

The guidance from Tyk is that the consistency you need should drive the design: per-region enforcement favors latency and isolation, while exact global limits require shared coordination and accept its availability and latency costs.

Real systems often accept a little inaccuracy. Salesforce combines local and Redis-backed caches with Redis atomic increments and accepts slight overshoot for performance. GitHub, at far larger scale, reports using sharded Redis with primary nodes for writes and replicas for reads, plus Redis expiration and Lua scripts to keep rate-limit operations atomic.

Two practical rules follow:

• Make the check atomic. A read followed by a separate write lets two concurrent requests both pass. Use a single atomic increment or a script.
• Decide what happens when the store is down. Failing open keeps the API available but unprotected. Failing closed protects the backend but blocks everyone. For most public APIs, fail open with a conservative local fallback limit is a sensible middle ground. Login endpoints may deserve the opposite choice.

Where you enforce also matters. An API gateway or edge layer stops excess traffic before it reaches your application, and application-level limits can then handle business rules such as per-plan quotas.

What to return when a client hits the limit

The standard response is HTTP 429. MDN explains that an API may return 429 Too Many Requests with a Retry-After header telling the client how long to wait before trying again.

Good limit responses share a few traits:

• A 429 status, not a generic 500 or a 403.
• A Retry-After header with the wait time.
• Limit headers on normal responses showing the limit and what remains, so clients can slow down before they are blocked. Rokka’s API, for instance, returns limit and remaining headers on its cache-invalidation endpoints and a retry-after header on 429.
• A short, readable error body that names the limit that was hit and links to your documentation.

Publish the limits in your docs. Surprise limits create support tickets, and well-documented ones let integrators design around them.

How clients should react: backoff and jitter

A limit only works if clients behave. The standard approach is retrying with exponential backoff, which means waiting longer after each failure. Adding jitter, a random variation in the wait, stops many clients from retrying at the same instant.

This is mainstream guidance from the big cloud providers. AWS SDKs can automatically retry requests that fail due to transient errors or throttling, using exponential backoff with jitter. Google Cloud recommends the same pattern for transient errors such as HTTP 429 and 5xx responses. Google also advises truncated exponential backoff with jitter to prevent synchronized retries, the so-called thundering herd.

A sound client does this:

1. Honor Retry-After when present, and fall back to exponential backoff when it is not.
2. Add random jitter to each wait.
3. Cap the maximum wait and the number of attempts.
4. Retry only safe or idempotent requests, or send an idempotency key.
5. Cache responses and batch calls so it needs fewer requests in the first place.

Retries are not always the cure. With Google’s Provisioned Throughput, frequent 429 errors usually mean the purchased capacity has been exceeded, so adding retries will not fix the underlying issue. If you see constant 429s on a paid, reserved tier, the answer is more capacity or fewer calls. Also note that AWS lets you turn retries off by setting the maximum number of attempts to 1, which is useful when you want a failure to surface immediately rather than be hidden.

Best practices checklist

This list pulls the pieces together:

• Analyze traffic first. Know what normal looks like before choosing numbers.
• Match the algorithm to the traffic. Bursty developer traffic wants a token bucket. Fragile backends want a leaky bucket.
• Use layered keys. A global per-client limit plus stricter endpoint limits covers most risks.
• Protect sensitive routes harder. Login, password reset and expensive operations deserve their own, lower limits.
• Keep limits fair in multi-tenant systems. Limit per tenant so one customer cannot consume everyone’s capacity.
• Communicate. Use 429, Retry-After and limit headers, and document everything.
• Monitor. Track blocked requests per key and per endpoint. A spike in blocks for one legitimate customer usually means your limit is wrong, not that they are attacking you.
• Review regularly. Traffic grows, features change and a limit chosen a year ago may be too tight or too loose today.
• Keep the limiter off the critical path where you can. Use local caches and accept small overshoot when exactness is not worth the latency.

Getting this built into your product

Rate limiting touches the gateway, the application, the data store and the client documentation, so it is easier to get right when one senior team owns the whole picture. At Ampersand, a Zurich-based studio, senior engineers lead each project from the first consultation to handover, and we build and integrate APIs, web applications and platforms for startups and growing teams. If you need help designing limits for a new platform, connecting systems whose APIs have strict quotas, or keeping an integration healthy after launch, see our services and plans. Our Run & monitor plan is CHF 750 per month, and our Guidance plan is CHF 799 per month.

FAQ

What is the difference between API throttling and rate limiting?

Rate limiting sets a hard cap on requests in a time window and rejects anything above it. Throttling is broader: it shapes traffic by delaying, queuing or sometimes rejecting requests depending on load. If you see rising latency with successful responses, you are probably being throttled. If you get immediate 429 errors, you hit a hard limit.

Which rate limiting algorithm should I use?

Start with a token bucket if your clients send legitimate bursts, or a sliding window counter if you want to avoid boundary spikes without heavy memory use. Fixed windows are fine for simple, low-risk cases. Choose a leaky bucket when a fragile downstream system needs a steady flow.

Should I limit by IP address or by API key?

Use the API key or user for authenticated traffic, because it identifies the actual client. Use IP address for anonymous endpoints such as login and signup, keeping in mind that many users can share one IP. In multi-tenant products, add a per-tenant limit so one customer cannot crowd out the others.

What should my API return when a limit is exceeded?

Return HTTP 429 Too Many Requests with a Retry-After header that tells the client how long to wait. Add limit and remaining headers to normal responses so clients can slow down on their own. Document the limits so integrators can plan for them.

How do I rate limit across multiple servers?

Use a shared store such as Redis with atomic operations if you need an exact global limit, and accept the extra latency and the new dependency. If approximate is enough, count locally per instance or per region and tolerate some overshoot. The right choice depends on how exact the limit must be.

Sources

• What Is Rate Limiting? Dev Proxy, Microsoft Learn
• What Is Rate Limiting? Akamai
• Rate Limit, API Patterns
• API Throttling: How It Works and How to Implement It, Gravitee
• What is API Rate Limiting? Zuplo
• API Rate Limiting and Throttling, rokka
• API Throttling vs Rate Limiting, Requestly
• Rate limiting strategies for APIs, Vercel
• Ratelimit Python SDK Algorithms, Upstash
• 429 Too Many Requests, MDN
• Coordinated Rate Limiting in Microservices, Salesforce Engineering
• How we scaled the GitHub API with a sharded, replicated rate limiter in Redis
• Cross-Region Rate Limits and Quotas with MDCB, Tyk
• Retry behavior, AWS
• Retry strategy, Generative AI on Vertex AI, Google Cloud
• Retry strategy, Model Armor, Google Cloud
• Shipping API integration pitfalls, Packlogica