Rate Limiting Explained for Business Systems

Rate limiting explained without the infrastructure jargon: it is the rule for how much traffic a system will accept before it starts refusing. A workflow that is fine on a quiet afternoon can hit that rule on a Monday morning, when a backfill, a campaign, or a full-list sync sends far more calls than the test did. The usual answer is HTTP 429, Too Many Requests. The records do not vanish. They stall, retry in a tight loop, or stop, and someone finds out when a customer asks where the update went.

Rate limiting explained: two sides of the same rule

Outgoing limits are the ones your automation hits. Your workflow automation calls a CRM, an accounts tool, a model API, or a partner. That system decides how many calls your account, key, or IP address may make in a window. When you go over, it refuses. The logic did not change. The pace did.

Incoming limits are the ones you set, on a form, a webhook, a login, or an API you expose to a partner. Without a ceiling, one misconfigured caller can crowd out everyone else, or keep retrying until your own service spends its time saying no.

A limit is a budget of attention, not a verdict on whether the request was valid. Fair use, here, means one caller does not consume the capacity everyone else needs. It is an operational rule, not a legal doctrine.

Why a 429 is not an outage

Vendors limit traffic so a shared service stays available, so abuse is slowed, and so one integration cannot take the platform down. On a usage-based API, a limit also stops an unbounded job from running away. That is a commercial risk as well as an operational one.

RFC 6585 defines 429 as too many requests in a given amount of time. It does not say how the server counts, or who “you” are. MDN notes the same: a limit may apply per resource, per address, or to a logged-in user or application. One product counts reads and writes together. Another counts each endpoint alone. A third counts the account, so a second tool on the same key uses the same allowance.

This article does not list vendor quotas. A copied number is already stale, and it may not be the number on your plan. Read the vendor’s documentation for the account and endpoint you use, and check it again when the integration changes.

A 429 means the service is up, and is asking this caller to slow down. Retry-After, if present, is a wait instruction. Immediate retries ignore it and keep you over the line. RFC 6585 also notes that answering every excess request still costs the server something, so a busy origin may drop the connection instead of sending 429. Timeouts can still be a pace problem.

What fails when volume arrives

The usual failure is a workflow tested on a handful of records, then pointed at a whole list. Each row is one or more calls through an API integration.

In n8n, a 429 surfaces as an error on the node. The product’s documentation quotes the message as “The service is receiving too many requests from you”, plus whatever the vendor sent. Other tools phrase it differently. The meaning is the same: refused for rate, not because the record was malformed.

Treat that as a pace problem. Restarting at once turns the retry into more traffic. A webhook that retries on any error does the same from the other side: the receiver is slow, the sender retries, and both sides look broken. That loop is a design fault, not bad luck.

Queues, backoff, and your own ceiling

Pace the work before you hit the wall. Put records in a queue and release them at a rate the other system has documented. A queue also shows you what is waiting.

Batch only if the API is built for it. Some vendors count the call. Some count each item inside it. Check before you treat a batch as fewer calls.

Backoff is for after a refusal, not instead of pacing. Wait, then wait longer, and add a little randomness so every failed job does not retry at the same instant. If Retry-After is present, wait at least that long. Cap the tries. After the cap, hand the item to a person. Retrying without a ceiling is how a small refusal becomes a stuck process.

On the way in, the same discipline applies. Cloudflare’s rate limiting rules match a class of request, count it over a period, and then block, challenge, or throttle. Their documentation is explicit: the rules are not designed to let an exact number of requests through. Counters can lag by a few seconds, so a few extra requests may still reach your origin. A limit reduces harm. It is not a hard stop at a chosen integer.

Set your own number, and write down why: how many form posts, login attempts, or webhook deliveries one account, key, or IP address may send before you slow them. Do not copy it from a template.

Working inside someone else’s limit

n8n’s documentation describes pacing controls. None of them raises the other side’s limit. Retry On Fail pauses between attempts. Loop Over Items, with a Wait node, breaks the input into chunks and pauses between them. The HTTP Request node can batch items and set an interval between batches. If the documented limit is too low for the volume you have, change the design: a queue, a batch endpoint, fewer repeat reads, or a conversation with the vendor about what your plan allows. A shorter wait is not a higher quota.

Cache reads you do not need to repeat. Do not assume a read and a write share a ceiling. If the API sends remaining-allowance or reset headers, watch those before you hit the wall. Keep logging of refusals, with endpoint, account, and time, and name the person who sees the unfinished items. A log nobody opens is not a process.

If calls and responses are still fuzzy, start with API basics. If the burst comes from events rather than a schedule, read webhooks.

What to write down

List the automations that call an external system. For each, note the account or key, whether it can burst, what it does on a 429, and who is told. If the last two answers are “it errors” and “nobody”, that is the gap.

Fernside scopes and builds this pacing as project work: queues, backoff, and logging around integrations you already depend on. That is a build, not a switch. AI systems is that engagement. If the system is already live and the question is how refusals are watched after it ships, that sits with managed systems: ongoing care, not a substitute for designing the queue.

If you want the pacing designed before the next backfill, send an enquiry.