API Rate Limiting and Quota Management: Token Buckets, Concurrency Control and Graceful Degradation

Know What the Platform's Limits Look Like

Mainstream Chinese platforms usually enforce multi-dimensional throttling: QPS/concurrency caps that reject excess calls with 429 or a platform-specific error code, daily call quotas counted per application and per API, and escalating penalties — sustained abuse can downgrade or even ban your API credentials, which is far worse than a single error. Limits vary widely across platforms and APIs, so always confirm against official documentation and keep the values in configuration, never hard-coded.

The Token Bucket: Standard Equipment for Client-Side Throttling

A token bucket generates tokens at a fixed rate; each call takes one token and waits when none are available. Its advantage is smoothness plus burst tolerance — tokens saved during quiet periods absorb short spikes such as promotion-day sync. Two engineering points matter:

Throttle globally, not per thread. With multiple instances, use a Redis-based distributed token bucket, or at least split the quota evenly across instances.
Cap concurrency too. Staying under QPS while spiking concurrent connections can still get rejected by the platform gateway; a semaphore set to the platform's recommended level is the safe default.

Quota Budgeting: Spend Daily Calls Like Money

For platforms with daily quotas, allocate budgets per pipeline so that "quota exhausted by noon, order sync down all afternoon" never happens: order pulling and inventory push are core and keep running, while product/master-data sync pauses when over budget and offline reconciliation jobs only run in off-peak windows.

Graceful Degradation: What to Do When Throttled

Being throttled should be an expected event, not an alarm: recognize the platform's throttling error codes and separate them from genuine business errors; retry with exponential backoff plus jitter to avoid a retry storm; automatically slow down or pause non-core pipelines to yield quota to order sync; and only page someone when the backlog crosses a threshold — with backlog size and estimated catch-up time in the alert. Qeasy's connectors ship with a per-platform rate-limit rule library and a quota dashboard, so teams don't rebuild this per platform.