Rate Limiting

Sophon's Gateway applies a single global token-bucket rate limiter to incoming HTTP requests. It is partitioned: authenticated traffic is limited per user, anonymous traffic is limited per IP. The goal is to absorb short bursts while capping sustained abuse, without ever throttling the things that legitimately spike — inbound channel webhooks and the Dashboard's static assets.

The limiter is built on ASP.NET Core's standard System.Threading.RateLimiting token-bucket primitives, keyed by a Gateway-resolved partition per request.

How partitioning works

For each request the limiter resolves a partition key:

• If the request carries a validated user identity (the NameIdentifier claim), the key is user:<id>.
• Otherwise it falls back to ip:<address>, derived from the first hop of X-Forwarded-For (when present) or the connection's remote IP.

Each key gets its own independent token bucket. The per-user limit is the unspoofable guarantee — it is keyed on the authenticated claim. Per-IP limiting is best-effort for anonymous traffic, since X-Forwarded-For is client-supplied; it assumes a trusted reverse proxy populates that header.

What is exempt

Two categories of traffic bypass the limiter entirely:

1. Excluded path prefixes — health checks, metrics, SignalR hubs, and all inbound webhooks. Webhooks are exempt because platform channel receivers authenticate via HMAC signatures (not JWT) and arrive from shared platform egress IPs that would otherwise exhaust the anonymous per-IP bucket on a busy channel.
2. Dashboard SPA static assets — JS, CSS, and font chunks. Static file serving is registered before the rate limiter in the pipeline, so a matched file short-circuits and is never counted against a bucket. Non-file requests fall through to the limiter and auth layers as normal.

The default excluded prefixes are /api/health, /health, /metrics, /hubs, and /webhooks.

Configuration

The limiter reads the Sophon:RateLimit configuration section. Following Sophon's standard mapping, each key is also settable via a SOPHON__RateLimit__<Key> environment variable. All values below are the source defaults.

In a token bucket, Burst is how many requests can fire back-to-back from a full bucket, and PermitLimit tokens are added back every WindowSeconds. With the defaults, an authenticated user can burst 20 requests, then sustain 120 per minute.

// appsettings.user.json (or via SOPHON__RateLimit__* env vars)
{
  "Sophon": {
    "RateLimit": {
      "Enabled": true,
      "PerUserPermitLimit": 120,
      "PerUserBurst": 20,
      "PerUserWindowSeconds": 60,
      "PerIpPermitLimit": 60,
      "PerIpBurst": 10,
      "PerIpWindowSeconds": 60
    }
  }
}

SignalR hub invocation limits

The /hubs exemption above only applies to the HTTP edge — a SignalR connection is one long-lived request, so an HTTP token bucket can't meaningfully meter it. Realtime traffic is limited at the invocation level instead: hub method calls (chat sends, audio streaming, subscription changes) are metered per authenticated user, falling back to the connection id for a connection that has no user identity, using the same token-bucket semantics as the HTTP limits. Exceeding the limit fails that invocation with an error naming the retry delay; it does not drop the connection.

Hub invocation limits live in the same configuration section, under the Sophon:RateLimit:Hub* keys (settable via SOPHON__RateLimit__Hub* environment variables, like every other key here). Leave them at the defaults unless you run high-frequency realtime workloads.

Every invocation costs exactly one token, regardless of how long it runs. That matters most for voice: an audio stream is a single long-lived hub invocation, so a whole spoken utterance spends one token, not one per audio chunk. Streaming voice is not a burst of small invocations, and raising these limits for it is unnecessary.

Because a long-running stream can't be metered by an invocation counter, it carries its own budget instead. Each audio stream is bounded by:

• A duration budget — the caller's max-utterance length, defaulting to 60 seconds, clamped to 1–300 seconds. When it's reached, intake ends gracefully and whatever was heard is still transcribed.
• A byte budget derived from that duration and the stream's sample rate, with headroom for client-side buffering bursts. A stream that exceeds it is flushed rather than left to grow.
• One active stream per connected client. A new utterance cancels its predecessor, unless that one has already finished taking audio.

Max utterance length is a per-user listening preference with a host default — see Voice Settings.

Cron stagger prevents scheduled-job bursts

Rate limiting protects the HTTP edge, but scheduled jobs are an internal source of synchronized load. When many cron jobs share the same schedule they can fire at the same instant — a thundering herd. Sophon's scheduler applies an optional random stagger to spread them out.

The Sophon:Scheduling section exposes one key:

When set, each job receives a random offset in [0, CronStaggerMaxSeconds]. This meaningfully phase-shifts repeating interval triggers so they no longer fire in lockstep. For raw cron-expression triggers the offset only delays first eligibility — the expression itself dictates each recurrence and cannot be individually phase-shifted — so jitter is not applied there.

Where to go next

• Cron & Scheduled Jobs — defining schedules and trigger types
• Operations — running Sophon in production