Proxy Overhead Benchmark

Last measured: 2026-07-14

A proxy only earns its place if it stays out of the way. This page reports how much latency the Spanlens proxy adds to a request, explains exactly how that number is produced, and gives you the command to reproduce it. We would rather publish a benchmark you can run than a marketing figure you have to trust.

The result

Synchronous proxy overhead per request, measured over 100,000 warm iterations using the same header-transform functions that run in production:

The proxy's own synchronous work is on the order of microseconds, not milliseconds. The rest of a request's time is the upstream provider answering your call, which Spanlens does not change.

Why it is this small

The expensive part of observability, writing the request, computing cost, masking PII, and inserting the log row, does not happen while your caller waits. Spanlens streams the provider response straight back to the client and dispatches logging as a fire-and-forget task that runs after the response has already been returned. Logging is off the response critical path by design, so it cannot slow the caller down. If the logging pipeline is briefly unavailable, work is captured in a fallback queue and replayed later rather than blocking the request.

What remains on the caller's path is the synchronous work measured above: stripping hop-by-hop and Spanlens-internal headers, attaching the upstream credential, and reconstructing the response. That is the number in the table.

Methodology

The benchmark imports the real production transform functions and exercises the non-streaming proxy hot path against an in-process mock upstream. It measures with nanosecond-resolution timers after a warmup, then reports sorted percentiles.

To keep the number honest, it deliberately excludes three things and reports why:

• Network time to the provider is excluded. That is OpenAI, Anthropic, or Gemini answering your request, not Spanlens. Measuring a real fetch would only measure their latency.
• The provider-key lookup(a database round-trip) is excluded from this CPU figure. In production it runs concurrently with request parsing and is reported live as part of the request's proxy_overhead_ms.
• Asynchronous logging is excluded because it runs after the response is returned. Including it would misrepresent what the caller actually waits for.

So this is a conservative, controlled measurement of synchronous proxy CPU cost, not an end-to-end latency claim. It was measured locally on Node 22; because the work is CPU-bound and small, results are broadly representative, and you can reproduce them on your own hardware.

Reproduce it

The benchmark script is part of the open-source repository. Clone it and run:

pnpm --filter server exec tsx scripts/bench-proxy-overhead.ts

The script is at apps/server/scripts/bench-proxy-overhead.ts. It prints JSON with the percentiles above. Your absolute numbers will vary with hardware; the order of magnitude, microseconds, should not.

Measured in production too

Every request Spanlens logs carries a proxy_overhead_ms value, so the overhead is observed on live traffic rather than only in a benchmark. If you self-host, the same field is available on your own requests. Read more about the request pipeline in the reliability documentation and the agent tracing overview.