Proxy Overhead Benchmark

Last measured: 2026-07-14

Measure the CPU work involved in transforming proxy headers and reconstructing a response. This page explains the measurement method and provides a command to reproduce it locally.

The result

Header-transform and response-construction CPU time, measured over 100,000 warm iterations using the same header-transform functions that run in production:

These results describe the measured CPU operations on a warmed local process. Authentication, database lookups, network routing, and deployment conditions also contribute to latency on a real request.

Asynchronous request logging

Spanlens supports streaming responses and saves request logs asynchronously. This reduces the time spent waiting for log storage during request handling. See the reliability guide for how request forwarding and background logging work.

The benchmark exercises these synchronous operations: stripping hop-by-hop and Spanlens-internal headers, attaching the upstream credential, and reconstructing the response. That is the number in the table.

Methodology

The benchmark imports the real production transform functions and exercises the non-streaming proxy hot path against an in-process mock upstream. It measures with nanosecond-resolution timers after a warmup, then reports sorted percentiles.

To keep the number honest, it deliberately excludes three things and reports why:

• Network time to the provider is excluded. That is OpenAI, Anthropic, or Gemini answering your request, not Spanlens. Measuring a real fetch would only measure their latency.
• The provider-key lookup (a database round-trip) is excluded from this CPU figure.
• Asynchronous logging is excluded because it runs after the response is returned. Including it would misrepresent what the caller actually waits for.

So this is a conservative, controlled measurement of synchronous proxy CPU cost, not an end-to-end latency claim. It was measured locally on Node 22; because the work is CPU-bound and small, results are broadly representative, and you can reproduce them on your own hardware.

Reproduce it

The benchmark script is part of the open-source repository. Clone it and run:

pnpm --filter server exec tsx scripts/bench-proxy-overhead.ts

The script is at apps/server/scripts/bench-proxy-overhead.ts. It prints JSON with the percentiles above. Your absolute numbers will vary with hardware; the order of magnitude, microseconds, should not.

Measured in production too

Every request Spanlens logs carries a proxy_overhead_ms value, so the overhead is observed on live traffic rather than only in a benchmark. If you self-host, the same field is available on your own requests. Read more about the request pipeline in the reliability documentation and the agent tracing overview.