Reliability

The Spanlens proxy sits in the critical path of your LLM calls. This page covers what we guarantee, what degrades when, and how to detect each failure mode from your side without waiting for our status page.

What the proxy is on the critical path for

The proxy passes your request to OpenAI / Anthropic / Gemini and streams the response back. The log row is written afterthe response leaves for the client, via Vercel's waitUntil(). Concretely:

Critical for your user-facing latency: proxy auth, provider key decrypt, upstream fetch, stream pump back to your client.
Not critical for your user: writing the log row, computing cost, parsing usage. These happen after the bytes are on the wire.

So even when the database is unhappy, your application keeps returning responses to end users. The visible symptom is missing rows in /requests, not failed API calls.

Failure modes and what happens

Failure	End-user impact	Dashboard impact	Auto-recovery
Upstream provider 5xx (OpenAI down)	Same as direct call: the SDK surfaces the 5xx.	Request still logged with the 5xx status_code.	Provider SDKs retry by default.
Provider 429 rate limit	Same as direct call: 429 returned.	Logged with status_code=429.	Provider SDKs retry with backoff.
Stream exceeds 290s budget	Stream closes gracefully; client sees an end-of-stream without sentinel.	Logged with `truncated: true`, partial response body kept.	Use `stream: true` with smaller `max_tokens`, or self-host (no Vercel 300s limit).
Non-streaming > 35s	504 returned.	Logged with status_code=504.	Switch to streaming; first byte still arrives in ~200ms.
The log insert fails (pooler saturated, statement timeout, schema behind the code)	None. Response already streamed.	Row queued in `requests_fallback`, so it shows up late.	Cron drains the queue every 5 min once the insert path is healthy.
Postgres unreachable	Warm instances keep authenticating from their 30s key cache. After that, new calls fail closed.	Dashboard reads fail. /api/v1/* returns 5xx.	Supabase managed availability (cloud) or your HA setup (self-host).
Postgres unreachable long enough for the queue to fill	None for calls that already went through.	Rows older than 7 days in the queue are dropped.	None. Restore the database before the TTL expires.

The fallback queue

The log row goes in over the pooled connection. When that insert throws, the logger catches it and writes the row into a table named requests_fallback instead, over PostgREST. Same database, different route in, which is what makes it useful: the two paths fail for different reasons. An exhausted pooler, a statement timeout, or a column the deployed schema does not have yet stops the direct insert while PostgREST keeps working.

A cron route, GET /cron/replay-fallback, runs every 5 minutes, pulls up to 50 rows in FIFO order, and inserts them into requests as one statement. Rows that land are deleted from the queue. Rows that do not get their retry_count bumped and stay put.

Expiry: rows are dropped after 7 days or 100 retries, whichever comes first.
Ordering: FIFO by created_at, not strict per-organization.
Duplicates: the replay insert ends in ON CONFLICT (created_at, id) DO NOTHING. If a batch lands but the queue delete blips, the next run re-inserts nothing and the queue still drains. A replayed row cannot double-count against your cost or quota.

Source: apps/server/src/lib/fallback-replay.ts and apps/server/src/lib/logger.ts.

Health endpoints

Three endpoints, three depths. All are public; no auth required.

Endpoint	Purpose	Returns
`GET /health`	Process liveness. Cheap; safe to poll every 10s.	`200` always (if process is up).
`GET /health/ready`	Readiness. Pings the database both ways it is reached, PostgREST and the pooled connection, plus the rate-limit store. Cheap enough for a 30s container healthcheck.	`200` if all healthy, `503` if a dependency is unreachable.
`GET /health/deep`	Component view. Adds the fallback queue depth, the slowest cron run in 24h, and webhook backlog counts. Meant for a 5 minute probe, not a 30 second one.	`200` if the pooled connection answers, `503` if it does not.

Sample response from /health/deep:

json

{
  "status": "ok",
  "timestamp": "2026-08-20T03:14:22.000Z",
  "version": "a1b2c3d",
  "postgresPool": { "ok": true, "latencyMs": 12, "probedInMs": 41 },
  "fallback": { "queue": 0 },
  "crons": { "max_runtime_ms": 1840 },
  "webhooks": { "backlog_count": 0, "dlq_count": 0 }
}

A null anywhere in there means the lookup itself failed, not that the number is zero. Worth distinguishing when you triage.

Monitor these from your own observability stack (Better Stack, UptimeRobot, Pingdom, Sentry Crons, anything that supports HTTP probes). We recommend two probes:

GET /health every 60s, alert if 2 consecutive failures.
GET /health/deep every 5 min, alert on 503 OR if fallback.queue > 1000 (queue not draining).

Status page

Public status: status.spanlens.io (when the service is down our marketing pages may be down too; bookmark this URL directly). The page tracks the proxy (liveness + deep health) and the dashboard independently, and posts incident updates within 15 minutes of first detection.

Subscribe by email or RSS directly on the status page (Subscribe button, top right). For real-time pages on critical work, set up your own probe against /health/deep as well, the status page lags real detection by minutes.

What you should do client-side

Retry on 5xx and 429 from the proxy

The official OpenAI / Anthropic SDKs already do this. If you wrote a raw HTTP client, add at least 2 retries with exponential backoff on 5xx and 429.

Do not retry on 401 / 403 / 400

401 means your Spanlens key is wrong. 403 means the key lacks permission (e.g. wrong project). 400 typically means missing provider key for the requested provider. None of these benefit from a retry; surface to the user.

Tolerate missing logs

Your application code should not block waiting for a Spanlens log to appear. A request returns to the user before the log is written; downstream features that depend on the log (e.g. real-time cost display) should poll with a small delay or accept eventual consistency.

Self-host if data residency matters more than ops effort

Self-hosting removes our cloud as a failure mode entirely. You take on running one Postgres database, and the latency budget shifts under your control. See Self-hosting.

Incident response checklist

If you see missing rows in /requests:

Check status.spanlens.io.
curl https://api.spanlens.io/health/deep. If fallback.queue > 0, the rows are queued and will replay automatically; no action needed.
Verify your application is hitting the proxy (Network tab in the browser, or your APM trace). If requests are not reaching api.spanlens.io, the gap is on your side.
If status page is green AND /health/deep returns 200 AND your requests are reaching us, email support@spanlens.io with the request id (x-spanlens-request-id response header) and we will trace the missing row.

SLOs (cloud, hobby and paid)

Metric	Target	How measured
Proxy availability	99.9% monthly	`GET /health` success rate from external probe.
Logging completeness	99.95% of calls eventually logged	Compared against upstream provider invoice token counts daily.
Proxy overhead (p95)	< 50 ms	`proxy_overhead_ms` column on every Request row.
Fallback drain (p95)	< 15 min after the insert path recovers	Time between queue size peak and queue size 0.

Targets above are for the cloud product. Self-host SLOs are whatever you achieve; the code is the same.

Next: scaling for high throughput.