Features

Complete log of every LLM call routed through Spanlens, model, tokens, cost, latency, full request/response bodies.

Save named filter combinations on the /requests page and restore them instantly from a dropdown on future visits.

End-user attribution and per-user analytics for LLM usage. Tag requests with x-spanlens-user and see who is spending what.

Agent tracing with nested span trees. See exactly where time goes when your LLM agent calls five tools in sequence.

Version-controlled prompt templates with typed variables and real-data A/B comparison. See latency, cost, and error rate for every version you ship.

Interactive console inside the Prompts tab, select a version, set model, temperature, and variables, then run it immediately and see cost and token counts.

Route live production traffic across two prompt versions and measure latency, cost, and error rate with statistical significance.

LLM-as-judge evaluation, automatically score production responses on a 0..1 scale and quantify quality per prompt version.

Reusable (input, expected_output) test sets. Use in Evals and Experiments instead of pulling from live production traffic.

Offline side-by-side comparison, run a dataset against two prompt versions and compare outputs, scores, and cost without touching production traffic.

Human star-rating for production responses. Pearson r correlation with LLM judge scores makes judge reliability visible at a glance.

Automatic PII detection and prompt-injection scanning on every LLM request and response, with optional blocking mode and real-time alert emails.

3-sigma statistical anomaly detection on latency, cost, and error rate per (provider, model) bucket. No ML, no configuration.

Threshold-based alert rules for budget, error rate, and p95 latency. Delivered via email (Resend), Slack, or Discord webhooks.

Receive Spanlens events (request created, trace completed, alert triggered) as real-time HTTP POST payloads on your own server.

Accurate per-request USD cost computed from provider token prices. Handles dated model variants via longest-prefix matching.

Model recommendations based on your real token distribution. Cheaper substitutes with estimated monthly savings, confidence tiers, and email alerts.

How Spanlens charges you. Plan quotas, overage rates per 100K requests, the hard cap that stops runaway spend, and what an invoice looks like.

Scope your traffic into projects (dev / staging / prod, per-service). Each Spanlens key (sl_live_…) carries its own pool of encrypted provider keys.

Publish a public read-only render of any trace or request via a share link, with redaction presets, view counts, and one-click revoke.

How Spanlens stores and protects your AI provider keys. AES-256-GCM encryption at rest, decrypted only in memory during proxy forwarding.

Multi-user workspaces with admin, editor, and viewer roles, email invitations with auto-accept, and an audit log of every membership event.

Chronological record of every organization-level change: API keys, provider keys, member invitations, role changes, and billing plan switches.

Download request logs, traces, anomalies, and security flags as CSV, JSONL, or JSON. Streamed exports handle millions of rows.