xtokenhub

# XTokenHub XTokenHub is a self-hosted aggregation gateway for LLM provider APIs. It collects the API keys you already hold across providers into a single channel-based panel, exposes standardized protocol endpoints to your clients, and reports token usage and cache hit rate live. The stated design principle is to pass traffic through in the provider's native protocol whenever possible, treating conversion as a fallback. ## Core capabilities - **Channel-based key management** — one entry per provider endpoint, with availability probing, enable/disable switching, and balance queries where a provider exposes them. - **Model grouping and routing** — each channel carries its own model list (pulled from upstream on demand), and all upstreams are merged into a single model listing endpoint. Routing uses priority plus weighted random selection, with automatic failover across channels serving the same model. - **Usage insight** — a dashboard reports request counts, token usage, cache hit rate and average latency, aggregated by model, channel or caller key, with a GitHub-style activity heatmap and daily trend charts pushed over WebSocket. - **Protocol conversion** — the gateway simultaneously exposes OpenAI-style chat and responses endpoints and an Anthropic-style messages endpoint. Requests are converted only when the inbound and upstream protocols differ; matching protocols are forwarded without rewriting, which the project says preserves tool calls and multimodal payloads. - **Gateway keys and per-caller stats** — client-facing keys are issued for different callers, and request/token totals are aggregated per key. - **Single-binary self-hosting** — the frontend is embedded in the Go binary, so a build produces one static artifact (no CGO) that can be copied to a Linux or macOS machine. ## How routing and metering work The README describes a four-step flow: 1. Create a channel with provider base URL, API key and model list. The API style (Bearer for OpenAI-compatible, x-api-key for Anthropic-compatible) is auto-detected from the base URL and can be overridden. Several base-URL mounting forms are supported, including bare domains, /v1 suffixes, sub-path mounts such as /anthropic, and versioned mounts. 2. A native-protocol probe sends a minimal request to each protocol endpoint; a 2xx response marks that protocol as native to the channel, which can also be corrected manually. 3. Incoming requests are filtered to enabled channels serving the model, native channels are preferred, and converted channels are used only as a fallback. Selection is priority-ascending with weighted randomness, and failures such as network errors, 401/403/408/429 or 5xx trigger failover. 4. Usage is parsed from the upstream response where reported (with streaming usage options added automatically); when upstream reports nothing, a local heuristic estimate is used. Cache hit rate comes from the provider's cached-token fields, and every request is written to a log table and pushed to the UI. ## Dashboard and API surface The admin API covers channels, gateway keys, request logs with retention cleanup, and a set of statistics endpoints (summary, daily trend, by-model, by-channel, by-key, lifetime totals, and trend-by-model), plus a health endpoint and a WebSocket endpoint for live events. Gateway endpoints include a merged model list and the chat, responses and messages routes. Caller authentication accepts either a Bearer token or an x-api-key header and can be switched off by configuration. ## Configuration and operations Configuration precedence is environment variables, then a YAML file, then built-in defaults. Data is stored in SQLite in WAL mode with a single writer connection. Request logs grow without bound by default, so a retention job deletes rows older than a configurable number of days, with settings for cycle interval, batch size and an optional VACUUM. The project notes that the SQLite file does not shrink automatically after deletion. ## Testing Unit tests live in a mirrored external test package layout and cover configuration loading, in-memory SQLite repositories, the event bus, WebSocket behaviour, provider probing and conversion via fake upstreams, gateway selection and stats persistence, and end-to-end handler/router paths. The README reports a full race-enabled test run and 87.6% statement coverage, plus frontend tests for WebSocket reconnection and data transforms. ## Known limitations stated by the project - The conversion path handles text chat only; tool calls, multimodal and cache-control payloads need native passthrough channels. - Balance queries currently cover DeepSeek only, because other providers' balance APIs are undocumented, require expiring cookie auth, or are not public. - Local token estimation is heuristic and used only as a fallback. - The admin API has no login authentication and is intended for self-hosted intranet use with external network isolation; keys are stored in plaintext. - Cache hit rate and per-key aggregation rely on request-log snapshots, so a deleted key's historical usage remains under its name. ## Companion app and license A separate SwiftUI menu bar app consumes the same admin API and WebSocket without requiring backend changes. XTokenHub is released under the MIT License.