TokenSeq Gateway · The engine

The gateway behind TokenSeq — in your domain.

Everything TokenSeq Cloud runs on is a product you can deploy yourself: the edge network, the routing, the multimodal services, the billing ledger and the control plane. Your channels, your accounts, your keys — nothing leaves your environment.

Five layers, one job each

Access (SDKs, agents and CLI tools, downstream relays, in-house apps) → Edge (TLS, declarative routing, session affinity, health probing) → Data plane (text, image, video services with billing and audit) → Adaptation (one pure-function adapter per upstream protocol) → Supply (a channel = one upstream account). A control plane — admin, console, single configuration store — sits alongside.

01 · ACCESS 02 · EDGE 03 · DATA PLANE 04 · ADAPTATION 05 · SUPPLY Official SDKsonly base_url changes Agents & CLI toolscoding assistants In-house applicationsenterprise apps Downstream relayspartners · white-label Self-built edge network Nodes in three regions take client traffic. Each node pulls, renders, validates, then swaps — a bad config never takes a node down. TLS · auto-cert declarative routing session affinity health probing connection reuse Asia Europe US Text service Scheduling · session pinning Metering · usage ledger streamed, unbuffered Image service Sync or async orchestration Own store · refund on failure quote → deduct → upstream Video service Task-based · submit and poll Own media store · spec gate never loses a task Protocol adaptation One pure-function adapter per upstream protocol — composed, not inherited; each testable on its own. normalised in · aligned field-by-field out · unsupported parameters stated, not faked Upstream channel pool A channel is one upstream account: protocol + address + one key. Admission is a dual switch (channel and model); traffic routes around anything off. Demoted on failure, never silently disabled. upstream-01 upstream-02 upstream-03 demoted · · · text · image · video models CONTROL PLANE Single configuration store Routes · certificates · channels · models · prices · keys · groups · ledger the only authority — data plane reads it live, no restarts Admin console channels · lifecycle models · pricing tiers evaluation centre monitoring · auto-suspend billing · reconciliation sites · brands · domains Customer console dashboard keys · group quotas usage · statements balance · invoices integration docs white-label per brand Model verification agent Blind capability tests · upstream provenance · fingerprints. Runs at onboarding and on schedule; failing channels leave service. not tested ≠ tested clean Every admin action is also an MCP tool One central token; agents and operators walk the same path. Self-healing A keeper restarts stuck processes, decoupled from the database — a database outage never affects liveness. live config CLOUD Our nodes, shared edge and upstream pool. Subscription. Nothing to operate. multi-region · pay per token SELF-HOSTED The same software in your environment. Your channels, accounts, data and keys never leave it. data plane deployed · control plane implemented per project · managed ops or hand-over
TokenSeq Gateway · architecture · redrawn

Self-built edge network

Edge nodes in Asia, Europe and the US take client traffic; the configuration store is the single authority. Each node periodically pulls, renders, validates, then swaps — a bad configuration can never take a node down. Certificates are issued and renewed automatically. Nodes forward independently of the configuration store: if it is slow or unavailable, traffic still flows.

Cache-friendly session affinity

Upstreams cache conversation prefixes. If routing drifts, that cache is wasted. The gateway pins a conversation to one path for its whole life — so cache hits and load balancing are achieved together, and first-token latency and cost both improve.

Adaptive routing

A short chain of gates decides each request: enable switches at channel and model level, exact specification match, and avoidance of recently failed channels. Every hop is recorded with status, reason and time. Failed channels are demoted, never disabled: slower is acceptable; no route is not.

Handling bursts

Each layer absorbs its own share: the edge keeps warm connections and fails fast; the process layer scales with the resources it is actually given; the tenant layer enforces concurrency and rate limits before any request body is read. Streaming is passed through unbuffered.

Multimodal services

  • ImageQuote → deduct → call upstream, in that order; one order, one ledger line. Upstream failures are translated once into stable error codes that decide refund, retry and what the client sees. Generated images are served from our own store at stable addresses.
  • VideoWhat we promise, how we judge, and what a channel currently accepts are kept as three separate specifications — so the published spec is never quietly wider than reality. Tasks are created and billed before any upstream call; every background failure refunds with a stable reason code.

Control plane

Per-modality management (channels, pricing, statistics, tiers, tests) with a symmetric structure; channel lifecycle from onboarding through blind evaluation, listing, scheduled inspection and automatic suspension / recovery; every administrative action is also an MCP tool, so agents and operators walk the same path.

Deployment

Cloud

Our nodes, shared edge and upstream pool, subscription.

Self-hosted

Data plane in your environment, control plane implemented per project; optional managed operations or hand-over after training.

Both support customisation of pages and functions.