Connecting AI tools

The proxy is a drop-in OpenAI endpoint, so any tool that speaks the OpenAI API can use it — point it at http://localhost:9376 (or wherever you've configured it to listen) instead of api.openai.com, and it just works. The proxy must be running first (zs-proxy proxy start) — connect reads the live endpoint and model list, and tells you to start it if it isn't up.

The connect command

zs-proxy connect                          # detect installed tools → confirm → configure
zs-proxy connect opencode                 # configure just one tool
zs-proxy connect opencode --model qwen3-coder   # ...with a specific default model
zs-proxy connect --all                    # configure every detected tool, no prompts
zs-proxy connect --list                   # show what would be detected/configured — no writes
zs-proxy connect opencode --print         # render the merged config to stdout — no writes

For each tool, connect merges a zerosignal provider into the tool's existing config — your other settings (and comments, for JSONC configs like opencode's) are preserved, the original file is backed up to <file>.bak, and the write is atomic. Re-running it is safe and idempotent.

Flag	What it does
[tool] (positional)	Configure just this tool instead of every detected one.
--all	Configure every detected tool, non-interactively.
--list	List detected tools and their config paths; writes nothing.
--print / --dry-run	Render the resulting config to stdout; writes nothing.
-y, --yes	Skip confirmation prompts.
--model <id>	Default model id to set in the tool's config.
--config-path <path>	Override the target config file (only with a single named tool).
--url <url>	Advanced: point the tool at a different OpenAI-compatible base URL instead of the local proxy (skips the running check).

Supported tools

Tool	connect name
opencode	opencode
OpenClaw	openclaw
Pi	pi
Hermes	hermes
Aider	aider
Continue	continue
Codex CLI	codex
Anything else	generic — prints the OPENAI_BASE_URL / OPENAI_API_KEY env vars to set yourself

No API key is required — admission to the network comes from your wallet's on-chain seal, not a static key — so connect writes a harmless placeholder key wherever a tool insists on a non-empty one.

Manual setup

For a tool that isn't in the list, or any OpenAI SDK / script, point it at:

• Base URL: http://localhost:9376/v1 (or your configured server.listen — see Configuration)
• API key: anything non-empty; the proxy ignores it

export OPENAI_BASE_URL="http://localhost:9376/v1"
export OPENAI_API_KEY="not-checked"

info

For popular GUI apps — SillyTavern, Open WebUI, LibreChat, Chatbox, Cherry Studio — there are screen-by-screen walkthroughs in How-to guides.

What the proxy serves

Endpoint	Notes
POST /v1/chat/completions	Standard chat completions, streaming or buffered. Prefer streaming — see below.
POST /v1/responses	The Responses API, streaming or buffered. Prefer streaming — see below.
POST /v1/completions	Legacy text-completion shape, translated to a chat request under the hood. The prompt must be a single string (a one-element string array works too).
POST /v1/images/generations	Image generation.
POST /v1/images/edits	Image edits (multipart upload).
GET /v1/models / GET /v1/models/{id}	The live, on-chain model catalog — the same one the chat app's model picker reads.
GET /v1/zs/operators	The operator directory — per-operator ids, models, and pricing. Where the ids for routing preferences come from.
GET /v1/zs/details	The network's aggregate capability and pricing summary (union across operators, no per-operator ids).
GET /healthz	Liveness probe — handy for scripts and supervisors.

Whichever tool you connect, requests must name a concrete model id from /v1/models — the chat app's "Auto" model lane is an app feature, not a proxy one. What the proxy does pick for you is the operator serving that model — see Models & operators.

Response length

Most tools send a max_tokens (or max_completion_tokens, or max_output_tokens) taken from their own built-in model card. Operators advertise their own output ceiling per model, and the two often disagree — a tool's card may say 65,536 for a model every operator serving it caps at 32,768.

That's fine. max_tokens is an upper bound, not a demand, so the proxy lowers an over-large value to the highest ceiling advertised for that model, which leaves the operators offering it eligible to serve you. You still get at most what you asked for, and the escrow reserve shrinks to match, so it costs you less rather than more. If any operator serving the model advertises no ceiling at all, your value is passed through untouched — that operator can honor it.

Only operators you could actually be routed to are counted, so a staging node or one your routing preferences exclude never decides your ceiling.

When the proxy does lower your value, the response carries X-Zs-Max-Output-Clamped with the ceiling it applied. That's worth checking if a reply stops earlier than your tool expected: without it, a reply cut off at the lowered ceiling looks the same as the model simply deciding to stop.

Send nothing and the proxy picks a ceiling sized to whichever operator it routes you to (see fallback_max_output_tokens in Configuration). Setting a smaller value than you need is still the best way to keep replies — and cost — tight.

Prefer streaming

Set stream: true on chat and responses requests whenever your tool lets you. Most tools already default to it; some agent frameworks and one-shot scripts don't, and that's where trouble shows up.

A buffered (non-streaming) request sends nothing over the connection until the model has finished generating the entire reply. Your request crosses the open internet on its way to an operator, and any layer in between — a CDN, a load balancer, a reverse proxy — can have an idle timeout of its own. Sixty seconds is a very common default. If the reply takes longer than that to generate, one of those layers gives up on a connection it sees as silent and answers 504 instead, even though the operator was working normally the whole time.

Streaming keeps bytes flowing: tokens as they're produced, plus an invisible keepalive during quiet stretches such as a long reasoning pass. Nothing in the path ever sees the connection go idle, so the timeout never fires. You also see the reply as it arrives instead of waiting for all of it.

Buffered requests are still supported and are fine for short replies. The longer the expected output — big context, reasoning models, agent turns that plan before answering — the more streaming matters.

Choosing where a request goes

Most tools never need to think about this — the proxy routes the same way the chat app does by default. If a tool (or your own code) needs finer control over which operator handles a request — pinning to one, setting a price ceiling, requiring tool support, or trading latency for cost — that's the provider field on the request body. See Proxy routing preferences.

Seeing what you paid

Every response carries its cost. For a non-streaming response, look at the X-Zs-Inference-Amount and related X-Zs-* response headers. For a streaming response, set the header X-Zs-Usage-Frame: 1 on your request and the proxy adds an event: zs.usage frame to the stream with the same breakdown as JSON — most tools that don't know about it simply ignore an SSE event they don't recognize, so it's safe to always set. See Pricing for what the numbers mean.

Troubleshooting

context_length_exceeded on a conversation that isn't long

A 400 like this one:

no operator serves <model> with input_tokens=36576 max_output_tokens=65536
  operators advertising this model:
  - op 1: max_output_tokens=32768 (over by 32768)

names the limit each candidate operator blew out. Read the per-operator lines rather than the error code — despite the name, this fires whenever no operator fits the request, and the two causes look different:

• context_window=N (request needs M) — the real thing. Your conversation plus the reserved output doesn't fit the model's window. Start a new conversation, trim the history, or pick a model with a larger window.

• max_output_tokens=N (over by M) — your tool asked for more output than that operator caps at. The proxy normally lowers this for you (see Response length), so seeing it as the only reason every operator was dropped means the lowering didn't apply. Either set a smaller max response length in your tool, or check the two things that hold it back:

  • an operator serving this model advertises no ceiling, which passes your value through untouched by design — the max_output_tokens= line will be missing from that operator's entry;
  • you set a provider.max_price ceiling (Routing preferences) that rules out the operators with the larger ceilings.

  If neither applies, check zs-proxy version — lowering arrived in a later release than the rest of this page describes.

504 errors, usually after about a minute

An error like this one:

504 operator returned HTTP 504 with a body this proxy could not parse as an
OpenAI error (content-type "text/html")

means something between the proxy and the operator gave up on the connection — a CDN or load balancer, not the operator's software, which is why the body is an HTML error page rather than a normal API error. The giveaway is the timing: these arrive at a round number, typically right around 60 seconds.

Almost always this is a buffered request that ran long. See Prefer streaming — setting stream: true fixes it, and is the first thing to try.

Worth knowing about retries, because the behavior differs on either side of a line:

• While the proxy is still setting up a request, it retries and reroutes for you — different network paths, and other operators serving the same model. Most transient trouble is absorbed here and you never see it.
• Once the request is paid for and sent, it goes out exactly once. There's no retry and no failover, because the payment is already committed on-chain. A 504 at this stage reaches your tool as-is.

So repeated attempts in your logs are usually your own tool's retry logic (the OpenAI SDKs retry 5xx responses by default), not the proxy's. Each of those attempts is a separate paid request.

No money is lost on a failed request: what was reserved on-chain is returned automatically once the ticket expires, without you doing anything.

If you're already streaming and still seeing 504s from the same operator, that operator's front door is genuinely unhealthy. Retrying will normally land you on a different one; to rule it out for good, add its id to ignore in routing preferences.

What's next

• How-to guides — per-tool setup for opencode, Codex CLI, Aider, and Continue, plus the GUI chat apps (SillyTavern, Open WebUI, LibreChat, Chatbox, Cherry Studio).
• Wallet & funding — keep the proxy funded.
• Configuration — change the listen address, network, or spend caps.
• Proxy CLI reference — every connect flag and the rest of the command surface.