Skip to main content

Relays

Every operator node is also a relay. By default, a user's request doesn't go straight to the node that will serve it. It first goes through one relay hop: some other operator's node, which forwards the sealed request to its target without being able to read it.

This keeps a user's identity separate from their prompt. The node that runs the model sees the relay's address instead of the user's, and the relay sees a sealed payload it can't decrypt. Every node must relay.

What your node does as a relay​

When your node is picked as a relay for someone else's request, it:

  1. Receives a request marked with a target operator and node, and the path to forward to (reserve, inference, etc.).
  2. Forwards the request to the target node's endpoint.
  3. Streams the target's response back to the client.

The payload your node forwards is sealed to the target node's encryption key, not yours. You can't decrypt it; you're a transport hop.

What your node does as a target​

When your node is the target (the one serving the request), the request normally reaches you from a relay, not from the user:

  • The connection's source address is the relay's, so you don't learn the user's network address. A user who turns relaying off connects to you directly.
  • Everything else (reserve, sealed prompt, receipt, settlement) works as in the payment flow.

How relays get chosen​

The client picks a fresh relay for each request:

  • Never the target or a sibling of it. Every node belonging to the target's operator is excluded, so a node can't relay for itself or for another node of the same operator.
  • Owner diversity is a hard rule. A relay sharing the target's owner address is excluded. Relay and target are always under two distinct owner keys, so no single operator holds both a user's network address and their prompt. Joining the two takes two owners colluding, or one party registering operators under several owner keys.
  • Only compatible nodes are eligible. A relay must speak a compatible protocol version and, best-effort, sit on a different /16 subnet than the target.
  • Reachable, faster relays are preferred. Selection favors relays measured as responsive, weighted by latency, with enough randomness that no single relay dominates. If no measured-reachable relay is eligible, the client falls back to the full eligible pool, so cold, unproven relays still get tried.

A fast, reliable, well-connected node is used as a relay more often, and reliable relaying builds the reachability reputation that keeps your node in rotation.

Reliability and reputation​

The network tracks relay reliability per node. If your node accepts relay traffic but fails to forward it (it's reachable, but can't reach the target), clients down-rank it as a relay for a while and prefer others. If your node is unreachable as a transport endpoint at all, it's demoted faster. Both recover automatically once your node forwards successfully again.

To be a good relay:

  • Stay reachable at your advertised endpoint, with low latency.
  • Forward promptly to targets and stream responses back without buffering the whole reply.
  • Keep your protocol version current, so you stay eligible for both serving and relaying.
  • Let your node's own response headers through, in particular X-Zs-Relay-Hop.

Don't strip the X-Zs-Relay-Hop header​

Your node stamps X-Zs-Relay-Hop: 1 on every response it produces while relaying. The header carries no information about the request, the client, or the target. It proves that your node handled the request, rather than the request dying at your CDN, load balancer, or reverse proxy before it arrived.

When a relayed request fails with a generic gateway error and no protocol error code, the client uses the header to decide who was at fault. If it's present, the client knows your node did its job and looks elsewhere. If the header is missing, the client down-ranks your node, and nothing appears in your logs. So anything in front of your node that strips unknown response headers gets you penalised for failures that weren't yours. Check for:

  • Cloudflare Transform Rules or WAF rules that remove response headers.
  • nginx proxy_hide_header / more_clear_headers with a broad pattern.
  • An API gateway or CDN configured to pass through only an allow-list of response headers — add X-Zs-Relay-Hop to it.

If you serve browser clients, the header must also be readable cross-origin. Your node already lists it in Access-Control-Expose-Headers; if your edge rewrites that header, keep X-Zs-Relay-Hop in the list. A browser treats a header it can't read the same as one that wasn't sent, so stripping it there has the same effect as stripping it outright.

Nodes running an older protocol version don't send this header, and clients don't hold its absence against them. It matters only once your node advertises support for it.

Fail-closed privacy​

info

The privacy guarantee is enforced on the client side, and with relaying on it fails closed: if no eligible relay is available for a target, the client refuses to send directly rather than expose the user's address.