Connect llama-swap with Tailcat

Tailcat provides end-to-end encrypted connections without a Tailscale account or control plane. A Tailcat connection token is case-sensitive and acts like a capability: anyone who has it can attempt to reach the server. Store and share it as carefully as an API credential.

Create stable keys

Install the tailcat CLI, then create a persistent server identity with a fixed relay region. Fixing the region keeps its connection token stable across restarts:

tailcat genkey --key=/path/to/server.private.json --fixed-region

To retrieve the public key for an existing client identity, use the root --key option before printpub:

tailcat --key=/path/to/client.private.json printpub

This prints the full nodekey:... public key. Put that exact value in the server's tailcat.allow list.

Use tailcat genkey --client --key=/path/to/client.private.json only when intentionally creating or rotating a client private key. After a rotation, update tailcat.allow and each affected peer's tailcatKey value before using the new identity.

tailcat:
  allow:
    - nodekey:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
  models:
    - chat
    - coding-model # a selectors entry
    - gpu-box/embeddings
  admin: false
  debug: false

Start llama-swap with both the configuration and the persistent server key:

llama-swap -config /path/to/config.yaml \
  -listen-tailcat /path/to/server.private.json

-listen-tailcat requires a valid Tailcat server PrivateKey JSON file and a non-empty tailcat.models list. Use models: ["*"] explicitly to expose all callable local and peer IDs.

tailcat.models accepts any public model ID: a model's own ID, one of its aliases, a selectors entry, a profile pin, or a peer's peerID/model name. Listing a selector exposes it the same way as a regular model ID, and requests still resolve through the selector's normal routing logic.

allow denies by default. llama-swap checks it on every HTTP request, using the node key that Tailcat authenticated for the connection. A client whose key is not listed can still complete the Tailcat handshake (it needs the connection token for that), but every request it sends gets 403 Forbidden. An empty or missing allow denies every client, and llama-swap logs a warning at startup and on reload when that happens.

To let anyone holding the connection token in, including ephemeral clients, use the wildcard:

tailcat:
  allow: ["*"]
  models: [chat]

An explicit list of keys is strongly recommended for a persistent address. If a client you expect to work gets 403, check that its printpub output is in allow exactly, and that it is using its saved key rather than an ephemeral one.

With admin: false (the default), Tailcat serves only /health, filtered model listings, and allowlisted model-dispatched inference routes. Set admin: true only when remote callers also need the UI, /api, logs, metrics, operations, or upstream passthrough. The model allowlist still applies in admin mode.

Set debug: true to include Tailcat transport diagnostics in the proxy log. It defaults to false to keep routine proxy logs concise.

Configuration reloads (-watch-config or SIGHUP) apply changes to allow, models, admin, and debug without restarting the listener, so connected clients stay connected and the token does not change. A key removed from allow loses access on its next request, even over an open connection. Only the listener key (-listen-tailcat) requires restarting llama-swap.

Find the server token

When the listener starts, llama-swap writes the connection token to its console log. Copy the value after the colon; it is case-sensitive:

[INFO] Tailcat listening on virtual TCP port 80: tcREPLACE_WITH_CONNECTION_TOKEN

You can also open the Tailcat page in the llama-swap UI and copy the token shown under Tailcat connection token. The page is available only while the listener is running.

If llama-swap configures apiKeys, Tailcat callers must also send the normal header, for example Authorization: Bearer <key> or x-api-key: <key>. Tailcat node authorization and HTTP API-key authentication are independent.

Connect a client

Use the token from the console log or Tailcat page. An ephemeral client works only when the server's allow is "*":

tailcat socks <token> curl http://server.tailcat/v1/models

Otherwise use a stable client key whose public key is in the server's allow list:

tailcat --key=/path/to/client.private.json socks <token> curl http://server.tailcat/v1/models

Add API-key headers to the curl command when required:

tailcat --key=/path/to/client.private.json socks <token> \
  curl -H 'Authorization: Bearer <api-key>' http://server.tailcat/v1/models

Tailcat uses virtual TCP port 80 for llama-swap; do not add another port to the connection URL or the server.tailcat hostname.

An unloaded remote model may take longer than a minute to send response headers. Tailcat peers wait up to 300 seconds by default; other peers wait 60. If a cold start takes longer, set peers.<name>.timeouts.responseHeader in seconds. A 502 with timeout awaiting response headers means this wait expired while the remote server was still loading or preparing the response.

peers:
  friend:
    proxy: tailcat://tcREPLACE_WITH_CONNECTION_TOKEN
    models: [large-model]
    timeouts:
      responseHeader: 600

List models with curl

To list the models exposed by a llama-swap server's Tailcat listener, run curl through Tailcat's SOCKS wrapper. Replace the token with the exact value shown on the Tailcat page; it is case-sensitive. This ephemeral example only works when the server's allow is "*".

tailcat socks tcREPLACE_WITH_CONNECTION_TOKEN \
  curl --fail --silent http://server.tailcat/v1/models

With a saved client key listed in the server's allow, and HTTP API keys, include the normal llama-swap authentication header:

tailcat --key=/path/to/client.private.json socks tcREPLACE_WITH_CONNECTION_TOKEN \
  curl --fail --silent \
    -H 'Authorization: Bearer <api-key>' \
    http://server.tailcat/v1/models

Configure a Tailcat peer

Use the destination token verbatim after tailcat://. Tokens are case-sensitive; do not lowercase them.

peers:
  gpu-box:
    proxy: tailcat://tcREPLACE_WITH_CONNECTION_TOKEN
    tailcatKey: /path/to/client.private.json
    models: [chat, embeddings]

Omit tailcatKey (or set it to ephemeral) to create one client identity for that peer for the lifetime of the llama-swap process. That only works when the destination's allow is "*"; otherwise use a saved client key and add its printpub output to the destination's allow list.

Avoid cyclic peer graphs. If host A routes a model to B and B routes that same request back to A, requests loop until they fail and may repeatedly start or hold models.