llama-swap
Browse the docs

Docs / API integration

Rewriting requests with filters

Use stripParams, setParams and setParamsByID while preserving the protected model parameter.

Filters modify the JSON request body before it reaches the upstream server. They exist on models and on peers, with the same three settings.

The request's model parameter is protected: stripParams cannot remove it, and setParams cannot override it.

stripParams — remove parameters

A comma-separated list of JSON keys to delete from the request body:

models:
  qwen-coder:
    filters:
      stripParams: "temperature, top_p, top_k"

Use it when the model has been tuned with specific sampling settings in cmd and you don't want clients overriding them, or when a peer rejects parameters it doesn't understand.

The model parameter can never be stripped. Stick to sampling parameters — stripping structural fields will break requests.

setParams — force values

models:
  qwen-coder:
    filters:
      setParams:
        temperature: 0.7
        top_p: 0.9

Runs on every request to the model and overrides whatever the client sent. Values may be strings, numbers, booleans, arrays or objects. model and other protected params cannot be overridden.

A key ending in ? is set-if-undefined instead: it applies only when the body does not already carry that parameter, so a client that sends its own value wins. Works in setParams and setParamsByID alike — see guides/api-integration/set-if-undefined.

      setParams:
        temperature: 0.7    # forced
        max_tokens?: 4096   # only if the request didn't set max_tokens

For peers this is how you inject provider-specific settings:

peers:
  openrouter:
    proxy: https://openrouter.ai/api
    apiKey: ${env.OPENROUTER_API_KEY}
    models: [z-ai/glm-4.7]
    filters:
      setParams:
        provider:
          data_collection: "deny"
          zdr: true

setParamsByID — variants without reloading

This is the interesting one. It maps a model ID to a set of parameters, and automatically creates a model alias for each key. One loaded model then serves several API-visible variants:

models:
  gpt-oss-120b:
    filters:
      setParamsByID:
        "${MODEL_ID}":
          chat_template_kwargs: { reasoning_effort: medium }
        "${MODEL_ID}:high":
          chat_template_kwargs: { reasoning_effort: high }
        "${MODEL_ID}:low":
          chat_template_kwargs: { reasoning_effort: low }

Clients can now request gpt-oss-120b, gpt-oss-120b:high or gpt-oss-120b:low. All three hit the same running process — no swap, no reload — with different parameters applied.

setParamsByID runs after setParams, so it wins on any key they share.

Order of operations

  1. Profile pins resolve the requested ID
  2. Aliases resolve
  3. stripParams removes keys
  4. setParams applies
  5. setParamsByID applies, overriding setParams
  6. The request is proxied

Filters apply like a pipe: stripParams | setParams | setParamsByID. A set-if-undefined key (key?) checks the body at its own stage, so a stripped key counts as undefined and a key an earlier stage set counts as defined.

When not to use filters

If you want a genuinely different configuration — different context size, different GPU layers — that is a separate model entry, not a filter. Filters only rewrite the request body; they cannot change how the server was started.

  • guides/api-integration/set-if-undefined — set a parameter only if the request didn't
  • reference/config/models — the annotated filters block
  • reference/config/peers — peer filters