Gateway

Chat Completions

The Gateway's only inference endpoint. OpenAI-compatible request and response, with routing, cost and quota reported in headers.

POSThttps://www.lobstack.ai/api/gateway/v1/chat/completions

Authenticate with an API key as a bearer token. Keys look like lsk_live_<8 hex><48 hex> and need the inference scope. Two legacy credentials are still accepted — an agent's gateway token and the platform agent secret — because rows minted before the agent-VM runtime was retired still authenticate. Nothing issues new ones.

Request body

messagesarrayrequired: Chat messages with a role of system, user, assistant or tool. content may be a string or an OpenAI content-part array; the array form is passed through to the provider untouched. Assistant messages may carry tool_calls, and tool messages a tool_call_id. An empty array is a 400.
modelstringdefault: "auto": A Lobstack model key, or "auto". Omit it and the Gateway uses the calling credential's configured model, falling back to auto. A named model is a ceiling, not a command: a simple request under a flagship model still routes down. See Token Intelligence.
streambooleandefault: false: Return server-sent events instead of a single JSON body. The routing and quota headers are still sent; the cost headers are not, because token counts do not exist yet when the headers flush.
max_tokensinteger: Maximum tokens to generate. Defaults to 4096 for Anthropic and for classic OpenAI-compatible providers, and to 8192 for OpenAI's GPT-5 and o-series models, where it is sent as max_completion_tokens because those models reject max_tokens.
temperaturenumber: Forwarded only when you set it, and only to models that accept it. The Gateway never invents a default. When it drops one it says so in x-lobstack-dropped-params. See Temperature below for the exception.
toolsarray: OpenAI function-tool definitions: { type: "function", function: { name, description?, parameters? } }. Passed straight through on OpenAI-compatible providers and translated to Anthropic's input_schema form for Claude models.
session_idstring: Free-form grouping key. Recorded on the ledger row so a multi-turn conversation can be costed as one thing. Not sent to the provider.

Other OpenAI parameters are ignored, not rejectedThe Gateway reads exactly the fields above off the body. Send top_p, tool_choice, n, stop, response_format or stream_options and the request succeeds with those fields dropped silently. They are not forwarded and no header reports it.

Response

200 OKjson

{
  "id": "chatcmpl-8f2b1e0c-4d5a-4b91-9c3e-6a7f0d2b5c41",
  "object": "chat.completion",
  "created": 1789000000,
  "model": "gemini-3.1-flash-lite",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "Western European Summer Time, UTC+1." },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 14, "completion_tokens": 9, "total_tokens": 23 }
}

id is chatcmpl- followed by the request id, so the completion and its trace row share an identifier. model is the Lobstack key that actually served the request — read it rather than assuming it echoes what you sent. There is always exactly one choice.

usage carries the provider's own token counts, which is what the ledger prices. Every other fact about the request is in the headers, listed in full on Metering & cost.

Streaming

Set "stream": true and the response is text/event-stream with X-Accel-Buffering: no, so an intermediate proxy cannot hold the whole answer and defeat the point. Chunks are OpenAI chat.completion.chunk objects.

The first chunk carries the role and empty content, so a client can render the assistant turn before any text arrives. Then come content deltas, then a chunk with a finish_reason, then a usage-only chunk, then data: [DONE].

Event sequencetext

data: {"id":"chatcmpl-8f2b…","object":"chat.completion.chunk","created":1789000000,
       "model":"gemini-3.1-flash-lite","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {…,"choices":[{"index":0,"delta":{"content":"Western European"},"finish_reason":null}]}

data: {…,"choices":[{"index":0,"delta":{"content":" Summer Time."},"finish_reason":null}]}

data: {…,"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {…,"choices":[],"usage":{"prompt_tokens":14,"completion_tokens":9,"total_tokens":23},"x_lobstack":{"request_id":"2f1c…","served_model":"gpt-5.6-luna","requested_model":"claude-opus-5","routed":true,"cost_usd":0.000017,"savings_usd":0.000352,"priced":true,"baseline_model":"claude-opus-5","baseline_reason":"named","baseline_cost_usd":0.000369}}

data: [DONE]

Usage and cost arrive on the final chunkThe final chunk has an empty choices array and is sent unconditionally — you do not need to ask for it with stream_options.include_usage, which the Gateway does not read anyway. It is the last data event before [DONE], and on a streamed request it is the only place the price appears: the cost, savings, baseline and priced headers are all absent, because the headers are already gone by the time the provider says how many tokens it used. The same figures come back in the body instead, under x_lobstack — including baseline_reason, which says whether the saving is measured against a model you named or against the most expensive one your plan allows. The usage object keeps the exact OpenAI shape, so an SDK that does not know us parses it unchanged and ignores the rest. Metering has the field list.

A failure part-way through a stream cannot change the status code, which is already 200. The Gateway sends an error object into the stream instead, followed by [DONE], and the trace row records the real status. Tokens the provider already produced are metered even if you hang up early: the generator is cancelled and the ledger write still runs.

Mid-stream failurejson

data: {"error":{"message":"provider error 529: overloaded","type":"gateway_error","code":529,
       "request_id":"8f2b1e0c-4d5a-4b91-9c3e-6a7f0d2b5c41"}}

data: [DONE]

Tool calls

Pass tools and the model may answer with tool_calls on the assistant message instead of content. Anthropic models go through a translation layer — tool_use blocks in, OpenAI tool_calls out — so one shape works across every provider. When a model returns tool calls with no text, content is an empty string rather than null.

Assistant turn with a tool calljson

{
  "index": 0,
  "message": {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "id": "toolu_01A9…",
        "type": "function",
        "function": { "name": "get_weather", "arguments": "{\"city\":\"Lisbon\"}" }
      }
    ]
  },
  "finish_reason": "tool_calls"
}

Streamed tool calls arrive as deltas inside delta.tool_calls, each carrying an index; the id and function.name appear on the first delta for that index and function.arguments accumulates across the rest. Assemble by index, not by arrival order.

Send the result back as a message with "role": "tool" and the tool_call_id you were given. Every turn of a tool loop is a separate request, separately scored and separately routed, so a loop can start on a small model and finish on a larger one.

Temperature

The Gateway forwards temperature only when you set it, and only to models that still accept it. Anthropic deprecated the sampling parameters on Opus 4.7 and everything after it, including Opus 5, Sonnet 5 and the Fable line, and the API returns a hard 400 rather than ignoring the field.

When a temperature is dropped for that reason the response carries x-lobstack-dropped-params: temperature. Erroring would fail a request that is otherwise serviceable; ignoring it silently would leave you wondering why sampling has no effect.

Model family	temperature sent?	Header?
Claude Haiku 4.5, Sonnet 4.5, Opus 4.6	Yes, when you set one	—
Claude Opus 5, Sonnet 5, Fable 5.1	No — provider rejects it	x-lobstack-dropped-params
OpenAI GPT-5 and o-series	No — provider only accepts its default	None. This one is silent.
Everything else	Yes, when you set one	—

One case is dropped without a headerOpenAI's GPT-5 and o-series models take only their default temperature, so the adapter omits the field for them. x-lobstack-dropped-params is driven by the Anthropic rule alone, so it is not set in that case. If you need to know whether sampling was applied on those models, assume it was not.

Full examples

Buffered

curl -i https://www.lobstack.ai/api/gateway/v1/chat/completions \
  -H "Authorization: Bearer $LOBSTACK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "max_tokens": 512,
    "messages": [
      { "role": "system", "content": "Answer in one sentence." },
      { "role": "user", "content": "Why did the deploy roll back?" }
    ]
  }'

Streaming

curl -N https://www.lobstack.ai/api/gateway/v1/chat/completions \
  -H "Authorization: Bearer $LOBSTACK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "stream": true,
    "messages": [{ "role": "user", "content": "List three causes of a rollback." }]
  }'

Errors

Every failure is JSON with the same shape, and every one carries the request id, including the ones that fail before authentication finishes.

400 Bad Requestjson

{
  "error": {
    "message": "messages[] is required",
    "type": "validation",
    "code": 400,
    "request_id": "8f2b1e0c-4d5a-4b91-9c3e-6a7f0d2b5c41"
  }
}

type is one of six error classes. Errors & retries covers what causes each one and which are worth retrying.