When an API key with quota or rate limits was deleted while a request was
still in flight, the key counter update matched no rows and returned
ErrAPIKeyNotFound, rolling back the whole billing transaction including the
balance/subscription charge. Skip the key-scoped counters in that case and
keep settling the user and account side as usual.
The create form shows the upstream billing auto-probe toggle for every
API-key platform and enables it by default. TypeSafe was added as an
API-key platform but not to IsUpstreamBillingProbeIdentity, so creating a
TypeSafe account with the default form state fails with
400 UPSTREAM_BILLING_PROBE_ACCOUNT_INVALID ("account is not an API key
account").
TypeSafe accounts store credentials.api_key/base_url like every other
API-key platform, which is all the probe reads. Add TypeSafe to the probe
identity set, and add typesafe.ai to the official API domains so accounts
on the default https://api.typesafe.ai base URL record "unsupported"
without sending the key to a path that cannot exist, matching the other
official providers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
intersectUpstreamModelMetadata falls back to null for every Codex tool
capability field whose value differs across peer accounts. Since
e6d191a83 added service_tiers to codexToolCapabilityFields, a model served
by accounts with different (or partially declared) tier lists ends up with
"service_tiers": null in the generated Codex manifest. Codex decodes
service_tiers as a required array and rejects the whole manifest, so the
client loses model metadata for every model.
Fall back to [] for conflicting service_tiers (do not advertise any tier
that is not shared by all peers), and normalize an explicit null to []
when applying capabilities.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The WS v2 passthrough relay only read output_tokens_details.image_tokens,
so image generation over WebSocket billed image input tokens at the text
input rate and ignored the hosted response.tool_usage.image_gen breakdown
that the HTTP Responses path already merges (extractOpenAIUsageFromJSONBytes
+ mergeHostedImageGenToolUsage).
The relay now reads image input tokens from input_tokens_details or
prompt_tokens_details, fills missing image input/output counters from
response.tool_usage.image_gen (usage values keep precedence), and carries
ImageInputTokens per turn and in the session aggregate into the
OpenAIForwardResult used for billing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Responses refusal arrives as a {"type":"refusal","refusal":...} content
part (streamed as response.refusal.delta), not as output_text. The
Responses → Chat Completions converters only read output_text, and
ResponsesContentPart had no refusal field, so Chat clients received an
empty assistant message when the model refused.
Map refusal parts to choices[0].message.refusal, response.refusal.delta
to delta.refusal, and let BufferedResponseAccumulator keep refusal deltas
so non-streaming replies rebuilt from deltas keep them too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The buffered Anthropic aggregators index finalResp.Content with the
upstream content_block_delta index after checking only idx < len. A
malformed or third-party Anthropic-compatible upstream that sends a
negative index makes the request goroutine panic, so the client gets no
response and the forward never returns a result to record usage.
Add the lower bound in all four buffered aggregators (Chat Completions
and Responses, on both the Anthropic gateway and the OpenAI native
Anthropic paths); the out-of-range delta is skipped like an index past
the end.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Legacy Chat function calling has no call IDs: the assistant turn carries
function_call and the role=function result refers to it only by name.
ChatCompletionsToResponses ignored the assistant function_call and used
the function name as the result's call_id, so the Responses upstream got
a function_call_output with no matching function_call and rejected the
replayed history.
Emit a function_call item for each legacy assistant call and give the
following same-name result the same call_id. IDs follow message order
(call_legacy_N), so the same history converts to the same input across
turns, and skip IDs already used by tool_calls. A result with no
preceding call keeps the previous name-based call_id.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chatMessageToResponsesItems only mapped "system" explicitly and sent every
other unknown role through the user converter, so Chat Completions
developer messages reached the Responses upstream as user turns and lost
their instruction priority.
The Responses API accepts role "developer" in input messages; convert it
like a system message and keep the original role. Downstream converters
already understand it (ResponsesToAnthropicRequest folds developer into
the system prompt, the Codex transform forwards it unchanged).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Chat Completions names a forced function as
{"type":"function","function":{"name":"X"}}, while the Responses API
expects {"type":"function","name":"X"}. ChatCompletionsToResponses passed
the Chat shape through unchanged, so OpenAI API-key accounts on
/v1/chat/completions forwarded the nested shape to /v1/responses and the
upstream rejected the request.
Flatten the named Chat choice; strings and objects already in the
Responses shape are kept as-is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Responses WebSocket handler reads the API key auth snapshot once at
connect and reuses it for every turn: RecordUsage takes the group pricing
from that snapshot, and the profit gate reinstalled in BeforeTurn takes
its selling price from the connect-time ctxkey.Group. A group price
change therefore never reaches an open connection until the client
reconnects, while HTTP requests pick it up once the auth cache entry is
invalidated.
Each later turn now re-reads the key through APIKeyService.GetByKey (the
auth cache the middleware uses) in BeforeTurn. When the key ID, group ID,
group platform and subscription type are unchanged, the fresh group is
swapped into ctxkey.Group for the turn's profit gate and passed to that
turn's RecordUsage. A failed lookup or a key moved to another group keeps
the connection snapshot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds two regression cases mirroring the #5009 OpenAI ones for Grok
platform accounts (issue #7774):
- TestOpenAIResponsesEmptyCompletedFailsOverGrok: an empty
response.completed on the forwardGrokResponses streaming path must
produce UpstreamFailoverError with an empty client body. Verified to
FAIL on main without the guard extension and PASS with it.
- TestOpenAIResponsesEmptyCompletedWithOutputSucceedsGrok: normal Grok
streams with semantic output and usage are untouched.
Fixes#7774
Issue #5009 added a failover guard for OpenAI Responses streams that
terminate with an empty response.completed (no output, no usage, no
error), but the guard is gated on PlatformOpenAI. Grok platform
accounts reach the same handler (forwardGrokResponses ->
handleStreamingResponseWithReasoning) and exhibit the same upstream
silent-refusal shape, so their empty completions are relayed to the
client as successful empty turns: Responses SDK clients map the empty
completed to finishReason stop and accept the turn with no retry.
Extend the guard to PlatformGrok, matching the unconditional guard on
the passthrough path (openai_gateway_passthrough.go).
Fixes#7774
The warning said "Other errors will return 500", but custom error codes
only gate normal account-error handling (stop scheduling, rate-limit
marking). Request retry and account failover decide independently: on the
OpenAI-compatible gateway, unselected 403/5xx still fail over to another
account and only exhaust to 502; only some non-failover paths (for example
an ordinary 400) are rewritten to a generic 500. Reword zh/en to describe
what the setting actually does, including that an empty list applies no
filtering.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Wesley Liddick