Compare commits

..
56 Commits
Author SHA1 Message Date
Wesley Liddick b73d8c3efe Merge pull request #4009 from heathermhuang/codex/fix-recent-grok-issues
fix: expand Grok API, CLI, billing, and setup support
2026-07-13 10:36:11 +08:00
Wesley Liddick 0d318195bf Merge pull request #4116 from Wei-Shaw/feat/alpha-search-per-call-billing
feat(billing): Codex alpha/search 网页搜索按次计费
2026-07-13 10:35:46 +08:00
shaw e5af699d0f test(contract): groups/available 契约夹具补 web_search_price_per_call 字段 2026-07-13 10:21:44 +08:00
Heatherm Huang f73031f436 test(grok): align scheduling reasons with upstream 2026-07-13 10:21:34 +08:00
shaw 64a2a31729 fix(billing): 修复复审发现的三处按次计费问题
- HIGH: GetByKeyForAuth 的 WithGroup 显式 Select 补 web_search_price_per_call 列,
  否则 auth 快照里分组覆盖价恒为 nil,覆盖价/免费配置在实际计费中静默失效
- MEDIUM: 按次搜索倍率改用不含高峰因子的基础倍率(与 image/video 按次不变式
  及分组表单价格预览承诺一致),calculateOpenAIRecordUsageCost 新增独立倍率参数
- LOW: 计费门槛从 <400 收紧为严格 2xx,1xx/3xx 透传不计费
2026-07-13 10:14:53 +08:00
Heatherm Huang 8a22dc7347 fix(grok): diagnose unavailable models by platform 2026-07-13 10:11:34 +08:00
Heatherm Huang aeb34d2003 fix(grok): sanitize composer reasoning parameters 2026-07-13 10:11:33 +08:00
Heatherm Huang c4ff604e93 test(grok): cover OAuth chat permission identity 2026-07-13 10:11:33 +08:00
Heatherm Huang ce3f12bbff test(grok): cover CLI identity at transport boundary 2026-07-13 10:11:33 +08:00
Heatherm Huang f187f08ae3 fix(grok): harden OAuth routing and CLI version guard 2026-07-13 10:11:33 +08:00
Heatherm Huang cbddb57dec fix(grok): display remaining quota capacity 2026-07-13 10:11:33 +08:00
Heatherm Huang 3375b4ed2b fix(grok): route OAuth subscriptions through CLI proxy 2026-07-13 10:11:33 +08:00
Heatherm Huang d9e466ad3a feat(grok): support xAI API key accounts
Allow Grok API-key accounts in Responses forwarding and connection tests, expose creation and edit defaults in the dashboard, and document the supported setup.
2026-07-13 10:11:33 +08:00
Heatherm Huang ad18ee7c4f fix(grok): use responses adapter for OpenCode 2026-07-13 10:09:53 +08:00
Heatherm Huang 038b25c0b1 fix(grok): resolve recent integration issues 2026-07-13 10:09:53 +08:00
shaw 7cbb36f278 feat(billing): Codex alpha/search 网页搜索按次计费
- alpha/search 成功请求(上游 2xx)按次计费落 usage_logs(billing_mode=per_request),
  上游错误透传/failover 不计费;使用 mandatory 池提交,池满同步兜底不丢扣费
- 单价默认 0.01 USD/次(官方 $10/1000 次),分组新增 web_search_price_per_call
  覆盖价(0=免费,负数/留空=默认价),实际扣费叠加分组费率倍数(与 token 口径一致)
- 分组字段全链路:ent schema + 迁移 174 + 端口快照 v15 + admin 创建/更新 + DTO
- 前端分组表单(openai 平台)新增单次价格配置,实时预览应用当前倍率后的单次价格
- 该端点鉴权维持仅 OpenAI 分组(非 OpenAI 分组 404)
2026-07-13 09:54:51 +08:00
Wesley Liddick 42f3c22830 Merge pull request #4060 from superman2003/feat/grok-prompt-cache-identity
feat(grok): enable Free OAuth prompt caching and quota recovery
2026-07-13 09:20:57 +08:00
shaw 8d51364c3d Merge remote-tracking branch 'origin/main' into feat/grok-prompt-cache-identity
# Conflicts:
#	backend/internal/handler/endpoint.go
#	backend/internal/service/openai_gateway_grok_test.go
2026-07-13 09:12:44 +08:00
Wesley Liddick 33b1d772f7 Merge pull request #4063 from yardbirds0/fix/codex-alpha-search
fix: 转发 Codex alpha/search 独立搜索端点
2026-07-13 08:58:01 +08:00
Wesley Liddick 1c214eaca4 Merge pull request #4059 from visa2/fix/openai-messages-codex-identity
fix(openai): restore Codex identity headers for OAuth Messages
2026-07-13 08:56:21 +08:00
Wesley Liddick 841481e051 Merge pull request #3971 from yardbirds0/codex/fix-remote-compaction-v2
fix: 保留 remote_compaction_v2 原生 Responses 请求链路
2026-07-13 08:54:53 +08:00
Wesley Liddick 2f4478fd32 Merge pull request #4019 from wucm667/fix/compact-keepalive-writer-nil-guard
fix(service): 补齐 compact keepalive writer nil 守卫
2026-07-13 08:54:31 +08:00
Wesley Liddick 877bee84a1 Merge pull request #4072 from jianjianai/codex/fix-upstream-review
修复 tool_search 参数解析与 compact writer 生命周期
2026-07-13 08:54:15 +08:00
Wesley Liddick d734dbdac4 Merge pull request #3997 from lyon-le/feat/openai-fast-policy-user-scope
feat(frontend): Fast/Flex 策略支持搜索选择用户
2026-07-13 08:54:01 +08:00
Wesley Liddick 05865d9b65 Merge pull request #4003 from wucm667/fix/codex-message-item-id-prefix
fix(codex): 剥离续链 message item 的非法 item_* id
2026-07-13 08:53:51 +08:00
superman2003 7050070aa3 fix(grok): route cacheable chat requests via responses 2026-07-12 11:50:56 +08:00
jjaw 73ffd13430 关联上游 issue #3818 #3887 #3961
相关修复 PR:#3989 #3994
2026-07-12 05:12:20 +08:00
jjaw f57d06d959 合并 compact writer 修复 2026-07-12 04:53:55 +08:00
jjaw 1a8401f5f3 合并 tool_search 参数修复 2026-07-12 04:53:54 +08:00
jjaw 06af8115f7 修复 compact 心跳 writer 生命周期 2026-07-12 04:53:41 +08:00
jjaw 5015b7a1c1 修复 tool_search 参数对象反序列化 2026-07-12 04:53:34 +08:00
visa2 d5b47c2142 fix(openai): restore Codex identity for OAuth Messages 2026-07-11 20:47:52 +08:00
superman2003 1dedb2097d fix(grok): persist quota exhaustion as rate limit 2026-07-11 20:41:32 +08:00
Tian Lee 52071d391b fix(openai): 转发 Codex alpha/search 独立搜索端点 2026-07-11 20:31:33 +08:00
superman2003 0478fd3668 feat(grok): enable free OAuth prompt caching 2026-07-11 19:34:01 +08:00
Tian Lee 84bb7d0709 fix: 保留 remote_compaction_v2 原生 Responses 链路 2026-07-11 12:51:53 +08:00
wucm667 6e2bb31281 fix(service): guard compact keepalive writer delegates 2026-07-11 08:52:46 +08:00
wucm667andClaude Opus 4.8 4d4ba64bf7 fix(codex): 剥离续链 message item 的非法 item_* id
OpenAI OAuth 转发续链请求时,type=message 的 item 的 id 被客户端以
item_* 形式回放,但上游要求以 msg 开头,返回 400
"Expected an ID that begins with 'msg'",sub2api 随后向客户端返回 502。
客户端自动重试会不断重放同一份坏上下文,导致连续失败。

filterCodexInputWithOptions 在 PreserveReferences=true 路径下为
type=message 增加与 #3785 (fd64d07e6) 平行的 id 前缀检查:非 msg 开头
即删除。合法的 msg* id 原样保留,不改写 item_* 为 msg_*,因为改写出的
id 未必对应真实上游对象。

原 TestFilterCodexInput_NonToolCallItemKeepsID 以 message + item_msg_001
断言"保留 id",该行为已被上游拒绝,改用 web_search_call 覆盖同一意图。

Fixes #3981

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 23:32:34 +08:00
Lyonle 0464856c4a feat(frontend): Fast/Flex 策略支持搜索选择用户
将手工 User ID 输入替换为带防抖的邮箱模糊搜索和多选标签,同时继续保存 user_ids。

回显已有用户邮箱并保留无法解析的历史 ID,补齐中英文文案路径及组件回归测试。
2026-07-10 22:27:15 +08:00
Wesley Liddick e316ebf528 Merge pull request #3989 from xlplbo/fix/codex-mcp-tools-bridge
fix(apicompat): 补齐 Codex 0.14x 工具链经 chat 回退桥的转换(custom/tool_search/namespace)
2026-07-10 22:09:25 +08:00
Wesley Liddick 07fac34713 Merge pull request #3991 from fengshao1227/fix/anthropic-usage-cache-creation-tokens
fix(apicompat): Responses↔Anthropic 转换补齐 cache_creation_input_tokens
2026-07-10 21:49:29 +08:00
Wesley Liddick 151b9265fc Merge pull request #3994 from fengshao1227/fix/compact-keepalive-nil-pointer
fix(handler): opsCaptureWriter 释放后 nil pointer panic
2026-07-10 21:49:07 +08:00
github-actions[bot] 6c588bb950 chore: sync VERSION to 0.1.151 [skip ci] 2026-07-10 13:24:13 +00:00
xlplboandClaude Fable 5 90e9d03dec fix(apicompat): 强制选择 tool_search 的 tool_choice 降级为指向代理的 function 选择
tool_search 工具未被丢弃而是降级为同名 function 代理,此前 tool_choice 过滤
把它连同其他服务端工具类型一起静默丢弃,强制工具搜索被退化为自动选择,
违反调用方明确指定的工具选择语义。改为映射到指向代理的 function 选择;
未声明 type=tool_search 时无可指向的代理,维持丢弃。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NWQyEgFKKbdve67G6qCoAU
2026-07-10 21:14:33 +08:00
xlplboandClaude Fable 5 e2b68d1f90 fix(apicompat): tool_choice 仅转发实际存在于转换结果中的工具
此前只检查转换后是否还剩任意工具,不校验 tool_choice 指向的工具是否幸存:
强制选择被丢弃的服务端工具(如 web_search)或指向不存在名字时,选择项被
原样转发,chat 上游因 tool_choice 指向未声明工具而 400。改为具名选择项仅
在目标存在于转换后工具集时转发,服务端工具类选择项随工具本身丢弃;
"auto" 等字符串形式保持原样转发。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NWQyEgFKKbdve67G6qCoAU
2026-07-10 21:06:45 +08:00
xlplboandClaude Fable 5 a2cdaa6419 fix(apicompat): 内置 tool_search 与同名工具撞名时显式拒绝请求
tool_search 服务端工具降级后的代理 function 不能改名(codex 的模型侧按
tool_search 这个名字调用),与客户端声明的同名 function/custom 工具无法
区分:去程产生重复声明,回程会把普通工具的调用劫持还原成 tool_search_call。
与 namespace 摊平撞名的处理一致,在请求转换阶段显式报错(网关 400);
重复声明 type=tool_search 去重后只产出一个代理,不拒绝。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NWQyEgFKKbdve67G6qCoAU
2026-07-10 21:06:26 +08:00
li bc3cb29027 fix(handler): 补齐 opsCaptureWriter 全部委托方法的 nil 守卫 2026-07-10 21:03:40 +08:00
li 89a551b964 fix(handler): opsCaptureWriter 释放后访问 nil panic
Fixes #3961
2026-07-10 20:56:31 +08:00
li 83f169e4fa fix(apicompat): 补齐 Responses→Anthropic 流式路径的 cache_creation_input_tokens 2026-07-10 20:49:44 +08:00
xlplbo eb4d005031 Merge remote-tracking branch 'upstream/main' 2026-07-10 20:46:38 +08:00
li 0d28f7f90d fix(apicompat): Responses↔Anthropic 转换补齐 cache_creation_input_tokens
Fixes #3935
2026-07-10 20:40:45 +08:00
xlplboandClaude Fable 5 f1082bb78f fix(apicompat): namespace 摊平名撞名时显式拒绝请求
摊平名与顶层 function/custom 工具撞名、或跨 namespace 摊平出同名时,
chat 上游无法按 namespace 区分调用归属:此前重复声明照发上游、回程
固定命中其中一条映射,调用可能被还原到错误工具。这类请求在原生
Responses 上游合法,歧义由摊平转换制造且不可消除,改为在请求转换
阶段直接报错(网关返回 400 invalid_request_error 并点名冲突双方);
同一 (namespace, 子工具) 的重复声明去重后不拒绝。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NWQyEgFKKbdve67G6qCoAU
2026-07-10 17:38:48 +08:00
xlplboandClaude Fable 5 7942338328 fix(apicompat): 回程还原 namespace 子工具调用,修复 Codex MCP 工具 unsupported call
Codex 0.14x 将 MCP 工具声明为 namespace 工具,chat 桥去程摊平为
"<namespace>__<name>" function 工具后,回程仅原样回传平铺名的
function_call 项;codex 按 namespace+name 路由查不到该名字,所有
MCP 工具调用被判为 unsupported call。

- NamespaceToolNames 构建摊平名 →(namespace, 子工具名)反查表
  (摊平名超长带截断哈希,无法按字符串切分还原)
- 非流式/流式回程命中映射时还原为裸子工具名 + namespace 字段,
  ResponsesOutput 新增 Namespace 字段并同步 wire 层白名单
- 回退桥入口将映射与 CustomTools/ToolSearchDeclared 一并穿入

已在测试机经 codex exec + MCP server 端到端验证。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NWQyEgFKKbdve67G6qCoAU
2026-07-10 17:38:17 +08:00
xlplboandClaude Fable 5 18e26c127c Merge remote-tracking branch 'upstream/main'
解决 backend/internal/service/openai_gateway_responses_chat_fallback.go 冲突:
将本地 tool_search/custom 工具改动(customTools/toolSearch 穿参与流状态标记)
移植到上游重构后的 CC 公共管线;同步适配上游新增调用点
openai_gateway_messages_chat_fallback.go 的 ChatCompletionsResponseToResponses 签名。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8zMzeFAumSddraEw1FUeG
2026-07-10 09:27:48 +08:00
xlplbo 27e29f0562 feat(apicompat): 添加 tool_search 支持,增强工具调用处理逻辑 2026-07-07 19:19:35 +08:00
xlplbo 75fb3c41c2 fix(apicompat): responses→chat 桥支持 custom 工具,修复 Codex exec 丢失
Codex 0.14x 的核心执行工具 exec 是 type=custom(freeform)工具,
responsesToolsToChatTools 丢弃所有非 function 工具,导致模型工具列表中
没有 exec、无法执行任何命令;SSE 序列化层 responsesItemWire 也缺少
custom_tool_call 分支,即使产出该类型 item,call_id/name/input 也会被
白名单剥离。

- custom 工具降级为单一 input:string 参数的 function 工具
- 回程(非流式+流式)按名字还原 custom_tool_call 项/事件
- 历史 custom_tool_call / custom_tool_call_output 项转换(此前被跳过)
- responsesItemWire 补 custom_tool_call 分支
- tools 全部被丢弃时不再转发 tool_choice(修复上游 400
  "'tool_choice' is only allowed when 'tools' are specified")

feat(apicompat): chat 桥补齐 tool_search/namespace/字符串工具声明

对齐参考实现(cc-switch transform_codex_chat.rs)的工具类型覆盖:
- tool_search → 固定 function 代理工具(query 必填/limit 可选),
  历史项 tool_search_call/tool_search_output 一并转换
- namespace → 子 function 工具摊平为 <namespace>__<name>,
  超 64 字符截断加 sha256 短哈希;历史项带 namespace 字段时同规则摊平
- tools: ["name"] 字符串简写按 custom 工具解析

fix(apicompat): 工具调用宣告延迟到类型可判定,兼容 name 迟到的上游

output_item.added 的 custom/function 判定依赖工具名。名字晚于首个分片
到达时,此前会按 function_call 宣告造成 added/done 项类型不一致。
改为存在 custom 工具且名字未到时延迟宣告,宣告时补发已累积的参数增量;
请求无 custom 工具时行为不变。

PR 标题与正文(可直接粘贴)

标题:fix(apicompat): responses→chat 桥丢弃 custom 工具导致 Codex 无法执行命令

正文:

## 问题

OpenAI APIKey 账号接入仅支持 /v1/chat/completions 的第三方上游
(DeepSeek/GLM/Kimi 等)时,Codex 0.14x 客户端文本对话正常,但无法执行
任何命令:exec 调用全部 "aborted",spawn_agent 派出的子智能体永远卡在
running,多轮后模型自述"环境无法执行文件系统命令"放弃。

## 根因

Codex 0.14x 的核心执行工具 exec 是 Responses API 的 custom(freeform)
工具,而非普通 function:

    {"type": "custom", "name": "exec", "format": {"type": "grammar", ...}}

两处缺陷叠加:

1. `responsesToolsToChatTools` 丢弃所有非 function 工具
   (chatcompletions_responses_bridge.go),模型工具列表中从未出现 exec;
2. SSE 序列化 `responsesItemWire` 按 item 类型白名单重组字段,无
   custom_tool_call 分支,call_id/name/input 会被剥离
   (responses_stream_event_wire.go)。

次生缺陷:工具全部被丢弃时 tool_choice 仍被转发,部分上游报 400
"'tool_choice' is only allowed when 'tools' are specified"。

## 改动

(三个提交的要点清单,见各提交说明)

设计与 cc-switch 的 Codex chat 转换层(transform_codex_chat.rs /
streaming_codex_chat.rs)对齐:custom 工具降级为 {"input": string}
function 工具;custom 调用的参数增量不可流式还原为自由文本,缓冲至
收尾一次性下发 custom_tool_call_input.delta/done。

## 测试

- 新增 17 个单元测试:请求工具转换、tool_choice 兜底、历史项转换、
  非流式/流式回程还原、wire 序列化字段完整性、name 迟到宣告、命名摊平
- `go test ./internal/pkg/apicompat/...` 与 service 相关测试全绿
- 真机验证:Codex CLI 0.142.5 → 本桥 → GLM/DeepSeek 系 chat-only 上游,
  exec 实际执行命令并多轮续作正常

## 影响面

仅影响 "APIKey 账号 + 判定不支持 Responses" 的 chat 直转路径:

- OAuth 账号与透传路径不经过本桥(透传为字节级原文转发)
- CC→Responses 反方向仅构造 ResponsesRequest,新增字段带 omitempty
  且不赋值,序列化输出不变
- 探测逻辑未动,路由判定不变
- 共享类型改动均为增量容错(新字段 omitempty;UnmarshalJSON 只新增
  字符串工具的解析成功路径)
2026-07-07 16:23:44 +08:00
128 changed files with 8265 additions and 501 deletions
+45 -10
View File
@@ -637,19 +637,19 @@ Simple Mode is designed for individual developers or internal teams who want qui
---
## Grok / xAI OAuth Support
## Grok / xAI Support
Sub2API supports Grok subscription accounts through xAI OAuth and forwards OpenAI-compatible Responses traffic to xAI.
Sub2API supports both Grok subscription accounts through xAI OAuth and standard xAI API-key accounts. Both account types forward OpenAI-compatible Responses traffic to xAI.
### Supported Scope
- Platform name: `grok`
- Account type: OAuth subscription accounts
- Public Responses targets: `/v1/responses`, `/responses`, and `/backend-api/codex/responses`, forwarded to `${XAI_BASE_URL:-https://api.x.ai/v1}/responses`
- Account types: OAuth subscription accounts and xAI API-key accounts
- Public Responses targets: `/v1/responses`, `/responses`, and `/backend-api/codex/responses`, forwarded to the Grok subscription proxy for OAuth accounts or `https://api.x.ai/v1/responses` for API-key accounts
- Public Claude-compatible target: `/v1/messages`, converted to xAI Responses and returned as Anthropic Messages output for Claude CLI style clients
- Public Chat Completions targets: `/v1/chat/completions` and `/chat/completions`, forwarded to `${XAI_BASE_URL:-https://api.x.ai/v1}/chat/completions`
- Public Chat Completions targets: `/v1/chat/completions` and `/chat/completions`, forwarded to the account-type-specific xAI upstream
- Codex CLI style Responses WebSocket ingress is accepted on the Responses targets and bridged to xAI HTTP/SSE Responses upstream
- Initial text models: `grok-4.3`, `grok-build-0.1`, `grok-4.20-0309-reasoning`, `grok-4.20-0309-non-reasoning`, and `grok-4.20-multi-agent-0309`
- Text models: `grok-4.5`, `grok-4.3`, `grok-build-0.1`, `grok-composer-2.5-fast`, `grok-4.20-0309-reasoning`, `grok-4.20-0309-non-reasoning`, and `grok-4.20-multi-agent-0309`
- Media targets for Grok groups: `/v1/images/generations`, `/images/generations`, `/v1/images/edits`, `/images/edits`, `/v1/videos/generations`, `/videos/generations`, `/v1/videos/{request_id}`, and `/videos/{request_id}`. Generation requests require the group image-generation permission.
- Media models: `grok-imagine`, `grok-imagine-image-quality`, `grok-imagine-image`, `grok-imagine-edit`, `grok-imagine-video`, and `grok-imagine-video-1.5`
- Out of scope for this provider: TTS, transcription, browser automation, cookies, and Grok web scraping
@@ -665,9 +665,10 @@ The Grok OAuth flow uses PKCE and does not require committing private secrets. T
| `XAI_OAUTH_REDIRECT_URI` | `http://127.0.0.1:56121/callback` |
| `XAI_OAUTH_AUTHORIZE_URL` | `https://auth.x.ai/oauth2/authorize` |
| `XAI_OAUTH_TOKEN_URL` | `https://auth.x.ai/oauth2/token` |
| `XAI_BASE_URL` | `https://api.x.ai/v1` |
| `XAI_BASE_URL` | `https://api.x.ai/v1`; runtime-diagnostics override (account `base_url` controls request forwarding) |
| `XAI_GROK_CLI_VERSION` | `0.2.93`; optional override for the client identity sent to `cli-chat-proxy.grok.com` |
Administrators can create or reauthorize Grok accounts from the dashboard, or use the admin API:
Administrators can create Grok OAuth or API-key accounts from the dashboard. OAuth authorization and reauthorization are also available through the admin API:
| Endpoint | Purpose |
|----------|---------|
@@ -676,13 +677,47 @@ Administrators can create or reauthorize Grok accounts from the dashboard, or us
| `POST /api/v1/admin/grok/oauth/refresh-token` | Validate or refresh a Grok refresh token |
| `POST /api/v1/admin/grok/accounts/:id/refresh` | Refresh an existing Grok account |
Credential storage reuses the existing account JSON fields: `access_token`, `refresh_token`, `token_type`, `expires_at`, optional `email`, optional `subscription_tier`, and `entitlement_status`.
OAuth credential storage reuses the existing account JSON fields: `access_token`, `refresh_token`, `token_type`, `expires_at`, `base_url`, optional `email`, optional `subscription_tier`, and `entitlement_status`. OAuth inference defaults to `https://cli-chat-proxy.grok.com/v1`; existing OAuth accounts that stored the old `https://api.x.ai/v1` default are redirected to the subscription proxy at runtime. Explicit custom upstreams remain unchanged.
For API-key accounts, select **Grok → API Key** in the create-account dialog. The official base URL defaults to `https://api.x.ai/v1`; credentials use the existing `base_url` and `api_key` account fields. OAuth accounts continue to use the subscription flow above.
### Grok Build CLI Configuration
1. In the Sub2API admin dashboard, add either a `grok` OAuth account and complete xAI authorization, or add a Grok API-key account.
2. Create a Grok group, attach the account to it, then create a Sub2API API key assigned to that group.
3. In the user API-key page, click **Use Key** and select **Grok CLI**. The modal generates the correct file and base URL for macOS/Linux or Windows. It also provides an OpenCode configuration on the **OpenCode** tab.
4. If configuring manually, save the following as `~/.grok/config.toml` (Windows: `%USERPROFILE%\.grok\config.toml`):
```toml
[models]
default = "sub2api-grok"
web_search = "sub2api-grok"
[model."sub2api-grok"]
model = "grok-4.5"
base_url = "https://your-sub2api.example.com/v1"
name = "Grok 4.5 via Sub2API"
description = "Grok 4.5 through a Sub2API Grok group"
api_key = "sk-your-sub2api-key"
api_backend = "responses"
context_window = 1000000
supports_backend_search = true
```
Back up an existing `config.toml` before merging the entry. The file contains a Sub2API API key, so keep it private and restrict its permissions where supported. Verify the effective configuration and make a smoke request:
```bash
grok inspect
grok -p "Reply with sub2api-ok" -m sub2api-grok
```
The `base_url` above is the public Sub2API URL ending in `/v1`, not `api.x.ai` or the internal xAI OAuth proxy URL.
### Usage And Quota Display
xAI quota is passive. Sub2API does not invent subscription quota values; it records whitelisted xAI rate-limit headers from successful or rate-limited upstream responses when xAI sends them. Before the first usable upstream response, the dashboard shows quota as unknown and still displays local Sub2API usage stats.
`401` responses mark the account as needing reauthorization. `403` responses are treated as entitlement or subscription-tier failures instead of token-refresh loops. `429` responses use `Retry-After` or a short cooldown to temporarily remove the account from scheduling.
`401` responses temporarily remove accounts with invalid credentials from scheduling. `403` responses are treated as access or entitlement failures instead of token-refresh loops. `429` responses use `Retry-After` or a short cooldown to temporarily remove the account from scheduling.
---
+1 -1
View File
@@ -1 +1 @@
0.1.150
0.1.151
+15 -1
View File
@@ -83,6 +83,8 @@ type Group struct {
VideoPrice720p *float64 `json:"video_price_720p,omitempty"`
// VideoPrice1080p holds the value of the "video_price_1080p" field.
VideoPrice1080p *float64 `json:"video_price_1080p,omitempty"`
// Codex alpha/search 网页搜索单次价格(USD/次);nil 表示使用默认价 0.01(官方 $10/1000 次)
WebSearchPricePerCall *float64 `json:"web_search_price_per_call,omitempty"`
// 是否仅允许 Claude Code 客户端
ClaudeCodeOnly bool `json:"claude_code_only,omitempty"`
// 非 Claude Code 请求降级使用的分组 ID
@@ -223,7 +225,7 @@ func (*Group) scanValues(columns []string) ([]any, error) {
values[i] = new([]byte)
case group.FieldPeakRateEnabled, group.FieldIsExclusive, group.FieldAllowImageGeneration, group.FieldAllowBatchImageGeneration, group.FieldImageRateIndependent, group.FieldVideoRateIndependent, group.FieldClaudeCodeOnly, group.FieldModelRoutingEnabled, group.FieldMcpXMLInject, group.FieldAllowMessagesDispatch, group.FieldRequireOauthOnly, group.FieldRequirePrivacySet:
values[i] = new(sql.NullBool)
case group.FieldRateMultiplier, group.FieldPeakRateMultiplier, group.FieldDailyLimitUsd, group.FieldWeeklyLimitUsd, group.FieldMonthlyLimitUsd, group.FieldImageRateMultiplier, group.FieldImagePrice1k, group.FieldImagePrice2k, group.FieldImagePrice4k, group.FieldBatchImageDiscountMultiplier, group.FieldBatchImageHoldMultiplier, group.FieldVideoRateMultiplier, group.FieldVideoPrice480p, group.FieldVideoPrice720p, group.FieldVideoPrice1080p:
case group.FieldRateMultiplier, group.FieldPeakRateMultiplier, group.FieldDailyLimitUsd, group.FieldWeeklyLimitUsd, group.FieldMonthlyLimitUsd, group.FieldImageRateMultiplier, group.FieldImagePrice1k, group.FieldImagePrice2k, group.FieldImagePrice4k, group.FieldBatchImageDiscountMultiplier, group.FieldBatchImageHoldMultiplier, group.FieldVideoRateMultiplier, group.FieldVideoPrice480p, group.FieldVideoPrice720p, group.FieldVideoPrice1080p, group.FieldWebSearchPricePerCall:
values[i] = new(sql.NullFloat64)
case group.FieldID, group.FieldDefaultValidityDays, group.FieldFallbackGroupID, group.FieldFallbackGroupIDOnInvalidRequest, group.FieldSortOrder, group.FieldRpmLimit:
values[i] = new(sql.NullInt64)
@@ -455,6 +457,13 @@ func (_m *Group) assignValues(columns []string, values []any) error {
_m.VideoPrice1080p = new(float64)
*_m.VideoPrice1080p = value.Float64
}
case group.FieldWebSearchPricePerCall:
if value, ok := values[i].(*sql.NullFloat64); !ok {
return fmt.Errorf("unexpected type %T for field web_search_price_per_call", values[i])
} else if value.Valid {
_m.WebSearchPricePerCall = new(float64)
*_m.WebSearchPricePerCall = value.Float64
}
case group.FieldClaudeCodeOnly:
if value, ok := values[i].(*sql.NullBool); !ok {
return fmt.Errorf("unexpected type %T for field claude_code_only", values[i])
@@ -749,6 +758,11 @@ func (_m *Group) String() string {
builder.WriteString(fmt.Sprintf("%v", *v))
}
builder.WriteString(", ")
if v := _m.WebSearchPricePerCall; v != nil {
builder.WriteString("web_search_price_per_call=")
builder.WriteString(fmt.Sprintf("%v", *v))
}
builder.WriteString(", ")
builder.WriteString("claude_code_only=")
builder.WriteString(fmt.Sprintf("%v", _m.ClaudeCodeOnly))
builder.WriteString(", ")
+8
View File
@@ -80,6 +80,8 @@ const (
FieldVideoPrice720p = "video_price_720p"
// FieldVideoPrice1080p holds the string denoting the video_price_1080p field in the database.
FieldVideoPrice1080p = "video_price_1080p"
// FieldWebSearchPricePerCall holds the string denoting the web_search_price_per_call field in the database.
FieldWebSearchPricePerCall = "web_search_price_per_call"
// FieldClaudeCodeOnly holds the string denoting the claude_code_only field in the database.
FieldClaudeCodeOnly = "claude_code_only"
// FieldFallbackGroupID holds the string denoting the fallback_group_id field in the database.
@@ -217,6 +219,7 @@ var Columns = []string{
FieldVideoPrice480p,
FieldVideoPrice720p,
FieldVideoPrice1080p,
FieldWebSearchPricePerCall,
FieldClaudeCodeOnly,
FieldFallbackGroupID,
FieldFallbackGroupIDOnInvalidRequest,
@@ -511,6 +514,11 @@ func ByVideoPrice1080p(opts ...sql.OrderTermOption) OrderOption {
return sql.OrderByField(FieldVideoPrice1080p, opts...).ToFunc()
}
// ByWebSearchPricePerCall orders the results by the web_search_price_per_call field.
func ByWebSearchPricePerCall(opts ...sql.OrderTermOption) OrderOption {
return sql.OrderByField(FieldWebSearchPricePerCall, opts...).ToFunc()
}
// ByClaudeCodeOnly orders the results by the claude_code_only field.
func ByClaudeCodeOnly(opts ...sql.OrderTermOption) OrderOption {
return sql.OrderByField(FieldClaudeCodeOnly, opts...).ToFunc()
+55
View File
@@ -215,6 +215,11 @@ func VideoPrice1080p(v float64) predicate.Group {
return predicate.Group(sql.FieldEQ(FieldVideoPrice1080p, v))
}
// WebSearchPricePerCall applies equality check predicate on the "web_search_price_per_call" field. It's identical to WebSearchPricePerCallEQ.
func WebSearchPricePerCall(v float64) predicate.Group {
return predicate.Group(sql.FieldEQ(FieldWebSearchPricePerCall, v))
}
// ClaudeCodeOnly applies equality check predicate on the "claude_code_only" field. It's identical to ClaudeCodeOnlyEQ.
func ClaudeCodeOnly(v bool) predicate.Group {
return predicate.Group(sql.FieldEQ(FieldClaudeCodeOnly, v))
@@ -1655,6 +1660,56 @@ func VideoPrice1080pNotNil() predicate.Group {
return predicate.Group(sql.FieldNotNull(FieldVideoPrice1080p))
}
// WebSearchPricePerCallEQ applies the EQ predicate on the "web_search_price_per_call" field.
func WebSearchPricePerCallEQ(v float64) predicate.Group {
return predicate.Group(sql.FieldEQ(FieldWebSearchPricePerCall, v))
}
// WebSearchPricePerCallNEQ applies the NEQ predicate on the "web_search_price_per_call" field.
func WebSearchPricePerCallNEQ(v float64) predicate.Group {
return predicate.Group(sql.FieldNEQ(FieldWebSearchPricePerCall, v))
}
// WebSearchPricePerCallIn applies the In predicate on the "web_search_price_per_call" field.
func WebSearchPricePerCallIn(vs ...float64) predicate.Group {
return predicate.Group(sql.FieldIn(FieldWebSearchPricePerCall, vs...))
}
// WebSearchPricePerCallNotIn applies the NotIn predicate on the "web_search_price_per_call" field.
func WebSearchPricePerCallNotIn(vs ...float64) predicate.Group {
return predicate.Group(sql.FieldNotIn(FieldWebSearchPricePerCall, vs...))
}
// WebSearchPricePerCallGT applies the GT predicate on the "web_search_price_per_call" field.
func WebSearchPricePerCallGT(v float64) predicate.Group {
return predicate.Group(sql.FieldGT(FieldWebSearchPricePerCall, v))
}
// WebSearchPricePerCallGTE applies the GTE predicate on the "web_search_price_per_call" field.
func WebSearchPricePerCallGTE(v float64) predicate.Group {
return predicate.Group(sql.FieldGTE(FieldWebSearchPricePerCall, v))
}
// WebSearchPricePerCallLT applies the LT predicate on the "web_search_price_per_call" field.
func WebSearchPricePerCallLT(v float64) predicate.Group {
return predicate.Group(sql.FieldLT(FieldWebSearchPricePerCall, v))
}
// WebSearchPricePerCallLTE applies the LTE predicate on the "web_search_price_per_call" field.
func WebSearchPricePerCallLTE(v float64) predicate.Group {
return predicate.Group(sql.FieldLTE(FieldWebSearchPricePerCall, v))
}
// WebSearchPricePerCallIsNil applies the IsNil predicate on the "web_search_price_per_call" field.
func WebSearchPricePerCallIsNil() predicate.Group {
return predicate.Group(sql.FieldIsNull(FieldWebSearchPricePerCall))
}
// WebSearchPricePerCallNotNil applies the NotNil predicate on the "web_search_price_per_call" field.
func WebSearchPricePerCallNotNil() predicate.Group {
return predicate.Group(sql.FieldNotNull(FieldWebSearchPricePerCall))
}
// ClaudeCodeOnlyEQ applies the EQ predicate on the "claude_code_only" field.
func ClaudeCodeOnlyEQ(v bool) predicate.Group {
return predicate.Group(sql.FieldEQ(FieldClaudeCodeOnly, v))
+98
View File
@@ -469,6 +469,20 @@ func (_c *GroupCreate) SetNillableVideoPrice1080p(v *float64) *GroupCreate {
return _c
}
// SetWebSearchPricePerCall sets the "web_search_price_per_call" field.
func (_c *GroupCreate) SetWebSearchPricePerCall(v float64) *GroupCreate {
_c.mutation.SetWebSearchPricePerCall(v)
return _c
}
// SetNillableWebSearchPricePerCall sets the "web_search_price_per_call" field if the given value is not nil.
func (_c *GroupCreate) SetNillableWebSearchPricePerCall(v *float64) *GroupCreate {
if v != nil {
_c.SetWebSearchPricePerCall(*v)
}
return _c
}
// SetClaudeCodeOnly sets the "claude_code_only" field.
func (_c *GroupCreate) SetClaudeCodeOnly(v bool) *GroupCreate {
_c.mutation.SetClaudeCodeOnly(v)
@@ -1218,6 +1232,10 @@ func (_c *GroupCreate) createSpec() (*Group, *sqlgraph.CreateSpec) {
_spec.SetField(group.FieldVideoPrice1080p, field.TypeFloat64, value)
_node.VideoPrice1080p = &value
}
if value, ok := _c.mutation.WebSearchPricePerCall(); ok {
_spec.SetField(group.FieldWebSearchPricePerCall, field.TypeFloat64, value)
_node.WebSearchPricePerCall = &value
}
if value, ok := _c.mutation.ClaudeCodeOnly(); ok {
_spec.SetField(group.FieldClaudeCodeOnly, field.TypeBool, value)
_node.ClaudeCodeOnly = value
@@ -1968,6 +1986,30 @@ func (u *GroupUpsert) ClearVideoPrice1080p() *GroupUpsert {
return u
}
// SetWebSearchPricePerCall sets the "web_search_price_per_call" field.
func (u *GroupUpsert) SetWebSearchPricePerCall(v float64) *GroupUpsert {
u.Set(group.FieldWebSearchPricePerCall, v)
return u
}
// UpdateWebSearchPricePerCall sets the "web_search_price_per_call" field to the value that was provided on create.
func (u *GroupUpsert) UpdateWebSearchPricePerCall() *GroupUpsert {
u.SetExcluded(group.FieldWebSearchPricePerCall)
return u
}
// AddWebSearchPricePerCall adds v to the "web_search_price_per_call" field.
func (u *GroupUpsert) AddWebSearchPricePerCall(v float64) *GroupUpsert {
u.Add(group.FieldWebSearchPricePerCall, v)
return u
}
// ClearWebSearchPricePerCall clears the value of the "web_search_price_per_call" field.
func (u *GroupUpsert) ClearWebSearchPricePerCall() *GroupUpsert {
u.SetNull(group.FieldWebSearchPricePerCall)
return u
}
// SetClaudeCodeOnly sets the "claude_code_only" field.
func (u *GroupUpsert) SetClaudeCodeOnly(v bool) *GroupUpsert {
u.Set(group.FieldClaudeCodeOnly, v)
@@ -2858,6 +2900,34 @@ func (u *GroupUpsertOne) ClearVideoPrice1080p() *GroupUpsertOne {
})
}
// SetWebSearchPricePerCall sets the "web_search_price_per_call" field.
func (u *GroupUpsertOne) SetWebSearchPricePerCall(v float64) *GroupUpsertOne {
return u.Update(func(s *GroupUpsert) {
s.SetWebSearchPricePerCall(v)
})
}
// AddWebSearchPricePerCall adds v to the "web_search_price_per_call" field.
func (u *GroupUpsertOne) AddWebSearchPricePerCall(v float64) *GroupUpsertOne {
return u.Update(func(s *GroupUpsert) {
s.AddWebSearchPricePerCall(v)
})
}
// UpdateWebSearchPricePerCall sets the "web_search_price_per_call" field to the value that was provided on create.
func (u *GroupUpsertOne) UpdateWebSearchPricePerCall() *GroupUpsertOne {
return u.Update(func(s *GroupUpsert) {
s.UpdateWebSearchPricePerCall()
})
}
// ClearWebSearchPricePerCall clears the value of the "web_search_price_per_call" field.
func (u *GroupUpsertOne) ClearWebSearchPricePerCall() *GroupUpsertOne {
return u.Update(func(s *GroupUpsert) {
s.ClearWebSearchPricePerCall()
})
}
// SetClaudeCodeOnly sets the "claude_code_only" field.
func (u *GroupUpsertOne) SetClaudeCodeOnly(v bool) *GroupUpsertOne {
return u.Update(func(s *GroupUpsert) {
@@ -3951,6 +4021,34 @@ func (u *GroupUpsertBulk) ClearVideoPrice1080p() *GroupUpsertBulk {
})
}
// SetWebSearchPricePerCall sets the "web_search_price_per_call" field.
func (u *GroupUpsertBulk) SetWebSearchPricePerCall(v float64) *GroupUpsertBulk {
return u.Update(func(s *GroupUpsert) {
s.SetWebSearchPricePerCall(v)
})
}
// AddWebSearchPricePerCall adds v to the "web_search_price_per_call" field.
func (u *GroupUpsertBulk) AddWebSearchPricePerCall(v float64) *GroupUpsertBulk {
return u.Update(func(s *GroupUpsert) {
s.AddWebSearchPricePerCall(v)
})
}
// UpdateWebSearchPricePerCall sets the "web_search_price_per_call" field to the value that was provided on create.
func (u *GroupUpsertBulk) UpdateWebSearchPricePerCall() *GroupUpsertBulk {
return u.Update(func(s *GroupUpsert) {
s.UpdateWebSearchPricePerCall()
})
}
// ClearWebSearchPricePerCall clears the value of the "web_search_price_per_call" field.
func (u *GroupUpsertBulk) ClearWebSearchPricePerCall() *GroupUpsertBulk {
return u.Update(func(s *GroupUpsert) {
s.ClearWebSearchPricePerCall()
})
}
// SetClaudeCodeOnly sets the "claude_code_only" field.
func (u *GroupUpsertBulk) SetClaudeCodeOnly(v bool) *GroupUpsertBulk {
return u.Update(func(s *GroupUpsert) {
+72
View File
@@ -640,6 +640,33 @@ func (_u *GroupUpdate) ClearVideoPrice1080p() *GroupUpdate {
return _u
}
// SetWebSearchPricePerCall sets the "web_search_price_per_call" field.
func (_u *GroupUpdate) SetWebSearchPricePerCall(v float64) *GroupUpdate {
_u.mutation.ResetWebSearchPricePerCall()
_u.mutation.SetWebSearchPricePerCall(v)
return _u
}
// SetNillableWebSearchPricePerCall sets the "web_search_price_per_call" field if the given value is not nil.
func (_u *GroupUpdate) SetNillableWebSearchPricePerCall(v *float64) *GroupUpdate {
if v != nil {
_u.SetWebSearchPricePerCall(*v)
}
return _u
}
// AddWebSearchPricePerCall adds value to the "web_search_price_per_call" field.
func (_u *GroupUpdate) AddWebSearchPricePerCall(v float64) *GroupUpdate {
_u.mutation.AddWebSearchPricePerCall(v)
return _u
}
// ClearWebSearchPricePerCall clears the value of the "web_search_price_per_call" field.
func (_u *GroupUpdate) ClearWebSearchPricePerCall() *GroupUpdate {
_u.mutation.ClearWebSearchPricePerCall()
return _u
}
// SetClaudeCodeOnly sets the "claude_code_only" field.
func (_u *GroupUpdate) SetClaudeCodeOnly(v bool) *GroupUpdate {
_u.mutation.SetClaudeCodeOnly(v)
@@ -1375,6 +1402,15 @@ func (_u *GroupUpdate) sqlSave(ctx context.Context) (_node int, err error) {
if _u.mutation.VideoPrice1080pCleared() {
_spec.ClearField(group.FieldVideoPrice1080p, field.TypeFloat64)
}
if value, ok := _u.mutation.WebSearchPricePerCall(); ok {
_spec.SetField(group.FieldWebSearchPricePerCall, field.TypeFloat64, value)
}
if value, ok := _u.mutation.AddedWebSearchPricePerCall(); ok {
_spec.AddField(group.FieldWebSearchPricePerCall, field.TypeFloat64, value)
}
if _u.mutation.WebSearchPricePerCallCleared() {
_spec.ClearField(group.FieldWebSearchPricePerCall, field.TypeFloat64)
}
if value, ok := _u.mutation.ClaudeCodeOnly(); ok {
_spec.SetField(group.FieldClaudeCodeOnly, field.TypeBool, value)
}
@@ -2364,6 +2400,33 @@ func (_u *GroupUpdateOne) ClearVideoPrice1080p() *GroupUpdateOne {
return _u
}
// SetWebSearchPricePerCall sets the "web_search_price_per_call" field.
func (_u *GroupUpdateOne) SetWebSearchPricePerCall(v float64) *GroupUpdateOne {
_u.mutation.ResetWebSearchPricePerCall()
_u.mutation.SetWebSearchPricePerCall(v)
return _u
}
// SetNillableWebSearchPricePerCall sets the "web_search_price_per_call" field if the given value is not nil.
func (_u *GroupUpdateOne) SetNillableWebSearchPricePerCall(v *float64) *GroupUpdateOne {
if v != nil {
_u.SetWebSearchPricePerCall(*v)
}
return _u
}
// AddWebSearchPricePerCall adds value to the "web_search_price_per_call" field.
func (_u *GroupUpdateOne) AddWebSearchPricePerCall(v float64) *GroupUpdateOne {
_u.mutation.AddWebSearchPricePerCall(v)
return _u
}
// ClearWebSearchPricePerCall clears the value of the "web_search_price_per_call" field.
func (_u *GroupUpdateOne) ClearWebSearchPricePerCall() *GroupUpdateOne {
_u.mutation.ClearWebSearchPricePerCall()
return _u
}
// SetClaudeCodeOnly sets the "claude_code_only" field.
func (_u *GroupUpdateOne) SetClaudeCodeOnly(v bool) *GroupUpdateOne {
_u.mutation.SetClaudeCodeOnly(v)
@@ -3129,6 +3192,15 @@ func (_u *GroupUpdateOne) sqlSave(ctx context.Context) (_node *Group, err error)
if _u.mutation.VideoPrice1080pCleared() {
_spec.ClearField(group.FieldVideoPrice1080p, field.TypeFloat64)
}
if value, ok := _u.mutation.WebSearchPricePerCall(); ok {
_spec.SetField(group.FieldWebSearchPricePerCall, field.TypeFloat64, value)
}
if value, ok := _u.mutation.AddedWebSearchPricePerCall(); ok {
_spec.AddField(group.FieldWebSearchPricePerCall, field.TypeFloat64, value)
}
if _u.mutation.WebSearchPricePerCallCleared() {
_spec.ClearField(group.FieldWebSearchPricePerCall, field.TypeFloat64)
}
if value, ok := _u.mutation.ClaudeCodeOnly(); ok {
_spec.SetField(group.FieldClaudeCodeOnly, field.TypeBool, value)
}
+2 -1
View File
@@ -865,6 +865,7 @@ var (
{Name: "video_price_480p", Type: field.TypeFloat64, Nullable: true, SchemaType: map[string]string{"postgres": "decimal(20,8)"}},
{Name: "video_price_720p", Type: field.TypeFloat64, Nullable: true, SchemaType: map[string]string{"postgres": "decimal(20,8)"}},
{Name: "video_price_1080p", Type: field.TypeFloat64, Nullable: true, SchemaType: map[string]string{"postgres": "decimal(20,8)"}},
{Name: "web_search_price_per_call", Type: field.TypeFloat64, Nullable: true, SchemaType: map[string]string{"postgres": "decimal(20,8)"}},
{Name: "claude_code_only", Type: field.TypeBool, Default: false},
{Name: "fallback_group_id", Type: field.TypeInt64, Nullable: true},
{Name: "fallback_group_id_on_invalid_request", Type: field.TypeInt64, Nullable: true},
@@ -915,7 +916,7 @@ var (
{
Name: "group_sort_order",
Unique: false,
Columns: []*schema.Column{GroupsColumns[40]},
Columns: []*schema.Column{GroupsColumns[41]},
},
},
}
+108 -1
View File
@@ -20842,6 +20842,8 @@ type GroupMutation struct {
addvideo_price_720p *float64
video_price_1080p *float64
addvideo_price_1080p *float64
web_search_price_per_call *float64
addweb_search_price_per_call *float64
claude_code_only *bool
fallback_group_id *int64
addfallback_group_id *int64
@@ -22608,6 +22610,76 @@ func (m *GroupMutation) ResetVideoPrice1080p() {
delete(m.clearedFields, group.FieldVideoPrice1080p)
}
// SetWebSearchPricePerCall sets the "web_search_price_per_call" field.
func (m *GroupMutation) SetWebSearchPricePerCall(f float64) {
m.web_search_price_per_call = &f
m.addweb_search_price_per_call = nil
}
// WebSearchPricePerCall returns the value of the "web_search_price_per_call" field in the mutation.
func (m *GroupMutation) WebSearchPricePerCall() (r float64, exists bool) {
v := m.web_search_price_per_call
if v == nil {
return
}
return *v, true
}
// OldWebSearchPricePerCall returns the old "web_search_price_per_call" field's value of the Group entity.
// If the Group object wasn't provided to the builder, the object is fetched from the database.
// An error is returned if the mutation operation is not UpdateOne, or the database query fails.
func (m *GroupMutation) OldWebSearchPricePerCall(ctx context.Context) (v *float64, err error) {
if !m.op.Is(OpUpdateOne) {
return v, errors.New("OldWebSearchPricePerCall is only allowed on UpdateOne operations")
}
if m.id == nil || m.oldValue == nil {
return v, errors.New("OldWebSearchPricePerCall requires an ID field in the mutation")
}
oldValue, err := m.oldValue(ctx)
if err != nil {
return v, fmt.Errorf("querying old value for OldWebSearchPricePerCall: %w", err)
}
return oldValue.WebSearchPricePerCall, nil
}
// AddWebSearchPricePerCall adds f to the "web_search_price_per_call" field.
func (m *GroupMutation) AddWebSearchPricePerCall(f float64) {
if m.addweb_search_price_per_call != nil {
*m.addweb_search_price_per_call += f
} else {
m.addweb_search_price_per_call = &f
}
}
// AddedWebSearchPricePerCall returns the value that was added to the "web_search_price_per_call" field in this mutation.
func (m *GroupMutation) AddedWebSearchPricePerCall() (r float64, exists bool) {
v := m.addweb_search_price_per_call
if v == nil {
return
}
return *v, true
}
// ClearWebSearchPricePerCall clears the value of the "web_search_price_per_call" field.
func (m *GroupMutation) ClearWebSearchPricePerCall() {
m.web_search_price_per_call = nil
m.addweb_search_price_per_call = nil
m.clearedFields[group.FieldWebSearchPricePerCall] = struct{}{}
}
// WebSearchPricePerCallCleared returns if the "web_search_price_per_call" field was cleared in this mutation.
func (m *GroupMutation) WebSearchPricePerCallCleared() bool {
_, ok := m.clearedFields[group.FieldWebSearchPricePerCall]
return ok
}
// ResetWebSearchPricePerCall resets all changes to the "web_search_price_per_call" field.
func (m *GroupMutation) ResetWebSearchPricePerCall() {
m.web_search_price_per_call = nil
m.addweb_search_price_per_call = nil
delete(m.clearedFields, group.FieldWebSearchPricePerCall)
}
// SetClaudeCodeOnly sets the "claude_code_only" field.
func (m *GroupMutation) SetClaudeCodeOnly(b bool) {
m.claude_code_only = &b
@@ -23642,7 +23714,7 @@ func (m *GroupMutation) Type() string {
// order to get all numeric fields that were incremented/decremented, call
// AddedFields().
func (m *GroupMutation) Fields() []string {
fields := make([]string, 0, 47)
fields := make([]string, 0, 48)
if m.created_at != nil {
fields = append(fields, group.FieldCreatedAt)
}
@@ -23739,6 +23811,9 @@ func (m *GroupMutation) Fields() []string {
if m.video_price_1080p != nil {
fields = append(fields, group.FieldVideoPrice1080p)
}
if m.web_search_price_per_call != nil {
fields = append(fields, group.FieldWebSearchPricePerCall)
}
if m.claude_code_only != nil {
fields = append(fields, group.FieldClaudeCodeOnly)
}
@@ -23856,6 +23931,8 @@ func (m *GroupMutation) Field(name string) (ent.Value, bool) {
return m.VideoPrice720p()
case group.FieldVideoPrice1080p:
return m.VideoPrice1080p()
case group.FieldWebSearchPricePerCall:
return m.WebSearchPricePerCall()
case group.FieldClaudeCodeOnly:
return m.ClaudeCodeOnly()
case group.FieldFallbackGroupID:
@@ -23959,6 +24036,8 @@ func (m *GroupMutation) OldField(ctx context.Context, name string) (ent.Value, e
return m.OldVideoPrice720p(ctx)
case group.FieldVideoPrice1080p:
return m.OldVideoPrice1080p(ctx)
case group.FieldWebSearchPricePerCall:
return m.OldWebSearchPricePerCall(ctx)
case group.FieldClaudeCodeOnly:
return m.OldClaudeCodeOnly(ctx)
case group.FieldFallbackGroupID:
@@ -24222,6 +24301,13 @@ func (m *GroupMutation) SetField(name string, value ent.Value) error {
}
m.SetVideoPrice1080p(v)
return nil
case group.FieldWebSearchPricePerCall:
v, ok := value.(float64)
if !ok {
return fmt.Errorf("unexpected type %T for field %s", value, name)
}
m.SetWebSearchPricePerCall(v)
return nil
case group.FieldClaudeCodeOnly:
v, ok := value.(bool)
if !ok {
@@ -24383,6 +24469,9 @@ func (m *GroupMutation) AddedFields() []string {
if m.addvideo_price_1080p != nil {
fields = append(fields, group.FieldVideoPrice1080p)
}
if m.addweb_search_price_per_call != nil {
fields = append(fields, group.FieldWebSearchPricePerCall)
}
if m.addfallback_group_id != nil {
fields = append(fields, group.FieldFallbackGroupID)
}
@@ -24435,6 +24524,8 @@ func (m *GroupMutation) AddedField(name string) (ent.Value, bool) {
return m.AddedVideoPrice720p()
case group.FieldVideoPrice1080p:
return m.AddedVideoPrice1080p()
case group.FieldWebSearchPricePerCall:
return m.AddedWebSearchPricePerCall()
case group.FieldFallbackGroupID:
return m.AddedFallbackGroupID()
case group.FieldFallbackGroupIDOnInvalidRequest:
@@ -24564,6 +24655,13 @@ func (m *GroupMutation) AddField(name string, value ent.Value) error {
}
m.AddVideoPrice1080p(v)
return nil
case group.FieldWebSearchPricePerCall:
v, ok := value.(float64)
if !ok {
return fmt.Errorf("unexpected type %T for field %s", value, name)
}
m.AddWebSearchPricePerCall(v)
return nil
case group.FieldFallbackGroupID:
v, ok := value.(int64)
if !ok {
@@ -24633,6 +24731,9 @@ func (m *GroupMutation) ClearedFields() []string {
if m.FieldCleared(group.FieldVideoPrice1080p) {
fields = append(fields, group.FieldVideoPrice1080p)
}
if m.FieldCleared(group.FieldWebSearchPricePerCall) {
fields = append(fields, group.FieldWebSearchPricePerCall)
}
if m.FieldCleared(group.FieldFallbackGroupID) {
fields = append(fields, group.FieldFallbackGroupID)
}
@@ -24689,6 +24790,9 @@ func (m *GroupMutation) ClearField(name string) error {
case group.FieldVideoPrice1080p:
m.ClearVideoPrice1080p()
return nil
case group.FieldWebSearchPricePerCall:
m.ClearWebSearchPricePerCall()
return nil
case group.FieldFallbackGroupID:
m.ClearFallbackGroupID()
return nil
@@ -24802,6 +24906,9 @@ func (m *GroupMutation) ResetField(name string) error {
case group.FieldVideoPrice1080p:
m.ResetVideoPrice1080p()
return nil
case group.FieldWebSearchPricePerCall:
m.ResetWebSearchPricePerCall()
return nil
case group.FieldClaudeCodeOnly:
m.ResetClaudeCodeOnly()
return nil
+12 -12
View File
@@ -1044,53 +1044,53 @@ func init() {
// group.DefaultVideoRateMultiplier holds the default value on creation for the video_rate_multiplier field.
group.DefaultVideoRateMultiplier = groupDescVideoRateMultiplier.Default.(float64)
// groupDescClaudeCodeOnly is the schema descriptor for claude_code_only field.
groupDescClaudeCodeOnly := groupFields[29].Descriptor()
groupDescClaudeCodeOnly := groupFields[30].Descriptor()
// group.DefaultClaudeCodeOnly holds the default value on creation for the claude_code_only field.
group.DefaultClaudeCodeOnly = groupDescClaudeCodeOnly.Default.(bool)
// groupDescModelRoutingEnabled is the schema descriptor for model_routing_enabled field.
groupDescModelRoutingEnabled := groupFields[33].Descriptor()
groupDescModelRoutingEnabled := groupFields[34].Descriptor()
// group.DefaultModelRoutingEnabled holds the default value on creation for the model_routing_enabled field.
group.DefaultModelRoutingEnabled = groupDescModelRoutingEnabled.Default.(bool)
// groupDescMcpXMLInject is the schema descriptor for mcp_xml_inject field.
groupDescMcpXMLInject := groupFields[34].Descriptor()
groupDescMcpXMLInject := groupFields[35].Descriptor()
// group.DefaultMcpXMLInject holds the default value on creation for the mcp_xml_inject field.
group.DefaultMcpXMLInject = groupDescMcpXMLInject.Default.(bool)
// groupDescSupportedModelScopes is the schema descriptor for supported_model_scopes field.
groupDescSupportedModelScopes := groupFields[35].Descriptor()
groupDescSupportedModelScopes := groupFields[36].Descriptor()
// group.DefaultSupportedModelScopes holds the default value on creation for the supported_model_scopes field.
group.DefaultSupportedModelScopes = groupDescSupportedModelScopes.Default.([]string)
// groupDescSortOrder is the schema descriptor for sort_order field.
groupDescSortOrder := groupFields[36].Descriptor()
groupDescSortOrder := groupFields[37].Descriptor()
// group.DefaultSortOrder holds the default value on creation for the sort_order field.
group.DefaultSortOrder = groupDescSortOrder.Default.(int)
// groupDescAllowMessagesDispatch is the schema descriptor for allow_messages_dispatch field.
groupDescAllowMessagesDispatch := groupFields[37].Descriptor()
groupDescAllowMessagesDispatch := groupFields[38].Descriptor()
// group.DefaultAllowMessagesDispatch holds the default value on creation for the allow_messages_dispatch field.
group.DefaultAllowMessagesDispatch = groupDescAllowMessagesDispatch.Default.(bool)
// groupDescRequireOauthOnly is the schema descriptor for require_oauth_only field.
groupDescRequireOauthOnly := groupFields[38].Descriptor()
groupDescRequireOauthOnly := groupFields[39].Descriptor()
// group.DefaultRequireOauthOnly holds the default value on creation for the require_oauth_only field.
group.DefaultRequireOauthOnly = groupDescRequireOauthOnly.Default.(bool)
// groupDescRequirePrivacySet is the schema descriptor for require_privacy_set field.
groupDescRequirePrivacySet := groupFields[39].Descriptor()
groupDescRequirePrivacySet := groupFields[40].Descriptor()
// group.DefaultRequirePrivacySet holds the default value on creation for the require_privacy_set field.
group.DefaultRequirePrivacySet = groupDescRequirePrivacySet.Default.(bool)
// groupDescDefaultMappedModel is the schema descriptor for default_mapped_model field.
groupDescDefaultMappedModel := groupFields[40].Descriptor()
groupDescDefaultMappedModel := groupFields[41].Descriptor()
// group.DefaultDefaultMappedModel holds the default value on creation for the default_mapped_model field.
group.DefaultDefaultMappedModel = groupDescDefaultMappedModel.Default.(string)
// group.DefaultMappedModelValidator is a validator for the "default_mapped_model" field. It is called by the builders before save.
group.DefaultMappedModelValidator = groupDescDefaultMappedModel.Validators[0].(func(string) error)
// groupDescMessagesDispatchModelConfig is the schema descriptor for messages_dispatch_model_config field.
groupDescMessagesDispatchModelConfig := groupFields[41].Descriptor()
groupDescMessagesDispatchModelConfig := groupFields[42].Descriptor()
// group.DefaultMessagesDispatchModelConfig holds the default value on creation for the messages_dispatch_model_config field.
group.DefaultMessagesDispatchModelConfig = groupDescMessagesDispatchModelConfig.Default.(domain.OpenAIMessagesDispatchModelConfig)
// groupDescModelsListConfig is the schema descriptor for models_list_config field.
groupDescModelsListConfig := groupFields[42].Descriptor()
groupDescModelsListConfig := groupFields[43].Descriptor()
// group.DefaultModelsListConfig holds the default value on creation for the models_list_config field.
group.DefaultModelsListConfig = groupDescModelsListConfig.Default.(domain.GroupModelsListConfig)
// groupDescRpmLimit is the schema descriptor for rpm_limit field.
groupDescRpmLimit := groupFields[43].Descriptor()
groupDescRpmLimit := groupFields[44].Descriptor()
// group.DefaultRpmLimit holds the default value on creation for the rpm_limit field.
group.DefaultRpmLimit = groupDescRpmLimit.Default.(int)
idempotencyrecordMixin := schema.IdempotencyRecord{}.Mixin()
+5
View File
@@ -142,6 +142,11 @@ func (Group) Fields() []ent.Field {
Optional().
Nillable().
SchemaType(map[string]string{dialect.Postgres: "decimal(20,8)"}),
field.Float("web_search_price_per_call").
Optional().
Nillable().
SchemaType(map[string]string{dialect.Postgres: "decimal(20,8)"}).
Comment("Codex alpha/search 网页搜索单次价格(USD/次);nil 表示使用默认价 0.01(官方 $10/1000 次)"),
// Claude Code 客户端限制 (added by migration 029)
field.Bool("claude_code_only").
+1 -1
View File
@@ -44,6 +44,7 @@ require (
go.uber.org/zap v1.24.0
golang.org/x/crypto v0.51.0
golang.org/x/image v0.39.0
golang.org/x/mod v0.35.0
golang.org/x/net v0.55.0
golang.org/x/sync v0.20.0
golang.org/x/term v0.43.0
@@ -176,7 +177,6 @@ require (
go.uber.org/multierr v1.9.0 // indirect
golang.org/x/arch v0.3.0 // indirect
golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect
golang.org/x/mod v0.35.0 // indirect
golang.org/x/sys v0.45.0 // indirect
golang.org/x/text v0.37.0 // indirect
golang.org/x/tools v0.44.0 // indirect
+8
View File
@@ -220,6 +220,8 @@ github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovk
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-runewidth v0.0.15 h1:UNAjwbU9l54TA3KzvqLGxwWjHmMgBUVhBiTjelZgg3U=
github.com/mattn/go-runewidth v0.0.15/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-sqlite3 v1.14.17 h1:mCRHCLDUBXgpKAqIKsaAaAsrAlbkeomtRFKXh2L6YIM=
github.com/mattn/go-sqlite3 v1.14.17/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S2DGjv9HUNg=
github.com/mdelapenya/tlscert v0.2.0 h1:7H81W6Z/4weDvZBNOfQte5GpIMo0lGYEeWbkGp5LJHI=
@@ -253,6 +255,8 @@ github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec=
github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
@@ -282,6 +286,8 @@ github.com/refraction-networking/utls v1.8.2 h1:j4Q1gJj0xngdeH+Ox/qND11aEfhpgoEv
github.com/refraction-networking/utls v1.8.2/go.mod h1:jkSOEkLqn+S/jtpEHPOsVv/4V4EVnelwbMQl4vCWXAM=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII=
@@ -314,6 +320,8 @@ github.com/spf13/afero v1.11.0 h1:WJQKhtpdm3v2IzqG8VMqrr6Rf3UYpEF239Jy9wNepM8=
github.com/spf13/afero v1.11.0/go.mod h1:GH9Y3pIexgf1MTIWtNGyogA5MwRIDXGUr+hbWNoBjkY=
github.com/spf13/cast v1.6.0 h1:GEiTHELF+vaR5dhz3VqZfFSzZjYbgeKDpBxQVS4GYJ0=
github.com/spf13/cast v1.6.0/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I=
github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.18.2 h1:LUXCnvUvSM6FXAsj6nnfc8Q2tp1dIgUfY9Kc8GsSOiQ=
@@ -98,7 +98,7 @@ func TestGrokOAuthHandlerQueryQuotaProbesUpstream(t *testing.T) {
require.Contains(t, rec.Body.String(), `"source":"active_probe"`)
require.Contains(t, rec.Body.String(), `"headers_observed":true`)
require.NotContains(t, rec.Body.String(), "access-token")
require.Equal(t, xai.DefaultBaseURL+"/responses", upstream.lastReq.URL.String())
require.Equal(t, xai.DefaultCLIBaseURL+"/responses", upstream.lastReq.URL.String())
require.Equal(t, "Bearer access-token", upstream.lastReq.Header.Get("Authorization"))
require.Contains(t, string(upstream.lastBody), `"store":false`)
require.NotNil(t, repo.updates[42])
@@ -110,6 +110,7 @@ type CreateGroupRequest struct {
VideoPrice480P *float64 `json:"video_price_480p"`
VideoPrice720P *float64 `json:"video_price_720p"`
VideoPrice1080P *float64 `json:"video_price_1080p"`
WebSearchPricePerCall *float64 `json:"web_search_price_per_call"`
ClaudeCodeOnly bool `json:"claude_code_only"`
FallbackGroupID *int64 `json:"fallback_group_id"`
FallbackGroupIDOnInvalidRequest *int64 `json:"fallback_group_id_on_invalid_request"`
@@ -163,6 +164,7 @@ type UpdateGroupRequest struct {
VideoPrice480P *float64 `json:"video_price_480p"`
VideoPrice720P *float64 `json:"video_price_720p"`
VideoPrice1080P *float64 `json:"video_price_1080p"`
WebSearchPricePerCall *float64 `json:"web_search_price_per_call"`
ClaudeCodeOnly *bool `json:"claude_code_only"`
FallbackGroupID *int64 `json:"fallback_group_id"`
FallbackGroupIDOnInvalidRequest *int64 `json:"fallback_group_id_on_invalid_request"`
@@ -334,6 +336,7 @@ func (h *GroupHandler) Create(c *gin.Context) {
VideoPrice480P: req.VideoPrice480P,
VideoPrice720P: req.VideoPrice720P,
VideoPrice1080P: req.VideoPrice1080P,
WebSearchPricePerCall: req.WebSearchPricePerCall,
ClaudeCodeOnly: req.ClaudeCodeOnly,
FallbackGroupID: req.FallbackGroupID,
FallbackGroupIDOnInvalidRequest: req.FallbackGroupIDOnInvalidRequest,
@@ -402,6 +405,7 @@ func (h *GroupHandler) Update(c *gin.Context) {
VideoPrice480P: req.VideoPrice480P,
VideoPrice720P: req.VideoPrice720P,
VideoPrice1080P: req.VideoPrice1080P,
WebSearchPricePerCall: req.WebSearchPricePerCall,
ClaudeCodeOnly: req.ClaudeCodeOnly,
FallbackGroupID: req.FallbackGroupID,
FallbackGroupIDOnInvalidRequest: req.FallbackGroupIDOnInvalidRequest,
+1
View File
@@ -199,6 +199,7 @@ func groupFromServiceBase(g *service.Group) Group {
VideoPrice480P: g.VideoPrice480P,
VideoPrice720P: g.VideoPrice720P,
VideoPrice1080P: g.VideoPrice1080P,
WebSearchPricePerCall: g.WebSearchPricePerCall,
ClaudeCodeOnly: g.ClaudeCodeOnly,
FallbackGroupID: g.FallbackGroupID,
FallbackGroupIDOnInvalidRequest: g.FallbackGroupIDOnInvalidRequest,
+2
View File
@@ -120,6 +120,8 @@ type Group struct {
VideoPrice480P *float64 `json:"video_price_480p"`
VideoPrice720P *float64 `json:"video_price_720p"`
VideoPrice1080P *float64 `json:"video_price_1080p"`
// Codex alpha/search 网页搜索单次价格(USD/次);null 表示使用默认价 0.01
WebSearchPricePerCall *float64 `json:"web_search_price_per_call"`
// Claude Code 客户端限制
ClaudeCodeOnly bool `json:"claude_code_only"`
+9 -3
View File
@@ -18,6 +18,7 @@ const (
EndpointMessages = "/v1/messages"
EndpointChatCompletions = "/v1/chat/completions"
EndpointEmbeddings = "/v1/embeddings"
EndpointAlphaSearch = "/v1/alpha/search"
EndpointResponses = "/v1/responses"
EndpointResponsesCompact = "/v1/responses/compact"
EndpointImagesGenerations = "/v1/images/generations"
@@ -75,6 +76,8 @@ func NormalizeInboundEndpoint(path string) string {
switch {
case strings.Contains(path, EndpointEmbeddings):
return EndpointEmbeddings
case strings.Contains(path, EndpointAlphaSearch) || isBareOrSubpathOf(strings.TrimRight(path, "/"), "/alpha/search") || isBareOrSubpathOf(strings.TrimRight(path, "/"), "/backend-api/codex/alpha/search"):
return EndpointAlphaSearch
case strings.Contains(path, EndpointChatCompletions):
return EndpointChatCompletions
case strings.Contains(path, EndpointMessages):
@@ -155,8 +158,11 @@ func isBareOrSubpathOf(path, root string) bool {
// account platform and the normalized inbound endpoint.
//
// Platform-specific rules:
// - OpenAI always forwards to /v1/responses (with optional subpath
// such as /v1/responses/compact preserved from the raw URL).
// - OpenAI and Grok text compatibility routes forward to /v1/responses
// (with optional subpath such as /v1/responses/compact preserved from
// the raw URL); native endpoints such as embeddings and alpha search
// retain their paths. Grok raw Chat requests override this through the
// forwarding result consumed by resolveOpenAIUpstreamEndpoint.
// - Anthropic → /v1/messages
// - Gemini → /v1beta/models
// - Antigravity → /v1/messages (Claude) or gemini (Gemini)
@@ -167,7 +173,7 @@ func DeriveUpstreamEndpoint(inbound, rawRequestPath, platform string) string {
switch platform {
case service.PlatformOpenAI, service.PlatformGrok:
if inbound == EndpointEmbeddings || inbound == EndpointImagesGenerations || inbound == EndpointImagesEdits || inbound == EndpointVideosGenerations || inbound == EndpointVideos {
if inbound == EndpointEmbeddings || inbound == EndpointAlphaSearch || inbound == EndpointImagesGenerations || inbound == EndpointImagesEdits || inbound == EndpointVideosGenerations || inbound == EndpointVideos {
return inbound
}
// OpenAI forwards everything to the Responses API.
+59
View File
@@ -25,6 +25,7 @@ func TestNormalizeInboundEndpoint(t *testing.T) {
{"/v1/messages", EndpointMessages},
{"/v1/chat/completions", EndpointChatCompletions},
{"/v1/embeddings", EndpointEmbeddings},
{"/v1/alpha/search", EndpointAlphaSearch},
{"/v1/responses", EndpointResponses},
{"/v1/responses/compact", EndpointResponsesCompact},
{"/v1/responses/compact/detail", EndpointResponsesCompact},
@@ -50,11 +51,13 @@ func TestNormalizeInboundEndpoint(t *testing.T) {
{"/responses", EndpointResponses},
{"/responses/compact", EndpointResponsesCompact},
{"/responses/compact/detail", EndpointResponsesCompact},
{"/alpha/search", EndpointAlphaSearch},
// Bare Codex direct alias route — root vs. compact.
{"/backend-api/codex/responses", EndpointResponses},
{"/backend-api/codex/responses/compact", EndpointResponsesCompact},
{"/backend-api/codex/responses/compact/detail", EndpointResponsesCompact},
{"/backend-api/codex/alpha/search", EndpointAlphaSearch},
// Must NOT generalize to arbitrary paths merely ending in
// "/responses" (or "/responses/compact") that are unrelated to
@@ -119,8 +122,11 @@ func TestDeriveUpstreamEndpoint(t *testing.T) {
{"openai from messages", EndpointMessages, "/v1/messages", service.PlatformOpenAI, EndpointResponses},
{"openai from completions", EndpointChatCompletions, "/v1/chat/completions", service.PlatformOpenAI, EndpointResponses},
{"openai embeddings", EndpointEmbeddings, "/v1/embeddings", service.PlatformOpenAI, EndpointEmbeddings},
{"openai alpha search", EndpointAlphaSearch, "/backend-api/codex/alpha/search", service.PlatformOpenAI, EndpointAlphaSearch},
{"openai image generations", EndpointImagesGenerations, "/v1/images/generations", service.PlatformOpenAI, EndpointImagesGenerations},
{"openai image edits", EndpointImagesEdits, "/openai/v1/images/edits", service.PlatformOpenAI, EndpointImagesEdits},
{"grok chat defaults to responses without runtime result", EndpointChatCompletions, "/v1/chat/completions", service.PlatformGrok, EndpointResponses},
{"grok responses", EndpointResponses, "/v1/responses", service.PlatformGrok, EndpointResponses},
{"grok video generations", EndpointVideosGenerations, "/v1/videos/generations", service.PlatformGrok, EndpointVideosGenerations},
{"grok video status", EndpointVideos, "/videos/req_123", service.PlatformGrok, EndpointVideos},
@@ -138,6 +144,59 @@ func TestDeriveUpstreamEndpoint(t *testing.T) {
}
}
func TestResolveOpenAIUpstreamEndpointPrefersForwardResult(t *testing.T) {
tests := []struct {
name string
account *service.Account
result *service.OpenAIForwardResult
runtimeEndpoint string
want string
}{
{
name: "grok raw chat result overrides stale context",
account: &service.Account{Platform: service.PlatformGrok, Type: service.AccountTypeOAuth},
result: &service.OpenAIForwardResult{UpstreamEndpoint: EndpointChatCompletions},
runtimeEndpoint: EndpointResponses,
want: EndpointChatCompletions,
},
{
name: "grok chat bridged to responses",
account: &service.Account{Platform: service.PlatformGrok, Type: service.AccountTypeOAuth},
result: &service.OpenAIForwardResult{UpstreamEndpoint: EndpointResponses},
want: EndpointResponses,
},
{
name: "grok empty result keeps responses default",
account: &service.Account{Platform: service.PlatformGrok, Type: service.AccountTypeOAuth},
result: &service.OpenAIForwardResult{},
want: EndpointResponses,
},
{
name: "grok raw error uses runtime endpoint",
account: &service.Account{Platform: service.PlatformGrok, Type: service.AccountTypeOAuth},
runtimeEndpoint: EndpointChatCompletions,
want: EndpointChatCompletions,
},
{
name: "openai behavior remains responses",
account: &service.Account{Platform: service.PlatformOpenAI, Type: service.AccountTypeOAuth},
result: &service.OpenAIForwardResult{},
want: EndpointResponses,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
rec := httptest.NewRecorder()
c, _ := gin.CreateTestContext(rec)
c.Request = httptest.NewRequest(http.MethodPost, EndpointChatCompletions, nil)
c.Set(ctxKeyInboundEndpoint, EndpointChatCompletions)
service.SetActualOpenAIUpstreamEndpoint(c, tt.runtimeEndpoint)
require.Equal(t, tt.want, resolveOpenAIUpstreamEndpoint(c, tt.account, tt.result))
})
}
}
// ──────────────────────────────────────────────────────────
// responsesSubpathSuffix
// ──────────────────────────────────────────────────────────
@@ -107,3 +107,31 @@ func classifyNoAccountErrorFromGin(
}
return classifyNoAccountError(ctx, diag, apiKey, routingModel, displayModel, platform)
}
func classifyOpenAICompatibleNoAccountErrorFromGin(
c *gin.Context,
diag service.ModelAvailabilityDiagnoser,
apiKey *service.APIKey,
routingModel string,
displayModel string,
) noAccountErrorClassification {
return classifyNoAccountErrorFromGin(
c,
diag,
apiKey,
routingModel,
displayModel,
openAICompatibleRequestPlatform(apiKey),
)
}
func openAICompatibleSelectionErrorForLog(err error, platform string) error {
if err == nil || platform != service.PlatformGrok {
return err
}
message := strings.ReplaceAll(err.Error(), "OpenAI accounts", "Grok accounts")
if message == err.Error() {
return err
}
return fmt.Errorf("%s", message)
}
@@ -4,6 +4,7 @@ package handler
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"testing"
@@ -114,6 +115,33 @@ func TestClassifyNoAccountError_ModelNotSupported_Returns404(t *testing.T) {
require.Equal(t, int64(42), *fd.calls[0].GroupID)
}
func TestClassifyOpenAICompatibleNoAccountError_GrokUsesGrokPlatform(t *testing.T) {
c := newTestGinContextWithRequest()
fd := &fakeDiagnoser{resp: service.ModelAvailabilityDiagnosis{HasAccountsInPool: true, HasModelSupport: false}}
groupID := int64(43)
apiKey := &service.APIKey{
GroupID: &groupID,
Group: &service.Group{
ID: groupID,
Platform: service.PlatformGrok,
},
}
cls := classifyOpenAICompatibleNoAccountErrorFromGin(c, fd, apiKey, "grok-4.5", "grok-4.5")
require.Equal(t, http.StatusNotFound, cls.Status)
require.Equal(t, "model_not_found", cls.ErrType)
require.True(t, cls.ModelNotFound)
require.Len(t, fd.calls, 1)
require.Equal(t, service.PlatformGrok, fd.calls[0].Platform)
logErr := openAICompatibleSelectionErrorForLog(
fmt.Errorf("no available OpenAI accounts supporting model: grok-4.5"),
service.PlatformGrok,
)
require.EqualError(t, logErr, "no available Grok accounts supporting model: grok-4.5")
}
func TestClassifyNoAccountError_HasModelSupport_KeepsRoutingMessageGenerationToCaller(t *testing.T) {
c := newTestGinContextWithRequest()
fd := &fakeDiagnoser{resp: service.ModelAvailabilityDiagnosis{HasAccountsInPool: true, HasModelSupport: true}}
@@ -0,0 +1,250 @@
package handler
import (
"context"
"errors"
"net/http"
"strconv"
"strings"
"time"
pkghttputil "github.com/Wei-Shaw/sub2api/internal/pkg/httputil"
"github.com/Wei-Shaw/sub2api/internal/pkg/ip"
"github.com/Wei-Shaw/sub2api/internal/pkg/logger"
middleware2 "github.com/Wei-Shaw/sub2api/internal/server/middleware"
"github.com/Wei-Shaw/sub2api/internal/service"
"github.com/gin-gonic/gin"
"github.com/tidwall/gjson"
"go.uber.org/zap"
)
// AlphaSearch proxies the standalone search endpoint used by Codex Responses Lite.
func (h *OpenAIGatewayHandler) AlphaSearch(c *gin.Context) {
streamStarted := false
defer h.recoverResponsesPanic(c, &streamStarted)
setOpenAIClientTransportHTTP(c)
requestStart := time.Now()
apiKey, ok := middleware2.GetAPIKeyFromContext(c)
if !ok || apiKey.Group == nil {
h.errorResponse(c, http.StatusUnauthorized, "authentication_error", "Invalid API key")
return
}
if apiKey.Group.Platform != service.PlatformOpenAI {
h.errorResponse(c, http.StatusNotFound, "not_found_error", "Codex alpha search is only available for OpenAI groups")
return
}
subject, ok := middleware2.GetAuthSubjectFromContext(c)
if !ok {
h.errorResponse(c, http.StatusInternalServerError, "api_error", "User context not found")
return
}
reqLog := requestLogger(
c,
"handler.openai_gateway.alpha_search",
zap.Int64("user_id", subject.UserID),
zap.Int64("api_key_id", apiKey.ID),
zap.Any("group_id", apiKey.GroupID),
)
if !h.ensureResponsesDependencies(c, reqLog) {
return
}
body, err := pkghttputil.ReadRequestBodyWithPrealloc(c.Request)
if err != nil {
if maxErr, ok := extractMaxBytesError(err); ok {
h.errorResponse(c, http.StatusRequestEntityTooLarge, "invalid_request_error", buildBodyTooLargeMessage(maxErr.Limit))
return
}
h.errorResponse(c, http.StatusBadRequest, "invalid_request_error", "Failed to read request body")
return
}
if len(body) == 0 {
h.errorResponse(c, http.StatusBadRequest, "invalid_request_error", "Request body is empty")
return
}
if !gjson.ValidBytes(body) {
logRequestBodyParseFailure(reqLog, body, nil)
h.errorResponse(c, http.StatusBadRequest, "invalid_request_error", "Failed to parse request body")
return
}
modelResult := gjson.GetBytes(body, "model")
if !modelResult.Exists() || modelResult.Type != gjson.String || strings.TrimSpace(modelResult.String()) == "" {
h.errorResponse(c, http.StatusBadRequest, "invalid_request_error", "model is required")
return
}
requestedModel := strings.TrimSpace(modelResult.String())
reqLog = reqLog.With(zap.String("model", requestedModel))
setOpsRequestContext(c, requestedModel, false)
setOpsEndpointContext(c, "", int16(service.RequestTypeSync))
channelMapping, _ := h.gatewayService.ResolveChannelMappingAndRestrict(c.Request.Context(), apiKey.GroupID, requestedModel)
forwardBody := openAIModelMappedBody(body, channelMapping.Mapped, channelMapping.MappedModel, h.gatewayService.ReplaceModelInBody)
subscription, _ := middleware2.GetSubscriptionFromContext(c)
service.SetOpsLatencyMs(c, service.OpsAuthLatencyMsKey, time.Since(requestStart).Milliseconds())
userRelease, acquired := h.acquireResponsesUserSlot(c, subject.UserID, subject.Concurrency, false, &streamStarted, reqLog)
if !acquired {
return
}
if userRelease != nil {
defer userRelease()
}
if err := h.billingCacheService.CheckBillingEligibility(c.Request.Context(), apiKey.User, apiKey, apiKey.Group, subscription, service.QuotaPlatform(c.Request.Context(), apiKey)); err != nil {
status, code, message, retryAfter := billingErrorDetails(err)
if retryAfter > 0 {
c.Header("Retry-After", strconv.Itoa(retryAfter))
}
h.errorResponse(c, status, code, message)
return
}
searchID := strings.TrimSpace(gjson.GetBytes(body, "id").String())
sessionHash := h.gatewayService.GenerateSessionHashWithFallback(c, nil, searchID)
failedAccountIDs := make(map[int64]struct{})
var lastFailoverErr *service.UpstreamFailoverError
switchCount := 0
routingStart := time.Now()
for {
selection, _, err := h.gatewayService.SelectAccountWithSchedulerForCapability(
c.Request.Context(),
apiKey.GroupID,
"",
sessionHash,
requestedModel,
failedAccountIDs,
service.OpenAIUpstreamTransportHTTPSSE,
service.OpenAIEndpointCapabilityChatCompletions,
false,
false,
service.PlatformOpenAI,
)
if err != nil || selection == nil || selection.Account == nil {
if len(failedAccountIDs) == 0 {
cls := classifyNoAccountErrorFromGin(c, h.gatewayService, apiKey, requestedModel, requestedModel, service.PlatformOpenAI)
if !cls.ModelNotFound {
markOpsRoutingCapacityLimitedIfNoAvailable(c, err)
}
h.errorResponse(c, cls.Status, cls.ErrType, cls.Message)
return
}
if lastFailoverErr != nil {
h.handleFailoverExhausted(c, lastFailoverErr, false)
} else {
h.errorResponse(c, http.StatusBadGateway, "upstream_error", "Upstream request failed")
}
return
}
account := selection.Account
setOpsSelectedAccount(c, account.ID, account.Platform)
accountRelease, acquired := h.acquireResponsesAccountSlot(c, apiKey.GroupID, sessionHash, selection, false, &streamStarted, reqLog)
if !acquired {
return
}
service.SetOpsLatencyMs(c, service.OpsRoutingLatencyMsKey, time.Since(routingStart).Milliseconds())
writerSizeBeforeForward := c.Writer.Size()
forwardStart := time.Now()
var result *service.OpenAIForwardResult
result, err = func() (*service.OpenAIForwardResult, error) {
if accountRelease != nil {
defer accountRelease()
}
return h.gatewayService.ForwardAlphaSearch(c.Request.Context(), c, account, forwardBody)
}()
service.SetOpsLatencyMs(c, service.OpsResponseLatencyMsKey, time.Since(forwardStart).Milliseconds())
if err == nil {
h.gatewayService.ReportOpenAIAccountScheduleResult(account.ID, true, nil)
if result != nil {
h.recordAlphaSearchUsage(c, apiKey, account, subscription, channelMapping, requestedModel, body, result, subject.UserID)
}
return
}
var failoverErr *service.UpstreamFailoverError
if !errors.As(err, &failoverErr) {
h.gatewayService.ReportOpenAIAccountScheduleResult(account.ID, false, nil)
if c.Writer.Size() == writerSizeBeforeForward {
h.errorResponse(c, http.StatusBadGateway, "upstream_error", "Upstream request failed")
}
reqLog.Warn("openai_alpha_search.forward_failed", zap.Int64("account_id", account.ID), zap.Error(err))
return
}
h.gatewayService.ReportOpenAIAccountScheduleResult(account.ID, false, nil)
if c.Writer.Size() != writerSizeBeforeForward {
h.handleFailoverExhausted(c, failoverErr, true)
return
}
h.gatewayService.RecordOpenAIAccountSwitch()
failedAccountIDs[account.ID] = struct{}{}
lastFailoverErr = failoverErr
if switchCount >= h.maxAccountSwitches {
h.handleFailoverExhausted(c, failoverErr, false)
return
}
switchCount++
if h.gatewayService.ShouldStopOpenAIOAuth429Failover(account, failoverErr.StatusCode, switchCount) {
h.handleFailoverExhausted(c, failoverErr, false)
return
}
reqLog.Warn("openai_alpha_search.upstream_failover_switching",
zap.Int64("account_id", account.ID),
zap.Int("upstream_status", failoverErr.StatusCode),
zap.Int("switch_count", switchCount),
)
}
}
// recordAlphaSearchUsage 为一次成功的 alpha/search 网页搜索落按次计费用量行
// (上游不返回 usage 字段,按 WebSearchCalls 走分组单价 × 倍率的按次口径)。
// 与 images 一致使用 mandatory 池提交,池满时同步兜底执行,保证扣费不丢。
func (h *OpenAIGatewayHandler) recordAlphaSearchUsage(
c *gin.Context,
apiKey *service.APIKey,
account *service.Account,
subscription *service.UserSubscription,
channelMapping service.ChannelMappingResult,
requestedModel string,
body []byte,
result *service.OpenAIForwardResult,
userID int64,
) {
userAgent := c.GetHeader("User-Agent")
clientIP := ip.GetClientIP(c)
requestPayloadHash := service.HashUsageRequestPayload(body)
inboundEndpoint := GetInboundEndpoint(c)
upstreamEndpoint := GetUpstreamEndpoint(c, account.Platform)
quotaPlatform := service.QuotaPlatform(c.Request.Context(), apiKey)
h.submitMandatoryUsageRecordTask(c.Request.Context(), func(ctx context.Context) {
if err := h.gatewayService.RecordUsage(ctx, &service.OpenAIRecordUsageInput{
Result: result,
APIKey: apiKey,
User: apiKey.User,
Account: account,
Subscription: subscription,
InboundEndpoint: inboundEndpoint,
UpstreamEndpoint: upstreamEndpoint,
UserAgent: userAgent,
IPAddress: clientIP,
RequestPayloadHash: requestPayloadHash,
APIKeyService: h.apiKeyService,
QuotaPlatform: quotaPlatform,
ChannelUsageFields: channelMapping.ToUsageFields(requestedModel, result.UpstreamModel),
}); err != nil {
logger.L().With(
zap.String("component", "handler.openai_gateway.alpha_search"),
zap.Int64("user_id", userID),
zap.Int64("api_key_id", apiKey.ID),
zap.Any("group_id", apiKey.GroupID),
zap.String("model", requestedModel),
zap.Int64("account_id", account.ID),
).Error("openai_alpha_search.record_usage_failed", zap.Error(err))
}
})
}
@@ -5,6 +5,7 @@ import (
"errors"
"net/http"
"strconv"
"strings"
"time"
"github.com/Wei-Shaw/sub2api/internal/pkg/ip"
@@ -150,11 +151,11 @@ func (h *OpenAIGatewayHandler) ChatCompletions(c *gin.Context) {
)
if err != nil {
reqLog.Warn("openai_chat_completions.account_select_failed",
zap.Error(err),
zap.Error(openAICompatibleSelectionErrorForLog(err, requestPlatform)),
zap.Int("excluded_account_count", len(failedAccountIDs)),
)
if len(failedAccountIDs) == 0 {
cls := classifyNoAccountErrorFromGin(c, h.gatewayService, apiKey, reqModel, reqModel, service.PlatformOpenAI)
cls := classifyOpenAICompatibleNoAccountErrorFromGin(c, h.gatewayService, apiKey, reqModel, reqModel)
if !cls.ModelNotFound {
markOpsRoutingCapacityLimitedIfNoAvailable(c, err)
}
@@ -170,7 +171,7 @@ func (h *OpenAIGatewayHandler) ChatCompletions(c *gin.Context) {
}
}
if selection == nil || selection.Account == nil {
cls := classifyNoAccountErrorFromGin(c, h.gatewayService, apiKey, reqModel, reqModel, service.PlatformOpenAI)
cls := classifyOpenAICompatibleNoAccountErrorFromGin(c, h.gatewayService, apiKey, reqModel, reqModel)
if !cls.ModelNotFound {
markOpsRoutingCapacityLimited(c)
}
@@ -298,7 +299,7 @@ func (h *OpenAIGatewayHandler) ChatCompletions(c *gin.Context) {
userAgent := c.GetHeader("User-Agent")
clientIP := ip.GetClientIP(c)
inboundEndpoint := GetInboundEndpoint(c)
upstreamEndpoint := resolveOpenAIUpstreamEndpoint(c, account)
upstreamEndpoint := resolveOpenAIUpstreamEndpoint(c, account, result)
quotaPlatform := service.QuotaPlatform(c.Request.Context(), apiKey)
cyberBlocked := service.GetOpsCyberPolicy(c) != nil
@@ -337,14 +338,22 @@ func (h *OpenAIGatewayHandler) ChatCompletions(c *gin.Context) {
}
// resolveOpenAIUpstreamEndpoint returns the actual upstream endpoint for an
// OpenAI account, used by every OpenAI usage-recording site. APIKey accounts
// whose upstream is forced or probed to not support the Responses API are
// served directly via /v1/chat/completions (the raw chat path) regardless of
// the inbound endpoint; everything else goes through the Responses API.
func resolveOpenAIUpstreamEndpoint(c *gin.Context, account *service.Account) string {
// OpenAI-compatible account. A forwarding result is authoritative because a
// single inbound route may choose raw Chat or a Responses bridge at runtime.
// The account-based derivation remains as a fallback for existing callers and
// forwarding paths that do not report their endpoint yet.
func resolveOpenAIUpstreamEndpoint(c *gin.Context, account *service.Account, result *service.OpenAIForwardResult) string {
if result != nil {
if endpoint := strings.TrimSpace(result.UpstreamEndpoint); endpoint != "" {
return endpoint
}
}
if endpoint := service.GetActualOpenAIUpstreamEndpoint(c); endpoint != "" {
return endpoint
}
if account != nil && account.Type == service.AccountTypeAPIKey &&
!openai_compat.ShouldUseResponsesAPI(account.Extra) {
return "/v1/chat/completions"
return EndpointChatCompletions
}
return GetUpstreamEndpoint(c, account.Platform)
}
@@ -23,46 +23,61 @@ func newCompactBodySignalTestContext(t *testing.T, path string, body []byte) *gi
return c
}
// body-signal 提升后必须与 path-based compact 走同一条链路:
// path 改写、requireCompact 判定、stream/store/prompt_cache_key 归一化删除。
// 回归防护:若 stream 字段存活,Forward 会用流式 handler 解析 compact 的
// JSON 响应,导致 "stream ended before a terminal event" 的换号 failover 风暴。
func TestNormalizeOpenAIResponsesCompactRequest_BodySignalPromoted(t *testing.T) {
func TestNormalizeOpenAIResponsesCompactRequest_RemoteV2StaysOnResponses(t *testing.T) {
h := &OpenAIGatewayHandler{}
body := []byte(`{
"model":"gpt-5.5",
"model":"gpt-5.6-sol",
"stream":true,
"store":true,
"prompt_cache_key":"pck-signal-1",
"reasoning":{"effort":"max","context":"all_turns"},
"input":[
{"type":"message","role":"user","content":"hello"},
{"type":"compaction_trigger"}
]
}`)
c := newCompactBodySignalTestContext(t, "/v1/responses", body)
c.Request.Header.Set("x-codex-beta-features", "responses_websockets_v2, remote_compaction_v2, another_feature")
normalized, ok := h.normalizeOpenAIResponsesCompactRequest(c, zap.NewNop(), body)
require.True(t, ok)
require.Equal(t, "/v1/responses/compact", c.Request.URL.Path)
require.True(t, isOpenAIRemoteCompactPath(c))
require.False(t, gjson.GetBytes(normalized, "stream").Exists())
require.False(t, gjson.GetBytes(normalized, "store").Exists())
require.False(t, gjson.GetBytes(normalized, "prompt_cache_key").Exists())
require.Equal(t, "gpt-5.5", gjson.GetBytes(normalized, "model").String())
require.True(t, gjson.GetBytes(normalized, "input").IsArray())
require.Equal(t, "/v1/responses", c.Request.URL.Path)
require.False(t, isOpenAIRemoteCompactPath(c))
require.Equal(t, body, normalized)
require.True(t, gjson.GetBytes(normalized, "stream").Bool())
require.True(t, gjson.GetBytes(normalized, "store").Bool())
require.Equal(t, "pck-signal-1", gjson.GetBytes(normalized, "prompt_cache_key").String())
require.Equal(t, "max", gjson.GetBytes(normalized, "reasoning.effort").String())
require.Equal(t, "all_turns", gjson.GetBytes(normalized, "reasoning.context").String())
reqStream, streamOK := parseOpenAICompatibleStream(normalized)
require.True(t, streamOK)
require.False(t, reqStream)
require.True(t, reqStream)
seed, exists := c.Get(service.OpenAICompactSessionSeedKeyForTest())
require.True(t, exists)
require.Equal(t, "pck-signal-1", seed)
_, seedExists := c.Get(service.OpenAICompactSessionSeedKeyForTest())
require.False(t, seedExists)
_, streamMarkerExists := c.Get(service.OpenAICompactClientStreamKeyForTest())
require.False(t, streamMarkerExists)
}
func TestNormalizeOpenAIResponsesCompactRequest_BodySignalTrailingSlash(t *testing.T) {
func TestNormalizeOpenAIResponsesCompactRequest_RemoteV2PathAliasesStayOnResponses(t *testing.T) {
h := &OpenAIGatewayHandler{}
body := []byte(`{"model":"gpt-5.6-sol","stream":true,"input":[{"type":"compaction_trigger"}]}`)
for _, path := range []string{"/v1/responses/", "/backend-api/codex/responses"} {
t.Run(path, func(t *testing.T) {
c := newCompactBodySignalTestContext(t, path, body)
c.Request.Header.Set("x-codex-beta-features", "remote_compaction_v2")
normalized, ok := h.normalizeOpenAIResponsesCompactRequest(c, zap.NewNop(), body)
require.True(t, ok)
require.Equal(t, path, c.Request.URL.Path)
require.Equal(t, body, normalized)
})
}
}
func TestNormalizeOpenAIResponsesCompactRequest_BodySignalTrailingSlashPromoted(t *testing.T) {
h := &OpenAIGatewayHandler{}
body := []byte(`{"model":"gpt-5.5","input":[{"type":"compaction_trigger"}]}`)
c := newCompactBodySignalTestContext(t, "/v1/responses/", body)
@@ -82,6 +97,64 @@ func TestNormalizeOpenAIResponsesCompactRequest_CodexDirectAliasPromoted(t *test
require.Equal(t, "/backend-api/codex/responses/compact", c.Request.URL.Path)
}
func TestNormalizeOpenAIResponsesCompactRequest_NonRemoteV2BodySignalPromoted(t *testing.T) {
h := &OpenAIGatewayHandler{}
tests := []struct {
name string
body []byte
betaHeader string
wantMarked bool
}{
{
name: "no_header",
body: []byte(`{"model":"gpt-5.5","stream":true,"input":[{"type":"compaction_trigger"}]}`),
wantMarked: true,
},
{
name: "unrelated_header",
body: []byte(`{"model":"gpt-5.5","stream":true,"input":[{"type":"compaction_trigger"}]}`),
betaHeader: "responses_websockets_v2",
wantMarked: true,
},
{
name: "wrong_case_header",
body: []byte(`{"model":"gpt-5.5","stream":true,"input":[{"type":"compaction_trigger"}]}`),
betaHeader: "REMOTE_COMPACTION_V2",
wantMarked: true,
},
{
name: "stream_false",
body: []byte(`{"model":"gpt-5.5","stream":false,"input":[{"type":"compaction_trigger"}]}`),
betaHeader: "remote_compaction_v2",
},
{
name: "stream_absent",
body: []byte(`{"model":"gpt-5.5","input":[{"type":"compaction_trigger"}]}`),
betaHeader: "remote_compaction_v2",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
c := newCompactBodySignalTestContext(t, "/v1/responses", tt.body)
if tt.betaHeader != "" {
c.Request.Header.Set("x-codex-beta-features", tt.betaHeader)
}
normalized, ok := h.normalizeOpenAIResponsesCompactRequest(c, zap.NewNop(), tt.body)
require.True(t, ok)
require.Equal(t, "/v1/responses/compact", c.Request.URL.Path)
require.False(t, gjson.GetBytes(normalized, "stream").Exists())
marked, exists := c.Get(service.OpenAICompactClientStreamKeyForTest())
require.Equal(t, tt.wantMarked, exists)
if tt.wantMarked {
require.Equal(t, true, marked)
}
})
}
}
func TestNormalizeOpenAIResponsesCompactRequest_NoTriggerUntouched(t *testing.T) {
h := &OpenAIGatewayHandler{}
body := []byte(`{"model":"gpt-5.5","stream":true,"input":[{"type":"message","role":"user","content":"hello"}]}`)
@@ -99,6 +172,7 @@ func TestNormalizeOpenAIResponsesCompactRequest_PathBasedNoDoubleSuffix(t *testi
h := &OpenAIGatewayHandler{}
body := []byte(`{"model":"gpt-5.5","stream":true,"store":true,"input":[{"type":"message","role":"user","content":"hello"}]}`)
c := newCompactBodySignalTestContext(t, "/v1/responses/compact", body)
c.Request.Header.Set("x-codex-beta-features", "remote_compaction_v2")
normalized, ok := h.normalizeOpenAIResponsesCompactRequest(c, zap.NewNop(), body)
require.True(t, ok)
@@ -118,36 +192,6 @@ func TestNormalizeOpenAIResponsesCompactRequest_SubpathNotPromoted(t *testing.T)
require.Equal(t, body, normalized)
}
// 回归 #3875:body-signal 原始请求 stream:true 时必须标记 client-stream,
// 供响应写回阶段把上游 unary JSON 合成回 Codex remote compact v2 所需的 SSE。
func TestNormalizeOpenAIResponsesCompactRequest_BodySignalStreamTrueMarksClientStream(t *testing.T) {
h := &OpenAIGatewayHandler{}
body := []byte(`{"model":"gpt-5.5","stream":true,"input":[{"type":"compaction_trigger"}]}`)
c := newCompactBodySignalTestContext(t, "/v1/responses", body)
_, ok := h.normalizeOpenAIResponsesCompactRequest(c, zap.NewNop(), body)
require.True(t, ok)
marked, exists := c.Get(service.OpenAICompactClientStreamKeyForTest())
require.True(t, exists)
require.Equal(t, true, marked)
}
func TestNormalizeOpenAIResponsesCompactRequest_BodySignalStreamFalseNotMarked(t *testing.T) {
h := &OpenAIGatewayHandler{}
for name, body := range map[string][]byte{
"stream_false": []byte(`{"model":"gpt-5.5","stream":false,"input":[{"type":"compaction_trigger"}]}`),
"stream_absent": []byte(`{"model":"gpt-5.5","input":[{"type":"compaction_trigger"}]}`),
} {
c := newCompactBodySignalTestContext(t, "/v1/responses", body)
_, ok := h.normalizeOpenAIResponsesCompactRequest(c, zap.NewNop(), body)
require.True(t, ok, name)
require.Equal(t, "/v1/responses/compact", c.Request.URL.Path, name)
_, exists := c.Get(service.OpenAICompactClientStreamKeyForTest())
require.False(t, exists, "case %s 不应标记 client-stream", name)
}
}
// path-based compact(Codex v1 unary 协议)即使 body 带 stream:true 也不标记,
// 保持 JSON 写回行为不变。
func TestNormalizeOpenAIResponsesCompactRequest_PathBasedStreamTrueNotMarked(t *testing.T) {
@@ -115,8 +115,9 @@ func (h *OpenAIGatewayHandler) CountTokens(c *gin.Context) {
)
service.SetOpsLatencyMs(c, service.OpsAuthLatencyMsKey, time.Since(requestStart).Milliseconds())
if err != nil {
reqLog.Warn("openai_count_tokens.account_select_failed", zap.Error(err))
cls := classifyNoAccountErrorFromGin(c, h.gatewayService, apiKey, currentRoutingModel, reqModel, service.PlatformOpenAI)
requestPlatform := openAICompatibleRequestPlatform(apiKey)
reqLog.Warn("openai_count_tokens.account_select_failed", zap.Error(openAICompatibleSelectionErrorForLog(err, requestPlatform)))
cls := classifyOpenAICompatibleNoAccountErrorFromGin(c, h.gatewayService, apiKey, currentRoutingModel, reqModel)
if !cls.ModelNotFound {
markOpsRoutingCapacityLimitedIfNoAvailable(c, err)
}
@@ -124,7 +125,7 @@ func (h *OpenAIGatewayHandler) CountTokens(c *gin.Context) {
return
}
if selection == nil || selection.Account == nil {
cls := classifyNoAccountErrorFromGin(c, h.gatewayService, apiKey, currentRoutingModel, reqModel, service.PlatformOpenAI)
cls := classifyOpenAICompatibleNoAccountErrorFromGin(c, h.gatewayService, apiKey, currentRoutingModel, reqModel)
if !cls.ModelNotFound {
markOpsRoutingCapacityLimited(c)
}
@@ -351,7 +351,7 @@ func (h *OpenAIGatewayHandler) Responses(c *gin.Context) {
)
if err != nil {
reqLog.Warn("openai.account_select_failed",
zap.Error(err),
zap.Error(openAICompatibleSelectionErrorForLog(err, requestPlatform)),
zap.Int("excluded_account_count", len(failedAccountIDs)),
)
if len(failedAccountIDs) == 0 {
@@ -360,7 +360,7 @@ func (h *OpenAIGatewayHandler) Responses(c *gin.Context) {
h.handleStreamingAwareError(c, http.StatusServiceUnavailable, "compact_not_supported", "No available OpenAI accounts support /responses/compact", streamStarted)
return
}
cls := classifyNoAccountErrorFromGin(c, h.gatewayService, apiKey, reqModel, reqModel, service.PlatformOpenAI)
cls := classifyOpenAICompatibleNoAccountErrorFromGin(c, h.gatewayService, apiKey, reqModel, reqModel)
if !cls.ModelNotFound {
markOpsRoutingCapacityLimitedIfNoAvailable(c, err)
}
@@ -375,7 +375,7 @@ func (h *OpenAIGatewayHandler) Responses(c *gin.Context) {
return
}
if selection == nil || selection.Account == nil {
cls := classifyNoAccountErrorFromGin(c, h.gatewayService, apiKey, reqModel, reqModel, service.PlatformOpenAI)
cls := classifyOpenAICompatibleNoAccountErrorFromGin(c, h.gatewayService, apiKey, reqModel, reqModel)
if !cls.ModelNotFound {
markOpsRoutingCapacityLimited(c)
}
@@ -522,7 +522,7 @@ func (h *OpenAIGatewayHandler) Responses(c *gin.Context) {
clientIP := ip.GetClientIP(c)
requestPayloadHash := service.HashUsageRequestPayload(body)
inboundEndpoint := GetInboundEndpoint(c)
upstreamEndpoint := resolveOpenAIUpstreamEndpoint(c, account)
upstreamEndpoint := resolveOpenAIUpstreamEndpoint(c, account, result)
quotaPlatform := service.QuotaPlatform(c.Request.Context(), apiKey)
// 使用量记录通过有界 worker 池提交,避免请求热路径创建无界 goroutine。
@@ -580,21 +580,33 @@ func isBareOpenAIResponsesPath(c *gin.Context) bool {
return strings.HasSuffix(normalizedPath, "/responses")
}
// normalizeOpenAIResponsesCompactRequest 统一处理两种入站 compact 形态:
// path-based(POST /v1/responses/compact)与 Codex remote compact v2 的
// body-signal(普通 POST /v1/responses 的 input 中携带 type=compaction_trigger,
// 见 #3777)。body-signal 命中时在 stream 解析、compact body 归一化与
// requireCompact 调度判定之前改写 URL path,使后续全部链路(含 passthrough
// 分支与上游 URL 构建)与 path-based 完全一致。
func isOpenAIRemoteCompactionV2Request(c *gin.Context, body []byte) bool {
stream, valid := parseOpenAICompatibleStream(body)
if !valid || !stream || c == nil || c.Request == nil {
return false
}
for _, header := range c.Request.Header.Values("x-codex-beta-features") {
for _, feature := range strings.Split(header, ",") {
if strings.TrimSpace(feature) == "remote_compaction_v2" {
return true
}
}
}
return false
}
// normalizeOpenAIResponsesCompactRequest keeps Codex remote compaction v2 on
// its native streaming /responses wire and preserves the legacy body-signal
// promotion for clients that do not explicitly advertise that protocol.
// 返回归一化后的 body;ok=false 表示错误响应已写出,调用方应直接 return。
func (h *OpenAIGatewayHandler) normalizeOpenAIResponsesCompactRequest(c *gin.Context, reqLog *zap.Logger, body []byte) ([]byte, bool) {
isCompactRequest := service.IsOpenAIResponsesCompactPathForTest(c)
if !isCompactRequest && isBareOpenAIResponsesPath(c) && service.HasCompactionTriggerInInput(body) {
if isOpenAIRemoteCompactionV2Request(c, body) {
return body, true
}
c.Request.URL.Path = strings.TrimRight(c.Request.URL.Path, "/") + "/compact"
isCompactRequest = true
// Codex remote compact v2 的原始请求是流式 /responses:白名单归一化会删除
// stream 并让上游走 unary JSON,但客户端仍按 SSE 消费响应。记录原始
// stream 意图,响应写回阶段据此把 JSON 合成回 SSE(#3875)。
clientStream := gjson.GetBytes(body, "stream").Bool()
if clientStream {
service.MarkOpenAICompactClientStream(c)
@@ -843,12 +855,12 @@ func (h *OpenAIGatewayHandler) Messages(c *gin.Context) {
)
if err != nil {
reqLog.Warn("openai_messages.account_select_failed",
zap.Error(err),
zap.Error(openAICompatibleSelectionErrorForLog(err, requestPlatform)),
zap.Int("excluded_account_count", len(failedAccountIDs)),
)
if len(failedAccountIDs) == 0 {
if err != nil {
cls := classifyNoAccountErrorFromGin(c, h.gatewayService, apiKey, currentRoutingModel, reqModel, service.PlatformOpenAI)
cls := classifyOpenAICompatibleNoAccountErrorFromGin(c, h.gatewayService, apiKey, currentRoutingModel, reqModel)
if !cls.ModelNotFound {
markOpsRoutingCapacityLimitedIfNoAvailable(c, err)
}
@@ -865,7 +877,7 @@ func (h *OpenAIGatewayHandler) Messages(c *gin.Context) {
}
}
if selection == nil || selection.Account == nil {
cls := classifyNoAccountErrorFromGin(c, h.gatewayService, apiKey, currentRoutingModel, reqModel, service.PlatformOpenAI)
cls := classifyOpenAICompatibleNoAccountErrorFromGin(c, h.gatewayService, apiKey, currentRoutingModel, reqModel)
if !cls.ModelNotFound {
markOpsRoutingCapacityLimited(c)
}
@@ -994,7 +1006,7 @@ func (h *OpenAIGatewayHandler) Messages(c *gin.Context) {
clientIP := ip.GetClientIP(c)
requestPayloadHash := service.HashUsageRequestPayload(body)
inboundEndpoint := GetInboundEndpoint(c)
upstreamEndpoint := resolveOpenAIUpstreamEndpoint(c, account)
upstreamEndpoint := resolveOpenAIUpstreamEndpoint(c, account, result)
quotaPlatform := service.QuotaPlatform(c.Request.Context(), apiKey)
cyberBlocked := service.GetOpsCyberPolicy(c) != nil
@@ -1444,7 +1456,7 @@ func (h *OpenAIGatewayHandler) ResponsesWebSocket(c *gin.Context) {
)
if err != nil {
reqLog.Warn("openai.websocket_account_select_failed",
zap.Error(err),
zap.Error(openAICompatibleSelectionErrorForLog(err, requestPlatform)),
zap.Int("excluded_account_count", len(failedAccountIDs)),
)
if lastFailoverErr != nil {
@@ -1601,7 +1613,7 @@ func (h *OpenAIGatewayHandler) ResponsesWebSocket(c *gin.Context) {
}
h.gatewayService.ReportOpenAIAccountScheduleResult(account.ID, true, result.FirstTokenMs)
inboundEndpoint := GetInboundEndpoint(c)
upstreamEndpoint := resolveOpenAIUpstreamEndpoint(c, account)
upstreamEndpoint := resolveOpenAIUpstreamEndpoint(c, account, result)
quotaPlatform := service.QuotaPlatform(c.Request.Context(), apiKey)
cyberBlocked := service.GetOpsCyberPolicy(c) != nil
h.submitOpenAIUsageRecordTask(ctx, result, func(taskCtx context.Context) {
@@ -2441,7 +2453,7 @@ func (h *OpenAIGatewayHandler) recordCyberPolicyIfMarked(c *gin.Context, apiKey
var accountID int64
if account != nil {
accountID = account.ID
upstreamEndpoint = resolveOpenAIUpstreamEndpoint(c, account)
upstreamEndpoint = resolveOpenAIUpstreamEndpoint(c, account, nil)
}
stream := false
if v, ok := c.Get(opsStreamKey); ok {
@@ -0,0 +1,90 @@
package handler
import (
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/Wei-Shaw/sub2api/internal/service"
"github.com/gin-gonic/gin"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestOpsCaptureWriter_NilInnerWriter_NoPanic(t *testing.T) {
w := &opsCaptureWriter{}
w.ResponseWriter = nil
assert.NotPanics(t, func() {
assert.Equal(t, 0, w.Status())
})
assert.NotPanics(t, func() {
assert.Equal(t, -1, w.Size())
})
assert.NotPanics(t, func() {
assert.False(t, w.Written())
})
assert.NotPanics(t, func() {
n, err := w.Write([]byte("test"))
assert.Equal(t, 0, n)
assert.NoError(t, err)
})
assert.NotPanics(t, func() {
n, err := w.WriteString("test")
assert.Equal(t, 0, n)
assert.NoError(t, err)
})
assert.NotPanics(t, func() {
h := w.Header()
assert.NotNil(t, h)
})
assert.NotPanics(t, func() {
w.WriteHeader(200)
})
assert.NotPanics(t, func() {
w.WriteHeaderNow()
})
assert.NotPanics(t, func() {
w.Flush()
})
assert.NotPanics(t, func() {
conn, rw, err := w.Hijack()
assert.Nil(t, conn)
assert.Nil(t, rw)
assert.Error(t, err)
})
assert.NotPanics(t, func() {
ch := w.CloseNotify()
assert.NotNil(t, ch)
})
assert.NotPanics(t, func() {
p := w.Pusher()
assert.Nil(t, p)
})
}
func TestOpsCaptureWriter_CompactKeepaliveRestoresOriginalWriter(t *testing.T) {
gin.SetMode(gin.TestMode)
router := gin.New()
outerStatus := -1
router.Use(func(c *gin.Context) {
c.Next()
outerStatus = c.Writer.Status()
})
router.Use(OpsErrorLoggerMiddleware(nil))
router.GET("/compact", func(c *gin.Context) {
service.MarkOpenAICompactClientStream(c)
stop := service.StartOpenAICompactSSEKeepalive(c, time.Hour)
defer stop()
c.Status(http.StatusOK)
})
recorder := httptest.NewRecorder()
request := httptest.NewRequest(http.MethodGet, "/compact", nil)
require.NotPanics(t, func() {
router.ServeHTTP(recorder, request)
})
require.Equal(t, http.StatusOK, outerStatus)
require.Equal(t, http.StatusOK, recorder.Code)
}
@@ -1,11 +1,14 @@
package handler
import (
"bufio"
"bytes"
"context"
"encoding/json"
"errors"
"log"
"net"
"net/http"
"runtime"
"runtime/debug"
"strconv"
@@ -498,7 +501,82 @@ func releaseOpsCaptureWriter(w *opsCaptureWriter) {
opsCaptureWriterPool.Put(w)
}
func (w *opsCaptureWriter) Status() int {
if w.ResponseWriter == nil {
return 0
}
return w.ResponseWriter.Status()
}
func (w *opsCaptureWriter) Size() int {
if w.ResponseWriter == nil {
return -1
}
return w.ResponseWriter.Size()
}
func (w *opsCaptureWriter) Written() bool {
if w.ResponseWriter == nil {
return false
}
return w.ResponseWriter.Written()
}
func (w *opsCaptureWriter) Header() http.Header {
if w.ResponseWriter == nil {
return http.Header{}
}
return w.ResponseWriter.Header()
}
func (w *opsCaptureWriter) WriteHeader(code int) {
if w.ResponseWriter == nil {
return
}
w.ResponseWriter.WriteHeader(code)
}
func (w *opsCaptureWriter) WriteHeaderNow() {
if w.ResponseWriter == nil {
return
}
w.ResponseWriter.WriteHeaderNow()
}
func (w *opsCaptureWriter) Flush() {
if w.ResponseWriter == nil {
return
}
w.ResponseWriter.Flush()
}
func (w *opsCaptureWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
if w.ResponseWriter == nil {
return nil, nil, errors.New("response writer released")
}
return w.ResponseWriter.Hijack()
}
func (w *opsCaptureWriter) CloseNotify() <-chan bool {
if w.ResponseWriter == nil {
ch := make(chan bool)
close(ch)
return ch
}
return w.ResponseWriter.CloseNotify()
}
func (w *opsCaptureWriter) Pusher() http.Pusher {
if w.ResponseWriter == nil {
return nil
}
return w.ResponseWriter.Pusher()
}
func (w *opsCaptureWriter) Write(b []byte) (int, error) {
if w.ResponseWriter == nil {
return 0, nil
}
if w.Status() >= 400 && w.limit > 0 && w.buf.Len() < w.limit {
remaining := w.limit - w.buf.Len()
if len(b) > remaining {
@@ -511,6 +589,9 @@ func (w *opsCaptureWriter) Write(b []byte) (int, error) {
}
func (w *opsCaptureWriter) WriteString(s string) (int, error) {
if w.ResponseWriter == nil {
return 0, nil
}
if w.Status() >= 400 && w.limit > 0 && w.buf.Len() < w.limit {
remaining := w.limit - w.buf.Len()
if len(s) > remaining {
@@ -102,9 +102,10 @@ func AnthropicToResponsesResponse(resp *AnthropicResponse) *ResponsesResponse {
resp.Usage.CacheReadInputTokens +
resp.Usage.CacheCreationInputTokens
out.Usage = &ResponsesUsage{
InputTokens: totalInputTokens,
OutputTokens: resp.Usage.OutputTokens,
TotalTokens: totalInputTokens + resp.Usage.OutputTokens,
InputTokens: totalInputTokens,
OutputTokens: resp.Usage.OutputTokens,
TotalTokens: totalInputTokens + resp.Usage.OutputTokens,
CacheCreationInputTokens: resp.Usage.CacheCreationInputTokens,
}
if resp.Usage.CacheReadInputTokens > 0 {
out.Usage.InputTokensDetails = &ResponsesInputTokensDetails{
@@ -497,9 +498,10 @@ func makeResponsesCompletedEvent(
// back to match OpenAI Responses semantics where input_tokens is the total.
totalInputTokens := state.InputTokens + state.CacheReadInputTokens + state.CacheCreationInputTokens
usage := &ResponsesUsage{
InputTokens: totalInputTokens,
OutputTokens: state.OutputTokens,
TotalTokens: totalInputTokens + state.OutputTokens,
InputTokens: totalInputTokens,
OutputTokens: state.OutputTokens,
TotalTokens: totalInputTokens + state.OutputTokens,
CacheCreationInputTokens: state.CacheCreationInputTokens,
}
if state.CacheReadInputTokens > 0 {
usage.InputTokensDetails = &ResponsesInputTokensDetails{
@@ -1,6 +1,8 @@
package apicompat
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"strings"
@@ -34,10 +36,25 @@ func ResponsesToChatCompletionsRequest(req *ResponsesRequest) (*ChatCompletionsR
out.ReasoningEffort = req.Reasoning.Effort
}
if len(req.Tools) > 0 {
out.Tools = responsesToolsToChatTools(req.Tools)
tools, err := responsesToolsToChatTools(req.Tools)
if err != nil {
return nil, err
}
out.Tools = tools
}
if len(req.ToolChoice) > 0 {
out.ToolChoice = responsesToolChoiceToChatToolChoice(req.ToolChoice)
// tools 全部被丢弃(如仅含 web_search/image_generation 等服务端工具)时不再转发
// tool_choice:上游会拒绝 "'tool_choice' is only allowed when 'tools' are specified"。
// 指向被丢弃工具的选择项同理(见 responsesToolChoiceToChatToolChoice)。
if len(out.Tools) > 0 && len(req.ToolChoice) > 0 {
declared := make(map[string]bool, len(out.Tools))
for _, tool := range out.Tools {
if tool.Function != nil {
declared[tool.Function.Name] = true
}
}
if tc := responsesToolChoiceToChatToolChoice(req.ToolChoice, declared); len(tc) > 0 {
out.ToolChoice = tc
}
}
if req.Text != nil {
out.ResponseFormat = responsesTextFormatToChatResponseFormat(req.Text.Format)
@@ -46,6 +63,72 @@ func ResponsesToChatCompletionsRequest(req *ResponsesRequest) (*ChatCompletionsR
return out, nil
}
// CustomToolNames 收集 Responses 请求中 custom/freeform 工具的名字。chat 桥回程时
// 需要据此把模型对这些工具的调用还原为 custom_tool_call 项(codex 只按该类型路由)。
func CustomToolNames(tools []ResponsesTool) map[string]bool {
var out map[string]bool
for _, tool := range tools {
if tool.Type == "custom" && tool.Name != "" {
if out == nil {
out = make(map[string]bool)
}
out[tool.Name] = true
}
}
return out
}
// NamespacedToolName 记录 namespace 子工具的原始归属(命名空间 + 裸子工具名)。
type NamespacedToolName struct {
Namespace string
Name string
}
// NamespaceToolNames 收集 Responses 请求中 namespace 子工具的摊平名 →(namespace,
// 子工具名)映射。chat 桥回程时需据此把模型对摊平工具的调用还原为带 namespace 字段
// 的 function_call 项:codex 按 namespace+name 路由,平铺名会被判为 unsupported
// call;摊平名超长时带截断哈希(见 flattenNamespaceToolName),无法按字符串切分还原。
// 摊平名撞名的请求已在转换阶段被显式拒绝(见 namespaceChildrenToChatTools),
// 此处映射不存在歧义。
func NamespaceToolNames(tools []ResponsesTool) map[string]NamespacedToolName {
var out map[string]NamespacedToolName
for _, tool := range tools {
if tool.Type != "namespace" || tool.Name == "" {
continue
}
children := tool.Tools
if len(children) == 0 {
children = tool.Children
}
for _, child := range children {
if child.Type != "function" || child.Name == "" {
continue
}
if out == nil {
out = make(map[string]NamespacedToolName)
}
out[flattenNamespaceToolName(tool.Name, child.Name)] = NamespacedToolName{
Namespace: tool.Name,
Name: child.Name,
}
}
}
return out
}
// HasToolSearchTool 判断 Responses 请求是否声明了 tool_search 服务端工具。chat 桥
// 回程时需据此把模型对代理工具的调用还原为 tool_search_call 项:codex 只在该项类型
// 且 execution=client 时执行 tool search,同名 function_call 会因 payload 不匹配
// 触发 fatal 中止整个 turn。
func HasToolSearchTool(tools []ResponsesTool) bool {
for _, tool := range tools {
if tool.Type == "tool_search" {
return true
}
}
return false
}
// responsesInputToChatMessages converts a Responses request's instructions +
// input[] into Chat Completions messages. It is a three-stage pipeline:
//
@@ -133,33 +216,68 @@ func buildChatMessagesFromItems(messages []ChatMessage, rawItems []json.RawMessa
if strings.TrimSpace(arguments) == "" {
arguments = "{}"
}
name := rawString(item["name"])
// namespace 子工具的历史调用带 namespace 字段,需与请求方向的摊平
// 命名(namespaceChildrenToChatTools)保持一致。
if ns := rawString(item["namespace"]); ns != "" {
name = flattenNamespaceToolName(ns, name)
}
toolCall := ChatToolCall{
ID: rawString(item["call_id"]),
Type: "function",
Function: ChatFunctionCall{
Name: name,
Arguments: arguments,
},
}
messages = appendAssistantToolCall(messages, toolCall, pendingReasoning)
pendingReasoning = ""
continue
case "tool_search_call":
// tool_search 调用的 arguments 是 JSON 对象(如 {"query": ...}),
// 原文即为降级 function 调用的 arguments 字符串。
arguments := strings.TrimSpace(string(bytesTrimSpace(item["arguments"])))
if s := rawString(item["arguments"]); s != "" {
arguments = s
}
if arguments == "" || arguments == "null" {
arguments = "{}"
}
toolCall := ChatToolCall{
ID: rawString(item["call_id"]),
Type: "function",
Function: ChatFunctionCall{
Name: toolSearchProxyName,
Arguments: arguments,
},
}
messages = appendAssistantToolCall(messages, toolCall, pendingReasoning)
pendingReasoning = ""
continue
case "custom_tool_call":
// custom/freeform 工具的历史调用:input 自由文本包进降级 function 工具
// 的 {"input": ...} 参数,与请求方向的工具降级(customToolInputSchema)
// 保持一致,模型才能把历史与当前工具定义对上。
arguments, _ := json.Marshal(map[string]string{"input": rawString(item["input"])})
toolCall := ChatToolCall{
ID: rawString(item["call_id"]),
Type: "function",
Function: ChatFunctionCall{
Name: rawString(item["name"]),
Arguments: arguments,
Arguments: string(arguments),
},
}
// Parallel tool calls arrive as consecutive function_call items and
// must share one assistant message; the matching tool replies then
// follow it. Merge into the immediately preceding assistant message.
if n := len(messages); n > 0 && messages[n-1].Role == "assistant" {
messages[n-1].ToolCalls = append(messages[n-1].ToolCalls, toolCall)
if messages[n-1].ReasoningContent == "" {
messages[n-1].ReasoningContent = pendingReasoning
}
} else {
messages = append(messages, ChatMessage{
Role: "assistant",
ToolCalls: []ChatToolCall{toolCall},
ReasoningContent: pendingReasoning,
})
}
messages = appendAssistantToolCall(messages, toolCall, pendingReasoning)
pendingReasoning = ""
continue
case "function_call_output":
content, _ := json.Marshal(rawString(item["output"]))
case "function_call_output", "custom_tool_call_output", "tool_search_output":
outputRaw := bytesTrimSpace(item["output"])
outputText := rawString(outputRaw)
if outputText == "" && len(outputRaw) > 0 && string(outputRaw) != "null" && string(outputRaw) != `""` {
// 对象/数组形式的输出(如 tool_search 的结果列表)整体字符串化。
outputText = string(outputRaw)
}
content, _ := json.Marshal(outputText)
messages = append(messages, ChatMessage{
Role: "tool",
ToolCallID: rawString(item["call_id"]),
@@ -184,9 +302,9 @@ func buildChatMessagesFromItems(messages []ChatMessage, rawItems []json.RawMessa
// Only genuine message items become chat messages. Codex emits other
// Responses item types with no Chat equivalent (web_search_call,
// local_shell_call, custom tool calls, file_search_call, ...). Converting
// them via the generic path would insert a spurious message between an
// assistant tool_calls message and its tool reply, which DeepSeek rejects
// local_shell_call, file_search_call, ...). Converting them via the
// generic path would insert a spurious message between an assistant
// tool_calls message and its tool reply, which DeepSeek rejects
// ("insufficient tool messages following tool_calls message"). Skip them.
if itemType != "" && itemType != "message" {
pendingReasoning = ""
@@ -213,6 +331,25 @@ func buildChatMessagesFromItems(messages []ChatMessage, rawItems []json.RawMessa
return messages, nil
}
// appendAssistantToolCall merges a tool call into the chat message list.
// Parallel tool calls arrive as consecutive *_call items and must share one
// assistant message; the matching tool replies then follow it. Merge into the
// immediately preceding assistant message.
func appendAssistantToolCall(messages []ChatMessage, toolCall ChatToolCall, pendingReasoning string) []ChatMessage {
if n := len(messages); n > 0 && messages[n-1].Role == "assistant" {
messages[n-1].ToolCalls = append(messages[n-1].ToolCalls, toolCall)
if messages[n-1].ReasoningContent == "" {
messages[n-1].ReasoningContent = pendingReasoning
}
return messages
}
return append(messages, ChatMessage{
Role: "assistant",
ToolCalls: []ChatToolCall{toolCall},
ReasoningContent: pendingReasoning,
})
}
// normalizeChatMessages is the single place that enforces the tool-call
// invariant the DeepSeek / OpenAI Chat Completions schema requires: an assistant
// message with tool_calls must be immediately followed by one tool message per
@@ -427,39 +564,187 @@ func chatContentFromSingleResponsesPart(partType string, part map[string]json.Ra
}
}
func responsesToolsToChatTools(tools []ResponsesTool) []ChatTool {
// customToolInputSchema 是 custom/freeform 工具降级为 function 工具时的参数 schema。
// chat 协议无法表达 custom 工具的自由文本输入(及其 grammar 约束),退化为单一
// input 字符串参数;回程时再从 arguments 的 input 字段还原(见
// extractCustomToolCallInput)。
const customToolInputSchema = `{"type":"object","properties":{"input":{"type":"string","description":"The raw input for this tool, passed through verbatim."}},"required":["input"]}`
func responsesToolsToChatTools(tools []ResponsesTool) ([]ChatTool, error) {
// 顶层 function/custom 工具名集合:namespace 子工具摊平后与其撞名时,chat
// 上游无法按 namespace 区分调用归属。这类请求在原生 Responses 上游是合法的
// (按 namespace+name 路由),歧义由摊平转换制造且无法消除,必须显式拒绝,
// 不能静默降级(重复声明发给上游、回程还原到错误工具)。
topLevel := make(map[string]bool)
for _, tool := range tools {
if (tool.Type == "function" || tool.Type == "custom") && tool.Name != "" {
topLevel[tool.Name] = true
}
}
flatOwner := make(map[string]NamespacedToolName)
toolSearchDeclared := false
out := make([]ChatTool, 0, len(tools))
for _, tool := range tools {
if tool.Type != "function" {
switch tool.Type {
case "function":
out = append(out, ChatTool{
Type: "function",
Function: &ChatFunction{
Name: tool.Name,
Description: tool.Description,
Parameters: tool.Parameters,
Strict: tool.Strict,
},
})
case "custom":
// codex 0.14x 的核心执行工具 exec 即为 custom 类型;丢弃它会让模型
// 无法执行任何命令,必须降级为 function 工具透传。
out = append(out, ChatTool{
Type: "function",
Function: &ChatFunction{
Name: tool.Name,
Description: tool.Description,
Parameters: json.RawMessage(customToolInputSchema),
},
})
case "tool_search":
// 代理不能改名(codex 的模型侧按 tool_search 这个名字调用),与客户端
// 声明的同名工具无法区分——回程会把普通工具的调用劫持成 tool_search_call,
// 必须显式拒绝;重复声明 type=tool_search 去重即可。
if topLevel[toolSearchProxyName] {
return nil, fmt.Errorf("built-in tool_search conflicts with a declared tool named %q; this upstream cannot disambiguate them, rename the tool", toolSearchProxyName)
}
if toolSearchDeclared {
continue
}
toolSearchDeclared = true
out = append(out, toolSearchProxyChatTool())
case "namespace":
flattened, err := namespaceChildrenToChatTools(tool, topLevel, flatOwner)
if err != nil {
return nil, err
}
out = append(out, flattened...)
}
// 其余类型(web_search、image_generation 等服务端工具)在 chat 上游没有
// 对应能力,维持丢弃。
}
return out, nil
}
// toolSearchProxyName 是 tool_search 服务端工具降级后的 function 工具名。模型对
// 它的调用以同名 function_call 原样回传,由 codex 端路由。
const toolSearchProxyName = "tool_search"
const toolSearchProxySchema = `{"type":"object","properties":{"query":{"type":"string","description":"Search query for tools or connectors to load."},"limit":{"type":"integer","description":"Maximum number of tool groups to return."}},"required":["query"]}`
func toolSearchProxyChatTool() ChatTool {
return ChatTool{
Type: "function",
Function: &ChatFunction{
Name: toolSearchProxyName,
Description: "Search and load Codex tools, plugins, connectors, and MCP namespaces for the current task.",
Parameters: json.RawMessage(toolSearchProxySchema),
},
}
}
// namespaceChildrenToChatTools 将 namespace 工具的子 function 工具摊平为顶层
// function 工具,名字加 "<namespace>__" 前缀。摊平名与顶层工具或其他 namespace
// 撞名时返回错误(歧义不可消除,显式拒绝);同一 (namespace, 子工具) 的重复声明
// 去重后不算冲突。
func namespaceChildrenToChatTools(tool ResponsesTool, topLevel map[string]bool, flatOwner map[string]NamespacedToolName) ([]ChatTool, error) {
if tool.Name == "" {
return nil, nil
}
children := tool.Tools
if len(children) == 0 {
children = tool.Children
}
var out []ChatTool
for _, child := range children {
if child.Type != "function" || child.Name == "" {
continue
}
flat := flattenNamespaceToolName(tool.Name, child.Name)
entry := NamespacedToolName{Namespace: tool.Name, Name: child.Name}
if topLevel[flat] {
return nil, fmt.Errorf("namespace tool %q/%q flattens to %q which conflicts with a top-level tool of the same name; this upstream cannot disambiguate them, rename one of the tools", tool.Name, child.Name, flat)
}
if prev, ok := flatOwner[flat]; ok {
if prev == entry {
continue
}
return nil, fmt.Errorf("namespace tools %q/%q and %q/%q both flatten to %q; this upstream cannot disambiguate them, rename one of the tools", prev.Namespace, prev.Name, tool.Name, child.Name, flat)
}
flatOwner[flat] = entry
out = append(out, ChatTool{
Type: "function",
Function: &ChatFunction{
Name: tool.Name,
Description: tool.Description,
Parameters: tool.Parameters,
Strict: tool.Strict,
Name: flat,
Description: child.Description,
Parameters: child.Parameters,
Strict: child.Strict,
},
})
}
return out
return out, nil
}
func responsesToolChoiceToChatToolChoice(raw json.RawMessage) json.RawMessage {
// chatToolNameMaxLen 是 Chat Completions function 工具名的通用长度上限。
const chatToolNameMaxLen = 64
// flattenNamespaceToolName 生成 namespace 子工具的摊平名;超长时截断并追加
// sha256 短哈希保证唯一性。
func flattenNamespaceToolName(namespace, name string) string {
full := namespace + "__" + name
if len(full) <= chatToolNameMaxLen {
return full
}
sum := sha256.Sum256([]byte(full))
suffix := "__" + hex.EncodeToString(sum[:4])
prefixLen := chatToolNameMaxLen - len(suffix)
var prefix strings.Builder
for _, ch := range full {
if prefix.Len()+len(string(ch)) > prefixLen {
break
}
_, _ = prefix.WriteRune(ch)
}
return prefix.String() + suffix
}
// responsesToolChoiceToChatToolChoice 把 Responses 的 tool_choice 转为 chat 形态。
// declared 是转换后实际声明的 chat 工具名集合:具名选择项仅在目标工具幸存时转发,
// 服务端工具(web_search 等)的选择项随工具本身丢弃——指向未声明工具的 tool_choice
// 会被 chat 上游 400 拒绝。返回 nil 表示丢弃 tool_choice。
func responsesToolChoiceToChatToolChoice(raw json.RawMessage, declared map[string]bool) json.RawMessage {
var choice map[string]json.RawMessage
if err := json.Unmarshal(raw, &choice); err != nil {
// "auto"/"none"/"required" 等字符串形式原样转发。
return raw
}
if rawString(choice["type"]) != "function" {
return raw
var name string
switch rawString(choice["type"]) {
case "tool_search":
// tool_search 未被丢弃而是降级为同名 function 代理(见
// responsesToolsToChatTools),强制选择它同样降级为 function 选择,
// 静默丢弃会把强制搜索退化为自动选择。
name = toolSearchProxyName
case "function", "custom":
// custom 工具已降级为 function 工具,指向它的 tool_choice 同样按 function 转换。
name = rawString(choice["name"])
if name == "" {
name = rawNestedString(choice["function"], "name")
}
if name == "" {
return raw
}
default:
return nil
}
name := rawString(choice["name"])
if name == "" {
name = rawNestedString(choice["function"], "name")
}
if name == "" {
return raw
if !declared[name] {
return nil
}
out, err := json.Marshal(map[string]any{
"type": "function",
@@ -473,9 +758,38 @@ func responsesToolChoiceToChatToolChoice(raw json.RawMessage) json.RawMessage {
return out
}
// extractCustomToolCallInput 从降级 function 调用的 arguments 中还原 custom 工具的
// 自由文本输入:优先取 {"input": "..."} 的 input 字段;模型未按 schema 输出时原样
// 回传,交由客户端校验、模型重试。
func extractCustomToolCallInput(arguments string) string {
trimmed := strings.TrimSpace(arguments)
if trimmed == "" {
return ""
}
var obj map[string]json.RawMessage
if err := json.Unmarshal([]byte(trimmed), &obj); err != nil {
return trimmed
}
if raw, ok := obj["input"]; ok {
var s string
if err := json.Unmarshal(raw, &s); err == nil {
return s
}
return trimmed
}
if len(obj) == 0 {
return ""
}
return trimmed
}
// ChatCompletionsResponseToResponses converts a non-streaming Chat Completions
// response into a Responses API response.
func ChatCompletionsResponseToResponses(resp *ChatCompletionsResponse, model string) *ResponsesResponse {
// response into a Responses API response. customTools 是客户端请求中 custom 工具
// 的名字集合(见 CustomToolNames),命中的调用会还原为 custom_tool_call 项;
// toolSearch 表示客户端声明了 tool_search 工具(见 HasToolSearchTool),代理工具
// 的调用会还原为 tool_search_call 项;namespaceTools 是 namespace 子工具的摊平名
// 映射(见 NamespaceToolNames),命中的调用还原为带 namespace 字段的 function_call 项。
func ChatCompletionsResponseToResponses(resp *ChatCompletionsResponse, model string, customTools map[string]bool, toolSearch bool, namespaceTools map[string]NamespacedToolName) *ResponsesResponse {
id := ""
if resp != nil {
id = resp.ID
@@ -500,7 +814,7 @@ func ChatCompletionsResponseToResponses(resp *ChatCompletionsResponse, model str
if len(resp.Choices) > 0 {
choice := resp.Choices[0]
out.Output = chatMessageToResponsesOutput(choice.Message)
out.Output = chatMessageToResponsesOutput(choice.Message, customTools, toolSearch, namespaceTools)
if choice.FinishReason == "length" {
out.Status = "incomplete"
out.IncompleteDetails = &ResponsesIncompleteDetails{Reason: "max_output_tokens"}
@@ -515,7 +829,7 @@ func ChatCompletionsResponseToResponses(resp *ChatCompletionsResponse, model str
return out
}
func chatMessageToResponsesOutput(message ChatMessage) []ResponsesOutput {
func chatMessageToResponsesOutput(message ChatMessage, customTools map[string]bool, toolSearch bool, namespaceTools map[string]NamespacedToolName) []ResponsesOutput {
var outputs []ResponsesOutput
if message.ReasoningContent != "" {
outputs = append(outputs, ResponsesOutput{
@@ -550,6 +864,39 @@ func chatMessageToResponsesOutput(message ChatMessage) []ResponsesOutput {
if strings.TrimSpace(arguments) == "" {
arguments = "{}"
}
if customTools[toolCall.Function.Name] {
outputs = append(outputs, ResponsesOutput{
Type: "custom_tool_call",
ID: generateItemID(),
CallID: toolCall.ID,
Name: toolCall.Function.Name,
Input: extractCustomToolCallInput(arguments),
Status: "completed",
})
continue
}
if toolSearch && toolCall.Function.Name == toolSearchProxyName {
outputs = append(outputs, ResponsesOutput{
Type: "tool_search_call",
ID: generateItemID(),
CallID: toolCall.ID,
Arguments: arguments,
Status: "completed",
})
continue
}
if ns, ok := namespaceTools[toolCall.Function.Name]; ok {
outputs = append(outputs, ResponsesOutput{
Type: "function_call",
ID: generateItemID(),
CallID: toolCall.ID,
Name: ns.Name,
Namespace: ns.Namespace,
Arguments: arguments,
Status: "completed",
})
continue
}
outputs = append(outputs, ResponsesOutput{
Type: "function_call",
ID: generateItemID(),
@@ -563,6 +910,21 @@ func chatMessageToResponsesOutput(message ChatMessage) []ResponsesOutput {
return outputs
}
// toolSearchCallArgumentsJSON 把降级 function 调用累积的 arguments 字符串还原为
// tool_search_call 线上要求的 JSON 对象;模型未按 schema 输出(非法 JSON)时按
// 字符串值兜底,交由 codex 解析报错后让模型重试。
func toolSearchCallArgumentsJSON(arguments string) json.RawMessage {
trimmed := strings.TrimSpace(arguments)
if trimmed == "" {
return json.RawMessage(`{}`)
}
if json.Valid([]byte(trimmed)) {
return json.RawMessage(trimmed)
}
fallback, _ := json.Marshal(arguments)
return fallback
}
func emptyResponsesMessageOutput() ResponsesOutput {
return ResponsesOutput{
Type: "message",
@@ -662,6 +1024,35 @@ type ChatCompletionsToResponsesStreamState struct {
ToolItemIDs map[int]string
ToolOutputIndex map[int]int
// CustomTools 是客户端请求中 custom/freeform 工具的名字集合(见
// CustomToolNames)。命中的调用按 custom_tool_call 生命周期下发,codex 才能
// 路由回它注册的 custom 工具。
CustomTools map[string]bool
// ToolSearchDeclared 表示客户端请求声明了 tool_search 工具(见
// HasToolSearchTool)。命中的代理调用按 tool_search_call 项还原,codex 只按
// 该项类型(且 execution=client)执行 tool search。
ToolSearchDeclared bool
// NamespaceTools 是 namespace 子工具的摊平名 → 原始归属映射(见
// NamespaceToolNames)。命中的调用还原为带 namespace 字段的 function_call 项,
// codex 按 namespace+name 路由。
NamespaceTools map[string]NamespacedToolName
// toolIsCustom 记录每个工具调用宣告时的类型判定,保证 added/done 事件的
// 项类型一致。
toolIsCustom map[int]bool
// toolIsToolSearch 记录工具调用是否判定为 tool_search 代理调用。
toolIsToolSearch map[int]bool
// toolNamespace 记录工具调用宣告时命中的 namespace 归属(见 NamespaceTools)。
toolNamespace map[int]NamespacedToolName
// toolAnnounced 记录 output_item.added 是否已发出。存在 custom 工具且名字
// 尚未到达时延迟宣告,待名字可判定类型后再补发(见 announceChatToolItem)。
toolAnnounced map[int]bool
FinishReason string
Usage *ResponsesUsage
}
@@ -669,12 +1060,16 @@ type ChatCompletionsToResponsesStreamState struct {
// NewChatCompletionsToResponsesStreamState returns an initialized stream state.
func NewChatCompletionsToResponsesStreamState(model string) *ChatCompletionsToResponsesStreamState {
return &ChatCompletionsToResponsesStreamState{
ResponseID: generateResponsesID(),
Model: model,
Created: time.Now().Unix(),
ToolCalls: make(map[int]*ChatToolCall),
ToolItemIDs: make(map[int]string),
ToolOutputIndex: make(map[int]int),
ResponseID: generateResponsesID(),
Model: model,
Created: time.Now().Unix(),
ToolCalls: make(map[int]*ChatToolCall),
ToolItemIDs: make(map[int]string),
ToolOutputIndex: make(map[int]int),
toolIsCustom: make(map[int]bool),
toolIsToolSearch: make(map[int]bool),
toolNamespace: make(map[int]NamespacedToolName),
toolAnnounced: make(map[int]bool),
}
}
@@ -758,19 +1153,8 @@ func ChatCompletionsChunkToResponsesEvents(
copyCall.Function.Arguments = ""
state.ToolCalls[idx] = &copyCall
stored = &copyCall
itemID := generateItemID()
state.ToolItemIDs[idx] = itemID
state.ToolItemIDs[idx] = generateItemID()
state.ToolOutputIndex[idx] = state.allocOutputIndex()
events = append(events, chatToResponsesEvent(state, "response.output_item.added", &ResponsesStreamEvent{
OutputIndex: state.ToolOutputIndex[idx],
Item: &ResponsesOutput{
Type: "function_call",
ID: itemID,
CallID: stored.ID,
Name: stored.Function.Name,
Status: "in_progress",
},
}))
} else {
if toolCall.ID != "" {
stored.ID = toolCall.ID
@@ -779,15 +1163,22 @@ func ChatCompletionsChunkToResponsesEvents(
stored.Function.Name = toolCall.Function.Name
}
}
events = append(events, announceChatToolItem(state, idx, stored, false)...)
if toolCall.Function.Arguments != "" {
stored.Function.Arguments += toolCall.Function.Arguments
events = append(events, chatToResponsesEvent(state, "response.function_call_arguments.delta", &ResponsesStreamEvent{
OutputIndex: state.ToolOutputIndex[idx],
ItemID: state.ToolItemIDs[idx],
Delta: toolCall.Function.Arguments,
CallID: stored.ID,
Name: stored.Function.Name,
}))
// 未宣告(名字未到)时仅累积,宣告时统一补发;custom 调用的
// arguments 是包裹 input 的 JSON 片段,无法增量还原为自由文本
// 输入,缓冲整份 arguments 收尾时一次性下发(见 closeChatToolItems);
// tool_search 调用同样收尾时随 output_item.done 全量下发。
if state.toolAnnounced[idx] && !state.toolIsCustom[idx] && !state.toolIsToolSearch[idx] {
events = append(events, chatToResponsesEvent(state, "response.function_call_arguments.delta", &ResponsesStreamEvent{
OutputIndex: state.ToolOutputIndex[idx],
ItemID: state.ToolItemIDs[idx],
Delta: toolCall.Function.Arguments,
CallID: stored.ID,
Name: stored.Function.Name,
}))
}
}
}
if choice.FinishReason != nil && *choice.FinishReason != "" {
@@ -998,6 +1389,64 @@ func ensureChatToResponsesTextPart(state *ChatCompletionsToResponsesStreamState)
})}
}
// announceChatToolItem 在类型可判定时发出工具调用的 output_item.added。custom
// 工具的判定依赖名字:名字未到且请求里存在 custom 工具时延迟宣告,避免 added/done
// 的项类型不一致;force 用于流收尾,名字始终未到时按 function_call 兜底。
func announceChatToolItem(
state *ChatCompletionsToResponsesStreamState,
idx int,
stored *ChatToolCall,
force bool,
) []ResponsesStreamEvent {
if state.toolAnnounced[idx] {
return nil
}
if !force && stored.Function.Name == "" && (len(state.CustomTools) > 0 || state.ToolSearchDeclared || len(state.NamespaceTools) > 0) {
return nil
}
state.toolAnnounced[idx] = true
isCustom := state.CustomTools[stored.Function.Name]
isToolSearch := !isCustom && state.ToolSearchDeclared && stored.Function.Name == toolSearchProxyName
state.toolIsCustom[idx] = isCustom
state.toolIsToolSearch[idx] = isToolSearch
itemType := "function_call"
if isCustom {
itemType = "custom_tool_call"
}
if isToolSearch {
itemType = "tool_search_call"
}
// namespace 子工具的调用仍按 function_call 生命周期下发,但 added/done 项要
// 还原为裸子工具名 + namespace 字段(codex 按 namespace+name 路由)。
itemName, itemNamespace := stored.Function.Name, ""
if ns, ok := state.NamespaceTools[stored.Function.Name]; ok && !isCustom && !isToolSearch {
state.toolNamespace[idx] = ns
itemName, itemNamespace = ns.Name, ns.Namespace
}
events := []ResponsesStreamEvent{chatToResponsesEvent(state, "response.output_item.added", &ResponsesStreamEvent{
OutputIndex: state.ToolOutputIndex[idx],
Item: &ResponsesOutput{
Type: itemType,
ID: state.ToolItemIDs[idx],
CallID: stored.ID,
Name: itemName,
Namespace: itemNamespace,
Status: "in_progress",
},
})}
// 迟到宣告时补发已累积的参数增量(custom/tool_search 的输入收尾统一下发,不补发)。
if !isCustom && !isToolSearch && stored.Function.Arguments != "" {
events = append(events, chatToResponsesEvent(state, "response.function_call_arguments.delta", &ResponsesStreamEvent{
OutputIndex: state.ToolOutputIndex[idx],
ItemID: state.ToolItemIDs[idx],
Delta: stored.Function.Arguments,
CallID: stored.ID,
Name: stored.Function.Name,
}))
}
return events
}
// closeChatToolItems emits function_call_arguments.done + output_item.done for
// every tool call opened during the stream, carrying the full call_id/name/
// arguments so codex can deserialize and execute the call. Mirrors cc-switch's
@@ -1016,17 +1465,72 @@ func closeChatToolItems(state *ChatCompletionsToResponsesStreamState) []Response
if !opened {
continue
}
// 名字始终未到导致尚未宣告的调用,收尾前按最终名字兜底宣告。
events = append(events, announceChatToolItem(state, i, toolCall, true)...)
arguments := toolCall.Function.Arguments
if strings.TrimSpace(arguments) == "" {
arguments = "{}"
}
outputIndex := state.ToolOutputIndex[i]
if state.toolIsCustom[i] {
// custom 调用按 custom_tool_call 生命周期收尾:input 在此处一次性下发
// (流中不产出增量,见 ChatCompletionsChunkToResponsesEvents)。
input := extractCustomToolCallInput(arguments)
if input != "" {
events = append(events, chatToResponsesEvent(state, "response.custom_tool_call_input.delta", &ResponsesStreamEvent{
OutputIndex: outputIndex,
ItemID: itemID,
Delta: input,
}))
}
events = append(events,
chatToResponsesEvent(state, "response.custom_tool_call_input.done", &ResponsesStreamEvent{
OutputIndex: outputIndex,
ItemID: itemID,
CallID: toolCall.ID,
Name: toolCall.Function.Name,
Input: input,
}),
chatToResponsesEvent(state, "response.output_item.done", &ResponsesStreamEvent{
OutputIndex: outputIndex,
Item: &ResponsesOutput{
Type: "custom_tool_call",
ID: itemID,
CallID: toolCall.ID,
Name: toolCall.Function.Name,
Input: input,
Status: "completed",
},
}),
)
continue
}
if state.toolIsToolSearch[i] {
// tool_search 调用按 tool_search_call 项收尾:codex 从 output_item.done
// 物化该调用(无参数增量事件),arguments 全量随项下发。
events = append(events, chatToResponsesEvent(state, "response.output_item.done", &ResponsesStreamEvent{
OutputIndex: outputIndex,
Item: &ResponsesOutput{
Type: "tool_search_call",
ID: itemID,
CallID: toolCall.ID,
Arguments: arguments,
Status: "completed",
},
}))
continue
}
// namespace 子工具调用在宣告时已记录归属,收尾项同样带还原名与 namespace。
name, namespace := toolCall.Function.Name, ""
if ns, ok := state.toolNamespace[i]; ok {
name, namespace = ns.Name, ns.Namespace
}
events = append(events,
chatToResponsesEvent(state, "response.function_call_arguments.done", &ResponsesStreamEvent{
OutputIndex: outputIndex,
ItemID: itemID,
CallID: toolCall.ID,
Name: toolCall.Function.Name,
Name: name,
Arguments: arguments,
}),
chatToResponsesEvent(state, "response.output_item.done", &ResponsesStreamEvent{
@@ -1035,7 +1539,8 @@ func closeChatToolItems(state *ChatCompletionsToResponsesStreamState) []Response
Type: "function_call",
ID: itemID,
CallID: toolCall.ID,
Name: toolCall.Function.Name,
Name: name,
Namespace: namespace,
Arguments: arguments,
Status: "completed",
},
@@ -1078,11 +1583,37 @@ func (state *ChatCompletionsToResponsesStreamState) chatOutput() []ResponsesOutp
if strings.TrimSpace(arguments) == "" {
arguments = "{}"
}
if state.toolIsCustom[i] {
outputs = append(outputs, ResponsesOutput{
Type: "custom_tool_call",
ID: generateItemID(),
CallID: toolCall.ID,
Name: toolCall.Function.Name,
Input: extractCustomToolCallInput(arguments),
Status: "completed",
})
continue
}
if state.toolIsToolSearch[i] {
outputs = append(outputs, ResponsesOutput{
Type: "tool_search_call",
ID: generateItemID(),
CallID: toolCall.ID,
Arguments: arguments,
Status: "completed",
})
continue
}
name, namespace := toolCall.Function.Name, ""
if ns, ok := state.toolNamespace[i]; ok {
name, namespace = ns.Name, ns.Namespace
}
outputs = append(outputs, ResponsesOutput{
Type: "function_call",
ID: generateItemID(),
CallID: toolCall.ID,
Name: toolCall.Function.Name,
Name: name,
Namespace: namespace,
Arguments: arguments,
Status: "completed",
})
@@ -0,0 +1,878 @@
package apicompat
// custom/freeform 工具(如 Codex 0.14x 的 exec)在 responses→chat 桥上的双向转换。
// 背景:Codex 的核心命令执行工具 exec 是 type=custom(输入为自由文本),此前被
// responsesToolsToChatTools 丢弃,导致模型工具列表中没有 exec、无法执行任何命令。
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestResponsesToChatCompletionsRequest_CustomToolBecomesFunctionTool(t *testing.T) {
req := &ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"run dir"`),
Tools: []ResponsesTool{
{Type: "custom", Name: "exec", Description: "Run JavaScript code"},
{Type: "function", Name: "wait", Parameters: json.RawMessage(`{"type":"object","properties":{}}`)},
},
}
out, err := ResponsesToChatCompletionsRequest(req)
require.NoError(t, err)
require.Len(t, out.Tools, 2)
assert.Equal(t, "function", out.Tools[0].Type)
assert.Equal(t, "exec", out.Tools[0].Function.Name)
assert.Equal(t, "Run JavaScript code", out.Tools[0].Function.Description)
assert.JSONEq(t, customToolInputSchema, string(out.Tools[0].Function.Parameters))
assert.Equal(t, "wait", out.Tools[1].Function.Name)
}
func TestResponsesToChatCompletionsRequest_DropsToolChoiceWhenNoConvertibleTools(t *testing.T) {
req := &ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{
{Type: "web_search"},
{Type: "image_generation"},
},
ToolChoice: json.RawMessage(`"auto"`),
}
out, err := ResponsesToChatCompletionsRequest(req)
require.NoError(t, err)
assert.Empty(t, out.Tools)
assert.Empty(t, out.ToolChoice, "tools 为空时转发 tool_choice 会被上游 400 拒绝")
}
func TestResponsesToChatCompletionsRequest_CustomToolChoiceMapsToFunctionChoice(t *testing.T) {
req := &ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"run dir"`),
Tools: []ResponsesTool{{Type: "custom", Name: "exec"}},
ToolChoice: json.RawMessage(`{"type":"custom","name":"exec"}`),
}
out, err := ResponsesToChatCompletionsRequest(req)
require.NoError(t, err)
assert.JSONEq(t, `{"type":"function","function":{"name":"exec"}}`, string(out.ToolChoice))
}
func TestResponsesInputToChatMessages_CustomToolCallHistory(t *testing.T) {
input := json.RawMessage(`[
{"role":"user","content":"list files"},
{"type":"custom_tool_call","call_id":"call_1","name":"exec","input":"dir"},
{"type":"custom_tool_call_output","call_id":"call_1","output":"main.go"}
]`)
messages, err := responsesInputToChatMessages("", input)
require.NoError(t, err)
require.Len(t, messages, 3)
assert.Equal(t, []string{"user", "assistant", "tool"}, chatMessageRoles(messages))
require.Len(t, messages[1].ToolCalls, 1)
toolCall := messages[1].ToolCalls[0]
assert.Equal(t, "call_1", toolCall.ID)
assert.Equal(t, "exec", toolCall.Function.Name)
assert.JSONEq(t, `{"input":"dir"}`, toolCall.Function.Arguments)
assert.Equal(t, "call_1", messages[2].ToolCallID)
assert.JSONEq(t, `"main.go"`, string(messages[2].Content))
}
func TestChatCompletionsResponseToResponses_CustomToolCallOutputItem(t *testing.T) {
resp := &ChatCompletionsResponse{
ID: "cc-1",
Choices: []ChatChoice{{
Message: ChatMessage{
Role: "assistant",
ToolCalls: []ChatToolCall{
{ID: "call_1", Function: ChatFunctionCall{Name: "exec", Arguments: `{"input": "dir"}`}},
{ID: "call_2", Function: ChatFunctionCall{Name: "wait", Arguments: `{"cell_id": 3}`}},
},
},
}},
}
out := ChatCompletionsResponseToResponses(resp, "glm-5.2", map[string]bool{"exec": true}, false, nil)
require.Len(t, out.Output, 2)
assert.Equal(t, "custom_tool_call", out.Output[0].Type)
assert.Equal(t, "call_1", out.Output[0].CallID)
assert.Equal(t, "exec", out.Output[0].Name)
assert.Equal(t, "dir", out.Output[0].Input)
assert.Empty(t, out.Output[0].Arguments)
assert.Equal(t, "function_call", out.Output[1].Type)
assert.Equal(t, "wait", out.Output[1].Name)
assert.Equal(t, `{"cell_id": 3}`, out.Output[1].Arguments)
}
func TestExtractCustomToolCallInput_FallsBackToRawArguments(t *testing.T) {
assert.Equal(t, "dir", extractCustomToolCallInput(`{"input": "dir"}`))
assert.Equal(t, "console.log(1)", extractCustomToolCallInput(`console.log(1)`))
assert.Equal(t, `{"other": "x"}`, extractCustomToolCallInput(`{"other": "x"}`))
assert.Equal(t, "", extractCustomToolCallInput(`{}`))
assert.Equal(t, "", extractCustomToolCallInput(""))
}
func TestChatCompletionsChunkToResponsesEvents_CustomToolCallStream(t *testing.T) {
state := NewChatCompletionsToResponsesStreamState("glm-5.2")
state.CustomTools = map[string]bool{"exec": true}
idx := 0
chunk := &ChatCompletionsChunk{
ID: "cc-1",
Choices: []ChatChunkChoice{{
Delta: ChatDelta{
ToolCalls: []ChatToolCall{{
Index: &idx,
ID: "call_1",
Function: ChatFunctionCall{Name: "exec", Arguments: `{"input": "dir"}`},
}},
},
}},
}
events := ChatCompletionsChunkToResponsesEvents(chunk, state)
events = append(events, FinalizeChatCompletionsResponsesStream(state)...)
var added, inputDone, itemDone *ResponsesStreamEvent
for i := range events {
evt := &events[i]
switch evt.Type {
case "response.output_item.added":
if evt.Item != nil && evt.Item.Type != "message" && evt.Item.Type != "reasoning" {
added = evt
}
case "response.custom_tool_call_input.done":
inputDone = evt
case "response.output_item.done":
if evt.Item != nil && evt.Item.Type == "custom_tool_call" {
itemDone = evt
}
case "response.function_call_arguments.delta", "response.function_call_arguments.done":
t.Fatalf("custom 工具调用不应产出 function_call 参数事件: %s", evt.Type)
}
}
require.NotNil(t, added, "缺少 custom_tool_call 的 output_item.added")
assert.Equal(t, "custom_tool_call", added.Item.Type)
assert.Equal(t, "exec", added.Item.Name)
require.NotNil(t, inputDone, "缺少 response.custom_tool_call_input.done")
assert.Equal(t, "dir", inputDone.Input)
assert.Equal(t, "call_1", inputDone.CallID)
require.NotNil(t, itemDone, "缺少 custom_tool_call 的 output_item.done")
assert.Equal(t, "call_1", itemDone.Item.CallID)
assert.Equal(t, "exec", itemDone.Item.Name)
assert.Equal(t, "dir", itemDone.Item.Input)
assert.Empty(t, itemDone.Item.Arguments)
// response.completed 的 output 数组同样携带 custom_tool_call 项。
final := events[len(events)-1]
require.Equal(t, "response.completed", final.Type)
require.NotNil(t, final.Response)
foundCustom := false
for _, item := range final.Response.Output {
if item.Type == "custom_tool_call" {
foundCustom = true
assert.Equal(t, "exec", item.Name)
assert.Equal(t, "dir", item.Input)
}
}
assert.True(t, foundCustom, "response.completed 缺少 custom_tool_call 输出项")
}
func TestResponsesToChatCompletionsRequest_ToolSearchToolBecomesProxyFunction(t *testing.T) {
req := &ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{{Type: "tool_search"}},
}
out, err := ResponsesToChatCompletionsRequest(req)
require.NoError(t, err)
require.Len(t, out.Tools, 1)
assert.Equal(t, "function", out.Tools[0].Type)
assert.Equal(t, "tool_search", out.Tools[0].Function.Name)
assert.Contains(t, string(out.Tools[0].Function.Parameters), `"query"`)
}
// codex 只在 ResponseItem 为 tool_search_call 变体且 execution=client 时执行
// tool search;同名 function_call 会命中 ToolSearchHandler 后因 payload 不匹配
// 触发 FunctionCallError::Fatal,直接中止整个 turn,因此回程必须还原项类型。
func TestChatCompletionsResponseToResponses_ToolSearchCallOutputItem(t *testing.T) {
resp := &ChatCompletionsResponse{
ID: "cc-1",
Choices: []ChatChoice{{
Message: ChatMessage{
Role: "assistant",
ToolCalls: []ChatToolCall{
{ID: "call_s", Function: ChatFunctionCall{Name: "tool_search", Arguments: `{"query":"gmail","limit":2}`}},
},
},
}},
}
out := ChatCompletionsResponseToResponses(resp, "glm-5.2", nil, true, nil)
require.Len(t, out.Output, 1)
item := out.Output[0]
assert.Equal(t, "tool_search_call", item.Type)
assert.Equal(t, "call_s", item.CallID)
// 线上形态:execution 必须为 "client"(codex 的必填字段,非 client 被忽略),
// arguments 必须是 JSON 对象而非字符串(codex 按对象解析 query/limit)。
b, err := json.Marshal(item)
require.NoError(t, err)
var m map[string]any
require.NoError(t, json.Unmarshal(b, &m))
assert.Equal(t, "client", m["execution"])
args, ok := m["arguments"].(map[string]any)
require.True(t, ok, "arguments 必须序列化为 JSON 对象")
assert.Equal(t, "gmail", args["query"])
}
func TestChatCompletionsResponseToResponses_ToolSearchNotDeclaredKeepsFunctionCall(t *testing.T) {
resp := &ChatCompletionsResponse{
Choices: []ChatChoice{{
Message: ChatMessage{
Role: "assistant",
ToolCalls: []ChatToolCall{
{ID: "call_s", Function: ChatFunctionCall{Name: "tool_search", Arguments: `{"query":"gmail"}`}},
},
},
}},
}
// 客户端未声明 type=tool_search 时,同名普通 function 工具不受影响。
out := ChatCompletionsResponseToResponses(resp, "glm-5.2", nil, false, nil)
require.Len(t, out.Output, 1)
assert.Equal(t, "function_call", out.Output[0].Type)
}
func TestChatCompletionsChunkToResponsesEvents_ToolSearchCallStream(t *testing.T) {
state := NewChatCompletionsToResponsesStreamState("glm-5.2")
state.ToolSearchDeclared = true
idx := 0
chunk := &ChatCompletionsChunk{
ID: "cc-1",
Choices: []ChatChunkChoice{{
Delta: ChatDelta{
ToolCalls: []ChatToolCall{{
Index: &idx,
ID: "call_s",
Function: ChatFunctionCall{Name: "tool_search", Arguments: `{"query":"gmail"}`},
}},
},
}},
}
events := ChatCompletionsChunkToResponsesEvents(chunk, state)
events = append(events, FinalizeChatCompletionsResponsesStream(state)...)
var added, itemDone *ResponsesStreamEvent
for i := range events {
evt := &events[i]
switch evt.Type {
case "response.output_item.added":
if evt.Item != nil && evt.Item.Type != "message" && evt.Item.Type != "reasoning" {
added = evt
}
case "response.output_item.done":
if evt.Item != nil && evt.Item.Type == "tool_search_call" {
itemDone = evt
}
case "response.function_call_arguments.delta", "response.function_call_arguments.done",
"response.custom_tool_call_input.delta", "response.custom_tool_call_input.done":
t.Fatalf("tool_search 调用不应产出 %s", evt.Type)
}
}
require.NotNil(t, added, "缺少 tool_search_call 的 output_item.added")
assert.Equal(t, "tool_search_call", added.Item.Type)
require.NotNil(t, itemDone, "缺少 tool_search_call 的 output_item.done")
assert.Equal(t, "call_s", itemDone.Item.CallID)
// SSE 线上形态经 responsesItemWire 白名单重组,必须单独断言。
sse, err := ResponsesEventToSSE(*itemDone)
require.NoError(t, err)
assert.Contains(t, sse, `"execution":"client"`)
assert.Contains(t, sse, `"arguments":{"query":"gmail"}`)
assert.Contains(t, sse, `"call_id":"call_s"`)
// response.completed 的 output 数组同样携带 tool_search_call 项。
final := events[len(events)-1]
require.Equal(t, "response.completed", final.Type)
require.NotNil(t, final.Response)
found := false
for _, item := range final.Response.Output {
if item.Type == "tool_search_call" {
found = true
assert.Equal(t, "call_s", item.CallID)
}
}
assert.True(t, found, "response.completed 缺少 tool_search_call 输出项")
}
func TestHasToolSearchTool(t *testing.T) {
assert.True(t, HasToolSearchTool([]ResponsesTool{{Type: "tool_search"}}))
assert.False(t, HasToolSearchTool([]ResponsesTool{{Type: "function", Name: "tool_search"}}))
assert.False(t, HasToolSearchTool(nil))
}
func TestResponsesToChatCompletionsRequest_NamespaceToolFlattensChildren(t *testing.T) {
req := &ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{{
Type: "namespace",
Name: "gmail",
Tools: []ResponsesTool{
{Type: "function", Name: "send", Description: "Send mail", Parameters: json.RawMessage(`{"type":"object","properties":{}}`)},
{Type: "custom", Name: "ignored_child"},
},
}},
}
out, err := ResponsesToChatCompletionsRequest(req)
require.NoError(t, err)
require.Len(t, out.Tools, 1, "namespace 子工具中仅 function 类型被摊平")
assert.Equal(t, "gmail__send", out.Tools[0].Function.Name)
assert.Equal(t, "Send mail", out.Tools[0].Function.Description)
}
func TestResponsesToolsParsing_StringToolBecomesCustom(t *testing.T) {
var req ResponsesRequest
require.NoError(t, json.Unmarshal([]byte(`{"model":"glm-5.2","input":"hi","tools":["exec",{"type":"function","name":"wait"}]}`), &req))
require.Len(t, req.Tools, 2)
assert.Equal(t, "custom", req.Tools[0].Type)
assert.Equal(t, "exec", req.Tools[0].Name)
assert.Equal(t, "function", req.Tools[1].Type)
assert.True(t, CustomToolNames(req.Tools)["exec"])
}
func TestFlattenNamespaceToolName_CapsAt64WithHashSuffix(t *testing.T) {
assert.Equal(t, "gmail__send", flattenNamespaceToolName("gmail", "send"))
long := flattenNamespaceToolName("very_long_namespace_prefix_for_testing_purposes", "and_a_rather_long_tool_name_too")
assert.LessOrEqual(t, len(long), 64)
assert.Contains(t, long, "__")
// 同输入结果稳定
assert.Equal(t, long, flattenNamespaceToolName("very_long_namespace_prefix_for_testing_purposes", "and_a_rather_long_tool_name_too"))
}
func TestResponsesInputToChatMessages_ToolSearchCallHistory(t *testing.T) {
input := json.RawMessage(`[
{"role":"user","content":"find tools"},
{"type":"tool_search_call","call_id":"call_s","arguments":{"query":"gmail"}},
{"type":"tool_search_output","call_id":"call_s","output":{"groups":["gmail"]}}
]`)
messages, err := responsesInputToChatMessages("", input)
require.NoError(t, err)
require.Len(t, messages, 3)
require.Len(t, messages[1].ToolCalls, 1)
assert.Equal(t, "tool_search", messages[1].ToolCalls[0].Function.Name)
assert.JSONEq(t, `{"query":"gmail"}`, messages[1].ToolCalls[0].Function.Arguments)
assert.Equal(t, "tool", messages[2].Role)
assert.Equal(t, "call_s", messages[2].ToolCallID)
assert.JSONEq(t, `"{\"groups\":[\"gmail\"]}"`, string(messages[2].Content))
}
func TestResponsesInputToChatMessages_NamespacedFunctionCallHistory(t *testing.T) {
input := json.RawMessage(`[
{"type":"function_call","call_id":"call_n","name":"send","namespace":"gmail","arguments":"{\"to\":\"a\"}"},
{"type":"function_call_output","call_id":"call_n","output":"ok"}
]`)
messages, err := responsesInputToChatMessages("", input)
require.NoError(t, err)
require.Len(t, messages, 2)
require.Len(t, messages[0].ToolCalls, 1)
assert.Equal(t, "gmail__send", messages[0].ToolCalls[0].Function.Name)
}
func TestChatCompletionsChunkToResponsesEvents_CustomToolNameArrivesLate(t *testing.T) {
state := NewChatCompletionsToResponsesStreamState("glm-5.2")
state.CustomTools = map[string]bool{"exec": true}
idx := 0
chunk1 := &ChatCompletionsChunk{Choices: []ChatChunkChoice{{Delta: ChatDelta{
ToolCalls: []ChatToolCall{{Index: &idx, ID: "call_1", Function: ChatFunctionCall{Arguments: `{"inp`}}},
}}}}
chunk2 := &ChatCompletionsChunk{Choices: []ChatChunkChoice{{Delta: ChatDelta{
ToolCalls: []ChatToolCall{{Index: &idx, Function: ChatFunctionCall{Name: "exec", Arguments: `ut": "dir"}`}}},
}}}}
var events []ResponsesStreamEvent
events = append(events, ChatCompletionsChunkToResponsesEvents(chunk1, state)...)
events = append(events, ChatCompletionsChunkToResponsesEvents(chunk2, state)...)
events = append(events, FinalizeChatCompletionsResponsesStream(state)...)
addedCount := 0
for _, evt := range events {
switch evt.Type {
case "response.output_item.added":
if evt.Item != nil && evt.Item.Type != "reasoning" && evt.Item.Type != "message" {
addedCount++
assert.Equal(t, "custom_tool_call", evt.Item.Type, "迟到的名字命中 custom 工具时按 custom_tool_call 宣告")
assert.Equal(t, "exec", evt.Item.Name)
}
case "response.function_call_arguments.delta", "response.function_call_arguments.done":
t.Fatalf("custom 调用不应产出 function 参数事件: %s", evt.Type)
case "response.custom_tool_call_input.done":
assert.Equal(t, "dir", evt.Input)
}
}
assert.Equal(t, 1, addedCount, "工具调用只宣告一次")
}
func TestChatCompletionsChunkToResponsesEvents_FunctionToolNameArrivesLate(t *testing.T) {
state := NewChatCompletionsToResponsesStreamState("glm-5.2")
state.CustomTools = map[string]bool{"exec": true}
idx := 0
chunk1 := &ChatCompletionsChunk{Choices: []ChatChunkChoice{{Delta: ChatDelta{
ToolCalls: []ChatToolCall{{Index: &idx, ID: "call_9", Function: ChatFunctionCall{Arguments: `{"cell`}}},
}}}}
chunk2 := &ChatCompletionsChunk{Choices: []ChatChunkChoice{{Delta: ChatDelta{
ToolCalls: []ChatToolCall{{Index: &idx, Function: ChatFunctionCall{Name: "wait", Arguments: `_id": 3}`}}},
}}}}
var events []ResponsesStreamEvent
events = append(events, ChatCompletionsChunkToResponsesEvents(chunk1, state)...)
events = append(events, ChatCompletionsChunkToResponsesEvents(chunk2, state)...)
events = append(events, FinalizeChatCompletionsResponsesStream(state)...)
deltas := ""
argsDone := ""
for _, evt := range events {
switch evt.Type {
case "response.function_call_arguments.delta":
deltas += evt.Delta
case "response.function_call_arguments.done":
argsDone = evt.Arguments
case "response.custom_tool_call_input.done":
t.Fatal("function 调用不应产出 custom 事件")
}
}
assert.Equal(t, `{"cell_id": 3}`, deltas, "宣告前累积的参数需在宣告时补发")
assert.Equal(t, `{"cell_id": 3}`, argsDone)
}
// 序列化层(MarshalJSON → responsesItemWire)单独走白名单重组,事件结构体上的字段
// 齐全不代表落到 SSE 线上的 JSON 齐全,必须在 wire 层再断言一次。
func TestResponsesEventToSSE_CustomToolCallItemCarriesAllFields(t *testing.T) {
evt := ResponsesStreamEvent{
Type: "response.output_item.done",
OutputIndex: 1,
Item: &ResponsesOutput{
Type: "custom_tool_call",
ID: "item_1",
CallID: "call_1",
Name: "exec",
Input: "dir",
Status: "completed",
},
}
sse, err := ResponsesEventToSSE(evt)
require.NoError(t, err)
assert.Contains(t, sse, `"call_id":"call_1"`)
assert.Contains(t, sse, `"name":"exec"`)
assert.Contains(t, sse, `"input":"dir"`)
assert.Contains(t, sse, `"type":"custom_tool_call"`)
}
func TestNamespaceToolNames_MapsFlattenedNames(t *testing.T) {
tools := []ResponsesTool{
{Type: "namespace", Name: "gmail", Tools: []ResponsesTool{
{Type: "function", Name: "send"},
{Type: "custom", Name: "skip_me"},
}},
{Type: "namespace", Name: "crm", Children: []ResponsesTool{
{Type: "function", Name: "query"},
}},
{Type: "function", Name: "wait"},
}
m := NamespaceToolNames(tools)
require.Len(t, m, 2)
assert.Equal(t, NamespacedToolName{Namespace: "gmail", Name: "send"}, m["gmail__send"])
assert.Equal(t, NamespacedToolName{Namespace: "crm", Name: "query"}, m["crm__query"])
// 摊平名超长时截断加哈希,无法按字符串切分还原,必须经映射反查。
longNS := "very_long_namespace_prefix_for_testing_purposes"
longChild := "and_a_rather_long_tool_name_too"
m2 := NamespaceToolNames([]ResponsesTool{{
Type: "namespace", Name: longNS,
Tools: []ResponsesTool{{Type: "function", Name: longChild}},
}})
assert.Equal(t, NamespacedToolName{Namespace: longNS, Name: longChild},
m2[flattenNamespaceToolName(longNS, longChild)])
assert.Nil(t, NamespaceToolNames(nil))
}
// 内置 tool_search 降级后的代理 function 与客户端声明的同名工具无法区分:回程会把
// 普通工具的调用劫持成 tool_search_call,必须显式拒绝(代理不能改名,codex 的模型
// 侧按 tool_search 这个名字调用)。
func TestResponsesToChatCompletionsRequest_RejectsToolSearchNameConflict(t *testing.T) {
// 与顶层 function 工具同名。
_, err := ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{
{Type: "tool_search"},
{Type: "function", Name: "tool_search"},
},
})
require.Error(t, err, "与内置 tool_search 代理撞名的 function 工具必须拒绝")
assert.Contains(t, err.Error(), "tool_search")
// 与顶层 custom 工具同名。
_, err = ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{
{Type: "custom", Name: "tool_search"},
{Type: "tool_search"},
},
})
require.Error(t, err, "与内置 tool_search 代理撞名的 custom 工具必须拒绝")
// 重复声明 type=tool_search 去重后只产出一个代理,不拒绝。
out, err := ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{{Type: "tool_search"}, {Type: "tool_search"}},
})
require.NoError(t, err)
require.Len(t, out.Tools, 1)
assert.Equal(t, "tool_search", out.Tools[0].Function.Name)
}
// tool_choice 指向被转换丢弃的工具(如 web_search)或不存在的名字时不能原样转发,
// chat 上游会因选择项指向未声明工具而 400;字符串形式与指向幸存工具的选择保持转发。
func TestResponsesToChatCompletionsRequest_DropsToolChoiceForDroppedTool(t *testing.T) {
// 强制选择被丢弃的 web_search:工具没了,选择项也必须丢。
out, err := ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{
{Type: "function", Name: "wait", Parameters: json.RawMessage(`{"type":"object","properties":{}}`)},
{Type: "web_search"},
},
ToolChoice: json.RawMessage(`{"type":"web_search"}`),
})
require.NoError(t, err)
require.Len(t, out.Tools, 1)
assert.Empty(t, out.ToolChoice, "指向被丢弃服务端工具的 tool_choice 必须丢弃")
// 具名选择指向不存在的工具名。
out, err = ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{{Type: "function", Name: "wait"}},
ToolChoice: json.RawMessage(`{"type":"function","name":"missing"}`),
})
require.NoError(t, err)
assert.Empty(t, out.ToolChoice, "指向不存在工具名的 tool_choice 必须丢弃")
// 字符串形式与指向幸存工具的选择保持原有转发行为。
out, err = ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{{Type: "function", Name: "wait"}},
ToolChoice: json.RawMessage(`"auto"`),
})
require.NoError(t, err)
assert.JSONEq(t, `"auto"`, string(out.ToolChoice))
out, err = ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{{Type: "function", Name: "wait"}},
ToolChoice: json.RawMessage(`{"type":"function","name":"wait"}`),
})
require.NoError(t, err)
assert.JSONEq(t, `{"type":"function","function":{"name":"wait"}}`, string(out.ToolChoice))
}
// tool_search 工具没有被丢弃而是降级为同名 function 代理,强制选择它的 tool_choice
// 必须同步降级为指向代理的 function 选择,不能静默丢弃(丢弃会把强制搜索退化为
// 自动选择,模型可以不执行搜索)。
func TestResponsesToChatCompletionsRequest_ToolSearchToolChoiceMapsToProxy(t *testing.T) {
out, err := ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{{Type: "tool_search"}},
ToolChoice: json.RawMessage(`{"type":"tool_search"}`),
})
require.NoError(t, err)
assert.JSONEq(t, `{"type":"function","function":{"name":"tool_search"}}`, string(out.ToolChoice))
// 未声明 type=tool_search 时强制选择它没有可指向的代理,丢弃选择项。
out, err = ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{{Type: "function", Name: "wait"}},
ToolChoice: json.RawMessage(`{"type":"tool_search"}`),
})
require.NoError(t, err)
assert.Empty(t, out.ToolChoice)
}
// 客户端请求在原生 Responses API 上合法(namespace 子工具按 namespace+name 路由),
// 是摊平转换让名字产生歧义;歧义无法消除时必须显式拒绝整个请求(400),而不是
// 静默降级——否则重复声明发给上游、回程还原到错误工具,问题只能靠抓包定位。
func TestResponsesToChatCompletionsRequest_RejectsAmbiguousFlattenedNames(t *testing.T) {
// 摊平名与顶层 function 工具撞名。
_, err := ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{
{Type: "function", Name: "gmail__send"},
{Type: "namespace", Name: "gmail", Tools: []ResponsesTool{{Type: "function", Name: "send"}}},
},
})
require.Error(t, err, "与顶层工具撞名的摊平必须拒绝")
assert.Contains(t, err.Error(), "gmail__send")
// 不同 namespace 组合产生相同摊平名。
_, err = ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{
{Type: "namespace", Name: "a", Tools: []ResponsesTool{{Type: "function", Name: "b__c"}}},
{Type: "namespace", Name: "a__b", Tools: []ResponsesTool{{Type: "function", Name: "c"}}},
},
})
require.Error(t, err, "跨 namespace 撞名的摊平必须拒绝")
assert.Contains(t, err.Error(), "a__b__c")
}
// 完全相同的 (namespace, 子工具) 重复声明不构成歧义:去重后正常转换,不拒绝。
func TestResponsesToChatCompletionsRequest_DedupesIdenticalNamespaceChildren(t *testing.T) {
out, err := ResponsesToChatCompletionsRequest(&ResponsesRequest{
Model: "glm-5.2",
Input: json.RawMessage(`"hi"`),
Tools: []ResponsesTool{
{Type: "namespace", Name: "gmail", Tools: []ResponsesTool{
{Type: "function", Name: "send"},
{Type: "function", Name: "send"},
}},
},
})
require.NoError(t, err)
require.Len(t, out.Tools, 1, "重复声明的同一子工具只声明一次")
assert.Equal(t, "gmail__send", out.Tools[0].Function.Name)
}
// codex 按 namespace+name 路由 namespace 子工具的调用:回程必须把摊平名还原为
// 裸子工具名并带独立 namespace 字段,平铺名的 function_call 会被 codex 判为
// unsupported call 拒绝执行。
func TestChatCompletionsResponseToResponses_NamespacedToolCallRestored(t *testing.T) {
resp := &ChatCompletionsResponse{
ID: "cc-1",
Choices: []ChatChoice{{
Message: ChatMessage{
Role: "assistant",
ToolCalls: []ChatToolCall{
{ID: "call_n", Function: ChatFunctionCall{Name: "mcp__svc__echo", Arguments: `{"text":"hi"}`}},
{ID: "call_9", Function: ChatFunctionCall{Name: "wait", Arguments: `{"cell_id": 3}`}},
},
},
}},
}
nsTools := map[string]NamespacedToolName{
"mcp__svc__echo": {Namespace: "mcp__svc", Name: "echo"},
}
out := ChatCompletionsResponseToResponses(resp, "glm-5.2", nil, false, nsTools)
require.Len(t, out.Output, 2)
item := out.Output[0]
assert.Equal(t, "function_call", item.Type)
assert.Equal(t, "echo", item.Name)
assert.Equal(t, "mcp__svc", item.Namespace)
assert.Equal(t, "call_n", item.CallID)
assert.Equal(t, `{"text":"hi"}`, item.Arguments)
// 非流式响应体走 ResponsesOutput.MarshalJSON,namespace 必须落到线上 JSON。
b, err := json.Marshal(item)
require.NoError(t, err)
assert.Contains(t, string(b), `"namespace":"mcp__svc"`)
assert.Contains(t, string(b), `"name":"echo"`)
// 未命中映射的普通 function 调用不受影响,且不携带 namespace 字段。
assert.Equal(t, "wait", out.Output[1].Name)
assert.Empty(t, out.Output[1].Namespace)
b2, err := json.Marshal(out.Output[1])
require.NoError(t, err)
assert.NotContains(t, string(b2), `"namespace"`)
}
func TestChatCompletionsChunkToResponsesEvents_NamespacedToolCallStream(t *testing.T) {
state := NewChatCompletionsToResponsesStreamState("glm-5.2")
state.NamespaceTools = map[string]NamespacedToolName{
"mcp__svc__echo": {Namespace: "mcp__svc", Name: "echo"},
}
idx := 0
chunk := &ChatCompletionsChunk{
ID: "cc-1",
Choices: []ChatChunkChoice{{
Delta: ChatDelta{
ToolCalls: []ChatToolCall{{
Index: &idx,
ID: "call_n",
Function: ChatFunctionCall{Name: "mcp__svc__echo", Arguments: `{"text":"hi"}`},
}},
},
}},
}
events := ChatCompletionsChunkToResponsesEvents(chunk, state)
events = append(events, FinalizeChatCompletionsResponsesStream(state)...)
var added, itemDone *ResponsesStreamEvent
for i := range events {
evt := &events[i]
switch evt.Type {
case "response.output_item.added":
if evt.Item != nil && evt.Item.Type != "message" && evt.Item.Type != "reasoning" {
added = evt
}
case "response.output_item.done":
if evt.Item != nil && evt.Item.Type == "function_call" {
itemDone = evt
}
case "response.custom_tool_call_input.delta", "response.custom_tool_call_input.done":
t.Fatalf("namespace 子工具调用不应产出 custom 事件: %s", evt.Type)
}
}
require.NotNil(t, added, "缺少 namespace 调用的 output_item.added")
assert.Equal(t, "function_call", added.Item.Type)
assert.Equal(t, "echo", added.Item.Name)
assert.Equal(t, "mcp__svc", added.Item.Namespace)
require.NotNil(t, itemDone, "缺少 namespace 调用的 output_item.done")
assert.Equal(t, "call_n", itemDone.Item.CallID)
assert.Equal(t, "echo", itemDone.Item.Name)
assert.Equal(t, "mcp__svc", itemDone.Item.Namespace)
assert.Equal(t, `{"text":"hi"}`, itemDone.Item.Arguments)
// SSE 线上形态经 responsesItemWire 白名单重组,必须单独断言 namespace 落线。
sse, err := ResponsesEventToSSE(*itemDone)
require.NoError(t, err)
assert.Contains(t, sse, `"namespace":"mcp__svc"`)
assert.Contains(t, sse, `"name":"echo"`)
assert.Contains(t, sse, `"call_id":"call_n"`)
// response.completed 的 output 数组同样携带还原后的 namespace 调用项。
final := events[len(events)-1]
require.Equal(t, "response.completed", final.Type)
require.NotNil(t, final.Response)
found := false
for _, item := range final.Response.Output {
if item.Type == "function_call" {
found = true
assert.Equal(t, "echo", item.Name)
assert.Equal(t, "mcp__svc", item.Namespace)
}
}
assert.True(t, found, "response.completed 缺少还原后的 namespace 调用项")
}
func TestChatCompletionsChunkToResponsesEvents_NamespacedToolNameArrivesLate(t *testing.T) {
state := NewChatCompletionsToResponsesStreamState("glm-5.2")
state.NamespaceTools = map[string]NamespacedToolName{
"mcp__svc__echo": {Namespace: "mcp__svc", Name: "echo"},
}
idx := 0
chunk1 := &ChatCompletionsChunk{Choices: []ChatChunkChoice{{Delta: ChatDelta{
ToolCalls: []ChatToolCall{{Index: &idx, ID: "call_n", Function: ChatFunctionCall{Arguments: `{"te`}}},
}}}}
chunk2 := &ChatCompletionsChunk{Choices: []ChatChunkChoice{{Delta: ChatDelta{
ToolCalls: []ChatToolCall{{Index: &idx, Function: ChatFunctionCall{Name: "mcp__svc__echo", Arguments: `xt":"hi"}`}}},
}}}}
var events []ResponsesStreamEvent
events = append(events, ChatCompletionsChunkToResponsesEvents(chunk1, state)...)
events = append(events, ChatCompletionsChunkToResponsesEvents(chunk2, state)...)
events = append(events, FinalizeChatCompletionsResponsesStream(state)...)
addedCount := 0
deltas := ""
for _, evt := range events {
switch evt.Type {
case "response.output_item.added":
if evt.Item != nil && evt.Item.Type != "reasoning" && evt.Item.Type != "message" {
addedCount++
assert.Equal(t, "echo", evt.Item.Name, "迟到的名字命中 namespace 映射时按还原名宣告")
assert.Equal(t, "mcp__svc", evt.Item.Namespace)
}
case "response.function_call_arguments.delta":
deltas += evt.Delta
}
}
assert.Equal(t, 1, addedCount, "工具调用只宣告一次")
assert.Equal(t, `{"text":"hi"}`, deltas, "宣告前累积的参数需在宣告时补发")
}
func TestChatCompletionsChunkToResponsesEvents_FunctionToolStreamUnaffected(t *testing.T) {
state := NewChatCompletionsToResponsesStreamState("glm-5.2")
state.CustomTools = map[string]bool{"exec": true}
idx := 0
chunk := &ChatCompletionsChunk{
Choices: []ChatChunkChoice{{
Delta: ChatDelta{
ToolCalls: []ChatToolCall{{
Index: &idx,
ID: "call_9",
Function: ChatFunctionCall{Name: "wait", Arguments: `{"cell_id": 3}`},
}},
},
}},
}
events := ChatCompletionsChunkToResponsesEvents(chunk, state)
events = append(events, FinalizeChatCompletionsResponsesStream(state)...)
sawArgsDelta := false
for _, evt := range events {
if evt.Type == "response.function_call_arguments.delta" {
sawArgsDelta = true
}
if evt.Type == "response.custom_tool_call_input.done" {
t.Fatal("function 工具不应产出 custom_tool_call 事件")
}
}
assert.True(t, sawArgsDelta, "function 工具应保持原有参数增量事件")
}
@@ -459,7 +459,7 @@ func TestChatCompletionsResponseToResponses_DeepSeekReasoningOnlyFallsBackToMess
}},
}
out := ChatCompletionsResponseToResponses(resp, "deepseek-reasoner")
out := ChatCompletionsResponseToResponses(resp, "deepseek-reasoner", nil, false, nil)
require.Len(t, out.Output, 2)
require.Equal(t, "reasoning", out.Output[0].Type)
@@ -493,7 +493,7 @@ func TestChatCompletionsResponseToResponses_DeepSeekReasoningToolCallDoesNotFall
}},
}
out := ChatCompletionsResponseToResponses(resp, "deepseek-reasoner")
out := ChatCompletionsResponseToResponses(resp, "deepseek-reasoner", nil, false, nil)
require.Len(t, out.Output, 2)
require.Equal(t, "reasoning", out.Output[0].Type)
@@ -0,0 +1,99 @@
package apicompat
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestAnthropicUsageFromResponsesUsage_CacheCreation(t *testing.T) {
usage := &ResponsesUsage{
InputTokens: 20,
OutputTokens: 5,
CacheCreationInputTokens: 6,
InputTokensDetails: &ResponsesInputTokensDetails{
CachedTokens: 4,
},
}
got := anthropicUsageFromResponsesUsage(usage)
assert.Equal(t, 10, got.InputTokens, "input = total(20) - cache_read(4) - cache_creation(6)")
assert.Equal(t, 5, got.OutputTokens)
assert.Equal(t, 4, got.CacheReadInputTokens)
assert.Equal(t, 6, got.CacheCreationInputTokens, "cache creation must be preserved")
}
func TestAnthropicUsageFromResponsesUsage_NoCacheCreation(t *testing.T) {
usage := &ResponsesUsage{
InputTokens: 10,
OutputTokens: 5,
InputTokensDetails: &ResponsesInputTokensDetails{
CachedTokens: 3,
},
}
got := anthropicUsageFromResponsesUsage(usage)
assert.Equal(t, 7, got.InputTokens)
assert.Equal(t, 3, got.CacheReadInputTokens)
assert.Equal(t, 0, got.CacheCreationInputTokens)
}
func TestResponsesEventToAnthropicEvents_StreamingCacheCreation(t *testing.T) {
state := NewResponsesEventToAnthropicState()
state.MessageStartSent = true
completedEvt := &ResponsesStreamEvent{
Type: "response.completed",
Response: &ResponsesResponse{
Status: "completed",
Usage: &ResponsesUsage{
InputTokens: 20,
OutputTokens: 5,
CacheCreationInputTokens: 6,
InputTokensDetails: &ResponsesInputTokensDetails{
CachedTokens: 4,
},
},
},
}
events := ResponsesEventToAnthropicEvents(completedEvt, state)
var deltaEvt *AnthropicStreamEvent
for i := range events {
if events[i].Type == "message_delta" {
deltaEvt = &events[i]
break
}
}
require.NotNil(t, deltaEvt, "should have message_delta event")
require.NotNil(t, deltaEvt.Usage)
assert.Equal(t, 6, deltaEvt.Usage.CacheCreationInputTokens, "streaming cache_creation must be preserved")
assert.Equal(t, 10, deltaEvt.Usage.InputTokens, "input = 20 - 4(read) - 6(creation)")
assert.Equal(t, 4, deltaEvt.Usage.CacheReadInputTokens)
}
func TestAnthropicToResponsesResponse_CacheCreation(t *testing.T) {
resp := AnthropicResponse{
ID: "msg_test",
Type: "message",
Role: "assistant",
Model: "claude-opus-4-6",
Usage: AnthropicUsage{
InputTokens: 10,
OutputTokens: 5,
CacheReadInputTokens: 4,
CacheCreationInputTokens: 6,
},
StopReason: "end_turn",
}
out := AnthropicToResponsesResponse(&resp)
require.NotNil(t, out.Usage)
assert.Equal(t, 20, out.Usage.InputTokens, "total = input(10) + cache_read(4) + cache_creation(6)")
assert.Equal(t, 6, out.Usage.CacheCreationInputTokens, "cache creation must round-trip")
}
@@ -86,6 +86,23 @@ func (e ResponsesStreamEvent) MarshalJSON() ([]byte, error) {
}
return json.Marshal(m)
case "response.custom_tool_call_input.delta", "response.custom_tool_call_input.done":
m := e.wireBase()
e.putItemID(m)
m["output_index"] = e.OutputIndex
if e.CallID != "" {
m["call_id"] = e.CallID
}
if e.Name != "" {
m["name"] = e.Name
}
if e.Type == "response.custom_tool_call_input.done" {
m["input"] = e.Input
} else {
m["delta"] = e.Delta
}
return json.Marshal(m)
default:
// response.created / completed / done / failed / incomplete and any
// event type not shaped above keep the default struct marshalling.
@@ -167,6 +184,23 @@ func responsesItemWire(item *ResponsesOutput) map[string]any {
m["call_id"] = item.CallID
m["name"] = item.Name
m["arguments"] = item.Arguments
// namespace 子工具的还原调用:codex 按 namespace+name 路由,缺少该字段
// 会被判为 unsupported call。
if item.Namespace != "" {
m["namespace"] = item.Namespace
}
case "custom_tool_call":
// custom/freeform 工具调用(如 codex 的 exec):input 为自由文本。缺少
// call_id/name 时 codex 无法路由该调用(表现为 unsupported call)。
m["call_id"] = item.CallID
m["name"] = item.Name
m["input"] = item.Input
case "tool_search_call":
// tool_search 调用还原项:execution 必须为 "client"(否则 codex 忽略该
// 调用),arguments 在线上是 JSON 对象而非字符串。
m["call_id"] = item.CallID
m["execution"] = "client"
m["arguments"] = toolSearchCallArgumentsJSON(item.Arguments)
}
return m
}
@@ -102,6 +102,26 @@ func TestWire_ArgumentsDonePresentEvenEmpty(t *testing.T) {
require.Equal(t, "", m["arguments"])
}
// TestWire_CustomToolCallInputIndexPresentAtZero guards the omitempty trap for
// custom_tool_call_input.delta/done: output_index must serialize even when 0
// (custom tool call as the first output item).
func TestWire_CustomToolCallInputIndexPresentAtZero(t *testing.T) {
d := marshalEvent(t, ResponsesStreamEvent{
Type: "response.custom_tool_call_input.delta", OutputIndex: 0, ItemID: "ct_1", Delta: "dir",
})
require.Contains(t, d, "output_index")
require.EqualValues(t, 0, d["output_index"])
require.Equal(t, "dir", d["delta"])
done := marshalEvent(t, ResponsesStreamEvent{
Type: "response.custom_tool_call_input.done", OutputIndex: 0, ItemID: "ct_1", CallID: "call_1", Name: "exec", Input: "dir",
})
require.Contains(t, done, "output_index")
require.EqualValues(t, 0, done["output_index"])
require.Equal(t, "dir", done["input"])
require.NotContains(t, done, "delta")
}
// TestWire_UnknownEventFallsBackToDefault ensures non-streamed event types keep
// default marshalling (the response object is preserved).
func TestWire_UnknownEventFallsBackToDefault(t *testing.T) {
@@ -111,3 +131,56 @@ func TestWire_UnknownEventFallsBackToDefault(t *testing.T) {
})
require.Contains(t, m, "response")
}
func TestResponsesOutputUnmarshal_ToolSearchObjectArguments(t *testing.T) {
var item ResponsesOutput
require.NoError(t, json.Unmarshal([]byte(`{
"type":"tool_search_call",
"id":"item_1",
"call_id":"call_1",
"execution":"client",
"arguments":{"query":"gmail","limit":2}
}`), &item))
require.Equal(t, "tool_search_call", item.Type)
require.Equal(t, `{"query":"gmail","limit":2}`, item.Arguments)
wire, err := json.Marshal(item)
require.NoError(t, err)
var decoded map[string]any
require.NoError(t, json.Unmarshal(wire, &decoded))
args, ok := decoded["arguments"].(map[string]any)
require.True(t, ok, "tool_search_call arguments must remain an object")
require.Equal(t, "gmail", args["query"])
}
func TestResponsesResponseUnmarshal_ToolSearchObjectArguments(t *testing.T) {
var response ResponsesResponse
require.NoError(t, json.Unmarshal([]byte(`{
"id":"response_1",
"object":"response",
"status":"completed",
"output":[{
"type":"tool_search_call",
"id":"item_1",
"call_id":"call_1",
"arguments":{"query":"gmail"}
}]
}`), &response))
require.Len(t, response.Output, 1)
require.Equal(t, `{"query":"gmail"}`, response.Output[0].Arguments)
}
func TestResponsesStreamEventUnmarshal_ToolSearchObjectArguments(t *testing.T) {
var event ResponsesStreamEvent
require.NoError(t, json.Unmarshal([]byte(`{
"type":"response.output_item.done",
"item":{
"type":"tool_search_call",
"id":"item_1",
"call_id":"call_1",
"arguments":{"query":"gmail"}
}
}`), &event))
require.NotNil(t, event.Item)
require.Equal(t, `{"query":"gmail"}`, event.Item.Arguments)
}
@@ -100,15 +100,16 @@ func anthropicUsageFromResponsesUsage(usage *ResponsesUsage) AnthropicUsage {
cachedTokens = usage.InputTokensDetails.CachedTokens
}
inputTokens := usage.InputTokens - cachedTokens
inputTokens := usage.InputTokens - cachedTokens - usage.CacheCreationInputTokens
if inputTokens < 0 {
inputTokens = 0
}
return AnthropicUsage{
InputTokens: inputTokens,
OutputTokens: usage.OutputTokens,
CacheReadInputTokens: cachedTokens,
InputTokens: inputTokens,
OutputTokens: usage.OutputTokens,
CacheReadInputTokens: cachedTokens,
CacheCreationInputTokens: usage.CacheCreationInputTokens,
}
}
@@ -181,9 +182,10 @@ type ResponsesEventToAnthropicState struct {
// OutputIndexToBlockIdx maps Responses output_index → Anthropic content block index.
OutputIndexToBlockIdx map[int]int
InputTokens int
OutputTokens int
CacheReadInputTokens int
InputTokens int
OutputTokens int
CacheReadInputTokens int
CacheCreationInputTokens int
ResponseID string
Model string
@@ -258,9 +260,10 @@ func FinalizeResponsesAnthropicStream(state *ResponsesEventToAnthropicState) []A
StopReason: stopReason,
},
Usage: &AnthropicUsage{
InputTokens: state.InputTokens,
OutputTokens: state.OutputTokens,
CacheReadInputTokens: state.CacheReadInputTokens,
InputTokens: state.InputTokens,
OutputTokens: state.OutputTokens,
CacheReadInputTokens: state.CacheReadInputTokens,
CacheCreationInputTokens: state.CacheCreationInputTokens,
},
},
AnthropicStreamEvent{Type: "message_stop"},
@@ -578,6 +581,7 @@ func resToAnthHandleCompleted(evt *ResponsesStreamEvent, state *ResponsesEventTo
state.InputTokens = usage.InputTokens
state.OutputTokens = usage.OutputTokens
state.CacheReadInputTokens = usage.CacheReadInputTokens
state.CacheCreationInputTokens = usage.CacheCreationInputTokens
}
if evt.Response != nil {
if evt.Response.Usage != nil {
@@ -585,6 +589,7 @@ func resToAnthHandleCompleted(evt *ResponsesStreamEvent, state *ResponsesEventTo
state.InputTokens = usage.InputTokens
state.OutputTokens = usage.OutputTokens
state.CacheReadInputTokens = usage.CacheReadInputTokens
state.CacheCreationInputTokens = usage.CacheCreationInputTokens
}
switch evt.Response.Status {
case "incomplete":
@@ -605,9 +610,10 @@ func resToAnthHandleCompleted(evt *ResponsesStreamEvent, state *ResponsesEventTo
StopReason: stopReason,
},
Usage: &AnthropicUsage{
InputTokens: state.InputTokens,
OutputTokens: state.OutputTokens,
CacheReadInputTokens: state.CacheReadInputTokens,
InputTokens: state.InputTokens,
OutputTokens: state.OutputTokens,
CacheReadInputTokens: state.CacheReadInputTokens,
CacheCreationInputTokens: state.CacheCreationInputTokens,
},
},
AnthropicStreamEvent{Type: "message_stop"},
+101 -1
View File
@@ -249,11 +249,31 @@ type ResponsesContentPart struct {
// ResponsesTool describes a tool in the Responses API.
type ResponsesTool struct {
Type string `json:"type"` // "function" | "web_search" | "local_shell" etc.
Type string `json:"type"` // "function" | "custom" | "web_search" | "local_shell" etc.
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Parameters json.RawMessage `json:"parameters,omitempty"`
Strict *bool `json:"strict,omitempty"`
// type=namespace 的子工具列表(tools 与 children 二选一,语义相同)。
Tools []ResponsesTool `json:"tools,omitempty"`
Children []ResponsesTool `json:"children,omitempty"`
}
// UnmarshalJSON 容忍字符串形式的工具声明:codex 会以 "name" 简写声明 custom 工具,
func (t *ResponsesTool) UnmarshalJSON(data []byte) error {
var name string
if err := json.Unmarshal(data, &name); err == nil {
*t = ResponsesTool{Type: "custom", Name: name}
return nil
}
type alias ResponsesTool
var a alias
if err := json.Unmarshal(data, &a); err != nil {
return err
}
*t = ResponsesTool(a)
return nil
}
// ResponsesResponse is the non-streaming response from POST /v1/responses.
@@ -301,11 +321,88 @@ type ResponsesOutput struct {
CallID string `json:"call_id,omitempty"`
Name string `json:"name,omitempty"`
Arguments string `json:"arguments,omitempty"`
// 来源为 namespace 子工具时的归属命名空间(codex 按 namespace+name 路由该调用)。
Namespace string `json:"namespace,omitempty"`
// type=custom_tool_call(custom/freeform 工具,input 为自由文本)
Input string `json:"input,omitempty"`
// type=web_search_call
Action *WebSearchAction `json:"action,omitempty"`
}
// MarshalJSON 处理 tool_search_call 项的线上形态(复用 CallID/Arguments 字段):
// execution 固定为 "client"(codex 的必填字段,非 client 的调用会被静默忽略),
// arguments 是 JSON 对象而非 function_call 语义下的字符串。其余类型走默认结构体
// 序列化,输出逐字节不变。
func (o ResponsesOutput) MarshalJSON() ([]byte, error) {
type responsesOutputAlias ResponsesOutput
if o.Type != "tool_search_call" {
return json.Marshal(responsesOutputAlias(o))
}
m := map[string]any{
"type": o.Type,
"id": o.ID,
"call_id": o.CallID,
"execution": "client",
"arguments": toolSearchCallArgumentsJSON(o.Arguments),
}
if o.Status != "" {
m["status"] = o.Status
}
return json.Marshal(m)
}
// UnmarshalJSON accepts both the Responses function-call string form and the
// tool_search_call object form for arguments. The bridge stores arguments as a
// string internally, so object arguments are retained as their raw JSON.
func (o *ResponsesOutput) UnmarshalJSON(data []byte) error {
type responsesOutputAlias ResponsesOutput
var kind struct {
Type string `json:"type"`
}
if err := json.Unmarshal(data, &kind); err != nil {
return err
}
if kind.Type != "tool_search_call" {
var decoded responsesOutputAlias
if err := json.Unmarshal(data, &decoded); err != nil {
return err
}
*o = ResponsesOutput(decoded)
return nil
}
var fields map[string]json.RawMessage
if err := json.Unmarshal(data, &fields); err != nil {
return err
}
arguments, hasArguments := fields["arguments"]
delete(fields, "arguments")
normalized, err := json.Marshal(fields)
if err != nil {
return err
}
var decoded responsesOutputAlias
if err := json.Unmarshal(normalized, &decoded); err != nil {
return err
}
*o = ResponsesOutput(decoded)
if !hasArguments || string(arguments) == "null" {
return nil
}
var argumentString string
if err := json.Unmarshal(arguments, &argumentString); err == nil {
o.Arguments = argumentString
} else {
o.Arguments = string(arguments)
}
return nil
}
// WebSearchAction describes the search action in a web_search_call output item.
type WebSearchAction struct {
Type string `json:"type,omitempty"` // "search"
@@ -444,6 +541,9 @@ type ResponsesStreamEvent struct {
Name string `json:"name,omitempty"`
Arguments string `json:"arguments,omitempty"`
// response.custom_tool_call_input.done
Input string `json:"input,omitempty"`
// response.reasoning_summary_text.delta / done
// Reuses Text/Delta fields above, SummaryIndex identifies which summary part
SummaryIndex int `json:"summary_index,omitempty"`
@@ -1286,6 +1286,38 @@ func (r *accountRepository) SetRateLimited(ctx context.Context, id int64, resetA
return nil
}
// SetRateLimitedIfLater atomically extends an account-level rate limit. Grok
// requests may finish concurrently, so an older response must not overwrite a
// later reset boundary observed by another request or instance.
func (r *accountRepository) SetRateLimitedIfLater(ctx context.Context, id int64, resetAt time.Time) error {
now := time.Now()
updated, err := r.client.Account.Update().
Where(
dbaccount.IDEQ(id),
dbaccount.Or(
dbaccount.RateLimitResetAtIsNil(),
dbaccount.RateLimitResetAtLT(resetAt),
),
).
SetRateLimitedAt(now).
SetRateLimitResetAt(resetAt).
Save(ctx)
if err != nil {
return err
}
if updated == 0 {
// This instance may not have observed the later value written elsewhere.
// Refresh its local scheduler snapshot even though no outbox event is needed.
r.syncSchedulerAccountSnapshot(ctx, id)
return nil
}
if err := enqueueSchedulerOutbox(ctx, r.sql, service.SchedulerOutboxEventAccountChanged, &id, nil, nil); err != nil {
logger.LegacyPrintf("repository.account", "[SchedulerOutbox] enqueue extended rate limit failed: account=%d err=%v", id, err)
}
r.syncSchedulerAccountSnapshot(ctx, id)
return nil
}
func (r *accountRepository) SetModelRateLimit(ctx context.Context, id int64, scope string, resetAt time.Time, reason ...string) error {
if scope == "" {
return nil
@@ -703,6 +703,25 @@ func (s *AccountRepoSuite) TestSetRateLimited() {
s.Require().WithinDuration(resetAt, *got.RateLimitResetAt, time.Second)
}
func (s *AccountRepoSuite) TestSetRateLimitedIfLaterDoesNotShortenReset() {
account := mustCreateAccount(s.T(), s.client, &service.Account{Name: "acc-rl-monotonic"})
later := time.Now().Add(30 * time.Minute).UTC().Truncate(time.Second)
earlier := time.Now().Add(5 * time.Minute).UTC().Truncate(time.Second)
cacheRecorder := &schedulerCacheRecorder{}
s.repo.schedulerCache = cacheRecorder
s.Require().NoError(s.repo.SetRateLimitedIfLater(s.ctx, account.ID, later))
s.Require().NoError(s.repo.SetRateLimitedIfLater(s.ctx, account.ID, earlier))
got, err := s.repo.GetByID(s.ctx, account.ID)
s.Require().NoError(err)
s.Require().NotNil(got.RateLimitResetAt)
s.Require().WithinDuration(later, *got.RateLimitResetAt, time.Second)
s.Require().Len(cacheRecorder.setAccounts, 2)
s.Require().NotNil(cacheRecorder.setAccounts[1].RateLimitResetAt)
s.Require().WithinDuration(later, *cacheRecorder.setAccounts[1].RateLimitResetAt, time.Second)
}
func (s *AccountRepoSuite) TestClearRateLimit() {
account := mustCreateAccount(s.T(), s.client, &service.Account{Name: "acc-clear"})
until := time.Now().Add(1 * time.Hour)
@@ -190,6 +190,7 @@ func (r *apiKeyRepository) GetByKeyForAuth(ctx context.Context, key string) (*se
group.FieldVideoPrice480p,
group.FieldVideoPrice720p,
group.FieldVideoPrice1080p,
group.FieldWebSearchPricePerCall,
group.FieldClaudeCodeOnly,
group.FieldFallbackGroupID,
group.FieldFallbackGroupIDOnInvalidRequest,
@@ -943,6 +944,7 @@ func groupEntityToService(g *dbent.Group) *service.Group {
VideoPrice480P: g.VideoPrice480p,
VideoPrice720P: g.VideoPrice720p,
VideoPrice1080P: g.VideoPrice1080p,
WebSearchPricePerCall: g.WebSearchPricePerCall,
DefaultValidityDays: g.DefaultValidityDays,
ClaudeCodeOnly: g.ClaudeCodeOnly,
FallbackGroupID: g.FallbackGroupID,
@@ -63,6 +63,7 @@ func (r *groupRepository) Create(ctx context.Context, groupIn *service.Group) er
SetNillableVideoPrice480p(groupIn.VideoPrice480P).
SetNillableVideoPrice720p(groupIn.VideoPrice720P).
SetNillableVideoPrice1080p(groupIn.VideoPrice1080P).
SetNillableWebSearchPricePerCall(groupIn.WebSearchPricePerCall).
SetDefaultValidityDays(groupIn.DefaultValidityDays).
SetClaudeCodeOnly(groupIn.ClaudeCodeOnly).
SetNillableFallbackGroupID(groupIn.FallbackGroupID).
@@ -215,6 +216,11 @@ func (r *groupRepository) Update(ctx context.Context, groupIn *service.Group) er
} else {
builder = builder.ClearVideoPrice1080p()
}
if groupIn.WebSearchPricePerCall != nil {
builder = builder.SetWebSearchPricePerCall(*groupIn.WebSearchPricePerCall)
} else {
builder = builder.ClearWebSearchPricePerCall()
}
// 处理 FallbackGroupID:nil 时清除,否则设置
if groupIn.FallbackGroupID != nil {
@@ -13,6 +13,7 @@ import (
"net"
"net/http"
"net/url"
"os"
"strings"
"sync"
"sync/atomic"
@@ -27,6 +28,7 @@ import (
"github.com/Wei-Shaw/sub2api/internal/pkg/tlsfingerprint"
"github.com/Wei-Shaw/sub2api/internal/service"
"github.com/Wei-Shaw/sub2api/internal/util/urlvalidator"
"golang.org/x/mod/semver"
)
// 默认配置常量
@@ -57,6 +59,13 @@ const (
defaultOpenAIHTTP2FallbackErrorThreshold = 2
defaultOpenAIHTTP2FallbackWindow = 60 * time.Second
defaultOpenAIHTTP2FallbackTTL = 10 * time.Minute
// The Grok CLI proxy rejects requests that do not identify a supported
// client version. Keep a known-good stable version in the binary while
// allowing operators to bump it without waiting for a Sub2API release.
grokCLIProxyHost = "cli-chat-proxy.grok.com"
grokCLIStableVersion = "0.2.93"
grokCLIVersionOverride = "XAI_GROK_CLI_VERSION"
)
const (
@@ -161,6 +170,7 @@ func NewHTTPUpstream(cfg *config.Config) service.HTTPUpstream {
// - 调用方必须关闭 resp.Body,否则会导致 inFlight 计数泄漏
// - inFlight > 0 的客户端不会被淘汰,确保活跃请求不被中断
func (s *httpUpstreamService) Do(req *http.Request, proxyURL string, accountID int64, accountConcurrency int) (*http.Response, error) {
applyGrokCLIProxyHeaders(req)
if err := s.validateRequestHost(req); err != nil {
return nil, err
}
@@ -207,6 +217,7 @@ func (s *httpUpstreamService) DoWithTLS(req *http.Request, proxyURL string, acco
if profile == nil {
return s.Do(req, proxyURL, accountID, accountConcurrency)
}
applyGrokCLIProxyHeaders(req)
upstreamProfile := service.HTTPUpstreamProfileDefault
if req != nil {
upstreamProfile = service.HTTPUpstreamProfileFromContext(req.Context())
@@ -250,6 +261,34 @@ func (s *httpUpstreamService) DoWithTLS(req *http.Request, proxyURL string, acco
return resp, nil
}
// applyGrokCLIProxyHeaders applies the official Grok Build client identity at
// the final shared transport boundary. Keying this behavior to the exact CLI
// proxy host keeps direct api.x.ai traffic unchanged and automatically covers
// Responses, Chat Completions, media, quota probes, and account tests.
func applyGrokCLIProxyHeaders(req *http.Request) {
if req == nil || req.URL == nil || !strings.EqualFold(strings.TrimSpace(req.URL.Hostname()), grokCLIProxyHost) {
return
}
if req.Header == nil {
req.Header = make(http.Header)
}
version := strings.TrimSpace(os.Getenv(grokCLIVersionOverride))
if !isSupportedGrokCLIVersion(version) {
version = grokCLIStableVersion
}
req.Header.Set("X-XAI-Token-Auth", "xai-grok-cli")
req.Header.Set("x-grok-client-version", version)
req.Header.Set("User-Agent", "xai-grok-workspace/"+version)
}
func isSupportedGrokCLIVersion(version string) bool {
canonical := "v" + version
minimum := "v" + grokCLIStableVersion
return semver.IsValid(canonical) &&
semver.Canonical(canonical) == canonical &&
semver.Compare(canonical, minimum) >= 0
}
// acquireClientWithTLS 获取或创建带 TLS 指纹的客户端
func (s *httpUpstreamService) acquireClientWithTLS(proxyURL string, accountID int64, accountConcurrency int, profile *tlsfingerprint.Profile, upstreamProfile service.HTTPUpstreamProfile) (*upstreamClientEntry, error) {
return s.getClientEntryWithTLS(proxyURL, accountID, accountConcurrency, profile, upstreamProfile, true, true)
@@ -15,6 +15,151 @@ import (
"github.com/stretchr/testify/suite"
)
func TestHTTPUpstreamDoAppliesGrokCLIIdentityBeforeOAuthRoundTrip(t *testing.T) {
t.Setenv("XAI_GROK_CLI_VERSION", "")
for _, endpoint := range []string{"responses", "chat/completions"} {
t.Run(endpoint, func(t *testing.T) {
upstream := NewHTTPUpstream(nil)
svc, ok := upstream.(*httpUpstreamService)
require.True(t, ok)
const accountID int64 = 4084
isolation := svc.getIsolationMode()
profile := service.HTTPUpstreamProfileDefault
proxyKey := directProxyKey
protocolMode := svc.resolveProtocolMode(profile, proxyKey, nil)
settings := svc.resolvePoolSettings(isolation, 1)
settings = svc.applyProfilePoolSettings(settings, profile)
cacheKey := buildCacheKey(isolation, proxyKey, accountID, protocolMode)
var capturedHeaders http.Header
svc.clients[cacheKey] = &upstreamClientEntry{
client: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
capturedHeaders = req.Header.Clone()
statusCode := http.StatusOK
if req.Header.Get("X-XAI-Token-Auth") != "xai-grok-cli" {
statusCode = http.StatusForbidden
}
return &http.Response{
StatusCode: statusCode,
Header: make(http.Header),
Body: http.NoBody,
Request: req,
}, nil
})},
proxyKey: proxyKey,
poolKey: buildPoolKey(settings, protocolMode),
protocolMode: protocolMode,
}
req, err := http.NewRequest(http.MethodPost, "https://cli-chat-proxy.grok.com/v1/"+endpoint, nil)
require.NoError(t, err)
req.Header.Set("User-Agent", "sub2api-grok/1.0")
resp, err := svc.Do(req, "", accountID, 1)
require.NoError(t, err)
require.Equal(t, http.StatusOK, resp.StatusCode)
require.NoError(t, resp.Body.Close())
require.Equal(t, "0.2.93", capturedHeaders.Get("x-grok-client-version"))
require.Equal(t, "xai-grok-cli", capturedHeaders.Get("X-XAI-Token-Auth"))
require.Equal(t, "xai-grok-workspace/0.2.93", capturedHeaders.Get("User-Agent"))
})
}
}
func TestApplyGrokCLIProxyHeaders(t *testing.T) {
t.Run("uses pinned stable version for the CLI proxy", func(t *testing.T) {
t.Setenv("XAI_GROK_CLI_VERSION", "")
req, err := http.NewRequest(http.MethodPost, "https://cli-chat-proxy.grok.com/v1/responses", nil)
require.NoError(t, err)
req.Header.Set("User-Agent", "sub2api-grok/1.0")
applyGrokCLIProxyHeaders(req)
require.Equal(t, "0.2.93", req.Header.Get("x-grok-client-version"))
require.Equal(t, "xai-grok-cli", req.Header.Get("X-XAI-Token-Auth"))
require.Equal(t, "xai-grok-workspace/0.2.93", req.Header.Get("User-Agent"))
})
t.Run("accepts a valid operator override", func(t *testing.T) {
t.Setenv("XAI_GROK_CLI_VERSION", "0.2.95-alpha.1")
req, err := http.NewRequest(http.MethodPost, "https://cli-chat-proxy.grok.com/v1/chat/completions", nil)
require.NoError(t, err)
applyGrokCLIProxyHeaders(req)
require.Equal(t, "0.2.95-alpha.1", req.Header.Get("x-grok-client-version"))
require.Equal(t, "xai-grok-workspace/0.2.95-alpha.1", req.Header.Get("User-Agent"))
})
t.Run("rejects an unsafe override", func(t *testing.T) {
t.Setenv("XAI_GROK_CLI_VERSION", "0.2.95\r\nX-Injected: true")
req, err := http.NewRequest(http.MethodPost, "https://cli-chat-proxy.grok.com/v1/responses", nil)
require.NoError(t, err)
applyGrokCLIProxyHeaders(req)
require.Equal(t, "0.2.93", req.Header.Get("x-grok-client-version"))
require.Empty(t, req.Header.Get("X-Injected"))
})
t.Run("rejects an override below the supported minimum", func(t *testing.T) {
t.Setenv("XAI_GROK_CLI_VERSION", "0.2.92")
req, err := http.NewRequest(http.MethodPost, "https://cli-chat-proxy.grok.com/v1/responses", nil)
require.NoError(t, err)
applyGrokCLIProxyHeaders(req)
require.Equal(t, "0.2.93", req.Header.Get("x-grok-client-version"))
require.Equal(t, "xai-grok-workspace/0.2.93", req.Header.Get("User-Agent"))
})
t.Run("rejects a prerelease override at the minimum version", func(t *testing.T) {
t.Setenv("XAI_GROK_CLI_VERSION", "0.2.93-beta.1")
req, err := http.NewRequest(http.MethodPost, "https://cli-chat-proxy.grok.com/v1/responses", nil)
require.NoError(t, err)
applyGrokCLIProxyHeaders(req)
require.Equal(t, "0.2.93", req.Header.Get("x-grok-client-version"))
require.Equal(t, "xai-grok-workspace/0.2.93", req.Header.Get("User-Agent"))
})
for _, version := range []string{
"0.2.093",
"0.2.94-alpha..1",
"0.3",
"1",
"0.2.95+build.1",
} {
t.Run("rejects invalid semver "+version, func(t *testing.T) {
t.Setenv("XAI_GROK_CLI_VERSION", version)
req, err := http.NewRequest(http.MethodPost, "https://cli-chat-proxy.grok.com/v1/responses", nil)
require.NoError(t, err)
applyGrokCLIProxyHeaders(req)
require.Equal(t, "0.2.93", req.Header.Get("x-grok-client-version"))
require.Equal(t, "xai-grok-workspace/0.2.93", req.Header.Get("User-Agent"))
})
}
t.Run("leaves direct xAI API requests unchanged", func(t *testing.T) {
t.Setenv("XAI_GROK_CLI_VERSION", "0.2.95")
req, err := http.NewRequest(http.MethodPost, "https://api.x.ai/v1/responses", nil)
require.NoError(t, err)
req.Header.Set("User-Agent", "sub2api-grok/1.0")
applyGrokCLIProxyHeaders(req)
require.Empty(t, req.Header.Get("x-grok-client-version"))
require.Empty(t, req.Header.Get("X-XAI-Token-Auth"))
require.Equal(t, "sub2api-grok/1.0", req.Header.Get("User-Agent"))
})
}
// HTTPUpstreamSuite HTTP 上游服务测试套件
// 使用 testify/suite 组织测试,支持 SetupTest 初始化
type HTTPUpstreamSuite struct {
@@ -366,6 +366,7 @@ func TestAPIContracts(t *testing.T) {
"video_price_480p": null,
"video_price_720p": null,
"video_price_1080p": null,
"web_search_price_per_call": null,
"allow_image_generation": false,
"allow_batch_image_generation": false,
"batch_image_discount_multiplier": 0,
@@ -147,6 +147,7 @@ func RegisterGatewayRoutes(
}
h.Gateway.Responses(c)
})
gateway.POST("/alpha/search", h.OpenAIGateway.AlphaSearch)
gateway.GET("/responses", func(c *gin.Context) {
h.OpenAIGateway.ResponsesWebSocket(c)
})
@@ -212,6 +213,7 @@ func RegisterGatewayRoutes(
}
r.POST("/responses", bodyLimit, clientRequestID, opsErrorLogger, endpointNorm, gin.HandlerFunc(apiKeyAuth), requireGroupAnthropic, responsesHandler)
r.POST("/responses/*subpath", bodyLimit, clientRequestID, opsErrorLogger, endpointNorm, gin.HandlerFunc(apiKeyAuth), requireGroupAnthropic, responsesHandler)
r.POST("/alpha/search", bodyLimit, clientRequestID, opsErrorLogger, endpointNorm, gin.HandlerFunc(apiKeyAuth), requireGroupAnthropic, h.OpenAIGateway.AlphaSearch)
r.GET("/responses", bodyLimit, clientRequestID, opsErrorLogger, endpointNorm, gin.HandlerFunc(apiKeyAuth), requireGroupAnthropic, func(c *gin.Context) {
h.OpenAIGateway.ResponsesWebSocket(c)
})
@@ -220,6 +222,7 @@ func RegisterGatewayRoutes(
{
codexDirect.POST("/responses", responsesHandler)
codexDirect.POST("/responses/*subpath", responsesHandler)
codexDirect.POST("/alpha/search", h.OpenAIGateway.AlphaSearch)
codexDirect.GET("/responses", func(c *gin.Context) {
h.OpenAIGateway.ResponsesWebSocket(c)
})
@@ -65,6 +65,36 @@ func TestGatewayRoutesOpenAIResponsesCompactPathIsRegistered(t *testing.T) {
}
}
func TestGatewayRoutesOpenAIAlphaSearchPathsAreRegistered(t *testing.T) {
router := newGatewayRoutesTestRouter()
registered := make(map[string]bool)
for _, route := range router.Routes() {
if route.Method == http.MethodPost {
registered[route.Path] = true
}
}
for _, path := range []string{
"/v1/alpha/search",
"/alpha/search",
"/backend-api/codex/alpha/search",
} {
require.True(t, registered[path], "POST %s should be registered", path)
}
}
func TestGatewayRoutesAlphaSearchRejectsNonOpenAIGroup(t *testing.T) {
router := newGatewayRoutesTestRouter(service.PlatformGrok)
req := httptest.NewRequest(http.MethodPost, "/v1/alpha/search", strings.NewReader(`{"model":"gpt-5.6-sol"}`))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
require.Equal(t, http.StatusNotFound, w.Code)
require.Contains(t, w.Body.String(), "only available for OpenAI groups")
}
func TestGatewayRoutesOpenAIImagesPathsAreRegistered(t *testing.T) {
router := newGatewayRoutesTestRouter()
+28
View File
@@ -6,6 +6,7 @@ import (
"errors"
"hash/fnv"
"log/slog"
"net/url"
"reflect"
"sort"
"strconv"
@@ -1255,12 +1256,39 @@ func (a *Account) GetGrokBaseURL() string {
return ""
}
baseURL := a.GetCredential("base_url")
if a.IsGrokOAuth() {
if strings.TrimSpace(baseURL) == "" || isOfficialGrokAPIBaseURL(baseURL) {
return xai.DefaultCLIBaseURL
}
}
if baseURL != "" {
return baseURL
}
return xai.DefaultBaseURL
}
func isOfficialGrokAPIBaseURL(raw string) bool {
parsed, err := url.Parse(strings.TrimSpace(raw))
if err != nil || parsed == nil || parsed.Opaque != "" || parsed.User != nil || parsed.RawQuery != "" || parsed.Fragment != "" {
return false
}
defaultURL, err := url.Parse(xai.DefaultBaseURL)
if err != nil {
return false
}
if !strings.EqualFold(parsed.Scheme, defaultURL.Scheme) || !strings.EqualFold(parsed.Hostname(), defaultURL.Hostname()) {
return false
}
if port := parsed.Port(); port != "" {
portNumber, err := strconv.Atoi(port)
if err != nil || portNumber != 443 {
return false
}
}
path := strings.TrimRight(parsed.Path, "/")
return path == "" || path == strings.TrimRight(defaultURL.Path, "/")
}
func (a *Account) GetGrokAccessToken() string {
if !a.IsGrok() {
return ""
@@ -4,6 +4,9 @@ package service
import (
"testing"
"github.com/Wei-Shaw/sub2api/internal/pkg/xai"
"github.com/stretchr/testify/require"
)
func TestGetBaseURL(t *testing.T) {
@@ -158,3 +161,135 @@ func TestGetGeminiBaseURL(t *testing.T) {
})
}
}
func TestGetGrokBaseURLUsesSubscriptionProxyForOAuth(t *testing.T) {
tests := []struct {
name string
account Account
expected string
}{
{
name: "oauth without base_url uses CLI subscription proxy",
account: Account{
Type: AccountTypeOAuth,
Platform: PlatformGrok,
Credentials: map[string]any{},
},
expected: xai.DefaultCLIBaseURL,
},
{
name: "oauth legacy API default is migrated at runtime to CLI subscription proxy",
account: Account{
Type: AccountTypeOAuth,
Platform: PlatformGrok,
Credentials: map[string]any{
"base_url": xai.DefaultBaseURL,
},
},
expected: xai.DefaultCLIBaseURL,
},
{
name: "oauth legacy API default with trailing slash is migrated at runtime",
account: Account{
Type: AccountTypeOAuth,
Platform: PlatformGrok,
Credentials: map[string]any{
"base_url": xai.DefaultBaseURL + "/",
},
},
expected: xai.DefaultCLIBaseURL,
},
{
name: "oauth legacy API root is migrated at runtime",
account: Account{
Type: AccountTypeOAuth,
Platform: PlatformGrok,
Credentials: map[string]any{
"base_url": "https://api.x.ai",
},
},
expected: xai.DefaultCLIBaseURL,
},
{
name: "oauth legacy API root with canonical HTTPS port is migrated at runtime",
account: Account{
Type: AccountTypeOAuth,
Platform: PlatformGrok,
Credentials: map[string]any{
"base_url": "HTTPS://API.X.AI:443/",
},
},
expected: xai.DefaultCLIBaseURL,
},
{
name: "oauth legacy API canonical port with leading zeroes is migrated at runtime",
account: Account{
Type: AccountTypeOAuth,
Platform: PlatformGrok,
Credentials: map[string]any{
"base_url": "https://api.x.ai:0443/v1",
},
},
expected: xai.DefaultCLIBaseURL,
},
{
name: "oauth legacy API encoded version path is migrated at runtime",
account: Account{
Type: AccountTypeOAuth,
Platform: PlatformGrok,
Credentials: map[string]any{
"base_url": "https://api.x.ai/%76%31",
},
},
expected: xai.DefaultCLIBaseURL,
},
{
name: "oauth legacy API encoded trailing slash is migrated at runtime",
account: Account{
Type: AccountTypeOAuth,
Platform: PlatformGrok,
Credentials: map[string]any{
"base_url": "https://api.x.ai/v1%2F",
},
},
expected: xai.DefaultCLIBaseURL,
},
{
name: "oauth non-default API port remains an explicit override",
account: Account{
Type: AccountTypeOAuth,
Platform: PlatformGrok,
Credentials: map[string]any{
"base_url": "https://api.x.ai:8443/v1",
},
},
expected: "https://api.x.ai:8443/v1",
},
{
name: "oauth explicit custom base_url remains supported",
account: Account{
Type: AccountTypeOAuth,
Platform: PlatformGrok,
Credentials: map[string]any{
"base_url": "https://custom.example.com/v1",
},
},
expected: "https://custom.example.com/v1",
},
{
name: "API key without base_url uses official credit-backed API",
account: Account{
Type: AccountTypeAPIKey,
Platform: PlatformGrok,
Credentials: map[string]any{},
},
expected: xai.DefaultBaseURL,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
require.Equal(t, tt.expected, tt.account.GetGrokBaseURL())
})
}
}
@@ -654,16 +654,10 @@ func (s *AccountTestService) testOpenAIAccountConnection(c *gin.Context, account
return s.processOpenAIStream(c, resp.Body)
}
// testGrokAccountConnection tests a Grok OAuth account through xAI's Responses API.
// testGrokAccountConnection tests a Grok OAuth or API-key account through xAI's Responses API.
func (s *AccountTestService) testGrokAccountConnection(c *gin.Context, account *Account, modelID string) error {
ctx := c.Request.Context()
if account.Type != AccountTypeOAuth {
return s.sendErrorAndEnd(c, fmt.Sprintf("Unsupported Grok account type: %s", account.Type))
}
if s.grokTokenProvider == nil {
return s.sendErrorAndEnd(c, "Grok token provider not configured")
}
if s.httpUpstream == nil {
return s.sendErrorAndEnd(c, "HTTP upstream not configured")
}
@@ -676,9 +670,24 @@ func (s *AccountTestService) testGrokAccountConnection(c *gin.Context, account *
testModelID = mapped
}
authToken, err := s.grokTokenProvider.GetAccessToken(ctx, account)
if err != nil {
return s.sendErrorAndEnd(c, fmt.Sprintf("Failed to get Grok access token: %s", err.Error()))
var authToken string
switch account.Type {
case AccountTypeOAuth:
if s.grokTokenProvider == nil {
return s.sendErrorAndEnd(c, "Grok token provider not configured")
}
var err error
authToken, err = s.grokTokenProvider.GetAccessToken(ctx, account)
if err != nil {
return s.sendErrorAndEnd(c, fmt.Sprintf("Failed to get Grok access token: %s", err.Error()))
}
case AccountTypeAPIKey:
authToken = strings.TrimSpace(account.GetCredential("api_key"))
if authToken == "" {
return s.sendErrorAndEnd(c, "Grok API key is missing")
}
default:
return s.sendErrorAndEnd(c, fmt.Sprintf("Unsupported Grok account type: %s", account.Type))
}
apiURL, err := xai.BuildResponsesURL(account.GetGrokBaseURL())
@@ -710,7 +719,7 @@ func (s *AccountTestService) testGrokAccountConnection(c *gin.Context, account *
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json, text/event-stream")
req.Header.Set("Authorization", "Bearer "+authToken)
req.Header.Set("User-Agent", "sub2api-grok/1.0")
applyGrokCLIHeaders(req.Header)
proxyURL := ""
if account.ProxyID != nil && account.Proxy != nil {
@@ -723,10 +732,19 @@ func (s *AccountTestService) testGrokAccountConnection(c *gin.Context, account *
}
defer func() { _ = resp.Body.Close() }()
if snapshot := xai.ParseQuotaHeaders(resp.Header, resp.StatusCode); snapshot != nil && s.accountRepo != nil {
now := time.Now()
snapshot := parseGrokQuotaSnapshot(resp.Header, resp.StatusCode, now)
if snapshot != nil && s.accountRepo != nil {
resetAt, limited := grokRateLimitResetAt(snapshot, now)
if limited {
normalizeGrokExhaustedWindowResets(snapshot, resetAt, now)
}
_ = s.accountRepo.UpdateExtra(ctx, account.ID, map[string]any{
grokQuotaSnapshotExtraKey: snapshot,
})
if limited {
persistGrokRateLimit(ctx, s.accountRepo, account, resetAt)
}
}
if resp.StatusCode != http.StatusOK {
@@ -3,6 +3,7 @@
package service
import (
"context"
"io"
"net/http"
"net/http/httptest"
@@ -15,6 +16,18 @@ import (
"github.com/tidwall/gjson"
)
type grokAccountTestRateLimitRepo struct {
*mockAccountRepoForGemini
rateLimitedCalls int
resetAt time.Time
}
func (r *grokAccountTestRateLimitRepo) SetRateLimited(_ context.Context, _ int64, resetAt time.Time) error {
r.rateLimitedCalls++
r.resetAt = resetAt
return nil
}
func TestAccountTestService_TestAccountConnection_GrokUsesXAIResponses(t *testing.T) {
gin.SetMode(gin.TestMode)
@@ -58,10 +71,81 @@ func TestAccountTestService_TestAccountConnection_GrokUsesXAIResponses(t *testin
err := svc.TestAccountConnection(c, account.ID, "grok", "", AccountTestModeDefault)
require.NoError(t, err)
require.Equal(t, "https://api.x.ai/v1/responses", upstream.lastReq.URL.String())
require.Equal(t, "https://cli-chat-proxy.grok.com/v1/responses", upstream.lastReq.URL.String())
require.Equal(t, "Bearer grok-access-token", upstream.lastReq.Header.Get("Authorization"))
require.Equal(t, grokCLIVersion, upstream.lastReq.Header.Get("X-Grok-Client-Version"))
require.Equal(t, "grok-4.3", gjson.GetBytes(upstream.lastBody, "model").String())
require.NotContains(t, rec.Body.String(), "claude")
require.Contains(t, rec.Body.String(), `"model":"grok-4.3"`)
require.Contains(t, rec.Body.String(), `"type":"test_complete"`)
}
func TestAccountTestService_Grok429PersistsRateLimitReset(t *testing.T) {
gin.SetMode(gin.TestMode)
account := &Account{
ID: 14,
Name: "grok-oauth-limited",
Platform: PlatformGrok,
Type: AccountTypeOAuth,
Status: StatusActive,
Schedulable: true,
Concurrency: 1,
Credentials: map[string]any{
"access_token": "grok-access-token",
"expires_at": time.Now().Add(time.Hour).UTC().Format(time.RFC3339),
},
}
baseRepo := &mockAccountRepoForGemini{accountsByID: map[int64]*Account{account.ID: account}}
repo := &grokAccountTestRateLimitRepo{mockAccountRepoForGemini: baseRepo}
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusTooManyRequests,
Header: http.Header{"Retry-After": []string{"45"}},
Body: io.NopCloser(strings.NewReader(`{"error":{"message":"rate limited"}}`)),
}}
svc := &AccountTestService{
accountRepo: repo,
grokTokenProvider: NewGrokTokenProvider(repo, nil),
httpUpstream: upstream,
}
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/api/v1/admin/accounts/14/test", nil)
err := svc.TestAccountConnection(c, account.ID, "grok", "", AccountTestModeDefault)
require.Error(t, err)
require.Equal(t, 1, repo.rateLimitedCalls)
require.WithinDuration(t, time.Now().Add(45*time.Second), repo.resetAt, time.Second)
}
func TestAccountTestService_Grok429WithoutQuotaHeadersUsesFallback(t *testing.T) {
gin.SetMode(gin.TestMode)
account := &Account{
ID: 15, Name: "grok-oauth-limited-no-headers", Platform: PlatformGrok,
Type: AccountTypeOAuth, Status: StatusActive, Schedulable: true, Concurrency: 1,
Credentials: map[string]any{
"access_token": "grok-access-token",
"expires_at": time.Now().Add(time.Hour).UTC().Format(time.RFC3339),
},
}
baseRepo := &mockAccountRepoForGemini{accountsByID: map[int64]*Account{account.ID: account}}
repo := &grokAccountTestRateLimitRepo{mockAccountRepoForGemini: baseRepo}
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusTooManyRequests,
Body: io.NopCloser(strings.NewReader(`{"error":{"message":"quota exhausted"}}`)),
}}
svc := &AccountTestService{
accountRepo: repo, grokTokenProvider: NewGrokTokenProvider(repo, nil), httpUpstream: upstream,
}
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/api/v1/admin/accounts/15/test", nil)
before := time.Now()
err := svc.TestAccountConnection(c, account.ID, "grok", "", AccountTestModeDefault)
require.Error(t, err)
require.Equal(t, 1, repo.rateLimitedCalls)
require.WithinDuration(t, before.Add(grokRateLimitFallbackCooldown), repo.resetAt, time.Second)
}
+5
View File
@@ -156,6 +156,7 @@ func (s *adminServiceImpl) CreateGroup(ctx context.Context, input *CreateGroupIn
videoPrice480P := normalizePrice(input.VideoPrice480P)
videoPrice720P := normalizePrice(input.VideoPrice720P)
videoPrice1080P := normalizePrice(input.VideoPrice1080P)
webSearchPricePerCall := normalizePrice(input.WebSearchPricePerCall)
imageRateMultiplier := 1.0
if input.ImageRateMultiplier != nil {
if *input.ImageRateMultiplier < 0 {
@@ -287,6 +288,7 @@ func (s *adminServiceImpl) CreateGroup(ctx context.Context, input *CreateGroupIn
VideoPrice480P: videoPrice480P,
VideoPrice720P: videoPrice720P,
VideoPrice1080P: videoPrice1080P,
WebSearchPricePerCall: webSearchPricePerCall,
ClaudeCodeOnly: input.ClaudeCodeOnly,
FallbackGroupID: input.FallbackGroupID,
FallbackGroupIDOnInvalidRequest: fallbackOnInvalidRequest,
@@ -543,6 +545,9 @@ func (s *adminServiceImpl) UpdateGroup(ctx context.Context, id int64, input *Upd
if input.VideoPrice1080P != nil {
group.VideoPrice1080P = normalizePrice(input.VideoPrice1080P)
}
if input.WebSearchPricePerCall != nil {
group.WebSearchPricePerCall = normalizePrice(input.WebSearchPricePerCall)
}
// Claude Code 客户端限制
if input.ClaudeCodeOnly != nil {
+8 -4
View File
@@ -220,8 +220,10 @@ type CreateGroupInput struct {
VideoPrice480P *float64
VideoPrice720P *float64
VideoPrice1080P *float64
ClaudeCodeOnly bool // 仅允许 Claude Code 客户端
FallbackGroupID *int64 // 降级分组 ID
// Codex alpha/search 网页搜索单次价格(USD/次,仅 openai 平台使用);nil/负数按默认价 0.01 处理
WebSearchPricePerCall *float64
ClaudeCodeOnly bool // 仅允许 Claude Code 客户端
FallbackGroupID *int64 // 降级分组 ID
// 无效请求兜底分组 ID(仅 anthropic 平台使用)
FallbackGroupIDOnInvalidRequest *int64
// 模型路由配置(仅 anthropic 平台使用)
@@ -274,8 +276,10 @@ type UpdateGroupInput struct {
VideoPrice480P *float64
VideoPrice720P *float64
VideoPrice1080P *float64
ClaudeCodeOnly *bool // 仅允许 Claude Code 客户端
FallbackGroupID *int64 // 降级分组 ID
// Codex alpha/search 网页搜索单次价格(USD/次);nil 表示不修改,负数表示清除回默认价 0.01
WebSearchPricePerCall *float64
ClaudeCodeOnly *bool // 仅允许 Claude Code 客户端
FallbackGroupID *int64 // 降级分组 ID
// 无效请求兜底分组 ID(仅 anthropic 平台使用)
FallbackGroupIDOnInvalidRequest *int64
// 模型路由配置(仅 anthropic 平台使用)
@@ -78,6 +78,7 @@ type APIKeyAuthGroupSnapshot struct {
VideoPrice480P *float64 `json:"video_price_480p,omitempty"`
VideoPrice720P *float64 `json:"video_price_720p,omitempty"`
VideoPrice1080P *float64 `json:"video_price_1080p,omitempty"`
WebSearchPricePerCall *float64 `json:"web_search_price_per_call,omitempty"`
ClaudeCodeOnly bool `json:"claude_code_only"`
FallbackGroupID *int64 `json:"fallback_group_id,omitempty"`
FallbackGroupIDOnInvalidRequest *int64 `json:"fallback_group_id_on_invalid_request,omitempty"`
@@ -14,7 +14,7 @@ import (
"github.com/dgraph-io/ristretto"
)
const apiKeyAuthSnapshotVersion = 14 // v14: include group video pricing fields
const apiKeyAuthSnapshotVersion = 15 // v15: include group web search per-call pricing
type apiKeyAuthCacheConfig struct {
l1Size int
@@ -270,6 +270,7 @@ func (s *APIKeyService) snapshotFromAPIKey(ctx context.Context, apiKey *APIKey)
VideoPrice480P: apiKey.Group.VideoPrice480P,
VideoPrice720P: apiKey.Group.VideoPrice720P,
VideoPrice1080P: apiKey.Group.VideoPrice1080P,
WebSearchPricePerCall: apiKey.Group.WebSearchPricePerCall,
ClaudeCodeOnly: apiKey.Group.ClaudeCodeOnly,
FallbackGroupID: apiKey.Group.FallbackGroupID,
FallbackGroupIDOnInvalidRequest: apiKey.Group.FallbackGroupIDOnInvalidRequest,
@@ -353,6 +354,7 @@ func (s *APIKeyService) snapshotToAPIKey(key string, snapshot *APIKeyAuthSnapsho
VideoPrice480P: snapshot.Group.VideoPrice480P,
VideoPrice720P: snapshot.Group.VideoPrice720P,
VideoPrice1080P: snapshot.Group.VideoPrice1080P,
WebSearchPricePerCall: snapshot.Group.WebSearchPricePerCall,
ClaudeCodeOnly: snapshot.Group.ClaudeCodeOnly,
FallbackGroupID: snapshot.Group.FallbackGroupID,
FallbackGroupIDOnInvalidRequest: snapshot.Group.FallbackGroupIDOnInvalidRequest,
+41 -4
View File
@@ -563,15 +563,19 @@ func (s *BillingService) initFallbackPricing() {
s.fallbackPrices["grok-4.3"] = &ModelPricing{
InputPricePerToken: 1.25e-6,
OutputPricePerToken: 2.5e-6,
CacheReadPricePerToken: 0,
CacheReadPricePerToken: 0.2e-6,
SupportsCacheBreakdown: false,
LongContextInputThreshold: 1000000,
LongContextInputMultiplier: 1,
}
// xAI Grok Build 0.1 (official docs: $1 input / $2 output per MTok)
// xAI Grok Build 0.1 (official docs: $1 input / $0.20 cached input /
// $2 output per MTok). Composer is available only through Grok Build and
// has no standalone public API rate card, so its aliases use this coding
// model rate instead of silently billing at zero.
s.fallbackPrices["grok-build-0.1"] = &ModelPricing{
InputPricePerToken: 1e-6,
OutputPricePerToken: 2e-6,
CacheReadPricePerToken: 0.2e-6,
SupportsCacheBreakdown: false,
}
}
@@ -745,9 +749,14 @@ func (s *BillingService) getFallbackPricing(model string) *ModelPricing {
switch modelLower {
case "grok", "grok-latest", "grok-4.5", "grok-4.5-latest", "grok-build-latest":
return s.fallbackPrices["grok-4.5"]
case "grok-4.3":
case "grok-4.3",
"grok-4.20-0309-reasoning",
"grok-4.20-0309-non-reasoning",
"grok-4.20-multi-agent-0309",
"grok-4.20-reasoning",
"grok-4.20-non-reasoning":
return s.fallbackPrices["grok-4.3"]
case "grok-build", "grok-build-0.1":
case "grok-build", "grok-build-0.1", "grok-composer", "grok-composer-2.5-fast", "composer-2.5":
return s.fallbackPrices["grok-build-0.1"]
}
@@ -1320,8 +1329,36 @@ const (
defaultGrokImagineVideo15Price480P = 0.08
defaultGrokImagineVideo15Price720P = 0.14
defaultGrokImagineVideo15Price1080P = 0.25
// Codex alpha/search 网页搜索单次默认价:OpenAI 官方 web search 定价 $10/1000 次。
defaultWebSearchPricePerCall = 0.01
)
// CalculateWebSearchCost 计算 Codex alpha/search 网页搜索按次费用。
// callCount: 搜索调用次数(每次请求为 1)
// groupPrice: 分组配置的单次价格(nil 表示使用默认价 0.01;0 表示免费)
// rateMultiplier: 分组费率倍数
func (s *BillingService) CalculateWebSearchCost(callCount int, groupPrice *float64, rateMultiplier float64) *CostBreakdown {
if callCount <= 0 {
return &CostBreakdown{}
}
unitPrice := defaultWebSearchPricePerCall
if groupPrice != nil && *groupPrice >= 0 {
unitPrice = *groupPrice
}
totalCost := unitPrice * float64(callCount)
// 应用倍率(保存时强制 > 0;负数按 0 处理避免按 1x 误扣)
if rateMultiplier < 0 {
rateMultiplier = 0
}
return &CostBreakdown{
TotalCost: totalCost,
ActualCost: totalCost * rateMultiplier,
BillingMode: string(BillingModePerRequest),
}
}
// CalculateImageCost 计算图片生成费用
// model: 请求的模型名称(用于获取 LiteLLM 默认价格)
// imageSize: 图片尺寸 "1K", "2K", "4K"
@@ -1039,6 +1039,58 @@ func TestGetModelPricing_Grok45OfficialFallback(t *testing.T) {
}
}
func TestGetModelPricing_GrokCatalogFallbacks(t *testing.T) {
svc := newTestBillingService()
tests := []struct {
name string
models []string
input float64
cacheRead float64
output float64
}{
{
name: "Grok 4.3 family",
models: []string{
"grok-4.3",
"grok-4.20-0309-reasoning",
"grok-4.20-0309-non-reasoning",
"grok-4.20-multi-agent-0309",
"grok-4.20-reasoning",
"grok-4.20-non-reasoning",
},
input: 1.25e-6,
cacheRead: 0.2e-6,
output: 2.5e-6,
},
{
name: "Grok coding and Composer family",
models: []string{
"grok-build",
"grok-build-0.1",
"grok-composer",
"grok-composer-2.5-fast",
"composer-2.5",
},
input: 1e-6,
cacheRead: 0.2e-6,
output: 2e-6,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
for _, model := range tt.models {
pricing, err := svc.GetModelPricing(model)
require.NoError(t, err, "model %s", model)
require.InDelta(t, tt.input, pricing.InputPricePerToken, 1e-12, "model %s input", model)
require.InDelta(t, tt.cacheRead, pricing.CacheReadPricePerToken, 1e-12, "model %s cached input", model)
require.InDelta(t, tt.output, pricing.OutputPricePerToken, 1e-12, "model %s output", model)
}
})
}
}
func TestCalculateCost_SupportsCacheBreakdown(t *testing.T) {
svc := &BillingService{
cfg: &config.Config{},
+5 -4
View File
@@ -333,7 +333,7 @@ func (s *OpenAIGatewayService) ForwardGrokMedia(
}
upstreamReq.Header.Set("Authorization", "Bearer "+token)
upstreamReq.Header.Set("Accept", "application/json")
upstreamReq.Header.Set("User-Agent", "sub2api-grok/1.0")
applyGrokCLIHeaders(upstreamReq.Header)
if endpoint.RequiresRequestBody() {
contentType = strings.TrimSpace(contentType)
if contentType == "" {
@@ -357,11 +357,10 @@ func (s *OpenAIGatewayService) ForwardGrokMedia(
requestIDHeader := firstNonEmpty(resp.Header.Get("x-request-id"), resp.Header.Get("xai-request-id"))
requestModel := requestInfo.Model
if resp.StatusCode >= 400 {
s.updateGrokUsageSnapshot(ctx, account.ID, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
return s.handleGrokMediaErrorResponse(ctx, resp, c, account, requestIDHeader, requestModel)
}
s.updateGrokUsageSnapshot(ctx, account.ID, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
s.updateGrokUsageSnapshot(ctx, account, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
respBody, err := ReadUpstreamResponseBody(resp.Body, s.cfg, c, openAITooLargeError)
if err != nil {
return nil, err
@@ -564,6 +563,9 @@ func (s *OpenAIGatewayService) handleGrokMediaErrorResponse(
requestedModel string,
) (*OpenAIForwardResult, error) {
body := s.readUpstreamErrorBody(resp)
// Reconcile readiness before configurable passthrough branches can return;
// otherwise a Grok 429 can remain schedulable.
s.handleGrokAccountUpstreamError(ctx, account, resp.StatusCode, resp.Header, body)
upstreamMsg := sanitizeUpstreamErrorMessage(strings.TrimSpace(extractUpstreamErrorMessage(body)))
if upstreamMsg == "" {
upstreamMsg = fmt.Sprintf("xAI upstream returned status %d", resp.StatusCode)
@@ -609,7 +611,6 @@ func (s *OpenAIGatewayService) handleGrokMediaErrorResponse(
return nil, fmt.Errorf("upstream error: %d (not in custom error codes) message=%s", resp.StatusCode, upstreamMsg)
}
s.handleGrokAccountUpstreamError(ctx, account, resp.StatusCode, resp.Header, body)
kind := "http_error"
if s.shouldFailoverUpstreamError(resp.StatusCode) {
kind = "failover"
@@ -235,7 +235,7 @@ func (s *GrokOAuthService) BuildAccountCredentials(tokenInfo *GrokTokenInfo) map
if tokenInfo.EntitlementStatus != "" {
creds["entitlement_status"] = tokenInfo.EntitlementStatus
}
creds["base_url"] = xai.DefaultBaseURL
creds["base_url"] = xai.DefaultCLIBaseURL
return creds
}
@@ -5,6 +5,7 @@ package service
import (
"context"
"testing"
"time"
"github.com/Wei-Shaw/sub2api/internal/pkg/xai"
"github.com/stretchr/testify/require"
@@ -66,3 +67,15 @@ func TestGrokOAuthServiceExchangeCodeRequiresStateForCallbackURLAndConsumesSessi
require.Contains(t, err.Error(), "GROK_OAUTH_SESSION_NOT_FOUND")
require.Zero(t, client.exchangeCalls)
}
func TestGrokOAuthServiceBuildAccountCredentialsDefaultsToSubscriptionProxy(t *testing.T) {
svc := NewGrokOAuthService(nil, &grokOAuthClientStub{})
defer svc.Stop()
credentials := svc.BuildAccountCredentials(&GrokTokenInfo{
AccessToken: "access-token",
ExpiresAt: time.Now().Add(time.Hour).Unix(),
})
require.Equal(t, xai.DefaultCLIBaseURL, credentials["base_url"])
}
@@ -82,7 +82,7 @@ func (s *GrokQuotaService) ProbeUsage(ctx context.Context, accountID int64) (*Gr
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
req.Header.Set("User-Agent", "sub2api-grok-quota-probe/1.0")
applyGrokCLIHeaders(req.Header)
resp, err := s.httpUpstream.Do(req, proxyURL, account.ID, maxInt(account.Concurrency, 1))
if err != nil {
@@ -91,9 +91,16 @@ func (s *GrokQuotaService) ProbeUsage(ctx context.Context, accountID int64) (*Gr
defer func() { _ = resp.Body.Close() }()
snapshot := xai.ObserveQuotaHeaders(resp.Header, resp.StatusCode, "active_probe")
resetAt, limited := grokRateLimitResetAt(snapshot, time.Now())
if limited {
normalizeGrokExhaustedWindowResets(snapshot, resetAt, time.Now())
}
_ = s.accountRepo.UpdateExtra(ctx, account.ID, map[string]any{
grokQuotaSnapshotExtraKey: snapshot,
})
if limited {
persistGrokRateLimit(ctx, s.accountRepo, account, resetAt)
}
result := &GrokQuotaProbeResult{
Source: "active_probe",
@@ -19,6 +19,10 @@ import (
type grokQuotaAccountRepo struct {
*mockAccountRepoForPlatform
updates map[int64]map[string]any
updateCalls int
rateLimitedCalls int
lastRateLimitedID int64
lastRateLimitResetAt time.Time
tempUnschedCalls int
lastTempUnschedID int64
lastTempUnschedUntil time.Time
@@ -26,6 +30,7 @@ type grokQuotaAccountRepo struct {
}
func (r *grokQuotaAccountRepo) UpdateExtra(_ context.Context, id int64, updates map[string]any) error {
r.updateCalls++
if r.updates == nil {
r.updates = make(map[int64]map[string]any)
}
@@ -33,6 +38,17 @@ func (r *grokQuotaAccountRepo) UpdateExtra(_ context.Context, id int64, updates
return nil
}
func (r *grokQuotaAccountRepo) SetRateLimited(_ context.Context, id int64, resetAt time.Time) error {
r.rateLimitedCalls++
r.lastRateLimitedID = id
r.lastRateLimitResetAt = resetAt
return nil
}
func (r *grokQuotaAccountRepo) SetRateLimitedIfLater(ctx context.Context, id int64, resetAt time.Time) error {
return r.SetRateLimited(ctx, id, resetAt)
}
func (r *grokQuotaAccountRepo) SetTempUnschedulable(_ context.Context, id int64, until time.Time, reason string) error {
r.tempUnschedCalls++
r.lastTempUnschedID = id
@@ -96,8 +112,9 @@ func TestGrokQuotaServiceProbeUsageStoresHeaders(t *testing.T) {
require.NotNil(t, result.Snapshot.Requests)
require.EqualValues(t, 10, *result.Snapshot.Requests.Limit)
require.EqualValues(t, 7, *result.Snapshot.Requests.Remaining)
require.Equal(t, "https://api.x.ai/v1/responses", upstream.lastReq.URL.String())
require.Equal(t, "https://cli-chat-proxy.grok.com/v1/responses", upstream.lastReq.URL.String())
require.Equal(t, "Bearer access-token", upstream.lastReq.Header.Get("Authorization"))
require.Equal(t, grokCLIVersion, upstream.lastReq.Header.Get("X-Grok-Client-Version"))
require.Equal(t, "grok-4.3", gjson.GetBytes(upstream.lastBody, "model").String())
require.Contains(t, string(upstream.lastBody), `"max_output_tokens":1`)
require.Contains(t, string(upstream.lastBody), `"store":false`)
@@ -285,6 +302,10 @@ func TestGrokQuotaServiceProbeUsageReturnsRateLimitedSnapshot(t *testing.T) {
require.NotNil(t, result.Snapshot)
require.NotNil(t, result.Snapshot.RetryAfterSeconds)
require.Equal(t, 45, *result.Snapshot.RetryAfterSeconds)
require.Equal(t, 1, repo.rateLimitedCalls)
require.Equal(t, account.ID, repo.lastRateLimitedID)
require.WithinDuration(t, time.Now().Add(45*time.Second), repo.lastRateLimitResetAt, time.Second)
require.Zero(t, repo.tempUnschedCalls)
}
func TestGrokQuotaServiceResetQuotaUnsupported(t *testing.T) {
+3
View File
@@ -50,6 +50,9 @@ type Group struct {
VideoPrice480P *float64
VideoPrice720P *float64
VideoPrice1080P *float64
// Codex alpha/search 网页搜索单次价格(USD/次,仅 openai 平台使用);
// nil 表示使用默认价 defaultWebSearchPricePerCall(官方 $10/1000 次)。
WebSearchPricePerCall *float64
// Claude Code 客户端限制
ClaudeCodeOnly bool
@@ -29,3 +29,10 @@ func videoPriceConfigFromAPIKey(apiKey *APIKey) *VideoPriceConfig {
func apiKeyHasConfiguredVideoPrice(apiKey *APIKey, resolution string) bool {
return apiKey != nil && apiKey.Group != nil && apiKey.Group.GetVideoPrice(resolution) != nil
}
func webSearchPricePerCallFromAPIKey(apiKey *APIKey) *float64 {
if apiKey == nil || apiKey.Group == nil {
return nil
}
return apiKey.Group.WebSearchPricePerCall
}
@@ -0,0 +1,165 @@
package service
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
"github.com/gin-gonic/gin"
"github.com/tidwall/gjson"
)
const (
chatgptCodexAlphaSearchURL = "https://chatgpt.com/backend-api/codex/alpha/search"
openAIPlatformAlphaSearchURL = "https://api.openai.com/v1/alpha/search"
)
// ForwardAlphaSearch proxies Codex standalone web search without binding the
// evolving alpha request or response schema.
//
// 返回值约定:仅当上游返回 2xx(一次真实成功的搜索)时返回非 nil 的
// *OpenAIForwardResult(WebSearchCalls=1,供按次计费);上游错误被原样透传
// 给客户端时返回 (nil, nil),不产生计费。
func (s *OpenAIGatewayService) ForwardAlphaSearch(ctx context.Context, c *gin.Context, account *Account, body []byte) (*OpenAIForwardResult, error) {
if s == nil || c == nil || account == nil {
return nil, fmt.Errorf("service, context, and account are required")
}
modelResult := gjson.GetBytes(body, "model")
requestedModel := strings.TrimSpace(modelResult.String())
if modelResult.Type != gjson.String || requestedModel == "" {
return nil, fmt.Errorf("model is required")
}
upstreamModel := normalizeOpenAIModelForUpstream(account, account.GetMappedModel(requestedModel))
if upstreamModel != "" && upstreamModel != requestedModel {
body = ReplaceModelInBody(body, upstreamModel)
}
token, _, err := s.GetAccessToken(ctx, account)
if err != nil {
return nil, err
}
req, err := s.buildOpenAIAlphaSearchRequest(ctx, c, account, body, token)
if err != nil {
return nil, err
}
proxyURL := ""
if account.ProxyID != nil && account.Proxy != nil {
proxyURL = account.Proxy.URL()
}
upstreamStart := time.Now()
resp, err := s.httpUpstream.Do(req, proxyURL, account.ID, account.Concurrency)
SetOpsLatencyMs(c, OpsUpstreamLatencyMsKey, time.Since(upstreamStart).Milliseconds())
if err != nil {
return nil, s.handleOpenAIUpstreamTransportError(ctx, c, account, err, true)
}
defer func() { _ = resp.Body.Close() }()
respBody, err := ReadUpstreamResponseBody(resp.Body, s.cfg, c, openAITooLargeError)
if err != nil {
return nil, fmt.Errorf("read alpha search response: %w", err)
}
if resp.StatusCode >= http.StatusBadRequest {
upstreamMessage := sanitizeUpstreamErrorMessage(strings.TrimSpace(extractUpstreamErrorMessage(respBody)))
if s.shouldFailoverOpenAIUpstreamResponse(resp.StatusCode, upstreamMessage, respBody) {
resp.Body = io.NopCloser(bytes.NewReader(respBody))
s.handleFailoverSideEffects(ctx, resp, account, respBody, upstreamModel)
return nil, &UpstreamFailoverError{
StatusCode: resp.StatusCode,
ResponseBody: respBody,
RetryableOnSameAccount: account.IsPoolMode() && account.IsPoolModeRetryableStatus(resp.StatusCode),
}
}
}
if !account.IsShadow() {
s.UpdateCodexUsageSnapshotFromHeaders(ctx, account.ID, resp.Header)
}
writeOpenAIPassthroughResponseHeaders(c.Writer.Header(), resp.Header, s.responseHeaderFilter)
contentType := resp.Header.Get("Content-Type")
if contentType == "" {
contentType = "application/json"
}
c.Data(resp.StatusCode, contentType, respBody)
if resp.StatusCode < http.StatusOK || resp.StatusCode >= http.StatusMultipleChoices {
// 非 2xx(错误/重定向)已原样透传给客户端:不是一次成功的搜索,不计费。
return nil, nil
}
return &OpenAIForwardResult{
RequestID: strings.TrimSpace(resp.Header.Get("x-request-id")),
Model: requestedModel,
UpstreamModel: upstreamModel,
Duration: time.Since(upstreamStart),
WebSearchCalls: 1,
}, nil
}
func (s *OpenAIGatewayService) buildOpenAIAlphaSearchRequest(ctx context.Context, c *gin.Context, account *Account, body []byte, token string) (*http.Request, error) {
clientBeta := ""
if c != nil {
clientBeta = c.GetHeader("OpenAI-Beta")
}
req, err := s.buildUpstreamRequestOpenAIPassthrough(ctx, c, account, body, token)
if err != nil {
return nil, err
}
targetURL, err := s.openAIAlphaSearchURL(account)
if err != nil {
return nil, err
}
parsedURL, err := url.Parse(targetURL)
if err != nil {
return nil, fmt.Errorf("parse alpha search URL: %w", err)
}
if c != nil && c.Request != nil && c.Request.URL != nil {
query := parsedURL.Query()
for key, values := range c.Request.URL.Query() {
for _, value := range values {
query.Add(key, value)
}
}
parsedURL.RawQuery = query.Encode()
}
req.URL = parsedURL
req.Header.Set("Accept", "application/json")
if clientBeta == "" {
req.Header.Del("OpenAI-Beta")
}
if version := strings.TrimSpace(c.GetHeader("Version")); version != "" {
req.Header.Set("Version", version)
} else if account.Type == AccountTypeOAuth {
req.Header.Set("Version", codexCLIVersion)
}
return req, nil
}
func (s *OpenAIGatewayService) openAIAlphaSearchURL(account *Account) (string, error) {
if account == nil {
return "", fmt.Errorf("account is required")
}
switch account.Type {
case AccountTypeOAuth:
return chatgptCodexAlphaSearchURL, nil
case AccountTypeAPIKey:
baseURL := account.GetOpenAIBaseURL()
if baseURL == "" {
return openAIPlatformAlphaSearchURL, nil
}
validatedURL, err := s.validateUpstreamBaseURL(baseURL)
if err != nil {
return "", err
}
return buildOpenAIEndpointURL(validatedURL, "/v1/alpha/search"), nil
default:
return "", fmt.Errorf("unsupported OpenAI account type: %s", account.Type)
}
}
@@ -0,0 +1,101 @@
//go:build unit
package service
import (
"context"
"testing"
"github.com/Wei-Shaw/sub2api/internal/config"
"github.com/stretchr/testify/require"
)
func TestCalculateWebSearchCostDefaultAndOverride(t *testing.T) {
t.Parallel()
s := &BillingService{}
// 默认价:官方 $10/1000 次 = 0.01/次
cost := s.CalculateWebSearchCost(1, nil, 1.0)
require.InDelta(t, 0.01, cost.TotalCost, 1e-12)
require.InDelta(t, 0.01, cost.ActualCost, 1e-12)
require.Equal(t, string(BillingModePerRequest), cost.BillingMode)
// 分组覆盖价 + 倍率
cost = s.CalculateWebSearchCost(1, float64Ptr(0.02), 2.5)
require.InDelta(t, 0.02, cost.TotalCost, 1e-12)
require.InDelta(t, 0.05, cost.ActualCost, 1e-12)
// 0 = 免费(区别于 nil = 默认价)
cost = s.CalculateWebSearchCost(1, float64Ptr(0), 3.0)
require.Zero(t, cost.TotalCost)
require.Zero(t, cost.ActualCost)
// 负数倍率按 0 处理,避免按 1x 误扣
cost = s.CalculateWebSearchCost(1, nil, -1)
require.InDelta(t, 0.01, cost.TotalCost, 1e-12)
require.Zero(t, cost.ActualCost)
// 次数 <= 0 不产生费用
cost = s.CalculateWebSearchCost(0, float64Ptr(0.02), 1.0)
require.Zero(t, cost.TotalCost)
require.Empty(t, cost.BillingMode)
}
func TestCalculateOpenAIRecordUsageCostWebSearchPerCall(t *testing.T) {
t.Parallel()
svc := &OpenAIGatewayService{billingService: &BillingService{}}
groupID := int64(11)
// 分组未配置单价:默认 0.01。按次搜索使用不含高峰因子的基础倍率(第 4 个倍率参数 2.0),
// 即使 token 倍率(含高峰,3.0)更高也不采用。
apiKey := &APIKey{ID: 1, GroupID: &groupID, Group: &Group{ID: groupID, Platform: PlatformOpenAI}}
result := &OpenAIForwardResult{Model: "gpt-5.6-sol", UpstreamModel: "gpt-5.6-sol", WebSearchCalls: 1}
cost, err := svc.calculateOpenAIRecordUsageCost(context.Background(), result, apiKey, []string{"gpt-5.6-sol"}, 3.0, 1.0, 1.0, 2.0, UsageTokens{}, "")
require.NoError(t, err)
require.Equal(t, string(BillingModePerRequest), cost.BillingMode)
require.InDelta(t, 0.01, cost.TotalCost, 1e-12)
require.InDelta(t, 0.02, cost.ActualCost, 1e-12)
// 分组配置单价 0.005
apiKey.Group.WebSearchPricePerCall = float64Ptr(0.005)
cost, err = svc.calculateOpenAIRecordUsageCost(context.Background(), result, apiKey, []string{"gpt-5.6-sol"}, 1.0, 1.0, 1.0, 1.0, UsageTokens{}, "")
require.NoError(t, err)
require.InDelta(t, 0.005, cost.TotalCost, 1e-12)
require.InDelta(t, 0.005, cost.ActualCost, 1e-12)
// WebSearchCalls = 0 时不得走按次分支(无定价数据会返回 pricing 错误,
// 证明回落到了 token 路径而不是被按次分支吞掉)。
result.WebSearchCalls = 0
_, err = svc.calculateOpenAIRecordUsageCost(context.Background(), result, apiKey, []string{"gpt-5.6-sol"}, 1.0, 1.0, 1.0, 1.0, UsageTokens{InputTokens: 10}, "")
require.Error(t, err)
}
func TestAPIKeyService_SnapshotRoundTrip_PreservesWebSearchPricePerCall(t *testing.T) {
svc := NewAPIKeyService(nil, nil, nil, nil, nil, nil, &config.Config{})
groupID := int64(9)
apiKey := &APIKey{
ID: 1,
UserID: 2,
GroupID: &groupID,
Key: "k-websearch",
Status: StatusActive,
User: &User{ID: 2, Status: StatusActive, Role: RoleUser},
Group: &Group{
ID: groupID,
Name: "openai",
Platform: PlatformOpenAI,
Status: StatusActive,
SubscriptionType: SubscriptionTypeStandard,
RateMultiplier: 1,
WebSearchPricePerCall: float64Ptr(0.008),
},
}
snapshot := svc.snapshotFromAPIKey(context.Background(), apiKey)
roundTrip := svc.snapshotToAPIKey(apiKey.Key, snapshot)
require.NotNil(t, roundTrip)
require.NotNil(t, roundTrip.Group)
require.NotNil(t, roundTrip.Group.WebSearchPricePerCall)
require.InDelta(t, 0.008, *roundTrip.Group.WebSearchPricePerCall, 1e-12)
}
@@ -0,0 +1,145 @@
package service
import (
"bytes"
"context"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/Wei-Shaw/sub2api/internal/config"
"github.com/gin-gonic/gin"
"github.com/stretchr/testify/require"
"github.com/tidwall/gjson"
)
func TestForwardAlphaSearchOAuthPreservesWire(t *testing.T) {
gin.SetMode(gin.TestMode)
body := []byte(`{
"id":"search-session",
"model":"gpt-5.6-sol",
"reasoning":{"effort":"max","context":"all_turns"},
"input":[{"type":"message","role":"user","content":[{"type":"input_text","text":"latest news"}]}],
"commands":{"search_query":[{"q":"OpenAI news","recency":1}]},
"settings":{"allowed_callers":["direct"],"external_web_access":true},
"max_output_tokens":2000,
"future_field":{"keep":true}
}`)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/alpha/search?feature=standalone", bytes.NewReader(body))
c.Request.Header.Set("Content-Type", "application/json")
c.Request.Header.Set("User-Agent", codexCLIUserAgent)
c.Request.Header.Set("Originator", "codex_cli_rs")
c.Request.Header.Set("Version", "0.144.1")
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(strings.NewReader(`{"encrypted_output":"ciphertext","output":"search result"}`)),
}}
service := &OpenAIGatewayService{cfg: &config.Config{}, httpUpstream: upstream}
account := &Account{
ID: 42,
Platform: PlatformOpenAI,
Type: AccountTypeOAuth,
Concurrency: 1,
Credentials: map[string]any{
"access_token": "oauth-token",
"chatgpt_account_id": "chatgpt-account",
},
}
result, err := service.ForwardAlphaSearch(context.Background(), c, account, body)
require.NoError(t, err)
require.NotNil(t, result)
require.Equal(t, 1, result.WebSearchCalls)
require.Equal(t, "gpt-5.6-sol", result.Model)
require.Equal(t, http.StatusOK, recorder.Code)
require.JSONEq(t, `{"encrypted_output":"ciphertext","output":"search result"}`, recorder.Body.String())
require.Equal(t, chatgptCodexAlphaSearchURL+"?feature=standalone", upstream.lastReq.URL.String())
require.Equal(t, "chatgpt.com", upstream.lastReq.Host)
require.Equal(t, "Bearer oauth-token", upstream.lastReq.Header.Get("Authorization"))
require.Equal(t, "chatgpt-account", upstream.lastReq.Header.Get("chatgpt-account-id"))
require.Equal(t, "application/json", upstream.lastReq.Header.Get("Accept"))
require.Equal(t, "0.144.1", upstream.lastReq.Header.Get("Version"))
require.Empty(t, upstream.lastReq.Header.Get("OpenAI-Beta"))
require.JSONEq(t, string(body), string(upstream.lastBody))
}
func TestForwardAlphaSearchAPIKeyMapsModelAndPassesThroughError(t *testing.T) {
gin.SetMode(gin.TestMode)
body := []byte(`{"id":"search-session","model":"gpt-5.6-sol","commands":{"search_query":[{"q":"news"}]}}`)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/alpha/search", bytes.NewReader(body))
c.Request.Header.Set("Content-Type", "application/json")
upstreamBody := `{"error":{"type":"invalid_request_error","message":"bad search"}}`
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusBadRequest,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(strings.NewReader(upstreamBody)),
}}
service := &OpenAIGatewayService{cfg: &config.Config{}, httpUpstream: upstream}
account := &Account{
ID: 7,
Platform: PlatformOpenAI,
Type: AccountTypeAPIKey,
Credentials: map[string]any{
"api_key": "sk-test",
"base_url": "https://compat.example/v4",
"model_mapping": map[string]any{
"gpt-5.6-sol": "upstream-5.6",
},
},
}
result, err := service.ForwardAlphaSearch(context.Background(), c, account, body)
require.NoError(t, err)
// 上游错误透传不是一次成功的搜索:不返回 result、不产生按次计费。
require.Nil(t, result)
require.Equal(t, http.StatusBadRequest, recorder.Code)
require.JSONEq(t, upstreamBody, recorder.Body.String())
require.Equal(t, "https://compat.example/v4/alpha/search", upstream.lastReq.URL.String())
require.Equal(t, "Bearer sk-test", upstream.lastReq.Header.Get("Authorization"))
require.Equal(t, "upstream-5.6", gjson.GetBytes(upstream.lastBody, "model").String())
require.True(t, gjson.GetBytes(upstream.lastBody, "commands.search_query").IsArray())
}
func TestForwardAlphaSearchReturnsFailoverBeforeWriting(t *testing.T) {
gin.SetMode(gin.TestMode)
body := []byte(`{"id":"search-session","model":"gpt-5.6-sol","commands":{}}`)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/alpha/search", bytes.NewReader(body))
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusTooManyRequests,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(strings.NewReader(`{"error":{"message":"rate limited"}}`)),
}}
service := &OpenAIGatewayService{cfg: &config.Config{}, httpUpstream: upstream}
account := &Account{
ID: 8,
Platform: PlatformOpenAI,
Type: AccountTypeAPIKey,
Credentials: map[string]any{
"api_key": "sk-test",
},
}
result, err := service.ForwardAlphaSearch(context.Background(), c, account, body)
require.Nil(t, result)
var failoverErr *UpstreamFailoverError
require.ErrorAs(t, err, &failoverErr)
require.Equal(t, http.StatusTooManyRequests, failoverErr.StatusCode)
require.Equal(t, openAIPlatformAlphaSearchURL, upstream.lastReq.URL.String())
require.False(t, c.Writer.Written())
require.Empty(t, recorder.Body.String())
}
@@ -114,14 +114,15 @@ func TestFilterCodexInput_OutputTypeKeepsItemID(t *testing.T) {
require.Equal(t, "o1", out["id"], "output item id should be preserved")
}
// TestFilterCodexInput_NonToolCallItemKeepsID ensures non-tool-call items
// (e.g. message) still keep their id when PreserveReferences is true.
// TestFilterCodexInput_NonToolCallItemKeepsID ensures items subject to neither
// the fc* (call-input) nor the msg* (message) prefix rule still keep their id
// when PreserveReferences is true.
// message is covered separately in openai_codex_message_item_id_test.go (#3981).
func TestFilterCodexInput_NonToolCallItemKeepsID(t *testing.T) {
input := []any{
map[string]any{
"type": "message",
"id": "item_msg_001",
"role": "user",
"type": "web_search_call",
"id": "ws_001",
},
}
@@ -130,7 +131,7 @@ func TestFilterCodexInput_NonToolCallItemKeepsID(t *testing.T) {
})
require.Len(t, filtered, 1)
msg, ok := filtered[0].(map[string]any)
item, ok := filtered[0].(map[string]any)
require.True(t, ok)
require.Equal(t, "item_msg_001", msg["id"], "non-tool-call items keep their id in preserve mode")
require.Equal(t, "ws_001", item["id"], "unconstrained items keep their id in preserve mode")
}
@@ -11,12 +11,32 @@ import (
// 若请求携带 version 且低于该值,上游直接 404(issue #3901,2026-07 实测)。
const codexUpstreamMinVersion = "0.144.0"
// ensureCodexIdentityHeaders 补齐 OAuth(ChatGPT 内部接口)出站请求所需的 Codex 身份头。
// 已有 User-Agent 与 version 保持不变,交给紧随其后的 enforceCodexIdentityHeaders
// 做官方身份配对与最低版本校正。
func ensureCodexIdentityHeaders(h http.Header) {
if h == nil {
return
}
if strings.TrimSpace(h.Get("user-agent")) == "" {
h.Set("user-agent", codexCLIUserAgent)
}
if strings.TrimSpace(h.Get("originator")) == "" {
h.Set("originator", "codex_cli_rs")
}
if strings.TrimSpace(h.Get("version")) == "" {
h.Set("version", codexCLIVersion)
}
h.Set("OpenAI-Beta", "responses=experimental")
}
// enforceCodexIdentityHeaders 收口 OAuth(ChatGPT 内部接口)出站请求的客户端身份头。
// 上游要求 originator 与 User-Agent 首段配套且为官方客户端标识,version 头(若携带)
// 不低于 0.144.0,任一不满足即 404(issue #3901)。以最终 User-Agent 为准推导配套
// originator;推导不出官方身份(第三方 UA / UA 缺失)时整体回退为默认 Codex CLI 身份。
//
// 仅对携带 originator 的请求生效——compat messages bridge 故意不带 originator,保持原样。
// 仅对携带 originator 的请求生效;需要从缺失身份头恢复的调用方应先调用
// ensureCodexIdentityHeaders。
// 必须在所有 User-Agent 改写(自定义 UA / ForceCodexCLI / 浏览器 UA 兜底)之后调用。
func enforceCodexIdentityHeaders(h http.Header) {
if h == nil || h.Get("originator") == "" {
@@ -7,6 +7,36 @@ import (
"github.com/stretchr/testify/require"
)
func TestEnsureCodexIdentityHeaders(t *testing.T) {
t.Run("补齐缺失身份头", func(t *testing.T) {
h := make(http.Header)
ensureCodexIdentityHeaders(h)
enforceCodexIdentityHeaders(h)
require.Equal(t, "codex_cli_rs", h.Get("originator"))
require.Equal(t, codexCLIUserAgent, h.Get("user-agent"))
require.Equal(t, codexCLIVersion, h.Get("version"))
require.Equal(t, "responses=experimental", h.Get("OpenAI-Beta"))
})
t.Run("保留已有官方UA和合法version并重新配对", func(t *testing.T) {
const tuiUA = "codex-tui/9.9.9 (Mac OS X 14.0; arm64) iTerm (codex-tui; 9.9.9)"
h := make(http.Header)
h.Set("user-agent", tuiUA)
h.Set("version", "9.9.9")
h.Set("OpenAI-Beta", "assistants=v2")
ensureCodexIdentityHeaders(h)
enforceCodexIdentityHeaders(h)
require.Equal(t, "codex-tui", h.Get("originator"))
require.Equal(t, tuiUA, h.Get("user-agent"))
require.Equal(t, "9.9.9", h.Get("version"))
require.Equal(t, "responses=experimental", h.Get("OpenAI-Beta"))
})
}
func TestEnforceCodexIdentityHeaders(t *testing.T) {
const tuiUA = "codex-tui/0.140.2 (Mac OS X 14.0; arm64) iTerm (codex-tui; 0.140.2)"
@@ -102,13 +132,14 @@ func TestEnforceCodexIdentityHeaders(t *testing.T) {
}
}
// compat messages bridge 故意不带 originator:收口必须保持 no-op,不得注入身份头。
// enforce 本身仍只负责收口:缺少 originator 时必须保持 no-op,由需要恢复身份的
// 调用方先显式调用 ensureCodexIdentityHeaders。
func TestEnforceCodexIdentityHeaders_NoOriginatorIsNoop(t *testing.T) {
h := make(http.Header)
h.Set("user-agent", "luna/1.0.0")
h.Set("user-agent", "third-party-client/1.0.0")
enforceCodexIdentityHeaders(h)
require.Empty(t, h.Get("originator"))
require.Equal(t, "luna/1.0.0", h.Get("user-agent"))
require.Equal(t, "third-party-client/1.0.0", h.Get("user-agent"))
}
@@ -0,0 +1,160 @@
//go:build unit
package service
import (
"testing"
"github.com/stretchr/testify/require"
)
// TestFilterCodexInput_StripsMessageItemID_WhenPreservingReferences
// verifies that message items with a non-msg id (e.g. item_*) have their id
// stripped even when PreserveReferences is true. OpenAI upstream requires
// message ids to begin with "msg" and rejects item_* with 400:
// "Expected an ID that begins with 'msg'." (#3981)
func TestFilterCodexInput_StripsMessageItemID_WhenPreservingReferences(t *testing.T) {
input := []any{
map[string]any{
"type": "message",
"id": "item_3bc5a3fa8ccde25f1c0000d4",
"role": "user",
"content": []any{
map[string]any{"type": "input_text", "text": "hello"},
},
},
}
filtered := filterCodexInputWithOptions(input, codexInputFilterOptions{
PreserveReferences: true,
})
require.Len(t, filtered, 1)
msg, ok := filtered[0].(map[string]any)
require.True(t, ok)
require.Equal(t, "message", msg["type"])
_, hasID := msg["id"]
require.False(t, hasID, "item_* id should be stripped from message")
require.Equal(t, "user", msg["role"], "role must be preserved")
require.NotNil(t, msg["content"], "content must be preserved")
}
// TestFilterCodexInput_KeepsMsgID_WhenPreservingReferences
// verifies that message items with a valid msg* id are kept when
// PreserveReferences is true, so context references are not lost.
func TestFilterCodexInput_KeepsMsgID_WhenPreservingReferences(t *testing.T) {
input := []any{
map[string]any{
"type": "message",
"id": "msg_validID123",
"role": "assistant",
},
}
filtered := filterCodexInputWithOptions(input, codexInputFilterOptions{
PreserveReferences: true,
})
require.Len(t, filtered, 1)
msg, ok := filtered[0].(map[string]any)
require.True(t, ok)
require.Equal(t, "msg_validID123", msg["id"], "valid msg* id must be preserved")
}
// TestFilterCodexInput_StripsMessageIDWhenNotPreservingReferences ensures the
// non-continuation path still drops every message id regardless of prefix.
func TestFilterCodexInput_StripsMessageIDWhenNotPreservingReferences(t *testing.T) {
for _, id := range []string{"item_abc", "msg_validID123"} {
input := []any{
map[string]any{
"type": "message",
"id": id,
"role": "user",
},
}
filtered := filterCodexInputWithOptions(input, codexInputFilterOptions{
PreserveReferences: false,
})
require.Len(t, filtered, 1)
msg, ok := filtered[0].(map[string]any)
require.True(t, ok)
_, hasID := msg["id"]
require.False(t, hasID, "id %q should be stripped when not preserving references", id)
}
}
// TestFilterCodexInput_MessageIDStripDoesNotMutateInput ensures the original
// input map is not modified in place when the id is stripped.
func TestFilterCodexInput_MessageIDStripDoesNotMutateInput(t *testing.T) {
original := map[string]any{
"type": "message",
"id": "item_abc",
"role": "user",
}
filtered := filterCodexInputWithOptions([]any{original}, codexInputFilterOptions{
PreserveReferences: true,
})
require.Len(t, filtered, 1)
require.Equal(t, "item_abc", original["id"], "original input must not be mutated")
}
// TestFilterCodexInput_MessageStripKeepsFunctionCallBehavior guards against a
// regression of #3785: message and function_call id rules are independent.
func TestFilterCodexInput_MessageStripKeepsFunctionCallBehavior(t *testing.T) {
input := []any{
map[string]any{
"type": "message",
"id": "item_msg_001",
"role": "user",
},
map[string]any{
"type": "function_call",
"id": "fc_validID123",
"call_id": "fc_validID123",
"name": "bash",
},
map[string]any{
"type": "function_call",
"id": "item_A9v0SNfS3VaLrfX0j3y4xhyK",
"call_id": "fc_abc123",
"name": "bash",
},
map[string]any{
"type": "function_call_output",
"id": "o1",
"call_id": "fc_abc123",
"output": "done",
},
}
filtered := filterCodexInputWithOptions(input, codexInputFilterOptions{
PreserveReferences: true,
})
require.Len(t, filtered, 4)
msg, ok := filtered[0].(map[string]any)
require.True(t, ok)
_, hasID := msg["id"]
require.False(t, hasID, "message item_* id should be stripped")
fcValid, ok := filtered[1].(map[string]any)
require.True(t, ok)
require.Equal(t, "fc_validID123", fcValid["id"], "valid fc* id must be preserved")
fcBad, ok := filtered[2].(map[string]any)
require.True(t, ok)
_, hasID = fcBad["id"]
require.False(t, hasID, "function_call item_* id should still be stripped")
require.Equal(t, "fc_abc123", fcBad["call_id"], "call_id pairing must survive")
out, ok := filtered[3].(map[string]any)
require.True(t, ok)
require.Equal(t, "o1", out["id"], "output item id should be preserved")
require.Equal(t, "fc_abc123", out["call_id"], "call_id pairing must survive")
}
@@ -1405,6 +1405,15 @@ func filterCodexInputWithOptions(input []any, opts codexInputFilterOptions) []an
ensureCopy()
delete(newItem, "id")
}
} else if typ == "message" {
// 同理,message 类 item 的 id 必须以 "msg" 开头(上游校验
// "Expected an ID that begins with 'msg'")。item_* 形式的 id
// 来自客户端回放,需要删除。
// 注意:不改写成 msg_*,改写出的 id 未必对应真实的上游对象。
if id, ok := m["id"].(string); ok && id != "" && !strings.HasPrefix(id, "msg") {
ensureCopy()
delete(newItem, "id")
}
}
filtered = append(filtered, newItem)
@@ -2,18 +2,10 @@ package service
import "github.com/tidwall/gjson"
// HasCompactionTriggerInInput detects the Codex remote compact v2 body signal:
// an input item with type "compaction_trigger". When the client sends this
// inside a normal POST /v1/responses (instead of POST /v1/responses/compact),
// the request must still be treated as a compact request — otherwise the
// upstream path, model mapping, and body normalization are all wrong, causing
// Codex to receive a non-compact response and fail with:
//
// "remote compaction v2 expected exactly one compaction output item, got 0"
//
// The gateway handler promotes such requests by rewriting the URL path to the
// compact form before stream parsing, compact body normalization, and
// compact-capable account scheduling, so both inbound forms share one code path.
// HasCompactionTriggerInInput detects an input item with
// type="compaction_trigger". The handler combines this body signal with the
// request path, stream flag, and Codex beta feature header to distinguish the
// native remote compaction v2 wire from the legacy /responses/compact bridge.
func HasCompactionTriggerInInput(body []byte) bool {
if len(body) == 0 {
return false
@@ -1,6 +1,9 @@
package service
import (
"bufio"
"errors"
"net"
"net/http"
"sync"
"time"
@@ -43,12 +46,14 @@ func StartOpenAICompactSSEKeepalive(c *gin.Context, interval time.Duration) func
if c == nil || c.Writer == nil || interval <= 0 || !openAICompactClientWantsStream(c) {
return func() {}
}
originalWriter := c.Writer
k := &openAICompactSSEKeepalive{
writer: c.Writer,
writer: originalWriter,
stop: make(chan struct{}),
}
c.Set(openAICompactSSEKeepaliveKey, k)
c.Writer = &openAICompactKeepaliveWriter{ResponseWriter: c.Writer, k: k}
wrappedWriter := &openAICompactKeepaliveWriter{ResponseWriter: originalWriter, k: k}
c.Writer = wrappedWriter
var reqDone <-chan struct{}
if c.Request != nil {
@@ -71,7 +76,14 @@ func StartOpenAICompactSSEKeepalive(c *gin.Context, interval time.Duration) func
timer.Reset(interval)
}
}()
return k.Stop
return func() {
k.Stop()
// Do not leave a pooled middleware writer reachable through the compact
// wrapper after the request finishes.
if current, ok := c.Writer.(*openAICompactKeepaliveWriter); ok && current == wrappedWriter {
c.Writer = originalWriter
}
}
}
// beat 在锁内提交(首次)响应头并写出一条 SSE 注释行;返回 false 表示心跳已
@@ -181,52 +193,105 @@ type openAICompactKeepaliveWriter struct {
// suspend 停拍心跳;幂等。任何响应构造(含 Header 访问——写响应必先操作
// 响应头)都视为请求侧接管 ResponseWriter。
func (w *openAICompactKeepaliveWriter) suspend() {
if w.k == nil {
return
}
w.k.Stop()
}
func (w *openAICompactKeepaliveWriter) Header() http.Header {
w.suspend()
if w.ResponseWriter == nil {
return http.Header{}
}
return w.ResponseWriter.Header()
}
func (w *openAICompactKeepaliveWriter) Write(data []byte) (int, error) {
w.suspend()
if w.ResponseWriter == nil {
return 0, nil
}
return w.ResponseWriter.Write(data)
}
func (w *openAICompactKeepaliveWriter) WriteString(s string) (int, error) {
w.suspend()
if w.ResponseWriter == nil {
return 0, nil
}
return w.ResponseWriter.WriteString(s)
}
func (w *openAICompactKeepaliveWriter) WriteHeader(code int) {
w.suspend()
if w.ResponseWriter == nil {
return
}
w.ResponseWriter.WriteHeader(code)
}
func (w *openAICompactKeepaliveWriter) WriteHeaderNow() {
w.suspend()
if w.ResponseWriter == nil {
return
}
w.ResponseWriter.WriteHeaderNow()
}
func (w *openAICompactKeepaliveWriter) Flush() {
w.suspend()
if w.ResponseWriter == nil {
return
}
w.ResponseWriter.Flush()
}
func (w *openAICompactKeepaliveWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
if w.ResponseWriter == nil {
return nil, nil, errors.New("response writer released")
}
return w.ResponseWriter.Hijack()
}
func (w *openAICompactKeepaliveWriter) CloseNotify() <-chan bool {
if w.ResponseWriter == nil {
ch := make(chan bool)
close(ch)
return ch
}
return w.ResponseWriter.CloseNotify()
}
func (w *openAICompactKeepaliveWriter) Pusher() http.Pusher {
if w.ResponseWriter == nil {
return nil
}
return w.ResponseWriter.Pusher()
}
func (w *openAICompactKeepaliveWriter) Status() int {
if w.k == nil || w.ResponseWriter == nil {
return 0
}
w.k.mu.Lock()
defer w.k.mu.Unlock()
return w.ResponseWriter.Status()
}
func (w *openAICompactKeepaliveWriter) Size() int {
if w.k == nil || w.ResponseWriter == nil {
return 0
}
w.k.mu.Lock()
defer w.k.mu.Unlock()
return w.ResponseWriter.Size()
}
func (w *openAICompactKeepaliveWriter) Written() bool {
if w.k == nil || w.ResponseWriter == nil {
return false
}
w.k.mu.Lock()
defer w.k.mu.Unlock()
return w.ResponseWriter.Written()
@@ -6,6 +6,7 @@ import (
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/tidwall/gjson"
)
@@ -141,6 +142,110 @@ func TestOpenAICompactKeepaliveWriter_RequestSideWriteSuspendsBeats(t *testing.T
require.Contains(t, rec.Body.String(), `{"error":"local reject"}`)
}
func TestOpenAICompactKeepaliveWriter_NilInnerWriter_NoPanic(t *testing.T) {
w := &openAICompactKeepaliveWriter{
k: &openAICompactSSEKeepalive{stop: make(chan struct{})},
}
w.ResponseWriter = nil
assert.NotPanics(t, func() {
assert.Equal(t, 0, w.Status())
})
assert.NotPanics(t, func() {
assert.Equal(t, 0, w.Size())
})
assert.NotPanics(t, func() {
assert.False(t, w.Written())
})
assert.NotPanics(t, func() {
assert.NotNil(t, w.Header())
})
assert.NotPanics(t, func() {
n, err := w.Write([]byte("test"))
assert.Equal(t, 0, n)
assert.NoError(t, err)
})
assert.NotPanics(t, func() {
n, err := w.WriteString("test")
assert.Equal(t, 0, n)
assert.NoError(t, err)
})
assert.NotPanics(t, func() {
w.WriteHeader(http.StatusOK)
})
assert.NotPanics(t, func() {
w.WriteHeaderNow()
})
assert.NotPanics(t, func() {
w.Flush()
})
assert.NotPanics(t, func() {
conn, rw, err := w.Hijack()
assert.Nil(t, conn)
assert.Nil(t, rw)
assert.Error(t, err)
})
assert.NotPanics(t, func() {
ch := w.CloseNotify()
assert.NotNil(t, ch)
})
assert.NotPanics(t, func() {
assert.Nil(t, w.Pusher())
})
}
func TestOpenAICompactKeepaliveWriter_NilKeepalive_NoPanic(t *testing.T) {
c, rec := newCompactBridgeTestContext(t, true)
w := &openAICompactKeepaliveWriter{ResponseWriter: c.Writer}
assert.NotPanics(t, func() {
assert.Equal(t, 0, w.Status())
})
assert.NotPanics(t, func() {
assert.Equal(t, 0, w.Size())
})
assert.NotPanics(t, func() {
assert.False(t, w.Written())
})
assert.NotPanics(t, func() {
w.Header().Set("X-Test", "ok")
})
assert.NotPanics(t, func() {
w.WriteHeader(http.StatusAccepted)
})
assert.NotPanics(t, func() {
n, err := w.WriteString("ok")
assert.Equal(t, 2, n)
assert.NoError(t, err)
})
assert.NotPanics(t, func() {
w.Flush()
})
require.Equal(t, "ok", rec.Header().Get("X-Test"))
require.Equal(t, "ok", rec.Body.String())
}
func TestOpenAICompactKeepaliveWriter_DelegatesWhenReady(t *testing.T) {
c, rec := newCompactBridgeTestContext(t, true)
stop := StartOpenAICompactSSEKeepalive(c, time.Hour)
defer stop()
w, ok := c.Writer.(*openAICompactKeepaliveWriter)
require.True(t, ok)
w.Header().Set("X-Test", "ok")
w.WriteHeader(http.StatusAccepted)
n, err := w.WriteString("ready")
require.NoError(t, err)
require.Equal(t, len("ready"), n)
require.Equal(t, http.StatusAccepted, w.Status())
require.Equal(t, len("ready"), w.Size())
require.True(t, w.Written())
require.Equal(t, "ok", rec.Header().Get("X-Test"))
require.Equal(t, "ready", rec.Body.String())
}
// fast policy block 在心跳提交后必须降级为 response.failed 终止事件。
func TestWriteOpenAIFastPolicyBlockedResponse_AfterKeepaliveCommit(t *testing.T) {
c, rec := newCompactBridgeTestContext(t, true)
@@ -837,8 +837,7 @@ func TestForwardAsAnthropic_ReusesOAuthCodexTurnState(t *testing.T) {
require.NoError(t, err)
require.NotNil(t, firstResult)
require.Empty(t, upstream.requests[0].Header.Get("x-codex-turn-state"))
require.Empty(t, upstream.requests[0].Header.Get("OpenAI-Beta"))
require.Empty(t, upstream.requests[0].Header.Get("originator"))
requireOpenAIMessagesCodexIdentity(t, upstream.requests[0], codexCLIUserAgent, "codex_cli_rs")
secondBody := []byte(`{"model":"claude-sonnet-4-5","max_tokens":16,"messages":[{"role":"user","content":"first"},{"role":"assistant","content":"ok"},{"role":"user","content":"second"}],"stream":false}`)
secondRec := httptest.NewRecorder()
@@ -852,12 +851,73 @@ func TestForwardAsAnthropic_ReusesOAuthCodexTurnState(t *testing.T) {
require.Equal(t, "turn_state_first", upstream.requests[1].Header.Get("x-codex-turn-state"))
require.Equal(t, generateSessionUUID(isolateOpenAISessionID(0, "stable-cache-key")), upstream.requests[1].Header.Get("session_id"))
require.Empty(t, upstream.requests[1].Header.Get("conversation_id"))
require.Empty(t, upstream.requests[1].Header.Get("OpenAI-Beta"))
require.Empty(t, upstream.requests[1].Header.Get("originator"))
requireOpenAIMessagesCodexIdentity(t, upstream.requests[1], codexCLIUserAgent, "codex_cli_rs")
require.False(t, gjson.GetBytes(upstream.bodies[1], "prompt_cache_key").Exists())
require.False(t, gjson.GetBytes(upstream.bodies[1], "previous_response_id").Exists())
}
func TestForwardAsAnthropic_OAuthRestoresCodexIdentityHeaders(t *testing.T) {
gin.SetMode(gin.TestMode)
const tuiUA = "codex-tui/9.9.9 (Mac OS X 14.0; arm64) iTerm (codex-tui; 9.9.9)"
tests := []struct {
name string
userAgent string
originator string
wantUserAgent string
wantOriginator string
}{
{
name: "官方UA逐字保留并重新配对",
userAgent: tuiUA,
originator: "opencode",
wantUserAgent: tuiUA,
wantOriginator: "codex-tui",
},
{
name: "第三方UA回退为默认Codex身份",
userAgent: "third-party-client/1.0.0",
originator: "opencode",
wantUserAgent: codexCLIUserAgent,
wantOriginator: "codex_cli_rs",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
body := []byte(`{"model":"claude-sonnet-4-5","max_tokens":16,"messages":[{"role":"user","content":"hello"}],"stream":false}`)
rec := httptest.NewRecorder()
c, _ := gin.CreateTestContext(rec)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/messages", bytes.NewReader(body))
c.Request.Header.Set("Content-Type", "application/json")
c.Request.Header.Set("User-Agent", tt.userAgent)
c.Request.Header.Set("originator", tt.originator)
upstream := &httpUpstreamRecorder{resp: openAICompatSSECompletedResponse("resp_identity", "gpt-5.4")}
svc := &OpenAIGatewayService{
httpUpstream: upstream,
cfg: &config.Config{Security: config.SecurityConfig{URLAllowlist: config.URLAllowlistConfig{Enabled: false}}},
}
account := &Account{
ID: 1,
Name: "openai-oauth",
Platform: PlatformOpenAI,
Type: AccountTypeOAuth,
Concurrency: 1,
Credentials: map[string]any{
"access_token": "oauth-token",
"chatgpt_account_id": "chatgpt-acc",
},
}
result, err := svc.ForwardAsAnthropic(context.Background(), c, account, body, "", "gpt-5.4")
require.NoError(t, err)
require.NotNil(t, result)
requireOpenAIMessagesCodexIdentity(t, upstream.lastReq, tt.wantUserAgent, tt.wantOriginator)
})
}
}
func TestForwardAsAnthropic_OAuthDigestFallbackReusesTurnStateWithoutExplicitKey(t *testing.T) {
t.Parallel()
gin.SetMode(gin.TestMode)
@@ -896,6 +956,7 @@ func TestForwardAsAnthropic_OAuthDigestFallbackReusesTurnStateWithoutExplicitKey
firstSessionID := upstream.requests[0].Header.Get("session_id")
require.NotEmpty(t, firstSessionID)
require.Empty(t, upstream.requests[0].Header.Get("x-codex-turn-state"))
requireOpenAIMessagesCodexIdentity(t, upstream.requests[0], codexCLIUserAgent, "codex_cli_rs")
require.False(t, gjson.GetBytes(upstream.bodies[0], "prompt_cache_key").Exists())
secondBody := []byte(`{"model":"claude-sonnet-4-5","max_tokens":16,"messages":[{"role":"user","content":"first"},{"role":"assistant","content":"ok"},{"role":"user","content":"second"}],"stream":false}`)
@@ -910,6 +971,7 @@ func TestForwardAsAnthropic_OAuthDigestFallbackReusesTurnStateWithoutExplicitKey
require.Equal(t, firstSessionID, upstream.requests[1].Header.Get("session_id"))
require.Equal(t, "turn_state_digest_first", upstream.requests[1].Header.Get("x-codex-turn-state"))
require.Empty(t, upstream.requests[1].Header.Get("conversation_id"))
requireOpenAIMessagesCodexIdentity(t, upstream.requests[1], codexCLIUserAgent, "codex_cli_rs")
require.False(t, gjson.GetBytes(upstream.bodies[1], "prompt_cache_key").Exists())
require.False(t, gjson.GetBytes(upstream.bodies[1], "previous_response_id").Exists())
}
@@ -1064,8 +1126,7 @@ func TestForwardAsAnthropic_OAuthKeepsSystemAsDeveloperInput(t *testing.T) {
instructions := gjson.GetBytes(upstream.lastBody, "instructions")
require.True(t, instructions.Exists())
require.Empty(t, instructions.String())
require.Empty(t, upstream.requests[0].Header.Get("OpenAI-Beta"))
require.Empty(t, upstream.requests[0].Header.Get("originator"))
requireOpenAIMessagesCodexIdentity(t, upstream.requests[0], codexCLIUserAgent, "codex_cli_rs")
}
func TestForwardAsAnthropic_OAuthAddsClaudeCodeTodoGuardForCompatModel(t *testing.T) {
@@ -1202,6 +1263,15 @@ func openAICompatSSECompletedResponse(responseID, model string) *http.Response {
}
}
func requireOpenAIMessagesCodexIdentity(t *testing.T, req *http.Request, wantUserAgent, wantOriginator string) {
t.Helper()
require.NotNil(t, req)
require.Equal(t, wantUserAgent, req.Header.Get("User-Agent"))
require.Equal(t, wantOriginator, req.Header.Get("originator"))
require.Equal(t, codexCLIVersion, req.Header.Get("version"))
require.Equal(t, "responses=experimental", req.Header.Get("OpenAI-Beta"))
}
func openAICompatSSEResponseWithoutUsage(responseID, model string) *http.Response {
body := strings.Join([]string{
`data: {"type":"response.completed","response":{"id":"` + responseID + `","object":"response","model":"` + model + `","status":"completed","output":[{"type":"message","id":"msg_1","role":"assistant","status":"completed","content":[{"type":"output_text","text":"ok"}]}]}}`,
@@ -88,6 +88,9 @@ func (s *OpenAIGatewayService) failoverOpenAIUpstreamHTTPError(
upstreamMsg string,
upstreamModel string,
) *UpstreamFailoverError {
if account != nil && account.Platform == PlatformGrok {
s.handleGrokAccountUpstreamError(ctx, account, resp.StatusCode, resp.Header, respBody)
}
if !s.shouldFailoverOpenAIUpstreamResponse(resp.StatusCode, upstreamMsg, respBody) {
return nil
}
@@ -109,7 +112,9 @@ func (s *OpenAIGatewayService) failoverOpenAIUpstreamHTTPError(
Message: upstreamMsg,
Detail: upstreamDetail,
})
s.handleOpenAIAccountUpstreamError(ctx, account, resp.StatusCode, resp.Header, respBody, upstreamModel)
if account.Platform != PlatformGrok {
s.handleOpenAIAccountUpstreamError(ctx, account, resp.StatusCode, resp.Header, respBody, upstreamModel)
}
return &UpstreamFailoverError{
StatusCode: resp.StatusCode,
ResponseBody: respBody,
@@ -159,6 +164,7 @@ func (s *OpenAIGatewayService) sendCCUpstreamRequest(
stream bool,
bearerToken string,
userAgent string,
grokCacheIdentity string,
) (*http.Response, error) {
upstreamCtx, releaseUpstreamCtx := detachUpstreamContext(ctx)
upstreamReq, err := http.NewRequestWithContext(upstreamCtx, http.MethodPost, targetURL, bytes.NewReader(body))
@@ -190,6 +196,10 @@ func (s *OpenAIGatewayService) sendCCUpstreamRequest(
// 账号级请求头覆写(仅 openai api_key 账号启用时生效)
account.ApplyHeaderOverrides(upstreamReq.Header)
if account.Platform == PlatformGrok {
applyGrokCLIHeaders(upstreamReq.Header)
applyGrokCacheHeaders(upstreamReq.Header, grokCacheIdentity)
}
proxyURL := ""
if account.Proxy != nil {
@@ -72,6 +72,16 @@ func (s *OpenAIGatewayService) ForwardAsChatCompletions(
}
if account.Platform == PlatformGrok {
if account.IsGrokOAuth() {
if eligible, reason := grokChatResponsesBridgeEligibility(body); eligible {
return s.forwardGrokChatCompletionsViaResponses(ctx, c, account, body, promptCacheKey, defaultMappedModel)
} else {
logger.L().Debug("grok chat_completions: using raw fallback",
zap.Int64("account_id", account.ID),
zap.String("reason", reason),
)
}
}
return s.forwardAsRawChatCompletions(ctx, c, account, body, defaultMappedModel)
}
@@ -76,6 +76,12 @@ func (s *OpenAIGatewayService) forwardAsRawChatCompletions(
// 2. Resolve model mapping (same as ForwardAsChatCompletions)
billingModel := resolveOpenAIForwardModel(account, originalModel, defaultMappedModel)
upstreamModel := normalizeOpenAIModelForUpstream(account, billingModel)
grokCacheIdentity := ""
if account.Platform == PlatformGrok {
// Resolve before image bridging or other body rewrites so the fallback is
// anchored to the client's stable conversation prefix.
grokCacheIdentity = resolveGrokCacheIdentity(c, body, "", upstreamModel)
}
reasoningEffort := extractOpenAIReasoningEffortFromBody(body, upstreamModel, billingModel, originalModel)
// 国产模型默认 effort 补充:需要 mappedModel 判定,推迟到 billingModel 算出之后。
reasoningEffort = ApplyThinkingEnabledFallback(reasoningEffort, body, billingModel)
@@ -134,6 +140,12 @@ func (s *OpenAIGatewayService) forwardAsRawChatCompletions(
return nil, fmt.Errorf("enable stream usage: %w", usageErr)
}
}
if account.Platform == PlatformGrok {
upstreamBody, err = stripGrokChatPromptCacheKey(upstreamBody)
if err != nil {
return nil, fmt.Errorf("remove Responses-only Grok prompt cache key: %w", err)
}
}
logger.L().Debug("openai chat_completions raw: forwarding without protocol conversion",
zap.Int64("account_id", account.ID),
@@ -148,11 +160,12 @@ func (s *OpenAIGatewayService) forwardAsRawChatCompletions(
if err != nil {
return nil, err
}
SetActualOpenAIUpstreamEndpoint(c, grokChatRawEndpoint)
customUA := account.GetOpenAIUserAgent()
if customUA == "" && account.Platform == PlatformGrok {
customUA = "sub2api-grok/1.0"
}
resp, err := s.sendCCUpstreamRequest(ctx, c, account, targetURL, upstreamBody, clientStream, token, customUA)
resp, err := s.sendCCUpstreamRequest(ctx, c, account, targetURL, upstreamBody, clientStream, token, customUA, grokCacheIdentity)
if err != nil {
return nil, err
}
@@ -162,7 +175,6 @@ func (s *OpenAIGatewayService) forwardAsRawChatCompletions(
if resp.StatusCode >= 400 {
respBody, upstreamMsg := s.readOpenAIUpstreamError(resp)
if account.Platform == PlatformGrok {
s.updateGrokUsageSnapshot(ctx, account.ID, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
appendOpsUpstreamError(c, OpsUpstreamErrorEvent{
Platform: account.Platform,
AccountID: account.ID,
@@ -189,7 +201,7 @@ func (s *OpenAIGatewayService) forwardAsRawChatCompletions(
}
if account.Platform == PlatformGrok {
s.updateGrokUsageSnapshot(ctx, account.ID, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
s.updateGrokUsageSnapshot(ctx, account, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
}
// 8. Forward response
@@ -202,6 +214,7 @@ func (s *OpenAIGatewayService) forwardAsRawChatCompletions(
}
if result != nil {
addOpenAIUsage(&result.Usage, bridgeUsage)
result.UpstreamEndpoint = grokChatRawEndpoint
}
return result, forwardErr
}
@@ -49,7 +49,6 @@ func (s *OpenAIGatewayService) Forward(ctx context.Context, c *gin.Context, acco
originalModel := reqModel
if account.Platform == PlatformGrok {
_ = promptCacheKey
return s.forwardGrokResponses(ctx, c, account, body, originalModel, reqStream, startTime)
}
+273 -23
View File
@@ -6,6 +6,7 @@ import (
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
"strings"
"time"
@@ -20,6 +21,9 @@ import (
const (
grokComposerImageBridgeVisionModel = "grok-build-0.1"
grokComposerImageBridgeMaxOutputTokens = 512
grokUpstreamUserAgent = "sub2api-grok/1.0"
grokCLIVersion = "0.2.93"
grokRateLimitFallbackCooldown = 2 * time.Minute
)
func (s *OpenAIGatewayService) forwardGrokResponses(
@@ -31,18 +35,23 @@ func (s *OpenAIGatewayService) forwardGrokResponses(
reqStream bool,
startTime time.Time,
) (*OpenAIForwardResult, error) {
if account.Type != AccountTypeOAuth {
return nil, fmt.Errorf("grok account type %s is not supported by subscription forwarding", account.Type)
if account.Type != AccountTypeOAuth && account.Type != AccountTypeAPIKey {
return nil, fmt.Errorf("grok account type %s is not supported by Responses forwarding", account.Type)
}
upstreamModel := account.GetMappedModel(originalModel)
if strings.TrimSpace(upstreamModel) == "" {
upstreamModel = "grok-4.3"
}
cacheIdentity := resolveGrokCacheIdentity(c, body, "", upstreamModel)
patchedBody, err := patchGrokResponsesBody(body, upstreamModel)
if err != nil {
return nil, err
}
patchedBody, err = applyGrokResponsesCacheIdentity(patchedBody, body, cacheIdentity, account.IsGrokOAuth())
if err != nil {
return nil, fmt.Errorf("apply grok prompt cache identity: %w", err)
}
token, _, err := s.GetAccessToken(ctx, account)
if err != nil {
@@ -51,7 +60,7 @@ func (s *OpenAIGatewayService) forwardGrokResponses(
upstreamCtx, releaseUpstreamCtx := detachUpstreamContext(ctx)
defer releaseUpstreamCtx()
upstreamReq, err := buildGrokResponsesRequest(upstreamCtx, c, account, patchedBody, token)
upstreamReq, err := buildGrokResponsesRequest(upstreamCtx, c, account, patchedBody, token, cacheIdentity)
if err != nil {
return nil, err
}
@@ -72,7 +81,6 @@ func (s *OpenAIGatewayService) forwardGrokResponses(
if resp.StatusCode >= 400 {
respBody := s.readUpstreamErrorBody(resp)
resp.Body = io.NopCloser(bytes.NewReader(respBody))
s.updateGrokUsageSnapshot(ctx, account.ID, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
upstreamMsg := sanitizeUpstreamErrorMessage(extractUpstreamErrorMessage(respBody))
if upstreamMsg == "" {
upstreamMsg = fmt.Sprintf("xAI upstream returned status %d", resp.StatusCode)
@@ -97,7 +105,7 @@ func (s *OpenAIGatewayService) forwardGrokResponses(
return s.handleErrorResponse(ctx, resp, c, account, patchedBody, upstreamModel)
}
s.updateGrokUsageSnapshot(ctx, account.ID, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
s.updateGrokUsageSnapshot(ctx, account, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
var usage *OpenAIUsage
var firstTokenMs *int
@@ -146,6 +154,10 @@ func patchGrokResponsesBody(body []byte, upstreamModel string) ([]byte, error) {
if err != nil {
return nil, err
}
out, err = sanitizeGrokResponsesModelCapabilities(out, upstreamModel)
if err != nil {
return nil, err
}
for _, unsupportedField := range []string{"prompt_cache_retention", "safety_identifier"} {
if gjson.GetBytes(out, unsupportedField).Exists() {
out, err = sjson.DeleteBytes(out, unsupportedField)
@@ -168,6 +180,10 @@ func patchGrokResponsesBody(body []byte, upstreamModel string) ([]byte, error) {
if err != nil {
return nil, err
}
out, err = sanitizeGrokResponsesInput(out)
if err != nil {
return nil, err
}
out, err = sanitizeGrokResponsesTools(out)
if err != nil {
return nil, err
@@ -175,6 +191,38 @@ func patchGrokResponsesBody(body []byte, upstreamModel string) ([]byte, error) {
return out, nil
}
func sanitizeGrokResponsesModelCapabilities(body []byte, upstreamModel string) ([]byte, error) {
if !grokModelRejectsReasoningEffort(upstreamModel) {
return body, nil
}
out := body
for _, field := range []string{"reasoning", "reasoning_effort", "reasoningEffort"} {
if !gjson.GetBytes(out, field).Exists() {
continue
}
var err error
out, err = sjson.DeleteBytes(out, field)
if err != nil {
return nil, fmt.Errorf("remove unsupported Grok Composer %s: %w", field, err)
}
}
return out, nil
}
func grokModelRejectsReasoningEffort(model string) bool {
model = strings.TrimSpace(strings.ToLower(model))
if slash := strings.LastIndex(model, "/"); slash >= 0 {
model = strings.TrimSpace(model[slash+1:])
}
switch model {
case "grok-composer", "grok-composer-2.5-fast", "composer-2.5":
return true
default:
return false
}
}
var grokResponsesUnsupportedRecursiveFields = map[string]struct{}{
"external_web_access": {},
}
@@ -223,6 +271,38 @@ func deleteJSONFields(value any, fields map[string]struct{}) bool {
}
}
// additional_tools is a Codex/Responses Lite private input carrier. xAI's
// Responses schema accepts ordinary message/function-call input items but
// rejects this carrier before inference with a ModelInput deserialization
// error. Top-level supported tools remain available through the separate
// sanitizeGrokResponsesTools path.
func sanitizeGrokResponsesInput(body []byte) ([]byte, error) {
if !bytes.Contains(body, []byte(`"additional_tools"`)) {
return body, nil
}
input := gjson.GetBytes(body, "input")
if !input.Exists() || !input.IsArray() {
return body, nil
}
rawItems := input.Array()
filtered := make([]json.RawMessage, 0, len(rawItems))
for _, item := range rawItems {
if strings.TrimSpace(item.Get("type").String()) == "additional_tools" {
continue
}
filtered = append(filtered, json.RawMessage(item.Raw))
}
if len(filtered) == len(rawItems) {
return body, nil
}
encoded, err := json.Marshal(filtered)
if err != nil {
return nil, err
}
return sjson.SetRawBytes(body, "input", encoded)
}
var grokResponsesSupportedToolTypes = map[string]struct{}{
"code_execution": {},
"code_interpreter": {},
@@ -457,7 +537,9 @@ func (s *OpenAIGatewayService) describeGrokComposerImage(
}
upstreamCtx, releaseUpstreamCtx := detachUpstreamContext(ctx)
upstreamReq, err := buildGrokResponsesRequest(upstreamCtx, c, account, body, token)
// Image-description probes are auxiliary requests, not conversation turns.
// Do not bind them to the caller's Grok prompt-cache identity.
upstreamReq, err := buildGrokResponsesRequest(upstreamCtx, c, account, body, token, "")
releaseUpstreamCtx()
if err != nil {
return "", OpenAIUsage{}, fmt.Errorf("build grok composer image bridge request: %w", err)
@@ -476,7 +558,6 @@ func (s *OpenAIGatewayService) describeGrokComposerImage(
if resp.StatusCode >= 400 {
respBody := s.readUpstreamErrorBody(resp)
s.updateGrokUsageSnapshot(ctx, account.ID, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
upstreamMsg := sanitizeUpstreamErrorMessage(extractUpstreamErrorMessage(respBody))
if upstreamMsg == "" {
upstreamMsg = fmt.Sprintf("xAI image bridge upstream returned status %d", resp.StatusCode)
@@ -501,7 +582,7 @@ func (s *OpenAIGatewayService) describeGrokComposerImage(
return "", OpenAIUsage{}, fmt.Errorf("grok composer image bridge upstream error: %s", upstreamMsg)
}
s.updateGrokUsageSnapshot(ctx, account.ID, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
s.updateGrokUsageSnapshot(ctx, account, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
respBody, err := ReadUpstreamResponseBody(resp.Body, s.cfg, c, nil)
if err != nil {
return "", OpenAIUsage{}, fmt.Errorf("read grok composer image bridge response: %w", err)
@@ -623,7 +704,7 @@ func addOpenAIUsage(dst *OpenAIUsage, usage OpenAIUsage) {
dst.ImageOutputTokens += usage.ImageOutputTokens
}
func buildGrokResponsesRequest(ctx context.Context, c *gin.Context, account *Account, body []byte, token string) (*http.Request, error) {
func buildGrokResponsesRequest(ctx context.Context, c *gin.Context, account *Account, body []byte, token, cacheIdentity string) (*http.Request, error) {
targetURL, err := xai.BuildResponsesURL(account.GetGrokBaseURL())
if err != nil {
return nil, err
@@ -635,7 +716,8 @@ func buildGrokResponsesRequest(ctx context.Context, c *gin.Context, account *Acc
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json, text/event-stream")
req.Header.Set("User-Agent", "sub2api-grok/1.0")
applyGrokCLIHeaders(req.Header)
applyGrokCacheHeaders(req.Header, cacheIdentity)
if c != nil {
if v := c.GetHeader("OpenAI-Beta"); strings.TrimSpace(v) != "" {
req.Header.Set("OpenAI-Beta", v)
@@ -644,33 +726,201 @@ func buildGrokResponsesRequest(ctx context.Context, c *gin.Context, account *Acc
return req, nil
}
func (s *OpenAIGatewayService) updateGrokUsageSnapshot(ctx context.Context, accountID int64, snapshot *xai.QuotaSnapshot) {
if s == nil || s.accountRepo == nil || accountID <= 0 || snapshot == nil {
// applyGrokCLIHeaders identifies subscription traffic as a supported Grok CLI
// version. The CLI gateway rejects otherwise valid OAuth requests without it.
func applyGrokCLIHeaders(headers http.Header) {
if headers == nil {
return
}
if s.codexSnapshotThrottle != nil && !s.codexSnapshotThrottle.Allow(accountID, time.Now()) {
headers.Set("User-Agent", grokUpstreamUserAgent)
headers.Set("X-Grok-Client-Version", grokCLIVersion)
}
func (s *OpenAIGatewayService) updateGrokUsageSnapshot(ctx context.Context, account *Account, snapshot *xai.QuotaSnapshot) {
if s == nil || account == nil || account.ID <= 0 || snapshot == nil {
return
}
_ = s.accountRepo.UpdateExtra(ctx, accountID, map[string]any{
grokQuotaSnapshotExtraKey: snapshot,
})
accountID := account.ID
now := time.Now()
resetAt, hasActiveLimit := grokRateLimitResetAt(snapshot, now)
if hasActiveLimit {
normalizeGrokExhaustedWindowResets(snapshot, resetAt, now)
}
critical := snapshot.StatusCode == http.StatusTooManyRequests || hasActiveLimit
if s.codexSnapshotThrottle != nil {
allowed := s.codexSnapshotThrottle.Allow(accountID, now)
if !critical && !allowed {
return
}
}
stateCtx := ctx
if hasActiveLimit {
var cancel context.CancelFunc
stateCtx, cancel = openAIAccountStateContext(ctx)
defer cancel()
}
if s.accountRepo != nil {
_ = s.accountRepo.UpdateExtra(stateCtx, accountID, map[string]any{
grokQuotaSnapshotExtraKey: snapshot,
})
}
// Error responses are reconciled by handleGrokAccountUpstreamError, which
// also installs the immediate in-memory scheduling block. Successful
// responses can still consume the last available request/token, so persist
// that exhausted window here as a real rate limit rather than relying only
// on the passive snapshot scheduler check.
if hasActiveLimit {
s.rateLimitGrok(stateCtx, account, resetAt)
}
}
func parseGrokQuotaSnapshot(headers http.Header, statusCode int, now time.Time) *xai.QuotaSnapshot {
snapshot := xai.ParseQuotaHeaders(headers, statusCode)
if snapshot == nil && statusCode == http.StatusTooManyRequests {
return &xai.QuotaSnapshot{
StatusCode: statusCode,
UpdatedAt: now.UTC().Format(time.RFC3339),
}
}
return snapshot
}
func normalizeGrokExhaustedWindowResets(snapshot *xai.QuotaSnapshot, resetAt, now time.Time) {
if snapshot == nil || !resetAt.After(now) {
return
}
for _, window := range []*xai.QuotaWindow{snapshot.Requests, snapshot.Tokens} {
if window == nil || window.Remaining == nil || *window.Remaining > 0 {
continue
}
candidate := time.Time{}
if window.ResetUnix != nil && *window.ResetUnix > 0 {
candidate = time.Unix(*window.ResetUnix, 0)
} else if parsed, err := time.Parse(time.RFC3339, strings.TrimSpace(window.ResetAt)); err == nil {
candidate = parsed
}
if !candidate.After(now) {
candidate = resetAt
}
resetUnix := candidate.Unix()
window.ResetUnix = &resetUnix
window.ResetAt = candidate.UTC().Format(time.RFC3339)
}
}
func grokRateLimitResetAt(snapshot *xai.QuotaSnapshot, now time.Time) (time.Time, bool) {
if snapshot == nil {
return time.Time{}, false
}
// Retry-After is xAI's explicit retry boundary. Use the observation time so
// a persisted snapshot does not start a fresh cooldown every time it is read.
retryAfterExpired := false
var resetAt time.Time
if snapshot.RetryAfterSeconds != nil && *snapshot.RetryAfterSeconds > 0 {
observedAt := now
if parsed, err := time.Parse(time.RFC3339, strings.TrimSpace(snapshot.UpdatedAt)); err == nil {
observedAt = parsed
}
retryAfterResetAt := observedAt.Add(time.Duration(*snapshot.RetryAfterSeconds) * time.Second)
if retryAfterResetAt.After(now) {
resetAt = retryAfterResetAt
} else {
retryAfterExpired = true
}
}
exhausted := false
for _, window := range []*xai.QuotaWindow{snapshot.Requests, snapshot.Tokens} {
if window == nil || window.Remaining == nil || *window.Remaining > 0 {
continue
}
exhausted = true
candidate := time.Time{}
if window.ResetUnix != nil && *window.ResetUnix > 0 {
candidate = time.Unix(*window.ResetUnix, 0)
} else if parsed, err := time.Parse(time.RFC3339, strings.TrimSpace(window.ResetAt)); err == nil {
candidate = parsed
}
if candidate.After(now) && candidate.After(resetAt) {
resetAt = candidate
}
}
if !resetAt.IsZero() {
return resetAt, true
}
// An observed Retry-After is an absolute boundary once combined with the
// snapshot timestamp. Do not turn an expired persisted snapshot into a new
// rolling fallback cooldown, but still allow a later explicit window reset.
if retryAfterExpired {
return time.Time{}, false
}
if exhausted || snapshot.StatusCode == http.StatusTooManyRequests {
return now.Add(grokRateLimitFallbackCooldown), true
}
return time.Time{}, false
}
func normalizeGrokRateLimitResetAt(account *Account, resetAt, now time.Time) time.Time {
if !resetAt.After(now) {
resetAt = now.Add(grokRateLimitFallbackCooldown)
}
if account != nil && account.RateLimitResetAt != nil && account.RateLimitResetAt.After(resetAt) {
resetAt = *account.RateLimitResetAt
}
return resetAt
}
type grokRateLimitExtendingRepository interface {
SetRateLimitedIfLater(ctx context.Context, id int64, resetAt time.Time) error
}
func persistGrokRateLimit(ctx context.Context, repo AccountRepository, account *Account, resetAt time.Time) {
if repo == nil || account == nil || account.ID <= 0 {
return
}
resetAt = normalizeGrokRateLimitResetAt(account, resetAt, time.Now())
stateCtx, cancel := openAIAccountStateContext(ctx)
defer cancel()
var err error
if extendingRepo, ok := repo.(grokRateLimitExtendingRepository); ok {
err = extendingRepo.SetRateLimitedIfLater(stateCtx, account.ID, resetAt)
} else {
err = repo.SetRateLimited(stateCtx, account.ID, resetAt)
}
if err != nil {
slog.Warn("persist_grok_rate_limit_failed", "account_id", account.ID, "reset_at", resetAt.UTC(), "error", err)
}
}
func (s *OpenAIGatewayService) rateLimitGrok(ctx context.Context, account *Account, resetAt time.Time) {
if s == nil || account == nil {
return
}
resetAt = normalizeGrokRateLimitResetAt(account, resetAt, time.Now())
runtimeUntil := resetAt
if account.TempUnschedulableUntil != nil && account.TempUnschedulableUntil.After(runtimeUntil) {
runtimeUntil = *account.TempUnschedulableUntil
}
s.BlockAccountScheduling(account, runtimeUntil, "429")
persistGrokRateLimit(ctx, s.accountRepo, account, resetAt)
}
func (s *OpenAIGatewayService) handleGrokAccountUpstreamError(ctx context.Context, account *Account, statusCode int, headers http.Header, responseBody []byte) {
if s == nil || account == nil {
return
}
now := time.Now()
s.updateGrokUsageSnapshot(ctx, account, parseGrokQuotaSnapshot(headers, statusCode, now))
switch statusCode {
case http.StatusUnauthorized:
s.tempUnscheduleGrok(ctx, account, 10*time.Minute, "grok oauth token unauthorized")
s.tempUnscheduleGrok(ctx, account, 10*time.Minute, "grok credentials unauthorized")
case http.StatusForbidden:
s.tempUnscheduleGrok(ctx, account, 30*time.Minute, "grok entitlement or subscription tier denied")
s.tempUnscheduleGrok(ctx, account, 30*time.Minute, "grok access or entitlement denied")
case http.StatusTooManyRequests:
cooldown := 2 * time.Minute
if snapshot := xai.ParseQuotaHeaders(headers, statusCode); snapshot != nil && snapshot.RetryAfterSeconds != nil && *snapshot.RetryAfterSeconds > 0 {
cooldown = time.Duration(*snapshot.RetryAfterSeconds) * time.Second
}
s.tempUnscheduleGrok(ctx, account, cooldown, "grok rate limited")
// updateGrokUsageSnapshot installs both runtime and durable rate-limit state.
default:
if statusCode >= 500 {
s.tempUnscheduleGrok(ctx, account, 2*time.Minute, "grok upstream temporary error")
@@ -0,0 +1,149 @@
package service
import (
"fmt"
"net/http"
"strings"
"github.com/gin-gonic/gin"
"github.com/tidwall/gjson"
"github.com/tidwall/sjson"
)
const (
grokConversationIDHeader = "X-Grok-Conv-Id"
grokFreeCacheNativeToolsJSON = `[{"type":"web_search"},{"type":"x_search"}]`
grokFreeCacheDisabledToolChoice = "none"
)
// resolveGrokCacheIdentity derives one stable, tenant-isolated routing identity
// for xAI's server-side prompt cache. The returned value is safe to expose to
// the upstream: it never contains the client's raw session identifier.
//
// A valid downstream API key is required. This intentionally fails closed on
// internal probes and incomplete request contexts instead of creating a cache
// identity that could be shared by unrelated tenants.
func resolveGrokCacheIdentity(c *gin.Context, body []byte, explicitKey, upstreamModel string) string {
apiKeyID := getAPIKeyIDFromContext(c)
if apiKeyID <= 0 {
return ""
}
// /responses/compact rejects tool_choice and does not represent a normal
// conversation turn. Keep both cache identity and Free-tier routing
// augmentation out of this path.
if isOpenAIResponsesCompactPath(c) {
return ""
}
model := strings.ToLower(strings.TrimSpace(upstreamModel))
if model == "" {
return ""
}
seed := explicitGrokCacheSeed(c, body, explicitKey)
if seed == "" {
seed = deriveOpenAIContentSessionSeed(body)
}
if seed == "" {
return ""
}
// generateSessionUUID hashes the whole seed before formatting it as a UUID.
// Include a versioned namespace so this identity cannot collide with other
// upstream session identifiers derived by sub2api.
isolatedSeed := fmt.Sprintf("grok-prompt-cache:v1:%d:%s:%s", apiKeyID, model, seed)
return generateSessionUUID(isolatedSeed)
}
func explicitGrokCacheSeed(c *gin.Context, body []byte, explicitKey string) string {
seed := ""
if c != nil {
seed = strings.TrimSpace(c.GetHeader("session_id"))
if seed == "" {
seed = strings.TrimSpace(c.GetHeader("conversation_id"))
}
if seed == "" {
seed = strings.TrimSpace(c.GetHeader(grokConversationIDHeader))
}
}
if seed == "" && len(body) > 0 {
seed = strings.TrimSpace(gjson.GetBytes(body, "prompt_cache_key").String())
}
if seed == "" {
seed = strings.TrimSpace(explicitKey)
}
return seed
}
func isGrokRequestContext(c *gin.Context) bool {
if c == nil {
return false
}
v, exists := c.Get("api_key")
if !exists {
return false
}
apiKey, ok := v.(*APIKey)
return ok && apiKey != nil && apiKey.Group != nil && apiKey.Group.Platform == PlatformGrok
}
// applyGrokResponsesCacheIdentity writes the cache routing identity into an
// xAI Responses request. Existing client values are deliberately replaced by
// the tenant-isolated value to prevent collisions on shared OAuth accounts.
//
// Free OAuth requests without native search tools are routed by xAI to the
// non-cacheable build-free model. For otherwise tool-free requests, add the
// native tools with tool_choice=none: this selects the cache-capable tier
// without allowing an actual search. Any explicit client tools or tool_choice
// disable this augmentation so client function-calling semantics stay intact.
func applyGrokResponsesCacheIdentity(body, intentSourceBody []byte, identity string, injectFreeTierTools bool) ([]byte, error) {
identity = strings.TrimSpace(identity)
if identity == "" {
if gjson.GetBytes(body, "prompt_cache_key").Exists() {
return sjson.DeleteBytes(body, "prompt_cache_key")
}
return body, nil
}
out, err := sjson.SetBytes(body, "prompt_cache_key", identity)
if err != nil {
return nil, err
}
if !injectFreeTierTools {
return out, nil
}
// Inspect the pre-sanitization source. patchGrokResponsesBody may remove an
// unsupported client tool and its tool_choice; that must not turn an
// explicit client tool intent into an eligible native-tool request.
if gjson.GetBytes(intentSourceBody, "tools").Exists() || gjson.GetBytes(intentSourceBody, "tool_choice").Exists() {
return out, nil
}
out, err = sjson.SetRawBytes(out, "tools", []byte(grokFreeCacheNativeToolsJSON))
if err != nil {
return nil, err
}
return sjson.SetBytes(out, "tool_choice", grokFreeCacheDisabledToolChoice)
}
// applyGrokCacheHeaders applies the documented Chat Completions conversation
// routing header. The request is built from a fresh header map, so client
// supplied x-grok headers cannot override this server-derived value.
func applyGrokCacheHeaders(headers http.Header, identity string) {
if headers == nil {
return
}
identity = strings.TrimSpace(identity)
if identity == "" {
headers.Del(grokConversationIDHeader)
return
}
headers.Set(grokConversationIDHeader, identity)
}
// stripGrokChatPromptCacheKey removes the Responses-only body field after it
// has been used as an identity seed. Chat Completions routes cache by header.
func stripGrokChatPromptCacheKey(body []byte) ([]byte, error) {
if !gjson.GetBytes(body, "prompt_cache_key").Exists() {
return body, nil
}
return sjson.DeleteBytes(body, "prompt_cache_key")
}
@@ -0,0 +1,273 @@
//go:build unit
package service
import (
"net/http"
"net/http/httptest"
"sync"
"testing"
"github.com/gin-gonic/gin"
"github.com/stretchr/testify/require"
"github.com/tidwall/gjson"
)
func newGrokCacheTestContext(apiKeyID int64) *gin.Context {
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
if apiKeyID > 0 {
c.Set("api_key", &APIKey{ID: apiKeyID, Group: &Group{Platform: PlatformGrok}})
}
return c
}
func TestResolveGrokCacheIdentityStableAcrossAppendOnlyTurns(t *testing.T) {
gin.SetMode(gin.TestMode)
c := newGrokCacheTestContext(101)
round1 := []byte(`{"model":"grok","instructions":"be concise","tools":[{"type":"function","name":"lookup","parameters":{"type":"object"}}],"input":[{"role":"user","content":"first question"}]}`)
round2 := []byte(`{"model":"grok","instructions":"be concise","tools":[{"type":"function","name":"lookup","parameters":{"type":"object"}}],"input":[{"role":"user","content":"first question"},{"role":"assistant","content":"first answer"},{"role":"user","content":"second question"}]}`)
first := resolveGrokCacheIdentity(c, round1, "", "grok-4.5")
second := resolveGrokCacheIdentity(c, round2, "", "grok-4.5")
require.NotEmpty(t, first)
require.Len(t, first, 36)
require.Equal(t, first, second)
}
func TestResolveGrokCacheIdentityIsolatesAPIKeyAndMappedModel(t *testing.T) {
gin.SetMode(gin.TestMode)
body := []byte(`{"model":"grok","input":"same prompt"}`)
base := resolveGrokCacheIdentity(newGrokCacheTestContext(201), body, "", "grok-4.5")
otherTenant := resolveGrokCacheIdentity(newGrokCacheTestContext(202), body, "", "grok-4.5")
otherModel := resolveGrokCacheIdentity(newGrokCacheTestContext(201), body, "", "grok-4.3")
require.NotEmpty(t, base)
require.NotEqual(t, base, otherTenant)
require.NotEqual(t, base, otherModel)
}
func TestResolveGrokCacheIdentityUsesAndIsolatesNativeConversationHeader(t *testing.T) {
gin.SetMode(gin.TestMode)
c := newGrokCacheTestContext(301)
c.Request.Header.Set(grokConversationIDHeader, "raw-native-conversation")
body1 := []byte(`{"model":"grok","input":"one"}`)
body2 := []byte(`{"model":"grok","input":"different body that must not replace the explicit session"}`)
first := resolveGrokCacheIdentity(c, body1, "body-cache-key", "grok-4.5")
second := resolveGrokCacheIdentity(c, body2, "another-body-cache-key", "grok-4.5")
require.Equal(t, "raw-native-conversation", (&OpenAIGatewayService{}).ExtractSessionID(c, body1))
require.Equal(t, first, second)
require.NotEqual(t, "raw-native-conversation", first)
require.NotContains(t, first, "raw-native-conversation")
}
func TestResolveGrokCacheIdentityExplicitHeaderPriority(t *testing.T) {
gin.SetMode(gin.TestMode)
body := []byte(`{"model":"grok","prompt_cache_key":"body-key","input":"hi"}`)
c := newGrokCacheTestContext(401)
c.Request.Header.Set(grokConversationIDHeader, "grok-key")
c.Request.Header.Set("conversation_id", "conversation-key")
c.Request.Header.Set("session_id", "session-key")
got := resolveGrokCacheIdentity(c, body, "explicit-argument", "grok-4.5")
onlySession := newGrokCacheTestContext(401)
onlySession.Request.Header.Set("session_id", "session-key")
want := resolveGrokCacheIdentity(onlySession, []byte(`{"model":"grok","input":"unrelated"}`), "", "grok-4.5")
require.Equal(t, want, got)
}
func TestResolveGrokCacheIdentityFailsClosedWithoutAPIKeyContext(t *testing.T) {
gin.SetMode(gin.TestMode)
c := newGrokCacheTestContext(0)
c.Request.Header.Set(grokConversationIDHeader, "native-session")
require.Empty(t, resolveGrokCacheIdentity(c, []byte(`{"model":"grok","input":"hi"}`), "", "grok-4.5"))
require.Empty(t, resolveGrokCacheIdentity(nil, []byte(`{"model":"grok","prompt_cache_key":"key"}`), "key", "grok-4.5"))
}
func TestGrokConversationHeaderIsScopedToGrokRequestScheduling(t *testing.T) {
gin.SetMode(gin.TestMode)
body := []byte(`{"model":"grok","prompt_cache_key":"body-session","input":"hi"}`)
grokContext := newGrokCacheTestContext(601)
grokContext.Request.Header.Set(grokConversationIDHeader, "native-grok-session")
require.Equal(t, "native-grok-session", (&OpenAIGatewayService{}).ExtractSessionID(grokContext, body))
openAIContext := newGrokCacheTestContext(601)
openAIContext.Set("api_key", &APIKey{ID: 601, Group: &Group{Platform: PlatformOpenAI}})
openAIContext.Request.Header.Set(grokConversationIDHeader, "must-be-ignored")
require.Equal(t, "body-session", (&OpenAIGatewayService{}).ExtractSessionID(openAIContext, body))
withoutGrokHeader := newGrokCacheTestContext(601)
withoutGrokHeader.Set("api_key", &APIKey{ID: 601, Group: &Group{Platform: PlatformOpenAI}})
require.Equal(t,
(&OpenAIGatewayService{}).GenerateSessionHash(withoutGrokHeader, body),
(&OpenAIGatewayService{}).GenerateSessionHash(openAIContext, body),
)
}
func TestApplyGrokCacheIdentityWritesResponsesBodyAndHeader(t *testing.T) {
sourceBody := []byte(`{"model":"grok-4.5","prompt_cache_key":"raw-client-key"}`)
body, err := applyGrokResponsesCacheIdentity(sourceBody, sourceBody, "isolated-id", true)
require.NoError(t, err)
require.Equal(t, "isolated-id", gjson.GetBytes(body, "prompt_cache_key").String())
require.Equal(t, "web_search", gjson.GetBytes(body, "tools.0.type").String())
require.Equal(t, "x_search", gjson.GetBytes(body, "tools.1.type").String())
require.Equal(t, grokFreeCacheDisabledToolChoice, gjson.GetBytes(body, "tool_choice").String())
headers := make(http.Header)
headers.Set(grokConversationIDHeader, "spoofed-client-value")
applyGrokCacheHeaders(headers, "isolated-id")
require.Equal(t, "isolated-id", headers.Get(grokConversationIDHeader))
applyGrokCacheHeaders(headers, "")
require.Empty(t, headers.Get(grokConversationIDHeader))
chatBody, err := stripGrokChatPromptCacheKey(body)
require.NoError(t, err)
require.False(t, gjson.GetBytes(chatBody, "prompt_cache_key").Exists())
unscopedSourceBody := []byte(`{"model":"grok","prompt_cache_key":"raw-client-key"}`)
unscopedBody, err := applyGrokResponsesCacheIdentity(unscopedSourceBody, unscopedSourceBody, "", true)
require.NoError(t, err)
require.False(t, gjson.GetBytes(unscopedBody, "prompt_cache_key").Exists())
require.False(t, gjson.GetBytes(unscopedBody, "tools").Exists())
require.False(t, gjson.GetBytes(unscopedBody, "tool_choice").Exists())
}
func TestApplyGrokCacheIdentityPreservesExplicitClientToolFields(t *testing.T) {
tests := []struct {
name string
body string
}{
{
name: "tools only",
body: `{"model":"grok","tools":[{"type":"function","name":"lookup","parameters":{"type":"object"}}]}`,
},
{
name: "empty tools array",
body: `{"model":"grok","tools":[]}`,
},
{
name: "null tools",
body: `{"model":"grok","tools":null}`,
},
{
name: "tool choice only",
body: `{"model":"grok","tool_choice":{"type":"function","name":"lookup"}}`,
},
{
name: "null tool choice",
body: `{"model":"grok","tool_choice":null}`,
},
{
name: "both fields",
body: `{"model":"grok","tools":[{"type":"web_search"}],"tool_choice":"auto"}`,
},
{
name: "unsupported tool",
body: `{"model":"grok","tools":[{"type":"namespace","name":"client_tools"}]}`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
beforeTools := gjson.Get(tt.body, "tools")
beforeChoice := gjson.Get(tt.body, "tool_choice")
body, err := applyGrokResponsesCacheIdentity([]byte(tt.body), []byte(tt.body), "isolated-id", true)
require.NoError(t, err)
require.Equal(t, "isolated-id", gjson.GetBytes(body, "prompt_cache_key").String())
require.Equal(t, beforeTools.Exists(), gjson.GetBytes(body, "tools").Exists())
require.Equal(t, beforeTools.Raw, gjson.GetBytes(body, "tools").Raw)
require.Equal(t, beforeChoice.Exists(), gjson.GetBytes(body, "tool_choice").Exists())
require.Equal(t, beforeChoice.Raw, gjson.GetBytes(body, "tool_choice").Raw)
})
}
}
func TestApplyGrokCacheIdentityUsesPreSanitizationToolIntent(t *testing.T) {
tests := []struct {
name string
intentBody string
}{
{
name: "unsupported tools removed by sanitizer",
intentBody: `{"model":"grok","tools":[{"type":"namespace","name":"client_tools"}]}`,
},
{
name: "tool choice removed with unsupported tool",
intentBody: `{"model":"grok","tools":[{"type":"namespace","name":"client_tools"}],"tool_choice":{"type":"namespace","name":"client_tools"}}`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// This is the shape apply receives after patchGrokResponsesBody has
// removed unsupported tools and their associated tool_choice.
patchedBody := []byte(`{"model":"grok-4.5","input":"hello"}`)
body, err := applyGrokResponsesCacheIdentity(patchedBody, []byte(tt.intentBody), "isolated-id", true)
require.NoError(t, err)
require.Equal(t, "isolated-id", gjson.GetBytes(body, "prompt_cache_key").String())
require.False(t, gjson.GetBytes(body, "tools").Exists())
require.False(t, gjson.GetBytes(body, "tool_choice").Exists())
})
}
}
func TestApplyGrokCacheIdentityWithoutFreeTierRoutingOnlyWritesIdentity(t *testing.T) {
sourceBody := []byte(`{"model":"grok-4.5","input":"hello"}`)
body, err := applyGrokResponsesCacheIdentity(sourceBody, sourceBody, "isolated-id", false)
require.NoError(t, err)
require.Equal(t, "isolated-id", gjson.GetBytes(body, "prompt_cache_key").String())
require.False(t, gjson.GetBytes(body, "tools").Exists())
require.False(t, gjson.GetBytes(body, "tool_choice").Exists())
}
func TestGrokCompactRequestSkipsCacheIdentityAndNativeTools(t *testing.T) {
gin.SetMode(gin.TestMode)
c := newGrokCacheTestContext(701)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/responses/compact", nil)
body := []byte(`{"model":"grok","input":"compact this","prompt_cache_key":"raw-client-key"}`)
identity := resolveGrokCacheIdentity(c, body, "", "grok-4.5")
patched, err := applyGrokResponsesCacheIdentity(body, body, identity, true)
require.NoError(t, err)
require.Empty(t, identity)
require.False(t, gjson.GetBytes(patched, "prompt_cache_key").Exists())
require.False(t, gjson.GetBytes(patched, "tools").Exists())
require.False(t, gjson.GetBytes(patched, "tool_choice").Exists())
}
func TestResolveGrokCacheIdentityConcurrentDeterminism(t *testing.T) {
gin.SetMode(gin.TestMode)
const workers = 50
body := []byte(`{"model":"grok","messages":[{"role":"system","content":"stable"},{"role":"user","content":"hello"}]}`)
identities := make(chan string, workers)
var wg sync.WaitGroup
for i := 0; i < workers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
identities <- resolveGrokCacheIdentity(newGrokCacheTestContext(501), body, "", "grok-4.5")
}()
}
wg.Wait()
close(identities)
var first string
for identity := range identities {
if first == "" {
first = identity
}
require.Equal(t, first, identity)
}
require.NotEmpty(t, first)
}
@@ -0,0 +1,320 @@
package service
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"time"
"github.com/Wei-Shaw/sub2api/internal/pkg/apicompat"
"github.com/Wei-Shaw/sub2api/internal/pkg/xai"
"github.com/gin-gonic/gin"
)
const (
grokChatResponsesEndpoint = "/v1/responses"
grokChatRawEndpoint = "/v1/chat/completions"
)
var grokChatResponsesBridgeTopLevelFields = map[string]struct{}{
"model": {},
"messages": {},
"stream": {},
"stream_options": {},
"max_tokens": {},
"max_completion_tokens": {},
"temperature": {},
"top_p": {},
"prompt_cache_key": {},
"tools": {},
"tool_choice": {},
"functions": {},
"function_call": {},
}
// grokChatResponsesBridgeEligibility deliberately accepts only request shapes
// whose Chat Completions semantics are preserved by the Responses bridge.
// Everything else stays on raw Chat Completions rather than being silently
// dropped or rewritten.
func grokChatResponsesBridgeEligibility(body []byte) (bool, string) {
var root map[string]json.RawMessage
if err := json.Unmarshal(body, &root); err != nil || root == nil {
return false, "invalid_json"
}
for _, field := range []string{"stop", "reasoning_effort"} {
if _, exists := root[field]; exists {
return false, "unsupported_" + field
}
}
for _, field := range []string{"tools", "functions"} {
if raw, exists := root[field]; exists && !grokChatNullOrEmptyArray(raw) {
return false, "unsupported_" + field
}
}
if raw, exists := root["tool_choice"]; exists && !grokChatNullOrNone(raw) {
return false, "unsupported_tool_choice"
}
if raw, exists := root["function_call"]; exists && !grokChatNullOrNone(raw) {
return false, "unsupported_function_call"
}
for field := range root {
if _, supported := grokChatResponsesBridgeTopLevelFields[field]; !supported {
return false, "unknown_field_" + field
}
}
var model string
if raw, ok := root["model"]; !ok || json.Unmarshal(raw, &model) != nil || strings.TrimSpace(model) == "" {
return false, "invalid_model"
}
if raw, ok := root["stream"]; ok {
var stream *bool
if json.Unmarshal(raw, &stream) != nil || stream == nil {
return false, "invalid_stream"
}
}
if raw, ok := root["stream_options"]; ok {
var options map[string]json.RawMessage
if json.Unmarshal(raw, &options) != nil || options == nil {
return false, "invalid_stream_options"
}
for field, value := range options {
if field != "include_usage" {
return false, "unknown_stream_option_" + field
}
var includeUsage *bool
if json.Unmarshal(value, &includeUsage) != nil || includeUsage == nil {
return false, "invalid_stream_include_usage"
}
}
}
for _, field := range []string{"max_tokens", "max_completion_tokens"} {
if raw, ok := root[field]; ok {
var value *int
if json.Unmarshal(raw, &value) != nil || value == nil || *value < 128 {
return false, "unsafe_" + field
}
}
}
if _, hasMaxTokens := root["max_tokens"]; hasMaxTokens {
if _, hasMaxCompletionTokens := root["max_completion_tokens"]; hasMaxCompletionTokens {
return false, "conflicting_max_tokens"
}
}
for _, field := range []string{"temperature", "top_p"} {
if raw, ok := root[field]; ok {
var value *float64
if json.Unmarshal(raw, &value) != nil || value == nil {
return false, "invalid_" + field
}
}
}
if raw, ok := root["prompt_cache_key"]; ok {
var key string
if json.Unmarshal(raw, &key) != nil {
return false, "invalid_prompt_cache_key"
}
}
var messages []map[string]json.RawMessage
rawMessages, ok := root["messages"]
if !ok || json.Unmarshal(rawMessages, &messages) != nil || len(messages) == 0 {
return false, "invalid_messages"
}
for _, message := range messages {
for field := range message {
if field != "role" && field != "content" {
return false, "unsafe_message_field_" + field
}
}
var role string
if raw, exists := message["role"]; !exists || json.Unmarshal(raw, &role) != nil {
return false, "invalid_message_role"
}
switch role {
case "system", "user", "assistant":
default:
return false, "unsupported_message_role_" + role
}
var content string
if raw, exists := message["content"]; !exists || json.Unmarshal(raw, &content) != nil {
// Structured content includes image_url and other parts whose exact
// behavior is not guaranteed by this bridge.
return false, "non_text_message_content"
}
if strings.TrimSpace(content) == "" {
return false, "empty_message_content"
}
}
return true, ""
}
func grokChatNullOrEmptyArray(raw json.RawMessage) bool {
if strings.TrimSpace(string(raw)) == "null" {
return true
}
var values []json.RawMessage
return json.Unmarshal(raw, &values) == nil && len(values) == 0
}
func grokChatNullOrNone(raw json.RawMessage) bool {
if strings.TrimSpace(string(raw)) == "null" {
return true
}
var value string
return json.Unmarshal(raw, &value) == nil && strings.EqualFold(strings.TrimSpace(value), "none")
}
func grokChatCacheIntentBody(body []byte) ([]byte, error) {
var root map[string]json.RawMessage
if err := json.Unmarshal(body, &root); err != nil {
return nil, err
}
for _, field := range []string{"tools", "tool_choice", "functions", "function_call"} {
delete(root, field)
}
return json.Marshal(root)
}
func grokChatResponsesRuntimeEligible(upstreamModel, cacheIdentity string) bool {
return strings.TrimSpace(upstreamModel) == "grok-4.5" && strings.TrimSpace(cacheIdentity) != ""
}
// forwardGrokChatCompletionsViaResponses converts a strictly compatible Chat
// request into xAI Responses format and reuses the established Responses-to-
// Chat response translators. It intentionally does not run the Codex OAuth
// transform because Grok CLI is a separate upstream protocol.
func (s *OpenAIGatewayService) forwardGrokChatCompletionsViaResponses(
ctx context.Context,
c *gin.Context,
account *Account,
body []byte,
promptCacheKey string,
defaultMappedModel string,
) (*OpenAIForwardResult, error) {
startTime := time.Now()
var chatReq apicompat.ChatCompletionsRequest
if err := json.Unmarshal(body, &chatReq); err != nil {
return nil, fmt.Errorf("parse grok chat completions request: %w", err)
}
originalModel := chatReq.Model
clientStream := chatReq.Stream
billingModel := resolveOpenAIForwardModel(account, originalModel, defaultMappedModel)
upstreamModel := normalizeOpenAIModelForUpstream(account, billingModel)
cacheIdentity := resolveGrokCacheIdentity(c, body, promptCacheKey, upstreamModel)
if !grokChatResponsesRuntimeEligible(upstreamModel, cacheIdentity) {
return s.forwardAsRawChatCompletions(ctx, c, account, body, defaultMappedModel)
}
responsesReq, err := apicompat.ChatCompletionsToResponses(&chatReq)
if err != nil {
return nil, fmt.Errorf("convert grok chat completions to responses: %w", err)
}
responsesReq.Model = upstreamModel
responsesReq.Stream = true
// These fields are useful to Codex but are not needed by the Grok CLI
// protocol. Keep the bridge request as close as possible to native Grok.
responsesReq.Include = nil
responsesReq.Store = nil
responsesBody, err := json.Marshal(responsesReq)
if err != nil {
return nil, fmt.Errorf("marshal grok responses bridge request: %w", err)
}
responsesBody, err = patchGrokResponsesBody(responsesBody, upstreamModel)
if err != nil {
return nil, fmt.Errorf("patch grok responses bridge request: %w", err)
}
intentBody, err := grokChatCacheIntentBody(body)
if err != nil {
return nil, fmt.Errorf("normalize grok responses bridge tool intent: %w", err)
}
responsesBody, err = applyGrokResponsesCacheIdentity(responsesBody, intentBody, cacheIdentity, true)
if err != nil {
return nil, fmt.Errorf("apply grok responses bridge cache identity: %w", err)
}
updatedBody, policyErr := s.applyOpenAIFastPolicyToBody(ctx, account, upstreamModel, responsesBody)
if policyErr != nil {
var blocked *OpenAIFastBlockedError
if errors.As(policyErr, &blocked) {
MarkOpsClientBusinessLimited(c, OpsClientBusinessLimitedReasonLocalPolicyDenied)
writeChatCompletionsError(c, http.StatusForbidden, "permission_error", blocked.Message)
}
return nil, policyErr
}
responsesBody = updatedBody
token, _, err := s.GetAccessToken(ctx, account)
if err != nil {
return nil, fmt.Errorf("get grok access token: %w", err)
}
upstreamCtx, releaseUpstreamCtx := detachUpstreamContext(ctx)
upstreamReq, err := buildGrokResponsesRequest(upstreamCtx, c, account, responsesBody, token, cacheIdentity)
releaseUpstreamCtx()
if err != nil {
return nil, fmt.Errorf("build grok responses bridge request: %w", err)
}
SetActualOpenAIUpstreamEndpoint(c, grokChatResponsesEndpoint)
proxyURL := ""
if account.ProxyID != nil && account.Proxy != nil {
proxyURL = account.Proxy.URL()
}
resp, err := s.httpUpstream.Do(upstreamReq, proxyURL, account.ID, account.Concurrency)
if err != nil {
return nil, s.handleOpenAIUpstreamTransportError(ctx, c, account, err, false)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode >= http.StatusBadRequest {
respBody, upstreamMsg := s.readOpenAIUpstreamError(resp)
if upstreamMsg == "" {
upstreamMsg = fmt.Sprintf("xAI upstream returned status %d", resp.StatusCode)
}
appendOpsUpstreamError(c, OpsUpstreamErrorEvent{
Platform: account.Platform,
AccountID: account.ID,
AccountName: account.Name,
UpstreamStatusCode: resp.StatusCode,
UpstreamRequestID: firstNonEmpty(resp.Header.Get("x-request-id"), resp.Header.Get("xai-request-id")),
Kind: "failover",
Message: upstreamMsg,
})
s.handleGrokAccountUpstreamError(ctx, account, resp.StatusCode, resp.Header, respBody)
if s.shouldFailoverUpstreamError(resp.StatusCode) {
return nil, &UpstreamFailoverError{
StatusCode: resp.StatusCode,
ResponseBody: respBody,
RetryableOnSameAccount: account.IsPoolMode() && account.IsPoolModeRetryableStatus(resp.StatusCode),
}
}
return s.handleChatCompletionsErrorResponse(resp, c, account, billingModel)
}
s.updateGrokUsageSnapshot(ctx, account, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
var result *OpenAIForwardResult
if clientStream {
result, err = s.handleChatStreamingResponse(resp, c, account, originalModel, billingModel, upstreamModel, startTime, len(body))
} else {
result, err = s.handleChatBufferedStreamingResponse(resp, c, account, originalModel, billingModel, upstreamModel, startTime)
}
if result != nil {
result.UpstreamEndpoint = grokChatResponsesEndpoint
result.ResponseHeaders = resp.Header.Clone()
if result.RequestID == "" {
result.RequestID = firstNonEmpty(resp.Header.Get("x-request-id"), resp.Header.Get("xai-request-id"))
}
result.ReasoningEffort = extractOpenAIReasoningEffortFromBody(body, upstreamModel, billingModel, originalModel)
}
return result, err
}
@@ -0,0 +1,358 @@
//go:build unit
package service
import (
"bytes"
"context"
"errors"
"io"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
"time"
"github.com/Wei-Shaw/sub2api/internal/pkg/xai"
"github.com/gin-gonic/gin"
"github.com/stretchr/testify/require"
"github.com/tidwall/gjson"
)
func TestGrokChatResponsesBridgeEligibility(t *testing.T) {
t.Parallel()
tests := []struct {
name string
body string
want bool
reason string
}{
{
name: "plain text chat",
body: `{"model":"grok","messages":[{"role":"system","content":"concise"},{"role":"user","content":"hi"}],"stream":false}`,
want: true,
},
{
name: "safe generation options",
body: `{"model":"grok","messages":[{"role":"user","content":"hi"}],"stream":true,"stream_options":{"include_usage":true},"max_completion_tokens":256,"temperature":0.2,"top_p":0.9,"prompt_cache_key":"session","tools":[],"functions":null,"tool_choice":"none"}`,
want: true,
},
{
name: "stop falls back",
body: `{"model":"grok","messages":[{"role":"user","content":"hi"}],"stop":"done"}`,
reason: "unsupported_stop",
},
{
name: "developer role falls back",
body: `{"model":"grok","messages":[{"role":"developer","content":"rules"},{"role":"user","content":"hi"}]}`,
reason: "unsupported_message_role_developer",
},
{
name: "image content falls back",
body: `{"model":"grok","messages":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"data:image/png;base64,QQ=="}}]}]}`,
reason: "non_text_message_content",
},
{
name: "function tools fall back",
body: `{"model":"grok","messages":[{"role":"user","content":"hi"}],"tools":[{"type":"function","function":{"name":"lookup"}}]}`,
reason: "unsupported_tools",
},
{
name: "automatic tool choice falls back",
body: `{"model":"grok","messages":[{"role":"user","content":"hi"}],"tools":[],"tool_choice":"auto"}`,
reason: "unsupported_tool_choice",
},
{
name: "reasoning effort falls back because conversion adds summary",
body: `{"model":"grok","messages":[{"role":"user","content":"hi"}],"reasoning_effort":"high"}`,
reason: "unsupported_reasoning_effort",
},
{
name: "both token limits fall back",
body: `{"model":"grok","messages":[{"role":"user","content":"hi"}],"max_tokens":256,"max_completion_tokens":256}`,
reason: "conflicting_max_tokens",
},
{
name: "empty message falls back",
body: `{"model":"grok","messages":[{"role":"assistant","content":""},{"role":"user","content":"hi"}]}`,
reason: "empty_message_content",
},
{
name: "tool history falls back",
body: `{"model":"grok","messages":[{"role":"assistant","content":"","tool_calls":[]}]}`,
reason: "unsafe_message_field_tool_calls",
},
{
name: "unknown field falls back",
body: `{"model":"grok","messages":[{"role":"user","content":"hi"}],"seed":7}`,
reason: "unknown_field_seed",
},
{
name: "small max tokens falls back because conversion clamps it",
body: `{"model":"grok","messages":[{"role":"user","content":"hi"}],"max_tokens":32}`,
reason: "unsafe_max_tokens",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, reason := grokChatResponsesBridgeEligibility([]byte(tt.body))
require.Equal(t, tt.want, got)
require.Equal(t, tt.reason, reason)
})
}
}
func TestGrokChatResponsesRuntimeEligibility(t *testing.T) {
t.Parallel()
require.True(t, grokChatResponsesRuntimeEligible("grok-4.5", "isolated-id"))
require.False(t, grokChatResponsesRuntimeEligible("grok-4.3", "isolated-id"))
require.False(t, grokChatResponsesRuntimeEligible("grok-4.5-build-free", "isolated-id"))
require.False(t, grokChatResponsesRuntimeEligible("grok-4.5", ""))
}
func TestForwardGrokChatViaResponsesNonStreamingCachesAndReturnsChat(t *testing.T) {
gin.SetMode(gin.TestMode)
body := []byte(`{"model":"grok","messages":[{"role":"system","content":"be concise"},{"role":"user","content":"hi"}],"stream":false,"prompt_cache_key":"stable-session","tools":[],"functions":null,"tool_choice":"none"}`)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, grokChatRawEndpoint, bytes.NewReader(body))
c.Set("api_key", &APIKey{ID: 7101})
account := grokChatBridgeTestAccount(71)
repo := &grokQuotaAccountRepo{mockAccountRepoForPlatform: &mockAccountRepoForPlatform{
accountsByID: map[int64]*Account{account.ID: account},
}}
upstream := &httpUpstreamRecorder{resp: grokChatBridgeCompletedResponse("resp_grok_chat_cache", 9856)}
svc := &OpenAIGatewayService{
httpUpstream: upstream,
grokTokenProvider: NewGrokTokenProvider(repo, nil),
accountRepo: repo,
}
result, err := svc.ForwardAsChatCompletions(context.Background(), c, account, body, "", "")
require.NoError(t, err)
require.NotNil(t, result)
require.Equal(t, xai.DefaultCLIBaseURL+"/responses", upstream.lastReq.URL.String())
require.Equal(t, grokChatResponsesEndpoint, result.UpstreamEndpoint)
require.Equal(t, "grok-4.5", result.UpstreamModel)
require.Equal(t, 9908, result.Usage.InputTokens)
require.Equal(t, 12, result.Usage.OutputTokens)
require.Equal(t, 9856, result.Usage.CacheReadInputTokens)
identity := gjson.GetBytes(upstream.lastBody, "prompt_cache_key").String()
require.NotEmpty(t, identity)
require.NotEqual(t, "stable-session", identity)
require.Equal(t, identity, upstream.lastReq.Header.Get(grokConversationIDHeader))
require.Equal(t, "web_search", gjson.GetBytes(upstream.lastBody, "tools.0.type").String())
require.Equal(t, "x_search", gjson.GetBytes(upstream.lastBody, "tools.1.type").String())
require.Equal(t, grokFreeCacheDisabledToolChoice, gjson.GetBytes(upstream.lastBody, "tool_choice").String())
require.True(t, gjson.GetBytes(upstream.lastBody, "stream").Bool())
require.Equal(t, "system", gjson.GetBytes(upstream.lastBody, "input.0.role").String())
require.Equal(t, "user", gjson.GetBytes(upstream.lastBody, "input.1.role").String())
require.False(t, gjson.GetBytes(upstream.lastBody, "instructions").Exists())
require.False(t, gjson.GetBytes(upstream.lastBody, "include").Exists())
require.False(t, gjson.GetBytes(upstream.lastBody, "store").Exists())
require.Equal(t, http.StatusOK, recorder.Code)
require.Equal(t, "cached ok", gjson.Get(recorder.Body.String(), "choices.0.message.content").String())
require.Equal(t, int64(9856), gjson.Get(recorder.Body.String(), "usage.prompt_tokens_details.cached_tokens").Int())
require.NotNil(t, repo.updates[account.ID][grokQuotaSnapshotExtraKey])
}
func TestForwardGrokChatViaResponsesStreamingPropagatesCachedUsage(t *testing.T) {
gin.SetMode(gin.TestMode)
body := []byte(`{"model":"grok","messages":[{"role":"user","content":"hi"}],"stream":true}`)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, grokChatRawEndpoint, bytes.NewReader(body))
c.Set("api_key", &APIKey{ID: 7201})
account := grokChatBridgeTestAccount(72)
repo := &grokQuotaAccountRepo{mockAccountRepoForPlatform: &mockAccountRepoForPlatform{
accountsByID: map[int64]*Account{account.ID: account},
}}
upstream := &httpUpstreamRecorder{resp: grokChatBridgeCompletedResponse("resp_grok_chat_stream", 4096)}
svc := &OpenAIGatewayService{
httpUpstream: upstream,
grokTokenProvider: NewGrokTokenProvider(repo, nil),
accountRepo: repo,
}
result, err := svc.ForwardAsChatCompletions(context.Background(), c, account, body, "", "")
require.NoError(t, err)
require.NotNil(t, result)
require.True(t, result.Stream)
require.Equal(t, grokChatResponsesEndpoint, result.UpstreamEndpoint)
require.Equal(t, 4096, result.Usage.CacheReadInputTokens)
require.Contains(t, recorder.Header().Get("Content-Type"), "text/event-stream")
require.Contains(t, recorder.Body.String(), `"content":"cached ok"`)
require.Contains(t, recorder.Body.String(), `"cached_tokens":4096`)
require.Contains(t, recorder.Body.String(), "data: [DONE]")
}
func TestForwardGrokChatRuntimeGateFallsBackToRaw(t *testing.T) {
gin.SetMode(gin.TestMode)
tests := []struct {
name string
setAPIKey bool
mappedModel string
wantUpstream string
}{
{name: "missing cache identity", wantUpstream: "grok-4.5"},
{name: "non cache capable mapped model", setAPIKey: true, mappedModel: "grok-4.3", wantUpstream: "grok-4.3"},
}
for index, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
body := []byte(`{"model":"grok","messages":[{"role":"user","content":"hi"}],"stream":false}`)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, grokChatRawEndpoint, bytes.NewReader(body))
if tt.setAPIKey {
c.Set("api_key", &APIKey{ID: int64(7301 + index)})
}
account := grokChatBridgeTestAccount(int64(73 + index))
if tt.mappedModel != "" {
account.Credentials["model_mapping"] = map[string]any{"grok": tt.mappedModel}
}
repo := &grokQuotaAccountRepo{mockAccountRepoForPlatform: &mockAccountRepoForPlatform{
accountsByID: map[int64]*Account{account.ID: account},
}}
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(strings.NewReader(
`{"id":"chat_raw","object":"chat.completion","model":"` + tt.wantUpstream + `","choices":[{"index":0,"message":{"role":"assistant","content":"raw ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":2,"completion_tokens":1,"total_tokens":3}}`,
)),
}}
svc := &OpenAIGatewayService{
httpUpstream: upstream,
grokTokenProvider: NewGrokTokenProvider(repo, nil),
accountRepo: repo,
}
result, err := svc.ForwardAsChatCompletions(context.Background(), c, account, body, "", "")
require.NoError(t, err)
require.NotNil(t, result)
require.Equal(t, xai.DefaultCLIBaseURL+"/chat/completions", upstream.lastReq.URL.String())
require.Equal(t, grokChatRawEndpoint, result.UpstreamEndpoint)
require.Equal(t, tt.wantUpstream, result.UpstreamModel)
require.False(t, gjson.GetBytes(upstream.lastBody, "tools").Exists())
require.Equal(t, "raw ok", gjson.Get(recorder.Body.String(), "choices.0.message.content").String())
})
}
}
func TestForwardGrokChatViaResponses429UsesGrokRateLimitPolicy(t *testing.T) {
gin.SetMode(gin.TestMode)
body := []byte(`{"model":"grok","messages":[{"role":"user","content":"hi"}],"stream":false}`)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, grokChatRawEndpoint, bytes.NewReader(body))
c.Set("api_key", &APIKey{ID: 7501})
account := grokChatBridgeTestAccount(75)
repo := &grokQuotaAccountRepo{mockAccountRepoForPlatform: &mockAccountRepoForPlatform{
accountsByID: map[int64]*Account{account.ID: account},
}}
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusTooManyRequests,
Header: http.Header{
"Content-Type": []string{"application/json"},
"Retry-After": []string{"45"},
},
Body: io.NopCloser(strings.NewReader(`{"error":{"message":"rate limited"}}`)),
}}
svc := &OpenAIGatewayService{
httpUpstream: upstream,
grokTokenProvider: NewGrokTokenProvider(repo, nil),
accountRepo: repo,
}
before := time.Now()
result, err := svc.ForwardAsChatCompletions(context.Background(), c, account, body, "", "")
require.Error(t, err)
require.Nil(t, result)
var failoverErr *UpstreamFailoverError
require.True(t, errors.As(err, &failoverErr))
require.Equal(t, http.StatusTooManyRequests, failoverErr.StatusCode)
require.Equal(t, xai.DefaultCLIBaseURL+"/responses", upstream.lastReq.URL.String())
require.Equal(t, grokChatResponsesEndpoint, GetActualOpenAIUpstreamEndpoint(c))
require.Equal(t, 1, repo.rateLimitedCalls)
require.Zero(t, repo.tempUnschedCalls)
require.WithinDuration(t, before.Add(45*time.Second), repo.lastRateLimitResetAt, time.Second)
require.True(t, svc.isOpenAIAccountRuntimeBlocked(account))
}
func TestForwardGrokRawChatErrorRecordsActualEndpoint(t *testing.T) {
gin.SetMode(gin.TestMode)
body := []byte(`{"model":"grok","messages":[{"role":"user","content":"hi"}],"stream":false,"stop":"done"}`)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, grokChatRawEndpoint, bytes.NewReader(body))
c.Set("api_key", &APIKey{ID: 7601})
account := grokChatBridgeTestAccount(76)
repo := &grokQuotaAccountRepo{mockAccountRepoForPlatform: &mockAccountRepoForPlatform{
accountsByID: map[int64]*Account{account.ID: account},
}}
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusBadRequest,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(strings.NewReader(`{"error":{"message":"bad request"}}`)),
}}
svc := &OpenAIGatewayService{
httpUpstream: upstream,
grokTokenProvider: NewGrokTokenProvider(repo, nil),
accountRepo: repo,
}
result, err := svc.ForwardAsChatCompletions(context.Background(), c, account, body, "", "")
require.Error(t, err)
require.Nil(t, result)
require.Equal(t, xai.DefaultCLIBaseURL+"/chat/completions", upstream.lastReq.URL.String())
require.Equal(t, grokChatRawEndpoint, GetActualOpenAIUpstreamEndpoint(c))
}
func grokChatBridgeTestAccount(id int64) *Account {
return &Account{
ID: id,
Name: "grok-cache-bridge",
Platform: PlatformGrok,
Type: AccountTypeOAuth,
Concurrency: 1,
Credentials: map[string]any{
"access_token": "access-token",
"expires_at": time.Now().Add(time.Hour).UTC().Format(time.RFC3339),
"base_url": xai.DefaultCLIBaseURL,
},
}
}
func grokChatBridgeCompletedResponse(responseID string, cachedTokens int) *http.Response {
body := strings.Join([]string{
`data: {"type":"response.output_text.delta","sequence_number":0,"delta":"cached ok"}`,
"",
`data: {"type":"response.completed","sequence_number":1,"response":{"id":"` + responseID + `","object":"response","model":"grok-4.5","status":"completed","output":[{"type":"message","id":"msg_1","role":"assistant","status":"completed","content":[{"type":"output_text","text":"cached ok"}]}],"usage":{"input_tokens":9908,"output_tokens":12,"total_tokens":9920,"input_tokens_details":{"cached_tokens":` + strconv.Itoa(cachedTokens) + `}}}}`,
"",
}, "\n")
return &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{
"Content-Type": []string{"text/event-stream"},
"Xai-Request-Id": []string{responseID + "-request"},
"X-Ratelimit-Limit-Requests": []string{"10"},
"X-Ratelimit-Remaining-Requests": []string{"9"},
},
Body: io.NopCloser(strings.NewReader(body)),
}
}
@@ -6,6 +6,7 @@ import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"mime/multipart"
"net/http"
@@ -41,6 +42,50 @@ func TestPatchGrokResponsesBodySetsMappedModelAndDropsUnsupportedFields(t *testi
require.Equal(t, "high", gjson.GetBytes(patched, "reasoning.effort").String())
}
func TestPatchGrokResponsesBodySanitizesComposerReasoningParameters(t *testing.T) {
t.Parallel()
tests := []struct {
name string
upstreamModel string
wantReasoning bool
}{
{name: "composer fast", upstreamModel: "grok-composer-2.5-fast"},
{name: "composer shorthand", upstreamModel: "grok-composer"},
{name: "composer legacy alias", upstreamModel: "composer-2.5"},
{name: "provider-prefixed composer", upstreamModel: "xai/grok-composer-2.5-fast"},
{name: "grok 4.5", upstreamModel: "grok-4.5", wantReasoning: true},
}
body := []byte(`{
"model": "grok",
"input": "hello",
"reasoning": {"effort": "medium", "summary": "auto"},
"reasoning_effort": "medium",
"reasoningEffort": "medium"
}`)
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
patched, err := patchGrokResponsesBody(body, tt.upstreamModel)
require.NoError(t, err)
require.True(t, json.Valid(patched))
require.Equal(t, tt.upstreamModel, gjson.GetBytes(patched, "model").String())
if tt.wantReasoning {
require.Equal(t, "medium", gjson.GetBytes(patched, "reasoning.effort").String())
require.Equal(t, "medium", gjson.GetBytes(patched, "reasoning_effort").String())
require.Equal(t, "medium", gjson.GetBytes(patched, "reasoningEffort").String())
return
}
require.False(t, gjson.GetBytes(patched, "reasoning").Exists())
require.False(t, gjson.GetBytes(patched, "reasoning_effort").Exists())
require.False(t, gjson.GetBytes(patched, "reasoningEffort").Exists())
})
}
}
func TestExtractGrokResponsesReasoningEffortSupportsOpenAICompatibleField(t *testing.T) {
t.Parallel()
@@ -161,6 +206,45 @@ func TestPatchGrokResponsesBodyDropsToolChoiceWhenNoSupportedToolsRemain(t *test
require.False(t, gjson.GetBytes(patched, "tool_choice").Exists())
}
func TestPatchGrokResponsesBodyDropsCodexAdditionalToolsInputItems(t *testing.T) {
t.Parallel()
body := []byte(`{
"model": "grok",
"input": [
{
"type": "additional_tools",
"role": "developer",
"tools": [
{"type": "namespace", "name": "image_gen"},
{"type": "function", "name": "wait"}
]
},
{
"type": "message",
"role": "developer",
"content": [{"type": "input_text", "text": "system prompt"}]
},
{
"type": "message",
"role": "user",
"content": [{"type": "input_text", "text": "hello"}]
}
]
}`)
patched, err := patchGrokResponsesBody(body, "grok-4.5")
require.NoError(t, err)
require.True(t, json.Valid(patched))
require.Equal(t, "grok-4.5", gjson.GetBytes(patched, "model").String())
require.Equal(t, 2, len(gjson.GetBytes(patched, "input").Array()))
require.False(t, gjson.GetBytes(patched, `input.#(type=="additional_tools")`).Exists())
require.Equal(t, "developer", gjson.GetBytes(patched, "input.0.role").String())
require.Equal(t, "system prompt", gjson.GetBytes(patched, "input.0.content.0.text").String())
require.Equal(t, "user", gjson.GetBytes(patched, "input.1.role").String())
require.Equal(t, "hello", gjson.GetBytes(patched, "input.1.content.0.text").String())
}
func TestBuildGrokResponsesRequestUsesAccountBaseURLAndBearerToken(t *testing.T) {
t.Setenv(xai.EnvAllowUnsafeURLOverrides, "true")
@@ -172,13 +256,15 @@ func TestBuildGrokResponsesRequestUsesAccountBaseURLAndBearerToken(t *testing.T)
},
}
req, err := buildGrokResponsesRequest(context.Background(), nil, account, []byte(`{"model":"grok-4.3"}`), "access-token")
req, err := buildGrokResponsesRequest(context.Background(), nil, account, []byte(`{"model":"grok-4.3"}`), "access-token", "isolated-cache-id")
require.NoError(t, err)
require.Equal(t, http.MethodPost, req.Method)
require.Equal(t, "https://xai.test/v1/responses", req.URL.String())
require.Equal(t, "Bearer access-token", req.Header.Get("Authorization"))
require.Equal(t, "application/json", req.Header.Get("Content-Type"))
require.Contains(t, req.Header.Get("Accept"), "text/event-stream")
require.Equal(t, grokCLIVersion, req.Header.Get("X-Grok-Client-Version"))
require.Equal(t, "isolated-cache-id", req.Header.Get(grokConversationIDHeader))
data, err := io.ReadAll(req.Body)
require.NoError(t, err)
@@ -196,7 +282,7 @@ func TestBuildGrokResponsesRequestRejectsUnsafeAccountBaseURL(t *testing.T) {
},
}
_, err := buildGrokResponsesRequest(context.Background(), nil, account, []byte(`{"model":"grok-4.3"}`), "access-token")
_, err := buildGrokResponsesRequest(context.Background(), nil, account, []byte(`{"model":"grok-4.3"}`), "access-token", "")
require.Error(t, err)
require.Contains(t, err.Error(), "invalid base url")
}
@@ -324,6 +410,7 @@ func TestForwardGrokMediaImagesGenerationNormalizesImagineAlias(t *testing.T) {
require.Equal(t, http.MethodPost, upstream.lastReq.Method)
require.Equal(t, "Bearer api-key", upstream.lastReq.Header.Get("Authorization"))
require.Equal(t, "application/json", upstream.lastReq.Header.Get("Content-Type"))
require.Equal(t, grokCLIVersion, upstream.lastReq.Header.Get("X-Grok-Client-Version"))
require.JSONEq(t, `{"model":"grok-imagine-image-quality","prompt":"draw a cat"}`, string(upstream.lastBody))
require.Equal(t, http.StatusOK, recorder.Code)
require.JSONEq(t, `{"data":[]}`, recorder.Body.String())
@@ -565,7 +652,7 @@ func TestBindGrokMediaVideoRequestAccountUsesRequestIDStickyHash(t *testing.T) {
require.Equal(t, int64(63), accountID)
}
func TestForwardGrokMediaErrorHonorsCustomErrorCodes(t *testing.T) {
func TestForwardGrokMedia429ReconcilesRateLimitBeforeCustomErrorBypass(t *testing.T) {
t.Setenv(xai.EnvAllowUnsafeURLOverrides, "true")
gin.SetMode(gin.TestMode)
@@ -585,18 +672,20 @@ func TestForwardGrokMediaErrorHonorsCustomErrorCodes(t *testing.T) {
"api_key": "api-key",
"base_url": "https://xai.test/v1",
"custom_error_codes_enabled": true,
"custom_error_codes": []any{float64(http.StatusTooManyRequests)},
"custom_error_codes": []any{float64(http.StatusBadRequest)},
},
}
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusBadRequest,
StatusCode: http.StatusTooManyRequests,
Header: http.Header{
"Content-Type": []string{"application/json"},
"Xai-Request-Id": []string{"xai-error-req"},
"Retry-After": []string{"45"},
},
Body: io.NopCloser(strings.NewReader(`{"error":{"message":"do not expose this upstream detail"}}`)),
}}
svc := &OpenAIGatewayService{httpUpstream: upstream}
repo := &grokQuotaAccountRepo{}
svc := &OpenAIGatewayService{httpUpstream: upstream, accountRepo: repo}
result, err := svc.ForwardGrokMedia(context.Background(), c, account, GrokMediaEndpointImagesGenerations, "", body, "application/json")
require.Error(t, err)
@@ -604,15 +693,19 @@ func TestForwardGrokMediaErrorHonorsCustomErrorCodes(t *testing.T) {
require.Equal(t, http.StatusInternalServerError, recorder.Code)
require.Contains(t, recorder.Body.String(), "Upstream gateway error")
require.NotContains(t, recorder.Body.String(), "do not expose")
require.Equal(t, 1, repo.rateLimitedCalls)
require.Zero(t, repo.tempUnschedCalls)
require.True(t, svc.isOpenAIAccountRuntimeBlocked(account))
}
func TestForwardAsChatCompletionsForGrokUsesXAIChatCompletionsAndSnapshots(t *testing.T) {
func TestForwardAsChatCompletionsForGrokStopFallsBackToXAIChatCompletions(t *testing.T) {
gin.SetMode(gin.TestMode)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
body := []byte(`{"model":"grok","messages":[{"role":"user","content":"hi"}],"stream":false}`)
body := []byte(`{"model":"grok","messages":[{"role":"user","content":"hi"}],"stream":false,"stop":"done","prompt_cache_key":"raw-client-cache-key"}`)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/chat/completions", bytes.NewReader(body))
c.Set("api_key", &APIKey{ID: 5101})
account := &Account{
ID: 51,
@@ -641,7 +734,7 @@ func TestForwardAsChatCompletionsForGrokUsesXAIChatCompletionsAndSnapshots(t *te
"X-Ratelimit-Limit-Tokens": []string{"1000"},
"X-Ratelimit-Remaining-Tokens": []string{"990"},
},
Body: io.NopCloser(strings.NewReader(`{"id":"chatcmpl","object":"chat.completion","model":"grok-4.3","choices":[],"usage":{"prompt_tokens":1,"completion_tokens":2}}`)),
Body: io.NopCloser(strings.NewReader(`{"id":"chatcmpl","object":"chat.completion","model":"grok-4.3","choices":[],"usage":{"prompt_tokens":1,"completion_tokens":2,"prompt_tokens_details":{"cached_tokens":1}}}`)),
}}
svc := &OpenAIGatewayService{
httpUpstream: upstream,
@@ -653,11 +746,15 @@ func TestForwardAsChatCompletionsForGrokUsesXAIChatCompletionsAndSnapshots(t *te
require.NoError(t, err)
require.Equal(t, xai.DefaultCLIBaseURL+"/chat/completions", upstream.lastReq.URL.String())
require.Equal(t, "Bearer access-token", upstream.lastReq.Header.Get("Authorization"))
require.NotEmpty(t, upstream.lastReq.Header.Get(grokConversationIDHeader))
require.NotEqual(t, "raw-client-cache-key", upstream.lastReq.Header.Get(grokConversationIDHeader))
require.Equal(t, "grok-4.5", gjson.GetBytes(upstream.lastBody, "model").String())
require.False(t, gjson.GetBytes(upstream.lastBody, "prompt_cache_key").Exists())
require.Equal(t, "grok", result.Model)
require.Equal(t, "grok-4.5", result.UpstreamModel)
require.Equal(t, 1, result.Usage.InputTokens)
require.Equal(t, 2, result.Usage.OutputTokens)
require.Equal(t, 1, result.Usage.CacheReadInputTokens)
require.NotNil(t, repo.updates[51][grokQuotaSnapshotExtraKey])
require.Equal(t, http.StatusOK, recorder.Code)
}
@@ -671,6 +768,7 @@ func TestForwardGrokResponsesStreamingUsesXAIResponsesAndSnapshots(t *testing.T)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/responses", bytes.NewReader(body))
c.Request.Header.Set("Content-Type", "application/json")
c.Request.Header.Set("OpenAI-Beta", "responses=experimental")
c.Set("api_key", &APIKey{ID: 5201})
account := &Account{
ID: 52,
@@ -719,6 +817,11 @@ func TestForwardGrokResponsesStreamingUsesXAIResponsesAndSnapshots(t *testing.T)
require.Equal(t, "Bearer access-token", upstream.lastReq.Header.Get("Authorization"))
require.Equal(t, "responses=experimental", upstream.lastReq.Header.Get("OpenAI-Beta"))
require.Equal(t, "grok-4.5", gjson.GetBytes(upstream.lastBody, "model").String())
require.NotEmpty(t, gjson.GetBytes(upstream.lastBody, "prompt_cache_key").String())
require.Equal(t, gjson.GetBytes(upstream.lastBody, "prompt_cache_key").String(), upstream.lastReq.Header.Get(grokConversationIDHeader))
require.Equal(t, "web_search", gjson.GetBytes(upstream.lastBody, "tools.0.type").String())
require.Equal(t, "x_search", gjson.GetBytes(upstream.lastBody, "tools.1.type").String())
require.Equal(t, "none", gjson.GetBytes(upstream.lastBody, "tool_choice").String())
require.Equal(t, "high", gjson.GetBytes(upstream.lastBody, "reasoning_effort").String())
require.True(t, gjson.GetBytes(upstream.lastBody, "stream").Bool())
require.True(t, result.Stream)
@@ -734,6 +837,83 @@ func TestForwardGrokResponsesStreamingUsesXAIResponsesAndSnapshots(t *testing.T)
require.NotNil(t, repo.updates[52][grokQuotaSnapshotExtraKey])
}
func TestForwardGrokResponsesAPIKeyUsesXAIResponses(t *testing.T) {
gin.SetMode(gin.TestMode)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
body := []byte(`{"model":"grok","input":"hi","stream":true}`)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/responses", bytes.NewReader(body))
c.Request.Header.Set("Content-Type", "application/json")
account := &Account{
ID: 53,
Name: "grok-api-key",
Platform: PlatformGrok,
Type: AccountTypeAPIKey,
Concurrency: 2,
Credentials: map[string]any{
"api_key": "xai-test-key",
"base_url": "https://api.x.ai/v1",
},
}
upstreamBody := strings.Join([]string{
`data: {"type":"response.output_text.delta","sequence_number":0,"delta":"ok"}`,
"",
`data: {"type":"response.completed","sequence_number":1,"response":{"id":"resp_grok_api_key","model":"grok-4.5","usage":{"input_tokens":2,"output_tokens":1}}}`,
"",
}, "\n")
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"text/event-stream"}},
Body: io.NopCloser(strings.NewReader(upstreamBody)),
}}
svc := &OpenAIGatewayService{httpUpstream: upstream}
result, err := svc.forwardGrokResponses(context.Background(), c, account, body, "grok", true, time.Now())
require.NoError(t, err)
require.Equal(t, "https://api.x.ai/v1/responses", upstream.lastReq.URL.String())
require.Equal(t, "Bearer xai-test-key", upstream.lastReq.Header.Get("Authorization"))
require.Equal(t, "grok-4.5", gjson.GetBytes(upstream.lastBody, "model").String())
require.Equal(t, "resp_grok_api_key", result.ResponseID)
require.Equal(t, 2, result.Usage.InputTokens)
require.Equal(t, 1, result.Usage.OutputTokens)
}
func TestAccountTestServiceGrokAPIKeyUsesXAIResponses(t *testing.T) {
gin.SetMode(gin.TestMode)
account := &Account{
ID: 54,
Name: "grok-api-key",
Platform: PlatformGrok,
Type: AccountTypeAPIKey,
Concurrency: 2,
Credentials: map[string]any{
"api_key": "xai-test-key",
"base_url": "https://api.x.ai/v1",
},
}
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"text/event-stream"}},
Body: io.NopCloser(strings.NewReader(
"data: {\"type\":\"response.output_text.delta\",\"delta\":\"ok\"}\n\n" +
"data: {\"type\":\"response.completed\"}\n\n",
)),
}}
svc := &AccountTestService{httpUpstream: upstream}
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/api/v1/admin/accounts/54/test", nil)
err := svc.testGrokAccountConnection(c, account, "grok")
require.NoError(t, err)
require.Equal(t, "https://api.x.ai/v1/responses", upstream.lastReq.URL.String())
require.Equal(t, "Bearer xai-test-key", upstream.lastReq.Header.Get("Authorization"))
require.Contains(t, recorder.Body.String(), `"type":"test_complete"`)
}
func TestForwardAsChatCompletionsForGrokStreamingUsesRawXAIChatCompletions(t *testing.T) {
gin.SetMode(gin.TestMode)
@@ -801,6 +981,205 @@ func TestForwardAsChatCompletionsForGrokStreamingUsesRawXAIChatCompletions(t *te
require.NotNil(t, repo.updates[53][grokQuotaSnapshotExtraKey])
}
func TestForwardGrokResponsesNonStreamingUsesCacheIdentityAndCachedUsage(t *testing.T) {
gin.SetMode(gin.TestMode)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
body := []byte(`{"model":"grok","input":"hi","stream":false,"tools":[{"type":"namespace","name":"client_tools"}],"tool_choice":{"type":"namespace","name":"client_tools"}}`)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/responses", bytes.NewReader(body))
c.Request.Header.Set("Content-Type", "application/json")
c.Set("api_key", &APIKey{ID: 5202})
account := &Account{
ID: 56,
Name: "grok",
Platform: PlatformGrok,
Type: AccountTypeOAuth,
Concurrency: 1,
Credentials: map[string]any{
"access_token": "access-token",
"expires_at": time.Now().Add(time.Hour).UTC().Format(time.RFC3339),
"base_url": xai.DefaultCLIBaseURL,
},
}
repo := &grokQuotaAccountRepo{
mockAccountRepoForPlatform: &mockAccountRepoForPlatform{
accountsByID: map[int64]*Account{56: account},
},
}
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{
"Content-Type": []string{"application/json"},
"Xai-Request-Id": []string{"xai-non-stream-req"},
},
Body: io.NopCloser(strings.NewReader(`{"id":"resp_grok_non_stream","object":"response","model":"grok-4.3","status":"completed","output":[{"type":"message","role":"assistant","content":[{"type":"output_text","text":"ok"}]}],"usage":{"input_tokens":7,"output_tokens":2,"total_tokens":9,"input_tokens_details":{"cached_tokens":4}}}`)),
}}
svc := &OpenAIGatewayService{
httpUpstream: upstream,
grokTokenProvider: NewGrokTokenProvider(repo, nil),
accountRepo: repo,
}
result, err := svc.forwardGrokResponses(context.Background(), c, account, body, "grok", false, time.Now())
require.NoError(t, err)
require.NotNil(t, result)
require.False(t, result.Stream)
require.Equal(t, "resp_grok_non_stream", result.ResponseID)
require.Equal(t, 7, result.Usage.InputTokens)
require.Equal(t, 2, result.Usage.OutputTokens)
require.Equal(t, 4, result.Usage.CacheReadInputTokens)
require.Equal(t, "grok-4.5", gjson.GetBytes(upstream.lastBody, "model").String())
identity := gjson.GetBytes(upstream.lastBody, "prompt_cache_key").String()
require.NotEmpty(t, identity)
require.Equal(t, identity, upstream.lastReq.Header.Get(grokConversationIDHeader))
// The sanitizer drops this unsupported client tool, but its explicit intent
// must still prevent native cache-routing tools from being injected.
require.False(t, gjson.GetBytes(upstream.lastBody, "tools").Exists())
require.False(t, gjson.GetBytes(upstream.lastBody, "tool_choice").Exists())
require.Equal(t, "resp_grok_non_stream", gjson.Get(recorder.Body.String(), "id").String())
}
func TestForwardGrokResponsesFailoverKeepsCacheIdentityAcrossAccounts(t *testing.T) {
gin.SetMode(gin.TestMode)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
body := []byte(`{"model":"grok","input":[{"role":"user","content":"stable prefix"}],"stream":false}`)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/responses", bytes.NewReader(body))
c.Set("api_key", &APIKey{ID: 5203})
newAccount := func(id int64, token string) *Account {
return &Account{
ID: id,
Name: fmt.Sprintf("grok-%d", id),
Platform: PlatformGrok,
Type: AccountTypeOAuth,
Concurrency: 1,
Credentials: map[string]any{
"access_token": token,
"expires_at": time.Now().Add(time.Hour).UTC().Format(time.RFC3339),
"base_url": xai.DefaultCLIBaseURL,
},
}
}
firstAccount := newAccount(58, "access-token-a")
secondAccount := newAccount(59, "access-token-b")
repo := &grokQuotaAccountRepo{
mockAccountRepoForPlatform: &mockAccountRepoForPlatform{
accountsByID: map[int64]*Account{58: firstAccount, 59: secondAccount},
},
}
upstream := &httpUpstreamRecorder{responses: []*http.Response{
{
StatusCode: http.StatusServiceUnavailable,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(strings.NewReader(`{"error":{"message":"temporary"}}`)),
},
{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(strings.NewReader(`{"id":"resp_after_failover","object":"response","model":"grok-4.3","status":"completed","output":[],"usage":{"input_tokens":5,"output_tokens":1}}`)),
},
}}
svc := &OpenAIGatewayService{
httpUpstream: upstream,
grokTokenProvider: NewGrokTokenProvider(repo, nil),
accountRepo: repo,
}
_, err := svc.forwardGrokResponses(context.Background(), c, firstAccount, body, "grok", false, time.Now())
var failoverErr *UpstreamFailoverError
require.ErrorAs(t, err, &failoverErr)
result, err := svc.forwardGrokResponses(context.Background(), c, secondAccount, body, "grok", false, time.Now())
require.NoError(t, err)
require.NotNil(t, result)
require.Len(t, upstream.requests, 2)
require.Len(t, upstream.bodies, 2)
firstIdentity := gjson.GetBytes(upstream.bodies[0], "prompt_cache_key").String()
secondIdentity := gjson.GetBytes(upstream.bodies[1], "prompt_cache_key").String()
require.NotEmpty(t, firstIdentity)
require.Equal(t, firstIdentity, secondIdentity)
require.Equal(t, firstIdentity, upstream.requests[0].Header.Get(grokConversationIDHeader))
require.Equal(t, secondIdentity, upstream.requests[1].Header.Get(grokConversationIDHeader))
require.Equal(t, "Bearer access-token-a", upstream.requests[0].Header.Get("Authorization"))
require.Equal(t, "Bearer access-token-b", upstream.requests[1].Header.Get("Authorization"))
}
func TestForwardAsChatCompletionsForGrokStreamingStopFallsBackToRawXAIChatCompletions(t *testing.T) {
gin.SetMode(gin.TestMode)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
body := []byte(`{"model":"grok","messages":[{"role":"user","content":"hi"}],"stream":true,"stop":"done"}`)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/chat/completions", bytes.NewReader(body))
c.Request.Header.Set("Content-Type", "application/json")
c.Request.Header.Set(grokConversationIDHeader, "native-client-conversation")
c.Set("api_key", &APIKey{ID: 5301})
account := &Account{
ID: 53,
Name: "grok",
Platform: PlatformGrok,
Type: AccountTypeOAuth,
Concurrency: 1,
Credentials: map[string]any{
"access_token": "access-token",
"expires_at": time.Now().Add(time.Hour).UTC().Format(time.RFC3339),
"base_url": xai.DefaultCLIBaseURL,
},
}
repo := &grokQuotaAccountRepo{
mockAccountRepoForPlatform: &mockAccountRepoForPlatform{
accountsByID: map[int64]*Account{53: account},
},
}
upstreamBody := strings.Join([]string{
`data: {"id":"chatcmpl_grok","object":"chat.completion.chunk","model":"grok-4.3","choices":[{"index":0,"delta":{"content":"ok"}}]}`,
"",
`data: {"id":"chatcmpl_grok","object":"chat.completion.chunk","model":"grok-4.3","choices":[],"usage":{"prompt_tokens":6,"completion_tokens":4,"total_tokens":10,"prompt_tokens_details":{"cached_tokens":1}}}`,
"",
"data: [DONE]",
"",
}, "\n")
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{
"Content-Type": []string{"text/event-stream"},
"X-Request-Id": []string{"chat-stream-req"},
"X-Ratelimit-Limit-Requests": []string{"10"},
"X-Ratelimit-Remaining-Requests": []string{"7"},
},
Body: io.NopCloser(strings.NewReader(upstreamBody)),
}}
svc := &OpenAIGatewayService{
cfg: rawChatCompletionsTestConfig(),
httpUpstream: upstream,
grokTokenProvider: NewGrokTokenProvider(repo, nil),
accountRepo: repo,
}
result, err := svc.ForwardAsChatCompletions(context.Background(), c, account, body, "", "")
require.NoError(t, err)
require.Equal(t, xai.DefaultCLIBaseURL+"/chat/completions", upstream.lastReq.URL.String())
require.Equal(t, "Bearer access-token", upstream.lastReq.Header.Get("Authorization"))
require.Equal(t, "text/event-stream", upstream.lastReq.Header.Get("Accept"))
require.Equal(t, "sub2api-grok/1.0", upstream.lastReq.Header.Get("User-Agent"))
require.Equal(t, grokCLIVersion, upstream.lastReq.Header.Get("X-Grok-Client-Version"))
require.NotEmpty(t, upstream.lastReq.Header.Get(grokConversationIDHeader))
require.NotEqual(t, "native-client-conversation", upstream.lastReq.Header.Get(grokConversationIDHeader))
require.Equal(t, "grok-4.5", gjson.GetBytes(upstream.lastBody, "model").String())
require.True(t, gjson.GetBytes(upstream.lastBody, "stream_options.include_usage").Bool())
require.True(t, result.Stream)
require.Equal(t, 6, result.Usage.InputTokens)
require.Equal(t, 4, result.Usage.OutputTokens)
require.Equal(t, 1, result.Usage.CacheReadInputTokens)
require.Contains(t, recorder.Body.String(), "data: [DONE]")
require.NotNil(t, repo.updates[53][grokQuotaSnapshotExtraKey])
}
func TestForwardAsChatCompletionsForGrokComposerBridgesImageInput(t *testing.T) {
gin.SetMode(gin.TestMode)
@@ -809,6 +1188,7 @@ func TestForwardAsChatCompletionsForGrokComposerBridgesImageInput(t *testing.T)
body := []byte(`{"model":"grok-composer-2.5-fast","messages":[{"role":"system","content":"You are concise."},{"role":"user","content":[{"type":"text","text":"What is shown?"},{"type":"image_url","image_url":{"url":"data:image/png;base64,QUJD"}}]}],"stream":false}`)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/chat/completions", bytes.NewReader(body))
c.Request.Header.Set("Content-Type", "application/json")
c.Set("api_key", &APIKey{ID: 5501})
account := &Account{
ID: 55,
@@ -858,9 +1238,11 @@ func TestForwardAsChatCompletionsForGrokComposerBridgesImageInput(t *testing.T)
require.NotNil(t, result)
require.Len(t, upstream.requests, 2)
require.Equal(t, xai.DefaultCLIBaseURL+"/responses", upstream.requests[0].URL.String())
require.Empty(t, upstream.requests[0].Header.Get(grokConversationIDHeader))
require.Equal(t, "grok-build-0.1", gjson.GetBytes(upstream.bodies[0], "model").String())
require.Equal(t, "input_image", gjson.GetBytes(upstream.bodies[0], "input.0.content.1.type").String())
require.Equal(t, xai.DefaultCLIBaseURL+"/chat/completions", upstream.requests[1].URL.String())
require.NotEmpty(t, upstream.requests[1].Header.Get(grokConversationIDHeader))
require.Equal(t, "grok-composer-2.5-fast", gjson.GetBytes(upstream.bodies[1], "model").String())
require.False(t, strings.Contains(string(upstream.bodies[1]), "image_url"))
require.Contains(t, gjson.GetBytes(upstream.bodies[1], "messages.1.content").String(), "Image 1 description")
@@ -878,6 +1260,9 @@ func TestForwardAsAnthropicForGrokUsesXAIResponses(t *testing.T) {
c, _ := gin.CreateTestContext(recorder)
body := []byte(`{"model":"grok","max_tokens":32,"stream":false,"messages":[{"role":"user","content":"hi"}]}`)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/messages", bytes.NewReader(body))
c.Set("api_key", &APIKey{ID: 5401})
c.Request.Header.Set("OpenAI-Beta", "grok-experimental")
c.Request.Header.Set("originator", "opencode")
account := &Account{
ID: 54,
@@ -896,7 +1281,7 @@ func TestForwardAsAnthropicForGrokUsesXAIResponses(t *testing.T) {
accountsByID: map[int64]*Account{54: account},
},
}
upstream := &httpUpstreamRecorder{resp: openAICompatSSECompletedResponse("resp_grok_messages", "grok-4.3")}
upstream := &httpUpstreamRecorder{resp: grokMessagesSSECompletedResponse("resp_grok_messages", 3)}
svc := &OpenAIGatewayService{
httpUpstream: upstream,
grokTokenProvider: NewGrokTokenProvider(repo, nil),
@@ -908,18 +1293,88 @@ func TestForwardAsAnthropicForGrokUsesXAIResponses(t *testing.T) {
require.Equal(t, xai.DefaultCLIBaseURL+"/responses", upstream.lastReq.URL.String())
require.Equal(t, "Bearer access-token", upstream.lastReq.Header.Get("Authorization"))
require.Equal(t, "sub2api-grok/1.0", upstream.lastReq.Header.Get("User-Agent"))
require.Equal(t, grokCLIVersion, upstream.lastReq.Header.Get("X-Grok-Client-Version"))
require.Equal(t, "grok-experimental", upstream.lastReq.Header.Get("OpenAI-Beta"))
require.Empty(t, upstream.lastReq.Header.Get("originator"))
require.Empty(t, upstream.lastReq.Header.Get("version"))
require.Equal(t, "grok-4.5", gjson.GetBytes(upstream.lastBody, "model").String())
require.NotEmpty(t, gjson.GetBytes(upstream.lastBody, "prompt_cache_key").String())
require.Equal(t, gjson.GetBytes(upstream.lastBody, "prompt_cache_key").String(), upstream.lastReq.Header.Get(grokConversationIDHeader))
require.Equal(t, "web_search", gjson.GetBytes(upstream.lastBody, "tools.0.type").String())
require.Equal(t, "x_search", gjson.GetBytes(upstream.lastBody, "tools.1.type").String())
require.Equal(t, "none", gjson.GetBytes(upstream.lastBody, "tool_choice").String())
require.Empty(t, upstream.lastReq.Header.Get("session_id"))
require.True(t, gjson.GetBytes(upstream.lastBody, "stream").Bool())
require.NotContains(t, string(upstream.lastBody), "chatgpt.com")
require.Equal(t, "grok", result.Model)
require.Equal(t, "grok-4.5", result.UpstreamModel)
require.Equal(t, 5, result.Usage.InputTokens)
require.Equal(t, 2, result.Usage.OutputTokens)
require.Equal(t, 3, result.Usage.CacheReadInputTokens)
require.Contains(t, recorder.Body.String(), `"type":"message"`)
require.Equal(t, int64(3), gjson.Get(recorder.Body.String(), "usage.cache_read_input_tokens").Int())
require.Contains(t, recorder.Body.String(), "ok")
}
func TestHandleGrokAccountUpstreamErrorTempUnschedulesReadinessStates(t *testing.T) {
func TestForwardAsAnthropicForGrokStreamingPreservesCacheUsage(t *testing.T) {
gin.SetMode(gin.TestMode)
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
body := []byte(`{"model":"grok","max_tokens":32,"stream":true,"messages":[{"role":"user","content":"hi"}]}`)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/messages", bytes.NewReader(body))
c.Set("api_key", &APIKey{ID: 5402})
account := &Account{
ID: 57,
Name: "grok",
Platform: PlatformGrok,
Type: AccountTypeOAuth,
Concurrency: 1,
Credentials: map[string]any{
"access_token": "access-token",
"expires_at": time.Now().Add(time.Hour).UTC().Format(time.RFC3339),
"base_url": xai.DefaultCLIBaseURL,
},
}
repo := &grokQuotaAccountRepo{
mockAccountRepoForPlatform: &mockAccountRepoForPlatform{
accountsByID: map[int64]*Account{57: account},
},
}
upstream := &httpUpstreamRecorder{resp: grokMessagesSSECompletedResponse("resp_grok_messages_stream", 2)}
svc := &OpenAIGatewayService{
httpUpstream: upstream,
grokTokenProvider: NewGrokTokenProvider(repo, nil),
accountRepo: repo,
}
result, err := svc.ForwardAsAnthropic(context.Background(), c, account, body, "", "")
require.NoError(t, err)
require.NotNil(t, result)
require.Equal(t, 2, result.Usage.CacheReadInputTokens)
identity := gjson.GetBytes(upstream.lastBody, "prompt_cache_key").String()
require.NotEmpty(t, identity)
require.Equal(t, identity, upstream.lastReq.Header.Get(grokConversationIDHeader))
require.Contains(t, recorder.Header().Get("Content-Type"), "text/event-stream")
require.Contains(t, recorder.Body.String(), `"cache_read_input_tokens":2`)
}
func grokMessagesSSECompletedResponse(responseID string, cachedTokens int) *http.Response {
body := strings.Join([]string{
fmt.Sprintf(`data: {"type":"response.completed","response":{"id":%q,"object":"response","model":"grok-4.3","status":"completed","output":[{"type":"message","id":"msg_1","role":"assistant","status":"completed","content":[{"type":"output_text","text":"ok"}]}],"usage":{"input_tokens":5,"output_tokens":2,"total_tokens":7,"input_tokens_details":{"cached_tokens":%d}}}}`, responseID, cachedTokens),
"",
"data: [DONE]",
"",
}, "\n")
return &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"text/event-stream"}},
Body: io.NopCloser(strings.NewReader(body)),
}
}
func TestHandleGrokAccountUpstreamErrorTempUnschedulesNonRateLimitStates(t *testing.T) {
tests := []struct {
name string
status int
@@ -931,24 +1386,23 @@ func TestHandleGrokAccountUpstreamErrorTempUnschedulesReadinessStates(t *testing
{
name: "unauthorized reauth",
status: http.StatusUnauthorized,
wantReason: "grok oauth token unauthorized",
wantReason: "grok credentials unauthorized",
wantMinCooldown: 10*time.Minute - time.Second,
wantMaxCooldown: 10*time.Minute + time.Second,
},
{
name: "forbidden entitlement",
status: http.StatusForbidden,
wantReason: "grok entitlement or subscription tier denied",
wantReason: "grok access or entitlement denied",
wantMinCooldown: 30*time.Minute - time.Second,
wantMaxCooldown: 30*time.Minute + time.Second,
},
{
name: "rate limited retry after",
status: http.StatusTooManyRequests,
headers: http.Header{"Retry-After": []string{"45"}},
wantReason: "grok rate limited",
wantMinCooldown: 44 * time.Second,
wantMaxCooldown: 46 * time.Second,
name: "upstream temporary error",
status: http.StatusInternalServerError,
wantReason: "grok upstream temporary error",
wantMinCooldown: 2*time.Minute - time.Second,
wantMaxCooldown: 2*time.Minute + time.Second,
},
}
@@ -963,6 +1417,7 @@ func TestHandleGrokAccountUpstreamErrorTempUnschedulesReadinessStates(t *testing
require.True(t, svc.isOpenAIAccountRuntimeBlocked(account))
require.Equal(t, 1, repo.tempUnschedCalls)
require.Zero(t, repo.rateLimitedCalls)
require.Equal(t, account.ID, repo.lastTempUnschedID)
require.Equal(t, tt.wantReason, repo.lastTempUnschedReason)
require.True(t, repo.lastTempUnschedUntil.After(before.Add(tt.wantMinCooldown)))
@@ -971,10 +1426,83 @@ func TestHandleGrokAccountUpstreamErrorTempUnschedulesReadinessStates(t *testing
}
}
func TestHandleGrokAccountUpstreamErrorDoesNotShortenExistingPause(t *testing.T) {
func TestHandleGrokAccountUpstreamError429SetsRateLimitedFromRetryAfter(t *testing.T) {
account := &Account{ID: 61, Platform: PlatformGrok, Type: AccountTypeOAuth}
repo := &grokQuotaAccountRepo{}
svc := &OpenAIGatewayService{accountRepo: repo}
before := time.Now()
svc.handleGrokAccountUpstreamError(context.Background(), account, http.StatusTooManyRequests, http.Header{"Retry-After": []string{"45"}}, nil)
require.True(t, svc.isOpenAIAccountRuntimeBlocked(account))
require.Equal(t, 1, repo.rateLimitedCalls)
require.Equal(t, account.ID, repo.lastRateLimitedID)
require.WithinDuration(t, before.Add(45*time.Second), repo.lastRateLimitResetAt, time.Second)
require.Zero(t, repo.tempUnschedCalls)
}
func TestHandleGrokAccountUpstreamError429UsesLatestExhaustedWindowReset(t *testing.T) {
now := time.Now()
requestReset := now.Add(10 * time.Minute).Truncate(time.Second)
tokenReset := now.Add(20 * time.Minute).Truncate(time.Second)
headers := http.Header{
"X-Ratelimit-Limit-Requests": []string{"10"},
"X-Ratelimit-Remaining-Requests": []string{"0"},
"X-Ratelimit-Reset-Requests": []string{fmt.Sprintf("%d", requestReset.Unix())},
"X-Ratelimit-Limit-Tokens": []string{"1000"},
"X-Ratelimit-Remaining-Tokens": []string{"0"},
"X-Ratelimit-Reset-Tokens": []string{fmt.Sprintf("%d", tokenReset.Unix())},
}
account := &Account{ID: 62, Platform: PlatformGrok, Type: AccountTypeOAuth}
repo := &grokQuotaAccountRepo{}
svc := &OpenAIGatewayService{accountRepo: repo}
svc.handleGrokAccountUpstreamError(context.Background(), account, http.StatusTooManyRequests, headers, nil)
require.Equal(t, 1, repo.rateLimitedCalls)
require.WithinDuration(t, tokenReset, repo.lastRateLimitResetAt, time.Second)
require.Zero(t, repo.tempUnschedCalls)
}
func TestHandleGrokAccountUpstreamError429UsesFallbackReset(t *testing.T) {
account := &Account{ID: 63, Platform: PlatformGrok, Type: AccountTypeOAuth}
repo := &grokQuotaAccountRepo{}
svc := &OpenAIGatewayService{accountRepo: repo}
before := time.Now()
svc.handleGrokAccountUpstreamError(context.Background(), account, http.StatusTooManyRequests, nil, nil)
require.Equal(t, 1, repo.rateLimitedCalls)
require.WithinDuration(t, before.Add(grokRateLimitFallbackCooldown), repo.lastRateLimitResetAt, time.Second)
require.Zero(t, repo.tempUnschedCalls)
}
func TestGrokRateLimitResetAtUsesFutureWindowAfterRetryAfterExpires(t *testing.T) {
now := time.Now().UTC().Truncate(time.Second)
observedAt := now.Add(-2 * time.Minute)
windowReset := now.Add(15 * time.Minute)
retryAfter := 30
snapshot := &xai.QuotaSnapshot{
StatusCode: http.StatusTooManyRequests,
UpdatedAt: observedAt.Format(time.RFC3339),
RetryAfterSeconds: &retryAfter,
Requests: &xai.QuotaWindow{
Limit: grokInt64PtrForTest(10),
Remaining: grokInt64PtrForTest(0),
ResetUnix: grokInt64PtrForTest(windowReset.Unix()),
},
}
resetAt, limited := grokRateLimitResetAt(snapshot, now)
require.True(t, limited)
require.WithinDuration(t, windowReset, resetAt, time.Second)
}
func TestHandleGrokAccountUpstreamError429DoesNotShortenExistingPause(t *testing.T) {
existingUntil := time.Now().Add(15 * time.Minute)
account := &Account{
ID: 62,
ID: 64,
Platform: PlatformGrok,
Type: AccountTypeOAuth,
TempUnschedulableUntil: &existingUntil,
@@ -985,11 +1513,167 @@ func TestHandleGrokAccountUpstreamErrorDoesNotShortenExistingPause(t *testing.T)
svc.handleGrokAccountUpstreamError(context.Background(), account, http.StatusTooManyRequests, http.Header{"Retry-After": []string{"45"}}, nil)
require.Equal(t, 1, repo.tempUnschedCalls)
require.WithinDuration(t, existingUntil, repo.lastTempUnschedUntil, time.Second)
require.Equal(t, 1, repo.rateLimitedCalls)
require.WithinDuration(t, time.Now().Add(45*time.Second), repo.lastRateLimitResetAt, time.Second)
require.Zero(t, repo.tempUnschedCalls)
value, ok := svc.openaiAccountRuntimeBlockUntil.Load(account.ID)
require.True(t, ok)
runtimeUntil, ok := value.(time.Time)
require.True(t, ok)
require.WithinDuration(t, existingUntil, runtimeUntil, time.Second)
}
func TestUpdateGrokUsageSnapshotExhaustedSuccessBypassesThrottleAndSetsRateLimited(t *testing.T) {
account := &Account{ID: 65, Platform: PlatformGrok, Type: AccountTypeOAuth}
repo := &grokQuotaAccountRepo{}
svc := &OpenAIGatewayService{
accountRepo: repo,
codexSnapshotThrottle: newAccountWriteThrottle(time.Hour),
}
now := time.Now()
// Consume the normal snapshot write allowance first.
svc.updateGrokUsageSnapshot(context.Background(), account, &xai.QuotaSnapshot{
StatusCode: http.StatusOK,
Requests: &xai.QuotaWindow{
Limit: grokInt64PtrForTest(10),
Remaining: grokInt64PtrForTest(9),
},
UpdatedAt: now.UTC().Format(time.RFC3339),
})
resetAt := now.Add(30 * time.Minute).Truncate(time.Second)
svc.updateGrokUsageSnapshot(context.Background(), account, &xai.QuotaSnapshot{
StatusCode: http.StatusOK,
Requests: &xai.QuotaWindow{
Limit: grokInt64PtrForTest(10),
Remaining: grokInt64PtrForTest(0),
ResetUnix: grokInt64PtrForTest(resetAt.Unix()),
ResetAt: resetAt.UTC().Format(time.RFC3339),
},
UpdatedAt: now.UTC().Format(time.RFC3339),
})
require.Equal(t, 2, repo.updateCalls)
require.Equal(t, 1, repo.rateLimitedCalls)
require.Equal(t, account.ID, repo.lastRateLimitedID)
require.WithinDuration(t, resetAt, repo.lastRateLimitResetAt, time.Second)
require.True(t, svc.isOpenAIAccountRuntimeBlocked(account))
}
func TestUpdateGrokUsageSnapshotAvailableSuccessDoesNotSetRateLimited(t *testing.T) {
repo := &grokQuotaAccountRepo{}
svc := &OpenAIGatewayService{accountRepo: repo}
account := &Account{ID: 66, Platform: PlatformGrok, Type: AccountTypeOAuth}
svc.updateGrokUsageSnapshot(context.Background(), account, &xai.QuotaSnapshot{
StatusCode: http.StatusOK,
Requests: &xai.QuotaWindow{
Limit: grokInt64PtrForTest(10),
Remaining: grokInt64PtrForTest(1),
},
UpdatedAt: time.Now().UTC().Format(time.RFC3339),
})
require.Equal(t, 1, repo.updateCalls)
require.Zero(t, repo.rateLimitedCalls)
}
func TestUpdateGrokUsageSnapshotExhaustedSuccessWithoutResetUsesFallback(t *testing.T) {
repo := &grokQuotaAccountRepo{}
svc := &OpenAIGatewayService{accountRepo: repo}
account := &Account{ID: 67, Platform: PlatformGrok, Type: AccountTypeOAuth}
before := time.Now()
svc.updateGrokUsageSnapshot(context.Background(), account, &xai.QuotaSnapshot{
StatusCode: http.StatusOK,
Tokens: &xai.QuotaWindow{
Limit: grokInt64PtrForTest(2_000_000),
Remaining: grokInt64PtrForTest(0),
},
UpdatedAt: before.UTC().Format(time.RFC3339),
})
require.Equal(t, 1, repo.rateLimitedCalls)
require.WithinDuration(t, before.Add(grokRateLimitFallbackCooldown), repo.lastRateLimitResetAt, time.Second)
stored, ok := repo.updates[account.ID][grokQuotaSnapshotExtraKey].(*xai.QuotaSnapshot)
require.True(t, ok)
require.NotNil(t, stored.Tokens.ResetUnix)
paused, _ := shouldAutoPauseGrokQuotaWindow("tokens", stored.Tokens, before.Add(time.Second))
require.True(t, paused)
paused, _ = shouldAutoPauseGrokQuotaWindow("tokens", stored.Tokens, repo.lastRateLimitResetAt.Add(time.Second))
require.False(t, paused)
}
func TestOpenAIWSHTTPBridgeGrok429PersistsRateLimit(t *testing.T) {
repo := &grokQuotaAccountRepo{}
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusTooManyRequests,
Header: http.Header{"Retry-After": []string{"45"}},
Body: io.NopCloser(strings.NewReader(`{"error":{"message":"rate limited"}}`)),
}}
svc := &OpenAIGatewayService{accountRepo: repo, httpUpstream: upstream}
account := &Account{ID: 68, Platform: PlatformGrok, Type: AccountTypeOAuth, Concurrency: 1}
before := time.Now()
result, err := svc.proxyOpenAIWSHTTPBridgeTurn(
context.Background(), nil, account, "token",
[]byte(`{"type":"response.create","model":"grok-4.3","input":"hi"}`),
64, "grok-4.3", "", "", "", "cache-id", 1,
func([]byte) error { return nil },
)
require.Error(t, err)
require.Nil(t, result)
require.Equal(t, 1, repo.rateLimitedCalls)
require.WithinDuration(t, before.Add(45*time.Second), repo.lastRateLimitResetAt, time.Second)
require.Zero(t, repo.tempUnschedCalls)
require.True(t, svc.isOpenAIAccountRuntimeBlocked(account))
}
func TestOpenAIWSHTTPBridgeGrokExhaustedSuccessPersistsRateLimit(t *testing.T) {
repo := &grokQuotaAccountRepo{}
resetAt := time.Now().Add(20 * time.Minute).UTC().Truncate(time.Second)
resp := grokMessagesSSECompletedResponse("resp_ws_limited", 0)
resp.Header.Set("X-Ratelimit-Limit-Requests", "10")
resp.Header.Set("X-Ratelimit-Remaining-Requests", "0")
resp.Header.Set("X-Ratelimit-Reset-Requests", fmt.Sprintf("%d", resetAt.Unix()))
upstream := &httpUpstreamRecorder{resp: resp}
svc := &OpenAIGatewayService{accountRepo: repo, httpUpstream: upstream}
account := &Account{ID: 69, Platform: PlatformGrok, Type: AccountTypeOAuth, Concurrency: 1}
result, err := svc.proxyOpenAIWSHTTPBridgeTurn(
context.Background(), nil, account, "token",
[]byte(`{"type":"response.create","model":"grok-4.3","input":"hi"}`),
64, "grok-4.3", "", "", "", "cache-id", 1,
func([]byte) error { return nil },
)
require.NoError(t, err)
require.NotNil(t, result)
require.Equal(t, 1, repo.rateLimitedCalls)
require.WithinDuration(t, resetAt, repo.lastRateLimitResetAt, time.Second)
require.True(t, svc.isOpenAIAccountRuntimeBlocked(account))
}
func TestFailoverOpenAIUpstreamHTTPErrorUsesOnlyGrokRateLimitPolicy(t *testing.T) {
gin.SetMode(gin.TestMode)
repo := &grokQuotaAccountRepo{}
svc := &OpenAIGatewayService{accountRepo: repo}
account := &Account{ID: 70, Platform: PlatformGrok, Type: AccountTypeOAuth}
resp := &http.Response{
StatusCode: http.StatusTooManyRequests,
Header: http.Header{"Retry-After": []string{"45"}},
}
recorder := httptest.NewRecorder()
c, _ := gin.CreateTestContext(recorder)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/messages", nil)
failoverErr := svc.failoverOpenAIUpstreamHTTPError(
context.Background(), c, account, resp,
[]byte(`{"error":{"message":"rate limited"}}`), "rate limited", "grok-4.3",
)
require.NotNil(t, failoverErr)
require.Equal(t, 1, repo.rateLimitedCalls)
require.Zero(t, repo.tempUnschedCalls)
}
@@ -244,12 +244,17 @@ func (s *OpenAIGatewayService) ForwardAsAnthropic(
return nil, policyErr
}
responsesBody = updatedBody
grokCacheIdentity := ""
if account.Platform == PlatformGrok {
grokCacheIdentity = resolveGrokCacheIdentity(c, responsesBody, promptCacheKey, upstreamModel)
patchedBody, patchErr := patchGrokResponsesBody(responsesBody, upstreamModel)
if patchErr != nil {
return nil, patchErr
}
responsesBody = patchedBody
responsesBody, patchErr = applyGrokResponsesCacheIdentity(patchedBody, responsesBody, grokCacheIdentity, account.IsGrokOAuth())
if patchErr != nil {
return nil, fmt.Errorf("apply grok prompt cache identity: %w", patchErr)
}
}
// 5. Get access token
@@ -261,15 +266,14 @@ func (s *OpenAIGatewayService) ForwardAsAnthropic(
// 6. Build upstream request
if account.Type == AccountTypeOAuth && account.Platform != PlatformGrok {
// Messages 兼容桥即使 body 未带 todo-guard/prompt_cache_key 标记(如映射到非
// gpt-5/codex 模型),也必须让 buildUpstreamRequest 走 bridge 分支:不带
// originator、User-Agent 逐字透传,避免身份收口(issue #3901)误改本路径
// 刻意最小化的请求形态(下方的 Del(OpenAI-Beta/originator) 兜底保持不变)。
// gpt-5/codex 模型),也必须让 buildUpstreamRequest 走 bridge 分支,以保留
// 既有 body/session/conversation 行为。身份头在 post-build 阶段统一恢复。
setOpenAICompatMessagesBridgeContext(c, true)
}
upstreamCtx, releaseUpstreamCtx := detachUpstreamContext(ctx)
var upstreamReq *http.Request
if account.Platform == PlatformGrok {
upstreamReq, err = buildGrokResponsesRequest(upstreamCtx, c, account, responsesBody, token)
upstreamReq, err = buildGrokResponsesRequest(upstreamCtx, c, account, responsesBody, token, grokCacheIdentity)
} else {
upstreamReq, err = s.buildUpstreamRequest(upstreamCtx, c, account, responsesBody, token, isStream, promptCacheKey, false)
}
@@ -280,7 +284,7 @@ func (s *OpenAIGatewayService) ForwardAsAnthropic(
// Override session_id with a deterministic UUID derived from the isolated
// session key, ensuring different API keys produce different upstream sessions.
if promptCacheKey != "" {
if account.Platform != PlatformGrok && promptCacheKey != "" {
isolatedSessionID := generateSessionUUID(isolateOpenAISessionID(apiKeyID, promptCacheKey))
upstreamReq.Header.Set("session_id", isolatedSessionID)
if upstreamReq.Header.Get("conversation_id") != "" {
@@ -288,12 +292,16 @@ func (s *OpenAIGatewayService) ForwardAsAnthropic(
}
}
if account.Type == AccountTypeOAuth && account.Platform != PlatformGrok {
// Anthropic Messages compatibility uses the ChatGPT Codex SSE endpoint.
// Match airgate-openai's request shape: the SSE endpoint does not need
// the Responses experimental beta header, and forcing originator can make
// ChatGPT select a different internal continuation path.
upstreamReq.Header.Del("OpenAI-Beta")
upstreamReq.Header.Del("originator")
// buildUpstreamRequest 保留 Messages bridge 的 body/session 兼容行为,并会先
// 清除身份头。真正发送前恢复完整 Codex 身份,避免 ChatGPT Codex 上游因缺失
// originator/OpenAI-Beta 返回 404(issue #3901)。
ensureCodexIdentityHeaders(upstreamReq.Header)
enforceCodexIdentityHeaders(upstreamReq.Header)
logger.L().Debug("openai messages: upstream identity restored",
zap.Int64("account_id", account.ID),
zap.String("upstream_model", upstreamModel),
zap.Bool("compat_identity_restored", true),
)
}
if account.Type == AccountTypeOAuth && promptCacheKey != "" && strings.TrimSpace(c.GetHeader("conversation_id")) == "" {
upstreamReq.Header.Del("conversation_id")
@@ -316,11 +324,6 @@ func (s *OpenAIGatewayService) ForwardAsAnthropic(
// 8. Handle error response with failover
if resp.StatusCode >= 400 {
respBody, upstreamMsg := s.readOpenAIUpstreamError(resp)
if account.Platform == PlatformGrok {
s.updateGrokUsageSnapshot(ctx, account.ID, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
s.handleGrokAccountUpstreamError(ctx, account, resp.StatusCode, resp.Header, respBody)
}
if previousResponseID != "" && (isOpenAICompatPreviousResponseNotFound(resp.StatusCode, upstreamMsg, respBody) || isOpenAICompatPreviousResponseUnsupported(resp.StatusCode, upstreamMsg, respBody)) {
if isOpenAICompatPreviousResponseUnsupported(resp.StatusCode, upstreamMsg, respBody) {
s.disableOpenAICompatSessionContinuation(ctx, c, account, promptCacheKey)
@@ -340,6 +343,9 @@ func (s *OpenAIGatewayService) ForwardAsAnthropic(
// Non-failover error: return Anthropic-formatted error to client
return s.handleAnthropicErrorResponse(resp, c, account, billingModel)
}
if account.Platform == PlatformGrok && account.Type == AccountTypeOAuth && !account.IsShadow() {
s.updateGrokUsageSnapshot(ctx, account, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
}
if account.Type == AccountTypeOAuth && promptCacheKey != "" {
if turnState := strings.TrimSpace(resp.Header.Get("x-codex-turn-state")); turnState != "" {
@@ -387,10 +393,8 @@ func (s *OpenAIGatewayService) ForwardAsAnthropic(
// Extract and save Codex usage snapshot from response headers (for OAuth accounts).
// 排除 spark 影子:其 codex_* 仅由 QueryUsage(/wham/usage bengalfox)更新(外审第7轮 P1)。
if handleErr == nil && account.Type == AccountTypeOAuth && !account.IsShadow() {
if account.Platform == PlatformGrok {
s.updateGrokUsageSnapshot(ctx, account.ID, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
} else if snapshot := ParseCodexRateLimitHeaders(resp.Header); snapshot != nil {
if handleErr == nil && account.Type == AccountTypeOAuth && !account.IsShadow() && account.Platform != PlatformGrok {
if snapshot := ParseCodexRateLimitHeaders(resp.Header); snapshot != nil {
s.updateCodexUsageSnapshot(ctx, account.ID, snapshot)
}
}
@@ -101,7 +101,7 @@ func (s *OpenAIGatewayService) forwardAnthropicViaRawChatCompletions(
if err != nil {
return nil, err
}
resp, err := s.sendCCUpstreamRequest(ctx, c, account, targetURL, chatBody, clientStream, apiKey, account.GetOpenAIUserAgent())
resp, err := s.sendCCUpstreamRequest(ctx, c, account, targetURL, chatBody, clientStream, apiKey, account.GetOpenAIUserAgent(), "")
if err != nil {
return nil, err
}
@@ -140,7 +140,7 @@ func (s *OpenAIGatewayService) bufferChatCompletionsAsAnthropic(
if err != nil {
return nil, err
}
responsesResp := apicompat.ChatCompletionsResponseToResponses(ccResp, originalModel)
responsesResp := apicompat.ChatCompletionsResponseToResponses(ccResp, originalModel, nil, false, nil)
anthropicResp := apicompat.ResponsesToAnthropic(responsesResp, originalModel)
@@ -356,6 +356,8 @@ func TestForwardAsAnthropic_ResponsesSupportedAccountStillUsesResponsesEndpoint(
c, _ := gin.CreateTestContext(rec)
c.Request = httptest.NewRequest(http.MethodPost, "/v1/messages", bytes.NewReader(body))
c.Request.Header.Set("Content-Type", "application/json")
c.Request.Header.Set("User-Agent", "third-party-client/1.0.0")
c.Request.Header.Set("originator", "opencode")
upstreamBody := strings.Join([]string{
`data: {"type":"response.completed","response":{"id":"resp_native","object":"response","model":"gpt-5.4","status":"completed","output":[{"type":"message","id":"msg_1","role":"assistant","status":"completed","content":[{"type":"output_text","text":"ok"}]}],"usage":{"input_tokens":5,"output_tokens":2,"total_tokens":7}}}`,
@@ -385,5 +387,9 @@ func TestForwardAsAnthropic_ResponsesSupportedAccountStillUsesResponsesEndpoint(
"responses-capable account must stay on /v1/responses, got %s", upstream.lastReq.URL.String())
require.True(t, gjson.GetBytes(upstream.lastBody, "input").Exists())
require.False(t, gjson.GetBytes(upstream.lastBody, "messages").Exists())
require.Equal(t, "third-party-client/1.0.0", upstream.lastReq.Header.Get("User-Agent"))
require.Equal(t, "opencode", upstream.lastReq.Header.Get("originator"))
require.Empty(t, upstream.lastReq.Header.Get("version"))
require.Empty(t, upstream.lastReq.Header.Get("OpenAI-Beta"))
require.Equal(t, "ok", gjson.Get(rec.Body.String(), "content.0.text").String())
}
@@ -39,6 +39,13 @@ func (s *OpenAIGatewayService) forwardResponsesViaRawChatCompletions(
clientStream := responsesReq.Stream
serviceTier := extractOpenAIServiceTierFromBody(body)
// custom 工具(如 codex 的 exec)降级为 function 工具转发,回程需按名字还原为
// custom_tool_call 项,先记下名字集合;tool_search 工具同理,回程还原为
// tool_search_call 项;namespace 子工具(如 MCP 工具)摊平转发,回程按映射还原
// 为带 namespace 字段的 function_call 项。
customTools := apicompat.CustomToolNames(responsesReq.Tools)
toolSearch := apicompat.HasToolSearchTool(responsesReq.Tools)
namespaceTools := apicompat.NamespaceToolNames(responsesReq.Tools)
chatReq, err := apicompat.ResponsesToChatCompletionsRequest(&responsesReq)
if err != nil {
@@ -85,7 +92,7 @@ func (s *OpenAIGatewayService) forwardResponsesViaRawChatCompletions(
if err != nil {
return nil, err
}
resp, err := s.sendCCUpstreamRequest(ctx, c, account, targetURL, chatBody, clientStream, apiKey, account.GetOpenAIUserAgent())
resp, err := s.sendCCUpstreamRequest(ctx, c, account, targetURL, chatBody, clientStream, apiKey, account.GetOpenAIUserAgent(), "")
if err != nil {
return nil, err
}
@@ -100,15 +107,18 @@ func (s *OpenAIGatewayService) forwardResponsesViaRawChatCompletions(
}
if clientStream {
return s.streamChatCompletionsAsResponses(c, resp, originalModel, billingModel, upstreamModel, reasoningEffort, serviceTier, startTime)
return s.streamChatCompletionsAsResponses(c, resp, originalModel, customTools, toolSearch, namespaceTools, billingModel, upstreamModel, reasoningEffort, serviceTier, startTime)
}
return s.bufferChatCompletionsAsResponses(c, resp, originalModel, billingModel, upstreamModel, reasoningEffort, serviceTier, startTime)
return s.bufferChatCompletionsAsResponses(c, resp, originalModel, customTools, toolSearch, namespaceTools, billingModel, upstreamModel, reasoningEffort, serviceTier, startTime)
}
func (s *OpenAIGatewayService) bufferChatCompletionsAsResponses(
c *gin.Context,
resp *http.Response,
originalModel string,
customTools map[string]bool,
toolSearch bool,
namespaceTools map[string]apicompat.NamespacedToolName,
billingModel string,
upstreamModel string,
reasoningEffort *string,
@@ -120,7 +130,7 @@ func (s *OpenAIGatewayService) bufferChatCompletionsAsResponses(
if err != nil {
return nil, err
}
responsesResp := apicompat.ChatCompletionsResponseToResponses(ccResp, originalModel)
responsesResp := apicompat.ChatCompletionsResponseToResponses(ccResp, originalModel, customTools, toolSearch, namespaceTools)
if s.responseHeaderFilter != nil {
responseheaders.WriteFilteredHeaders(c.Writer.Header(), resp.Header, s.responseHeaderFilter)
@@ -144,6 +154,9 @@ func (s *OpenAIGatewayService) streamChatCompletionsAsResponses(
c *gin.Context,
resp *http.Response,
originalModel string,
customTools map[string]bool,
toolSearch bool,
namespaceTools map[string]apicompat.NamespacedToolName,
billingModel string,
upstreamModel string,
reasoningEffort *string,
@@ -154,6 +167,9 @@ func (s *OpenAIGatewayService) streamChatCompletionsAsResponses(
writeStreamHeaders := s.newStreamHeaderWriter(c, resp.Header)
state := apicompat.NewChatCompletionsToResponsesStreamState(originalModel)
state.CustomTools = customTools
state.ToolSearchDeclared = toolSearch
state.NamespaceTools = namespaceTools
clientDisconnected := false
writeEvents := func(events []apicompat.ResponsesStreamEvent) {
@@ -23,17 +23,7 @@ import (
// ExtractSessionID extracts the raw session ID from headers or body without hashing.
// Used by ForwardAsAnthropic to pass as prompt_cache_key for upstream cache.
func (s *OpenAIGatewayService) ExtractSessionID(c *gin.Context, body []byte) string {
if c == nil {
return ""
}
sessionID := strings.TrimSpace(c.GetHeader("session_id"))
if sessionID == "" {
sessionID = strings.TrimSpace(c.GetHeader("conversation_id"))
}
if sessionID == "" && len(body) > 0 {
sessionID = strings.TrimSpace(gjson.GetBytes(body, "prompt_cache_key").String())
}
return sessionID
return explicitOpenAIRequestSessionID(c, body)
}
func explicitOpenAISessionID(c *gin.Context, body []byte) string {
@@ -51,11 +41,33 @@ func explicitOpenAISessionID(c *gin.Context, body []byte) string {
return sessionID
}
// explicitOpenAIRequestSessionID extends the common OpenAI session signals
// with Grok's native conversation header only for requests authenticated to a
// Grok group. This keeps an unrelated x-grok-conv-id header from changing
// scheduling or upstream session behavior for non-Grok groups.
func explicitOpenAIRequestSessionID(c *gin.Context, body []byte) string {
if c == nil {
return ""
}
sessionID := strings.TrimSpace(c.GetHeader("session_id"))
if sessionID == "" {
sessionID = strings.TrimSpace(c.GetHeader("conversation_id"))
}
if sessionID == "" && isGrokRequestContext(c) {
sessionID = strings.TrimSpace(c.GetHeader(grokConversationIDHeader))
}
if sessionID == "" && len(body) > 0 {
sessionID = strings.TrimSpace(gjson.GetBytes(body, "prompt_cache_key").String())
}
return sessionID
}
// GenerateExplicitSessionHash generates a sticky-session hash only from explicit
// client session signals. It intentionally skips content-derived fallback and is
// used by stateless endpoints such as /v1/images.
func (s *OpenAIGatewayService) GenerateExplicitSessionHash(c *gin.Context, body []byte) string {
sessionID := explicitOpenAISessionID(c, body)
sessionID := explicitOpenAIRequestSessionID(c, body)
if sessionID == "" {
return ""
}
@@ -70,14 +82,15 @@ func (s *OpenAIGatewayService) GenerateExplicitSessionHash(c *gin.Context, body
// Priority:
// 1. Header: session_id
// 2. Header: conversation_id
// 3. Body: prompt_cache_key (opencode)
// 4. Body: content-based fallback (model + system + tools + first user message)
// 3. Header: x-grok-conv-id (Grok groups only)
// 4. Body: prompt_cache_key (opencode)
// 5. Body: content-based fallback (model + system + tools + first user message)
func (s *OpenAIGatewayService) GenerateSessionHash(c *gin.Context, body []byte) string {
if c == nil {
return ""
}
sessionID := explicitOpenAISessionID(c, body)
sessionID := explicitOpenAIRequestSessionID(c, body)
if sessionID == "" && len(body) > 0 {
sessionID = deriveOpenAIContentSessionSeed(body)
}
@@ -49,6 +49,7 @@ const (
openAIWSRetryBackoffMaxDefault = 2 * time.Second
openAIWSRetryJitterRatioDefault = 0.2
openAICompactSessionSeedKey = "openai_compact_session_seed"
openAIUpstreamEndpointContextKey = "openai_actual_upstream_endpoint"
codexCLIVersion = "0.144.1"
// Codex 限额快照仅用于后台展示/诊断,不需要每个成功请求都立即落库。
openAICodexSnapshotPersistMinInterval = 30 * time.Second
@@ -66,6 +67,7 @@ var openaiAllowedHeaders = map[string]bool{
"user-agent": true,
"originator": true,
"session_id": true,
"x-codex-beta-features": true,
"x-codex-turn-state": true,
"x-codex-turn-metadata": true,
}
@@ -81,6 +83,7 @@ var openaiPassthroughAllowedHeaders = map[string]bool{
"user-agent": true,
"originator": true,
"session_id": true,
"x-codex-beta-features": true,
"x-codex-turn-state": true,
"x-codex-turn-metadata": true,
}
@@ -223,6 +226,9 @@ type OpenAIForwardResult struct {
// UpstreamModel is the actual model sent to the upstream provider after mapping.
// Empty when no mapping was applied (requested model was used as-is).
UpstreamModel string
// UpstreamEndpoint is the actual upstream API path used for this request.
// It avoids guessing when one downstream protocol can use multiple upstream endpoints.
UpstreamEndpoint string
// ServiceTier records the OpenAI Responses API service tier, e.g. "priority" / "flex".
// Nil means the request did not specify a recognized tier.
ServiceTier *string
@@ -246,11 +252,40 @@ type OpenAIForwardResult struct {
VideoResolution string
// VideoDurationSeconds 是提交时请求的生成时长(xAI 按输出秒数计费),已归一化到 1-15 秒。
VideoDurationSeconds int
// WebSearchCalls 是 Codex alpha/search 网页搜索调用次数(每次成功请求为 1)。
// 上游不返回 usage 字段,>0 时走按次计费(分组单价 × 次数 × 倍率)。
WebSearchCalls int
wsReplayInput []json.RawMessage
wsReplayInputExists bool
}
// SetActualOpenAIUpstreamEndpoint records the endpoint selected by the current
// forwarding attempt. It covers error paths where no OpenAIForwardResult is
// available for usage and operations logging.
func SetActualOpenAIUpstreamEndpoint(c *gin.Context, endpoint string) {
if c == nil {
return
}
if endpoint = strings.TrimSpace(endpoint); endpoint != "" {
c.Set(openAIUpstreamEndpointContextKey, endpoint)
}
}
// GetActualOpenAIUpstreamEndpoint returns the endpoint recorded by the latest
// forwarding attempt in this request.
func GetActualOpenAIUpstreamEndpoint(c *gin.Context) string {
if c == nil {
return ""
}
value, exists := c.Get(openAIUpstreamEndpointContextKey)
if !exists {
return ""
}
endpoint, _ := value.(string)
return strings.TrimSpace(endpoint)
}
type OpenAIWSRetryMetricsSnapshot struct {
RetryAttemptsTotal int64 `json:"retry_attempts_total"`
RetryBackoffMsTotal int64 `json:"retry_backoff_ms_total"`
@@ -178,7 +178,7 @@ func (s *OpenAIGatewayService) RecordUsage(ctx context.Context, input *OpenAIRec
if result.ServiceTier != nil {
serviceTier = strings.TrimSpace(*result.ServiceTier)
}
cost, err = s.calculateOpenAIRecordUsageCost(ctx, result, apiKey, billingModels, multiplier, imageMultiplier, videoMultiplier, tokens, serviceTier)
cost, err = s.calculateOpenAIRecordUsageCost(ctx, result, apiKey, billingModels, multiplier, imageMultiplier, videoMultiplier, baseMultiplier, tokens, serviceTier)
if err != nil {
if !isUsagePricingUnavailableError(err) {
return err
@@ -363,10 +363,18 @@ func (s *OpenAIGatewayService) calculateOpenAIRecordUsageCost(
multiplier float64,
imageMultiplier float64,
videoMultiplier float64,
webSearchMultiplier float64,
tokens UsageTokens,
serviceTier string,
) (*CostBreakdown, error) {
billingModel := firstUsageBillingModel(billingModels)
if result != nil && result.WebSearchCalls > 0 {
// Codex alpha/search 网页搜索按次计费:上游不返回 usage/token 字段,单价只取
// 分组覆盖价(nil 时默认 0.01 = 官方 $10/1000 次),不参与渠道级模型定价。
// 倍率与 image/video 按次口径一致:使用不含高峰因子的基础倍率
//(用户专属 > 分组 rate_multiplier > 系统默认),与分组表单的价格预览承诺一致。
return s.billingService.CalculateWebSearchCost(result.WebSearchCalls, webSearchPricePerCallFromAPIKey(apiKey), webSearchMultiplier), nil
}
if isGrokVideoUsageResult(result, billingModels) {
if resolved := s.resolveOpenAIChannelPricing(ctx, billingModel, apiKey); resolved == nil || resolved.Mode != BillingModeToken {
return s.calculateOpenAIVideoCost(ctx, billingModel, apiKey, result, videoMultiplier), nil
@@ -223,13 +223,17 @@ func TestOpenAIGatewayServiceForwardOAuthCompactDowngradesMaxEffort(t *testing.T
require.Equal(t, "xhigh", *result.ReasoningEffort)
}
func TestOpenAIGatewayServiceForwardOAuthResponsesPreservesMaxEffort(t *testing.T) {
func TestOpenAIGatewayServiceForwardOAuthRemoteCompactV2PreservesResponsesWire(t *testing.T) {
gin.SetMode(gin.TestMode)
upstream := &httpUpstreamRecorder{
resp: &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(strings.NewReader(`{"usage":{"input_tokens":1,"output_tokens":2}}`)),
Header: http.Header{"Content-Type": []string{"text/event-stream"}},
Body: io.NopCloser(strings.NewReader(
"data: {\"type\":\"response.output_item.done\",\"item\":{\"type\":\"compaction\",\"encrypted_content\":\"summary\"}}\n\n" +
"data: {\"type\":\"response.completed\",\"response\":{\"usage\":{\"input_tokens\":1,\"output_tokens\":2}}}\n\n" +
"data: [DONE]\n\n",
)),
},
}
cfg := &config.Config{}
@@ -244,6 +248,9 @@ func TestOpenAIGatewayServiceForwardOAuthResponsesPreservesMaxEffort(t *testing.
Credentials: map[string]any{
"access_token": "oauth-token",
"chatgpt_account_id": "chatgpt-acc",
"compact_model_mapping": map[string]any{
"gpt-5.6-sol": "gpt-5.6-sol-openai-compact",
},
},
Status: StatusActive,
Schedulable: true,
@@ -251,16 +258,82 @@ func TestOpenAIGatewayServiceForwardOAuthResponsesPreservesMaxEffort(t *testing.
rec := httptest.NewRecorder()
c, _ := gin.CreateTestContext(rec)
c.Request = httptest.NewRequest(http.MethodPost, "/openai/v1/responses", nil)
c.Request.Header.Set("x-codex-beta-features", "remote_compaction_v2")
SetOpenAIClientTransport(c, OpenAIClientTransportHTTP)
body := []byte(`{"model":"gpt-5.6-sol","instructions":"response-test","input":"hello","reasoning":{"effort":"max"}}`)
body := []byte(`{"model":"gpt-5.6-sol","stream":true,"instructions":"response-test","input":[{"type":"message","role":"user","content":"hello"},{"type":"compaction_trigger"}],"reasoning":{"effort":"max","context":"all_turns"}}`)
result, err := svc.Forward(context.Background(), c, account, body)
require.NoError(t, err)
require.NotNil(t, result)
require.NotNil(t, upstream.lastReq)
require.Equal(t, chatgptCodexURL, upstream.lastReq.URL.String())
require.Equal(t, "gpt-5.6-sol", gjson.GetBytes(upstream.lastBody, "model").String())
require.True(t, gjson.GetBytes(upstream.lastBody, "stream").Bool())
require.Equal(t, "compaction_trigger", gjson.GetBytes(upstream.lastBody, "input.#(type==\"compaction_trigger\").type").String())
require.Equal(t, "max", gjson.GetBytes(upstream.lastBody, "reasoning.effort").String())
require.Equal(t, "all_turns", gjson.GetBytes(upstream.lastBody, "reasoning.context").String())
require.Equal(t, "remote_compaction_v2", upstream.lastReq.Header.Get("x-codex-beta-features"))
require.Contains(t, rec.Body.String(), `"type":"compaction"`)
require.Contains(t, rec.Body.String(), `"encrypted_content":"summary"`)
require.NotNil(t, result.ReasoningEffort)
require.Equal(t, "max", *result.ReasoningEffort)
}
func TestOpenAIGatewayServiceForwardAPIKeyRemoteCompactV2PreservesResponsesWire(t *testing.T) {
gin.SetMode(gin.TestMode)
upstream := &httpUpstreamRecorder{
resp: &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"text/event-stream"}},
Body: io.NopCloser(strings.NewReader(
"data: {\"type\":\"response.output_item.done\",\"item\":{\"type\":\"compaction\",\"encrypted_content\":\"summary\"}}\n\n" +
"data: {\"type\":\"response.completed\",\"response\":{\"usage\":{\"input_tokens\":1,\"output_tokens\":2}}}\n\n" +
"data: [DONE]\n\n",
)),
},
}
cfg := &config.Config{}
cfg.Security.URLAllowlist.Enabled = false
svc := &OpenAIGatewayService{cfg: cfg, httpUpstream: upstream}
account := &Account{
ID: 11,
Name: "openai-apikey-responses",
Platform: PlatformOpenAI,
Type: AccountTypeAPIKey,
Concurrency: 1,
Credentials: map[string]any{
"api_key": "sk-test",
"base_url": "https://example.com/v1",
"compact_model_mapping": map[string]any{
"gpt-5.6-sol": "gpt-5.6-sol-openai-compact",
},
},
Extra: map[string]any{"use_responses_api": true},
Status: StatusActive,
Schedulable: true,
}
rec := httptest.NewRecorder()
c, _ := gin.CreateTestContext(rec)
c.Request = httptest.NewRequest(http.MethodPost, "/openai/v1/responses", nil)
c.Request.Header.Set("x-codex-beta-features", "remote_compaction_v2")
SetOpenAIClientTransport(c, OpenAIClientTransportHTTP)
body := []byte(`{"model":"gpt-5.6-sol","stream":true,"instructions":"response-test","input":[{"type":"message","role":"user","content":"hello"},{"type":"compaction_trigger"}],"reasoning":{"effort":"max","context":"all_turns"}}`)
result, err := svc.Forward(context.Background(), c, account, body)
require.NoError(t, err)
require.NotNil(t, result)
require.NotNil(t, upstream.lastReq)
require.Equal(t, "https://example.com/v1/responses", upstream.lastReq.URL.String())
require.Equal(t, "gpt-5.6-sol", gjson.GetBytes(upstream.lastBody, "model").String())
require.True(t, gjson.GetBytes(upstream.lastBody, "stream").Bool())
require.Equal(t, "compaction_trigger", gjson.GetBytes(upstream.lastBody, "input.#(type==\"compaction_trigger\").type").String())
require.Equal(t, "max", gjson.GetBytes(upstream.lastBody, "reasoning.effort").String())
require.Equal(t, "all_turns", gjson.GetBytes(upstream.lastBody, "reasoning.context").String())
require.Equal(t, "remote_compaction_v2", upstream.lastReq.Header.Get("x-codex-beta-features"))
require.Contains(t, rec.Body.String(), `"type":"compaction"`)
require.Contains(t, rec.Body.String(), `"encrypted_content":"summary"`)
require.NotNil(t, result.ReasoningEffort)
require.Equal(t, "max", *result.ReasoningEffort)
}
@@ -347,6 +347,7 @@ func TestOpenAIGatewayService_OAuthPassthrough_StreamKeepsToolNameAndBodyNormali
c.Request.Header.Set("Accept-Encoding", "gzip")
c.Request.Header.Set("Proxy-Authorization", "Basic abc")
c.Request.Header.Set("X-Test", "keep")
c.Request.Header.Set("x-codex-beta-features", "remote_compaction_v2")
originalBody := []byte(`{"model":"gpt-5.2","stream":true,"store":true,"instructions":"local-test-instructions","input":[{"type":"text","text":"hi"}]}`)
@@ -409,6 +410,7 @@ func TestOpenAIGatewayService_OAuthPassthrough_StreamKeepsToolNameAndBodyNormali
require.Empty(t, upstream.lastReq.Header.Get("Accept-Encoding"))
require.Empty(t, upstream.lastReq.Header.Get("Proxy-Authorization"))
require.Empty(t, upstream.lastReq.Header.Get("X-Test"))
require.Equal(t, "remote_compaction_v2", upstream.lastReq.Header.Get("x-codex-beta-features"))
// 3) required OAuth headers are present
require.Equal(t, "chatgpt.com", upstream.lastReq.Host)
@@ -1373,6 +1375,7 @@ func TestOpenAIGatewayService_APIKeyPassthrough_PreservesBodyAndUsesResponsesEnd
c.Request = httptest.NewRequest(http.MethodPost, "/v1/responses", bytes.NewReader(nil))
c.Request.Header.Set("User-Agent", "curl/8.0")
c.Request.Header.Set("X-Test", "keep")
c.Request.Header.Set("x-codex-beta-features", "remote_compaction_v2")
originalBody := []byte(`{"model":"gpt-5.2","stream":false,"service_tier":"flex","max_output_tokens":128,"input":[{"type":"text","text":"hi"}]}`)
resp := &http.Response{
@@ -1410,6 +1413,7 @@ func TestOpenAIGatewayService_APIKeyPassthrough_PreservesBodyAndUsesResponsesEnd
require.Equal(t, "https://api.openai.com/v1/responses", upstream.lastReq.URL.String())
require.Equal(t, "Bearer sk-api-key", upstream.lastReq.Header.Get("Authorization"))
require.Equal(t, "curl/8.0", upstream.lastReq.Header.Get("User-Agent"))
require.Equal(t, "remote_compaction_v2", upstream.lastReq.Header.Get("x-codex-beta-features"))
require.Empty(t, upstream.lastReq.Header.Get("X-Test"))
}
@@ -421,6 +421,10 @@ func (s *OpenAIGatewayService) ProxyResponsesWebSocketFromClient(
storeDisabled,
)
currentBridgePayload := firstPayload
// Keep the first turn as the stable conversation seed. The mapped model
// is resolved again for each turn below so an in-connection model switch
// cannot reuse another model's upstream cache identity.
grokCacheSeedPayload := firstPayload.payloadRaw
var bridgeReplayInput []json.RawMessage
bridgeReplayInputExists := false
for turn := 1; ; turn++ {
@@ -469,6 +473,13 @@ func (s *OpenAIGatewayService) ProxyResponsesWebSocketFromClient(
openAIWSRawPayloadHasToolCallOutput(currentBridgePayload.payloadRaw),
)
}
grokCacheIdentity := ""
if account.Platform == PlatformGrok {
grokCacheIdentity, err = resolveGrokWSCacheIdentity(c, account, grokCacheSeedPayload, currentBridgePayload.originalModel)
if err != nil {
return fmt.Errorf("resolve Grok websocket cache identity: %w", err)
}
}
result, bridgeErr := s.proxyOpenAIWSHTTPBridgeTurn(
ctx,
c,
@@ -480,6 +491,7 @@ func (s *OpenAIGatewayService) ProxyResponsesWebSocketFromClient(
currentBridgePayload.imageBillingModel,
currentBridgePayload.imageSizeTier,
currentBridgePayload.imageInputSize,
grokCacheIdentity,
turn,
writeClientMessage,
)
@@ -74,6 +74,11 @@ func (s *OpenAIGatewayService) buildOpenAIWSHeaders(
if v := strings.TrimSpace(c.Request.Header.Get("accept-language")); v != "" {
headers.Set("accept-language", v)
}
for _, value := range c.Request.Header.Values("x-codex-beta-features") {
if value = strings.TrimSpace(value); value != "" {
headers.Add("x-codex-beta-features", value)
}
}
}
// OAuth 账号:将 apiKeyID 混入 session 标识符,防止跨用户会话碰撞。
if account != nil && account.Type == AccountTypeOAuth {
@@ -602,6 +602,7 @@ func TestOpenAIGatewayService_Forward_WSv2_OAuthStoreFalseByDefault(t *testing.T
c.Request.Header.Set("User-Agent", "codex_cli_rs/0.98.0")
c.Request.Header.Set("session_id", "sess-oauth-1")
c.Request.Header.Set("conversation_id", "conv-oauth-1")
c.Request.Header.Set("x-codex-beta-features", "remote_compaction_v2")
cfg := &config.Config{}
cfg.Security.URLAllowlist.Enabled = false
@@ -661,6 +662,7 @@ func TestOpenAIGatewayService_Forward_WSv2_OAuthStoreFalseByDefault(t *testing.T
require.True(t, gjson.Get(requestJSON, "stream").Exists(), "WSv2 payload 应保留 stream 字段")
require.True(t, gjson.Get(requestJSON, "stream").Bool(), "OAuth Codex 规范化后应强制 stream=true")
require.Equal(t, openAIWSBetaV2Value, captureDialer.lastHeaders.Get("OpenAI-Beta"))
require.Equal(t, "remote_compaction_v2", captureDialer.lastHeaders.Get("x-codex-beta-features"))
// OAuth 账号的 session_id/conversation_id 应被 isolateOpenAISessionID 隔离,
// 测试中未设置 api_key 到 context,apiKeyID=0。
require.Equal(t, isolateOpenAISessionID(0, "sess-oauth-1"), captureDialer.lastHeaders.Get("session_id"))
@@ -12,6 +12,7 @@ import (
"time"
"github.com/Wei-Shaw/sub2api/internal/config"
"github.com/Wei-Shaw/sub2api/internal/pkg/xai"
"github.com/gin-gonic/gin"
"github.com/tidwall/gjson"
)
@@ -155,6 +156,7 @@ func (s *OpenAIGatewayService) proxyOpenAIWSHTTPBridgeTurn(
imageBillingModel string,
imageSizeTier string,
imageInputSize string,
grokCacheIdentity string,
turn int,
writeClientMessage func([]byte) error,
) (*OpenAIForwardResult, error) {
@@ -179,21 +181,19 @@ func (s *OpenAIGatewayService) proxyOpenAIWSHTTPBridgeTurn(
upstreamCtx, releaseUpstreamCtx := detachUpstreamContext(ctx)
var upstreamReq *http.Request
if account.Platform == PlatformGrok {
upstreamModel := strings.TrimSpace(gjson.GetBytes(body, "model").String())
if originalModel != "" {
if mappedModel := normalizeOpenAIModelForUpstream(account, account.GetMappedModel(originalModel)); mappedModel != "" {
upstreamModel = mappedModel
}
}
if upstreamModel == "" {
upstreamModel = "grok-4.3"
}
upstreamModel := resolveGrokWSUpstreamModel(account, body, originalModel)
grokIntentSourceBody := body
body, err = patchGrokResponsesBody(body, upstreamModel)
if err != nil {
releaseUpstreamCtx()
return nil, err
}
upstreamReq, err = buildGrokResponsesRequest(upstreamCtx, c, account, body, token)
body, err = applyGrokResponsesCacheIdentity(body, grokIntentSourceBody, grokCacheIdentity, account.IsGrokOAuth())
if err != nil {
releaseUpstreamCtx()
return nil, fmt.Errorf("apply grok prompt cache identity: %w", err)
}
upstreamReq, err = buildGrokResponsesRequest(upstreamCtx, c, account, body, token, grokCacheIdentity)
} else {
upstreamReq, err = s.buildUpstreamRequestOpenAIPassthrough(upstreamCtx, c, account, body, token)
}
@@ -222,6 +222,9 @@ func (s *OpenAIGatewayService) proxyOpenAIWSHTTPBridgeTurn(
if resp.StatusCode >= 400 {
respBody, _ := io.ReadAll(io.LimitReader(resp.Body, openAIWSHTTPBridgeErrorBodyLimitBytes))
if account.Platform == PlatformGrok {
s.handleGrokAccountUpstreamError(ctx, account, resp.StatusCode, resp.Header, respBody)
}
upstreamMsg := sanitizeUpstreamErrorMessage(strings.TrimSpace(extractUpstreamErrorMessage(respBody)))
if upstreamMsg == "" {
upstreamMsg = http.StatusText(resp.StatusCode)
@@ -229,6 +232,9 @@ func (s *OpenAIGatewayService) proxyOpenAIWSHTTPBridgeTurn(
_ = writeClientMessage(buildOpenAIWSHTTPBridgeErrorEvent(resp.StatusCode, upstreamMsg))
return nil, fmt.Errorf("upstream http bridge error: status=%d message=%s", resp.StatusCode, upstreamMsg)
}
if account.Platform == PlatformGrok {
s.updateGrokUsageSnapshot(ctx, account, xai.ParseQuotaHeaders(resp.Header, resp.StatusCode))
}
responseID := ""
usage := OpenAIUsage{}
@@ -407,3 +413,25 @@ func (s *OpenAIGatewayService) proxyOpenAIWSHTTPBridgeTurn(
}
return resultWithUsage(), errors.New("upstream http bridge stream ended before terminal event")
}
func resolveGrokWSCacheIdentity(c *gin.Context, account *Account, payload []byte, originalModel string) (string, error) {
body, err := prepareOpenAIWSHTTPBridgeBody(payload)
if err != nil {
return "", err
}
upstreamModel := resolveGrokWSUpstreamModel(account, body, originalModel)
return resolveGrokCacheIdentity(c, body, "", upstreamModel), nil
}
func resolveGrokWSUpstreamModel(account *Account, body []byte, originalModel string) string {
upstreamModel := strings.TrimSpace(gjson.GetBytes(body, "model").String())
if account != nil && originalModel != "" {
if mappedModel := normalizeOpenAIModelForUpstream(account, account.GetMappedModel(originalModel)); mappedModel != "" {
upstreamModel = mappedModel
}
}
if upstreamModel == "" {
upstreamModel = "grok-4.3"
}
return upstreamModel
}
@@ -3,6 +3,7 @@ package service
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
@@ -127,6 +128,7 @@ func TestOpenAIWSHTTPBridgeRelaysSSEFramesAsWebSocketMessages(t *testing.T) {
"",
"",
"",
"",
1,
writeClient,
)
@@ -179,21 +181,28 @@ func TestOpenAIWSHTTPBridgeRelaysSSEFramesAsWebSocketMessages(t *testing.T) {
func TestProxyResponsesWebSocketFromClientForGrokUsesXAIHTTPBridge(t *testing.T) {
gin.SetMode(gin.TestMode)
sseBody := strings.Join([]string{
`data: {"type":"response.created","response":{"id":"resp_grok_ws","model":"grok-4.3"}}`,
"",
`data: {"type":"response.output_text.delta","response":{"id":"resp_grok_ws"},"delta":"ok"}`,
"",
`data: {"type":"response.completed","response":{"id":"resp_grok_ws","model":"grok-4.3","usage":{"input_tokens":4,"output_tokens":2}}}`,
"",
}, "\n")
upstream := &httpUpstreamRecorder{resp: &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{
"Content-Type": []string{"text/event-stream"},
"Xai-Request-Id": []string{"xai-ws-req"},
},
Body: io.NopCloser(strings.NewReader(sseBody)),
bridgeResponse := func(responseID, requestID string, cachedTokens int) *http.Response {
sseBody := strings.Join([]string{
`data: {"type":"response.created","response":{"id":"` + responseID + `","model":"grok-4.3"}}`,
"",
`data: {"type":"response.output_text.delta","response":{"id":"` + responseID + `"},"delta":"ok"}`,
"",
`data: {"type":"response.completed","response":{"id":"` + responseID + `","model":"grok-4.3","usage":{"input_tokens":4,"output_tokens":2,"input_tokens_details":{"cached_tokens":` + fmt.Sprintf("%d", cachedTokens) + `}}}}`,
"",
}, "\n")
return &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{
"Content-Type": []string{"text/event-stream"},
"Xai-Request-Id": []string{requestID},
},
Body: io.NopCloser(strings.NewReader(sseBody)),
}
}
upstream := &httpUpstreamRecorder{responses: []*http.Response{
bridgeResponse("resp_grok_ws_1", "xai-ws-req-1", 0),
bridgeResponse("resp_grok_ws_2", "xai-ws-req-2", 3),
bridgeResponse("resp_grok_ws_3", "xai-ws-req-3", 0),
}}
svc := &OpenAIGatewayService{
cfg: &config.Config{
@@ -241,6 +250,7 @@ func TestProxyResponsesWebSocketFromClientForGrokUsesXAIHTTPBridge(t *testing.T)
req := r.Clone(r.Context())
req.Header = req.Header.Clone()
ginCtx.Request = req
ginCtx.Set("api_key", &APIKey{ID: 7101})
errCh <- svc.ProxyResponsesWebSocketFromClient(r.Context(), ginCtx, conn, account, "access-token", firstMessage, nil)
}))
@@ -271,6 +281,33 @@ func TestProxyResponsesWebSocketFromClientForGrokUsesXAIHTTPBridge(t *testing.T)
require.Equal(t, "response.created", gjson.GetBytes(created, "type").String())
require.Equal(t, "response.output_text.delta", gjson.GetBytes(delta, "type").String())
require.Equal(t, "response.completed", gjson.GetBytes(completed, "type").String())
require.Equal(t, "resp_grok_ws_1", gjson.GetBytes(completed, "response.id").String())
writeCtx, cancelWrite = context.WithTimeout(context.Background(), 3*time.Second)
err = clientConn.Write(writeCtx, coderws.MessageText, []byte(`{"type":"response.create","generate":true,"model":"grok","stream":true,"previous_response_id":"resp_grok_ws_1","input":"second turn"}`))
cancelWrite()
require.NoError(t, err)
created = readEvent()
delta = readEvent()
completed = readEvent()
require.Equal(t, "response.created", gjson.GetBytes(created, "type").String())
require.Equal(t, "response.output_text.delta", gjson.GetBytes(delta, "type").String())
require.Equal(t, "response.completed", gjson.GetBytes(completed, "type").String())
require.Equal(t, "resp_grok_ws_2", gjson.GetBytes(completed, "response.id").String())
writeCtx, cancelWrite = context.WithTimeout(context.Background(), 3*time.Second)
err = clientConn.Write(writeCtx, coderws.MessageText, []byte(`{"type":"response.create","generate":true,"model":"grok-4.3","stream":true,"previous_response_id":"resp_grok_ws_2","input":"third turn with a different model"}`))
cancelWrite()
require.NoError(t, err)
created = readEvent()
delta = readEvent()
completed = readEvent()
require.Equal(t, "response.created", gjson.GetBytes(created, "type").String())
require.Equal(t, "response.output_text.delta", gjson.GetBytes(delta, "type").String())
require.Equal(t, "response.completed", gjson.GetBytes(completed, "type").String())
require.Equal(t, "resp_grok_ws_3", gjson.GetBytes(completed, "response.id").String())
_ = clientConn.Close(coderws.StatusNormalClosure, "done")
select {
@@ -280,10 +317,30 @@ func TestProxyResponsesWebSocketFromClientForGrokUsesXAIHTTPBridge(t *testing.T)
require.Fail(t, "proxy did not finish after client close")
}
require.Len(t, upstream.requests, 3)
require.Len(t, upstream.bodies, 3)
require.Equal(t, xai.DefaultCLIBaseURL+"/responses", upstream.lastReq.URL.String())
require.Equal(t, "Bearer access-token", upstream.lastReq.Header.Get("Authorization"))
require.Equal(t, "sub2api-grok/1.0", upstream.lastReq.Header.Get("User-Agent"))
require.Equal(t, "grok-4.5", gjson.GetBytes(upstream.lastBody, "model").String())
require.Equal(t, grokCLIVersion, upstream.lastReq.Header.Get("X-Grok-Client-Version"))
require.Equal(t, "grok-4.5", gjson.GetBytes(upstream.bodies[0], "model").String())
require.Equal(t, "grok-4.5", gjson.GetBytes(upstream.bodies[1], "model").String())
require.Equal(t, "grok-4.3", gjson.GetBytes(upstream.bodies[2], "model").String())
require.NotEmpty(t, gjson.GetBytes(upstream.lastBody, "prompt_cache_key").String())
require.Equal(t, gjson.GetBytes(upstream.lastBody, "prompt_cache_key").String(), upstream.lastReq.Header.Get(grokConversationIDHeader))
require.Equal(t, "web_search", gjson.GetBytes(upstream.lastBody, "tools.0.type").String())
require.Equal(t, "x_search", gjson.GetBytes(upstream.lastBody, "tools.1.type").String())
require.Equal(t, "none", gjson.GetBytes(upstream.lastBody, "tool_choice").String())
firstIdentity := gjson.GetBytes(upstream.bodies[0], "prompt_cache_key").String()
secondIdentity := gjson.GetBytes(upstream.bodies[1], "prompt_cache_key").String()
thirdIdentity := gjson.GetBytes(upstream.bodies[2], "prompt_cache_key").String()
require.NotEmpty(t, firstIdentity)
require.Equal(t, firstIdentity, secondIdentity)
require.NotEmpty(t, thirdIdentity)
require.NotEqual(t, firstIdentity, thirdIdentity)
require.Equal(t, firstIdentity, upstream.requests[0].Header.Get(grokConversationIDHeader))
require.Equal(t, secondIdentity, upstream.requests[1].Header.Get(grokConversationIDHeader))
require.Equal(t, thirdIdentity, upstream.requests[2].Header.Get(grokConversationIDHeader))
require.False(t, gjson.GetBytes(upstream.lastBody, "type").Exists())
require.False(t, gjson.GetBytes(upstream.lastBody, "generate").Exists())
require.False(t, gjson.GetBytes(upstream.lastBody, "prompt_cache_retention").Exists())
+132 -9
View File
@@ -218,6 +218,9 @@ func (l *openAIWSConnLease) Release() {
return
}
l.conn.release()
if l.pool != nil {
l.pool.notifyAccountPoolChanged(l.accountID)
}
}
type openAIWSConn struct {
@@ -225,6 +228,7 @@ type openAIWSConn struct {
ws openAIWSClientConn
handshakeHeaders http.Header
betaFeatures string
leaseCh chan struct{}
closedCh chan struct{}
@@ -498,6 +502,10 @@ func (c *openAIWSConn) handshakeHeader(name string) string {
return strings.TrimSpace(c.handshakeHeaders.Get(strings.TrimSpace(name)))
}
func (c *openAIWSConn) matchesBetaFeatures(betaFeatures string) bool {
return c != nil && c.betaFeatures == betaFeatures
}
func (c *openAIWSConn) isPrewarmed() bool {
if c == nil {
return false
@@ -516,6 +524,7 @@ type openAIWSAccountPool struct {
mu sync.Mutex
conns map[string]*openAIWSConn
pinnedConns map[string]int
changedCh chan struct{}
creating int
lastCleanupAt time.Time
lastAcquire *openAIWSAcquireRequest
@@ -525,6 +534,23 @@ type openAIWSAccountPool struct {
prewarmFailAt time.Time
}
func (ap *openAIWSAccountPool) changeChannelLocked() chan struct{} {
if ap.changedCh == nil {
ap.changedCh = make(chan struct{})
}
return ap.changedCh
}
func (ap *openAIWSAccountPool) signalChangedLocked() {
if ap == nil {
return
}
if ap.changedCh != nil {
close(ap.changedCh)
}
ap.changedCh = make(chan struct{})
}
type OpenAIWSPoolMetricsSnapshot struct {
AcquireTotal int64
AcquireReuseTotal int64
@@ -786,7 +812,9 @@ func (p *openAIWSConnPool) acquire(ctx context.Context, req openAIWSAcquireReque
return nil, errors.New("ws url is empty")
}
retryAcquire:
accountID := req.Account.ID
betaFeatures := normalizeOpenAIWSBetaFeatures(req.Headers)
effectiveMaxConns := p.effectiveMaxConnsByAccount(req.Account)
if effectiveMaxConns <= 0 {
return nil, errOpenAIWSConnQueueFull
@@ -814,7 +842,7 @@ func (p *openAIWSConnPool) acquire(ctx context.Context, req openAIWSAcquireReque
return nil, errOpenAIWSPreferredConnUnavailable
}
preferredConn, ok := ap.conns[preferredConnID]
if !ok || preferredConn == nil {
if !ok || !preferredConn.matchesBetaFeatures(betaFeatures) {
p.recordConnPickDuration(time.Since(pickStartedAt))
ap.mu.Unlock()
closeOpenAIWSConns(evicted)
@@ -895,7 +923,7 @@ func (p *openAIWSConnPool) acquire(ctx context.Context, req openAIWSAcquireReque
}
if preferredConnID != "" {
if conn, ok := ap.conns[preferredConnID]; ok && conn.tryAcquire() {
if conn, ok := ap.conns[preferredConnID]; ok && conn.matchesBetaFeatures(betaFeatures) && conn.tryAcquire() {
connPick := time.Since(pickStartedAt)
p.recordConnPickDuration(connPick)
ap.mu.Unlock()
@@ -917,7 +945,7 @@ func (p *openAIWSConnPool) acquire(ctx context.Context, req openAIWSAcquireReque
}
}
best := p.pickLeastBusyConnLocked(ap, "")
best := p.pickLeastBusyConnLocked(ap, "", betaFeatures)
if best != nil && best.tryAcquire() {
connPick := time.Since(pickStartedAt)
p.recordConnPickDuration(connPick)
@@ -939,7 +967,7 @@ func (p *openAIWSConnPool) acquire(ctx context.Context, req openAIWSAcquireReque
return lease, nil
}
for _, conn := range ap.conns {
if conn == nil || conn == best {
if conn == nil || conn == best || !conn.matchesBetaFeatures(betaFeatures) {
continue
}
if conn.tryAcquire() {
@@ -965,6 +993,37 @@ func (p *openAIWSConnPool) acquire(ctx context.Context, req openAIWSAcquireReque
}
}
if !req.ForceNewConn && len(ap.conns)+ap.creating >= effectiveMaxConns {
compatible := p.pickLeastBusyConnLocked(ap, "", betaFeatures)
if idle := p.pickOldestIdleConnWithDifferentBetaFeaturesLocked(ap, betaFeatures); idle != nil {
delete(ap.conns, idle.id)
evicted = append(evicted, idle)
p.metrics.scaleDownTotal.Add(1)
} else if compatible == nil {
hasConnection := false
for _, conn := range ap.conns {
if conn != nil {
hasConnection = true
break
}
}
if !hasConnection && ap.creating == 0 {
ap.mu.Unlock()
closeOpenAIWSConns(evicted)
return nil, errOpenAIWSConnClosed
}
changedCh := ap.changeChannelLocked()
ap.mu.Unlock()
closeOpenAIWSConns(evicted)
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-changedCh:
goto retryAcquire
}
}
}
if req.ForceNewConn && len(ap.conns)+ap.creating >= effectiveMaxConns {
if idle := p.pickOldestIdleConnLocked(ap); idle != nil {
delete(ap.conns, idle.id)
@@ -988,6 +1047,7 @@ func (p *openAIWSConnPool) acquire(ctx context.Context, req openAIWSAcquireReque
if dialErr != nil {
ap.prewarmFails++
ap.prewarmFailAt = time.Now()
ap.signalChangedLocked()
ap.mu.Unlock()
return nil, dialErr
}
@@ -1016,7 +1076,7 @@ func (p *openAIWSConnPool) acquire(ctx context.Context, req openAIWSAcquireReque
return nil, errOpenAIWSConnQueueFull
}
target := p.pickLeastBusyConnLocked(ap, req.PreferredConnID)
target := p.pickLeastBusyConnLocked(ap, req.PreferredConnID, betaFeatures)
connPick := time.Since(pickStartedAt)
p.recordConnPickDuration(connPick)
if target == nil {
@@ -1089,6 +1149,22 @@ func (p *openAIWSConnPool) pickOldestIdleConnLocked(ap *openAIWSAccountPool) *op
return oldest
}
func (p *openAIWSConnPool) pickOldestIdleConnWithDifferentBetaFeaturesLocked(ap *openAIWSAccountPool, betaFeatures string) *openAIWSConn {
if ap == nil || len(ap.conns) == 0 {
return nil
}
var oldest *openAIWSConn
for _, conn := range ap.conns {
if conn == nil || conn.matchesBetaFeatures(betaFeatures) || conn.isLeased() || conn.waiters.Load() > 0 || p.isConnPinnedLocked(ap, conn.id) {
continue
}
if oldest == nil || conn.lastUsedAt().Before(oldest.lastUsedAt()) {
oldest = conn
}
}
return oldest
}
func (p *openAIWSConnPool) getOrCreateAccountPool(accountID int64) *openAIWSAccountPool {
if p == nil || accountID <= 0 {
return nil
@@ -1101,6 +1177,7 @@ func (p *openAIWSConnPool) getOrCreateAccountPool(accountID int64) *openAIWSAcco
ap := &openAIWSAccountPool{
conns: make(map[string]*openAIWSConn),
pinnedConns: make(map[string]int),
changedCh: make(chan struct{}),
}
actual, _ := p.accounts.LoadOrStore(accountID, ap)
if typed, ok := actual.(*openAIWSAccountPool); ok && typed != nil {
@@ -1126,6 +1203,16 @@ func (p *openAIWSConnPool) getAccountPool(accountID int64) (*openAIWSAccountPool
return ap, typed && ap != nil
}
func (p *openAIWSConnPool) notifyAccountPoolChanged(accountID int64) {
ap, ok := p.getAccountPool(accountID)
if !ok || ap == nil {
return
}
ap.mu.Lock()
ap.signalChangedLocked()
ap.mu.Unlock()
}
func (p *openAIWSConnPool) isConnPinnedLocked(ap *openAIWSAccountPool, connID string) bool {
if ap == nil || connID == "" || len(ap.pinnedConns) == 0 {
return false
@@ -1212,17 +1299,20 @@ func (p *openAIWSConnPool) cleanupAccountLocked(ap *openAIWSAccountPool, now tim
p.metrics.scaleDownTotal.Add(int64(redundant))
}
}
if len(evicted) > 0 {
ap.signalChangedLocked()
}
return evicted
}
func (p *openAIWSConnPool) pickLeastBusyConnLocked(ap *openAIWSAccountPool, preferredConnID string) *openAIWSConn {
func (p *openAIWSConnPool) pickLeastBusyConnLocked(ap *openAIWSAccountPool, preferredConnID, betaFeatures string) *openAIWSConn {
if ap == nil || len(ap.conns) == 0 {
return nil
}
preferredConnID = stringsTrim(preferredConnID)
if preferredConnID != "" {
if conn, ok := ap.conns[preferredConnID]; ok {
if conn, ok := ap.conns[preferredConnID]; ok && conn.matchesBetaFeatures(betaFeatures) {
return conn
}
}
@@ -1230,7 +1320,7 @@ func (p *openAIWSConnPool) pickLeastBusyConnLocked(ap *openAIWSAccountPool, pref
var bestWaiters int32
var bestLastUsed time.Time
for _, conn := range ap.conns {
if conn == nil {
if conn == nil || !conn.matchesBetaFeatures(betaFeatures) {
continue
}
waiters := conn.waiters.Load()
@@ -1395,10 +1485,12 @@ func (p *openAIWSConnPool) prewarmConns(accountID int64, req openAIWSAcquireRequ
if err != nil {
ap.prewarmFails++
ap.prewarmFailAt = time.Now()
ap.signalChangedLocked()
ap.mu.Unlock()
continue
}
if len(ap.conns) >= p.effectiveMaxConnsByAccount(req.Account) {
ap.signalChangedLocked()
ap.mu.Unlock()
conn.close()
continue
@@ -1406,6 +1498,7 @@ func (p *openAIWSConnPool) prewarmConns(accountID int64, req openAIWSAcquireRequ
ap.conns[conn.id] = conn
ap.prewarmFails = 0
ap.prewarmFailAt = time.Time{}
ap.signalChangedLocked()
ap.mu.Unlock()
}
}
@@ -1424,6 +1517,7 @@ func (p *openAIWSConnPool) evictConn(accountID int64, connID string) {
if len(ap.pinnedConns) > 0 {
delete(ap.pinnedConns, connID)
}
ap.signalChangedLocked()
}
ap.mu.Unlock()
}
@@ -1476,9 +1570,11 @@ func (p *openAIWSConnPool) UnpinConn(accountID int64, connID string) {
count := ap.pinnedConns[connID]
if count <= 1 {
delete(ap.pinnedConns, connID)
ap.signalChangedLocked()
return
}
ap.pinnedConns[connID] = count - 1
ap.signalChangedLocked()
}
func (p *openAIWSConnPool) dialConn(ctx context.Context, req openAIWSAcquireRequest) (*openAIWSConn, error) {
@@ -1501,7 +1597,9 @@ func (p *openAIWSConnPool) dialConn(ctx context.Context, req openAIWSAcquireRequ
}
}
id := p.nextConnID(req.Account.ID)
return newOpenAIWSConn(id, req.Account.ID, conn, handshakeHeaders), nil
pooledConn := newOpenAIWSConn(id, req.Account.ID, conn, handshakeHeaders)
pooledConn.betaFeatures = normalizeOpenAIWSBetaFeatures(req.Headers)
return pooledConn, nil
}
func (p *openAIWSConnPool) nextConnID(accountID int64) string {
@@ -1679,6 +1777,31 @@ func cloneOpenAIWSAcquireRequestPtr(req *openAIWSAcquireRequest) *openAIWSAcquir
return &copied
}
func normalizeOpenAIWSBetaFeatures(headers http.Header) string {
features := make(map[string]struct{})
for name, values := range headers {
if !strings.EqualFold(strings.TrimSpace(name), "x-codex-beta-features") {
continue
}
for _, value := range values {
for _, feature := range strings.Split(value, ",") {
if feature = strings.TrimSpace(feature); feature != "" {
features[feature] = struct{}{}
}
}
}
}
if len(features) == 0 {
return ""
}
normalized := make([]string, 0, len(features))
for feature := range features {
normalized = append(normalized, feature)
}
sort.Strings(normalized)
return strings.Join(normalized, ",")
}
func cloneHeader(src http.Header) http.Header {
if src == nil {
return nil

Some files were not shown because too many files have changed in this diff Show More