Commit Graph
2862 Commits
Author SHA1 Message Date
Wesley Liddick 16765bde69 Merge pull request #3230 from DaydreamCoding/feat/openai-cyber-policy-passthrough
feat(openai): cyber_policy 硬阻断全链路透传、审计与计费
2026-06-16 16:55:51 +08:00
shaw b816949291 feat(openai-quota): query + reset rate-limit credits for OpenAI accounts
Adds an admin-side action that mirrors the Codex Desktop "rate-limit reset"
flow against chatgpt.com upstream for OpenAI OAuth accounts.

Backend
- OpenAIQuotaService.QueryUsage / ResetCredit hit /wham/usage and
  /wham/rate-limit-reset-credits/consume with the Codex Desktop header set,
  reusing OpenAITokenProvider for refreshed tokens and PrivacyClientFactory
  for the impersonated Chrome TLS fingerprint.
- Honors the account's configured proxy by reading the eager-loaded
  account.Proxy directly (falls back to proxyRepo only when missing).
- GET /api/v1/admin/openai/accounts/:id/quota
  POST /api/v1/admin/openai/accounts/:id/reset-quota
- Wire DI for the new service + handler dependency.

Frontend
- OpenAIQuotaResetCell renders a single action row in AccountUsageCell's
  OpenAI section: the existing local "查询" (active sampling) is injected
  via #pre-actions, alongside a "次数 N" button that doubles as the
  upstream query trigger and the available-credit indicator, and a "重置"
  button that consumes one credit.
- No duplicate 5h/7d window display; the local UsageProgressBar owns those
  bars to avoid confusion.
2026-06-16 16:55:07 +08:00
shaw f069c9ae00 fix(outbox-dedup): buildSchedulerGroupPayload typed-nil broke dedup_key consistency
#3255 introduced payload-aware dedup_key (sha256 over event_type, account_id,
group_id, payload_json). enqueueSchedulerOutbox checks "if payload != nil"
before json.Marshal — but Go interface containing a typed-nil map (returned
by buildSchedulerGroupPayload(empty)) is NOT == nil at the interface level.

So an ungrouped account's account_changed event went through this path:
  payload := buildSchedulerGroupPayload(account.GroupIDs)  // typed-nil map
  enqueueSchedulerOutbox(..., payload)                     // interface != nil
    → json.Marshal(typedNilMap) = "null"
    → dedup_key hash = sha256(... + "null")

While other call sites pass literal nil:
  enqueueSchedulerOutbox(..., nil)                         // interface == nil
    → payloadJSON stays empty
    → dedup_key hash = sha256(... + "")

The two dedup_keys differ for what should be the same logical event,
silently degrading dedup effectiveness in bursts on ungrouped accounts.

Fix: change buildSchedulerGroupPayload return type from map[string]any to
any so empty input returns true untyped-nil. All call sites pass the
result straight to enqueueSchedulerOutbox(payload any) — no inspection,
no breakage.

Adds regression test TestEnqueueSchedulerOutbox_UngroupedAccountDedupesWithLiteralNilPayload
asserting (1) typed-nil regression doesn't sneak back, (2) dedup_key for
empty-groups payload matches the literal-nil-payload key.
2026-06-16 14:21:09 +08:00
shaw acaffe29ec fix(account-repo): refresh candidates SQL excluded healthy accounts; fix CI build
Post-merge audit of #3272 found two regressions:

1. ListOAuthRefreshCandidates used "AND NOT (a AND b)" which, under PG
   3-valued logic, evaluates to NULL when both temp_unschedulable_until
   and temp_unschedulable_reason are NULL — i.e., the common healthy
   account state. Such rows were silently excluded from the background
   token refresh worker, so their OAuth access tokens would never get
   refreshed and eventually start returning 401.

   Verified empirically against PostgreSQL: only 3 of 5 test rows
   matched before the fix; after switching to "(a AND b) IS NOT TRUE"
   the expected 4 rows match.

2. The new ListOAuthRefreshCandidates method on AccountRepository was
   not implemented on stubAccountRepo in api_contract_test.go (build
   tag "unit"), breaking "make test-unit" which CI runs in
   .github/workflows/backend-ci.yml.

Tests:
- Added IS NOT TRUE and "AND NOT (" assertions to the SQL-shape unit
  test so the predicate can't regress to the broken form again.
- "go test -tags=unit ./internal/..." now passes cleanly.
2026-06-16 14:08:50 +08:00
shaw 31dc8913ac chore(outbox-cleanup): add 10s grace to defend against id-vs-commit race
PG sequences advance outside transactions, so a slow committer can hold
an id that gets surpassed by the watermark before its commit becomes
visible. Without the grace period, cleanup would delete such rows before
the snapshot poller ever sees them. 10s is a comfortable upper bound
on realistic enqueueSchedulerOutbox commit latency.
2026-06-16 11:57:32 +08:00
jjaw cb14935e9a fix: cleanup consumed scheduler outbox rows 2026-06-16 11:56:40 +08:00
shaw 1fdbe52f96 chore(migrations): renumber scheduler outbox dedup migrations 151/152 -> 152/153
#3232 (already merged) occupies 151. Renumber #3255's dedup migrations to
avoid collision and keep linear ordering. Updated runner constant + tests.
2026-06-16 11:50:25 +08:00
jjaw b3ec6288ad fix: release scheduler outbox dedup on claim 2026-06-16 11:48:13 +08:00
jjaw 60cf89ae26 fix: recover scheduler outbox invalid dedup index 2026-06-16 11:48:13 +08:00
jjaw 3ef70b045d fix: safely coalesce scheduler outbox events 2026-06-16 11:48:12 +08:00
jjaw 34e66ec0a5 fix: outbox scheduler snapshot coalesce 2026-06-16 11:48:12 +08:00
jjaw 9b270f11d7 refactor: inline token refresh retry reason prefix 2026-06-16 11:44:39 +08:00
jjaw 74199b6a6e fix: reduce token refresh retry amplification 2026-06-16 11:44:39 +08:00
jjaw b0579c4891 fix: move user wait queue accounting off hot path 2026-06-16 11:41:54 +08:00
jjaw 8b698ff4c1 fix account list parameter limit 2026-06-16 11:39:15 +08:00
Wesley Liddick 62fef6ebed Merge pull request #3178 from jianjianai/fix/fix-failover-error-body-reuse
fix: 修复 OpenAI failover 错误响应体重复读取 / reuse cached OpenAI failover error body
2026-06-16 11:13:10 +08:00
Wesley Liddick 8ddc138094 Merge pull request #3232 from jianjianai/fix1/account-expiry-autopause-index
fix:account expiry autopause index / 账号过期自动禁用索引优化
2026-06-16 11:12:57 +08:00
Wesley Liddick 2f1f1971b4 Merge pull request #3195 from jianjianai/fix/invalid-refresh-token-nonretryable
fix: treat invalid_refresh_token as non-retryable / 将 invalid_refresh_token 判定为不可重试
2026-06-16 11:12:45 +08:00
Wesley Liddick 15b0753c1d Merge pull request #3283 from wucm667/fix/non-json-200-response-failover
fix(gateway): 上游 200 返回非 JSON 错误报文时纳入 failover,避免坏上游被反复命中误扣费
2026-06-16 11:12:21 +08:00
Wesley Liddick a0fc2c4143 Merge pull request #3218 from wucm667/fix/antigravity-system-role-message
fix(antigravity): 处理 messages 中 role:system 消息,修复 Claude Code 接入 400
2026-06-16 11:12:08 +08:00
Wesley Liddick 28bed8b866 Merge pull request #3252 from wucm667/fix/upstream-zstd-response-decompression
fix(gateway): 解压 zstd 上游响应体,修复非流式 usage 静默记 0 的计费丢失
2026-06-16 11:11:51 +08:00
Wesley Liddick 715594847b Merge pull request #3267 from codeQuest-fly/codex/fix-sub2api-injection
feat: configure Claude OAuth system prompt blocks
2026-06-16 10:43:38 +08:00
dailingfei b63b411654 fix: remove unused billing attribution helper 2026-06-15 17:16:26 +08:00
wucm667 ab9987b2e2 fix(gateway): fail over on non-JSON 2xx responses 2026-06-15 11:04:24 +08:00
dailingfei 8ce7b9a8f6 feat: configure Claude OAuth system prompt blocks 2026-06-13 04:12:13 +08:00
wucm667 c1c28ac7bb fix(gateway): 解压 zstd 上游响应体 2026-06-12 15:06:01 +08:00
wucm667 edfd5e3736 fix(apicompat): default tool strict to false 2026-06-12 15:01:58 +08:00
DaydreamCodingandClaude Opus 4.8 b62b573f7f feat(openai): cyber_policy 硬阻断全链路透传、审计与计费
上游对单次请求下发 error.code=cyber_policy 硬阻断时,网关在所有端点
(/v1/responses、/v1/chat/completions、/v1/messages、WebSocket)及流式/
非流式路径下,将该结果原样透传给客户端,绝不 failover、换号或同步拦截;
命中后异步完成审计与计费:

- 风控中心记录 cyber_policy 留痕并发送通知邮件,落库先于发信,SMTP 阻塞
  不影响留痕
- ops 错误请求记录,状态码对齐客户端实际接收(流式 200 / 非流式 400)
- 用量明细标记 request_type=cyber,按上游真实 token 计费,HTTP 与
  WebSocket 计费口径统一,零 token 命中不误扣
- 会话级自动屏蔽(管理员开关,默认关):命中的会话在可配 TTL 内本地拦截
  不再发往上游,仅屏蔽该会话不影响同 Key 其他会话
- 封号计数排除开关:可选让 cyber 命中不计入自动封号,命中当次不判定且
  历史行在违规计数中一并排除

WebSocket 多轮连接下 cyber 标记按 turn 生命周期管理,逐轮独立检测与记录;
透传的错误响应不被兜底逻辑追加内容污染。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 01:47:01 +08:00
jjaw e4c255a77a fix:account expiry autopause index 2026-06-12 01:38:09 +08:00
wucm667 65559ac589 fix(antigravity): merge system role messages 2026-06-11 18:37:00 +08:00
jjaw 727ac3f689 fix: add app_session_terminated to non-retryable refresh errors / 将 app_session_terminated 添加到不可重试的刷新错误中 2026-06-10 22:32:04 +08:00
github-actions[bot] e34ad2b194 chore: sync VERSION to 0.1.136 [skip ci] 2026-06-10 07:02:12 +00:00
jjaw fa8f1749f5 fix: treat invalid_refresh_token as non-retryable / 将 invalid_refresh_token 判定为不可重试 2026-06-10 14:21:44 +08:00
shaw 0acf00c4a1 Add admin compliance acknowledgement gate 2026-06-10 14:16:51 +08:00
Wesley Liddick c32e29bab0 Merge pull request #3187 from jianjianai/fix/gateway-debug-log-loop
优化调度日志循环开销 / Reduce debug logging overhead in scheduler hot path
2026-06-10 09:58:40 +08:00
Wesley Liddick 2b1c5f8563 Merge pull request #3186 from jianjianai/fix/account-group-scheduler-indexes
优化账号分组调度索引 / Add two concurrent composite indexes
2026-06-10 09:58:31 +08:00
Wesley Liddick 0c997d41a9 Merge pull request #3176 from jianjianai/fix/precompute-model-body-replacement
优化 OpenAI 网关 failover 流程,避免账号切换时重复对请求体执行 JSON model 替换。
2026-06-10 09:58:19 +08:00
Wesley Liddick f0748b98f6 Merge pull request #3173 from jianjianai/fix/idempotency-utf8-truncation
fix idempotency response utf8 truncation / 响应缓存按字节截断时可能切断 UTF-8 多字节字符的问题。
2026-06-10 09:58:09 +08:00
Wesley Liddick dd709f5985 Merge pull request #3181 from codeQuest-fly/fix/gateway-upstream-error-double-write
fix: avoid double-writing error frame on non-stream upstream errors
2026-06-10 09:26:18 +08:00
Wesley Liddick 5cb17fdc4f Merge pull request #3184 from touwaeriol/fix/bedrock-beta-and-error-passthrough
fix(gateway): prevent error passthrough double-write and Bedrock beta token leakage
2026-06-10 09:00:21 +08:00
shaw d662c97302 feat: claude-fable-5 2026-06-10 08:58:06 +08:00
jjaw 2c27548b82 优化调度日志循环开销 2026-06-10 01:03:01 +08:00
jjaw 30c00a91d8 优化账号分组调度索引 2026-06-10 00:59:50 +08:00
erio 448936d965 fix(ci): fix gofmt, errcheck, and test for supported context-management beta token 2026-06-10 00:46:19 +08:00
erio 12962bab24 refactor(bedrock): merge header filtering into ApplyBedrockCCCompat
Move anthropic-beta header filtering from separate FilterBedrockBetaHeader
into ApplyBedrockCCCompat, so one function handles all CC compat processing
(body cleanup + header filtering). Change signature from ctx to *gin.Context
to access request headers. Remove the redundant separate call in handler.
2026-06-10 00:18:09 +08:00
erio bf28a0098a fix(bedrock): filter unsupported top-level fields and fix beta token cleanup
- Remove `provider` and `metadata` fields from Bedrock request body.
  Bedrock returns 400 ValidationException for unknown top-level fields.
- When filtered beta tokens list is empty, delete any pre-existing
  `anthropic_beta` field in body to prevent client-injected tokens
  from leaking to Bedrock unfiltered.
- Add `context-management-2025-06-27` and `fine-grained-tool-streaming-2025-05-14`
  to bedrockSupportedBetaTokens whitelist per AWS documentation.
2026-06-10 00:16:30 +08:00
erio 20f3f2049b fix(gateway): complete MarkResponseCommitted coverage for all platforms
Add MarkResponseCommitted to 7 error-writing helper functions and
4 early-return branches in OpenAI handlers, plus 3 inline c.Data paths.

Helper functions (1 line each, covers 60+ call sites):
- writeGatewayCCError, writeResponsesError (Anthropic compat)
- writeClaudeError, writeGoogleError (Gemini + Antigravity)
- writeChatCompletionsError (Gemini Chat Completions compat)

OpenAI handleErrorResponse/handleCompatErrorResponse:
- passthrough rule and ShouldHandleErrorCode branches return before
  the existing MarkResponseCommitted, now each has its own Mark.

Inline c.Data: Gemini ForwardNative (2) + Antigravity ForwardGemini (1).
2026-06-10 00:16:08 +08:00
erio 6c88631690 fix(gateway): prevent double-write on error passthrough responses
Service layer writes a complete JSON error response then returns error.
Handler's ensureForwardErrorResponse couldn't distinguish this from
"no response written" and appended an SSE event, corrupting the body.

Use gin.Context flag: service marks MarkResponseCommitted(c) after
writing, ensureForwardErrorResponse checks IsResponseCommitted(c)
and skips. Zero function signature changes, zero error wrapping.
2026-06-10 00:15:51 +08:00
dailingfei 914c059f4a fix: avoid double-writing error frame on non-stream upstream errors
When a Forward implementation already wrote a complete non-SSE (JSON) error
response to the client and returned an error -- e.g. the case-400 passthrough
in GatewayService.handleErrorResponse -- the handler unconditionally called
ensureForwardErrorResponse, which detected the writer was already written and
appended a fallback `data: {"type":"error",...}` SSE frame. The client then
received a corrupted body: the upstream JSON immediately followed by a stray
`data:` line.

Add gatewayForwardErrorAlreadyCommunicated (and the OpenAI counterpart) to
detect this case -- writer size changed AND Content-Type is not
text/event-stream -- and skip the fallback. SSE streams that only flushed
keepalive pings or partial data still receive a protocol-compliant terminal
frame, so strict SDKs (Codex CLI) do not see a silent EOF.

Applied consistently across the Messages / ChatCompletions / Responses
gateway handlers and the OpenAI chat/images handlers. Added regression tests
covering JSON passthrough, mid-stream SSE 400, nil-error and no-write cases.
2026-06-09 22:46:06 +08:00
jjaw 46bd7968a3 fix: reuse OpenAI failover error body 2026-06-09 21:56:35 +08:00