Commit Graph
1422 Commits
Author SHA1 Message Date
Wesley Liddick b918874f81 Merge pull request #5403 from cyhhao/fix/codex-capacity-exponential-backoff
fix(openai): back off capacity retries exponentially
2026-08-11 13:58:06 +08:00
pigzwy 9096492b55 feat(billing): support safe upstream response model billing 2026-08-10 18:45:14 +08:00
Wesley Liddick 10a4c6e3ad Merge pull request #5234 from wucm667/fix/issue-5230-deduplicate-latest-turn-audit
fix(security-audit): deduplicate websocket turn audits
2026-08-10 10:53:17 +08:00
Wesley Liddick f3c7a1a8c4 Merge pull request #5295 from wucm667/fix/issue-5289-streaming-upstream-error
fix: emit response.failed when compact keepalive commits headers but no SSE payload
2026-08-10 10:52:49 +08:00
Wesley Liddick 30d0405388 Merge pull request #5464 from wucm667/fix/issue-5455-api-key-input-validation
fix(api-key): validate quota and expiry inputs
2026-08-10 10:52:04 +08:00
wucm667 f5c108c836 fix(api-key): validate quota and expiry inputs 2026-08-09 22:42:51 +08:00
lyen1688 bbc8b6e906 完善大文件备份分卷上传与恢复 2026-08-09 20:58:07 +08:00
shaw 563a72ca73 feat: add default-off switch for email domain registration quota
PR #5423 relaxed the email suffix whitelist: once a whitelist is
configured, non-whitelisted registrable domains are each allowed to
register one account. That behavior activated unconditionally.

Add registration_email_domain_quota_enabled (default false) to gate it:

- Off (default): restore pre-#5423 strict whitelist semantics — with a
  non-empty whitelist, non-whitelisted domains are rejected with
  EMAIL_SUFFIX_NOT_ALLOWED; the register/verify views restore the
  client-side whitelist pre-check and allowed-domain hint.
- On: keep #5423 behavior — one account per non-whitelisted registrable
  domain (EMAIL_DOMAIN_REGISTRATION_LIMIT).
- Empty whitelist keeps allowing all domains in both states.

Gating lives in validateRegistrationEmailQuota and (as a race-safety
backstop) createUserWithRegistrationEmailGuard; the repository-level
domain lock + in-tx recheck is unchanged. The admin update field is
*bool (omitted = keep current) so stale full-payload saves cannot
silently flip the switch. Email binding and OAuth auto-signup keep
their strict policy, and pending-OAuth bind-login for existing
accounts is unaffected because the handler resolves existing emails
before the quota check.

Frontend adds the toggle to admin settings (zh/en copy; whitelist hint
restored to strict wording, quota wording moved to the new toggle) and
exposes the flag via public settings + SSR injection payload.

Tests: #5423 quota tests now enable the switch explicitly; new
default-off regression tests cover register/send-code/async/pending
OAuth/OIDC create-account plus both register views; API contract JSON
and the injection drift guard are updated.
2026-08-09 15:53:40 +08:00
Wesley Liddick f2da30bcd9 Merge pull request #5423 from lyen1688/feat/email-domain-registration-quota
完善邮箱域名注册额度策略
2026-08-09 15:16:26 +08:00
shaw d92edc01be Merge origin/main into feat/channel-monitor-v2-ops-ui
Resolves three conflicts, all of the "both branches appended to the same
block" shape. Every one is resolved as a union of both sides; nothing from
either parent is dropped.

- handler/admin/setting_handler_update.go: keep ChannelMonitorHideThroughput
  (V2) alongside GrokDefaultTextModel / GrokCrossClientModelMapEnabled /
  GrokDefaultBaseURLMode (#5408). UpdateSettings writes every key on each
  save, so dropping either side would reset those settings to zero values.
- service/domain_constants.go: keep SettingKeyChannelMonitorHideThroughput
  and the three SettingKeyGrok* constants.
- repository/migrations_runner.go: keep the 195 checksum rule (V2) and the
  218/219/220 rules (#5408).
2026-08-09 12:11:35 +08:00
lyen1688 4999231d61 修复邮箱域名注册额度策略 2026-08-08 21:05:20 +08:00
IanShaw027 7eb1310701 fix(grok): close free-by-default billing and related review blockers
H1/H2: bill search and voice with code defaults when group prices are
nil (explicit 0 remains free); bump API key auth snapshot to v19 and
refresh incomplete media/search/audio projections.

M1–M6: free-quota soft gate fails open on cache miss with background
refresh and 60s default TTL; correct password_auth config docs; default
cross-client model map to true (→ grok-4.5); audit /tts and /web_search;
exclude composite from migration 220 video-price clears; never let a
search surcharge mask token pricing failures.
2026-08-08 14:39:22 +08:00
IanShaw027 cec922d335 fix(grok): clear golangci-lint findings on complete-integration branch
Check Close/CloseNow errors, drop unused helpers and dead constants,
lowercase ST1005 error strings, and stop discarding unwrap status as an
unused assignment so CI golangci-lint passes.
2026-08-08 12:56:40 +08:00
IanShaw027 1f58e25ab3 Merge upstream/main into feat/grok-complete-integration
冲突集中在 chat completions / messages 两条 Responses 转发路径:
upstream 给 OpenAIForwardResult 增加了 UpstreamResponseModel 与
UpstreamResponseModelConflict(配套 beginUpstreamResponseModelObservation
观测器),本分支在同样位置把返回值改成了具名变量以便挂 Grok 原生搜索计数。
两侧不互斥,合并结果同时保留上游的响应模型观测字段与 Grok SearchCount 逻辑。

frontend/pnpm-lock.yaml 取 upstream 版本:package.json 与 upstream 完全一致,
本地差异只是 pnpm install 的重解析噪音。
2026-08-08 11:12:56 +08:00
IanShaw027 f3bac4619e feat(keys): expand Grok client samples and tune free soft-gate default
Ship Use Key templates that match Grok Build / Codex best practice: env
vars + multi-model config.toml with api_backend=responses, env_key over
hardcoded secrets, and clearer shell/path guidance for Claude/Codex/OpenCode.

Also set free_quota_token_limit default to 500k (24h soft-gate), clean up
personal-dev-only comments, and keep billing test fixtures aligned.
2026-08-08 10:26:16 +08:00
IanShaw027 e01ce90d47 fix(grok): harden voice request ids, video pending, and search pricing alerts
Mint durable grok_audio/grok_realtime usage ids, avoid CLI headers on api.x.ai
voice, retry video pending store and fail-closed when snapshot is missing without
status duration, align pure-video ImageCount tests, and escalate unset search
price_per_1k to error-level logs.
2026-08-08 09:45:12 +08:00
IanShaw027 12db0f906a fix(grok): drop account-test ZDR path and align media CLI headers
Remove optional upload_url / fake connectivity-only success from admin video
tests. Stamp Grok CLI headers only on the CLI proxy so OAuth media against
api.x.ai can complete and preview video like the gateway path.
2026-08-08 09:45:12 +08:00
IanShaw027 35faaa6d21 feat(grok): register custom-voices CRUD and audio download gateway routes
Forward list/get/patch/delete and reference-audio paths with safe path segment
encoding, method passthrough, and empty-body GET/DELETE handling.
2026-08-08 08:48:38 +08:00
IanShaw027 85b65284ec fix(grok): set async video duration_ms from create accept to done discovery
Store CreatedAt on pending billing at video create and use wall-clock E2E
latency when status/content first observes official done+video.url, so usage
logs no longer record only the single poll hop.
2026-08-08 08:48:38 +08:00
IanShaw027 0d98176c59 fix(channel-monitor-v2): correct aggregation privacy and backfill 2026-08-08 01:59:44 +08:00
IanShaw027 68faeac837 fix(grok): restore base URL resolution and operator settings wiring
Honor account GetGrokBaseURLOr policy for official vs custom endpoints,
and wire settings resolution used by responses/chat URL builders.
2026-08-08 01:07:19 +08:00
IanShaw027 6d632eec45 fix(grok): tighten OAuth SSO flow and hide password login
Require oauth state/redirect consistency, fail closed on missing proxy,
and remove password login from create/reauth UI (admin-only password path stays off by default).
2026-08-08 01:07:19 +08:00
IanShaw027 d0767eab9d feat(grok): admin account test modes with real media preview
Add mode-first connectivity probes for text/image/video/search/tts/stt/realtime,
standalone voice and web-search paths, media upload options, and in-browser
image/audio/video preview (including ZDR-safe b64 images and edit validation).
2026-08-08 01:07:08 +08:00
cyh 74fcdf3d42 fix(openai): preserve exponential capacity backoff 2026-08-08 01:02:02 +08:00
Wesley Liddick 8991574873 Merge pull request #5345 from puppywang/fix/oauth-pending-account-takeover
fix(security): block OAuth account takeover via pending exchange
2026-08-07 23:20:22 +08:00
Brisbanehuang db0bff82c7 feat(usage): audit upstream response models
(cherry picked from commit 839036224f795c8ee5dc6718a2a14372a45eea44)
2026-08-07 09:40:11 -04:00
li 02fbcbe3ad fix(ratelimit): 守卫按端点来源门控,并与冷却键对齐模型口径
上一版守卫只看模型类型,不区分请求从哪个端点进来。OAuth 账号的 /v1/images/*
上游同样是 Codex Responses(openai_images_responses.go → handleOpenAIImagesErrorResponse
→ handleOpenAIAccountUpstreamError → HandleUpstreamModelNotFound),所以专用生图
端点也会命中 plan-gated 分支。账号确实不具备生图能力时跳过冷却,会让调度层失去
唯一的刹车:每个请求都完整走一遍号池,对上游形成无上界的 400 放大。

改动:
- 新增 ctxkey.OpenAIImagesEndpoint 与 WithOpenAIImagesEndpoint /
  OpenAIImagesEndpointFromContext,在 handler/openai_images.go 入口置位;
  与 OpenAIImageGenerationIntent 区分——后者在 /v1/responses 带图片模型时也会置位。
- 守卫下移到 modelKey 计算之后,抽成 shouldSkipCodexPlanGatedImageModelCooldown,
  仅在 plan-gated 分支、且非 /v1/images/* 入站时生效。
- 同时判断 requestedModel 与最终 modelKey:冷却键走 account.GetMappedModel,
  账号可以把文本别名映射到 gpt-image-*,只判请求模型会漏掉这种形态。
2026-08-07 20:49:39 +08:00
IanShaw027 d7c9e7167b fix(grok): 三轮评审 — 流式 Search 去重与调度/计费加固
- P0: 直播 SSE SearchCount 跨事件 call_id 去重,避免 ~2× 附加费
- SearchCount/Audio/WebSearchCalls 走 mandatory usage task
- web_search:uuid 等 forced request_id 优先于 client/local
- Sanitize 始终剥离 cookie;ApplyOAuth 清 grok_needs_reauth_at
- free 判定:paid 证据压过陈旧 free 凭据
- Gateway 列表应用 free soft-gate;token/body-read 可 failover
- web_search 重试支持 WaitPlan 获取;周 PeriodEnd 不再回填月 end
2026-08-07 17:58:23 +08:00
IanShaw027 245d069602 fix(grok): 二轮评审残留 — Search 叠加计费与 fail-closed 安全
- SearchCost 叠加 token(openai/gateway),未定价 warn
- Token URL 校验失败回落 DefaultTokenURL,禁止 Effective 旁路
- free 判定收窄 paidSignal(仅 plan/月额度),usage% 不否决 free
- web_search: mandatory 计费、uuid request_id、上游 failover 重选账号
- 调度阈值:7d/30d 不跨期 until + 48h stale 可选跳过
- SanitizeStoredCredentials 接入 create/update/bulk/SSO/ApplyOAuth
- ApplyOAuth 成功清除 grok_needs_reauth;SSO 允许 header_override_enabled
- VideoModelPrices 视为媒体定价完整;realtime 正常关闭仍计费
2026-08-07 17:40:21 +08:00
IanShaw027 ccf7ba3ba7 fix(grok): P2 分层清理、主对话 SearchCount 与调度 7d/30d
- DoGrokNativeResponsesJSON 去 gin.Context,UA 固定 CLI 身份
- GrokOAuthService 去掉 redis 直依赖,session store 由 wire 注入
- 主对话/流式响应统计 web_search_call/x_search/tool_search 写入 SearchCount
- Grok 调度阈值候选并入官方 weekly/monthly billing %
2026-08-07 17:21:01 +08:00
IanShaw027 7a81468282 fix(grok): 按 review 优先级修复计费漏扣、auth 投影与 free 门禁
P0: content 与 status 共用 claim 计费;稳定 grok-video request_id;
auth 热路径投影 video_model_prices。
P1: claim 失败释放可重试;视频绑定 TTL≥24h;SSO 凭据白名单与脱敏;
Token URL 校验;free soft-gate 默认 1M 并统一 free 判定;
Voice 预检余额;STT 抗低报;Realtime 失败不计费;search 未定价告警。
2026-08-07 17:15:43 +08:00
IanShaw027 93a04567d4 style(grok): 清理搜索处理器末尾空行 2026-08-07 17:03:48 +08:00
IanShaw027 856a96a217 test(grok): 对齐配额模型同步与 CLI 身份断言 2026-08-07 16:55:36 +08:00
IanShaw027 61b975b5dd test(grok): 对齐跨客户端映射与路由审计契约 2026-08-07 16:45:25 +08:00
IanShaw027 21d0905c78 fix(grok): 按官方 status=done + video.url 计费并对齐字段
官方完成响应为 status=done、video.url、video.duration、顶层 model;
分辨率仅在创建请求中声明,status 无此字段故回退 create 快照/默认 480p。
拒绝 completed 等非官方状态与仅 URL 的宽松判定。
2026-08-07 16:36:15 +08:00
IanShaw027 962da308cc fix(grok): 限制导入探测队列并去重任务 2026-08-07 16:36:12 +08:00
IanShaw027 b59a3df5b5 fix(grok): 视频在 status 返回 video.url 后再按上游参数计费
创建/generations 仅绑定账号并缓存 create-time 定价快照;轮询到
video.url / download_url 等成功 URL 时一次性扣费。时长/分辨率/模型
优先取上游 status 字段,缺失则回退创建快照,再套默认 8s/480p。
Redis SetNX + request_id 去重,避免重复轮询双扣。
2026-08-07 16:29:57 +08:00
IanShaw027 79df1647d4 feat(grok): 账单绝对金额、调度阈值 UI、搜索/Voice 计费与 /v1/web_search
- BillingSummary 输出 prepaid/monthly_used/on_demand 绝对金额,并映射官方 7d/30d 进度条
- 账号编辑页支持 credentials.account_scheduling_threshold 覆盖;Settings 文案细化
- groups.search_price_per_1k + 管理端/缓存全链路;SearchCount/AudioUsage 请求级计费
- Voice TTS/STT/Realtime 成功路径 RecordUsage;独立 /v1/web_search 原生 Grok 搜索
2026-08-07 15:52:55 +08:00
IanShaw027 99ad01f6de feat(grok): 网关 Voice TTS/STT/Realtime 与分组音频定价
- 中继 xAI Voice HTTP(/tts /stt /custom-voices)与 Realtime WS(/realtime)
  仅 Grok 分组可用;账号选调度沿用 chat 能力,不依赖创作中心
- Voice URL 固定走 api.x.ai(CLI proxy base 自动回落)
- 分组级 audio_realtime / TTS / STT 单价:schema + migration 218 +
  admin API + GroupsView 表单与 i18n(无创作中心 UI)
2026-08-07 15:08:39 +08:00
IanShaw027 7c62382d04 feat(grok): 吸收调度阈值、配额解析、批量用量与 CLI 身份
从 personal-dev 取优并接线到 feat/grok-complete-integration:

- 调度阈值:grok_sched_* 写入、ApplyAccountSchedulingThreshold、
  account_scheduling_thresholds 设置与 Settings UI、网关过滤
- 配额头:x-rate-limit 别名、相对秒 reset、tier/entitlement 扩展
- 批量 usage:GetUsageBatch + POST /usage/batch + AccountsView 合并加载
- CLI 身份:xai.cli_identity 统一 pin/UA;service 层 applyGrokCLIHeaders
  与 transport 最终改写对齐;media eligibility 模块落地
- UsageCell:月度 cents→USD 与 30d 进度条展示

保留既有 failure taxonomy / sticky / free recovery 等 HEAD cools。
2026-08-07 14:53:18 +08:00
IanShaw027 d0930c4bdb fix(grok): 完善密码与SSO授权能力控制 2026-08-07 14:13:07 +08:00
IanShaw027 451abc3aa5 feat(grok): 加固媒体计费门控并补齐分组按模型价输入
生成类媒体仅在存在真实 image/video 计费单元时写 usage,避免空结果误扣费。
Create/Update 分组输入支持 video_model_prices,与仓储 JSONB 字段打通。
本阶段不引入假 voice 路由(上游音频路径依赖更广协议层)。
2026-08-07 13:08:22 +08:00
IanShaw027 eb6c9663e7 feat(grok): 视频按模型族配置每秒单价,并补齐管理端映射设置
分组新增 video_model_prices(JSONB);计费优先模型×分辨率覆盖,
其次旧分辨率三列,最后官方按模型族默认价。同步补齐 admin 设置
中的 grok_default_text_model / 跨客户端映射开关,并保持
grok-imagine-video-1.5 请求模型 identity 不被静默改写。
2026-08-07 12:44:08 +08:00
IanShaw027 370bdcf695 feat(grok): 补齐密码登录与 SSO 校验,统一 OAuth 凭证形态
在 main 既有 SSO→Build 批量导入之上增加 sso-token 校验与账号密码授权。
密码仅用于换取 SSO 再转 Build OAuth,明文与 raw SSO 均不落库。
2026-08-07 12:38:49 +08:00
IanShaw027 a5beecb92a feat(channel-monitor-v2): 接入模式开关、路由门控与依赖注入
注册 admin/user 路由与 feature/mode 守卫,串联 Wire DI 与设置读写,
公开 channel_monitor_mode 与 hide_throughput 等运行时标志。
2026-08-07 11:05:32 +08:00
IanShaw027 ead73264c7 feat(channel-monitor-v2): 实现被动聚合、分层保留与只读 API
基于 usage_logs/ops_error_logs 做分钟聚合与 5m/1h/12h/1d rollup,提供
snapshot/matrix/errors/users 等接口;V1 在 mode=v2 时停用主动探测,
并对用户端按需脱敏绝对量与 RPM/TPM。
2026-08-07 11:05:32 +08:00
Puppywang 02e50cc22d fix(security): block OAuth account takeover via pending exchange 2026-08-07 03:09:57 +08:00
shaw 8e102b3a0f fix: 完善腾讯验证码区域适配与 CSP 白名单
修复国内站和国际站 SDK 构造、验证容器、票据重置及动态资源加载问题,并补充认证流程回归测试。
2026-08-06 20:34:37 +08:00
wucm667 2f109e74ca fix: emit response.failed when compact keepalive commits headers but no SSE payload
When OpenAI Responses streaming has compact keepalive commit 200 headers
but no semantic SSE event has been written, upstream 4xx errors were
silently swallowed, causing clients to hang until timeout.

The fix distinguishes between 'headers committed' and 'meaningful SSE
output written'. Compact keepalive may commit headers for heartbeat
purposes without writing actual SSE events. In that case, the Responses
stream still needs its protocol-correct terminal response.failed event.

Preserves existing behavior when a complete upstream response or
meaningful SSE payload was already communicated, and preserves image
keepalive semantics.

Fixes #5289
2026-08-05 16:28:16 +08:00
feeeei 26e0a89323 人机验证增加阿里云验证码 2.0
沿用腾讯天御验证码引入的多服务商模型:aliyun_captcha_enabled 作为独立
开关,与 Cloudflare Turnstile、腾讯天御三方互斥(保存校验 + 运行时
CAPTCHA_PROVIDER_CONFLICT)。后台「安全与认证」合并为单张人机验证卡片:
总开关 + 服务商单选(Turnstile / 腾讯天御 / 阿里云),选中即启用该家并
关闭其它,落库仍是三个独立开关键,由前端映射保证互斥。

阿里云侧同时支持 aliyun 中国站与国际站(alibabacloud.com):两站前端脚本、
region 取值与服务端 API 完全一致,仅账号与 AccessKey 相互独立,因此由
「服务地域」决定线路即可——中国内地走 captcha.cn-shanghai.aliyuncs.com,
非中国内地(新加坡)走 captcha.ap-southeast-1.aliyuncs.com,AccessKey
取自持有该实例的账号,无需在配置中区分站点。

- AliyunCaptchaService 对称 TencentCaptchaService:服务端校验走官方 SDK
  VerifyIntelligentCaptcha,调用异常按 fail-closed 拦截,与 Turnstile
  网络错误行为对称;保存设置时真实探测 AK/SK 有效性
- 保护面对齐腾讯扩展入口:VerifyTencentCaptchaIfEnabled 通用化为
  VerifyActionCaptchaIfEnabled,OAuth 登录启动、passkey 登录在阿里云
  启用时同样拦截;Turnstile 维持既有覆盖不扩大
- 前端 AliyunCaptchaWidget 为表单内预验证按钮(popup 模式),同时暴露
  verify() 供 OAuth 启动、passkey 等动作入口程序化弹窗;未预验证直接
  提交时弹窗兜底。SDK 按钮绑定异步完成,弹窗未出现前按 tick 重试触发,
  并轮询弹窗可见性识别用户关闭
- captchaVerifyParam 复用 turnstile_token 请求字段提交;公开设置下发
  aliyun_captcha_enabled / scene_id / prefix / region
- CSP 放行验证码 CDN:script-src/style-src 加 *.alicdn.com
2026-08-04 20:57:15 +08:00