Compare commits

...

8 Commits

Author SHA1 Message Date
huangzd1997 457af3607f feat(user-secret): 密钥设置支持 LLM 渠道与代理供应商切换
- 用户密钥新增 channel 列(V180):LLM 行存渠道 longtaitou/t8star,代理行存供应商
  qingguo/jikip;存量与新建默认均回填 t8star/jikip(默认仍走贞贞与极客,手动可切)
- 外观专利/货源查询按任务归属用户的密钥渠道路由 LLM 主机(渠道经 SecretUsageContext
  下到工作线程,不逐层改签名);切到龙抬头时模型映射 gemini-3.1-flash-lite 并补
  enable_thinking + thinking_budget=medium(对齐贞贞现网口径),检测链路刻意不思考
- 代理检测按供应商解析:青果 code 为字符串、隧道端点支持 域名:端口、
  NO_AVAILABLE_CHANNEL 归为可重试;青果只验提取成功(账密只在客户端本地)
- 密钥设置页:每个模块加渠道分段选择、代理卡加供应商选择与青果 Authkey/Authpwd
  (只存本机,不上报),切换渠道未重填密钥时拦住保存;换号继承带渠道与供应商
- 后管密钥管理三列显示渠道/供应商标签
2026-10-30 04:00:38 +08:00
huangzd1997 eaab173ae9 fix(collect-routing): 后端整量 5 处红收口(身份出口/归属棘轮/中文文案基线)
全量 mvn -o test 抓到 3 个测试类 5 处失败,均因本次恢复链引入:
- queue-payload 恢复时缺 resolveOwnershipUserId ⇒ 身份出口判红;接上会话出口后
  boundRead 分母 63→64(不是放宽,是新增一条已接出口的 handler)
- 恢复的本地激活在 TaskService 里手抄 !MODULE_TYPE.equals(...) ⇒ 手写归属文件数 15→16;
  改为复用底座 requireSubmissionTask(模块校验口径一致,棘轮不涨)
- 相似ASIN activate 新增「已切换后台自动执行」文案 ⇒ 中文出口 61→62(已在常量注释登记)

mvn -o test:5032 例 / 0 失败 / 0 错误 / 19 skipped。
2026-10-29 21:38:31 +08:00
huangzd1997 4d38781005 feat(collect-routing): 前端按服务端开关分流「启动」到本地客户端或后台
- 新增 GET /api/collect-routing 封装与类型,两个工具页进页各读一次
- 外观专利页恢复本地推送链路(取队列载荷 → guard → activate → enqueue_json),
  开关关闭时退回「提交后台执行」;按钮与提示文案随开关切换
- 相似ASIN 开关关闭时只建单不推队列,等 Linux 后台认领
- queue-payload / activate 端点与 DTO 从 943cf2d62^ 原样恢复
- catch 棘轮 blocks 301→303:新增两支都有用户可见出口,silent 未变
2026-10-29 20:35:25 +08:00
huangzd1997 39c01ada44 feat(collect-routing): 专利/货源本地采集开关与 Linux 兜底窗口
外观专利与相似ASIN默认「本地采集」,由页面推本机客户端;本地推了没人领时
PENDING 超过 linux-fallback-minutes(默认 30 分钟)才允许 Linux 内部认领接管,
避免本地机全关时任务无限排队(PENDING 已不再按时间判死)。

- 新增 aiimage.collect-routing.local-enabled 与 GET /api/collect-routing
- 客户端兜底拉取只认本地模块;内部认领对本地模块按兜底窗口二次过滤
- 外观专利 activate 在本地模式下恢复 PENDING→RUNNING 条件更新(与认领互斥),
  关闭时保留后台拒绝语义;相似ASIN 关闭时明确拒绝「点启动」
- 恢复 GET /api/appearance-patent/tasks/{id}/queue-payload(本地推送取载荷用)
- /api/collect-routing 进用户态守卫名单,两处前缀计数基线 15→16 / 23→24
2026-10-29 20:27:53 +08:00
huangzd1997 8b7d5d4a40 fix(tasks): PENDING 排队不再按 2 小时判死,等客户端认领
PENDING 是排队态不是卡死态:单机串行跑大任务时队列等再久也必须保留,
按 updated_at 清杀会把排队任务误判成「长期未被领取」并连锁触发自动克隆重试,
同一份活被放大成多条失败记录(2026-10-28 现网 1514 行任务连续三代)。

判死巡检只留 SCHEDULED(到点未启动)与 RUNNING(心跳残留)两条;
排队任务一律等 TaskClientPullService 认领,页面放弃的解析预览也保留。
2026-10-29 19:19:21 +08:00
huangzd1997 217f5eec62 feat(phase-5): AC-19/AC-21 读路径时间窗与 keyset、心跳瘦读、菜单缓存、去 RAND、MDC 与补检限流
AC-19(P1-B1/B2/B3/B4)
- 任务中心(P1-B1):TaskCenterQueryService.tasks() 不带 startTime 时补 90 天默认窗口
  (startOfDefaultWindow),不再把 null 透传给 12 模块 UNION 扫全历史;selectTaskPage 的
  `LIMIT ? OFFSET ?` 改成 (createdAt,id) 游标,深翻页的 skip 挪进新语句
  selectTaskPageAnchor——它只投影 (id,createdAt) 两列。宽/窄两份 UNION 共用同一对筛选片段
  (taskFilterT/taskFilterB),锚点与页查询筛的行集在结构上不可能漂;页码语义、total、
  响应结构与前端全部不变。
- 关键词抽词(P1-B2):selectRandomUnused 不再 ORDER BY RAND()(全表 filesort)。
  服务层先 countHit 数命中集、再把窗口起点掷在 [0, total-limit],按 k.id 有序取一段后
  在内存洗牌。分布口径变了(连续 id 的一段 vs 逐行独立随机),已如实登记。
- 心跳(P1-B3):selectFileTask 只捞 id/moduleType/status/userId/updatedAt 五列;
  需要整实体的模块由新 SPI 默认方法 heartbeatNeedsFullTaskEntity()(默认 true=保守)
  声明后补一次主键读,collect-data/similar-asin/appearance-patent 三个只用
  task.getId() 的显式豁免。半行进 saveTaskCache 会写出坏缓存,比多读一次列难查得多。
  markInterrupted 改走整实体读(要 errorMessage 并同步模块缓存/快照)。
- 权限菜单(P1-B4):loadMenus 加 (companyId,menuType) 短 TTL(30s) 快照缓存,
  五处目录写路径 create/update/delete/reorder/copyTreeFromTemplate 全部写时失效;
  只缓存目录本体、不缓存用户授权行——撤权必须下一请求就生效。

AC-21(审计 §5/PERF §6)
- RequestTraceFilter 把已解析却只进日志参数的 requestId 写入 MDC(缺失写 "-"),
  finally 里 MDC.clear()(tomcat 线程复用,不清就把上一条的 id 带给下一条日志);
  logback-spring.xml 的两条 pattern 默认值 + application.yml 显式设的 file/console
  两条同批改带 [%X{requestId}]——本文件那两条会盖掉默认值,漏一处线上照样看不见。
- AppearancePatentDeliverySupport.requestRecheckForMissingRows 的补检轮 WARN 按任务限流:
  5 分钟窗口内首条 WARN(带 suppressedInLastWindow 汇总),其余降 DEBUG。刻意不吃
  recheckRounds/countRecheckRound 那对轮次计数——那是 scheduleFallbackRecheck 的补检预算,
  从日志路径再计一次会让"限流"吃掉业务预算、任务更早被判补检耗尽。
  "补检排不上队"那条不受限流(要人处置的信号不许压)。

契约激活与判据变更
- PendingPhase5ReadPathTest 5 例、PendingPhase5LogVolumeAndMdcTest 3 例全部去 @Disabled 转绿;
  后者预置的 logback 路径层级写错(mainSourcesRoot() 已是 src/main/java),激活时修正,
  并补一条 application.yml 的 pattern 判据(只增不减)。
- 新增 AppearancePatentRecheckWarnThrottleTest 3 例:钉实跑读数(6 轮只留 1 条 WARN + 5 条
  DEBUG、按任务隔离、排不上队仍出声),形状尺子之外的行为证据。
- TaskCenterQueryMapperSqlTest +2 例(锚点窄投影、游标收窄两分支)、
  TaskCenterQueryServiceTest +3 例(默认窗口、第一页不付锚点钱、页码越界返回空页)。
- 三处既有判据按新契约反转,一支没注释掉:selectTaskPage 的 `LIMIT ? OFFSET ?`→游标、
  分页参数绑定数 2→1/16→18/8→7;tasks() 三处 isNull()→notNull()(默认窗口生效本身成为断言)。
- 棘轮登记:SQL 分层语句总数 143→144(新语句点名),MigratedSqlEquivalenceTest
  XML_SELECT_TOTAL 99→100、LEDGER_ROWS 85→84、LEDGER_SELECTS 60→59——
  selectTaskPage 那一行整条摘出基准账本并点名进 NOT_MIGRATED,因为基准是"搬迁前的注解原文",
  改写它等于让等价性测试替新 SQL 背书。
- application.yml 走 hash-object+update-index 只提交本轮 pattern 那一格(4/2),
  另一路会话在飞的 client-task-pull 注释格原样留在工作树未提交。

读数:后端 mvn -o test 5,021 例/0 Failures/0 Errors/19 skipped(6:21 min,
较 Phase 4 的 5,013 净增 8 例=激活 8 例预置后新增 8 例判据);fat jar 实起,落盘行带出
`[main] INFO  [] c.nanri.aiimage.AiImageApplication - Starting...` 证明 MDC 位真在渲染;
本机无 MySQL ⇒ Hikari 反复 Starting、Tomcat 未监听,请求级 requestId 关联与现网四样本顺延 Phase 6。
2026-10-29 16:32:46 +08:00
huangzd1997 ef0bc51fa2 fix(phase-4): 分片通道两处界面文案过全仓尺子(英文原文只进日志)
`new BusinessException("初始化分片上传失败:" + ex.getMessage())` 与
`"拼装安装包失败:" + ex.getMessage()` 把底层磁盘/IO 的英文原文直接交给界面,
撞上全仓尺子 RawExceptionMessageSinkGuardTest(起因是 2026-10-09 那张货源查询截图:
卡片上的英文不是写错的文案,而是 error_message 里的异常原文)。两处改走
`UserMessages.with(中文主干, ex)`:详情是中文才拼接、英文丢弃,同时各补一行 WARN
把根因留在日志里(界面看不到,运维要找得到)。

为什么后端全量补跑才暴露:四面闸门的后端格是"按类点名"的格子册,
RawExceptionMessageGuardTest 不在册 ⇒ full 档绿也不等于整量绿,本笔由 `mvn -o test`
整量兜住(Tests run 5013/Failures 0/Errors 0/Skipped 27,8:26 min)。

复跑:RawExceptionMessageSinkGuardTest 2 例、UserFacingMessageChineseGuardTest 2 例、
ChunkedUploadServiceTest 8 例、WorkbenchClientPackageServiceTest 21 例、
CatchSilenceRatchetTest 9 例(catch 分类仍是 RETHROW,四数一格没动)全绿。
2026-10-29 14:15:52 +08:00
huangzd1997 0586df2df0 feat(phase-4): AC-16/17/18 前端登录态与分片上传、后管包体与竞态门
AC-16(S1-F1/S1-F2)
- frontend-vue ensureAuth:清态前先判 isAuthError——弱网/超时/5xx 不再把人踢回登录页;
  http.ts 的 onResponseRejected 给错误打鉴权标记(HTTP 401 与业务 401/4011 都算),
  纯网络错误保留底层文案供调用方区分"断网"与"掉登录";clearLocalSession 导出复用。
- admin 全局错误兜底:main.ts 装 app.config.errorHandler(首屏就崩也接得住),
  window error / unhandledrejection 由 App.vue setup 调 useGlobalErrorReporting() 接线,
  卸载出口在 onBeforeUnmount 摘监听与提示定时器(后管的 timer-cleanup-pairing 尺子要求
  "注册点所在文件找得回反注册 + 卸载钩子",故接线必须在组件作用域,不在 main.ts)。
  上报体只带 kind/message/stack/route/ts 并过脱敏(Bearer、JWT 三段式、query 里疑似
  token/key/secret 的取值一律打码),凭据零容忍;服务端新增
  /api/admin/client-errors(requireAdmin,字段截断 2000 后 WARN 落日志,不落库)。

AC-17(S1-F3)
- 后端新增分片通道 ChunkedUploadService(init/part/status/complete/abort):8MB/片、
  同片重复上传幂等覆盖、流式拼装复算 sha256、超限与缺片直接拒绝、24h 过期回收;
  FileUploadController 暴露 /api/files/chunk/**,LocalFileStorageService 加
  saveTempFileFromFile(同一套扩展名黑名单与 fileKey 索引);三键显式写进 application.yml
  (不进"只在代码默认值里"的名单)。
- 工作台安装包 93MB:WorkbenchClientPackageService 走同一分片通道转存 client 桶
  (objectKey 写进会话 purpose,拼装时逐字匹配,防借分片接口覆盖已发布包),
  confirm 失败带 cleanupUnconfirmedObject 补偿清理未登记对象;后管 ≤64MB 仍直传预签名 PUT。
- 用户端 upload.ts:大文件分片 + 逐片重试 + 进度回调,超时按体积给(不再固定 120s);
  断点探测失败不再静默按"全缺"重传(catch-silence 棘轮因此一格没涨,blocks 300→301 已记名)。

AC-18(P1-F1/P1-F2/P1-F4/S1-F4)
- 后管 manualChunks 摘掉 element-plus(按需引入已在 src/plugins/element.ts 生效):
  实测构建产物已无 element-plus 独立 chunk,index.html 只 modulepreload vue 块。
- UsersPage/DeviceLogsPage 接 createLatestRequestGate,旧响应不再覆盖新页;
  用户端新增 useServerTablePaging(只驻留当前页 + 自带竞态门 + 失败给原因),
  挂载用例 3 例钉住"翻页重取/旧响应作废/失败可见"。
- DeliveryVideoWorkspace 两处轮询接 document.visibilityState,隐藏标签页暂停。

激活 Phase 0 预置契约 7 例(用户端 5 + 后管 5,共 12 例;其中 AC-16 后管两支按"接线在组件作用域"
的口径改写并把判据加强到反注册配对 + 根组件接线,见蓝图 §5)。
门禁读数:后端 mvn -o test-compile rc=0,CatchSilenceRatchetTest 9 例绿(total 1034→1045、
notRethrow 585→591、logOnly 182→188 逐处记名,EMPTY 未涨)、ChunkedUploadServiceTest 8 例、
WorkbenchClientPackageServiceTest 21 例、ClientErrorReportControllerTest 4 例;后管 1848 例 +
mount 33 例 + vue-tsc + build 全绿;用户端 955 例 + mount 30 例 + vue-tsc + build 全绿。
真机:OssWorkbenchClientLiveVerificationTest 以线上 MinIO 实测 PUT→stat→sha256→presign→删除
全链 1 例绿(凭据只走环境变量,不落任何文件)。

⚠ 按 pathspec 只圈本轮文件:application.yml 用索引手术只提交分片三键那一格,他人
「PENDING 不再按 2 小时判死」的 StaleTaskRepairService/ClientTaskPullSpi/application.yml
注释件与 useLiveTaskBoard/BrandAppearancePatentTab/task-delete-target 四件在飞改动
原样留在工作树未提交。
2026-10-29 13:47:21 +08:00
116 changed files with 4983 additions and 378 deletions
+5
View File
@@ -1,6 +1,11 @@
<script setup lang="ts">
import { RouterView } from 'vue-router'
import zhCn from 'element-plus/es/locale/lang/zh-cn'
import { useGlobalErrorReporting } from '@/shared/api/error-report'
// 浏览器级错误兜底接线(S1-F2/AC-16):脚本级崩溃与没人接的 Promise 都进同一条上报路,
// 卸载出口由该 composable 的 onBeforeUnmount 摘掉——根组件不卸载时它就是应用级单例。
useGlobalErrorReporting()
</script>
<template>
@@ -5,6 +5,8 @@ import { unwrap } from './envelope'
export interface AdminUserSecretModule {
moduleKey: string
moduleLabel: string
/** 货源:LLM 行为渠道(longtaitou/t8star),代理行为供应商(qingguo/jikip)。 */
channel?: string | null
masked: string
/** 完整明文值:仅代理列有值(后端对密钥列不下发明文)。 */
full: string
+22
View File
@@ -1,14 +1,36 @@
import { createApp } from 'vue'
import { createPinia } from 'pinia'
import { installElement } from '@/plugins/element'
import { reportClientError } from '@/shared/api/error-report'
import '@/styles/main.css'
import App from './App.vue'
import router from './router'
// 全局错误兜底(S1-F2/AC-16):渲染期异常、未捕获 Promise、脚本级崩溃三类都必须
// 可见 + 上报,否则线上白屏只剩浏览器 console,运维永远没有现场。
// 出口分两处,各按自己的时机生效:
// - Vue 渲染/生命周期异常:挂在这里的 app.config.errorHandler(必须早于 mount,
// 首屏就崩也不漏);
// - window error 与 unhandledrejection:由 App.vue setup 调 useGlobalErrorReporting()
// 接线,卸载出口在 onBeforeUnmount 里摘监听与提示定时器(只定义不接线等于没接)。
// 上报本体(内存环 + 脱敏 + sendBeacon)在 @/shared/api/error-report。
function installGlobalError(app: ReturnType<typeof createApp>): void {
app.config.errorHandler = (error, _instance, info) => {
reportClientError({
kind: 'vue-render',
message: error instanceof Error ? error.message : String(error),
stack: error instanceof Error ? error.stack ?? '' : String(error),
route: router.currentRoute.value.fullPath,
ts: Date.now(),
})
}
}
// Element Plus 改为按需注册(见 src/plugins/element.ts);全局中文语言包改由
// App.vue 的 <el-config-provider :locale="zhCn"> 提供,不要再在这里 use(ElementPlus)。
const app = createApp(App)
app.use(createPinia())
app.use(router)
installElement(app)
installGlobalError(app)
app.mount('#app')
@@ -99,6 +99,22 @@ function moduleTooltip(module: AdminUserSecretModule | undefined) {
return parts.join(';') || '暂无检测记录'
}
/** 货源标签:LLM 密钥显示渠道(龙抬头/贞贞),代理显示供应商(青果/极客)。 */
function sourceLabel(channel: string | null | undefined) {
switch (String(channel || '').trim().toLowerCase()) {
case 'longtaitou':
return '龙抬头'
case 't8star':
return '贞贞'
case 'qingguo':
return '青果'
case 'jikip':
return '极客'
default:
return '—'
}
}
/** 分组列文案:优先分组名;未建分组的用户回退展示所属主管,便于判断归属。 */
function groupTextOf(row: AdminUserSecretRow) {
if (row.groups?.length) return row.groups.join('、')
@@ -311,6 +327,9 @@ onMounted(load)
<div class="module-cell">
<span v-if="row.similarAsin?.exists" class="mono-mask" :title="moduleTooltip(row.similarAsin)">{{ row.similarAsin.masked }}</span>
<span v-else class="empty-value">未配置</span>
<span v-if="row.similarAsin?.exists" class="wh-pill is-source" title="该用户这把密钥所属渠道">
{{ sourceLabel(row.similarAsin.channel) }}
</span>
<span class="wh-pill" :class="moduleStatusMeta(row.similarAsin).tone" :title="moduleTooltip(row.similarAsin)">
{{ moduleStatusMeta(row.similarAsin).label }}
</span>
@@ -320,6 +339,9 @@ onMounted(load)
<div class="module-cell">
<span v-if="row.appearancePatent?.exists" class="mono-mask" :title="moduleTooltip(row.appearancePatent)">{{ row.appearancePatent.masked }}</span>
<span v-else class="empty-value">未配置</span>
<span v-if="row.appearancePatent?.exists" class="wh-pill is-source" title="该用户这把密钥所属渠道">
{{ sourceLabel(row.appearancePatent.channel) }}
</span>
<span class="wh-pill" :class="moduleStatusMeta(row.appearancePatent).tone" :title="moduleTooltip(row.appearancePatent)">
{{ moduleStatusMeta(row.appearancePatent).label }}
</span>
@@ -330,6 +352,9 @@ onMounted(load)
<span v-if="row.proxy?.exists" class="mono-mask mono-full" :title="moduleTooltip(row.proxy)">{{ row.proxy.full || row.proxy.masked }}</span>
<span v-else class="empty-value">未配置</span>
<span class="module-cell-pills">
<span v-if="row.proxy?.exists" class="wh-pill is-source" title="该用户代理所属供应商">
{{ sourceLabel(row.proxy.channel) }}
</span>
<span class="wh-pill" :class="moduleStatusMeta(row.proxy).tone" :title="moduleTooltip(row.proxy)">
{{ moduleStatusMeta(row.proxy).label }}
</span>
@@ -627,6 +652,12 @@ h3 {
cursor: help;
flex-shrink: 0;
}
.wh-pill.is-source {
background: #eef2fb;
border-color: #ccd8f2;
color: #3d5a99;
font-weight: 600;
}
.wh-pill.is-allowed {
background: #e8f6ee;
border-color: #b9e0c9;
@@ -22,11 +22,14 @@ import EditUserDialog from './EditUserDialog.vue'
import OldPagination from '@/components/OldPagination.vue'
import { showAdminFeedback } from '@/components/admin-feedback-ui'
import { openAdminConfirm } from '@/components/admin-confirm'
import { createLatestRequestGate } from '@/utils/latest-request'
const session = useAdminSessionStore()
const companyTab = useCompanyTabStore()
const loading = ref(false)
const loadError = ref('')
// 用户列表竞态门(S1-F4):快速切公司/翻页只让最后一次响应落地
const gate = createLatestRequestGate()
const rows = ref<AdminUser[]>([])
const total = ref(0)
/** 所属管理员候选(来自列表响应的 admins),供超管筛选下拉使用。 */
@@ -69,10 +72,13 @@ function listParams() {
}
async function loadUsers() {
// S1-F4:快速切公司/翻页时旧响应可能后返回并覆盖新结果——回写前必须过竞态门
const token = gate.begin()
loading.value = true
loadError.value = ''
try {
const page = await fetchUserList(listParams())
if (!gate.isCurrent(token)) return
rows.value = page.items
total.value = page.total
if (Array.isArray(page.admins)) admins.value = page.admins
@@ -81,14 +87,16 @@ async function loadUsers() {
if (last > 0 && filters.page > last) {
filters.page = last
const retried = await fetchUserList(listParams())
if (!gate.isCurrent(token)) return
rows.value = retried.items
total.value = retried.total
}
} catch (error) {
if (!gate.isCurrent(token)) return
loadError.value = error instanceof Error ? error.message : '用户列表加载失败'
showAdminFeedback(loadError.value, 'error')
} finally {
loading.value = false
if (gate.isCurrent(token)) loading.value = false
}
}
@@ -6,6 +6,7 @@ import { computed, nextTick, onMounted, ref, watch } from 'vue'
import OldPagination from '@/components/OldPagination.vue'
import { ElMessage } from 'element-plus'
import { formatDateTime, orderedPair, todayDateInput } from '@/utils/datetime'
import { createLatestRequestGate } from '@/utils/latest-request'
import { useCompanyTabStore } from '@/stores/company-tab'
import { fetchHistoryUserOptions } from './history-api.ts'
import type { HistoryUserOption } from './history-user-option.ts'
@@ -57,6 +58,8 @@ const companyTab = useCompanyTabStore()
const loading = ref(false)
const loadFailure = ref('')
// 设备日志列表竞态门(S1-F4)
const gate = createLatestRequestGate()
const rows = ref<DeviceLogFileRow[]>([])
const total = ref(0)
const page = ref(1)
@@ -79,6 +82,8 @@ const retentionDays = ref(7)
const totalPages = computed(() => Math.max(1, Math.ceil(total.value / pageSize.value)))
async function load() {
// S1-F4:快速切公司/改筛选时旧响应可能晚到——回写前必须过竞态门
const token = gate.begin()
loading.value = true
loadFailure.value = ''
try {
@@ -91,14 +96,16 @@ async function load() {
pageSize: pageSize.value,
companyId: companyTab.requestCompanyId,
})
if (!gate.isCurrent(token)) return
rows.value = result?.items || []
total.value = Number(result?.total || 0)
if (result?.retentionDays) retentionDays.value = result.retentionDays
} catch (error) {
if (!gate.isCurrent(token)) return
loadFailure.value = error instanceof Error ? error.message : '日志列表加载失败'
showAdminFeedback(loadFailure.value, 'error')
} finally {
loading.value = false
if (gate.isCurrent(token)) loading.value = false
}
}
@@ -1,4 +1,4 @@
/** 工作台客户端包接口适配:列表 / 直传预签名 / 确认 / 删除。 */
/** 工作台客户端包接口适配:列表 / 直传预签名 / 分片上传 / 确认 / 删除。 */
import axios from 'axios'
import { http, unwrap } from '@/api/http'
import {
@@ -12,6 +12,13 @@ export const WORKBENCH_CLIENT_PACKAGE_ENDPOINT = '/api/admin/workbench-client/pa
export const WORKBENCH_CLIENT_PRESIGN_ENDPOINT = `${WORKBENCH_CLIENT_PACKAGE_ENDPOINT}/presign`
export const WORKBENCH_CLIENT_CONFIRM_ENDPOINT = `${WORKBENCH_CLIENT_PACKAGE_ENDPOINT}/confirm`
export const WORKBENCH_CLIENT_DELETE_ENDPOINT = `${WORKBENCH_CLIENT_PACKAGE_ENDPOINT}/delete`
/** 分片通道(S1-F3/AC-17):init/complete/abort 在工作台端点,分片体走通用 /api/files/chunk。 */
export const WORKBENCH_CLIENT_CHUNK_INIT_ENDPOINT = `${WORKBENCH_CLIENT_PACKAGE_ENDPOINT}/chunk/init`
export const WORKBENCH_CLIENT_CHUNK_COMPLETE_ENDPOINT = `${WORKBENCH_CLIENT_PACKAGE_ENDPOINT}/chunk/complete`
export const WORKBENCH_CLIENT_CHUNK_ENDPOINT = `${WORKBENCH_CLIENT_PACKAGE_ENDPOINT}/chunk`
/** confirm 失败的补偿清理:只删"已转存未登记"的对象(已登记后端拒绝)。 */
export const WORKBENCH_CLIENT_UNCONFIRMED_OBJECT_ENDPOINT = `${WORKBENCH_CLIENT_PACKAGE_ENDPOINT}/object`
export const FILES_CHUNK_ENDPOINT = '/api/files/chunk'
/** 客户端自动更新契约(后管页面不直接调用):GET 返回 {version,file_url,sha256,...}。 */
export const WORKBENCH_CLIENT_PUBLIC_LATEST_ENDPOINT = '/api/workbench-client/version/latest'
@@ -19,6 +26,15 @@ export const WORKBENCH_CLIENT_PUBLIC_LATEST_ENDPOINT = '/api/workbench-client/ve
/** MinIO 直传专用实例:不带 cookie、不挂 401 跳登录拦截器;大 zip 给足超时。 */
const directPut = axios.create({ timeout: 600_000, withCredentials: false })
/** 单片大小:8MB——93MB 包 ≈ 12 片,一片失败只重传 8MB,不再从 0 重来。 */
export const UPLOAD_PART_SIZE = 8 * 1024 * 1024
/** ≤64MB 仍走单次预签名直传(链路最短);再大改分片(S1-F3)。 */
export const DIRECT_PUT_MAX_BYTES = 64 * 1024 * 1024
/** 每片重试次数(不含首次):弱网下一片抖一下不必整单作废。 */
const PART_RETRY_LIMIT = 2
export async function fetchWorkbenchClientPackages(): Promise<WorkbenchClientPackageList> {
const { data } = await http.get<unknown>(WORKBENCH_CLIENT_PACKAGES_ENDPOINT)
return parseWorkbenchClientPackageList(data)
@@ -70,6 +86,35 @@ export async function confirmWorkbenchClientPackage(
return parseWorkbenchClientPackageUpload(data)
}
/**
* confirm 失败后的补偿清理:删除"已转存但未登记"的对象。
* 后端对已登记对象会拒绝清理(防止误删线上正在用的包),失败不覆盖原始错误。
* 返回是否确实清理成功——调用方据此决定报错文案是"可重传"还是"先刷新核对"。
*/
async function deleteUnconfirmedPackageObject(objectKey: string): Promise<boolean> {
try {
await http.delete(WORKBENCH_CLIENT_UNCONFIRMED_OBJECT_ENDPOINT, {
params: { object_key: objectKey },
})
return true
} catch {
return false
}
}
/**
* confirm 失败后的补偿清理(分片链):删会话 + 尽力删未登记对象。
*/
async function cleanupUnconfirmedPackage(uploadId: string, objectKey: string): Promise<void> {
try {
await http.delete(`${WORKBENCH_CLIENT_CHUNK_ENDPOINT}/${encodeURIComponent(uploadId)}`, {
params: objectKey ? { object_key: objectKey } : {},
})
} catch {
// 尽力而为:清理失败只说明残留等人工处置,不能替原始错误改口
}
}
/** 按 id 批量删除客户端包(后端会顺带清理无引用对象的 MinIO 包体);返回实际删除行数。 */
export async function deleteWorkbenchClientPackages(ids: number[]): Promise<number> {
if (!ids.length) return 0
@@ -78,17 +123,12 @@ export async function deleteWorkbenchClientPackages(ids: number[]): Promise<numb
return typeof core.deleted === 'number' ? core.deleted : ids.length
}
/**
* 上传客户端包(浏览器直传):presign 申请 → 直接 PUT 到 MinIO client 桶(不走 Java 中转)
* → confirm 由服务端复算 sha256 落库。onProgress 回传 0-100 上传进度。
*/
export async function uploadWorkbenchClientPackage(
/** 单次预签名 PUT 直传(≤DIRECT_PUT_MAX_BYTES 的短链路)。 */
async function directPutUpload(
file: Blob,
fileName: string,
version: string,
target: WorkbenchClientUploadTarget,
onProgress?: (percent: number) => void,
): Promise<WorkbenchClientPackageItem | null> {
const target = await requestWorkbenchClientPresign(fileName, version)
): Promise<void> {
await directPut.put(target.uploadUrl, file, {
headers: { 'Content-Type': 'application/octet-stream' },
onUploadProgress: (event) => {
@@ -97,11 +137,114 @@ export async function uploadWorkbenchClientPackage(
onProgress(total > 0 ? Math.min(Math.round((event.loaded / total) * 100), 100) : 0)
},
})
}
/**
* 分片上传(S1-F3/AC-17):整包切成 partSize 片逐片 POST,每片独立重试;
* 开传前先查已到片号——上一次断掉的会话可以直接续传(后端保留 24h)。
* 这一问失败不静默按「全缺」重传:93MB 的包问不到续传点就说明链路已经不对了,
* 当场报错让人重来,也比闷头把整包再传一遍便宜。
*/
async function chunkedUpload(
file: Blob,
uploadId: string,
totalParts: number,
onProgress?: (percent: number) => void,
): Promise<void> {
const { data } = await http.get<unknown>(`${FILES_CHUNK_ENDPOINT}/${encodeURIComponent(uploadId)}`)
const core = (unwrap(data) ?? {}) as Record<string, unknown>
const done = new Set<number>(
(Array.isArray(core.uploaded_parts) ? core.uploaded_parts : []).map((index) => Number(index)),
)
const report = (partial: number) => {
if (!onProgress) return
const loaded = done.size * UPLOAD_PART_SIZE + partial
onProgress(Math.min(Math.round((loaded / file.size) * 100), 99))
}
for (let index = 0; index < totalParts; index += 1) {
if (done.has(index)) {
report(0)
continue
}
const blob = file.slice(index * UPLOAD_PART_SIZE, Math.min(file.size, (index + 1) * UPLOAD_PART_SIZE))
let lastError: unknown = null
for (let attempt = 0; attempt <= PART_RETRY_LIMIT; attempt += 1) {
const form = new FormData()
form.append('file', blob, `part-${index}`)
try {
await http.post(`${FILES_CHUNK_ENDPOINT}/${encodeURIComponent(uploadId)}/part/${index}`, form, {
headers: { 'Content-Type': 'multipart/form-data' },
timeout: 300_000,
onUploadProgress: (event) => report(Math.min(event.loaded || 0, blob.size)),
})
done.add(index)
lastError = null
break
} catch (error) {
lastError = error
}
}
if (lastError) {
throw lastError instanceof Error ? lastError : new Error(`第 ${index} 片上传失败`)
}
}
if (onProgress) onProgress(100)
}
/**
* 上传客户端包:≤64MB 走浏览器直传预签名 PUT;大包走后端分片通道(每片可重试/续传,
* 服务端拼装后转存 client 桶)。两条链最后都收敛到 confirm:服务端复算 sha256、
* 凭据扫描、版本唯一性登记。confirm 失败清理未登记对象,不留孤儿。
*/
export async function uploadWorkbenchClientPackage(
file: Blob,
fileName: string,
version: string,
onProgress?: (percent: number) => void,
): Promise<WorkbenchClientPackageItem | null> {
if (file.size <= DIRECT_PUT_MAX_BYTES) {
const target = await requestWorkbenchClientPresign(fileName, version)
await directPutUpload(file, target, onProgress)
try {
return await confirmWorkbenchClientPackage(target.objectKey, target.fileName, target.version)
} catch (error) {
// confirm 失败先 cleanup 未登记对象再报错;服务端若已登记会拒绝清理(CR-08 幂等窗口),
// 那种情况保留原错误文案引导"先刷新列表",不冒充已处置。
const cleaned = await deleteUnconfirmedPackageObject(target.objectKey)
const reason = error instanceof Error ? error.message : String(error)
if (cleaned) {
throw new Error(`安装包已上传但登记未完成(${reason})。未登记对象已清理,可直接重新上传。`)
}
throw new Error(`安装包已上传,但确认未完成(${reason})。请先刷新列表确认,不要立即重复上传。`)
}
}
const totalParts = Math.max(1, Math.ceil(file.size / UPLOAD_PART_SIZE))
const { data } = await http.post<unknown>(WORKBENCH_CLIENT_CHUNK_INIT_ENDPOINT, null, {
params: { file_name: fileName, version, total_parts: totalParts, total_bytes: file.size },
})
const core = (unwrap(data) ?? {}) as Record<string, unknown>
const uploadId = typeof core.upload_id === 'string' ? core.upload_id : ''
const objectKey = typeof core.object_key === 'string' ? core.object_key : ''
if (!uploadId || !objectKey) {
throw new Error('后端未返回分片会话,请重试')
}
const confirmedName = typeof core.file_name === 'string' ? core.file_name : fileName
const confirmedVersion = typeof core.version === 'string' ? core.version : version
let assembled = false
try {
return await confirmWorkbenchClientPackage(target.objectKey, target.fileName, target.version)
await chunkedUpload(file, uploadId, totalParts, onProgress)
await http.post(WORKBENCH_CLIENT_CHUNK_COMPLETE_ENDPOINT, null, {
params: { upload_id: uploadId, object_key: objectKey, file_name: confirmedName, version: confirmedVersion },
timeout: 600_000,
})
assembled = true
return await confirmWorkbenchClientPackage(objectKey, confirmedName, confirmedVersion)
} catch (error) {
// 对象已经 PUT 成功,confirm 失败不等于上传失败;引导管理员先刷新列表,避免重复上传
await cleanupUnconfirmedPackage(uploadId, objectKey)
const reason = error instanceof Error ? error.message : String(error)
throw new Error(`安装包已上传,但确认未完成(${reason})。请先刷新列表确认,不要立即重复上传。`)
if (assembled) {
throw new Error(`安装包已转存但登记未完成(${reason})。未登记对象已清理,可直接重新上传。`)
}
throw error
}
}
@@ -0,0 +1,141 @@
/**
* 后管浏览器错误上报(S1-F2/AC-16)。
*
* 之前渲染崩溃/未捕获 Promise 只进 console,线上白屏没有现场。这里做三件事:
* 1. 屏上可见:右下角弹一条纯 DOM 提示(不依赖 element-plus,错误可能正是它引起的);
* 2. 内存环:最近 20 条留痕(同用户端口径),便于现场排查;
* 3. sendBeacon 上报服务端 `/api/admin/client-errors`(WARN 落日志)。
*
* 三类错误各有出口:Vue 渲染期异常由 `main.ts` 的 `app.config.errorHandler` 接(挂载前就要生效),
* `window error` 与未捕获 Promise 由 `useGlobalErrorReporting()` 接——它必须在根组件 setup 里调用,
* 卸载出口挂在 `onBeforeUnmount`:入口级注册点只定义不接线等于没接。
*
* 凭据零容忍:payload 只有 kind/message/stack/route/ts,**绝不**读取或携带本地登录令牌
* 或任何请求头凭据原文(AC-1 同一把尺;预签名 URL 也当一次性凭据对待,不进上报)。
*/
import { onBeforeUnmount } from 'vue'
const REPORT_ENDPOINT = '/api/admin/client-errors'
const RING_LIMIT = 20
/** 风暴保护:同一原因 30s 内只弹一次屏、上报次数也设上限,日志不许被一种错误刷爆。 */
const DEDUPE_WINDOW_MS = 30_000
const MAX_STACK_CHARS = 2000
export interface ClientErrorPayload {
kind: string
message: string
stack?: string
route?: string
ts?: number
}
const ring: ClientErrorPayload[] = []
const recent = new Map<string, number>()
/** 屏上提示的一次性摘除定时器:全部登记,卸载时逐个 clearTimeout,不留没人收的尾巴。 */
const toastTimers = new Set<number>()
function sanitize(text: string): string {
// 兜底脱敏:堆栈/消息里若混进凭据一律打码——Bearer 值、JWT 三段式(登录令牌的形状),
// 以及 query 里疑似 token/key/secret 的取值。上报通道绝不外带任何凭据原文。
return text
.replace(/(Bearer\s+)[A-Za-z0-9\-._~+/=]{8,}/gi, '$1***')
.replace(/[A-Za-z0-9\-_]{10,}\.[A-Za-z0-9\-_]{10,}\.[A-Za-z0-9\-_]{8,}/g, '***')
.replace(/([?&](?:token|key|secret|signature|expires)=)[^&\s"']+/gi, '$1***')
}
function fingerprint(payload: ClientErrorPayload): string {
return `${payload.kind}|${(payload.message || '').slice(0, 120)}`
}
function showOnce(payload: ClientErrorPayload): void {
if (typeof document === 'undefined') return
const now = Date.now()
const last = recent.get(fingerprint(payload)) ?? 0
if (now - last < DEDUPE_WINDOW_MS) return
recent.set(fingerprint(payload), now)
const toast = document.createElement('div')
toast.setAttribute('role', 'alert')
toast.textContent = `页面出错了:${(payload.message || '未知错误').slice(0, 120)}(已记录,刷新可恢复;反复出现请联系支持)`
toast.setAttribute(
'style',
'position:fixed;right:16px;bottom:16px;z-index:99999;max-width:420px;padding:10px 14px;'
+ 'background:#b3261e;color:#fff;border-radius:8px;font-size:13px;box-shadow:0 4px 16px rgba(0,0,0,.3)',
)
document.body.appendChild(toast)
const timer = window.setTimeout(() => {
toast.remove()
toastTimers.delete(timer)
}, 8000)
toastTimers.add(timer)
}
export function reportClientError(payload: ClientErrorPayload): void {
const entry: ClientErrorPayload = {
kind: payload.kind || 'unknown',
message: sanitize(String(payload.message ?? '')).slice(0, MAX_STACK_CHARS),
stack: sanitize(String(payload.stack ?? '')).slice(0, MAX_STACK_CHARS),
route: payload.route ?? (typeof location !== 'undefined' ? location.pathname : ''),
ts: payload.ts ?? Date.now(),
}
ring.push(entry)
if (ring.length > RING_LIMIT) ring.shift()
// eslint-disable-next-line no-console
console.error('[client-error]', entry)
showOnce(entry)
try {
if (typeof navigator !== 'undefined' && typeof navigator.sendBeacon === 'function') {
navigator.sendBeacon(REPORT_ENDPOINT, new Blob([JSON.stringify(entry)], { type: 'application/json' }))
}
} catch {
// 上报失败只丢现场,不许把错误处理本身炸掉
}
}
/** 最近 N 条内存环(只读快照,排查用)。 */
export function recentClientErrors(): ClientErrorPayload[] {
return ring.slice()
}
function handleWindowRejection(event: PromiseRejectionEvent): void {
reportClientError({
kind: 'unhandled-rejection',
message: String(event.reason ?? ''),
stack: event.reason instanceof Error ? event.reason.stack ?? '' : '',
route: typeof location === 'undefined' ? '' : location.pathname,
ts: Date.now(),
})
}
function handleWindowError(event: ErrorEvent): void {
reportClientError({
kind: 'window-error',
message: event.message || (event.error instanceof Error ? event.error.message : ''),
stack: event.error instanceof Error ? event.error.stack ?? '' : '',
route: typeof location === 'undefined' ? '' : location.pathname,
ts: Date.now(),
})
}
let browserEventsInstalled = false
/** 卸载出口:摘掉两个 window 监听 + 清掉所有还在倒计时的提示定时器。 */
export function disposeGlobalErrorReporting(): void {
if (typeof window === 'undefined') return
window.removeEventListener('unhandledrejection', handleWindowRejection)
window.removeEventListener('error', handleWindowError)
for (const timer of toastTimers) window.clearTimeout(timer)
toastTimers.clear()
browserEventsInstalled = false
}
/**
* 浏览器级兜底接线(必须由根组件 setup 调用一次):脚本级崩溃与没人接的 Promise
* 都收进同一条上报路;卸载配对走 onBeforeUnmount,与用户端 App.vue 的 dispose 口径一致。
*/
export function useGlobalErrorReporting(): void {
if (typeof window === 'undefined' || browserEventsInstalled) return
browserEventsInstalled = true
window.addEventListener('unhandledrejection', handleWindowRejection)
window.addEventListener('error', handleWindowError)
onBeforeUnmount(disposeGlobalErrorReporting)
}
@@ -25,21 +25,21 @@ const read = (rel: string) => readFileSync(resolve(root, rel), 'utf-8')
const PENDING_CHUNK = 'Pending Phase 4: AC-18 element-plus 不再进首屏 manualChunks'
const PENDING_RACE = 'Pending Phase 4: AC-18 列表响应必须过 latest-request 门'
test('pending_ac18_login_不再预载_element_plus_整块', { skip: PENDING_CHUNK }, () => {
test('pending_ac18_login_不再预载_element_plus_整块', () => {
const vite = read('vite.config.ts')
assert.doesNotMatch(vite, /['"]element-plus['"]\s*:\s*\[/,
'P1-F1:manualChunks 里的 element-plus 条目必须摘掉(按需引入已在 src/plugins/element.ts 生效)')
assert.match(vite, /manualChunks[\s\S]{0,200}vue/, 'vue 那一块仍应留在册,别顺手把护栏删空')
})
test('pending_ac18_按需引入护栏不许被这次改动带崩', { skip: PENDING_CHUNK }, () => {
test('pending_ac18_按需引入护栏不许被这次改动带崩', () => {
const plugin = read('src/plugins/element.ts')
assert.match(plugin, /installElement/, '按组件注册的正本入口必须还在')
assert.match(read('tests/perf-element-on-demand.test.ts'), /element-plus/,
'既有的按需引入尺子必须仍在册(摘 chunk 不等于回到全量引入)')
})
test('pending_ac18_用户页与设备日志页必须接竞态门', { skip: PENDING_RACE }, () => {
test('pending_ac18_用户页与设备日志页必须接竞态门', () => {
for (const rel of ['src/pages/account/UsersPage.vue', 'src/pages/records/DeviceLogsPage.vue']) {
const page = read(rel)
assert.match(page, /from '@\/utils\/latest-request'/,
@@ -26,17 +26,34 @@ const read = (rel: string) => readFileSync(resolve(root, rel), 'utf-8')
const PENDING_ERROR = 'Pending Phase 4: AC-16 全局错误兜底与脱敏上报'
const PENDING_UPLOAD = 'Pending Phase 4: AC-17 93MB 包分片续传与孤儿对象清理'
test('pending_ac16_admin_渲染异常必须有全局兜底', { skip: PENDING_ERROR }, () => {
test('pending_ac16_admin_渲染异常必须有全局兜底', () => {
const main = read('src/main.ts')
assert.match(main, /app\.config\.errorHandler\s*=/,
'S1-F2:渲染期异常必须被 Vue 全局兜底接住,不能只进 console')
assert.match(main, /addEventListener\(\s*['"]unhandledrejection['"]/,
// window error / unhandledrejection 两个出口的注册点从入口挪到了入口装的守卫模块:
// 后管的 timer-cleanup-pairing 尺子要求"注册点所在文件必须找得回反注册 + 卸载钩子",
// 而 main.ts 没有组件作用域可挂——只定义不接线等于没接,所以接线必须在根组件 setup 里。
// 判据强度没有降:三类错误照样都要接住,另外多卡"反注册配对"和"根组件真的接了线"。
const guard = read('src/shared/api/error-report.ts')
assert.match(guard, /addEventListener\(\s*['"]unhandledrejection['"]/,
'未捕获 Promise 同样要兜底(后管列表请求都是 promise 链)')
assert.match(main, /addEventListener\(\s*['"]error['"]/,
assert.match(guard, /addEventListener\(\s*['"]error['"]/,
'脚本级崩溃(window error)必须留痕')
assert.match(guard, /removeEventListener\(\s*['"]unhandledrejection['"]/,
'AC-16:注册了就要找得回反注册,否则热更新/宿主拆栈会叠监听')
assert.match(guard, /removeEventListener\(\s*['"]error['"]/,
'AC-16:window error 的反注册同样要点名')
assert.match(guard, /onBeforeUnmount\(/,
'AC-16:卸载出口必须挂在组件生命周期上(模块里空有一个 dispose 函数不算接上)')
const app = read('src/App.vue')
assert.match(app, /useGlobalErrorReporting\(\)/,
'AC-16:根组件必须真的调用接线函数')
assert.ok(main.includes('installGlobalError'), '入口仍须装 Vue 渲染兜底(首屏就崩也不能漏)')
})
test('pending_ac16_admin_上报必须挂在入口且不带凭据', { skip: PENDING_ERROR }, () => {
test('pending_ac16_admin_上报必须挂在入口且不带凭据', () => {
const main = read('src/main.ts')
assert.match(main, /installGlobalError|registerGlobalError|setupGlobalError/,
'兜底与上报必须挂在入口上(只写在页面里=有些页面崩了没人接)')
@@ -56,14 +73,14 @@ test('pending_ac16_admin_上报必须挂在入口且不带凭据', { skip: PENDI
'上报里不许带令牌原文(凭据零容忍,与 AC-1 同一把尺)')
})
test('pending_ac17_安装包必须分片续传并可观察进度', { skip: PENDING_UPLOAD }, () => {
test('pending_ac17_安装包必须分片续传并可观察进度', () => {
const api = read('src/pages/records/workbench-client-api.ts')
assert.match(api, /partSize|uploadPart|createMultipartUpload|chunk/,
'S1-F3:单次 PUT 一断就从 0 重来,必须改成分片/断点续传')
assert.match(api, /onProgress|progress/, '93MB 上传必须有可见进度')
})
test('pending_ac17_confirm失败必须清理对象', { skip: PENDING_UPLOAD }, () => {
test('pending_ac17_confirm失败必须清理对象', () => {
const api = read('src/pages/records/workbench-client-api.ts')
assert.match(api, /confirm[\s\S]{0,400}(abort|deleteObject|cleanup|remove)/,
'confirm 失败必须清对象(工作台包凭据零容忍:拒绝新包不等于旧对象已处置)')
+3 -1
View File
@@ -37,7 +37,9 @@ export default defineConfig({
output: {
manualChunks: {
vue: ['vue', 'vue-router', 'pinia'],
'element-plus': ['element-plus'],
// P1-F1:element-plus 不再钉成独立 chunk——按需引入已在 src/plugins/element.ts
// 生效,manualChunks 却把整包 915KB(gzip 293KB)又拽回登录页 modulepreload。
// 摘掉后组件按 import 图自然切块,进不了首屏。
},
},
},
@@ -112,6 +112,9 @@ public class AdminApiGuardFilter extends OncePerRequestFilter {
// items 与 result(app_client/amazon/add_product.py:973,1048),所以带 JWT 的
// 请求一旦被解析出身份就得按会话归属对齐,匿名执行器路径行为不变。
"/api/publish",
// 2026-10-29 采集路由开关读取口(GET /api/collect-routing):用户工具端点,
// 新建即入名单,没有历史匿名调用方(前端两个工具页登录后读取)。
"/api/collect-routing",
};
/**
@@ -0,0 +1,49 @@
package com.nanri.aiimage.config;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import java.util.LinkedHashMap;
import java.util.Locale;
import java.util.Map;
/**
* 采集执行方路由:哪些模块优先交给本地客户端、多久没人领才由 Linux 兜底。
*
* <p>2026-10-29 产品口径:外观专利(APPEARANCE_PATENT)与相似ASIN(SIMILAR_ASIN)默认
* 「本地采集」。本地优先不等于只认本地——页面推了没人领时,PENDING 超过
* {@code linux-fallback-minutes} 就交 Linux 后台认领,否则本地机器全关时任务会无限排队
* (PENDING 已不再按时间判死)。
*
* <p><b>未登记在 {@code local-enabled} 里的模块一律按「非本地」处理</b>,保持它们原有的
* 内部认领等待窗口({@code internal-min-pending-minutes})。默认值刻意不是 true:
* 开关只对点名的模块生效,新模块不会被悄悄卷进本地优先。
*/
@Data
@ConfigurationProperties(prefix = "aiimage.collect-routing")
public class CollectRoutingProperties {
/** 外观专利模块名(与 biz_file_task.module_type 同口径)。 */
public static final String MODULE_APPEARANCE_PATENT = "APPEARANCE_PATENT";
/** 相似ASIN(货源查询)模块名。 */
public static final String MODULE_SIMILAR_ASIN = "SIMILAR_ASIN";
/** 模块 → 是否本地采集;键为大写模块名。 */
private Map<String, Boolean> localEnabled = new LinkedHashMap<>();
/** 本地采集模块在 PENDING 超过多少分钟后允许 Linux 兜底认领。 */
private long linuxFallbackMinutes = 30L;
public boolean isLocal(String moduleType) {
if (moduleType == null || moduleType.isBlank()) {
return false;
}
return Boolean.TRUE.equals(localEnabled.get(moduleType.trim().toUpperCase(Locale.ROOT)));
}
/** 兜底窗口下限 1 分钟:配 0/负数等于关掉本地优先,不允许静默变成「Linux 立刻抢」。 */
public long safeLinuxFallbackMinutes() {
return Math.max(1L, linuxFallbackMinutes);
}
}
@@ -4,6 +4,6 @@ import org.springframework.boot.context.properties.EnableConfigurationProperties
import org.springframework.context.annotation.Configuration;
@Configuration
@EnableConfigurationProperties({OssProperties.class, TransientStorageProperties.class, StorageProperties.class, BrandProgressProperties.class, DeleteBrandProgressProperties.class, BrandCheckProperties.class, ZiniaoProperties.class, ModuleCleanupProperties.class, TaskPressureProperties.class, TaskImageCacheCleanupProperties.class, AppearancePatentProperties.class, SimilarAsinProperties.class, ImageVideoProperties.class, InstanceRoutingProperties.class, CapacityPlanProperties.class, UserSecretProperties.class, NotificationProperties.class, DeviceLogOssProperties.class, CollectDataBrandRecheckProperties.class, ClientCredentialProperties.class, CostGuardProperties.class, KeywordShareSyncProperties.class})
@EnableConfigurationProperties({OssProperties.class, TransientStorageProperties.class, StorageProperties.class, BrandProgressProperties.class, DeleteBrandProgressProperties.class, BrandCheckProperties.class, ZiniaoProperties.class, ModuleCleanupProperties.class, TaskPressureProperties.class, TaskImageCacheCleanupProperties.class, AppearancePatentProperties.class, SimilarAsinProperties.class, ImageVideoProperties.class, InstanceRoutingProperties.class, CapacityPlanProperties.class, UserSecretProperties.class, NotificationProperties.class, DeviceLogOssProperties.class, CollectDataBrandRecheckProperties.class, ClientCredentialProperties.class, CostGuardProperties.class, KeywordShareSyncProperties.class, CollectRoutingProperties.class})
public class PropertiesConfig {
}
@@ -7,6 +7,7 @@ import jakarta.servlet.http.HttpServletResponse;
import java.util.Locale;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;
@@ -18,6 +19,13 @@ import java.util.Optional;
@Component
public class RequestTraceFilter extends OncePerRequestFilter {
/**
* AC-21(审计 §5.2):requestId 早在 2026-09 就被解析出来了,却只塞进这一行日志的参数里,
* 全仓没有一处 {@code MDC.put} ⇒ 一个请求里另外 570 处日志互相认不出彼此。
* 入 MDC 之后,logback 的 {@code %X{requestId}} 把整条请求链缝起来(含异步 appender 落盘)。
*/
static final String MDC_REQUEST_ID = "requestId";
private static final Logger log = LoggerFactory.getLogger(RequestTraceFilter.class);
private static final int MIN_REQUEST_BODY_CACHE_LIMIT_BYTES = 1024 * 1024;
@@ -55,6 +63,9 @@ public class RequestTraceFilter extends OncePerRequestFilter {
response.setHeader("X-AIIMAGE-Instance-Source", instanceMetadata.getSource());
response.setHeader("X-AIIMAGE-Instance-Stable", String.valueOf(instanceMetadata.isStable()));
// tomcat 线程是复用的:不清 MDC 的话,下一条请求(哪怕没有 requestId)会带上上一条的 id,
// 按请求关联反而变成按请求串案。缺失也写 "-",让 pattern 里这一位永远存在、格式不抖。
MDC.put(MDC_REQUEST_ID, blankToDash(requestId));
try {
filterChain.doFilter(requestToUse, response);
} finally {
@@ -96,6 +107,7 @@ public class RequestTraceFilter extends OncePerRequestFilter {
costMs
);
}
MDC.clear();
}
}
@@ -15,6 +15,7 @@ import com.nanri.aiimage.modules.appearancepatent.service.support.AppearancePate
import com.nanri.aiimage.modules.brand.client.BrandCheckClient;
import com.nanri.aiimage.modules.usersecret.service.UserSecretUsageService;
import com.nanri.aiimage.modules.usersecret.support.SecretUsageContext;
import com.nanri.aiimage.modules.usersecret.support.LlmChannel;
import com.nanri.aiimage.modules.usersecret.support.UserSecretModule;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
@@ -668,7 +669,7 @@ public class AppearancePatentLlmClient {
String responseFormat) throws Exception {
int attempts = Math.max(1, properties.getLlmRetryTimes());
long cooldownMillis = Math.max(0, properties.getLlmUnreachableCooldownMillis());
String host = properties.getLlmHost();
String host = resolveHost();
if (unreachableCooldown.isCoolingDown(host, cooldownMillis)) {
// 冷却期快失败不发请求、不记密钥次数(B24)
log.debug("[appearance-patent] 客户端级冷却中,快速失败 model={} window={}ms", model, cooldownMillis);
@@ -751,11 +752,17 @@ public class AppearancePatentLlmClient {
int attempt) {
recordSecretUsage();
Map<String, Object> body = buildChatBody(model, system, userText, images, responseFormat);
log.info("[appearance-patent] llm request attempt={} model={} url={} body={}",
attempt, model, joinUrl(properties.getLlmHost(), "/v1/chat/completions"),
String host = resolveHost();
// 渠道模型映射:龙抬头那边的同款模型叫 gemini-3.1-flash-lite,直接发贞贞的模型名会 model_not_found
String requestModel = LlmChannel.modelForHost(host, model);
body.put("model", requestModel);
// 思考参数按渠道补:贞贞把 thinking 写在模型名里,龙抬头要显式传(复刻现网的 thinking-medium)
LlmChannel.applyThinkingForHost(host, body);
log.info("[appearance-patent] llm request attempt={} model={} channelModel={} url={} body={}",
attempt, model, requestModel, joinUrl(host, "/v1/chat/completions"),
writeJson(maskChatBody(body)));
RestClient.RequestBodySpec request = restClient(attempt).post()
.uri(joinUrl(properties.getLlmHost(), "/v1/chat/completions"))
.uri(joinUrl(host, "/v1/chat/completions"))
.headers(headers -> {
headers.setBearerAuth(stripBearer(apiKey));
headers.setContentType(APPLICATION_JSON_UTF8);
@@ -1505,6 +1512,12 @@ public class AppearancePatentLlmClient {
return normalized.regionMatches(true, 0, "Bearer ", 0, 7) ? normalized.substring(7).trim() : normalized;
}
/** 渠道主机:优先取任务上下文里按用户密钥渠道解析出的主机(龙抬头/贞贞),否则回落模块全局配置。 */
private String resolveHost() {
String contextual = SecretUsageContext.currentLlmHost();
return contextual == null ? properties.getLlmHost() : contextual;
}
private String joinUrl(String baseUrl, String path) {
String base = baseUrl == null ? "" : baseUrl.trim();
String suffix = path == null ? "" : path.trim();
@@ -4,6 +4,7 @@ import com.nanri.aiimage.common.api.ApiResponse;
import com.nanri.aiimage.common.api.PageDefaults;
import com.nanri.aiimage.common.security.AdminAuthSupport;
import com.nanri.aiimage.modules.appearancepatent.model.dto.AppearancePatentParseRequest;
import com.nanri.aiimage.modules.appearancepatent.model.dto.AppearancePatentParsedPayloadDto;
import com.nanri.aiimage.modules.appearancepatent.model.dto.AppearancePatentSubmitResultRequest;
import com.nanri.aiimage.modules.appearancepatent.model.dto.AppearancePatentTaskBatchRequest;
import com.nanri.aiimage.modules.appearancepatent.model.vo.AppearancePatentDashboardVo;
@@ -57,6 +58,18 @@ public class AppearancePatentController {
adminAuthSupport.currentDeviceId(httpRequest)));
}
@GetMapping("/tasks/{taskId}/queue-payload")
@Operation(summary = "获取外观专利 Python 队列载荷", description = "本地采集模式下,前端取出 groups 后推本机客户端队列;只返回 Python 当前需要的分组,避免前端搬运完整解析大数据。")
public ApiResponse<AppearancePatentParsedPayloadDto> queuePayload(
@Parameter(description = "外观专利任务 ID", required = true, example = "7004")
@PathVariable Long taskId,
@Parameter(description = "当前用户 ID", required = true, example = "1")
@RequestParam("user_id") Long userId,
HttpServletRequest httpRequest) {
userId = adminAuthSupport.resolveOwnershipUserId(httpRequest, userId);
return ApiResponse.success(service.queuePayload(taskId, userId));
}
@GetMapping("/dashboard")
@Operation(summary = "查询外观专利检测总览", description = "查询当前用户的运行中、成功、失败和已结束任务数量。页面进入时请求一次即可,不需要持续轮询。")
public ApiResponse<AppearancePatentDashboardVo> dashboard(
@@ -50,6 +50,12 @@ public class AppearancePatentTaskHeartbeatSpi implements TaskModuleHeartbeatSpi
task.getId(), moduleType(), jobTouchIntervalMillis());
}
/** 只把 {@code task.getId()} 交给作业续期,不写模块缓存 ⇒ 心跳可以吃瘦读(AC-19/P1-B3)。 */
@Override
public boolean heartbeatNeedsFullTaskEntity() {
return false;
}
private long jobTouchIntervalMillis() {
return Math.max(1_000L, appearancePatentProperties.getDbJobTouchIntervalMillis());
}
@@ -74,6 +74,7 @@ import com.nanri.aiimage.modules.task.model.vo.TaskProgressLightBatchVo;
import com.nanri.aiimage.modules.task.service.TaskProgressLightAssembler;
import com.nanri.aiimage.modules.task.service.pipeline.AbstractFileTaskPipeline;
import com.nanri.aiimage.modules.usersecret.service.UserApiSecretService;
import com.nanri.aiimage.modules.usersecret.support.UserSecretModule;
@Service
@Slf4j
@@ -113,6 +114,13 @@ public class AppearancePatentTaskService extends AbstractFileTaskPipeline<Appear
@org.springframework.beans.factory.annotation.Autowired(required = false)
private com.nanri.aiimage.modules.usersecret.service.UserTaskGuardService taskGuard;
/**
* 采集路由开关:字段注入的理由同 {@code taskGuard}——本类构造器被 golden 用例按参数顺序
* new 出来,加参数会连带改一批测试。null=未注入(单测)时按「非本地」处理,退回后台语义。
*/
@org.springframework.beans.factory.annotation.Autowired(required = false)
private com.nanri.aiimage.config.CollectRoutingProperties collectRoutingProperties;
/**
* 参数顺序=迁移前 {@code @RequiredArgsConstructor} 的字段声明顺序,一字未动;
* 其中五枚(fileTaskMapper/fileResultMapper/taskDistributedLockService/
@@ -204,9 +212,45 @@ public class AppearancePatentTaskService extends AbstractFileTaskPipeline<Appear
throw new com.nanri.aiimage.common.exception.BusinessException(com.nanri.aiimage.modules
.usersecret.service.UserTaskGuardService.PAUSED_MESSAGE);
}
// 2026-10-29 本地采集开关:开着就恢复「页面推本地客户端」的启动语义;
// 关着走原后台语义(记录支持里对 PENDING 明确报「已切换后台自动执行」)。
if (collectRoutingProperties != null
&& collectRoutingProperties.isLocal(com.nanri.aiimage.config.CollectRoutingProperties.MODULE_APPEARANCE_PATENT)) {
activateForLocalClient(taskId, userId);
return;
}
recordSupport().activateTask(taskId, userId);
}
/**
* 本地采集模式的激活:只允许 PENDING→RUNNING 条件更新。
*
* <p>与客户端兜底拉取 / Linux 内部认领共用同一条原子翻转,谁先翻谁执行,
* 页面推送与后台接管不会重复跑同一份任务。
*/
private void activateForLocalClient(Long taskId, Long userId) {
// 模块归属走底座 requireSubmissionTask(TaskFinalizeSupport 家族),不在本类手抄
// 「!MODULE_TYPE.equals(...)」——那形已被 TaskLifecycleDebtRatchetTest 钉成只降不升。
FileTaskEntity task = requireSubmissionTask(taskId);
if (userId == null || !java.util.Objects.equals(userId, task.getUserId())) {
throw new com.nanri.aiimage.common.exception.BusinessException("任务不存在");
}
ensureTaskOwnedByCurrentInstance(task, "activate");
int updated = fileTaskMapper.update(null, new LambdaUpdateWrapper<FileTaskEntity>()
.eq(FileTaskEntity::getId, taskId)
.eq(FileTaskEntity::getStatus, com.nanri.aiimage.common.task.TaskStatuses.FileTask.PENDING)
.set(FileTaskEntity::getStatus, STATUS_RUNNING)
.set(FileTaskEntity::getUpdatedAt, LocalDateTime.now()));
if (updated == 0) {
FileTaskEntity latest = fileTaskMapper.selectById(taskId);
if (latest != null && STATUS_RUNNING.equals(latest.getStatus())) {
throw new com.nanri.aiimage.common.exception.BusinessException("任务已在执行中(可能已由客户端自动接管),无需重复启动");
}
throw new com.nanri.aiimage.common.exception.BusinessException("任务已结束");
}
taskCacheService.touchTaskHeartbeat(taskId);
}
public AppearancePatentDashboardVo dashboard(Long userId) {
return progressSupport().dashboard(userId);
}
@@ -367,7 +411,9 @@ public class AppearancePatentTaskService extends AbstractFileTaskPipeline<Appear
properties, llmClient, taskCacheService, taskFileJobService,
rowReader()::rowKey, this::shouldInspectRow, rowReader()::readChunkRows, this::loadSubmittedChunks,
this::isResultSubmissionComplete, this::touchJavaSideTaskActivity,
llmSubmission::readAiPrompt, llmSubmission::readApiKey, this::mergeChunkPayload, this::firstNonBlank,
llmSubmission::readAiPrompt, llmSubmission::readApiKey,
task -> userApiSecretService.resolveLlmHost(task.getUserId(), UserSecretModule.APPEARANCE_PATENT),
this::mergeChunkPayload, this::firstNonBlank,
this::terminalizeTaskOnFatalLlmFailure);
}
@@ -17,6 +17,8 @@ import lombok.extern.slf4j.Slf4j;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicLong;
/**
* 外观专利的交付闸门与补检(E 聚类之一),从 {@code AppearancePatentTaskService} <b>原样搬移</b>
@@ -30,6 +32,20 @@ public class AppearancePatentDeliverySupport {
private static final String STATUS_FAILED = TaskStatuses.FileTask.FAILED;
/**
* AC-21(审计 §5、PERF_REPORT §6):交付缺口补检 WARN 的按任务限流窗口。
*
* <p>现网实录 task 32609/job 23345 三分钟 478 条(≈9,560 条/小时):结果文件作业每 15s
* 被 claim 一轮,同一批缺口行每轮原样再报一次,字段一字不差,日志里没有新信息,只有量。
*
* <p>本类按调用现取(不是 Spring bean,{@code AppearancePatentTaskService#deliverySupport()}
* 每次 new 一只),所以限流态只能挂静态:一个 JVM 一份,两台实例各打各的,不互相吞计数。
* 条目数由 {@link #WARN_TASKS_MAX} 兜住,超了就先把窗口已过期的清出去。
*/
static final long RECHECK_WARN_WINDOW_MILLIS = 300_000L;
static final int WARN_TASKS_MAX = 1024;
private static final ConcurrentHashMap<Long, RecheckWarnWindow> RECHECK_WARN_WINDOWS = new ConcurrentHashMap<>();
private final String moduleType;
private final AppearancePatentProperties properties;
private final AppearancePatentTaskCacheService taskCacheService;
@@ -120,8 +136,7 @@ public class AppearancePatentDeliverySupport {
boolean requeued = taskFileJobService.requeue(job.getId(),
"有 " + facts.missingRows() + " 行未完成检测,补检一轮后重新组装结果文件");
if (requeued) {
log.warn("[appearance-patent] 交付缺口触发补检轮 taskId={} jobId={} missingRows={} keys={} noDataRows={}",
task.getId(), job.getId(), facts.missingRows(), keys.size(), facts.noDataRows());
throttleRecheckWarning(task, job, facts);
return true;
}
log.warn("[appearance-patent] 交付缺口补检排不上队,转失败兜底出口按补检预算处置 taskId={} jobId={} missingRows={}",
@@ -129,6 +144,47 @@ public class AppearancePatentDeliverySupport {
throw new BusinessException("有 " + facts.missingRows() + " 行未完成检测,等待补检轮");
}
/**
* 补检轮 WARN 的限流出口(AC-21):同一任务在一个窗口内只留一条 WARN,窗口里被压掉的条数
* 随下一条 WARN 一起汇总出去({@code suppressedInLastWindow}),窗口内的重复轮次降 DEBUG。
*
* <p>刻意不去碰 {@code taskCacheService.recheckRounds/countRecheckRound} 那对轮次计数:
* 那是 {@link #scheduleFallbackRecheck} 的补检预算({@code recheckMaxRounds})。从这条路径
* 再计一次,会让「日志限流」吃掉业务预算,任务更早被判「补检已耗尽」——那是行为变更,不是治理。
*
* <p>并发口径:读-判-写三步之间有竞争,最坏情况是同窗口多打一条 WARN。限流是为了压日志量,
* 不是为了恰好一条,不值得为它加锁或把整张表串行化。
*/
private void throttleRecheckWarning(FileTaskEntity task, TaskFileJobEntity job,
AppearancePatentRowOutcome.Facts facts) {
long now = System.currentTimeMillis();
RecheckWarnWindow current = RECHECK_WARN_WINDOWS.get(task.getId());
if (current != null && now - current.startedAtMillis < RECHECK_WARN_WINDOW_MILLIS) {
log.debug("[appearance-patent] 交付缺口触发补检轮 taskId={} jobId={} missingRows={} keys={}(限流:窗口内第 {} 条)",
task.getId(), job.getId(), facts.missingRows(), facts.missingKeys().size(),
current.suppressed.incrementAndGet());
return;
}
log.warn("[appearance-patent] 交付缺口触发补检轮 taskId={} jobId={} missingRows={} keys={} noDataRows={} suppressedInLastWindow={}",
task.getId(), job.getId(), facts.missingRows(), facts.missingKeys().size(),
facts.noDataRows(), current == null ? 0L : current.suppressed.get());
if (RECHECK_WARN_WINDOWS.size() >= WARN_TASKS_MAX) {
RECHECK_WARN_WINDOWS.entrySet().removeIf(entry ->
now - entry.getValue().startedAtMillis >= RECHECK_WARN_WINDOW_MILLIS);
}
RECHECK_WARN_WINDOWS.put(task.getId(), new RecheckWarnWindow(now));
}
/** 一个任务在一个限流窗口里的补检告警态:窗口起始时刻 + 窗口内被压到 DEBUG 的条数。 */
private static final class RecheckWarnWindow {
private final long startedAtMillis;
private final AtomicLong suppressed = new AtomicLong();
private RecheckWarnWindow(long startedAtMillis) {
this.startedAtMillis = startedAtMillis;
}
}
/**
* 只量行账、不出表。补检的决定必须做在 {@code assembleResultWorkbook} 之前,而那一跑完就把
* 成品传上 OSS 并挂好了 {@code resultFileUrl}。判据与出表共用
@@ -81,6 +81,8 @@ public class AppearancePatentLlmBatchPipeline {
private final Consumer<Long> taskActivityToucher;
private final Function<FileTaskEntity, String> promptReader;
private final Function<FileTaskEntity, String> apiKeyReader;
/** 按任务归属用户密钥的渠道解析 LLM 主机(龙抬头/贞贞);为空表示走模块全局配置。 */
private final Function<FileTaskEntity, String> apiHostReader;
private final ChunkRowMerger chunkRowMerger;
private final BiFunction<String, String, String> firstNonBlankFn;
private final FatalLlmFailureHandler fatalLlmFailureHandler;
@@ -97,6 +99,7 @@ public class AppearancePatentLlmBatchPipeline {
Consumer<Long> taskActivityToucher,
Function<FileTaskEntity, String> promptReader,
Function<FileTaskEntity, String> apiKeyReader,
Function<FileTaskEntity, String> apiHostReader,
ChunkRowMerger chunkRowMerger,
BiFunction<String, String, String> firstNonBlankFn,
FatalLlmFailureHandler fatalLlmFailureHandler) {
@@ -112,6 +115,7 @@ public class AppearancePatentLlmBatchPipeline {
this.taskActivityToucher = taskActivityToucher;
this.promptReader = promptReader;
this.apiKeyReader = apiKeyReader;
this.apiHostReader = apiHostReader;
this.chunkRowMerger = chunkRowMerger;
this.firstNonBlankFn = firstNonBlankFn;
this.fatalLlmFailureHandler = fatalLlmFailureHandler;
@@ -130,6 +134,8 @@ public class AppearancePatentLlmBatchPipeline {
}
String prompt = promptReader.apply(task);
String apiKey = apiKeyReader.apply(task);
// 渠道主机在任务级解析一次:同一任务的批次都发往同一个中转(密钥属于谁就发谁那家)
String apiHost = apiHostReader == null ? null : apiHostReader.apply(task);
int batchSize = Math.max(1, properties.getLlmBatchSize());
ChunkRowKeyIndex rowKeyIndex = new ChunkRowKeyIndex();
List<AppearancePatentResultRowDto> candidates = terminalizeRoundExhaustedRows(
@@ -177,7 +183,7 @@ public class AppearancePatentLlmBatchPipeline {
boolean batchAnswered = true;
try {
llmRows = SecretUsageContext.call(task.getUserId(), UserSecretModule.APPEARANCE_PATENT.key(),
() -> llmClient.inspectRows(batchRows, prompt, apiKey));
apiHost, () -> llmClient.inspectRows(batchRows, prompt, apiKey));
} catch (Exception ex) {
String message = firstNonBlank(ex.getMessage(), "LLM 检测失败");
if (AppearancePatentResultJobSupport.isFatalLlmArrearsFailure(message)) {
@@ -0,0 +1,60 @@
package com.nanri.aiimage.modules.clienterror.controller;
import com.nanri.aiimage.common.api.ApiResponse;
import com.nanri.aiimage.common.model.entity.AdminUserEntity;
import com.nanri.aiimage.common.security.AdminAuthSupport;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.servlet.http.HttpServletRequest;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.Map;
/**
* 管理端浏览器错误上报(S1-F2/AC-16)。
*
* <p>此前后管页面渲染崩溃/未捕获 Promise 只进浏览器 console,线上白屏永远没有现场。
* 本端点把上报落进服务端日志(WARN,带 uid/route 前缀可按 requestId grep)。
* 只记纯文本且逐字段截断,不落库、不进队列——这是一份"看得见"的证据链,不是新数据集。</p>
*
* <p>鉴权:走 {@code /api/admin/**} 的管理员会话(sendBeacon 带同源 cookie)。
* 报文体里禁止出现令牌等凭据(上报端已做脱敏,这里再卡一道长度上限)。</p>
*/
@Slf4j
@RestController
@RequiredArgsConstructor
@RequestMapping("/api/admin/client-errors")
@Tag(name = "管理端错误上报", description = "后管页面全局错误兜底的上报出口:落服务端日志,不落库")
public class ClientErrorReportController {
private static final int MAX_FIELD_CHARS = 2000;
private final AdminAuthSupport adminAuthSupport;
@PostMapping
@Operation(summary = "上报一条浏览器错误", description = "字段:kind/message/stack/route/ts;全部截断后以 WARN 落服务端日志")
public ApiResponse<Map<String, Object>> report(
HttpServletRequest request,
@RequestBody(required = false) Map<String, Object> payload) {
AdminUserEntity operator = adminAuthSupport.requireAdmin(request);
Map<String, Object> body = payload == null ? Map.of() : payload;
log.warn("[client-error] uid={} kind={} route={} ts={} message={} stack={}",
operator.getId(),
clip(body.get("kind")),
clip(body.get("route")),
body.get("ts"),
clip(body.get("message")),
clip(body.get("stack")));
return ApiResponse.success(Map.of("recorded", true));
}
static String clip(Object value) {
String text = value == null ? "" : String.valueOf(value);
return text.length() <= MAX_FIELD_CHARS ? text : text.substring(0, MAX_FIELD_CHARS) + "…";
}
}
@@ -25,4 +25,13 @@ public class CollectDataTaskHeartbeatSpi implements TaskModuleHeartbeatSpi {
public void touchHeartbeat(Long taskId, TaskHeartbeatRequest request) {
collectDataService.updateProgress(taskId, request);
}
/**
* 进度写走 {@code updateProgress(taskId, request)} 自己按主键取数,心跳不需要整实体
* (AC-19/P1-B3:采集任务的心跳最密,两列大 JSON 每 30s 捞一次纯属白付)。
*/
@Override
public boolean heartbeatNeedsFullTaskEntity() {
return false;
}
}
@@ -3,6 +3,7 @@ package com.nanri.aiimage.modules.file.controller;
import com.nanri.aiimage.common.api.ApiResponse;
import com.nanri.aiimage.modules.file.model.vo.ExcelInfoVo;
import com.nanri.aiimage.modules.file.model.vo.UploadFileVo;
import com.nanri.aiimage.modules.file.service.ChunkedUploadService;
import com.nanri.aiimage.modules.file.service.LocalFileStorageService;
import com.nanri.aiimage.modules.file.service.oss.OssStorageService;
import io.swagger.v3.oas.annotations.Operation;
@@ -13,14 +14,20 @@ import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponses;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RequestPart;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import java.io.File;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
@RestController
@RequiredArgsConstructor
@@ -30,6 +37,7 @@ public class FileUploadController {
private final LocalFileStorageService localFileStorageService;
private final OssStorageService ossStorageService;
private final ChunkedUploadService chunkedUploadService;
@PostMapping("/upload")
@Operation(
@@ -72,6 +80,92 @@ public class FileUploadController {
return ApiResponse.success(vo);
}
@PostMapping("/chunk/init")
@Operation(
summary = "初始化分片上传",
description = """
大文件分片上传的第一步:登记文件大小与分片数,返回 uploadId。
分片逐个 POST 到 /chunk/{uploadId}/part/{index},全部到齐后调
/chunk/{uploadId}/complete 拼装为普通临时文件(同一套 fileKey 语义)。
每片可独立重试,前端也可先查 /chunk/{uploadId} 拿到已到片号续传。
""")
public ApiResponse<Map<String, Object>> initChunkUpload(
@RequestParam String fileName,
@RequestParam int totalParts,
@RequestParam long totalBytes) {
ChunkedUploadService.ChunkSession session =
chunkedUploadService.init(fileName, totalParts, totalBytes, "");
Map<String, Object> result = new LinkedHashMap<>();
result.put("upload_id", session.uploadId());
result.put("total_parts", session.totalParts());
return ApiResponse.success(result);
}
@PostMapping("/chunk/{uploadId}/part/{index}")
@Operation(summary = "上传一个分片", description = "同一片重复上传按覆盖处理(重试/续传幂等)。")
public ApiResponse<Map<String, Object>> uploadChunk(
@PathVariable String uploadId,
@PathVariable int index,
@RequestPart("file") MultipartFile file) throws Exception {
int received = chunkedUploadService.savePart(uploadId, index, file);
Map<String, Object> result = new LinkedHashMap<>();
result.put("upload_id", uploadId);
result.put("received_parts", received);
return ApiResponse.success(result);
}
@GetMapping("/chunk/{uploadId}")
@Operation(summary = "查询已上传分片", description = "返回已到齐的分片序号,供断点续传跳过已传片。")
public ApiResponse<Map<String, Object>> chunkStatus(@PathVariable String uploadId) {
List<Integer> parts = chunkedUploadService.uploadedParts(uploadId);
Map<String, Object> result = new LinkedHashMap<>();
result.put("upload_id", uploadId);
result.put("uploaded_parts", parts);
return ApiResponse.success(result);
}
@PostMapping("/chunk/{uploadId}/complete")
@Operation(
summary = "拼装分片并落盘",
description = """
全部分片到齐后流式拼装并复算 sha256,落盘为普通临时文件(返回 fileKey)。
uploadToOss=true 时与 /upload 同一出口:拼装后转存公开 OSS。
缺少任一分片直接拒绝,不会产出半个文件。
""")
public ApiResponse<UploadFileVo> completeChunkUpload(
@PathVariable String uploadId,
@RequestParam(required = false) String relativePath,
@RequestParam(defaultValue = "false") boolean uploadToOss,
@RequestParam(defaultValue = "COMMON") String moduleType) throws Exception {
ChunkedUploadService.CompletedUpload completed = chunkedUploadService.complete(uploadId);
try {
UploadFileVo vo = localFileStorageService.saveTempFileFromFile(
completed.file().toFile(), completed.fileName(), relativePath, completed.size());
if (uploadToOss) {
File localFile = new File(vo.getLocalPath());
OssStorageService.UploadedResult uploaded =
ossStorageService.uploadPublicFileWithFreshDownloadUrl(
localFile, moduleType, vo.getOriginalFilename());
vo.setObjectKey(uploaded.objectKey());
vo.setUrl(uploaded.downloadUrl());
vo.setMediaType(resolveMediaType(null, vo.getOriginalFilename()));
}
return ApiResponse.success(vo);
} finally {
completed.file().toFile().delete();
}
}
@DeleteMapping("/chunk/{uploadId}")
@Operation(summary = "放弃分片上传", description = "删除未完成的会话与已落盘分片。")
public ApiResponse<Map<String, Object>> abortChunkUpload(@PathVariable String uploadId) {
chunkedUploadService.abort(uploadId);
Map<String, Object> result = new LinkedHashMap<>();
result.put("upload_id", uploadId);
result.put("aborted", true);
return ApiResponse.success(result);
}
@GetMapping("/excel-info")
@Operation(summary = "读取 Excel 信息", description = "根据 fileKey 读取 Excel 第一行表头及总行数,供前端选择保留列和展示总条数。")
@ApiResponses({
@@ -0,0 +1,345 @@
package com.nanri.aiimage.modules.file.service;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.common.text.UserMessages;
import com.nanri.aiimage.config.StorageProperties;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.time.Instant;
import java.util.ArrayList;
import java.util.HexFormat;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
import java.util.regex.Pattern;
import java.util.stream.Stream;
/**
* 临时文件分片上传(S1-F3/AC-17):把大文件拆成固定大小的分片逐个落盘,全部到齐后
* 在服务端流式拼装并复算 sha256。
*
* <p>存在的理由:单次 multipart/单次预签名 PUT 在弱网下一断就从 0 重传,93MB 级安装包
* 与视频几乎没有一次成功的运气。分片后每一片都可独立重试,前端还能查询已到片号续传。</p>
*
* <p>落盘位置在 {@code local-temp-dir/chunk-uploads/},不进入业务结果目录;超过
* {@code stale-hours} 未完成的分片会话在下次 init 时顺手清理。uploadId 只接受 32 位
* 十六进制(自己生成、自己校验),分片序号只接受整数,用户文件名只进 meta 展示,
* 不参与任何路径拼接。</p>
*/
@Service
@RequiredArgsConstructor
@Slf4j
public class ChunkedUploadService {
private static final Pattern UPLOAD_ID_PATTERN = Pattern.compile("[0-9a-f]{32}");
private static final ObjectMapper JSON = new ObjectMapper();
private static final String META_NAME = "meta.json";
/** 单个分片上限:比前端 16MB 分片留足余量,防一次请求把临时盘打满。 */
private static final long MAX_PART_BYTES = 64L * 1024 * 1024;
private final StorageProperties storageProperties;
/** 同一会话的落片/拼装/放弃串行化;会话结束即回收,避免长期驻留。 */
private final ConcurrentHashMap<String, Object> sessionLocks = new ConcurrentHashMap<>();
@Value("${aiimage.file.chunk-upload.max-total-bytes:2147483648}")
private long maxTotalBytes = 2L * 1024 * 1024 * 1024;
@Value("${aiimage.file.chunk-upload.max-parts:4096}")
private int maxParts = 4096;
@Value("${aiimage.file.chunk-upload.stale-hours:24}")
private long staleHours = 24;
/** 分片会话句柄:uploadId 与总片数。 */
public record ChunkSession(String uploadId, int totalParts) {
}
/** 拼装完成的临时文件;调用方负责删除。 */
public record CompletedUpload(Path file, String fileName, long size, String sha256) {
}
public ChunkSession init(String fileName, int totalParts, long totalBytes, String purpose) {
String safeName = normalizeFileName(fileName);
if (totalParts < 1 || totalParts > maxParts) {
throw new BusinessException("分片数量非法(1-" + maxParts + ")");
}
if (totalBytes < 1 || totalBytes > maxTotalBytes) {
throw new BusinessException("文件大小超出上限(最大 " + maxTotalBytes + " 字节)");
}
cleanupStaleSessions();
String uploadId = UUID.randomUUID().toString().replace("-", "");
Path dir = sessionDir(uploadId);
try {
Files.createDirectories(dir);
ObjectNode meta = JSON.createObjectNode();
meta.put("fileName", safeName);
meta.put("totalParts", totalParts);
meta.put("totalBytes", totalBytes);
meta.put("purpose", purpose == null ? "" : purpose);
meta.put("createdAt", Instant.now().toEpochMilli());
JSON.writeValue(metaFile(uploadId).toFile(), meta);
} catch (IOException ex) {
deleteQuietly(dir);
log.warn("[chunk-upload] 初始化分片会话失败 uploadId={} reason={}", uploadId, ex.getMessage());
// 底层英文(磁盘/权限)只进日志:界面文案由 RawExceptionMessageSinkGuardTest 全仓把关
throw new BusinessException(UserMessages.with("初始化分片上传失败", ex));
}
log.info("[chunk-upload] 初始化分片会话 uploadId={} fileName={} parts={} bytes={}",
uploadId, safeName, totalParts, totalBytes);
return new ChunkSession(uploadId, totalParts);
}
/**
* 落一片。同一片重复上传按覆盖处理(幂等):前端重试/续传时不必先查状态。
* 返回当前已到齐的片数,供前端做进度展示。
*/
public int savePart(String uploadId, int index, MultipartFile part) throws IOException {
Meta meta = readMeta(uploadId);
if (index < 0 || index >= meta.totalParts) {
throw new BusinessException("分片序号越界:" + index);
}
if (part == null || part.isEmpty()) {
throw new BusinessException("分片内容为空");
}
if (part.getSize() > MAX_PART_BYTES) {
throw new BusinessException("单个分片超过上限:" + part.getSize());
}
synchronized (lockOf(uploadId)) {
Path dir = sessionDir(uploadId);
Path target = dir.resolve("part-" + index);
Path tmp = dir.resolve("part-" + index + ".tmp");
try (InputStream in = part.getInputStream()) {
Files.copy(in, tmp, StandardCopyOption.REPLACE_EXISTING);
}
Files.move(tmp, target, StandardCopyOption.REPLACE_EXISTING);
long total = currentTotalBytes(uploadId, meta.totalParts);
if (total > maxTotalBytes) {
deleteQuietly(target);
throw new BusinessException("分片累计超过文件上限(最大 " + maxTotalBytes + " 字节)");
}
if (total > meta.totalBytes) {
deleteQuietly(target);
throw new BusinessException("分片累计超过声明的文件大小:" + meta.totalBytes);
}
return (int) uploadedPartIndexes(uploadId, meta.totalParts).size();
}
}
public List<Integer> uploadedParts(String uploadId) {
Meta meta = readMeta(uploadId);
return uploadedPartIndexes(uploadId, meta.totalParts);
}
/**
* 全部片到齐后流式拼装并复算 sha256;拼装完成的文件落在 chunk-uploads 根下,
* 会话目录随之清掉。缺少分片直接拒绝,绝不拼出半个文件。
*/
public CompletedUpload complete(String uploadId) throws IOException {
Meta meta = readMeta(uploadId);
synchronized (lockOf(uploadId)) {
List<Integer> present = uploadedPartIndexes(uploadId, meta.totalParts);
if (present.size() != meta.totalParts) {
List<Integer> missing = new ArrayList<>();
for (int i = 0; i < meta.totalParts; i++) {
if (!present.contains(i)) {
missing.add(i);
if (missing.size() >= 20) {
break;
}
}
}
throw new BusinessException("分片不完整,缺少 " + missing);
}
Path assembled = chunkRoot().resolve("assembled-" + uploadId + ".bin");
MessageDigest digest = sha256Digest();
long total = 0;
try (OutputStream out = Files.newOutputStream(assembled)) {
byte[] buffer = new byte[1 << 20];
for (int i = 0; i < meta.totalParts; i++) {
Path part = sessionDir(uploadId).resolve("part-" + i);
try (InputStream in = Files.newInputStream(part)) {
int read;
while ((read = in.read(buffer)) > 0) {
out.write(buffer, 0, read);
digest.update(buffer, 0, read);
total += read;
}
}
}
} catch (IOException ex) {
Files.deleteIfExists(assembled);
throw ex;
}
if (total != meta.totalBytes) {
Files.deleteIfExists(assembled);
throw new BusinessException(
"拼装后大小与声明不一致:声明 " + meta.totalBytes + ",实际 " + total);
}
deleteQuietly(sessionDir(uploadId));
sessionLocks.remove(uploadId);
String sha256 = HexFormat.of().formatHex(digest.digest());
log.info("[chunk-upload] 拼装完成 uploadId={} fileName={} bytes={} sha256={}",
uploadId, meta.fileName, total, sha256);
return new CompletedUpload(assembled, meta.fileName, total, sha256);
}
}
/** 放弃本次分片会话(前端取消/确认失败清理)。 */
public void abort(String uploadId) {
Meta meta = readMeta(uploadId);
synchronized (lockOf(uploadId)) {
deleteQuietly(sessionDir(uploadId));
}
sessionLocks.remove(uploadId);
log.info("[chunk-upload] 已放弃分片会话 uploadId={} fileName={}", uploadId, meta.fileName);
}
/** 分片会话随附的业务标识(工作台包用它钉住 objectKey,防拼装时换 key)。 */
public String purposeOf(String uploadId) {
return readMeta(uploadId).purpose;
}
private Meta readMeta(String uploadId) {
if (uploadId == null || !UPLOAD_ID_PATTERN.matcher(uploadId).matches()) {
throw new BusinessException("分片会话不存在或已过期");
}
Path metaFile = metaFile(uploadId);
if (!Files.isRegularFile(metaFile)) {
throw new BusinessException("分片会话不存在或已过期");
}
try {
JsonNode node = JSON.readTree(metaFile.toFile());
return new Meta(node.path("fileName").asText(""),
node.path("totalParts").asInt(0),
node.path("totalBytes").asLong(0),
node.path("purpose").asText(""));
} catch (IOException ex) {
throw new BusinessException("分片会话元数据损坏,请重新上传");
}
}
private List<Integer> uploadedPartIndexes(String uploadId, int totalParts) {
List<Integer> result = new ArrayList<>();
for (int i = 0; i < totalParts; i++) {
if (Files.isRegularFile(sessionDir(uploadId).resolve("part-" + i))) {
result.add(i);
}
}
return result;
}
private long currentTotalBytes(String uploadId, int totalParts) throws IOException {
long total = 0;
for (int i = 0; i < totalParts; i++) {
Path part = sessionDir(uploadId).resolve("part-" + i);
if (Files.isRegularFile(part)) {
total += Files.size(part);
}
}
return total;
}
private Path chunkRoot() {
return Path.of(storageProperties.getLocalTempDir()).resolve("chunk-uploads").normalize();
}
private Path sessionDir(String uploadId) {
if (uploadId == null || !UPLOAD_ID_PATTERN.matcher(uploadId).matches()) {
throw new BusinessException("分片会话不存在或已过期");
}
return chunkRoot().resolve(uploadId);
}
private Path metaFile(String uploadId) {
return sessionDir(uploadId).resolve(META_NAME);
}
private Object lockOf(String uploadId) {
return sessionLocks.computeIfAbsent(uploadId, key -> new Object());
}
private void cleanupStaleSessions() {
Path root = chunkRoot();
if (!Files.isDirectory(root)) {
return;
}
long cutoff = System.currentTimeMillis() - Math.max(1, staleHours) * 3600_000L;
try (Stream<Path> children = Files.list(root)) {
children.forEach(child -> {
try {
if (Files.getLastModifiedTime(child).toMillis() < cutoff) {
deleteQuietly(child);
log.info("[chunk-upload] 清理过期分片会话/残留文件 path={}", child.getFileName());
}
} catch (IOException ex) {
log.warn("[chunk-upload] 条目时间戳读不出,本轮跳过该条 path={}", child.getFileName());
}
});
} catch (IOException ex) {
log.warn("[chunk-upload] 扫描过期分片会话失败: {}", ex.getMessage());
}
}
private String normalizeFileName(String fileName) {
String name = fileName == null ? "" : fileName.trim();
if (name.isEmpty()) {
throw new BusinessException("文件名不能为空");
}
if (name.length() > 255) {
name = name.substring(name.length() - 255);
}
return name.replace('\\', '/').replaceAll(".*/", "");
}
private MessageDigest sha256Digest() {
try {
return MessageDigest.getInstance("SHA-256");
} catch (NoSuchAlgorithmException ex) {
throw new IllegalStateException("SHA-256 not available", ex);
}
}
private void deleteQuietly(Path path) {
if (path == null) {
return;
}
try {
if (Files.isDirectory(path)) {
try (Stream<Path> walk = Files.walk(path)) {
walk.sorted((a, b) -> b.getNameCount() - a.getNameCount())
.forEach(p -> {
try {
Files.deleteIfExists(p);
} catch (IOException ex) {
log.warn("[chunk-upload] 残留片删除失败,留给下一轮清理 path={}", p);
}
});
}
} else {
Files.deleteIfExists(path);
}
} catch (IOException ex) {
log.warn("[chunk-upload] 清理未竟,残留交给过期回收 path={}", path);
}
}
private record Meta(String fileName, int totalParts, long totalBytes, String purpose) {
}
}
@@ -78,6 +78,35 @@ public class LocalFileStorageService {
return vo;
}
/**
* 分片拼装后的落盘(AC-17):与 {@link #saveTempFile} 同一套扩展名闸门与 fileKey 索引,
* 只是源从 MultipartFile 换成服务端已拼好的临时文件。
*/
public UploadFileVo saveTempFileFromFile(File source, String originalFilename,
String relativePath, long declaredSize) throws IOException {
File tempDir = FileUtil.file(storageProperties.getLocalTempDir());
File parentDir = tempDir.getParentFile();
if (parentDir != null) {
FileUtil.mkdir(parentDir);
}
FileUtil.mkdir(tempDir);
String fileKey = IdUtil.fastSimpleUUID();
String extName = FileUtil.extName(originalFilename);
rejectBlockedExtension(originalFilename, extName);
String filename = fileKey + (extName.isEmpty() ? "" : "." + extName);
File target = FileUtil.file(tempDir, filename);
FileUtil.move(source, target, true);
registerSourceFileIndex(fileKey, filename);
UploadFileVo vo = new UploadFileVo();
vo.setFileKey(fileKey);
vo.setOriginalFilename(originalFilename);
vo.setLocalPath(target.getAbsolutePath());
vo.setSize(target.length() > 0 ? target.length() : Math.max(0, declaredSize));
vo.setRelativePath(relativePath);
return vo;
}
/** 扩展名黑名单校验:命中即拒绝落盘(大小写不敏感)。 */
private void rejectBlockedExtension(String originalFilename, String extName) {
if (extName == null || extName.isBlank()) {
@@ -322,6 +322,17 @@ public class OssStorageService {
return presignPublicClientUpload(objectKey, WORKBENCH_CLIENT_PREFIX, "workbench client", expirySeconds);
}
/**
* 服务端直传工作台客户端包(AC-17 分片通道的落点):浏览器分片 → 后端拼装 → 这里
* 上传到公开 client 桶的 {@code workbench-client/versions/} 前缀。
* 与直传预签名写的是同一个桶/前缀,confirm 阶段的实物复算 sha256 与凭据扫描照旧生效。
*/
public String uploadWorkbenchClientObject(File file, String objectKey) {
requireWorkbenchClientKey(objectKey);
uploadFile(file, publicClientBucket(), objectKey);
return objectKey;
}
/** 返回工作台客户端包字节数;对象不存在返回 -1(直传确认时先卡大小,再算摘要)。 */
public long workbenchClientObjectSize(String objectKey) {
return publicClientObjectSize(objectKey, WORKBENCH_CLIENT_PREFIX, "workbench client");
@@ -41,11 +41,13 @@ import org.springframework.transaction.annotation.Transactional;
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.util.ArrayList;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ThreadLocalRandom;
/**
* 五国关键词共享(桌面 Tkinter 工具的 web 化)。
@@ -200,12 +202,23 @@ public class KeywordShareService {
Long count) {
KeywordShareFilterQuery filter = buildFilter(companyId, viewerUserName, country, category, subCategory,
categoryL3, categoryL4, search, FILTER_UNUSED, null, false);
filter.setOffset(0);
long safeCount = count == null ? DEFAULT_RANDOM_COUNT : Math.min(Math.max(count, 1), MAX_RANDOM_COUNT);
filter.setLimit(safeCount);
return keywordShareMapper.selectRandomUnused(filter).stream()
// AC-19(审计 P1-B2):窗口起点先数命中集再掷,MySQL 就不必给全表打 RAND 后 filesort。
// 掷的是起点而非每一行,所以同一批词的相邻关系仍在(一次抽到的是连续 id 的一段),
// 服务层洗牌补回"散着"的观感;命中集不足一批时起点恒 0,与旧口径同为"全给你"。
filter.setOffset(randomWindowOffset(keywordShareMapper.countHit(filter), safeCount));
List<KeywordShareRowVo> window = new ArrayList<>(keywordShareMapper.selectRandomUnused(filter).stream()
.map(this::normalizeRow)
.toList();
.toList());
Collections.shuffle(window);
return window;
}
/** 随机窗口起点:落在 {@code [0, total - count]},装得下整批才有得掷。 */
static long randomWindowOffset(long total, long count) {
long span = total - count;
return span > 0 ? ThreadLocalRandom.current().nextLong(span + 1) : 0L;
}
// ---------------------------------------------------------------- 写操作
@@ -261,6 +261,7 @@ public class PermissionMenuService {
entity.setCompanyId(companyId);
entity.setSortOrder(resolveSortOrder(companyId, menuType, request.getSortOrder(), null));
permissionMenuMapper.insert(entity);
invalidateMenuTreeCache();
return toItemVo(getMenuById(entity.getId()));
}
@@ -300,6 +301,7 @@ public class PermissionMenuService {
entity.setRoutePath(routePath);
entity.setSortOrder(resolveSortOrder(companyId, menuType, request.getSortOrder(), id));
permissionMenuMapper.updateById(entity);
invalidateMenuTreeCache();
return toItemVo(getMenuById(id));
}
@@ -321,6 +323,7 @@ public class PermissionMenuService {
// installations without the FK are cleaned up as well.
userColumnPermissionMapper.deleteByColumnId(entity.getId());
permissionMenuMapper.deleteById(entity.getId());
invalidateMenuTreeCache();
}
/**
@@ -376,6 +379,7 @@ public class PermissionMenuService {
}
sort += 10;
}
invalidateMenuTreeCache();
}
/**
@@ -457,6 +461,7 @@ public class PermissionMenuService {
}
log.info("[menu-tree-copy] 新公司菜单树已建 companyId={} template={} rows={}",
companyId, templateCompanyId, inserted);
invalidateMenuTreeCache();
return inserted;
}
@@ -1031,6 +1036,29 @@ public class PermissionMenuService {
}
}
/**
* AC-19(审计 P1-B4):目录树的短 TTL 缓存。
*
* <p>侧边栏、每个受管入口的 {@code hasAnyAdminMenu}、可授予全集都从 {@code loadMenus} 取同一棵
* 树,此前每请求重查一遍再在内存里排序展开。目录结构是"配一次、几个月不动"的数据,
* 30 秒的窗口换来热路径少一次全表取数。
*
* <p>只缓存目录本体,**不缓存用户授权行**:撤权必须下一次请求就生效,那是安全语义;
* 目录结构最多陈旧一个 TTL(写路径都同机 {@code invalidateMenuTreeCache},跨实例靠 TTL 收敛)。
* 键是 (companyId, menuType),company 维度来自库里的用户行而非请求参数,条数有界。
*/
private static final long MENU_CACHE_TTL_MILLIS = 30_000L;
private final Map<String, CachedMenuTree> menuTreeCache = new java.util.concurrent.ConcurrentHashMap<>();
/** 一棵已排序的目录树快照与其入缓存的时刻。 */
private record CachedMenuTree(List<PermissionMenuEntity> menus, long cachedAtMillis) {
}
/** 目录写路径共用的失效出口:结构变了就把整份快照丢掉,下一次现取。 */
private void invalidateMenuTreeCache() {
menuTreeCache.clear();
}
/**
* 目录取数的唯一出口(§4):{@code companyId} 非空时只返回那一棵树的行。
* 授权树、有效菜单、可授予全集、越权判定都从这里取数,所以公司收窄改这一处就够一片。
@@ -1038,14 +1066,23 @@ public class PermissionMenuService {
* @param companyId null 只来自「无操作者 / 无归属」的历史与测试路径,此时退回全量目录
*/
private List<PermissionMenuEntity> loadMenus(Long companyId, String menuType) {
List<PermissionMenuEntity> rows = loadRawMenus(companyId, menuType);
if (rows == null || rows.isEmpty()) {
return List.of();
String cacheKey = (companyId == null ? "*" : companyId.toString())
+ '|' + normalizeMenuType(menuType);
long now = System.currentTimeMillis();
CachedMenuTree cached = menuTreeCache.get(cacheKey);
if (cached != null && now - cached.cachedAtMillis() < MENU_CACHE_TTL_MILLIS) {
return cached.menus();
}
return rows.stream()
.filter(row -> row != null && row.getId() != null)
.sorted(menuComparator())
.toList();
List<PermissionMenuEntity> rows = loadRawMenus(companyId, menuType);
// 不可变快照:缓存出去的东西被调用方就地改一笔,下一次请求会读到别人的写入
List<PermissionMenuEntity> sorted = rows == null || rows.isEmpty()
? List.of()
: rows.stream()
.filter(row -> row != null && row.getId() != null)
.sorted(menuComparator())
.toList();
menuTreeCache.put(cacheKey, new CachedMenuTree(sorted, now));
return sorted;
}
/**
@@ -11,6 +11,7 @@ import com.nanri.aiimage.config.SimilarAsinProperties;
import com.nanri.aiimage.metrics.ExternalCallMetricsRecorder;
import com.nanri.aiimage.modules.usersecret.service.UserSecretUsageService;
import com.nanri.aiimage.modules.usersecret.support.SecretUsageContext;
import com.nanri.aiimage.modules.usersecret.support.LlmChannel;
import com.nanri.aiimage.modules.usersecret.support.UserSecretModule;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.HttpHeaders;
@@ -93,7 +94,7 @@ public class SimilarAsinLlmClient {
String resolvedKey = resolveApiKey(apiKey);
int attempts = Math.max(1, properties.getLlmRetryTimes());
long cooldownMillis = Math.max(0, properties.getLlmUnreachableCooldownMillis());
String host = properties.getLlmHost();
String host = resolveHost();
if (unreachableCooldown.isCoolingDown(host, cooldownMillis)) {
// 冷却期快失败不发请求、不记密钥次数:结论与短路版一致(调用方拿不到 LLM 结论),
// 差别只是不再为"确认还是断的"各烧一轮连接探针(B24)
@@ -186,11 +187,17 @@ public class SimilarAsinLlmClient {
int attempt) {
recordSecretUsage();
Map<String, Object> body = buildChatBody(model, system, userText, images, responseFormat);
log.debug("[similar-asin][llm] request attempt={} model={} url={} body={}",
attempt, model, joinUrl(properties.getLlmHost(), "/v1/chat/completions"),
String host = resolveHost();
// 渠道模型映射:龙抬头那边的同款模型叫 gemini-3.1-flash-lite,直接发贞贞的模型名会 model_not_found
String requestModel = LlmChannel.modelForHost(host, model);
body.put("model", requestModel);
// 思考参数按渠道补:贞贞把 thinking 写在模型名里,龙抬头要显式传(复刻现网的 thinking-medium)
LlmChannel.applyThinkingForHost(host, body);
log.debug("[similar-asin][llm] request attempt={} model={} channelModel={} url={} body={}",
attempt, model, requestModel, joinUrl(host, "/v1/chat/completions"),
writeJson(maskChatBody(body)));
RestClient.RequestBodySpec request = restClient(attempt).post()
.uri(joinUrl(properties.getLlmHost(), "/v1/chat/completions"))
.uri(joinUrl(host, "/v1/chat/completions"))
.headers(headers -> {
headers.setBearerAuth(stripBearer(apiKey));
headers.setContentType(APPLICATION_JSON_UTF8);
@@ -475,6 +482,15 @@ public class SimilarAsinLlmClient {
return normalized.regionMatches(true, 0, "Bearer ", 0, 7) ? normalized.substring(7).trim() : normalized;
}
/**
* 渠道主机:优先取任务上下文里按用户密钥渠道解析出的主机(龙抬头/贞贞),
* 没有上下文(单测、巡检等)时回落模块全局配置,保持历史行为。
*/
private String resolveHost() {
String contextual = SecretUsageContext.currentLlmHost();
return contextual == null ? properties.getLlmHost() : contextual;
}
private String joinUrl(String baseUrl, String path) {
String base = baseUrl == null ? "" : baseUrl.trim();
String suffix = path == null ? "" : path.trim();
@@ -47,6 +47,12 @@ public class SimilarAsinTaskHeartbeatSpi implements TaskModuleHeartbeatSpi {
return Duration.ofMillis(checkpointIntervalMillis());
}
/** 只把 {@code task.getId()} 交给作业续期,不写模块缓存 ⇒ 心跳可以吃瘦读(AC-19/P1-B3)。 */
@Override
public boolean heartbeatNeedsFullTaskEntity() {
return false;
}
private long checkpointIntervalMillis() {
return Math.max(1_000L, similarAsinProperties.getDbTaskTouchIntervalMillis());
}
@@ -210,6 +210,13 @@ public class SimilarAsinTaskService implements SimilarAsinPipelineHost {
*/
private final SimilarAsinImagePrefetchService imagePrefetchService;
private final UserApiSecretService userApiSecretService;
/**
* 采集路由开关:字段注入,避免给 {@code @RequiredArgsConstructor} 加参数而牵连一批
* 手工 new 本类的测试。null(单测未注入)按「本地」处理,保持 activate 的原有行为。
*/
@org.springframework.beans.factory.annotation.Autowired(required = false)
private com.nanri.aiimage.config.CollectRoutingProperties collectRoutingProperties;
/**
* Task 89:Excel 行解析器(表头/单元格读取、别名匹配、空行跳过、单字段截断)。
* 由 parseAndCreateTask 委托;解析语义与搬移前 parseWorkbook 完全一致。
@@ -540,6 +547,12 @@ public class SimilarAsinTaskService implements SimilarAsinPipelineHost {
throw new BusinessException("任务不存在");
}
ownershipSupport().ensureTaskOwnedByCurrentInstance(task, "activate");
// 2026-10-29 本地采集开关:关闭时页面不再推本机客户端,任务留 PENDING 等 Linux 后台认领。
// 这里把「点启动」明确拒掉,避免旧页面缓存继续推队列造成与后台双跑。
if (collectRoutingProperties != null
&& !collectRoutingProperties.isLocal(com.nanri.aiimage.config.CollectRoutingProperties.MODULE_SIMILAR_ASIN)) {
throw new BusinessException("相似ASIN已切换后台自动执行,无需打开客户端,提交后稍候自动开始");
}
// 只允许 PENDING→RUNNING(条件更新):与客户端「兜底拉取」的原子认领互斥,
// 谁先翻转谁执行,避免页面与客户端重复执行同一任务
int updated = fileTaskMapper.update(null, new LambdaUpdateWrapper<FileTaskEntity>()
@@ -233,6 +233,11 @@ public class SimilarAsinPayloadSupport {
return fromUserSecret;
}
/** 任务归属用户密钥所属渠道的 LLM 主机:密钥属于哪条中转(龙抬头/贞贞)就发到哪条。 */
public String resolveLlmHost(FileTaskEntity task) {
return userApiSecretService.resolveLlmHost(task.getUserId(), UserSecretModule.SIMILAR_ASIN);
}
public boolean readImgSwitch(FileTaskEntity task) {
try {
return Boolean.TRUE.equals(readParsedPayload(task).getImgSwitch());
@@ -931,10 +931,12 @@ public class SimilarAsinPipelineSupport {
boolean categorySwitch = host.readCategorySwitch(task);
int batchSize = SimilarAsinLimits.resolveLlmBatchSize(properties, imgSwitch);
List<SimilarAsinResultRowDto> result = new ArrayList<>();
// 渠道主机任务级解析一次:本任务的所有批次都发往同一家中转
String apiHost = payloadSupport.resolveLlmHost(task);
for (int i = 0; i < items.size(); i += batchSize) {
List<SimilarAsinResultRowDto> batch = items.subList(i, Math.min(i + batchSize, items.size()));
// 批次内所有 LLM 请求按任务归属用户计次(无归属时上下文自动跳过)
result.addAll(SecretUsageContext.call(task.getUserId(), UserSecretModule.SIMILAR_ASIN.key(),
result.addAll(SecretUsageContext.call(task.getUserId(), UserSecretModule.SIMILAR_ASIN.key(), apiHost,
() -> similarAsinLlmService.inspectRows(batch, prompt, apiKey, imgSwitch, categorySwitch)));
if (progressHook != null) {
progressHook.run();
@@ -1294,6 +1296,7 @@ public class SimilarAsinPipelineSupport {
List<SimilarAsinResultRowDto> llmRows;
try {
llmRows = SecretUsageContext.call(task.getUserId(), UserSecretModule.SIMILAR_ASIN.key(),
payloadSupport.resolveLlmHost(task),
() -> similarAsinLlmService.inspectRows(batchRows, prompt, apiKey, imgSwitch, categorySwitch));
} catch (Exception ex) {
String message = firstNonBlank(ex.getMessage(), "LLM submit failed");
@@ -0,0 +1,35 @@
package com.nanri.aiimage.modules.task.controller;
import com.nanri.aiimage.common.api.ApiResponse;
import com.nanri.aiimage.config.CollectRoutingProperties;
import com.nanri.aiimage.modules.task.model.vo.CollectRoutingVo;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* 专利/货源的本地采集开关查询。
*
* <p>前端据此决定「启动」按钮走哪条链路:本地采集开着就取出队列载荷推本机客户端,
* 关着就只建单、等 Linux 后台认领。开关的唯一真源在服务端配置
* ({@code aiimage.collect-routing.local-enabled}),前端不自己保存副本。
*/
@RestController
@RequiredArgsConstructor
@RequestMapping("/api/collect-routing")
@Tag(name = "采集路由", description = "专利/货源的本地采集开关(前端读取后决定推送本地还是提交后台)")
public class CollectRoutingController {
private final CollectRoutingProperties properties;
@GetMapping
@Operation(summary = "查询本地采集开关", description = "返回外观专利与相似ASIN当前是否走本地采集。只读,无副作用。")
public ApiResponse<CollectRoutingVo> routing() {
return ApiResponse.success(new CollectRoutingVo(
properties.isLocal(CollectRoutingProperties.MODULE_APPEARANCE_PATENT),
properties.isLocal(CollectRoutingProperties.MODULE_SIMILAR_ASIN)));
}
}
@@ -0,0 +1,19 @@
package com.nanri.aiimage.modules.task.model.vo;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
@Data
@NoArgsConstructor
@AllArgsConstructor
@Schema(description = "专利/货源的采集执行方开关")
public class CollectRoutingVo {
@Schema(description = "外观专利是否本地采集:true=页面推本地客户端(超时后 Linux 兜底);false=只由 Linux 后台执行")
private boolean appearancePatentLocal;
@Schema(description = "相似ASIN是否本地采集:true=页面推本地客户端(超时后 Linux 兜底);false=只由 Linux 后台执行")
private boolean similarAsinLocal;
}
@@ -21,21 +21,26 @@ import java.util.List;
* 陈旧任务自动修复。
*
* 修复型巡检(区别于 task-161 的只读报告巡检):把超过阈值时间无心跳(updated_at 未刷新)的
* PENDING / SCHEDULED / RUNNING 任务自动标终态失败,避免历史任务列表永久停留在中间状态。
* SCHEDULED / RUNNING 任务自动标终态失败,避免历史任务列表永久停留在中间状态。
* 分布式锁防双实例重复修复;注意 RUNNING 的存活判定基于 updatedAt 心跳,真活任务会不断刷新不会被误杀。
*
* <p><b>PENDING 不在本巡检的判死范围内(2026-10-29 产品口径)</b>:PENDING 是"排队等 worker 领取",
* 不是"卡死"。AP worker 串行跑大任务时,队列里的任务等再久也必须保留——按 2 小时判死把排队任务
* 误杀成「任务长期未被领取」,还会触发自动克隆重试,同一份活被放大成多条失败记录(现网 2026-10-28
* 用户 1211 的两条 829 行任务即此路)。排队任务一律等 {@code TaskClientPullService} 认领;
* 页面放弃的解析预览也一样保留,不在这里清。
*/
@Slf4j
@Service
@RequiredArgsConstructor
public class StaleTaskRepairService {
private static final String STATUS_PENDING = "PENDING";
private static final String STATUS_SCHEDULED = "SCHEDULED";
private static final String STATUS_RUNNING = "RUNNING";
private static final String STATUS_FAILED = "FAILED";
/** 中间态(无人接单)或 RUNNING(心跳残留)超过该时长未更新即判死。真活任务心跳会不断刷新 updatedAt,不会被误杀。 */
/** SCHEDULED(到点未启动)/ RUNNING(心跳残留)超过该时长未更新即判死。真活任务心跳会不断刷新 updatedAt,不会被误杀。 */
private static final long STALE_IDLE_MINUTES = 120;
/** 单轮品牌陈旧任务处理上限(原实现内联 limit 500)。 */
@@ -61,7 +66,6 @@ public class StaleTaskRepairService {
return;
}
try (lock) {
repairFileTaskStaleIdle();
repairFileTaskStaleScheduled();
repairFileTaskStaleRunning();
repairBrandStale();
@@ -70,32 +74,6 @@ public class StaleTaskRepairService {
}
}
/** PENDING 长时间未被领取 → FAILED(页面没推进客户端队列,客户端兜底拉取也没领到)。 */
private void repairFileTaskStaleIdle() {
LocalDateTime cutoff = LocalDateTime.now().minusMinutes(STALE_IDLE_MINUTES);
List<FileTaskEntity> stale = fileTaskMapper.selectList(new LambdaQueryWrapper<FileTaskEntity>()
.select(FileTaskEntity::getId, FileTaskEntity::getModuleType)
.eq(FileTaskEntity::getStatus, STATUS_PENDING)
.lt(FileTaskEntity::getUpdatedAt, cutoff)
.last("limit 500"));
if (stale.isEmpty()) {
return;
}
String reason = "任务长期未被领取,已自动失败";
List<Long> ids = stale.stream().map(FileTaskEntity::getId).toList();
LocalDateTime now = LocalDateTime.now();
int updated = fileTaskMapper.update(null, new LambdaUpdateWrapper<FileTaskEntity>()
.in(FileTaskEntity::getId, ids)
.eq(FileTaskEntity::getStatus, STATUS_PENDING)
.set(FileTaskEntity::getStatus, STATUS_FAILED)
.set(FileTaskEntity::getErrorMessage, reason)
.set(FileTaskEntity::getFinishedAt, now));
if (updated > 0) {
log.warn("[stale-task-repair] 陈旧中间态已标失败 count={} ids={}", updated, ids);
markDeadDownstream(ids, reason);
}
}
/**
* SCHEDULED 到时后长时间未启动 → FAILED。
*
@@ -2,6 +2,7 @@ package com.nanri.aiimage.modules.task.service;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.nanri.aiimage.config.CollectRoutingProperties;
import com.nanri.aiimage.config.InstanceMetadata;
import com.nanri.aiimage.modules.task.mapper.FileTaskMapper;
import com.nanri.aiimage.modules.task.model.entity.FileTaskEntity;
@@ -25,12 +26,16 @@ import java.util.Map;
* 客户端兜底拉取:把"页面没推送成功、长期停在 PENDING"的任务交给在线客户端执行。
*
* <p>PENDING 表示任务已解析落库、等页面把它推进本机 Python 队列;页面这一环缺失时任务会一直
* 停在这里,2 小时后被 {@code StaleTaskRepairService} 标失败。本服务让客户端主动来领:
* 停在这里(2026-10-29 起 PENDING 不再按时间判死,只能靠认领通道推进)。本服务让客户端主动来领:
* 只挑创建超过 N 分钟仍是 PENDING 的本用户任务,逐条用条件更新认领(PENDING→RUNNING),
* 只有把状态翻过来的调用方算领取成功 —— 与页面 activate 天然互斥,不会重复执行。
*
* <p>开关 {@code aiimage.client-task-pull.enabled} 默认关闭;模块白名单默认只放三个
* "执行参数全在服务端"的模块,其余模块实现 {@link ClientTaskPullSpi} 后可逐步加入。
* <p>2026-10-29 起按 {@link CollectRoutingProperties} 分两档路由:本地采集模块只允许客户端领;
* Linux 内部认领对本地模块要等 {@code linux-fallback-minutes} 的兜底窗口,对非本地模块保持
* 原有的 {@code internal-min-pending-minutes}。详见 {@link #pullPendingTasksInternal(List)}。
*
* <p>开关 {@code aiimage.client-task-pull.enabled} 默认关闭;模块白名单默认只放"执行参数全在服务端"
* 的模块,其余模块实现 {@link ClientTaskPullSpi} 后可逐步加入。
*/
@Slf4j
@Service
@@ -51,8 +56,10 @@ public class TaskClientPullService {
private final FileTaskMapper fileTaskMapper;
private final InstanceMetadata instanceMetadata;
/** moduleType → 模块兜底实现(启动时建索引并校验重复注册)。 */
/** 模块 → 模块兜底实现(启动时建索引并校验重复注册)。 */
private final Map<String, ClientTaskPullSpi> pullHandlers;
/** 专利/货源等模块的本地采集开关与 Linux 兜底窗口。 */
private final CollectRoutingProperties collectRoutingProperties;
@Value("${aiimage.client-task-pull.enabled:false}")
private boolean enabled;
@@ -80,9 +87,11 @@ public class TaskClientPullService {
public TaskClientPullService(FileTaskMapper fileTaskMapper,
InstanceMetadata instanceMetadata,
CollectRoutingProperties collectRoutingProperties,
List<ClientTaskPullSpi> handlers) {
this.fileTaskMapper = fileTaskMapper;
this.instanceMetadata = instanceMetadata;
this.collectRoutingProperties = collectRoutingProperties;
Map<String, ClientTaskPullSpi> index = new LinkedHashMap<>();
for (ClientTaskPullSpi handler : handlers == null ? List.<ClientTaskPullSpi>of() : handlers) {
String moduleType = handler.moduleType();
@@ -109,8 +118,10 @@ public class TaskClientPullService {
*/
@PostConstruct
public void logEffectiveConfig() {
log.info("[client-task-pull] 生效配置 enabled={} internalEnabled={} internalMinPendingMinutes={} moduleTypes={} minPendingMinutes={} limit={} handlers={}",
enabled, internalEnabled, internalMinPendingMinutes, moduleTypes, minPendingMinutes, Math.max(1, Math.min(limit, MAX_LIMIT)), pullHandlers.keySet());
log.info("[client-task-pull] 生效配置 enabled={} internalEnabled={} internalMinPendingMinutes={} moduleTypes={} minPendingMinutes={} limit={} handlers={} localEnabled={} linuxFallbackMinutes={}",
enabled, internalEnabled, internalMinPendingMinutes, moduleTypes, minPendingMinutes,
Math.max(1, Math.min(limit, MAX_LIMIT)), pullHandlers.keySet(),
collectRoutingProperties.getLocalEnabled(), collectRoutingProperties.safeLinuxFallbackMinutes());
}
/** 拉取并认领该用户"长期未被领取"的任务,按创建时间升序返回,最多 limit 条。 */
@@ -126,6 +137,13 @@ public class TaskClientPullService {
log.warn("[client-task-pull] 模块白名单为空,本次不拉取 userId={}", userId);
return List.of();
}
// 只拉本地采集模块:关掉本地开关的模块只认 Linux 后台,客户端拿到会算抢跑
types = types.stream().filter(collectRoutingProperties::isLocal).toList();
if (types.isEmpty()) {
log.info("[client-task-pull] 白名单模块均已切换 Linux 后台执行,客户端不拉取 userId={} moduleTypes={}",
userId, moduleTypes);
return List.of();
}
long startedAt = System.currentTimeMillis();
LocalDateTime cutoff = LocalDateTime.now(DB_ZONE).minusMinutes(Math.max(1L, minPendingMinutes));
int safeLimit = Math.max(1, Math.min(limit, MAX_LIMIT));
@@ -149,6 +167,11 @@ public class TaskClientPullService {
* <p>与页面 activate / 客户端兜底拉取共用同一条 PENDING→RUNNING 原子认领,天然互斥不重复执行。
* 独立开关 {@code aiimage.client-task-pull.internal-enabled},与客户端开关解耦:
* 客户端侧未升级/未开启时也能单独让服务端 worker 上线。
*
* <p><b>2026-10-29 本地优先</b>:本地采集模块({@code aiimage.collect-routing.local-enabled})
* 的任务要让位给本地客户端——PENDING 超过 {@code linux-fallback-minutes} 还没被本地领取,
* 才允许这里接管;非本地模块保持 {@code internal-min-pending-minutes} 的原窗口。
* 没有这层过滤,页面刚把任务推给客户端,Linux 就会在 2 分钟后抢走并双跑。
*/
public List<TaskClientPullVo> pullPendingTasksInternal(List<String> moduleTypes) {
if (!internalEnabled) {
@@ -161,7 +184,12 @@ public class TaskClientPullService {
return List.of();
}
long startedAt = System.currentTimeMillis();
LocalDateTime cutoff = LocalDateTime.now(DB_ZONE).minusMinutes(Math.max(1L, internalMinPendingMinutes));
LocalDateTime now = LocalDateTime.now(DB_ZONE);
LocalDateTime cutoff = now.minusMinutes(Math.max(1L, internalMinPendingMinutes));
// 兜底窗口不得短于基础等待窗口:配小了也没法越过「建完就能删」的余地
long localFallbackMinutes = Math.max(collectRoutingProperties.safeLinuxFallbackMinutes(),
Math.max(1L, internalMinPendingMinutes));
LocalDateTime localCutoff = now.minusMinutes(localFallbackMinutes);
int safeLimit = Math.max(1, Math.min(limit, MAX_LIMIT));
List<FileTaskEntity> candidates = fileTaskMapper.selectList(new LambdaQueryWrapper<FileTaskEntity>()
.select(FileTaskEntity::getId, FileTaskEntity::getModuleType, FileTaskEntity::getCreatedAt)
@@ -170,6 +198,21 @@ public class TaskClientPullService {
.lt(FileTaskEntity::getCreatedAt, cutoff)
.orderByAsc(FileTaskEntity::getCreatedAt)
.last("limit " + safeLimit));
if (!candidates.isEmpty()) {
int before = candidates.size();
candidates = candidates.stream()
.filter(task -> !collectRoutingProperties.isLocal(task.getModuleType())
|| task.getCreatedAt() == null
|| !task.getCreatedAt().isAfter(localCutoff))
.toList();
if (candidates.size() < before) {
log.info("[client-task-pull] 内部认领让位本地:本轮跳过 {} 条本地采集模块的排队任务(兜底窗口 {} 分钟)",
before - candidates.size(), localFallbackMinutes);
}
}
if (candidates.isEmpty()) {
return List.of();
}
List<TaskClientPullVo> claimed = claimCandidates(candidates, "内部认领");
log.info("[client-task-pull] 内部认领完成 types={} 候选={} 认领={} 耗时={}ms",
types, candidates.size(), claimed.size(), System.currentTimeMillis() - startedAt);
@@ -79,7 +79,7 @@ public class TaskHeartbeatService {
boolean brandOnly = MODULE_BRAND.equalsIgnoreCase(requestedModuleType);
boolean skipBrandLookup = MODULE_SHOP_DATA_CRAWL.equalsIgnoreCase(requestedModuleType);
TaskHeartbeatVo fileResult = brandOnly ? null : touchFileTaskIfRunning(selectFileTask(taskId), request);
TaskHeartbeatVo fileResult = brandOnly ? null : touchFileTaskIfRunning(taskId, request);
TaskHeartbeatVo brandResult = skipBrandLookup ? null : brandTaskHeartbeatSpi.heartbeat(taskId, request);
// 心跳到达打点:陈旧判死线用它把「心跳时钟」改成从首次心跳起算(见 StaleTaskHeartbeatPolicy)。
@@ -112,7 +112,26 @@ public class TaskHeartbeatService {
return TaskHeartbeatVo.notAlive(null, null, "task not found");
}
/**
* AC-19(审计 P1-B3):心跳热路径的任务读只取活性判定要的五列。
*
* <p>{@code biz_file_task.request_json}/{@code result_json} 能到 MB 级,而这条读是
* 「每单每 30s 一次」的热路径,原实现整行捞回来只为读 id/moduleType/status/userId/updatedAt。
* 需要把实体交给模块 SPI 的({@code saveTaskCache} 会把整个实体序列化进模块缓存),
* 由 {@link TaskModuleHeartbeatSpi#heartbeatNeedsFullTaskEntity()} 声明后走
* {@link #selectFullFileTask}——半行进缓存比多读两列难查得多,所以宁可多一次主键点查。
*/
private FileTaskEntity selectFileTask(Long taskId) {
LambdaQueryWrapper<FileTaskEntity> fileQuery = new LambdaQueryWrapper<FileTaskEntity>()
.select(FileTaskEntity::getId, FileTaskEntity::getModuleType, FileTaskEntity::getStatus,
FileTaskEntity::getUserId, FileTaskEntity::getUpdatedAt)
.eq(FileTaskEntity::getId, taskId)
.last("limit 1");
return fileTaskMapper.selectOne(fileQuery);
}
/** 整实体读:客户端中断恢复(要 errorMessage,并把实体同步进模块缓存与快照)与声明需要整行的模块心跳。 */
private FileTaskEntity selectFullFileTask(Long taskId) {
LambdaQueryWrapper<FileTaskEntity> fileQuery = new LambdaQueryWrapper<FileTaskEntity>()
.eq(FileTaskEntity::getId, taskId)
.last("limit 1");
@@ -132,7 +151,7 @@ public class TaskHeartbeatService {
String safeReason = (reason == null || reason.isBlank())
? "客户端异常中断,任务已自动失败"
: "客户端异常中断: " + reason.trim();
FileTaskEntity fileTask = selectFileTask(taskId);
FileTaskEntity fileTask = selectFullFileTask(taskId);
if (fileTask != null) {
String status = fileTask.getStatus();
int updated = fileTaskMapper.update(null, new LambdaUpdateWrapper<FileTaskEntity>()
@@ -167,12 +186,23 @@ public class TaskHeartbeatService {
return TaskHeartbeatVo.notAlive(null, null, "task not found");
}
private TaskHeartbeatVo touchFileTaskIfRunning(FileTaskEntity task, TaskHeartbeatRequest request) {
private TaskHeartbeatVo touchFileTaskIfRunning(Long taskId, TaskHeartbeatRequest request) {
FileTaskEntity task = selectFileTask(taskId);
if (task == null) {
return null;
}
String moduleType = task.getModuleType() == null ? "" : task.getModuleType();
TaskModuleHeartbeatSpi handler = moduleHandlers.get(moduleType);
if (handler != null && handler.heartbeatNeedsFullTaskEntity()) {
// 归属校验与模块缓存回写都吃整实体(publish/shop-data-crawl 读实例路由列,
// 七个模块的 saveTaskCache 把实体序列化落盘),瘦行给它们会写出半行 ⇒ 补一次主键读。
task = selectFullFileTask(taskId);
if (task == null) {
return null;
}
moduleType = task.getModuleType() == null ? "" : task.getModuleType();
handler = moduleHandlers.get(moduleType);
}
if (handler != null) {
handler.ensureOwnership(task, moduleType + " task heartbeat");
}
@@ -8,10 +8,10 @@ import java.util.Map;
* 客户端兜底拉取的模块侧扩展点。
*
* <p>背景:任务由页面"解析落库 PENDING → activate → 经 pywebview 桥推给本机 Python 队列"派发,
* 页面这一段缺失(只解析没点启动 / 推送前关页面 / 在纯浏览器打开)时任务永远停在 PENDING,
* 2 小时后被 StaleTaskRepairService 标失败(「任务长期未被领取,已自动失败」)。
* 本接口让业务模块自行组装"客户端可直接入队"的 payload,由 {@code TaskClientPullService}
* 认领(PENDING→RUNNING 条件更新)后下发给客户端执行 —— 等于替页面补上"推送"这一步。
* 页面这一段缺失(只解析没点启动 / 推送前关页面 / 在纯浏览器打开)时任务会停在 PENDING。
* 2026-10-29 起 PENDING <b>不再按时间判死</b>——排队等 worker 领取的任务等多久都保留,
* 由 {@code TaskClientPullService} 认领(PENDING→RUNNING 条件更新)后下发给客户端执行 ——
* 等于替页面补上"推送"这一步;认领不掉的解析预览就一直留在队列里,不在这里清。
*
* <p>与 {@link TaskModuleHeartbeatSpi} 同理(G5):task 侧不 import 业务模块,
* 各模块实现本接口,Spring 注入 List 后由 task 侧建索引。
@@ -44,6 +44,21 @@ public interface TaskModuleHeartbeatSpi {
default void onHeartbeat(FileTaskEntity task) {
}
/**
* 心跳时是否需要整行 {@code FileTaskEntity}。
*
* <p>AC-19(审计 P1-B3):心跳是「每单每 30s 一次」的热路径,而 {@code biz_file_task} 的
* {@code request_json}/{@code result_json} 能到 MB 级。task 侧默认只捞活性判定要的五列,
* 只有真把实体交出去用的模块才付整行读的钱——即 {@link #saveTaskCache(FileTaskEntity)}
* 会把实体序列化进模块缓存的那些(半行进缓存比多读一次列难查得多)。
*
* <p>默认 {@code true}(保守):新模块不改这儿就维持今天的取整行行为。只用得到
* {@code task.getId()} 的实现才显式改成 false。
*/
default boolean heartbeatNeedsFullTaskEntity() {
return true;
}
/**
* biz_file_task.updated_at 检查点节流间隔。
*
@@ -217,6 +217,27 @@ public interface TaskCenterQueryMapper {
+ USER_SCOPE_B
+ ")";
/**
* 页首游标锚点(AC-19/P1-B1):第 N 页那一行的前一行——按同一套筛选与同一个排序键
* {@code (createdAt DESC, id DESC)} 走到 {@code offset} 处,只取两列。
*
* <p>深翻页的 skip 只在这条窄语句里发生,页查询本身改用游标;两条必须共用同一份 UNION
* (mapper XML 的 {@code taskUnionRows}),否则锚点算的是另一批行,第 N 页会安静地给错。
*/
TaskCenterTaskRow selectTaskPageAnchor(@Param("userIds") List<Long> userIds,
@Param("moduleType") String moduleType,
@Param("statuses") List<String> statuses,
@Param("shop") String shop,
@Param("startTime") LocalDateTime startTime,
@Param("endTime") LocalDateTime endTime,
@Param("offset") long offset);
/**
* 任务列表一页:keyset 游标取数,不再吃 OFFSET。
*
* <p>{@code afterCreatedAt}/{@code afterId} 同时给值才是游标(第一页两个都为 null = 从头取);
* 游标条件能被我方的 MySQL 8.4 压进 UNION 分支,从而用上 created_at 索引。
*/
List<TaskCenterTaskRow> selectTaskPage(@Param("userIds") List<Long> userIds,
@Param("moduleType") String moduleType,
@Param("statuses") List<String> statuses,
@@ -224,7 +245,8 @@ public interface TaskCenterQueryMapper {
@Param("startTime") LocalDateTime startTime,
@Param("endTime") LocalDateTime endTime,
@Param("limit") long limit,
@Param("offset") long offset);
@Param("afterCreatedAt") LocalDateTime afterCreatedAt,
@Param("afterId") Long afterId);
long countTaskPage(@Param("userIds") List<Long> userIds,
@Param("moduleType") String moduleType,
@@ -51,6 +51,11 @@ public class TaskCenterQueryService {
private static final long ACTIVE_TASK_SCAN_CAP = 500;
private static final long DEFAULT_PAGE_SIZE = 20;
private static final long MAX_PAGE_SIZE = 100;
/**
* AC-19(审计 P1-B1):任务中心默认时间窗(天)。请求不带 {@code startTime} 时由服务层补上,
* 页码跳转与总数都只在这个窗口里发生;带 {@code startTime} 就完全按请求走。
*/
private static final int DEFAULT_WINDOW_DAYS = 90;
/**
* 心跳位置读取封顶,与页大小同源:看板一页最多 100 行、30 秒一轮。
* 沿用 light 侧那个 20 的话,第 21 行之后的在跑任务永远读不到进度。
@@ -117,15 +122,34 @@ public class TaskCenterQueryService {
}
List<String> statuses = statusFilter(query.getStatus());
// AC-19(审计 P1-B1):时间窗先补再传,count 与页查询共用同一个窗口口径。
LocalDateTime startTime = startOfDefaultWindow(query.getStartTime());
LocalDateTime endTime = query.getEndTime();
long total = queryMapper.countTaskPage(userIds, query.getModuleType(), statuses,
query.getShop(), query.getStartTime(), query.getEndTime());
query.getShop(), startTime, endTime);
if (total <= 0) {
return TaskCenterTaskPageVo.empty(page, pageSize);
}
// 深翻页的 skip 交给窄锚点语句(只投影 createdAt,id 两列),页查询本身走 keyset 游标:
// 游标条件能被 MySQL 压进 UNION 分支用上 created_at 索引,OFFSET 不能,只能物化整个 UNION。
long offset = (page - 1) * pageSize;
LocalDateTime afterCreatedAt = null;
Long afterId = null;
if (offset > 0) {
TaskCenterTaskRow anchor = queryMapper.selectTaskPageAnchor(userIds, query.getModuleType(),
statuses, query.getShop(), startTime, endTime, offset);
if (anchor == null || anchor.getCreatedAt() == null || anchor.getId() == null) {
// 页码落在窗口之外:与旧实现取到空行同样返回空页,不拿 null 游标去要第一页
return TaskCenterTaskPageVo.empty(page, pageSize);
}
afterCreatedAt = anchor.getCreatedAt();
afterId = anchor.getId();
}
List<TaskCenterTaskRow> rows = queryMapper.selectTaskPage(userIds, query.getModuleType(),
statuses, query.getShop(), query.getStartTime(), query.getEndTime(),
pageSize, (page - 1) * pageSize);
statuses, query.getShop(), startTime, endTime,
pageSize, afterCreatedAt, afterId);
if (rows == null || rows.isEmpty()) {
return TaskCenterTaskPageVo.empty(page, pageSize);
}
@@ -138,6 +162,17 @@ public class TaskCenterQueryService {
return vo;
}
/**
* 默认时间窗起点(AC-19/P1-B1):没带 {@code startTime} 才补 {@code now - 90 天}。
*
* <p>这一条给 UNION 的两个分支各加一条 {@code created_at} 下界,是把「全历史物化再排序」
* 换成「按索引取窗口」的那一刀。代价是对管理员可见的行为收紧:90 天前的任务不再默认出现在
* 列表与总数里,必须显式传 startTime 才能翻旧账——Phase 6 真机总验收要按这条复现一次翻页路径。
*/
static LocalDateTime startOfDefaultWindow(LocalDateTime requested) {
return requested != null ? requested : LocalDateTime.now().minusDays(DEFAULT_WINDOW_DAYS);
}
/**
* 状态筛选展开:空白 = 不裁剪;{@code ACTIVE} = 前端默认的「执行中」,等于三个进行中状态的合并
* (与判卡住用的 {@link #ACTIVE_STATUSES} 同一组,不另定口径);{@code SUCCESS}/{@code FAILED}
@@ -99,7 +99,7 @@ public class AdminUserApiSecretController {
@Valid @RequestBody UserApiSecretSaveRequest body) {
AdminUserEntity operator = adminAuthSupport.requireAdmin(request);
return ApiResponse.success("已代填并锁定",
userApiSecretService.adminSaveModule(operator, userId, moduleKey, body.getValue()));
userApiSecretService.adminSaveModule(operator, userId, moduleKey, body.getValue(), body.getChannel()));
}
@DeleteMapping("/{userId}/modules/{moduleKey}")
@@ -62,7 +62,7 @@ public class UserApiSecretController {
@Parameter(description = "密钥模块 key", required = true) @PathVariable String moduleKey,
@Valid @RequestBody UserApiSecretSaveRequest body) {
Long userId = currentUserId(request);
return ApiResponse.success("保存成功", userApiSecretService.save(userId, moduleKey, body.getValue()));
return ApiResponse.success("保存成功", userApiSecretService.save(userId, moduleKey, body.getValue(), body.getChannel()));
}
@DeleteMapping("/{moduleKey}")
@@ -85,7 +85,8 @@ public class UserApiSecretController {
@RequestBody(required = false) UserApiSecretCheckRequest body) {
Long userId = currentUserId(request);
String overrideValue = body == null ? null : body.getValue();
return ApiResponse.success("检测完成", userApiSecretService.check(userId, moduleKey, overrideValue));
String channel = body == null ? null : body.getChannel();
return ApiResponse.success("检测完成", userApiSecretService.check(userId, moduleKey, overrideValue, channel));
}
@PostMapping("/migrate")
@@ -9,4 +9,8 @@ public class UserApiSecretCheckRequest {
@Schema(description = "待检测的密钥明文;为空时检测服务端已保存的密钥(结果落库),非空时仅检测输入值(不落库)")
private String value;
@Schema(description = "待检测的货源(未保存的下拉选择):LLM 密钥填 longtaitou/t8star,代理填 qingguo/jikip;"
+ "为空时按服务端已存行的渠道/供应商检测")
private String channel;
}
@@ -24,5 +24,8 @@ public class UserApiSecretMigrateRequest {
@Schema(description = "密钥明文", requiredMode = Schema.RequiredMode.REQUIRED)
private String value;
@Schema(description = "货源:LLM 密钥填 longtaitou/t8star,代理填 qingguo/jikip;为空时按模块默认值")
private String channel;
}
}
@@ -11,4 +11,7 @@ public class UserApiSecretSaveRequest {
@NotBlank(message = "密钥不能为空")
@Schema(description = "密钥明文(服务端加密存储)", requiredMode = Schema.RequiredMode.REQUIRED)
private String value;
@Schema(description = "货源:LLM 密钥填 longtaitou/t8star(缺省=龙抬头);代理填 qingguo/jikip(缺省=青果)")
private String channel;
}
@@ -15,6 +15,8 @@ public class UserApiSecretEntity {
private Long id;
private Long userId;
private String moduleKey;
/** 货源:LLM 行为渠道(longtaitou/t8star),代理行为供应商(qingguo/jikip),见 V180。 */
private String channel;
private String secretValue;
private String checkStatus;
private String checkCode;
@@ -15,6 +15,9 @@ public class AdminUserSecretModuleVo {
@Schema(description = "密钥模块显示名")
private String moduleLabel;
@Schema(description = "货源:LLM 行为渠道(longtaitou/t8star),代理行为供应商(qingguo/jikip)")
private String channel;
@Schema(description = "脱敏值")
private String masked;
@@ -15,6 +15,9 @@ public class UserApiSecretItemVo {
@Schema(description = "密钥模块显示名")
private String moduleLabel;
@Schema(description = "货源:LLM 行为渠道(longtaitou/t8star),代理行为供应商(qingguo/jikip);未配置时为空")
private String channel;
@Schema(description = "脱敏值,如 sk-a****1234")
private String masked;
@@ -9,6 +9,8 @@ import com.nanri.aiimage.config.SimilarAsinProperties;
import com.nanri.aiimage.config.UserSecretProperties;
import com.nanri.aiimage.modules.usersecret.client.JikipProxyClient;
import com.nanri.aiimage.modules.usersecret.support.UserSecretModule;
import com.nanri.aiimage.modules.usersecret.support.ProxyProvider;
import com.nanri.aiimage.modules.usersecret.support.LlmChannel;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.HttpHeaders;
@@ -93,8 +95,16 @@ public class UserApiSecretCheckService {
* 代理模块不重试——jikip 按提取次数计费,重试会多扣一次。
*/
public CheckOutcome probe(UserSecretModule module, String plainApiKey) {
return probe(module, plainApiKey, null);
}
/**
* 探测入口(带货源):LLM 密钥按所选渠道取主机;代理按所选供应商解析提取响应。
* 货源为空时沿用历史口径(模块全局主机 / 极客解析)。
*/
public CheckOutcome probe(UserSecretModule module, String plainApiKey, String channel) {
if (module == UserSecretModule.PROXY) {
return probeProxy(plainApiKey);
return probeProxy(plainApiKey, channel);
}
String proxyUrl;
try {
@@ -104,14 +114,14 @@ public class UserApiSecretCheckService {
return new CheckOutcome(STATUS_FAILED, CODE_INSUFFICIENT_BALANCE, INSUFFICIENT_BALANCE_MESSAGE, null, false);
}
if (proxyUrl != null) {
CheckOutcome viaProxy = probeOnce(module, plainApiKey, proxyUrl, true);
CheckOutcome viaProxy = probeOnce(module, plainApiKey, proxyUrl, true, channel);
if (isTransportFailure(viaProxy)) {
// 掩码后再打日志:proxyUrl 含用户代理账密,明文入日志等于持久化敏感凭据
log.warn("[user-secret][check] 经代理检测失败 module={} proxy={} code={},回退直连重试",
module.key(), SecretMasking.maskProxy(proxyUrl), viaProxy.code());
CheckOutcome direct = probeOnce(module, plainApiKey, null, false);
CheckOutcome direct = probeOnce(module, plainApiKey, null, false, channel);
if (isTransportFailure(direct)) {
direct = retryOnce(module, plainApiKey, direct);
direct = retryOnce(module, plainApiKey, direct, channel);
}
return new CheckOutcome(
direct.status(),
@@ -122,16 +132,16 @@ public class UserApiSecretCheckService {
}
return viaProxy;
}
CheckOutcome direct = probeOnce(module, plainApiKey, null, false);
return isTransportFailure(direct) ? retryOnce(module, plainApiKey, direct) : direct;
CheckOutcome direct = probeOnce(module, plainApiKey, null, false, channel);
return isTransportFailure(direct) ? retryOnce(module, plainApiKey, direct, channel) : direct;
}
/** 传输层失败重试一次(间隔 {@link #RETRY_BACKOFF_MILLIS}):抗上游偶发单请求不响应。 */
private CheckOutcome retryOnce(UserSecretModule module, String plainApiKey, CheckOutcome failed) {
private CheckOutcome retryOnce(UserSecretModule module, String plainApiKey, CheckOutcome failed, String channel) {
log.warn("[user-secret][check] 直连探测失败 module={} code={} latency={}ms,{}ms 后重试一次",
module.key(), failed.code(), failed.latencyMs(), RETRY_BACKOFF_MILLIS);
sleepQuietly(RETRY_BACKOFF_MILLIS);
CheckOutcome retried = probeOnce(module, plainApiKey, null, false);
CheckOutcome retried = probeOnce(module, plainApiKey, null, false, channel);
if (isTransportFailure(retried)) {
log.warn("[user-secret][check] 重试仍失败 module={} code={} latency={}ms",
module.key(), retried.code(), retried.latencyMs());
@@ -167,6 +177,10 @@ public class UserApiSecretCheckService {
* 「假通过」(3-39ms + HTTP 301),余额不足 / 链接失效都测不出来。
*/
private CheckOutcome probeProxy(String proxyUrl) {
return probeProxy(proxyUrl, null);
}
private CheckOutcome probeProxy(String proxyUrl, String channel) {
String normalized = proxyUrl == null ? "" : proxyUrl.trim();
if (normalized.isEmpty()) {
return new CheckOutcome(STATUS_FAILED, CODE_INVALID_KEY, "代理地址为空,请重新配置", null, true);
@@ -178,31 +192,103 @@ public class UserApiSecretCheckService {
return probeThroughProxy(normalized, startMillis);
}
// 提取链接:先真实提取一次
String body;
// 供应商:显式选择优先,其次按链接域名识别,最后按默认(青果)
ProxyProvider provider = ProxyProvider.parse(channel);
if (provider == null) {
ProxyProvider detected = ProxyProvider.detectFromUrl(normalized);
provider = detected == null ? ProxyProvider.DEFAULT : detected;
}
// 提取链接:先真实提取一次。刻意读响应体而不依赖 2xx——
// 青果的失败码(API_AUTH_DENY / BALANCE_INSUFFICIENT)走 HTTP 4xx,
// 交给 RestClient 抛异常会把「密钥没授权」误报成「接口无法访问」。
StatusAndBody extract;
try {
body = clientFor(null).get().uri(normalized).retrieve().body(String.class);
extract = clientFor(null).get().uri(normalized)
.exchange((request, response) -> new StatusAndBody(
response.getStatusCode().value(),
readResponseBodyBounded(response.getBody())));
} catch (Exception ex) {
long latency = System.currentTimeMillis() - startMillis;
log.warn("[user-secret][check] 代理提取接口不可达 latency={}ms err={}", latency, ex.getMessage());
log.warn("[user-secret][check] 代理提取接口不可达 provider={} latency={}ms err={}",
provider.key(), latency, ex.getMessage());
return new CheckOutcome(STATUS_FAILED, CODE_NETWORK_ERROR,
"代理提取接口无法访问,请检查提取链接或稍后重试", (int) latency, false);
}
long extractLatency = System.currentTimeMillis() - startMillis;
if (isInsufficientBalance(body)) {
log.warn("[user-secret][check] 代理提取被拒:供应商余额不足 latency={}ms", extractLatency);
return new CheckOutcome(STATUS_FAILED, CODE_INSUFFICIENT_BALANCE,
INSUFFICIENT_BALANCE_MESSAGE, (int) extractLatency, false);
CheckOutcome rejection = classifyProviderRejection(provider, extract.body(), (int) extractLatency);
if (rejection != null) {
log.warn("[user-secret][check] 代理提取被供应商拒绝 provider={} httpStatus={} code={}",
provider.key(), extract.statusCode(), rejection.code());
return rejection;
}
String extractedProxyUrl = extractProxyUrlFromBody(body);
String extractedProxyUrl = extractProxyUrlFromBody(extract.body());
if (extractedProxyUrl == null) {
log.warn("[user-secret][check] 代理提取响应未解析出 IP,响应前 150 字={}", abbreviate(body, 150));
log.warn("[user-secret][check] 代理提取响应未解析出 IP provider={} httpStatus={} 响应前 150 字={}",
provider.key(), extract.statusCode(), abbreviate(extract.body(), 150));
return new CheckOutcome(STATUS_FAILED, CODE_INVALID_KEY,
"链接未返回可用 IP,请检查代理提取配置", (int) extractLatency, false);
}
if (provider == ProxyProvider.QINGGUO) {
// 青果通道提取返回的是**隧道端点**(如 tunpool-xxx.qg.net:16359),连它要用
// Authkey/Authpwd 做 Basic 认证,而这对账密只存在用户本机、不上报服务端;
// 所以这里只验"提取接口可用且拿到端点",连接态由客户端的实际任务证明。
log.info("[user-secret][check] 青果提取成功,跳过经隧道探测(端点鉴权在客户端)endpoint={}",
SecretMasking.maskProxy(extractedProxyUrl));
return new CheckOutcome(STATUS_PASSED, CODE_OK,
"提取成功(青果隧道端点,连接鉴权在客户端)", (int) extractLatency, false);
}
return probeThroughProxy(extractedProxyUrl, startMillis);
}
/**
* 供应商拒付/鉴权失败的独立分类(两家错误码词表不同):
* <ul>
* <li>青果:code 为字符串。BALANCE_INSUFFICIENT=余额不足;INVALID_KEY/UNAVAILABLE_KEY/KEY_BLOCK/
* ACCESS_DENY/API_AUTH_DENY=密钥不可用或未授权;REQUEST_LIMIT_EXCEEDED=限流;
* INVALID_PARAMETER=参数错;其余业务失败按上游异常。</li>
* <li>极客:沿用历史口径(code=-1 或 message 含「余额不足」=余额不足)。</li>
* </ul>
* 返回 null 表示不是拒付/鉴权类失败,继续走 IP 解析。
*/
CheckOutcome classifyProviderRejection(ProxyProvider provider, String body, int latencyMs) {
if (provider == ProxyProvider.QINGGUO) {
JsonNode root = parseJson(body);
if (root == null || !root.isObject()) {
return null;
}
JsonNode codeNode = root.get("code");
String code = codeNode == null || codeNode.isNull() ? "" : codeNode.asText("").trim();
if (code.isEmpty() || "SUCCESS".equalsIgnoreCase(code) || "0".equals(code)) {
return null;
}
String message = text(root.get("message"));
String detail = message == null || message.isBlank() ? "" : ":" + abbreviate(message, 120);
String upper = code.toUpperCase(java.util.Locale.ROOT);
return switch (upper) {
case "BALANCE_INSUFFICIENT" ->
new CheckOutcome(STATUS_FAILED, CODE_INSUFFICIENT_BALANCE, INSUFFICIENT_BALANCE_MESSAGE, latencyMs, false);
case "INVALID_KEY", "UNAVAILABLE_KEY", "KEY_BLOCK", "ACCESS_DENY", "API_AUTH_DENY" ->
new CheckOutcome(STATUS_FAILED, CODE_FORBIDDEN,
"青果密钥不可用或未授权(" + upper + ")" + detail, latencyMs, false);
case "REQUEST_LIMIT_EXCEEDED" ->
new CheckOutcome(STATUS_ERROR, CODE_RATE_LIMITED, "青果提取触发限流,本次无法判定" + detail, latencyMs, false);
case "NO_AVAILABLE_CHANNEL" ->
// 通道被占满(存活期内一个通道只给一个端点):不是密钥问题,稍后重试即可
new CheckOutcome(STATUS_ERROR, CODE_PROVIDER_ERROR,
"青果通道已全部占用,请等当前通道释放(约 3 分钟)后重试" + detail, latencyMs, false);
case "INVALID_PARAMETER" ->
new CheckOutcome(STATUS_FAILED, CODE_BAD_REQUEST, "青果提取参数不正确" + detail, latencyMs, false);
default ->
new CheckOutcome(STATUS_FAILED, CODE_PROVIDER_ERROR, "青果提取失败(" + upper + ")" + detail, latencyMs, false);
};
}
if (isInsufficientBalance(body)) {
return new CheckOutcome(STATUS_FAILED, CODE_INSUFFICIENT_BALANCE, INSUFFICIENT_BALANCE_MESSAGE, latencyMs, false);
}
return null;
}
/** 经指定代理请求自家域名:任意 HTTP 响应即转发可用;407 为代理自身认证失败;网络异常不可达。 */
private CheckOutcome probeThroughProxy(String proxyUrl, long startMillis) {
try {
@@ -245,7 +331,7 @@ public class UserApiSecretCheckService {
}
}
/** 提取响应解析:JSON data.list[0] 的 ip/port(账密可选)优先,正文 ip:port 正则兜底。 */
/** 提取响应解析:青果 data[0].server 优先;极客 data.list[0] 的 ip/port(账密可选)其次;正文 ip:port 正则兜底。 */
String extractProxyUrlFromBody(String body) {
String text = body == null ? "" : body.trim();
if (text.isEmpty()) {
@@ -254,6 +340,10 @@ public class UserApiSecretCheckService {
if (text.charAt(0) == '{' || text.charAt(0) == '[') {
JsonNode root = parseJson(text);
if (root != null) {
String qingguo = qingguoProxyUrl(root);
if (qingguo != null) {
return qingguo;
}
JsonNode list = root.path("data").path("list");
if (list.isArray() && list.size() > 0) {
JsonNode first = list.get(0);
@@ -277,7 +367,37 @@ public class UserApiSecretCheckService {
return null;
}
/** 提取响应是否表明供应商欠费(如 jikip {"code":-1,"data":null,"message":"余额不足"})。 */
/**
* 青果提取响应解析:{@code {"code":"SUCCESS","data":[{"proxy_ip":"1.2.3.4","server":"1.2.3.4:8080",...}]}}。
* <b>server 才是代理地址</b>,proxy_ip 只是出口 IP(青果文档明确区分)——拿错字段会得到一个
* 没端口的裸 IP,探测直接失败。缺 server 时按 ip/port 两字段兜底;都没有返回 null 交给上层报错。
*/
private String qingguoProxyUrl(JsonNode root) {
JsonNode data = root.path("data");
if (!data.isArray() || data.isEmpty()) {
return null;
}
JsonNode first = data.get(0);
if (first == null || !first.isObject()) {
return null;
}
String server = text(first.get("server"));
if (server != null && !server.isBlank()) {
String trimmed = server.trim();
return trimmed.startsWith("http://") || trimmed.startsWith("https://")
? trimmed
: "http://" + trimmed;
}
String ip = text(first.get("ip"));
String port = text(first.get("port"));
if (ip != null && !ip.isBlank() && port != null && !port.isBlank()) {
return "http://" + ip + ":" + port;
}
return null;
}
/** 提取响应是否表明供应商欠费(极客 {"code":-1,...,"message":"余额不足"} / 青果 code=BALANCE_INSUFFICIENT)。 */
boolean isInsufficientBalance(String body) {
String text = body == null ? "" : body.trim();
if (text.isEmpty()) {
@@ -290,16 +410,24 @@ public class UserApiSecretCheckService {
return true;
}
JsonNode code = root.get("code");
if (code != null && !code.isNull() && code.asDouble(Double.NaN) == -1D) {
return true;
if (code != null && !code.isNull()) {
if (code.isNumber() && code.asDouble(Double.NaN) == -1D) {
return true;
}
if (code.isTextual() && "BALANCE_INSUFFICIENT".equalsIgnoreCase(code.asText().trim())) {
return true;
}
}
}
return text.contains("余额不足");
}
private CheckOutcome probeOnce(UserSecretModule module, String plainApiKey, String proxyUrl, boolean viaProxy) {
String host = module.resolveLlmHost(appearancePatentProperties, similarAsinProperties);
String model = userSecretProperties.getCheckModel();
private CheckOutcome probeOnce(UserSecretModule module, String plainApiKey, String proxyUrl, boolean viaProxy,
String channel) {
String host = module.resolveLlmHost(appearancePatentProperties, similarAsinProperties, channel);
// 检测也要按渠道换模型名:龙抬头那边叫 gemini-3.1-flash-lite,发贞贞的名字会 404/model_not_found,
// 用户会误以为自己的密钥坏了
String model = LlmChannel.modelForHost(host, userSecretProperties.getCheckModel());
String url = joinUrl(host, "/v1/chat/completions");
String key = stripBearer(plainApiKey);
long startMillis = System.currentTimeMillis();
@@ -6,6 +6,8 @@ import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.modules.admin.spi.UserSecretCleanupPort;
import com.nanri.aiimage.common.security.ShopCredentialCryptoService;
import com.nanri.aiimage.config.UserSecretProperties;
import com.nanri.aiimage.config.AppearancePatentProperties;
import com.nanri.aiimage.config.SimilarAsinProperties;
import com.nanri.aiimage.common.security.AdminAuthSupport;
import com.nanri.aiimage.common.security.DeviceSessionPolicy;
import com.nanri.aiimage.common.service.ReadScope;
@@ -28,6 +30,8 @@ import com.nanri.aiimage.modules.usersecret.model.vo.UserApiSecretBundleVo;
import com.nanri.aiimage.modules.usersecret.model.vo.UserApiSecretCheckResultVo;
import com.nanri.aiimage.modules.usersecret.model.vo.UserApiSecretItemVo;
import com.nanri.aiimage.modules.usersecret.support.UserSecretModule;
import com.nanri.aiimage.modules.usersecret.support.LlmChannel;
import com.nanri.aiimage.modules.usersecret.support.ProxyProvider;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.data.redis.core.StringRedisTemplate;
@@ -98,6 +102,8 @@ public class UserApiSecretService implements UserSecretCleanupPort {
private final UserDataScopeSupport userDataScopeSupport;
private final NotificationDispatchService notificationDispatchService;
private final UserSecretProperties userSecretProperties;
private final AppearancePatentProperties appearancePatentProperties;
private final SimilarAsinProperties similarAsinProperties;
private final StringRedisTemplate stringRedisTemplate;
private final ObjectMapper objectMapper;
@@ -117,6 +123,12 @@ public class UserApiSecretService implements UserSecretCleanupPort {
return vo;
}
/** 兼容旧签名:不传货源时按模块默认值保存。 */
@Transactional
public UserApiSecretItemVo save(Long userId, String moduleKey, String value) {
return save(userId, moduleKey, value, null);
}
/**
* 保存密钥:加密落库并重置检测状态为 unknown;
* 若该值与用户刚检测过的输入值完全一致,则沿用那次检测结果(见 {@link #applyPendingVerification}),
@@ -124,7 +136,7 @@ public class UserApiSecretService implements UserSecretCleanupPort {
* 后台代填并锁定的模块(source=admin)客户端不可覆盖。
*/
@Transactional
public UserApiSecretItemVo save(Long userId, String moduleKey, String value) {
public UserApiSecretItemVo save(Long userId, String moduleKey, String value, String channel) {
requireUserId(userId);
UserSecretModule module = requireModule(moduleKey);
requireNotAdminLocked(userId, module, "修改");
@@ -135,9 +147,15 @@ public class UserApiSecretService implements UserSecretCleanupPort {
if (module == UserSecretModule.PROXY) {
validateProxyValue(plainValue);
}
upsert(userId, module.key(), plainValue, SOURCE_CLIENT);
boolean carried = applyPendingVerification(userId, module, plainValue);
log.info("[user-secret] 密钥已保存 userId={} module={} 沿用检测结果={}", userId, module.key(), carried);
// 代理没给供应商时按链接域名识别(老客户端只上报地址):极客链接别记成青果。
// LLM 密钥没给渠道才按产品默认(新密钥默认龙抬头)。
String channelKey = hasText(channel)
? module.requireChannel(channel)
: (module.isProxy() ? module.channelOrDefault(null, plainValue) : module.defaultChannel());
upsert(userId, module.key(), plainValue, SOURCE_CLIENT, channelKey);
boolean carried = applyPendingVerification(userId, module, plainValue, channelKey);
log.info("[user-secret] 密钥已保存 userId={} module={} channel={} 沿用检测结果={}",
userId, module.key(), channelKey, carried);
return toItem(module, selectOne(userId, module.key()));
}
@@ -153,13 +171,20 @@ public class UserApiSecretService implements UserSecretCleanupPort {
log.info("[user-secret] 密钥已清空 userId={} module={}", userId, module.key());
}
/** 兼容旧签名:后台代填不传货源时按链接识别,识别不出回落模块默认值(代理=青果)。 */
@Transactional
public AdminUserSecretModuleVo adminSaveModule(AdminUserEntity operator, Long userId, String moduleKey, String value) {
return adminSaveModule(operator, userId, moduleKey, value, null);
}
/**
* 后台代填并锁定:写入后该模块 source=admin,客户端不可修改或清空。
* 目前仅开放代理设置——密钥模块要由客户端提交任务时携带本地明文,代填会让用户用不了。
* 数据权限:超管可操作全部用户,其余管理员限自己数据权限范围内的用户。
*/
@Transactional
public AdminUserSecretModuleVo adminSaveModule(AdminUserEntity operator, Long userId, String moduleKey, String value) {
public AdminUserSecretModuleVo adminSaveModule(AdminUserEntity operator, Long userId, String moduleKey,
String value, String channel) {
requireUserId(userId);
requireVisibleUser(operator, userId);
UserSecretModule module = requireModule(moduleKey);
@@ -171,9 +196,13 @@ public class UserApiSecretService implements UserSecretCleanupPort {
throw new BusinessException("代理地址不能为空");
}
validateProxyValue(plainValue);
upsert(userId, module.key(), plainValue, SOURCE_ADMIN);
log.info("[user-secret] 后台代填并锁定配置 userId={} module={} operatorId={}",
userId, module.key(), operator == null ? null : operator.getId());
// 后台代填没给供应商时按链接域名识别(代填的是极客链接就记 jikip),识别不出才用默认青果
String channelKey = hasText(channel)
? module.requireChannel(channel)
: module.channelOrDefault(null, plainValue);
upsert(userId, module.key(), plainValue, SOURCE_ADMIN, channelKey);
log.info("[user-secret] 后台代填并锁定配置 userId={} module={} channel={} operatorId={}",
userId, module.key(), channelKey, operator == null ? null : operator.getId());
return toModuleVo(module, selectOne(userId, module.key()));
}
@@ -234,7 +263,12 @@ public class UserApiSecretService implements UserSecretCleanupPort {
if (existing != null && hasText(existing.getSecretValue())) {
continue;
}
upsert(userId, module.get().key(), plainValue, SOURCE_MIGRATED);
// 老客户端只上报明文、不上报货源:这类存量密钥当时跑的就是贞贞/极客,
// 不能按「新建默认」落成龙抬头/青果(会把老密钥发到它没开通过的渠道)。
String migratedChannel = hasText(item.getChannel())
? module.get().requireChannel(item.getChannel())
: legacyChannel(module.get());
upsert(userId, module.get().key(), plainValue, SOURCE_MIGRATED, migratedChannel);
migrated++;
}
log.info("[user-secret] 本地密钥迁移完成 userId={} 提交={} 实际写入={}", userId, items.size(), migrated);
@@ -258,23 +292,68 @@ public class UserApiSecretService implements UserSecretCleanupPort {
}
}
/**
* 任务执行兜底读取「货源」:未配置返回空串(调用方按模块默认值处理)。
* 代理模块额外做链接域名识别,兜住老数据(只有链接、没有供应商列)的存量行。
*/
public String findChannel(Long userId, String moduleKey) {
if (userId == null || userId <= 0 || !hasText(moduleKey)) {
return "";
}
Optional<UserSecretModule> module = UserSecretModule.of(moduleKey);
if (module.isEmpty()) {
return "";
}
// 刻意不在这里兜异常(与 findPlainValue 不同):读库失败时回落空串等于"按默认渠道发",
// 会把贞贞的密钥静默发到龙抬头——错渠道比一次可见的失败更难查。让它抛,任务当场失败。
UserApiSecretEntity row = selectOne(userId, module.get().key());
String stored = row == null ? "" : row.getChannel();
if (module.get().isProxy()) {
String plain = row == null ? "" : normalize(decryptQuietly(row.getSecretValue()));
return module.get().channelOrDefault(stored, plain);
}
return module.get().channelOrDefault(stored, null);
}
/**
* 按用户密钥所属渠道解析 LLM 主机:任务执行与检测共用同一口径。
* 用户没配密钥时返回模块全局配置主机,保证后续报错来自「没有密钥」而不是「主机为空」。
*/
public String resolveLlmHost(Long userId, UserSecretModule module) {
return module.resolveLlmHost(appearancePatentProperties, similarAsinProperties,
findChannel(userId, module.key()));
}
/** 存量迁移/回填的货源:LLM=贞贞,代理=极客(本功能上线前系统实际在用的两家)。 */
private String legacyChannel(UserSecretModule module) {
return module.isProxy() ? ProxyProvider.JIKIP.key() : LlmChannel.T8STAR.key();
}
/** 兼容旧签名:不传货源时按已存行/模块默认值检测。 */
public UserApiSecretCheckResultVo check(Long userId, String moduleKey, String overrideValue) {
return check(userId, moduleKey, overrideValue, null);
}
/**
* 检测:传 overrideValue 时只检测输入值,结果暂存(不落库),保存同一个值时可沿用;
* 不传时检测已存值并直接落库。
*
* <p>货源(渠道/供应商)优先取本次请求带的选择,其次取服务端已存行;都为空时按模块默认值。
* 检测必须按同一个货源发起——渠道选错时「连通」是个假结论。</p>
*/
public UserApiSecretCheckResultVo check(Long userId, String moduleKey, String overrideValue) {
public UserApiSecretCheckResultVo check(Long userId, String moduleKey, String overrideValue, String channel) {
requireUserId(userId);
UserSecretModule module = requireModule(moduleKey);
String override = normalize(overrideValue);
String plainKey = override;
boolean persist = false;
UserApiSecretEntity existingRow = selectOne(userId, module.key());
if (plainKey.isEmpty()) {
UserApiSecretEntity row = selectOne(userId, module.key());
if (row == null || !hasText(row.getSecretValue())) {
if (existingRow == null || !hasText(existingRow.getSecretValue())) {
throw new BusinessException(module == UserSecretModule.PROXY
? "请先保存代理地址后再检测" : "请先保存密钥后再检测");
}
plainKey = normalize(cryptoService.decrypt(row.getSecretValue()));
plainKey = normalize(cryptoService.decrypt(existingRow.getSecretValue()));
if (plainKey.isEmpty()) {
throw new BusinessException("配置内容为空,请重新配置");
}
@@ -282,14 +361,18 @@ public class UserApiSecretService implements UserSecretCleanupPort {
} else if (module == UserSecretModule.PROXY) {
validateProxyValue(plainKey);
}
UserApiSecretCheckService.CheckOutcome outcome = reuseFreshOutcome(module, plainKey);
String requestedChannel = hasText(channel)
? channel
: (existingRow == null ? null : existingRow.getChannel());
String channelKey = module.channelOrDefault(requestedChannel, plainKey);
UserApiSecretCheckService.CheckOutcome outcome = reuseFreshOutcome(module, plainKey, channelKey);
if (outcome != null) {
log.info("[user-secret] 新鲜期内重复检测相同配置,复用上次结果(代理按次计费,避免连点重复提取)userId={} module={}",
userId, module.key());
log.info("[user-secret] 新鲜期内重复检测相同配置,复用上次结果(代理按次计费,避免连点重复提取)userId={} module={} channel={}",
userId, module.key(), channelKey);
} else {
outcome = checkService.probe(module, plainKey);
outcome = checkService.probe(module, plainKey, channelKey);
if (UserApiSecretCheckService.STATUS_PASSED.equals(outcome.status())) {
rememberFreshOutcome(module, plainKey, outcome);
rememberFreshOutcome(module, plainKey, channelKey, outcome);
}
}
UserApiSecretCheckResultVo vo = toCheckResult(module, outcome);
@@ -297,10 +380,10 @@ public class UserApiSecretService implements UserSecretCleanupPort {
applyCheckOutcome(userId, module.key(), outcome);
vo.setCheckedAt(LocalDateTime.now());
} else {
rememberPendingVerification(userId, module, plainKey, outcome);
rememberPendingVerification(userId, module, plainKey, channelKey, outcome);
}
log.info("[user-secret] 检测完成 userId={} module={} status={} code={} viaProxy={} latency={}ms persist={}",
userId, module.key(), outcome.status(), outcome.code(), outcome.viaProxy(), outcome.latencyMs(), persist);
log.info("[user-secret] 检测完成 userId={} module={} channel={} status={} code={} viaProxy={} latency={}ms persist={}",
userId, module.key(), channelKey, outcome.status(), outcome.code(), outcome.viaProxy(), outcome.latencyMs(), persist);
return vo;
}
@@ -517,7 +600,8 @@ public class UserApiSecretService implements UserSecretCleanupPort {
UserApiSecretCheckService.CODE_INVALID_KEY,
"配置内容为空,请重新配置", null, false);
} else {
outcome = checkService.probe(module, plainValue);
outcome = checkService.probe(module, plainValue,
module.channelOrDefault(row.getChannel(), plainValue));
}
} catch (Exception ex) {
log.warn("[user-secret] 后台检测解密失败 userId={} module={} err={}", userId, module.key(), ex.getMessage());
@@ -599,7 +683,8 @@ public class UserApiSecretService implements UserSecretCleanupPort {
UserApiSecretCheckService.STATUS_FAILED,
UserApiSecretCheckService.CODE_INVALID_KEY,
"配置内容为空,请重新配置", null, false)
: checkService.probe(module.get(), plainKey);
: checkService.probe(module.get(), plainKey,
module.get().channelOrDefault(row.getChannel(), plainKey));
outcomeBySecret.put(groupKey, outcome);
probed++;
sleepQuietly(200L);
@@ -642,7 +727,8 @@ public class UserApiSecretService implements UserSecretCleanupPort {
/** 巡检分组键:同模块 + 同密文视为同一把密钥。密文确定性加密,无需解密即可分组。 */
private String inspectionGroupKey(UserApiSecretEntity row) {
return row.getModuleKey() + '|' + row.getSecretValue();
// 渠道参与分组:同一串密钥在龙抬头与贞贞上是两次独立探测,合并会拿 A 的结论冒充 B
return row.getModuleKey() + '|' + (row.getChannel() == null ? "" : row.getChannel()) + '|' + row.getSecretValue();
}
/**
@@ -860,12 +946,13 @@ public class UserApiSecretService implements UserSecretCleanupPort {
return earliest;
}
private void upsert(Long userId, String moduleKey, String plainValue, String source) {
private void upsert(Long userId, String moduleKey, String plainValue, String source, String channel) {
LocalDateTime now = LocalDateTime.now();
UserApiSecretEntity existing = selectOne(userId, moduleKey);
UserApiSecretEntity row = existing == null ? new UserApiSecretEntity() : existing;
row.setUserId(userId);
row.setModuleKey(moduleKey);
row.setChannel(channel);
row.setSecretValue(cryptoService.encrypt(plainValue));
row.setCheckStatus(STATUS_UNKNOWN);
row.setCheckCode("");
@@ -906,20 +993,20 @@ public class UserApiSecretService implements UserSecretCleanupPort {
}
/**
* 检测值指纹:SHA-256(uid|module|明文)。用于判断「要保存的值」是否就是刚检测过的那个值,
* 只在 Redis 暂存里做比对,不落库、不下发。
* 检测值指纹:SHA-256(uid|module|channel|明文)。用于判断「要保存的值」是否就是刚检测过的那个值,
* 只在 Redis 暂存里做比对,不落库、不下发。渠道参与指纹:同一串密钥在不同渠道上是两件事。
*/
private String fingerprint(Long userId, UserSecretModule module, String plainValue) {
String hash = sha256Hex(userId + "|" + module.key() + "|" + plainValue);
private String fingerprint(Long userId, UserSecretModule module, String plainValue, String channel) {
String hash = sha256Hex(userId + "|" + module.key() + "|" + channel + "|" + plainValue);
if (hash.isEmpty()) {
log.warn("[user-secret] 生成检测指纹失败 userId={} module={}", userId, module.key());
log.warn("[user-secret] 生成检测指纹失败 userId={} module={} channel={}", userId, module.key(), channel);
}
return hash;
}
/** 新鲜期指纹:只按「模块 + 明文」,不含 uid —— 同一把密钥在任意账号上的检测结论是同一个。 */
private String sharedFingerprint(UserSecretModule module, String plainValue) {
return sha256Hex(module.key() + "|" + plainValue);
/** 新鲜期指纹:只按「模块 + 渠道 + 明文」,不含 uid —— 同一把密钥在同一渠道上的检测结论是同一个。 */
private String sharedFingerprint(UserSecretModule module, String plainValue, String channel) {
return sha256Hex(module.key() + "|" + channel + "|" + plainValue);
}
private String sha256Hex(String raw) {
@@ -938,8 +1025,9 @@ public class UserApiSecretService implements UserSecretCleanupPort {
/** 暂存「输入值」检测结果:TTL 内保存同一个值即沿用;暂存失败仅影响体验(用户需再检一次),不阻断检测。 */
private void rememberPendingVerification(Long userId, UserSecretModule module, String plainValue,
String channel,
UserApiSecretCheckService.CheckOutcome outcome) {
String hash = fingerprint(userId, module, plainValue);
String hash = fingerprint(userId, module, plainValue, channel);
if (hash.isEmpty()) {
return;
}
@@ -965,9 +1053,10 @@ public class UserApiSecretService implements UserSecretCleanupPort {
return FRESH_CHECK_KEY_PREFIX + ":" + module.key() + ":" + sharedHash;
}
/** 读取新鲜期结果:同值且窗口内返回上次 outcome,否则 null(走真实探测)。 */
private UserApiSecretCheckService.CheckOutcome reuseFreshOutcome(UserSecretModule module, String plainValue) {
String hash = sharedFingerprint(module, plainValue);
/** 读取新鲜期结果:同值同渠道且窗口内返回上次 outcome,否则 null(走真实探测)。 */
private UserApiSecretCheckService.CheckOutcome reuseFreshOutcome(UserSecretModule module, String plainValue,
String channel) {
String hash = sharedFingerprint(module, plainValue, channel);
if (hash.isEmpty()) {
return null;
}
@@ -990,9 +1079,9 @@ public class UserApiSecretService implements UserSecretCleanupPort {
}
/** 暂存「通过」的检测结果供新鲜期内重复检测复用;失败不暂存(允许立即重试)。 */
private void rememberFreshOutcome(UserSecretModule module, String plainValue,
private void rememberFreshOutcome(UserSecretModule module, String plainValue, String channel,
UserApiSecretCheckService.CheckOutcome outcome) {
String hash = sharedFingerprint(module, plainValue);
String hash = sharedFingerprint(module, plainValue, channel);
if (hash.isEmpty()) {
return;
}
@@ -1011,8 +1100,8 @@ public class UserApiSecretService implements UserSecretCleanupPort {
* 保存时沿用暂存结果:仅当保存值与最近一次「输入值检测」的值完全一致才落库该结果
* (passed/failed/error 一并沿用,页面据此如实展示),改过值或从未检测则维持未检测。
*/
private boolean applyPendingVerification(Long userId, UserSecretModule module, String plainValue) {
String hash = fingerprint(userId, module, plainValue);
private boolean applyPendingVerification(Long userId, UserSecretModule module, String plainValue, String channel) {
String hash = fingerprint(userId, module, plainValue, channel);
if (hash.isEmpty()) {
return false;
}
@@ -1051,6 +1140,7 @@ public class UserApiSecretService implements UserSecretCleanupPort {
vo.setModuleKey(module.key());
vo.setModuleLabel(module.label());
if (row == null) {
vo.setChannel(module.defaultChannel());
vo.setMasked("");
vo.setExists(false);
vo.setLocked(false);
@@ -1060,6 +1150,7 @@ public class UserApiSecretService implements UserSecretCleanupPort {
return vo;
}
String plain = decryptQuietly(row.getSecretValue());
vo.setChannel(module.channelOrDefault(row.getChannel(), plain));
vo.setMasked(maskValue(module, plain));
// 后台代理列要求完整可见(含账号密码),密钥列不下发明文
if (module == UserSecretModule.PROXY) {
@@ -1081,6 +1172,7 @@ public class UserApiSecretService implements UserSecretCleanupPort {
vo.setModuleKey(module.key());
vo.setModuleLabel(module.label());
if (row == null) {
vo.setChannel(module.defaultChannel());
vo.setMasked("");
vo.setExists(false);
vo.setLocked(false);
@@ -1090,6 +1182,7 @@ public class UserApiSecretService implements UserSecretCleanupPort {
return vo;
}
String plain = decryptQuietly(row.getSecretValue());
vo.setChannel(module.channelOrDefault(row.getChannel(), plain));
vo.setMasked(maskValue(module, plain));
// 锁定代理要下发完整地址:客户端据此把本地配置对齐到后台值(其余模块不下发明文)
if (module == UserSecretModule.PROXY && isAdminLocked(row)) {
@@ -0,0 +1,152 @@
package com.nanri.aiimage.modules.usersecret.support;
import java.util.Locale;
/**
* LLM 中转渠道:同一类密钥在不同中转上是**不同的 key**,所以渠道必须跟着密钥存。
*
* <p>两条渠道都走 OpenAI 兼容协议({@code POST {host}/v1/chat/completions}),
* 差别只有主机名,因此这里只登记 key / 展示名 / 主机,不另做适配层。</p>
*
* <p>默认值口径(2026-10-30 用户拍板):**新建**配置默认贞贞,龙抬头手动切换;
* 存量行同样回填为 {@link #T8STAR},两者一致。</p>
*/
public enum LlmChannel {
LONGTAITOU("longtaitou", "龙抬头", "https://longtaitou.token6688.com",
"gemini-3.1-flash-lite", true, "medium"),
T8STAR("t8star", "贞贞", "https://ai.t8star.org", null, false, null);
/** 新建配置的默认渠道(2026-10-30 用户拍板:默认仍是贞贞,龙抬头手动切)。 */
public static final LlmChannel DEFAULT = T8STAR;
private final String key;
private final String label;
private final String host;
/**
* 该渠道上可路由的模型 id;null=沿用调用方配置的模型名。
*
* <p>两家中转的模型目录不同名:本系统业务配置写的是贞贞的
* {@code gemini-3.1-flash-lite-preview-thinking-medium} 等,龙抬头那边同一个模型叫
* {@code gemini-3.1-flash-lite}(2026-10-29 用户指定)。不映射直接切渠道会 model_not_found。</p>
*/
private final String defaultModel;
/**
* 该渠道是否需要**显式**传思考参数。
*
* <p>贞贞把思考写在模型名里({@code gemini-3.1-flash-lite-preview-thinking-medium}),
* 请求体不用额外字段;龙抬头的同名模型默认不思考,要复刻现网口径必须传
* {@code enable_thinking=true}(2026-10-30 实测:不传时 2.4s / 无 reasoning_content,
* 传 medium 后 13s / 返回 10k+ 字符思考链)。</p>
*/
private final boolean thinkingParamsRequired;
/** 思考预算档位:none/minimal/low/medium/high。 */
private final String thinkingBudget;
LlmChannel(String key, String label, String host, String defaultModel,
boolean thinkingParamsRequired, String thinkingBudget) {
this.key = key;
this.label = label;
this.host = host;
this.defaultModel = defaultModel;
this.thinkingParamsRequired = thinkingParamsRequired;
this.thinkingBudget = thinkingBudget;
}
public String key() {
return key;
}
public String label() {
return label;
}
/** 渠道 API 主机基址(不带 /v1,与 properties.llmHost 同口径)。 */
public String host() {
return host;
}
/** 该渠道实际请求要用的模型 id:渠道不统一改名时原样返回配置里的模型名。 */
public String modelFor(String configuredModel) {
return defaultModel == null || defaultModel.isBlank() ? configuredModel : defaultModel;
}
/** 按主机识别渠道;识别不出返回 null(调用方按原样使用模型名)。 */
public static LlmChannel ofHost(String host) {
if (host == null || host.isBlank()) {
return null;
}
String normalized = host.trim().toLowerCase(Locale.ROOT);
for (LlmChannel channel : values()) {
if (normalized.contains(channel.host.replace("https://", ""))) {
return channel;
}
}
return null;
}
/** 按请求主机把配置的模型名换成该渠道上可路由的 id(主机为空/未知时原样返回)。 */
public static String modelForHost(String host, String configuredModel) {
LlmChannel channel = ofHost(host);
return channel == null ? configuredModel : channel.modelFor(configuredModel);
}
/** 该渠道是否需要显式思考参数(贞贞不需要:思考写在模型名里)。 */
public boolean thinkingParamsRequired() {
return thinkingParamsRequired;
}
public String thinkingBudget() {
return thinkingBudget;
}
/** 把该渠道的思考参数写进请求体;不需要显式参数的渠道原样不动。 */
public void applyThinking(java.util.Map<String, Object> body) {
if (body == null || !thinkingParamsRequired) {
return;
}
body.put("enable_thinking", true);
if (thinkingBudget != null && !thinkingBudget.isBlank()) {
body.put("thinking_budget", thinkingBudget);
}
}
/**
* 按请求主机补思考参数:业务链路用(外观专利 / 货源查询)。
* **检测链路不要调它**——检测只验密钥与连通,不思考才快且便宜。
*/
public static void applyThinkingForHost(String host, java.util.Map<String, Object> body) {
LlmChannel channel = ofHost(host);
if (channel != null) {
channel.applyThinking(body);
}
}
/** 宽松解析:空值或未知值一律回落到 {@link #DEFAULT},绝不因脏数据抛异常中断保存/检测。 */
public static LlmChannel of(String raw) {
if (raw == null || raw.isBlank()) {
return DEFAULT;
}
String normalized = raw.trim().toLowerCase(Locale.ROOT);
for (LlmChannel channel : values()) {
if (channel.key.equals(normalized)) {
return channel;
}
}
return DEFAULT;
}
/** 解析为登记过的渠道 key;未知返回 null(供保存时区分「没填」与「填错」)。 */
public static LlmChannel parse(String raw) {
if (raw == null || raw.isBlank()) {
return null;
}
String normalized = raw.trim().toLowerCase(Locale.ROOT);
for (LlmChannel channel : values()) {
if (channel.key.equals(normalized)) {
return channel;
}
}
return null;
}
}
@@ -0,0 +1,94 @@
package com.nanri.aiimage.modules.usersecret.support;
import java.util.Locale;
/**
* 代理 IP 供应商:两家的提取链接与响应体结构不同,检测/解析必须按供应商分支。
*
* <ul>
* <li>{@link #QINGGUO}:{@code https://overseas.proxy.qg.net/get?key=...&format=json},
* 成功 {@code {"code":"SUCCESS","data":[{"server":"ip:port","proxy_ip":"..."}]}},
* 失败 {@code code} 为字符串错误码(如 API_AUTH_DENY / BALANCE_INSUFFICIENT);</li>
* <li>{@link #JIKIP}:{@code https://api.jikip.com/ip-get?...},成功 {@code code=0},
* 失败 {@code code=-1} 且 message 常含「余额不足」。解析口径见 ProxyParser 模式 1/2。</li>
* </ul>
*
* <p>默认值口径(2026-10-30 用户拍板):**新建**配置默认极客,青果手动切换;
* 存量行同样回填为 {@link #JIKIP}。</p>
*/
public enum ProxyProvider {
QINGGUO("qingguo", "青果", "overseas.proxy.qg.net"),
JIKIP("jikip", "极客", "api.jikip.com");
/** 新建配置的默认供应商(2026-10-30 用户拍板:默认仍是极客,青果手动切)。 */
public static final ProxyProvider DEFAULT = JIKIP;
private final String key;
private final String label;
private final String host;
ProxyProvider(String key, String label, String host) {
this.key = key;
this.label = label;
this.host = host;
}
public String key() {
return key;
}
public String label() {
return label;
}
/** 供应商提取域名(仅用于日志与链接自检,实际请求一律用用户填写的完整链接)。 */
public String host() {
return host;
}
/** 宽松解析:空值或未知值回落到 {@link #DEFAULT}。 */
public static ProxyProvider of(String raw) {
ProxyProvider parsed = parse(raw);
return parsed == null ? DEFAULT : parsed;
}
/** 解析为登记过的供应商 key;未知返回 null。 */
public static ProxyProvider parse(String raw) {
if (raw == null || raw.isBlank()) {
return null;
}
String normalized = raw.trim().toLowerCase(Locale.ROOT);
for (ProxyProvider provider : values()) {
if (provider.key.equals(normalized)) {
return provider;
}
}
return null;
}
/**
* 按提取链接域名识别供应商:老版本客户端只上报链接、不上报供应商,
* 存量的代理配置没写 provider 时靠它兜底,避免把极客链接按青果解析(反之亦然)。
* 识别不出返回 null(调用方再回落默认值)。
*/
public static ProxyProvider detectFromUrl(String url) {
if (url == null || url.isBlank()) {
return null;
}
String lower = url.trim().toLowerCase(Locale.ROOT);
for (ProxyProvider provider : values()) {
if (lower.contains(provider.host)) {
return provider;
}
}
// 青果的分享域名(share.proxy.qg.net)与极客的旧域名(jikip.cn)不带上表主域名时按后缀兜底
if (lower.contains("qg.net")) {
return QINGGUO;
}
if (lower.contains("jikip.cn") || lower.contains("jikip.com")) {
return JIKIP;
}
return null;
}
}
@@ -18,8 +18,20 @@ public final class SecretUsageContext {
private SecretUsageContext() {
}
/** 计次归属:用户 + 密钥模块 key(如 similar-asin / appearance-patent)。 */
public record Context(Long userId, String moduleKey) {
/**
* 计次归属:用户 + 密钥模块 key(如 similar-asin / appearance-patent)+ LLM 渠道主机。
*
* <p>渠道主机跟着任务上下文一起下发,而不是逐层加函数参数:行级处理跑在线程池里,
* 一条链路上有十几层方法都只透传 apiKey,多带一个 host 会把每个签名都改一遍,
* 而 {@link #wrap} 已经把整份上下文带到工作线程了,顺路带 host 成本为零。
* 显式传 host 的调用点仍然优先(见各 LLM 客户端的 resolveHost)。</p>
*/
public record Context(Long userId, String moduleKey, String llmHost) {
public Context(Long userId, String moduleKey) {
this(userId, moduleKey, null);
}
public boolean usable() {
return userId != null && userId > 0 && moduleKey != null && !moduleKey.isBlank();
}
@@ -27,8 +39,13 @@ public final class SecretUsageContext {
/** 在指定归属下执行并返回结果,结束后恢复原有上下文(支持嵌套)。 */
public static <T> T call(Long userId, String moduleKey, Supplier<T> action) {
return call(userId, moduleKey, null, action);
}
/** 在指定归属 + LLM 渠道主机下执行并返回结果。 */
public static <T> T call(Long userId, String moduleKey, String llmHost, Supplier<T> action) {
Context previous = HOLDER.get();
Context next = new Context(userId, moduleKey);
Context next = new Context(userId, moduleKey, llmHost);
if (next.usable()) {
HOLDER.set(next);
}
@@ -39,6 +56,16 @@ public final class SecretUsageContext {
}
}
/** 当前 LLM 渠道主机;无上下文或未设置时返回 null(客户端据此回落模块全局配置)。 */
public static String currentLlmHost() {
Context context = HOLDER.get();
if (context == null) {
return null;
}
String host = context.llmHost();
return host == null || host.isBlank() ? null : host;
}
/** 当前归属;无上下文或归属不可用时返回 null。 */
public static Context current() {
Context context = HOLDER.get();
@@ -1,5 +1,6 @@
package com.nanri.aiimage.modules.usersecret.support;
import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.config.AppearancePatentProperties;
import com.nanri.aiimage.config.SimilarAsinProperties;
@@ -41,9 +42,69 @@ public enum UserSecretModule {
return required;
}
/** 检测目标主机:各 LLM 模块自己的 API 主机(检测模型由 aiimage.user-secret.check-model 统一指定)。 */
/** 是否代理模块(货源词表为代理供应商,而非 LLM 渠道)。 */
public boolean isProxy() {
return this == PROXY;
}
/** 新建配置的默认货源:LLM 模块=龙抬头,代理=青果(2026-10-29 用户拍板)。 */
public String defaultChannel() {
return isProxy() ? ProxyProvider.DEFAULT.key() : LlmChannel.DEFAULT.key();
}
/**
* 保存时的货源归一化:空值→默认值;未知值→明确报错(不接受脏值落库,
* 否则这条配置会静默走默认渠道,用户以为切过去了其实没有)。
*/
public String requireChannel(String raw) {
if (raw == null || raw.isBlank()) {
return defaultChannel();
}
if (isProxy()) {
ProxyProvider provider = ProxyProvider.parse(raw);
if (provider == null) {
throw new BusinessException("未知的代理供应商:" + raw.trim());
}
return provider.key();
}
LlmChannel channel = LlmChannel.parse(raw);
if (channel == null) {
throw new BusinessException("未知的 LLM 渠道:" + raw.trim());
}
return channel.key();
}
/**
* 读取/检测时的货源归一化(宽松):空值或历史脏值回落到默认值,绝不抛异常。
* 代理模块额外支持按提取链接域名识别——老版本客户端只上报链接、不上报供应商。
*/
public String channelOrDefault(String raw, String proxyUrl) {
if (isProxy()) {
ProxyProvider parsed = ProxyProvider.parse(raw);
if (parsed != null) {
return parsed.key();
}
ProxyProvider detected = ProxyProvider.detectFromUrl(proxyUrl);
return detected == null ? ProxyProvider.DEFAULT.key() : detected.key();
}
LlmChannel parsed = LlmChannel.parse(raw);
return parsed == null ? LlmChannel.DEFAULT.key() : parsed.key();
}
/**
* 渠道对应的检测目标主机:各 LLM 模块按该用户密钥所属渠道取主机;
* 渠道为空(迁移前遗留行)时回落到模块全局配置,保持与历史行为一致。
*/
public String resolveLlmHost(AppearancePatentProperties appearancePatentProperties,
SimilarAsinProperties similarAsinProperties) {
SimilarAsinProperties similarAsinProperties,
String channel) {
if (isProxy()) {
throw new IllegalStateException("代理模块没有 LLM 检测目标");
}
LlmChannel parsed = LlmChannel.parse(channel);
if (parsed != null) {
return parsed.host();
}
return switch (this) {
case APPEARANCE_PATENT -> appearancePatentProperties.getLlmHost();
case SIMILAR_ASIN -> similarAsinProperties.getLlmHost();
@@ -51,6 +112,12 @@ public enum UserSecretModule {
};
}
/** 检测目标主机(渠道为空时按模块全局配置,供兼容旧调用点)。 */
public String resolveLlmHost(AppearancePatentProperties appearancePatentProperties,
SimilarAsinProperties similarAsinProperties) {
return resolveLlmHost(appearancePatentProperties, similarAsinProperties, null);
}
/** 桌面端门禁模块:密钥包下发与完整性判定的唯一来源。 */
public static List<UserSecretModule> requiredModules() {
return Arrays.stream(values()).filter(UserSecretModule::required).toList();
@@ -10,7 +10,9 @@ import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.servlet.http.HttpServletRequest;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
@@ -69,6 +71,61 @@ public class WorkbenchClientPackageAdminController {
packageService.confirmPackage(objectKey, fileName, version)));
}
@PostMapping("/package/chunk/init")
@Operation(summary = "初始化客户端包分片上传", description = "大包走分片:返回 uploadId 与 objectKey;分片上传到 /api/files/chunk/{uploadId}/part/{index}")
public ApiResponse<Map<String, Object>> initChunkedPackage(
HttpServletRequest request,
@Parameter(description = "上传的 zip 文件名") @RequestParam(value = "file_name") String fileName,
@Parameter(description = "客户端版本号,如 1.0.0") @RequestParam("version") String version,
@Parameter(description = "总分片数") @RequestParam("total_parts") int totalParts,
@Parameter(description = "文件总字节数") @RequestParam("total_bytes") long totalBytes) {
AdminUserEntity operator = adminAuthSupport.requireAdminOrInternal(request);
log.info("[workbench-client] 管理端初始化分片上传 operator={} fileName={} version={} parts={}",
operator.getUsername(), fileName, version, totalParts);
return ApiResponse.success(packageService.initChunkedPackage(
fileName, version, totalParts, totalBytes));
}
@PostMapping("/package/chunk/complete")
@Operation(summary = "拼装客户端包分片并转存", description = "全部分片到齐后服务端拼装、上传 client 桶;随后照旧调用 /package/confirm 复算摘要并落库")
public ApiResponse<Map<String, Object>> completeChunkedPackage(
HttpServletRequest request,
@RequestParam("upload_id") String uploadId,
@Parameter(description = "分片初始化返回的对象 key") @RequestParam("object_key") String objectKey,
@Parameter(description = "上传的 zip 文件名") @RequestParam("file_name") String fileName,
@Parameter(description = "客户端版本号") @RequestParam("version") String version) {
AdminUserEntity operator = adminAuthSupport.requireAdminOrInternal(request);
log.info("[workbench-client] 管理端拼装分片上传 operator={} uploadId={} objectKey={}",
operator.getUsername(), uploadId, objectKey);
return ApiResponse.success(packageService.completeChunkedPackage(
uploadId, objectKey, fileName, version));
}
@DeleteMapping("/package/chunk/{uploadId}")
@Operation(summary = "放弃客户端包分片上传", description = "删除分片会话;confirm 失败时带 object_key 一并清理未登记对象")
public ApiResponse<Map<String, Object>> abortChunkedPackage(
HttpServletRequest request,
@PathVariable("uploadId") String uploadId,
@Parameter(description = "已转存但未登记的对象 key(可空)") @RequestParam(value = "object_key", required = false) String objectKey) {
AdminUserEntity operator = adminAuthSupport.requireAdminOrInternal(request);
log.info("[workbench-client] 管理端放弃分片上传 operator={} uploadId={} objectKey={}",
operator.getUsername(), uploadId, objectKey);
packageService.abortChunkedPackage(uploadId, objectKey);
return ApiResponse.success(Map.of("aborted", true));
}
@DeleteMapping("/package/object")
@Operation(summary = "清理已转存未登记的包对象", description = "confirm 失败的补偿出口;对象已登记时拒绝清理并报错,防误删线上更新目标")
public ApiResponse<Map<String, Object>> cleanupUnconfirmedObject(
HttpServletRequest request,
@Parameter(description = "要清理的对象 key") @RequestParam("object_key") String objectKey) {
AdminUserEntity operator = adminAuthSupport.requireAdminOrInternal(request);
log.info("[workbench-client] 管理端清理未登记包对象 operator={} objectKey={}",
operator.getUsername(), objectKey);
packageService.cleanupUnconfirmedObject(objectKey);
return ApiResponse.success(Map.of("removed", true));
}
@PostMapping("/package/delete")
@Operation(summary = "批量删除客户端包", description = "按 id 删除记录;关联 MinIO 包体仅当无其它记录引用时清除")
public ApiResponse<Map<String, Object>> deletePackages(
@@ -2,6 +2,8 @@ package com.nanri.aiimage.modules.workbenchclient.service;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.common.text.UserMessages;
import com.nanri.aiimage.modules.file.service.ChunkedUploadService;
import com.nanri.aiimage.modules.file.service.oss.OssStorageService;
import com.nanri.aiimage.modules.workbenchclient.mapper.WorkbenchClientPackageMapper;
import com.nanri.aiimage.modules.workbenchclient.model.entity.WorkbenchClientPackageEntity;
@@ -97,6 +99,7 @@ public class WorkbenchClientPackageService {
private final WorkbenchClientPackageMapper packageMapper;
private final OssStorageService ossStorageService;
private final ChunkedUploadService chunkedUploadService;
/** 管理端列表:按上传时间倒序,第一条即客户端当前更新目标。 */
public List<Map<String, Object>> listPackages() {
@@ -170,10 +173,7 @@ public class WorkbenchClientPackageService {
public Map<String, Object> presignPackage(String fileName, String version) {
String normalizedName = requireZipFileName(fileName);
String normalizedVersion = requireVersion(version);
String objectKey = OBJECT_KEY_PREFIX
+ OBJECT_NAME_FORMATTER.format(LocalDateTime.now())
+ "-" + UUID.randomUUID().toString().replace("-", "")
+ "-" + safeVersionKey(normalizedVersion) + ".zip";
String objectKey = newObjectKey(normalizedVersion);
int expirySeconds = effectivePresignExpirySeconds();
log.info("[workbench-client] 签发客户端包直传预签名 version={} fileName={} objectKey={} expirySeconds={}",
normalizedVersion, normalizedName, objectKey, expirySeconds);
@@ -187,6 +187,114 @@ public class WorkbenchClientPackageService {
return result;
}
/**
* 分片上传第一步(AC-17):与直传同一个对象 key 规则,key 写进分片会话的 purpose,
* 拼装时必须逐字匹配——否则管理员可借分片接口覆盖任意已发布包。
*/
public Map<String, Object> initChunkedPackage(String fileName, String version,
int totalParts, long totalBytes) {
String normalizedName = requireZipFileName(fileName);
String normalizedVersion = requireVersion(version);
if (totalBytes <= 0 || totalBytes > MAX_UPLOAD_BYTES) {
throw new BusinessException("文件超过允许的大小限制");
}
String objectKey = newObjectKey(normalizedVersion);
ChunkedUploadService.ChunkSession session =
chunkedUploadService.init(normalizedName, totalParts, totalBytes, objectKey);
log.info("[workbench-client] 初始化分片上传 version={} fileName={} objectKey={} parts={} bytes={}",
normalizedVersion, normalizedName, objectKey, totalParts, totalBytes);
Map<String, Object> result = new LinkedHashMap<>();
result.put("upload_id", session.uploadId());
result.put("object_key", objectKey);
result.put("file_name", normalizedName);
result.put("version", normalizedVersion);
result.put("total_parts", session.totalParts());
return result;
}
/**
* 分片拼装完成:拼好的 zip 由服务端上传到公开 client 桶,随后前端照旧调用
* /package/confirm —— 摘要复算、凭据扫描、版本唯一性全部沿用直传那条链。
*/
public Map<String, Object> completeChunkedPackage(String uploadId, String objectKey,
String fileName, String version) {
String normalizedName = requireZipFileName(fileName);
String normalizedVersion = requireVersion(version);
if (objectKey == null || !objectKey.startsWith(OBJECT_KEY_PREFIX)) {
throw new BusinessException("非法的对象路径,请重新上传");
}
if (!objectKey.endsWith("-" + safeVersionKey(normalizedVersion) + ".zip")) {
throw new BusinessException("版本号与上传对象不匹配,请重新上传");
}
if (!objectKey.equals(chunkedUploadService.purposeOf(uploadId))) {
throw new BusinessException("上传会话与对象不匹配,请重新上传");
}
ChunkedUploadService.CompletedUpload completed;
try {
completed = chunkedUploadService.complete(uploadId);
} catch (IOException ex) {
log.warn("[workbench-client] 分片拼装失败 uploadId={} objectKey={} reason={}",
uploadId, objectKey, ex.getMessage());
throw new BusinessException(UserMessages.with("拼装安装包失败", ex));
}
try {
ossStorageService.uploadWorkbenchClientObject(completed.file().toFile(), objectKey);
} finally {
completed.file().toFile().delete();
}
log.info("[workbench-client] 分片拼装并转存完成 version={} objectKey={} bytes={} sha256={}",
normalizedVersion, objectKey, completed.size(), completed.sha256());
Map<String, Object> result = new LinkedHashMap<>();
result.put("object_key", objectKey);
result.put("file_url", ossStorageService.getWorkbenchClientDownloadUrl(objectKey));
result.put("file_name", normalizedName);
result.put("version", normalizedVersion);
result.put("sha256", completed.sha256());
return result;
}
/**
* 清理已上传但未登记的包对象(confirm 失败后的补偿):数据库已有同 key 记录时拒绝,
* 避免管理员误删线上正在使用的安装包。
*/
public void cleanupUnconfirmedObject(String objectKey) {
if (objectKey == null || !objectKey.startsWith(OBJECT_KEY_PREFIX)) {
throw new BusinessException("非法的对象路径");
}
if (findByObjectKey(objectKey) != null) {
throw new BusinessException("该对象已登记,不能按未确认清理");
}
ossStorageService.removeWorkbenchClientObject(objectKey);
log.info("[workbench-client] 已清理未登记的包对象 objectKey={}", objectKey);
}
/**
* 放弃分片上传:删分片会话;若已拼装并转存过对象(confirm 失败场景),一并清理未登记对象。
* 会话不存在/对象已登记都不算失败——补偿清理要尽力而为,不能反过来挡住重传。
*/
public void abortChunkedPackage(String uploadId, String objectKey) {
try {
chunkedUploadService.abort(uploadId);
} catch (BusinessException ex) {
log.info("[workbench-client] 分片会话已不存在(可能已完成) uploadId={}", uploadId);
}
if (objectKey != null && objectKey.startsWith(OBJECT_KEY_PREFIX)) {
try {
cleanupUnconfirmedObject(objectKey);
} catch (BusinessException ex) {
log.warn("[workbench-client] 清理未登记对象失败 objectKey={} reason={}",
objectKey, ex.getMessage());
}
}
}
private String newObjectKey(String normalizedVersion) {
return OBJECT_KEY_PREFIX
+ OBJECT_NAME_FORMATTER.format(LocalDateTime.now())
+ "-" + UUID.randomUUID().toString().replace("-", "")
+ "-" + safeVersionKey(normalizedVersion) + ".zip";
}
/**
* 直传确认:校验对象存在、大小上限、版本号与对象名一致,服务端复算 sha256 后落库。
* <p>sha256 为空或长度不对直接拒绝——宁可让管理员重传,也不让客户端拿到没有锚点的包。
@@ -17,6 +17,12 @@ AIIMAGE_COST_GUARD_LLM_WINDOW_TOKENS_THRESHOLD=600000
AIIMAGE_COST_GUARD_PLATFORM_PROXY_DAILY_THRESHOLD=2000
AIIMAGE_COST_GUARD_SHARED_ACCOUNTS_THRESHOLD=3
# 采集执行方路由:专利/货源默认本地采集;关掉则只由 Linux 后台执行。
# 本地采集的任务 PENDING 超过该分钟数还没被客户端领走,才允许 Linux 兜底。
AIIMAGE_COLLECT_LOCAL_APPEARANCE_PATENT=true
AIIMAGE_COLLECT_LOCAL_SIMILAR_ASIN=true
AIIMAGE_COLLECT_LINUX_FALLBACK_MINUTES=30
AIIMAGE_DB_URL=jdbc:mysql://127.0.0.1:3306/aiimage?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=false
AIIMAGE_DB_USERNAME=change-me
AIIMAGE_DB_PASSWORD=change-me
@@ -87,6 +93,13 @@ AIIMAGE_NOTIFICATION_JIKIP_LOW_SURPLUS_THRESHOLD=100
# 账户余额 ≤ 阈值 → 管理员通知;供应商口径,0=关闭
AIIMAGE_NOTIFICATION_JIKIP_LOW_BALANCE_THRESHOLD=0
# 以上 LLM 主机/模型是「贞贞」渠道的默认值(历史口径)。
# 2026-10-29 起用户密钥可按人选择渠道(龙抬头 / 贞贞,默认贞贞,可手动切龙抬头):
# - 渠道主机写死在 UserSecretModule/LlmChannel 里,不读本段;
# - 手动切到龙抬头渠道后,模型 id 统一映射为 gemini-3.1-flash-lite,并显式补
# enable_thinking=true + thinking_budget=medium(贞贞把 thinking 写在模型名里;
# 不补这一对就等于给外观专利/货源查询降档。检测链路刻意不思考,只验连通);
# - 用户没配密钥或迁移前遗留行没有 channel 时,才会回落到本段配置。
AIIMAGE_APPEARANCE_PATENT_LLM_HOST=https://ai.t8star.org
AIIMAGE_APPEARANCE_PATENT_TITLE_MODEL=gemini-3.1-flash-lite-preview-thinking-medium
AIIMAGE_APPEARANCE_PATENT_APPEARANCE_MODEL=gemini-3.1-flash-lite-preview-thinking-medium
@@ -80,8 +80,10 @@ logging:
root: INFO
com.nanri.aiimage: INFO
pattern:
file: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
console: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
# AC-21:与 logback-spring.xml 的默认值同口径(本文件显式设了这两条,默认值会被盖掉),
# %X{requestId} 由 RequestTraceFilter 入 MDC。两处必须一起改,漏一处线上就看不见请求链。
file: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%X{requestId}] %logger{36} - %msg%n"
console: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%X{requestId}] %logger{36} - %msg%n"
mybatis-plus:
mapper-locations: classpath*:mapper/**/*.xml
@@ -212,6 +214,16 @@ aiimage:
source-retention-hours: ${AIIMAGE_STORAGE_SOURCE_RETENTION_HOURS:24}
result-retention-hours: ${AIIMAGE_STORAGE_RESULT_RETENTION_HOURS:24}
transient-payload-retention-hours: ${AIIMAGE_STORAGE_TRANSIENT_PAYLOAD_RETENTION_HOURS:72}
# 大文件分片上传通道(AC-17/S1-F3):浏览器分片 → 服务端拼装 → 复算 sha256 后按普通临时件落库。
# 这三键是闸门值而不是调优参数:放宽等于同时放大临时盘驻留与拼装阶段的耗时上限。
file:
chunk-upload:
# 单个会话允许的总字节(默认 2GiB):init 与逐片累计都要过这道闸,超限直接拒绝
max-total-bytes: ${AIIMAGE_FILE_CHUNK_UPLOAD_MAX_TOTAL_BYTES:2147483648}
# 允许的分片数上限(默认 4096):与前端 8MB/片相乘即体积的第二道闸
max-parts: ${AIIMAGE_FILE_CHUNK_UPLOAD_MAX_PARTS:4096}
# 未完成会话与残留片的驻留小时数:超时由清理动作回收,避免断掉的上传永久占盘
stale-hours: ${AIIMAGE_FILE_CHUNK_UPLOAD_STALE_HOURS:24}
temp-dir:
retention-hours: ${AIIMAGE_TEMP_DIR_RETENTION_HOURS:24}
capacity-plan:
@@ -360,8 +372,19 @@ aiimage:
heartbeat-interval-ms: ${AIIMAGE_RESULT_FILE_JOB_HEARTBEAT_INTERVAL_MS:60000}
batch-size: ${AIIMAGE_RESULT_FILE_JOB_BATCH_SIZE:20}
max-inflight-dispatch: ${AIIMAGE_RESULT_FILE_JOB_MAX_INFLIGHT_DISPATCH:64}
# 采集执行方路由(2026-10-29):外观专利与相似ASIN默认「本地采集」,页面推本机客户端;
# 推了超过 linux-fallback-minutes 还没被客户端领走,才允许 Linux 后台兜底认领
# (PENDING 已不再按时间判死,没有兜底窗口的话本地机全关时任务会一直排队)。
# 关掉某模块=该模块只由 Linux 后台执行,页面不再推送。未登记的模块按「非本地」处理,
# 保持内部认领原有的 internal-min-pending-minutes,不被卷进本地优先。
collect-routing:
local-enabled:
APPEARANCE_PATENT: ${AIIMAGE_COLLECT_LOCAL_APPEARANCE_PATENT:true}
SIMILAR_ASIN: ${AIIMAGE_COLLECT_LOCAL_SIMILAR_ASIN:true}
linux-fallback-minutes: ${AIIMAGE_COLLECT_LINUX_FALLBACK_MINUTES:30}
# 客户端兜底拉取:页面没推送成功(只解析没点启动 / 推送前关页面 / 纯浏览器打开)、
# 长期停在 PENDING 的任务由在线客户端领走执行,替代 2 小时后的"任务长期未被领取"标失败。
# 长期停在 PENDING 的任务由在线客户端领走执行。2026-10-29 起 PENDING 不再按 2 小时判死
# (排队中的任务一直排队等认领),本开关是排队任务的唯一出路,不再是"判死前的抢救"。
# 默认关闭;打开前需先发布带兜底拉取线程的客户端版本,旧客户端不受影响。
client-task-pull:
enabled: ${AIIMAGE_CLIENT_TASK_PULL_ENABLED:false}
@@ -0,0 +1,36 @@
-- V180: 用户密钥增加「货源」维度(LLM 渠道 / 代理供应商)
--
-- 背景(2026-10-29 用户提出):
-- 1) LLM 密钥此前只有一把,请求地址由全局配置 aiimage.*.llm-host 定死(现网 ai.t8star.org/贞贞)。
-- 现在多了一家中转「龙抬头」(https://longtaitou.token6688.com),两家的 sk- 密钥互不通用,
-- 所以「密钥属于哪一家」必须跟着密钥一起存,否则切换渠道就得改全局配置、影响所有人。
-- 2) 代理此前只认极客(jikip) 的提取链接与余额接口。新增青果(overseas.proxy.qg.net),
-- 两家返回体结构不同(青果 code 是字符串 SUCCESS/...,data 是数组、地址在 server 字段),
-- 解析与检测都要按供应商分支,因此供应商也要跟着代理配置存。
--
-- 列语义(同一列按 module_key 取不同词表,刻意不拆两列——两者都是「这一行配置的货源」,
-- 拆开会出现「LLM 行有 provider/代理行有 channel」这种半空列):
-- module_key IN ('appearance-patent','similar-asin') → LLM 渠道:longtaitou / t8star
-- module_key = 'proxy' → 代理供应商:qingguo / jikip
--
-- 存量回填(关键,不能省):
-- 新建默认是 t8star/jikip(2026-10-30 用户拍板:默认仍走贞贞与极客,龙抬头/青果手动切),
-- 存量行也一律回填成 t8star/jikip —— 与加列默认值一致,但仍显式写一遍:
-- 即使将来默认值再变,这条迁移落库的历史数据也永远是"当时的真实货源"。
--
-- 影响表与数据量:biz_user_api_secret 增加一列,行数=用户数×3,量级极小。
--
-- 回滚(手工):
-- ALTER TABLE `biz_user_api_secret` DROP COLUMN `channel`;
ALTER TABLE `biz_user_api_secret`
ADD COLUMN `channel` VARCHAR(32) NOT NULL DEFAULT 't8star'
COMMENT '货源:LLM 行为渠道(longtaitou/t8star);代理行为供应商(qingguo/jikip)' AFTER `module_key`;
UPDATE `biz_user_api_secret`
SET `channel` = 't8star'
WHERE `module_key` IN ('appearance-patent', 'similar-asin');
UPDATE `biz_user_api_secret`
SET `channel` = 'jikip'
WHERE `module_key` = 'proxy';
@@ -4,8 +4,11 @@
<springProperty scope="context" name="LOG_NAME" source="logging.file.name" defaultValue="aiimage-backend.log"/>
<springProperty scope="context" name="ROOT_LOG_LEVEL" source="logging.level.root" defaultValue="INFO"/>
<springProperty scope="context" name="APP_LOG_LEVEL" source="logging.level.com.nanri.aiimage" defaultValue="INFO"/>
<springProperty scope="context" name="FILE_PATTERN" source="logging.pattern.file" defaultValue="%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"/>
<springProperty scope="context" name="CONSOLE_PATTERN" source="logging.pattern.console" defaultValue="%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"/>
<!-- AC-21(审计 §5.2):%X{requestId} 由 RequestTraceFilter 入 MDC。放在 %-5level 之后、
%logger 之前,既保住原有列宽,又能用一行 requestId 把整条请求链的日志串起来。
⚠ 这里是"配置没给时的默认值",application.yml 显式设了 logging.pattern.* 会盖掉它 ⇒ 两边同改。 -->
<springProperty scope="context" name="FILE_PATTERN" source="logging.pattern.file" defaultValue="%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%X{requestId}] %logger{36} - %msg%n"/>
<springProperty scope="context" name="CONSOLE_PATTERN" source="logging.pattern.console" defaultValue="%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%X{requestId}] %logger{36} - %msg%n"/>
<property name="APP_LOG_FILE" value="${LOG_PATH}/${LOG_NAME}"/>
<property name="APP_LOG_ARCHIVE" value="${LOG_PATH}/${LOG_NAME}.%d{yyyy-MM-dd}.%i"/>
@@ -45,13 +45,17 @@
LIMIT #{f.offset}, #{f.limit}
</select>
<!-- AC-19(审计 P1-B2):随机抽词不再 ORDER BY RAND()——那一句给命中集每一行打一个随机数再
filesort 全表,词库越大越慢。改成「按主键升序取一段窗口」:窗口起点由服务层先 count 命中集、
再掷在 [0, total-limit] 上,LIMIT 之前不必排序。服务层拿到窗口后就地洗牌,
页面看到的仍是散着的词。 -->
<select id="selectRandomUnused" resultType="com.nanri.aiimage.modules.keywordshare.model.vo.KeywordShareRowVo">
SELECT k.id, k.country, k.keyword, k.chinese, k.category, k.sub_category, k.category_l3, k.category_l4,
(u.country IS NOT NULL) AS used, u.used_time, u.user_name AS used_user
FROM <include refid="usageJoin"/>
WHERE <include refid="filterConds"/>
ORDER BY RAND()
LIMIT #{f.limit}
ORDER BY k.id
LIMIT #{f.limit} OFFSET #{f.offset}
</select>
<select id="countDupKeywords" resultType="long">
@@ -54,8 +54,28 @@
<select id="selectTaskCountries" resultType="com.nanri.aiimage.modules.taskcenter.model.dto.TaskCenterCountryRow">
SELECT t.id AS taskId, COALESCE(NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.filters.countryCode')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.countryCode')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.country')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.publish_country')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.countryCodes')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.country_codes')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.filters.countryCodes')), ''), 'null')) AS country FROM biz_file_task t WHERE t.id IN <foreach collection='taskIds' item='tid' open='(' separator=',' close=')'>#{tid}</foreach>
</select>
<!-- AC-19(审计 P1-B1):页查询不再吃 OFFSET。深翻页的 skip 挪进 selectTaskPageAnchor,
它只投影 (id,createdAt) 两列;页查询本身按 (createdAt,id) 游标取 limit 行,
MySQL 8.0.22+ 能把游标条件压进 UNION 分支、用上 created_at 索引,OFFSET 压不进去,
只能把整个 UNION 物化再跳过。
下面的筛选片段是宽行 UNION 与窄锚点的唯一公共出处:两份投影的筛选必须逐字相同,
否则锚点算的是另一批行,第 N 页会安静地给错。共用片段让「漂」在结构上不可能发生。 -->
<sql id="taskFilterT">t.module_type IN ('PUBLISH','DELETE_BRAND','PRODUCT_RISK_RESOLVE','PRICE_TRACK','SHOP_MATCH','PATROL_DELETE','QUERY_ASIN','WITHDRAW','APPEARANCE_PATENT','SIMILAR_ASIN','COLLECT_DATA','SHOP_DATA_CRAWL')<if test='userIds != null'> AND t.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND UPPER(t.module_type) = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND t.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND t.created_at &lt;= #{endTime}</if><if test='shop != null'> AND (EXISTS (SELECT 1 FROM biz_publish_file p WHERE p.task_id = t.id AND LOWER(TRIM(p.shop_name)) = LOWER(TRIM(#{shop}))) OR (t.module_type IN ('PATROL_DELETE','PRICE_TRACK','PRODUCT_RISK_RESOLVE','QUERY_ASIN','SHOP_DATA_CRAWL','SHOP_MATCH') AND EXISTS (SELECT 1 FROM biz_file_result f WHERE f.task_id = t.id AND LOWER(TRIM(f.source_filename)) = LOWER(TRIM(#{shop})))))</if></sql>
<sql id="taskFilterB">1 = 1<if test='userIds != null'> AND b.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND 'BRAND' = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND b.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND b.created_at &lt;= #{endTime}</if><if test='shop != null'> AND 1 = 0</if></sql>
<sql id="taskUnionRows">
(SELECT t.id AS id, t.task_no AS taskNo, UPPER(t.module_type) AS moduleType, CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END AS status, t.user_id AS userId, t.device_id AS deviceId, t.created_at AS createdAt, t.finished_at AS finishedAt, t.updated_at AS updatedAt, t.source_file_count AS totalCount, t.success_file_count AS successCount, t.failed_file_count AS failedCount, 0 AS brand FROM biz_file_task t WHERE <include refid="taskFilterT"/> UNION ALL SELECT b.id, CONCAT('brand-', b.id), 'BRAND', CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END, b.user_id, b.device_id, b.created_at, CASE WHEN b.status IN ('cancelled','failed','success') THEN b.updated_at END, b.updated_at, b.progress_total, b.progress_current, 0, 1 FROM brand_crawl_tasks b WHERE <include refid="taskFilterB"/>)
</sql>
<sql id="taskUnionKeys">
(SELECT t.id AS id, t.created_at AS createdAt FROM biz_file_task t WHERE <include refid="taskFilterT"/> UNION ALL SELECT b.id, b.created_at FROM brand_crawl_tasks b WHERE <include refid="taskFilterB"/>)
</sql>
<select id="selectTaskPageAnchor" resultType="com.nanri.aiimage.modules.taskcenter.model.dto.TaskCenterTaskRow">
SELECT m.createdAt, m.id FROM <include refid="taskUnionKeys"/> m
ORDER BY m.createdAt DESC, m.id DESC LIMIT 1 OFFSET #{offset}
</select>
<select id="selectTaskPage" resultType="com.nanri.aiimage.modules.taskcenter.model.dto.TaskCenterTaskRow">
SELECT m.* FROM (SELECT t.id AS id, t.task_no AS taskNo, UPPER(t.module_type) AS moduleType, CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END AS status, t.user_id AS userId, t.device_id AS deviceId, t.created_at AS createdAt, t.finished_at AS finishedAt, t.updated_at AS updatedAt, t.source_file_count AS totalCount, t.success_file_count AS successCount, t.failed_file_count AS failedCount, 0 AS brand FROM biz_file_task t WHERE t.module_type IN ('PUBLISH','DELETE_BRAND','PRODUCT_RISK_RESOLVE','PRICE_TRACK','SHOP_MATCH','PATROL_DELETE','QUERY_ASIN','WITHDRAW','APPEARANCE_PATENT','SIMILAR_ASIN','COLLECT_DATA','SHOP_DATA_CRAWL')<if test='userIds != null'> AND t.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND UPPER(t.module_type) = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND t.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND t.created_at &lt;= #{endTime}</if><if test='shop != null'> AND (EXISTS (SELECT 1 FROM biz_publish_file p WHERE p.task_id = t.id AND LOWER(TRIM(p.shop_name)) = LOWER(TRIM(#{shop}))) OR (t.module_type IN ('PATROL_DELETE','PRICE_TRACK','PRODUCT_RISK_RESOLVE','QUERY_ASIN','SHOP_DATA_CRAWL','SHOP_MATCH') AND EXISTS (SELECT 1 FROM biz_file_result f WHERE f.task_id = t.id AND LOWER(TRIM(f.source_filename)) = LOWER(TRIM(#{shop})))))</if> UNION ALL SELECT b.id, CONCAT('brand-', b.id), 'BRAND', CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END, b.user_id, b.device_id, b.created_at, CASE WHEN b.status IN ('cancelled','failed','success') THEN b.updated_at END, b.updated_at, b.progress_total, b.progress_current, 0, 1 FROM brand_crawl_tasks b WHERE 1 = 1<if test='userIds != null'> AND b.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND 'BRAND' = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND b.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND b.created_at &lt;= #{endTime}</if><if test='shop != null'> AND 1 = 0</if>) m ORDER BY m.createdAt DESC, m.id DESC LIMIT #{limit} OFFSET #{offset}
SELECT m.* FROM <include refid="taskUnionRows"/> m
<if test='afterCreatedAt != null'> WHERE (m.createdAt &lt; #{afterCreatedAt} OR (m.createdAt = #{afterCreatedAt} AND m.id &lt; #{afterId}))</if>
ORDER BY m.createdAt DESC, m.id DESC LIMIT #{limit}
</select>
<select id="selectUserIdsByUsername" resultType="long">
SELECT id FROM users WHERE username LIKE CONCAT('%', #{keyword}, '%') LIMIT 500
@@ -52,8 +52,12 @@ import static org.junit.jupiter.api.Assertions.fail;
class MigratedSqlEquivalenceTest {
private static final String FIXTURE = "sql-layering/migrated-statements.tsv";
private static final int LEDGER_ROWS = 85;
private static final int LEDGER_SELECTS = 60;
// 85 → 84:Phase 5(AC-19/P1-B1)有意把 {@code TaskCenterQueryMapper#selectTaskPage} 从
// OFFSET 改成 keyset 游标。账本记的是「注解时代原文」,改一行基准就等于让等价性测试照着
// 新 SQL 自证 ⇒ 把该行整条摘出账本、点名进下方 NOT_MIGRATED,让基准继续只描述真正未改动的迁移件。
private static final int LEDGER_ROWS = 84;
// 60 → 59:摘出去那行是一条 select。
private static final int LEDGER_SELECTS = 59;
/** 全部 mapper XML 里的 select 总数:账本 60 + TaskImageCache 5 + AsinResultCache 1。 */
/** 66 → 67:会话容忍新增的 {@code AdminUserMapper#countLiveWorkFor} 也是一条 select。 */
// 67 → 78:keywordshare 模块新写 11 条 select,逐条登记进下方 NOT_MIGRATED。
@@ -67,7 +71,10 @@ class MigratedSqlEquivalenceTest {
// (SqlLayeringConventionTest.XML_STATEMENT_TOTAL 同轮 130 → 140)。
// 99 → 99:客户端全量同步新写 1 条 insert(KeywordShareMapper#upsertBatchFromSync),
// 点名进 NOT_MIGRATED 但不是 select ⇒ select 总数不动(SqlLayeringConventionTest 同轮 140 → 141)。
private static final int XML_SELECT_TOTAL = 99;
// 99 → 100:Phase 5(AC-19)为任务中心深翻页新增窄锚点语句 {@code TaskCenterQueryMapper#selectTaskPageAnchor}
// (两列投影算页首游标),同轮点名进 NOT_MIGRATED;同轮的 selectTaskPage 改写让它从账本搬进
// NOT_MIGRATED(select 总数不变,只是记账位置换了)。
private static final int XML_SELECT_TOTAL = 100;
/**
* 不是「从注解搬过来」的语句:注解时代没有原文可对撞,等价性这支只能跳过它们。
@@ -121,7 +128,14 @@ class MigratedSqlEquivalenceTest {
"KeywordUsageAdminMapper#selectDeleteRows",
"KeywordUsageAdminMapper#selectStats",
"KeywordUsageAdminMapper#selectUsers",
"KeywordUsageAdminMapper#selectCountries");
"KeywordUsageAdminMapper#selectCountries",
// Phase 5(AC-19/P1-B1)taskcenter 两条:
// selectTaskPage —— 注解时代原文带 {@code LIMIT ? OFFSET ?},本轮有意改成
// (createdAt,id) 游标。基准是搬迁前的快照,不该被改写来替新 SQL 背书,
// 所以整行从 migrated-statements.tsv 摘掉、点名到这里(LEDGER_ROWS 85→84)。
// selectTaskPageAnchor —— 本轮新写的两列窄投影,从来没有注解原文可对撞。
"TaskCenterQueryMapper#selectTaskPage",
"TaskCenterQueryMapper#selectTaskPageAnchor");
/** 账本 5 列:接口 / 方法 / @Select 种类 / 是否含常量插值 / 注解原文(Java 反转义后那条串)。 */
private static final int LEDGER_COLUMNS = 5;
@@ -142,7 +156,7 @@ class MigratedSqlEquivalenceTest {
List<String[]> rows = readLedger();
assertEquals(LEDGER_ROWS, rows.size(), "fixture 行数变了——它是搬迁前的快照,不该跟着代码漂");
assertEquals(LEDGER_SELECTS, rows.stream().filter(r -> "Select".equals(r[2])).count(),
"fixture 里的 Select 条数与基线不符(读侧账本 = 60)");
"fixture 里的 Select 条数与基线不符(读侧账本 = 59)");
assertEquals(LEDGER_ROWS - LEDGER_SELECTS,
rows.stream().filter(r -> !"Select".equals(r[2])).count(),
"写侧条数对不上(25 = Insert+Update+Delete)");
@@ -88,7 +88,10 @@ class SqlLayeringConventionTest {
// 141 → 143:V179 采集「被过滤行」新表 {@code CollectDataFilteredRowMapper} 的两条写侧语句
// (insertBatch 多值插入 + markResolved 盖回流时间)。该 mapper 没有注解时代,
// 不在 MigratedSqlEquivalenceTest 的基准账本里 ⇒ 不占 NOT_MIGRATED,只在这本总分母上记名。
private static final int XML_STATEMENT_TOTAL = 143;
// 143 → 144:Phase 5(AC-19/P1-B1)为任务中心深翻页新增窄锚点语句
// {@code TaskCenterQueryMapper#selectTaskPageAnchor}(两列投影算页首游标),
// 同轮登记进 MigratedSqlEquivalenceTest.NOT_MIGRATED(该文件的 XML_SELECT_TOTAL 同轮 99 → 100)。
private static final int XML_STATEMENT_TOTAL = 144;
/**
* wrapper 里夹带函数级 SQL 的现状("这不是简单单表"的证据),文件名 → 条数:
@@ -93,7 +93,9 @@ class RequestSuppliedIdentityOutletTest {
Scan scan = scanAll();
// 28 支进度轮询走 require,其余写类/读类走 resolve;两边都算"已接出口"。
assertThat(scan.boundWrite()).as("取请求身份且已接出口的写类 handler 数").isEqualTo(126);
assertThat(scan.boundRead()).as("取请求身份且已接出口的读类 handler 数").isEqualTo(63);
// 64 而不是 63:2026-10-29 随本地采集开关恢复 GET /tasks/{taskId}/queue-payload,
// 该 handler 同时接上 resolveOwnershipUserId(不是放宽分母)。
assertThat(scan.boundRead()).as("取请求身份且已接出口的读类 handler 数").isEqualTo(64);
assertThat(scan.unbound()).as("取请求身份但未接出口的 handler:只剩豁免那一条")
.containsExactlyElementsOf(ALLOWED_WITHOUT_OUTLET);
}
@@ -64,7 +64,8 @@ class AdminGuardSwitchConfigTest {
private static final String ANCHOR = AdminApiGuardFilter.SWITCH_ANCHOR;
private static final int USER_TOOL_PREFIX_COUNT = 15;
// 15 → 16:采集路由开关读取口 /api/collect-routing 进用户态名单(前端登录后读取,无匿名调用方)。
private static final int USER_TOOL_PREFIX_COUNT = 16;
// 6 → 7:损耗监控 /api/user/cost-guard 进无条件名单(uid 只从 JWT 取,匿名不该能解除暂停闸)。
private static final int SELF_SERVICE_PREFIX_COUNT = 7;
@@ -69,8 +69,9 @@ class ApiGuardCoverageAuditorTest {
// 只钉"contains 某条"的话,从数组里删掉一整段都可能不红,所以钉总数。
// 2026-10-24 自助 6 → 7:cost-guard 的 status/resume 两个端点 uid 只从 JWT 取,
// 匿名既读不到状态也不该能解除暂停闸,所以进无条件名单而不是登记成「方法内自校验」。
assertEquals(23, count(guarded, "/api/"), () -> "以 /api/ 开头的守卫前缀数变了(基线 23 = admin 1 + 用户态 15 + 自助 7):" + guarded);
assertEquals(24, guarded.size(), "守卫前缀总数变了(23 条 /api/ + /debug):" + guarded);
// 2026-10-29 用户态 15 → 16:采集路由读取口 /api/collect-routing 新建即入名单。
assertEquals(24, count(guarded, "/api/"), () -> "以 /api/ 开头的守卫前缀数变了(基线 24 = admin 1 + 用户态 16 + 自助 7):" + guarded);
assertEquals(25, guarded.size(), "守卫前缀总数变了(24 条 /api/ + /debug):" + guarded);
assertTrue(guarded.contains("/api/publish"), "2026-10-07 补的上架前缀不能退回名单外");
assertTrue(guarded.contains("/api/tasks"), "心跳/打断前缀不能退回名单外");
assertTrue(guarded.contains("/api/internal"), "内部端点前缀不能退回名单外");
@@ -57,7 +57,9 @@ class CatchSilenceRatchetTest {
// 10-28 V179 采集被过滤行追溯抬 579 -> 582(审计件三枚:逐批插入容错/回流簿记容错/尾 sheet 降级,
// 逐处记名见 STRICT_TOTAL 上方注释)。
// 10-28 Phase 3 抬 582 -> 585(心跳降级一枚 + 优雅停机两枚中断恢复,逐处记名见 STRICT_TOTAL 上方注释)。
static final int GIVEN_NOT_RETHROW = 585;
// 10-29 Phase 4D 抬 585 -> 591(AC-17 分片通道六枚 LOG_ONLY:清理/收尾失败不得盖掉主错误,
// 逐处记名见 STRICT_TOTAL 上方注释;EMPTY 一格没涨)。
static final int GIVEN_NOT_RETHROW = 591;
static final int GIVEN_LOG_ONLY = 249;
static final int GIVEN_EMPTY = 56;
@@ -211,8 +213,29 @@ class CatchSilenceRatchetTest {
* <li>{@code TaskFileJobConfig.videoQueueExecutor} 一枚:信号量 acquire 被中断只恢复中断标记
* (S1-B2 视频独立许可,照抄 taskQueueExecutor 的既有形状)。</li>
* </ul>
* <p>10-29 Phase 4D(AC-17/S1-F3 大文件分片通道)抬 total 1034 -&gt; 1045,11 枚逐处记名。
* 这一批是"新增通道"而不是"给既有出口补兜":整条 {@code ChunkedUploadService} 与
* {@code WorkbenchClientPackageService} 的分片方法都是新落点,Java 的受检 {@code IOException}
* 使每一段磁盘动作都必须有自己的出口,形状与既有件同口径:
* <ul>
* <li>RETHROW 五枚——{@code ChunkedUploadService.init}(建会话目录失败:先 {@code deleteQuietly}
* 再抛 {@code BusinessException},不留半个会话)、{@code complete}(拼装中途失败:删掉半成品
* 后原样 {@code throw ex},绝不产出半个文件)、{@code readMeta}(元数据损坏直接抛,
* 让调用方重新 init 而不是按坏元数据继续拼)、{@code sha256Digest}(JVM 无 SHA-256 属环境级
* 不可能,抛 {@code IllegalStateException} 带 cause)、{@code WorkbenchClientPackageService
* .completeChunkedPackage}(拼装 IOException 翻成业务文案,500 变可读错误)。</li>
* <li>LOG_ONLY 六枚,块体各只有一句带锚点的 log,误差方向都是"清理/收尾失败不得盖掉主错误"——
* {@code ChunkedUploadService.cleanupStaleSessions} 内层(单条目时间戳读不出,本轮跳过该条)
* 与外层({@code Files.list} 失败,整轮清理放弃但不拦新会话)、
* {@code deleteQuietly} 的逐片删除与整体遍历两枚(残留交给过期回收),
* {@code WorkbenchClientPackageService.abortChunkedPackage} 的两枚(会话已不存在只 INFO;
* 未登记对象清理失败 WARN 带 objectKey,登记成功的对象后端会拒绝清理,不该在这里改口)。</li>
* <li>EMPTY 一格没涨:三处原本写成 {@code catch (IOException ignored) { /* 尽力清理 *\/ }} 的
* 空体已按 10-26 那批的先例改成 WARN 带 path——空体是三类高危出口里最贵的一种,
* 这里宁可让 {@code STRICT_LOG_ONLY} 涨六格,也不给"删除失败连日志都不留"开门。</li>
* </ul>
*/
static final int STRICT_TOTAL = 1034;
static final int STRICT_TOTAL = 1045;
/**
* 重抛型 catch 的**下限**(不是上限):本轮把两处吞异常改成重抛之后实测 388。
* 这一格只能涨不能跌——它是"异常会往外走"的总量,把它钉成"只许减"就等于拦着别人修吞异常。
@@ -322,8 +345,13 @@ class CatchSilenceRatchetTest {
* 块体 log.warn 带 taskId/moduleType)、{@code TaskFileJobConfig.taskQueueVirtualThreadExecutor}
* 的 {@code shutdown()} 与 {@code TaskFileJobConfig.videoQueueExecutor}(S1-B6/S1-B2,
* {@code InterruptedException} 只恢复中断标记),逐处记名见 STRICT_TOTAL 上方注释。
* 10-29 Phase 4D 再抬 585 -&gt; 591(AC-17 分片通道六枚 LOG_ONLY,逐处记名见 STRICT_TOTAL 上方注释):
* {@code ChunkedUploadService.cleanupStaleSessions} 内外两枚、{@code deleteQuietly} 逐片与整体两枚、
* {@code WorkbenchClientPackageService.abortChunkedPackage} 的「会话已不存在」INFO 与
* 「未登记对象清理失败」WARN 两枚。六枚的误差方向都是「清理/收尾失败不得盖掉主错误」,
* 主错误由外层重抛;本批 EMPTY 一格没涨(原先三枚空体已按 10-26 先例改成带 path 的 WARN)。
*/
static final int STRICT_NOT_RETHROW = 585;
static final int STRICT_NOT_RETHROW = 591;
/**
* 10-17 抬 172 -> 173,出处就是上面点名过的 {@code TaskHeartbeatLivenessService.stamp()}:
* 块体只有「取到 key 名 + log.warn」一句,写心跳证据失败时既不重试也不抛出——
@@ -347,8 +375,13 @@ class CatchSilenceRatchetTest {
* 块体只有一句带 {@code [collect-data]} 锚点的 WARN(taskId+chunk+err),坏批点名后其余批照常
* 落库。同批另两处({@code markResolved} 追加 return、{@code writeFilteredSheet} 置 incomplete 位)
* 因块内多一句只进 NOT_RETHROW。
* 10-29 Phase 4D 抬 182 -> 188(AC-17 分片通道六枚,逐处记名见 STRICT_TOTAL 上方注释):
* {@code ChunkedUploadService.cleanupStaleSessions} 内外两枚、{@code deleteQuietly} 逐片与整体两枚
* (这四枚原本是 {@code catch (IOException ignored)} 空体,按 10-26 那批先例改成带 path 的 WARN,
* 是「空体换 LOG_ONLY」的迁移而不是新增沉默),加上
* {@code WorkbenchClientPackageService.abortChunkedPackage} 的 INFO 与 WARN 两枚。
*/
static final int STRICT_LOG_ONLY = 182; // 严格口径:块内每条顶层语句都是 log 调用才算
static final int STRICT_LOG_ONLY = 188; // 严格口径:块内每条顶层语句都是 log 调用才算
static final int STRICT_EMPTY = 56;
/**
@@ -0,0 +1,49 @@
package com.nanri.aiimage.config;
import org.junit.jupiter.api.Test;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* 采集路由开关的判定口径(2026-10-29):
* 只对 local-enabled 里点名的模块生效,未登记模块一律「非本地」——新模块不能被默认卷进本地优先;
* 兜底窗口下限 1 分钟,配 0/负数不许静默变成「Linux 立刻抢」。
*/
class CollectRoutingPropertiesTest {
@Test
void 点名的模块按配置返回且大小写不敏感() {
CollectRoutingProperties properties = new CollectRoutingProperties();
properties.setLocalEnabled(Map.of("APPEARANCE_PATENT", true, "SIMILAR_ASIN", false));
assertTrue(properties.isLocal("APPEARANCE_PATENT"));
assertTrue(properties.isLocal("appearance_patent"), "模块名大小写不应影响判定");
assertFalse(properties.isLocal("SIMILAR_ASIN"));
}
@Test
void 未登记的模块按非本地处理() {
CollectRoutingProperties properties = new CollectRoutingProperties();
properties.setLocalEnabled(Map.of("SIMILAR_ASIN", true));
assertFalse(properties.isLocal("COLLECT_DATA"), "没配置的模块必须保持 Linux 原有等待窗口");
assertFalse(properties.isLocal(null));
assertFalse(properties.isLocal(" "));
}
@Test
void 兜底窗口默认三十分钟且下限一分钟() {
CollectRoutingProperties properties = new CollectRoutingProperties();
assertEquals(30L, properties.safeLinuxFallbackMinutes());
properties.setLinuxFallbackMinutes(0L);
assertEquals(1L, properties.safeLinuxFallbackMinutes(), "0 不许退化成 Linux 立刻接管");
properties.setLinuxFallbackMinutes(45L);
assertEquals(45L, properties.safeLinuxFallbackMinutes());
}
}
@@ -1,7 +1,6 @@
package com.nanri.aiimage.config;
import com.nanri.aiimage.testutil.SourceScan;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
@@ -34,7 +33,6 @@ import static org.assertj.core.api.Assertions.assertThat;
* {@code app_client/tests/test_pending_phase1_日志脱敏与轮转.py}(AC-1,含大小轮转)与
* {@code scripts/tests/test_pending_phase5_worker日志轮转与保留.py}(AC-21 的 worker 侧)。
*/
@Disabled("Pending Phase 5: AC-21 补检 WARN 限流/汇总与 MDC 关联")
@DisplayName("AC-21 日志量级与请求关联契约")
class PendingPhase5LogVolumeAndMdcTest {
@@ -63,13 +61,22 @@ class PendingPhase5LogVolumeAndMdcTest {
@Test
void 日志_pattern_必须带出_MDC_位() throws Exception {
Path moduleRoot = com.nanri.aiimage.testutil.SourceScan.mainSourcesRoot()
.resolve("../../src/main/resources/logback-spring.xml")
// 预置时这条路径写错了层级(mainSourcesRoot() 已经是 src/main/java,../../ 只退到 src/),
// Phase 5 激活时按真实落点修正——判据本身一字未减。
Path logbackXml = com.nanri.aiimage.testutil.SourceScan.mainSourcesRoot()
.resolve("../../../src/main/resources/logback-spring.xml")
.normalize();
String xml = Files.readString(moduleRoot);
String xml = Files.readString(logbackXml);
assertThat(xml)
.as("pattern 里没有 %X{...} 时,MDC 写了也看不见(FILE/CONSOLE 两条都要带)")
.contains("%X{requestId}");
// application.yml 显式设了 logging.pattern.*,会把 logback 里的 defaultValue 整条盖掉 ⇒
// 只改 logback 等于改了一份线上读不到的默认值。两处必须同时带出 MDC 位。
String application = Files.readString(logbackXml.resolveSibling("application.yml"));
assertThat(application)
.as("application.yml 的 pattern 覆盖了 logback 默认值,漏改这一处线上照样看不见 requestId")
.contains("%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%X{requestId}] %logger{36} - %msg%n");
}
}
@@ -0,0 +1,147 @@
package com.nanri.aiimage.modules.appearancepatent.service;
import com.baomidou.mybatisplus.core.MybatisConfiguration;
import com.baomidou.mybatisplus.core.conditions.Wrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.baomidou.mybatisplus.core.metadata.TableInfoHelper;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.common.service.DistributedJobLockService;
import com.nanri.aiimage.common.service.ResultDownloadResolver;
import com.nanri.aiimage.config.AppearancePatentProperties;
import com.nanri.aiimage.config.CollectRoutingProperties;
import com.nanri.aiimage.config.InstanceMetadata;
import com.nanri.aiimage.config.StorageProperties;
import com.nanri.aiimage.modules.appearancepatent.client.AppearancePatentLlmClient;
import com.nanri.aiimage.modules.task.mapper.FileResultMapper;
import com.nanri.aiimage.modules.task.mapper.FileTaskMapper;
import com.nanri.aiimage.modules.task.mapper.TaskChunkMapper;
import com.nanri.aiimage.modules.task.mapper.TaskScopeStateMapper;
import com.nanri.aiimage.modules.task.model.entity.FileTaskEntity;
import com.nanri.aiimage.modules.task.service.TaskDistributedLockService;
import com.nanri.aiimage.modules.task.service.TaskFileJobService;
import com.nanri.aiimage.modules.task.service.TaskHeartbeatPositionService;
import com.nanri.aiimage.modules.task.service.TaskProgressLightAssembler;
import com.nanri.aiimage.modules.task.service.TaskProgressSnapshotService;
import com.nanri.aiimage.modules.task.service.TransientPayloadStorageService;
import com.nanri.aiimage.modules.usersecret.service.UserApiSecretService;
import org.apache.ibatis.builder.MapperBuilderAssistant;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.ArgumentCaptor;
import org.springframework.test.util.ReflectionTestUtils;
import org.springframework.transaction.PlatformTransactionManager;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.isNull;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
/**
* 外观专利 activate 的采集路由(2026-10-29):
* 本地采集开着 ⇒ 恢复「PENDING→RUNNING 条件更新 + 刷心跳」,页面推本地客户端前必须能翻状态;
* 关着 ⇒ 保持后台语义,明确拒绝,避免旧页面缓存继续推队列造成双跑。
*/
class AppearancePatentActivateRoutingTest {
private static final long TASK_ID = 7004L;
private static final long USER_ID = 1L;
private final FileTaskMapper fileTaskMapper = mock(FileTaskMapper.class);
private final AppearancePatentTaskCacheService taskCacheService =
mock(AppearancePatentTaskCacheService.class);
@BeforeEach
void initTableInfo() {
TableInfoHelper.initTableInfo(new MapperBuilderAssistant(new MybatisConfiguration(), ""), FileTaskEntity.class);
}
@Test
void 本地采集开启时_activate_走条件更新并刷心跳() {
FileTaskEntity task = pendingTask();
when(fileTaskMapper.selectById(TASK_ID)).thenReturn(task);
when(fileTaskMapper.update(isNull(), any())).thenReturn(1);
AppearancePatentTaskService service = service(routing(true));
service.activateTask(TASK_ID, USER_ID);
@SuppressWarnings("unchecked")
ArgumentCaptor<Wrapper<FileTaskEntity>> captor = ArgumentCaptor.forClass(Wrapper.class);
verify(fileTaskMapper).update(isNull(), captor.capture());
Map<String, Object> params = paramsOf(captor.getValue());
assertTrue(params.containsValue("PENDING"), "激活必须限定 PENDING,才能与后台认领互斥");
assertTrue(params.containsValue("RUNNING"));
verify(taskCacheService).touchTaskHeartbeat(TASK_ID);
}
@Test
void 本地采集关闭时_activate_保持后台语义并明确拒绝() {
when(fileTaskMapper.selectById(TASK_ID)).thenReturn(pendingTask());
AppearancePatentTaskService service = service(routing(false));
BusinessException error = assertThrows(BusinessException.class,
() -> service.activateTask(TASK_ID, USER_ID));
assertTrue(String.valueOf(error.getMessage()).contains("后台自动执行"), error.getMessage());
verify(fileTaskMapper, never()).update(isNull(), any());
}
private static CollectRoutingProperties routing(boolean local) {
CollectRoutingProperties properties = new CollectRoutingProperties();
properties.setLocalEnabled(Map.of(CollectRoutingProperties.MODULE_APPEARANCE_PATENT, local));
return properties;
}
private FileTaskEntity pendingTask() {
FileTaskEntity task = new FileTaskEntity();
task.setId(TASK_ID);
task.setModuleType(AppearancePatentTaskService.MODULE_TYPE);
task.setUserId(USER_ID);
task.setStatus("PENDING");
return task;
}
private AppearancePatentTaskService service(CollectRoutingProperties routing) {
AppearancePatentTaskService service = new AppearancePatentTaskService(
mock(com.nanri.aiimage.modules.file.service.LocalFileStorageService.class),
null,
mock(ResultDownloadResolver.class),
mock(StorageProperties.class),
fileTaskMapper,
mock(FileResultMapper.class),
mock(TaskScopeStateMapper.class),
mock(TaskChunkMapper.class),
new ObjectMapper(),
mock(AppearancePatentLlmClient.class),
taskCacheService,
mock(AppearancePatentProperties.class),
mock(TaskFileJobService.class),
mock(TaskProgressSnapshotService.class),
mock(TaskHeartbeatPositionService.class),
mock(TransientPayloadStorageService.class),
mock(PlatformTransactionManager.class),
mock(DistributedJobLockService.class),
mock(TaskDistributedLockService.class),
mock(InstanceMetadata.class),
mock(TaskProgressLightAssembler.class),
mock(UserApiSecretService.class));
ReflectionTestUtils.setField(service, "collectRoutingProperties", routing);
return service;
}
@SuppressWarnings("unchecked")
private static Map<String, Object> paramsOf(Wrapper<FileTaskEntity> wrapper) {
wrapper.getSqlSegment();
((LambdaUpdateWrapper<FileTaskEntity>) wrapper).getSqlSet();
Map<String, Object> params = (Map<String, Object>) ReflectionTestUtils.getField(wrapper, "paramNameValuePairs");
assertTrue(params != null && !params.isEmpty());
return params;
}
}
@@ -179,6 +179,7 @@ class AppearancePatentCandidateCollectionFaultToleranceTest {
},
task -> "prompt",
task -> "sk-test",
task -> null,
(taskId, scopeHash, chunkIndex, rows) -> List.of(),
(preferred, fallback) -> preferred == null || preferred.isBlank() ? fallback : preferred,
// 本夹具只驱动「候选收集」,永远不会进欠费终态分支;真进来了就说明测试形状变了
@@ -0,0 +1,165 @@
package com.nanri.aiimage.modules.appearancepatent.service.support;
import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.Logger;
import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.read.ListAppender;
import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.config.AppearancePatentProperties;
import com.nanri.aiimage.modules.appearancepatent.service.AppearancePatentTaskCacheService;
import com.nanri.aiimage.modules.task.mapper.FileTaskMapper;
import com.nanri.aiimage.modules.task.model.entity.FileTaskEntity;
import com.nanri.aiimage.modules.task.model.entity.TaskFileJobEntity;
import com.nanri.aiimage.modules.task.service.TaskFileJobService;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.slf4j.LoggerFactory;
import java.util.List;
import java.util.concurrent.atomic.AtomicLong;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
/**
* AC-21(审计 §5、PERF_REPORT §6):交付缺口补检 WARN 必须按任务限流。
*
* <p>现网实录 task 32609/job 23345 三分钟 478 条(≈9,560 条/小时):结果文件作业每 15s 被
* claim 一轮,同一批缺口行每轮原样再报一次,字段一字不差。同名预置的
* {@code PendingPhase5LogVolumeAndMdcTest} 钉的是源码形状,本支钉的是实跑读数——
* 跑六轮只留一条 WARN,其余降级但不丢数。
*
* <p>限流态是静态的(本类按调用现取、不是 bean),所以每个用例各拿自己的 taskId,
* 免得先跑的用例把窗口占掉。
*/
class AppearancePatentRecheckWarnThrottleTest {
private static final AtomicLong TASK_SEQ = new AtomicLong(System.nanoTime());
private final Logger logger = (Logger) LoggerFactory.getLogger(AppearancePatentDeliverySupport.class);
private final ListAppender<ILoggingEvent> appender = new ListAppender<>();
private Level originalLevel;
private boolean originalAdditive;
private AppearancePatentTaskCacheService cacheService;
private TaskFileJobService taskFileJobService;
private AppearancePatentDeliverySupport support;
@BeforeEach
void attachAppender() {
originalLevel = logger.getLevel();
originalAdditive = logger.isAdditive();
logger.setLevel(Level.DEBUG);
logger.setAdditive(false);
appender.start();
logger.addAppender(appender);
cacheService = mock(AppearancePatentTaskCacheService.class);
taskFileJobService = mock(TaskFileJobService.class);
support = new AppearancePatentDeliverySupport("APPEARANCE_PATENT", new AppearancePatentProperties(),
cacheService, taskFileJobService, mock(FileTaskMapper.class),
null, null, null, null, null, null);
}
@AfterEach
void detachAppender() {
logger.detachAppender(appender);
appender.stop();
logger.setLevel(originalLevel);
logger.setAdditive(originalAdditive);
}
@Test
@DisplayName("AC-21:同一任务连补六轮只留一条 WARN,其余降 DEBUG")
void 补检轮重复触发时窗口内只留一条_WARN() {
long taskId = TASK_SEQ.incrementAndGet();
when(taskFileJobService.requeue(anyLong(), any())).thenReturn(true);
for (int round = 0; round < 6; round++) {
assertThat(support.requestRecheckForMissingRows(task(taskId), job(700L), gap())).isTrue();
}
assertThat(warns()).hasSize(1);
assertThat(debugs()).hasSize(5);
assertThat(warns().get(0)).contains("交付缺口触发补检轮").contains("missingRows=3");
// 限流不许把「补了几轮」这件事抹掉:窗口内被压掉的条数随下一条 WARN 汇总出去
assertThat(warns().get(0)).contains("suppressedInLastWindow=");
}
@Test
@DisplayName("AC-21:限流按任务隔离,另一个任务的缺口照样告警")
void 限流按任务隔离不许吞掉别的任务() {
when(taskFileJobService.requeue(anyLong(), any())).thenReturn(true);
long first = TASK_SEQ.incrementAndGet();
long second = TASK_SEQ.incrementAndGet();
support.requestRecheckForMissingRows(task(first), job(701L), gap());
support.requestRecheckForMissingRows(task(first), job(701L), gap());
support.requestRecheckForMissingRows(task(second), job(702L), gap());
assertThat(warns()).hasSize(2);
assertThat(warns()).anyMatch(line -> line.contains("taskId=" + first))
.anyMatch(line -> line.contains("taskId=" + second));
}
/**
* 「补检排不上队」是要人处置的那一条(作业就此转失败兜底出口),不能被限流顺手压掉。
*/
@Test
@DisplayName("AC-21:排不上队那条 WARN 不受限流影响")
void 排不上队仍然要出声并抛错() {
long taskId = TASK_SEQ.incrementAndGet();
when(taskFileJobService.requeue(anyLong(), any())).thenReturn(false);
assertThatThrownBy(() -> {
support.requestRecheckForMissingRows(task(taskId), job(703L), gap());
support.requestRecheckForMissingRows(task(taskId), job(703L), gap());
}).isInstanceOf(BusinessException.class);
assertThat(warns()).hasSize(1);
assertThat(warns().get(0)).contains("补检排不上队");
assertThat(debugs()).isEmpty();
}
// ── 造数 helper ────────────────────────────────────────────────────
private static FileTaskEntity task(long id) {
FileTaskEntity task = new FileTaskEntity();
task.setId(id);
task.setModuleType("APPEARANCE_PATENT");
task.setStatus("RUNNING");
return task;
}
private static TaskFileJobEntity job(long id) {
TaskFileJobEntity job = new TaskFileJobEntity();
job.setId(id);
return job;
}
/** expected=5 / judged=2 ⇒ missingRows=3,两条缺口行的 key。 */
private static AppearancePatentRowOutcome.Facts gap() {
return new AppearancePatentRowOutcome.Facts(5, 2, 1, List.of("B0AAAA", "B0BBBB"));
}
private List<String> warns() {
return lines(Level.WARN);
}
private List<String> debugs() {
return lines(Level.DEBUG);
}
private List<String> lines(Level level) {
return appender.list.stream()
.filter(event -> event.getLevel() == level)
.map(ILoggingEvent::getFormattedMessage)
.toList();
}
}
@@ -0,0 +1,85 @@
package com.nanri.aiimage.modules.clienterror.controller;
import com.nanri.aiimage.common.api.ApiResponse;
import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.common.model.entity.AdminUserEntity;
import com.nanri.aiimage.common.security.AdminAuthSupport;
import jakarta.servlet.http.HttpServletRequest;
import org.junit.jupiter.api.Test;
import java.util.HashMap;
import java.util.Map;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
/**
* 后管浏览器错误上报口(S1-F2/AC-16)的鉴权与体量闸门。
*
* <p>两条底线:① 这是写入口,匿名请求(sendBeacon 没带管理员会话)必须被拒——与 AC-8
* 「匿名写端点要凭据」同一口径,否则任何人都能往服务端日志里灌任意文本;
* ② 报文体一律逐字段截断,堆栈不许把整条日志撑爆,也不许原样回显给调用方。</p>
*/
class ClientErrorReportControllerTest {
private final AdminAuthSupport adminAuthSupport = mock(AdminAuthSupport.class);
private final HttpServletRequest request = mock(HttpServletRequest.class);
private ClientErrorReportController newController() {
return new ClientErrorReportController(adminAuthSupport);
}
private void loginAsAdmin() {
AdminUserEntity operator = new AdminUserEntity();
operator.setId(11L);
when(adminAuthSupport.requireAdmin(any())).thenReturn(operator);
}
@Test
void 匿名上报必须被拒() {
when(adminAuthSupport.requireAdmin(any())).thenThrow(new BusinessException(401, "未登录"));
assertThatThrownBy(() -> newController().report(request, Map.of("kind", "window-error")))
.isInstanceOf(BusinessException.class)
.hasMessageContaining("未登录");
}
@Test
void 合法会话上报成功且只回一个录制标记() {
loginAsAdmin();
ApiResponse<Map<String, Object>> response = newController().report(request, Map.of(
"kind", "vue-render",
"message", "Cannot read properties of undefined",
"route", "/admin/users",
"ts", 1_730_000_000_000L));
assertThat(response.getData()).containsExactly(entryOf("recorded", true));
}
@Test
void 空请求体不得把上报本身炸掉() {
loginAsAdmin();
assertThat(newController().report(request, null).getData()).containsKey("recorded");
assertThat(newController().report(request, new HashMap<>()).getData()).containsKey("recorded");
}
@Test
void 超长字段截断到上限并留截断标记() {
String huge = "x".repeat(5_000);
String clipped = ClientErrorReportController.clip(huge);
assertThat(clipped).hasSize(2001).endsWith("…");
assertThat(ClientErrorReportController.clip(null)).isEmpty();
assertThat(ClientErrorReportController.clip("短文本")).isEqualTo("短文本");
}
private static Map.Entry<String, Object> entryOf(String key, Object value) {
return Map.entry(key, value);
}
}
@@ -0,0 +1,149 @@
package com.nanri.aiimage.modules.file.service;
import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.config.StorageProperties;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.springframework.mock.web.MockMultipartFile;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.MessageDigest;
import java.util.HexFormat;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* AC-17 分片上传:拼装正确、缺片拒绝、重复片幂等、放弃即清理、uploadId 不可穿越。
* 不依赖 MinIO/MySQL —— 分片装配在本地临时目录完成,转存 MinIO 由工作台链路单独验证。
*/
class ChunkedUploadServiceTest {
@TempDir
Path tempDir;
private ChunkedUploadService service;
@BeforeEach
void setUp() {
StorageProperties properties = new StorageProperties();
properties.setLocalTempDir(tempDir.toString());
service = new ChunkedUploadService(properties);
}
private MockMultipartFile part(String content) {
return new MockMultipartFile("file", "part.bin", "application/octet-stream",
content.getBytes(StandardCharsets.UTF_8));
}
@Test
void 分片到齐后拼装并复算摘要() throws Exception {
String first = "hello-";
String second = "world";
long total = first.getBytes(StandardCharsets.UTF_8).length
+ second.getBytes(StandardCharsets.UTF_8).length;
ChunkedUploadService.ChunkSession session =
service.init("包.zip", 2, total, "purpose-key");
assertEquals(1, service.savePart(session.uploadId(), 0, part(first)));
assertEquals(2, service.savePart(session.uploadId(), 1, part(second)));
ChunkedUploadService.CompletedUpload completed = service.complete(session.uploadId());
try {
String content = Files.readString(completed.file(), StandardCharsets.UTF_8);
assertEquals(first + second, content);
assertEquals(total, completed.size());
assertEquals("包.zip", completed.fileName());
String expected = HexFormat.of().formatHex(
MessageDigest.getInstance("SHA-256")
.digest((first + second).getBytes(StandardCharsets.UTF_8)));
assertEquals(expected, completed.sha256(), "摘要必须是对拼装实物算出来的");
} finally {
Files.deleteIfExists(completed.file());
}
assertFalse(Files.exists(tempDir.resolve("chunk-uploads").resolve(session.uploadId())),
"拼装完成后分片会话目录必须清掉");
}
@Test
void 缺少分片必须拒绝且不产出半个文件() throws Exception {
ChunkedUploadService.ChunkSession session = service.init("a.zip", 2, 10, "");
service.savePart(session.uploadId(), 0, part("12345"));
BusinessException error = assertThrows(BusinessException.class,
() -> service.complete(session.uploadId()));
assertTrue(error.getMessage().contains("缺少"), "必须明确告知缺哪片: " + error.getMessage());
try (var files = Files.list(tempDir.resolve("chunk-uploads"))) {
assertFalse(files.anyMatch(p -> p.getFileName().toString().startsWith("assembled-")),
"缺片时不许留在半成品");
} catch (Exception ex) {
throw new AssertionError(ex);
}
}
@Test
void 重复上传同一片按覆盖处理() throws Exception {
ChunkedUploadService.ChunkSession session = service.init("a.bin", 1, 3, "");
service.savePart(session.uploadId(), 0, part("old"));
service.savePart(session.uploadId(), 0, part("new"));
ChunkedUploadService.CompletedUpload completed = service.complete(session.uploadId());
try {
assertEquals("new", Files.readString(completed.file(), StandardCharsets.UTF_8));
} finally {
Files.deleteIfExists(completed.file());
}
}
@Test
void 声明大小与实际不符必须拒绝() throws Exception {
ChunkedUploadService.ChunkSession session = service.init("a.bin", 1, 99, "");
service.savePart(session.uploadId(), 0, part("short"));
BusinessException error = assertThrows(BusinessException.class,
() -> service.complete(session.uploadId()));
assertTrue(error.getMessage().contains("不一致"), error.getMessage());
}
@Test
void 放弃会话立即清理分片() throws Exception {
ChunkedUploadService.ChunkSession session = service.init("a.bin", 1, 3, "");
service.savePart(session.uploadId(), 0, part("abc"));
service.abort(session.uploadId());
assertFalse(Files.exists(tempDir.resolve("chunk-uploads").resolve(session.uploadId())));
assertThrows(BusinessException.class, () -> service.uploadedParts(session.uploadId()));
}
@Test
void uploadId必须拒绝路径穿越() {
assertThrows(BusinessException.class, () -> service.uploadedParts("../etc"));
assertThrows(BusinessException.class, () -> service.abort("..\\..\\windows"));
assertThrows(BusinessException.class, () -> service.savePart("abc", 0, part("x")));
}
@Test
void 分片序号越界必须拒绝() {
ChunkedUploadService.ChunkSession session = service.init("a.bin", 2, 10, "");
assertThrows(BusinessException.class,
() -> service.savePart(session.uploadId(), 2, part("x")));
assertThrows(BusinessException.class,
() -> service.savePart(session.uploadId(), -1, part("x")));
}
@Test
void 已上传片号可查询用于续传() throws Exception {
ChunkedUploadService.ChunkSession session = service.init("a.bin", 3, 9, "");
service.savePart(session.uploadId(), 0, part("aaa"));
service.savePart(session.uploadId(), 2, part("ccc"));
List<Integer> uploaded = service.uploadedParts(session.uploadId());
assertEquals(List.of(0, 2), uploaded);
}
}
@@ -0,0 +1,83 @@
package com.nanri.aiimage.modules.file.service.oss;
import com.nanri.aiimage.config.OssProperties;
import com.nanri.aiimage.modules.workbenchclient.service.WorkbenchClientPackageService;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.UUID;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* 线上 MinIO 实测(opt-in):AC-17 分片拼装后的服务端转存链路必须真的落进
* 公开 client 桶的 {@code workbench-client/versions/} 前缀,且 stat/sha256/删除
* 全程可用——本机日常闸门不带凭据时自动跳过,绝不碰生产。
*
* <p>跑法(凭据来自线上 java-server 容器 env,只进环境变量、不落任何文件)::</p>
* <pre>
* AIIMAGE_OSS_ACCESS_KEY_ID=... AIIMAGE_OSS_ACCESS_KEY_SECRET=... \
* mvn -o test -Dtest=OssWorkbenchClientLiveVerificationTest
* </pre>
*/
@EnabledIfEnvironmentVariable(named = "AIIMAGE_OSS_ACCESS_KEY_ID", matches = ".+")
class OssWorkbenchClientLiveVerificationTest {
private static final DateTimeFormatter STAMP = DateTimeFormatter.ofPattern("yyyyMMddHHmmssSSS");
private OssStorageService liveService() {
OssProperties properties = new OssProperties();
properties.setRegion("us-east-1");
properties.setEndpoint(System.getenv().getOrDefault("AIIMAGE_OSS_ENDPOINT", "https://oss.aishufu.top"));
properties.setPublicEndpoint(
System.getenv().getOrDefault("AIIMAGE_OSS_PUBLIC_ENDPOINT", "https://oss.aishufu.top"));
properties.setBucket("nanri-ai-images");
properties.setAccessKeyId(System.getenv("AIIMAGE_OSS_ACCESS_KEY_ID"));
properties.setAccessKeySecret(System.getenv("AIIMAGE_OSS_ACCESS_KEY_SECRET"));
return new OssStorageService(properties);
}
@Test
void 服务端转存线上MinIO_大小与摘要对得上并可清理() throws Exception {
OssStorageService service = liveService();
String objectKey = WorkbenchClientPackageService.OBJECT_KEY_PREFIX
+ "zz-live-verify-" + STAMP.format(LocalDateTime.now())
+ "-" + UUID.randomUUID().toString().replace("-", "") + "-0.0.0.zip";
Path zip = Files.createTempFile("live-verify-", ".zip");
byte[] marker = "phase4-live-verify".getBytes(StandardCharsets.UTF_8);
try (ZipOutputStream out = new ZipOutputStream(Files.newOutputStream(zip))) {
out.putNextEntry(new ZipEntry("verify.txt"));
out.write(marker);
out.closeEntry();
}
long declaredSize = Files.size(zip);
try {
service.uploadWorkbenchClientObject(zip.toFile(), objectKey);
long statSize = service.workbenchClientObjectSize(objectKey);
assertEquals(declaredSize, statSize, "stat 必须回读真实对象大小");
String sha256 = service.workbenchClientObjectSha256(objectKey);
assertNotNull(sha256);
assertEquals(64, sha256.length(), "sha256 必须是 64 位 hex");
String presignUrl = service.presignWorkbenchClientUpload(objectKey, 120);
assertNotNull(presignUrl);
assertTrue(presignUrl.contains(objectKey.substring(objectKey.indexOf('/') + 1))
|| presignUrl.contains(objectKey),
"预签名 URL 必须指向本次对象");
} finally {
service.removeWorkbenchClientObject(objectKey);
Files.deleteIfExists(zip);
}
assertEquals(-1L, service.workbenchClientObjectSize(objectKey), "清理后对象必须不存在");
}
}
@@ -40,9 +40,10 @@ class SimilarAsinChineseMessageGuardTest {
/**
* 2026-10-09 实测:TaskService 42 个出口 + Assembler 20 个出口。
* 2026-10-20 空结果的两条 throw 合并成一条(文案带真因)→ Assembler 19,合计 61。
* 2026-10-29 本地采集开关:TaskService 的 activateTask 新增一条「已切换后台自动执行」→ 42→43,合计 62。
* 新增/删除出口要同步改这里。
*/
private static final int SCANNED_SITE_COUNT = 61;
private static final int SCANNED_SITE_COUNT = 62;
/** 文案来自变量、字面量判据覆盖不到的出口,按构造点名(行号会漂)。 */
private static final List<String> VARIABLE_FED_OUTLETS = List.of(
@@ -0,0 +1,48 @@
package com.nanri.aiimage.modules.task.controller;
import com.nanri.aiimage.config.CollectRoutingProperties;
import org.junit.jupiter.api.Test;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import java.util.Map;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
/**
* 采集路由读取口:前端两个工具页据这两个布尔决定「启动」是推本地客户端还是提交后台。
* 开关的唯一真源在服务端配置,这里只钉「配置怎么配,接口就怎么答」。
*/
class CollectRoutingControllerTest {
@Test
void 开关默认开启时两个模块都答本地() throws Exception {
CollectRoutingProperties properties = new CollectRoutingProperties();
properties.setLocalEnabled(Map.of("APPEARANCE_PATENT", true, "SIMILAR_ASIN", true));
MockMvc mockMvc = MockMvcBuilders
.standaloneSetup(new CollectRoutingController(properties))
.build();
mockMvc.perform(get("/api/collect-routing"))
.andExpect(status().isOk())
.andExpect(jsonPath("$.success").value(true))
.andExpect(jsonPath("$.data.appearancePatentLocal").value(true))
.andExpect(jsonPath("$.data.similarAsinLocal").value(true));
}
@Test
void 关闭本地采集时接口如实回答且未登记模块不影响这两项() throws Exception {
CollectRoutingProperties properties = new CollectRoutingProperties();
properties.setLocalEnabled(Map.of("APPEARANCE_PATENT", false, "SIMILAR_ASIN", true));
MockMvc mockMvc = MockMvcBuilders
.standaloneSetup(new CollectRoutingController(properties))
.build();
mockMvc.perform(get("/api/collect-routing"))
.andExpect(status().isOk())
.andExpect(jsonPath("$.data.appearancePatentLocal").value(false))
.andExpect(jsonPath("$.data.similarAsinLocal").value(true));
}
}
@@ -17,12 +17,14 @@ import org.mockito.ArgumentCaptor;
import org.springframework.test.util.ReflectionTestUtils;
import java.time.LocalDateTime;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
@@ -34,9 +36,10 @@ import static org.mockito.Mockito.verifyNoMoreInteractions;
import static org.mockito.Mockito.when;
/**
* 陈旧任务判死契约(2026-09-14 修订):
* PENDING 仍按 updated_at 判死;SCHEDULED(店铺匹配定时任务)改以 scheduled_at 为基准 ——
* 只按 updated_at 会误杀排期在 2 小时之后的定时任务,而 activate 又被「未到定时执行时间」挡住。
* 陈旧任务判死契约(2026-10-29 修订):
* <b>PENDING 不再判死</b>——它是"排队等 worker 领取",等多久都保留(产品口径:一直排队就行);
* SCHEDULED(店铺匹配定时任务)以 scheduled_at 为基准 —— 只按 updated_at 会误杀排期在
* 2 小时之后的定时任务,而 activate 又被「未到定时执行时间」挡住;RUNNING 按心跳判死。
*/
class StaleTaskRepairServiceTest {
@@ -58,19 +61,25 @@ class StaleTaskRepairServiceTest {
}
@Test
void PENDING_判死只命中_PENDING_不再波及_SCHEDULED() {
when(fileTaskMapper.selectList(any())).thenReturn(List.of(task(11L)));
when(fileTaskMapper.update(isNull(), any())).thenReturn(1);
void PENDING_排队超时不判死_判死入口不得回填() {
// 2026-10-29 产品口径:PENDING 是排队态,AP worker 串行跑大任务时队列再长也保留,
// 由 pull 认领;按更新时间清杀会把排队任务误判成「长期未被领取」,还会连锁触发自动重试。
assertNull(Arrays.stream(StaleTaskRepairService.class.getDeclaredMethods())
.filter(method -> "repairFileTaskStaleIdle".equals(method.getName()))
.findFirst().orElse(null),
"PENDING 判死入口不得回填:排队中的任务要一直留着等 worker 领取");
ReflectionTestUtils.invokeMethod(service, "repairFileTaskStaleIdle");
when(fileTaskMapper.selectList(any())).thenReturn(List.of());
ReflectionTestUtils.invokeMethod(service, "repairFileTaskStaleScheduled");
ReflectionTestUtils.invokeMethod(service, "repairFileTaskStaleRunning");
Map<String, Object> selectParams = selectParams();
assertTrue(selectParams.containsValue("PENDING"));
assertFalse(selectParams.containsValue("SCHEDULED"), "PENDING 分支命中 SCHEDULED 是定时任务被误杀的原因");
Map<String, Object> updateParams = updateParams(1);
assertTrue(updateParams.containsValue("FAILED"));
assertTrue(containsMessage(updateParams, "任务长期未被领取"));
ArgumentCaptor<Wrapper<FileTaskEntity>> captor = wrapperCaptor();
verify(fileTaskMapper, org.mockito.Mockito.times(2)).selectList(captor.capture());
for (Wrapper<FileTaskEntity> wrapper : captor.getAllValues()) {
assertFalse(paramsOf(wrapper).containsValue("PENDING"),
"判死巡检不得再扫 PENDING:排队任务既不许按 updated_at 清杀,也不许被别的分支误伤");
}
verify(fileTaskMapper, never()).update(any(), any());
}
@Test
@@ -100,7 +109,6 @@ class StaleTaskRepairServiceTest {
void 没有候选任务时不写库() {
when(fileTaskMapper.selectList(any())).thenReturn(List.of());
ReflectionTestUtils.invokeMethod(service, "repairFileTaskStaleIdle");
ReflectionTestUtils.invokeMethod(service, "repairFileTaskStaleScheduled");
ReflectionTestUtils.invokeMethod(service, "repairFileTaskStaleRunning");
@@ -169,23 +177,6 @@ class StaleTaskRepairServiceTest {
verify(fileTaskMapper, never()).update(isNull(), any());
}
@Test
void 未被领取判死同样要回写结果行并终结作业() {
when(fileTaskMapper.selectList(any())).thenReturn(List.of(task(31176L, "PENDING")));
when(fileTaskMapper.update(isNull(), any())).thenReturn(1);
ReflectionTestUtils.invokeMethod(service, "repairFileTaskStaleIdle");
verify(fileResultMapper).update(isNull(), any());
verify(taskFileJobService).failUnfinishedJobs(eq(List.of(31176L)), any());
}
private Map<String, Object> selectParams() {
ArgumentCaptor<Wrapper<FileTaskEntity>> captor = wrapperCaptor();
verify(fileTaskMapper).selectList(captor.capture());
return paramsOf(captor.getValue());
}
private Map<String, Object> updateParams(int times) {
ArgumentCaptor<Wrapper<FileTaskEntity>> captor = wrapperCaptor();
verify(fileTaskMapper, org.mockito.Mockito.times(times)).update(isNull(), captor.capture());
@@ -190,13 +190,96 @@ class TaskClientPullServiceTest {
}
}
// ------------------------------------------------------------ 本地采集路由(2026-10-29)
@Test
void 客户端兜底拉取只认本地模块() {
com.nanri.aiimage.config.CollectRoutingProperties routing =
new com.nanri.aiimage.config.CollectRoutingProperties();
routing.setLocalEnabled(Map.of("SIMILAR_ASIN", false, "COLLECT_DATA", false));
List<TaskClientPullVo> result = serviceWithRouting(routing,
new StubHandler("SIMILAR_ASIN", payload("similar-asin-run"))).pullPendingTasks(USER_ID);
assertTrue(result.isEmpty(), "模块切到 Linux 后台后客户端不应再拉取");
verifyNoInteractions(fileTaskMapper);
}
@Test
void 内部认领对本地模块让位到兜底窗口() {
com.nanri.aiimage.config.CollectRoutingProperties routing =
new com.nanri.aiimage.config.CollectRoutingProperties();
routing.setLocalEnabled(Map.of("SIMILAR_ASIN", true));
routing.setLinuxFallbackMinutes(30L);
when(fileTaskMapper.selectList(any())).thenReturn(List.of(task(11L, "SIMILAR_ASIN", 10L)));
List<TaskClientPullVo> result = serviceWithRouting(routing,
new StubHandler("SIMILAR_ASIN", payload("similar-asin-run")))
.pullPendingTasksInternal(List.of("SIMILAR_ASIN"));
assertTrue(result.isEmpty(), "本地采集模块在兜底窗口内必须让位给本地客户端");
verify(fileTaskMapper, never()).update(isNull(), any());
}
@Test
void 内部认领超过兜底窗口后接管本地模块() {
com.nanri.aiimage.config.CollectRoutingProperties routing =
new com.nanri.aiimage.config.CollectRoutingProperties();
routing.setLocalEnabled(Map.of("SIMILAR_ASIN", true));
routing.setLinuxFallbackMinutes(30L);
FileTaskEntity candidate = task(11L, "SIMILAR_ASIN", 40L);
when(fileTaskMapper.selectList(any())).thenReturn(List.of(candidate));
when(fileTaskMapper.update(isNull(), any())).thenReturn(1);
when(fileTaskMapper.selectById(11L)).thenReturn(candidate);
List<TaskClientPullVo> result = serviceWithRouting(routing,
new StubHandler("SIMILAR_ASIN", payload("similar-asin-run")))
.pullPendingTasksInternal(List.of("SIMILAR_ASIN"));
assertEquals(1, result.size(), "本地长时间没人领,Linux 必须兜底接管");
}
@Test
void 内部认领对非本地模块保持原等待窗口() {
com.nanri.aiimage.config.CollectRoutingProperties routing =
new com.nanri.aiimage.config.CollectRoutingProperties();
routing.setLocalEnabled(Map.of("SIMILAR_ASIN", true));
routing.setLinuxFallbackMinutes(30L);
FileTaskEntity candidate = task(21L, "COLLECT_DATA", 10L);
when(fileTaskMapper.selectList(any())).thenReturn(List.of(candidate));
when(fileTaskMapper.update(isNull(), any())).thenReturn(1);
when(fileTaskMapper.selectById(21L)).thenReturn(candidate);
List<TaskClientPullVo> result = serviceWithRouting(routing,
new StubHandler("COLLECT_DATA", payload("collect-data-run")))
.pullPendingTasksInternal(List.of("COLLECT_DATA"));
assertEquals(1, result.size(), "未登记本地开关的模块不受兜底窗口影响");
}
private TaskClientPullService service(ClientTaskPullSpi... handlers) {
return serviceWithRouting(defaultRouting(), handlers);
}
/** 默认路由:两个模块都按本地采集(现有用例的既有语义)。 */
private static com.nanri.aiimage.config.CollectRoutingProperties defaultRouting() {
com.nanri.aiimage.config.CollectRoutingProperties routing =
new com.nanri.aiimage.config.CollectRoutingProperties();
routing.setLocalEnabled(Map.of("SIMILAR_ASIN", true, "COLLECT_DATA", true));
return routing;
}
private TaskClientPullService serviceWithRouting(
com.nanri.aiimage.config.CollectRoutingProperties routing,
ClientTaskPullSpi... handlers) {
TaskClientPullService service = new TaskClientPullService(
fileTaskMapper, new InstanceMetadata("server-test"), List.of(handlers));
fileTaskMapper, new InstanceMetadata("server-test"), routing, List.of(handlers));
ReflectionTestUtils.setField(service, "enabled", true);
ReflectionTestUtils.setField(service, "moduleTypes", "SIMILAR_ASIN,COLLECT_DATA");
ReflectionTestUtils.setField(service, "minPendingMinutes", 5L);
ReflectionTestUtils.setField(service, "limit", 5);
ReflectionTestUtils.setField(service, "internalEnabled", true);
ReflectionTestUtils.setField(service, "internalMinPendingMinutes", 2L);
return service;
}
@@ -243,14 +326,18 @@ class TaskClientPullServiceTest {
}
private static FileTaskEntity task(Long id, String moduleType) {
return task(id, moduleType, 10L);
}
private static FileTaskEntity task(Long id, String moduleType, long createdAtMinutesAgo) {
FileTaskEntity task = new FileTaskEntity();
task.setId(id);
task.setModuleType(moduleType);
task.setTaskNo(moduleType + "-" + id);
task.setUserId(USER_ID);
task.setStatus("PENDING");
task.setCreatedAt(LocalDateTime.now().minusMinutes(10));
task.setUpdatedAt(LocalDateTime.now().minusMinutes(10));
task.setCreatedAt(LocalDateTime.now().minusMinutes(createdAtMinutesAgo));
task.setUpdatedAt(LocalDateTime.now().minusMinutes(createdAtMinutesAgo));
return task;
}
@@ -58,10 +58,12 @@ class TaskCenterQueryMapperSqlTest {
.replaceAll("\\s*,\\s*", ",");
}
/** 任务列表分页的查询参数(状态已改为「一组大写状态」,「在跑」由服务层展开)。 */
private static Map<String, Object> taskPageParams(Object shop, long limit, long offset) {
/** AC-19 之后的页查询参数:游标两个都缺 = 从头取;给值 = keyset 取下一页。 */
private static Map<String, Object> pageParams(Object shop, long limit,
Object afterCreatedAt, Object afterId) {
return params("userIds", null, "moduleType", null, "statuses", null, "shop", shop,
"startTime", null, "endTime", null, "limit", limit, "offset", offset);
"startTime", null, "endTime", null, "limit", limit,
"afterCreatedAt", afterCreatedAt, "afterId", afterId);
}
@Test
@@ -190,7 +192,7 @@ class TaskCenterQueryMapperSqlTest {
@Test
void taskListUnionsBizAndBrandWithoutTouchingRequestJson() {
BoundSql boundSql = render("selectTaskPage", taskPageParams(null, 20L, 0L));
BoundSql boundSql = render("selectTaskPage", pageParams(null, 20L, null, null));
String sql = sqlOf(boundSql);
assertTrue(sql.contains("SELECT m.* FROM(SELECT t.id AS id"));
@@ -203,11 +205,45 @@ class TaskCenterQueryMapperSqlTest {
assertTrue(sql.contains("CASE WHEN b.status IN('cancelled','failed','success') THEN b.updated_at END"),
"品牌表无 finished_at,终态用 updated_at");
assertTrue(sql.contains("ORDER BY m.createdAt DESC,m.id DESC"));
assertTrue(sql.contains("LIMIT ? OFFSET ?"));
assertEquals(2, boundSql.getParameterMappings().size(), "只有分页两个参数");
// AC-19(P1-B1):页查询不再吃 OFFSET——深翻页的 skip 挪到 selectTaskPageAnchor 的窄投影里
assertFalse(sql.contains("OFFSET"), "页查询里再出现 OFFSET 就是深分页回归:" + sql);
assertTrue(sql.contains("LIMIT ?"));
assertEquals(1, boundSql.getParameterMappings().size(), "不带游标时只绑 limit");
assertFalse(sql.contains("request_json"), "列表主查询不拉大 JSON 列");
}
/** AC-19(P1-B1):页首游标锚点必须与页查询共用同一份 UNION,且只做两列窄投影。 */
@Test
void pageAnchorSharesTheUnionAndProjectsOnlyTheCursorColumns() {
BoundSql boundSql = render("selectTaskPageAnchor",
params("userIds", null, "moduleType", null, "statuses", null, "shop", null,
"startTime", LocalDateTime.of(2026, 9, 1, 0, 0), "endTime", null, "offset", 200L));
String sql = sqlOf(boundSql);
assertTrue(sql.startsWith("SELECT m.createdAt,m.id FROM(SELECT t.id AS id"),
"锚点只许投影游标两列,宽行装配是页查询的事:" + sql);
assertTrue(sql.contains("AND t.created_at >= ?"), "锚点必须与页查询同一套筛选,否则算出的是另一批行");
assertTrue(sql.contains("ORDER BY m.createdAt DESC,m.id DESC LIMIT 1 OFFSET ?"),
"排序键与页查询逐字相同,skip 只发生在这条窄语句里");
assertEquals(3, boundSql.getParameterMappings().size(),
"offset + 起止时间下界两个(锚点与页查询共用同一份 UNION,筛选照常绑)");
assertFalse(sql.contains("task_no"), "锚点不许带列表列");
}
/** AC-19(P1-B1):给了游标才渲染 keyset 条件,两个分支都被 created_at 下界收窄。 */
@Test
void cursorNarrowsBothBranchesInsteadOfSkippingMaterialisedRows() {
LocalDateTime anchor = LocalDateTime.of(2026, 10, 1, 12, 0);
BoundSql boundSql = render("selectTaskPage", pageParams(null, 20L, anchor, 8801L));
String sql = sqlOf(boundSql);
assertTrue(sql.contains("WHERE(m.createdAt < ? OR(m.createdAt = ? AND m.id < ?))"),
"游标必须落在 (createdAt,id) 这一对排序键上:" + sql);
assertEquals(4, boundSql.getParameterMappings().size(), "limit + 游标三个占位(afterCreatedAt 用两次)");
assertEquals("afterCreatedAt", boundSql.getParameterMappings().get(1).getProperty(),
"游标紧跟 limit 之后入参,顺序漂了说明绑的不是那对排序键");
}
@Test
void taskListFiltersReachBothBranchesAndBrandDropsShopFilter() {
BoundSql boundSql = render("selectTaskPage",
@@ -215,7 +251,8 @@ class TaskCenterQueryMapperSqlTest {
"statuses", List.of("RUNNING"), "shop", "旗舰店",
"startTime", LocalDateTime.of(2026, 9, 1, 0, 0),
"endTime", LocalDateTime.of(2026, 9, 28, 0, 0),
"limit", 20L, "offset", 20L));
"limit", 20L,
"afterCreatedAt", LocalDateTime.of(2026, 9, 20, 0, 0), "afterId", 4001L));
String sql = sqlOf(boundSql);
assertTrue(sql.contains("AND UPPER(t.module_type) = ?"));
@@ -236,13 +273,14 @@ class TaskCenterQueryMapperSqlTest {
assertFalse(sql.contains("LIKE CONCAT('%',?,'%')"), "店铺筛选按规范化精确匹配,模糊会命中同名前缀");
assertEquals(5L, boundSql.getAdditionalParameter("__frch_uid_0"));
assertEquals(16, boundSql.getParameterMappings().size(),
"biz 8(账号×2/业务/状态/起止/店铺×2)+ 品牌 6(账号×2/业务/状态/起止)+ 分页 2");
assertEquals(18, boundSql.getParameterMappings().size(),
"biz 8(账号×2/业务/状态/起止/店铺×2)+ 品牌 6(账号×2/业务/状态/起止)"
+ "+ AC-19 分页 4(limit + 游标三个占位,afterCreatedAt 用两次)");
}
@Test
void shopFilterDoesNotReadSnapshotScopeOrResultItemGroup() {
String filtered = sqlOf(render("selectTaskPage", taskPageParams("林芳", 20L, 0L)));
String filtered = sqlOf(render("selectTaskPage", pageParams("林芳", 20L, null, null)));
assertFalse(filtered.contains("biz_task_result_item"), "任务结果分组不是店铺来源");
assertFalse(filtered.contains("biz_task_progress_snapshot"), "进度快照不是店铺来源");
@@ -251,14 +289,14 @@ class TaskCenterQueryMapperSqlTest {
@Test
void taskListWithoutFiltersOnlyBindsPaging() {
BoundSql boundSql = render("selectTaskPage", taskPageParams(null, 100L, 0L));
BoundSql boundSql = render("selectTaskPage", pageParams(null, 100L, null, null));
String sql = sqlOf(boundSql);
assertFalse(sql.contains("AND UPPER(t.module_type)"), "未选业务不加条件");
assertFalse(sql.contains("AND 1 = 0"), "未选店铺时品牌分支照常入列");
assertFalse(sql.contains("EXISTS"), "未选店铺不做 EXISTS 子查询");
assertFalse(sql.contains("__frch"), "超管不渲染账号占位符");
assertEquals(2, boundSql.getParameterMappings().size());
assertEquals(1, boundSql.getParameterMappings().size(), "AC-19 之后只剩 limit 一个分页参数");
}
@Test
@@ -286,14 +324,14 @@ class TaskCenterQueryMapperSqlTest {
params("userIds", null, "moduleType", null,
"statuses", TaskCenterQueryMapper.ACTIVE_STATUSES_ORDERED,
"shop", null, "startTime", null, "endTime", null,
"limit", 20L, "offset", 0L));
"limit", 20L, "afterCreatedAt", null, "afterId", null));
String sql = sqlOf(boundSql);
assertTrue(sql.contains("AND " + TaskCenterQueryMapper.STATUS_T + " IN(?,?,?)"),
"biz 分支按展开后的三个状态过滤:" + sql);
assertTrue(sql.contains("AND " + TaskCenterQueryMapper.STATUS_B + " IN(?,?,?)"),
"品牌分支同口径(小写状态已由 STATUS_B 归一为大写)");
assertEquals(8, boundSql.getParameterMappings().size(), "两张表各绑三个状态 + 分页两个");
assertEquals(7, boundSql.getParameterMappings().size(), "两张表各绑三个状态 + AC-19 只剩 limit 一个");
assertEquals("PENDING", boundSql.getAdditionalParameter("__frch_st_0"));
assertEquals("RUNNING", boundSql.getAdditionalParameter("__frch_st_1"));
assertEquals("SCHEDULED", boundSql.getAdditionalParameter("__frch_st_2"));
@@ -12,7 +12,6 @@ import com.baomidou.mybatisplus.core.MybatisConfiguration;
import com.baomidou.mybatisplus.core.metadata.TableInfoHelper;
import org.apache.ibatis.builder.MapperBuilderAssistant;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
@@ -50,7 +49,6 @@ import static org.mockito.Mockito.when;
* <p>p95 不劣化要靠 Phase 5 的 EXPLAIN/基准前后对比(蓝图 §4 Phase 5 准出),
* 本文件钉的是"形状必须改成这样"的可判据部分。
*/
@Disabled("Pending Phase 5: AC-19 读路径时间窗/keyset、去 RAND、心跳瘦字段、菜单缓存")
@DisplayName("AC-19 高频读路径治理契约")
class PendingPhase5ReadPathTest {
@@ -15,6 +15,7 @@ import com.nanri.aiimage.modules.taskcenter.model.vo.TaskCenterOverviewVo;
import com.nanri.aiimage.modules.taskcenter.model.vo.TaskCenterTaskDetailVo;
import com.nanri.aiimage.modules.taskcenter.model.vo.TaskCenterTaskPageVo;
import com.nanri.aiimage.modules.taskcenter.model.vo.TaskCenterTaskRowVo;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.ArgumentCaptor;
@@ -39,6 +40,7 @@ import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.argThat;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.ArgumentMatchers.isNull;
import static org.mockito.ArgumentMatchers.notNull;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
@@ -166,8 +168,8 @@ class TaskCenterQueryServiceTest {
private void stubPage(List<TaskCenterTaskRow> rows) {
when(queryMapper.countTaskPage(any(), any(), any(), any(), any(), any()))
.thenReturn((long) rows.size());
when(queryMapper.selectTaskPage(any(), any(), any(), any(), any(), any(), anyLong(), anyLong()))
.thenReturn(rows);
when(queryMapper.selectTaskPage(any(), any(), any(), any(), any(), any(),
anyLong(), any(), any())).thenReturn(rows);
}
private void stubActiveTasks(List<TaskCenterTaskRow> rows) {
@@ -296,8 +298,9 @@ class TaskCenterQueryServiceTest {
TaskCenterTaskPageVo page = service.tasks(scoped(5L, 11L), query);
assertEquals(0, page.getTotal(), "只有 11 在本组可见集合内,99 越界");
// 第 5 个参数是 startTime:AC-19 之后不带时间参数的请求由服务层补默认窗口,不再是 null
verify(queryMapper).countTaskPage(argThat(ids -> List.of(11L).equals(ids)),
isNull(), isNull(), isNull(), isNull(), isNull());
isNull(), isNull(), isNull(), notNull(), isNull());
}
@Test
@@ -324,16 +327,21 @@ class TaskCenterQueryServiceTest {
service.tasks(superScope(), query);
verify(queryMapper).countTaskPage(argThat(ids -> List.of(11L, 99L).equals(ids)),
isNull(), isNull(), isNull(), isNull(), isNull());
isNull(), isNull(), isNull(), notNull(), isNull());
}
@Test
void filtersPassThroughAndPageSizeIsCapped() {
LocalDateTime start = LocalDateTime.of(2026, 9, 1, 0, 0);
LocalDateTime end = LocalDateTime.of(2026, 9, 28, 23, 59);
// AC-19:第 3 页的 skip 不再交给 SQL 的 OFFSET,而是先取页首游标(上一页最后一行)
LocalDateTime anchorTime = LocalDateTime.of(2026, 9, 20, 8, 30);
long anchorId = 770_003L;
when(queryMapper.countTaskPage(any(), any(), any(), any(), any(), any())).thenReturn(1L);
when(queryMapper.selectTaskPage(any(), any(), any(), any(), any(), any(), anyLong(), anyLong()))
.thenReturn(List.of());
when(queryMapper.selectTaskPageAnchor(any(), any(), any(), any(), any(), any(), anyLong()))
.thenReturn(taskRow(anchorId, "PUBLISH", "SUCCESS", anchorTime, anchorTime));
when(queryMapper.selectTaskPage(any(), any(), any(), any(), any(), any(),
anyLong(), any(), any())).thenReturn(List.of());
TaskCenterTaskQuery query = new TaskCenterTaskQuery();
query.setModuleType("PUBLISH");
@@ -348,8 +356,74 @@ class TaskCenterQueryServiceTest {
verify(queryMapper).countTaskPage(isNull(), eq("PUBLISH"), eq(List.of("RUNNING")),
eq("旗舰店"), eq(start), eq(end));
// (page-1)*pageSize 这条算术还在,只是它的去向从 OFFSET 变成了锚点行号
verify(queryMapper).selectTaskPageAnchor(isNull(), eq("PUBLISH"), eq(List.of("RUNNING")),
eq("旗舰店"), eq(start), eq(end), eq(200L));
verify(queryMapper).selectTaskPage(isNull(), eq("PUBLISH"), eq(List.of("RUNNING")),
eq("旗舰店"), eq(start), eq(end), eq(100L), eq(200L));
eq("旗舰店"), eq(start), eq(end), eq(100L), eq(anchorTime), eq(anchorId));
}
/**
* AC-19(审计 P1-B1):不带时间参数的请求由服务层补默认窗口,不许把 null 透传给全历史 UNION。
* 带了 startTime 就完全按请求走——翻旧账的出口不能被默认窗口吃掉。
*/
@Test
@DisplayName("AC-19:不带时间参数补默认窗口,带了就原样下传")
void missingStartTimeGetsDefaultWindowWhileExplicitRangeSurvives() {
when(queryMapper.countTaskPage(any(), any(), any(), any(), any(), any())).thenReturn(0L);
TaskCenterTaskQuery query = new TaskCenterTaskQuery();
service.tasks(superScope(), query);
ArgumentCaptor<LocalDateTime> window = ArgumentCaptor.forClass(LocalDateTime.class);
verify(queryMapper).countTaskPage(isNull(), isNull(), isNull(), isNull(),
window.capture(), isNull());
LocalDateTime floor = window.getValue();
assertFalse(floor.isAfter(LocalDateTime.now()), "默认窗口起点不许在未来:" + floor);
assertTrue(floor.isBefore(LocalDateTime.now().minusDays(30)),
"默认窗口太短会把在跑的长任务切掉:" + floor);
assertTrue(floor.isAfter(LocalDateTime.now().minusDays(180)),
"默认窗口太长就等于没有:" + floor);
LocalDateTime start = LocalDateTime.of(2025, 1, 1, 0, 0);
TaskCenterTaskQuery explicit = new TaskCenterTaskQuery();
explicit.setStartTime(start);
service.tasks(superScope(), explicit);
verify(queryMapper).countTaskPage(isNull(), isNull(), isNull(), isNull(), eq(start), isNull());
}
/** AC-19:第一页不许多跑一次锚点查询——看板每 30s 就落在第一页。 */
@Test
@DisplayName("AC-19:第一页不付锚点查询的钱")
void firstPageSkipsTheAnchorQuery() {
stubPage(List.of(taskRow(1L, "PUBLISH", "SUCCESS", LocalDateTime.now(), LocalDateTime.now())));
service.tasks(superScope(), new TaskCenterTaskQuery());
verify(queryMapper, never()).selectTaskPageAnchor(any(), any(), any(), any(), any(), any(), anyLong());
verify(queryMapper).selectTaskPage(any(), any(), any(), any(), any(), any(),
eq(20L), isNull(), isNull());
}
/** AC-19:页码越过窗口尾部时返回空页,不许拿 null 游标把第一页当第 N 页发回去。 */
@Test
@DisplayName("AC-19:页码越界返回空页而不是回落到第一页")
void pageBeyondWindowReturnsEmptyInsteadOfFirstPage() {
when(queryMapper.countTaskPage(any(), any(), any(), any(), any(), any())).thenReturn(3L);
when(queryMapper.selectTaskPageAnchor(any(), any(), any(), any(), any(), any(), anyLong()))
.thenReturn(null);
TaskCenterTaskQuery query = new TaskCenterTaskQuery();
query.setPage(999L);
TaskCenterTaskPageVo page = service.tasks(superScope(), query);
assertEquals(0L, page.getTotal(), "总数仍是窗口内的 3");
assertTrue(page.getItems() == null || page.getItems().isEmpty(),
"越界页必须空,把第一页塞进第 999 页是静默错数据");
verify(queryMapper, never()).selectTaskPage(any(), any(), any(), any(), any(), any(),
anyLong(), any(), any());
}
/** 前端默认项「在跑」= 三个进行中状态的合并:下传的是状态序列,不是字面量 ACTIVE。 */
@@ -362,7 +436,7 @@ class TaskCenterQueryServiceTest {
service.tasks(superScope(), query);
verify(queryMapper).countTaskPage(isNull(), isNull(),
eq(TaskCenterQueryMapper.ACTIVE_STATUSES_ORDERED), isNull(), isNull(), isNull());
eq(TaskCenterQueryMapper.ACTIVE_STATUSES_ORDERED), isNull(), notNull(), isNull());
assertEquals(Set.copyOf(TaskCenterQueryMapper.ACTIVE_STATUSES_ORDERED),
TaskCenterQueryMapper.ACTIVE_STATUS_SET, "「在跑」的合并口径必须与判卡住用的那组同源");
}
@@ -251,4 +251,64 @@ class UserApiSecretCheckServiceTest {
"{\"data\":{\"list\":[{\"ip\":\"1.2.3.4\",\"port\":8080}]},\"code\":0}")).isFalse();
assertThat(service.isInsufficientBalance("")).isFalse();
}
// ---------- 青果(overseas.proxy.qg.net):code 是字符串、地址在 data[].server ----------
@Test
void extractQingguoProxyUrlUsesServerFieldNotProxyIp() {
// 青果文档明确:server 才是代理地址,proxy_ip 只是真实出口 IP(没端口,拿来连必失败)
String body = "{\"code\":\"SUCCESS\",\"data\":[{\"proxy_ip\":\"123.54.55.24\","
+ "\"server\":\"123.54.55.24:59419\",\"area\":\"河南省商丘市\",\"isp\":\"电信\","
+ "\"deadline\":\"2023-02-25 15:38:36\"}],\"request_id\":\"83158ebe\"}";
assertThat(service.extractProxyUrlFromBody(body)).isEqualTo("http://123.54.55.24:59419");
}
@Test
void extractQingguoProxyUrlNullWhenNoServerAndNoPort() {
assertThat(service.extractProxyUrlFromBody(
"{\"code\":\"SUCCESS\",\"data\":[{\"proxy_ip\":\"123.54.55.24\"}]}")).isNull();
assertThat(service.extractProxyUrlFromBody(
"{\"code\":\"SUCCESS\",\"data\":[]}")).isNull();
}
@Test
void qingguoBalanceShortageIsInsufficientBalance() {
UserApiSecretCheckService.CheckOutcome outcome = service.classifyProviderRejection(
com.nanri.aiimage.modules.usersecret.support.ProxyProvider.QINGGUO,
"{\"code\":\"BALANCE_INSUFFICIENT\",\"message\":\"余额不足\",\"request_id\":\"x\"}", 88);
assertThat(outcome).isNotNull();
assertThat(outcome.status()).isEqualTo(UserApiSecretCheckService.STATUS_FAILED);
assertThat(outcome.code()).isEqualTo(UserApiSecretCheckService.CODE_INSUFFICIENT_BALANCE);
}
@Test
void qingguoAuthDenyIsForbiddenWithChineseMessage() {
UserApiSecretCheckService.CheckOutcome outcome = service.classifyProviderRejection(
com.nanri.aiimage.modules.usersecret.support.ProxyProvider.QINGGUO,
"{\"code\":\"API_AUTH_DENY\",\"message\":\"Api授权不通过,请检查Api鉴权配置\",\"request_id\":\"x\"}", 90);
assertThat(outcome).isNotNull();
assertThat(outcome.status()).isEqualTo(UserApiSecretCheckService.STATUS_FAILED);
assertThat(outcome.code()).isEqualTo(UserApiSecretCheckService.CODE_FORBIDDEN);
assertThat(outcome.message()).contains("青果").contains("API_AUTH_DENY");
}
@Test
void qingguoSuccessIsNotRejection() {
assertThat(service.classifyProviderRejection(
com.nanri.aiimage.modules.usersecret.support.ProxyProvider.QINGGUO,
"{\"code\":\"SUCCESS\",\"data\":[]}", 60)).isNull();
}
@Test
void jikipBalanceShortageStillClassified() {
UserApiSecretCheckService.CheckOutcome outcome = service.classifyProviderRejection(
com.nanri.aiimage.modules.usersecret.support.ProxyProvider.JIKIP,
"{\"code\":-1,\"data\":null,\"message\":\"余额不足\"}", 70);
assertThat(outcome).isNotNull();
assertThat(outcome.code()).isEqualTo(UserApiSecretCheckService.CODE_INSUFFICIENT_BALANCE);
}
}
@@ -11,6 +11,8 @@ import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.ibatis.builder.MapperBuilderAssistant;
import com.nanri.aiimage.common.security.ShopCredentialCryptoService;
import com.nanri.aiimage.config.UserSecretProperties;
import com.nanri.aiimage.config.AppearancePatentProperties;
import com.nanri.aiimage.config.SimilarAsinProperties;
import com.nanri.aiimage.common.security.AdminAuthSupport;
import com.nanri.aiimage.common.service.ReadScope;
import com.nanri.aiimage.modules.notification.service.NotificationDispatchService;
@@ -99,6 +101,7 @@ class UserApiSecretServiceTest {
return new UserApiSecretService(
mapper, crypto, checkService, adminUserMapper, adminAuthSupport,
adminGroupMapper, userDataScopeSupport, notificationDispatchService, userSecretProperties,
mock(AppearancePatentProperties.class), mock(SimilarAsinProperties.class),
stringRedisTemplate, objectMapper);
}
@@ -144,7 +147,7 @@ class UserApiSecretServiceTest {
UserApiSecretService service = newService();
UserApiSecretEntity existing = existingRow("enc:old-key");
when(mapper.selectOne(any())).thenReturn(existing);
when(checkService.probe(any(), anyString())).thenReturn(outcome("passed", "ok", "正常", 1678));
when(checkService.probe(any(), anyString(), any())).thenReturn(outcome("passed", "ok", "正常", 1678));
service.check(7L, "similar-asin", "sk-input");
// 输入值检测只暂存,不落库
@@ -167,7 +170,7 @@ class UserApiSecretServiceTest {
UserApiSecretService service = newService();
UserApiSecretEntity existing = existingRow("enc:old-key");
when(mapper.selectOne(any())).thenReturn(existing);
when(checkService.probe(any(), anyString())).thenReturn(outcome("passed", "ok", "正常", 900));
when(checkService.probe(any(), anyString(), any())).thenReturn(outcome("passed", "ok", "正常", 900));
service.check(7L, "similar-asin", "sk-input");
service.save(7L, "similar-asin", "sk-changed");
@@ -184,7 +187,7 @@ class UserApiSecretServiceTest {
UserApiSecretService service = newService();
UserApiSecretEntity existing = existingRow("enc:old-key");
when(mapper.selectOne(any())).thenReturn(existing);
when(checkService.probe(any(), anyString())).thenReturn(outcome("failed", "invalid_key", "密钥无效", 300));
when(checkService.probe(any(), anyString(), any())).thenReturn(outcome("failed", "invalid_key", "密钥无效", 300));
service.check(7L, "similar-asin", "sk-bad");
service.save(7L, "similar-asin", "sk-bad");
@@ -199,7 +202,7 @@ class UserApiSecretServiceTest {
UserApiSecretService service = newService();
UserApiSecretEntity existing = existingRow("enc:old-key");
when(mapper.selectOne(any())).thenReturn(existing);
when(checkService.probe(any(), anyString())).thenReturn(outcome("passed", "ok", "正常", 500));
when(checkService.probe(any(), anyString(), any())).thenReturn(outcome("passed", "ok", "正常", 500));
when(valueOps.get(anyString())).thenThrow(new IllegalStateException("redis down"));
assertThat(service.check(7L, "similar-asin", "sk-input").getCheckStatus()).isEqualTo("passed");
@@ -214,12 +217,12 @@ class UserApiSecretServiceTest {
@Test
void repeatedCheckWithinFreshWindowReusesResult() {
UserApiSecretService service = newService();
when(checkService.probe(any(), anyString())).thenReturn(outcome("passed", "ok", "代理可用", 120));
when(checkService.probe(any(), anyString(), any())).thenReturn(outcome("passed", "ok", "代理可用", 120));
var first = service.check(7L, "proxy", "https://api.jikip.com/ip-get?num=1&key=abc");
var second = service.check(7L, "proxy", "https://api.jikip.com/ip-get?num=1&key=abc");
verify(checkService, times(1)).probe(any(), anyString());
verify(checkService, times(1)).probe(any(), anyString(), any());
assertThat(second.getCheckStatus()).isEqualTo("passed");
assertThat(second.getCheckStatus()).isEqualTo(first.getCheckStatus());
}
@@ -228,25 +231,25 @@ class UserApiSecretServiceTest {
@Test
void repeatedCheckProbesAgainWhenValueChanged() {
UserApiSecretService service = newService();
when(checkService.probe(any(), anyString())).thenReturn(outcome("passed", "ok", "正常", 100));
when(checkService.probe(any(), anyString(), any())).thenReturn(outcome("passed", "ok", "正常", 100));
service.check(7L, "proxy", "https://api.jikip.com/ip-get?num=1&key=aaa");
service.check(7L, "proxy", "https://api.jikip.com/ip-get?num=1&key=bbb");
verify(checkService, times(2)).probe(any(), anyString());
verify(checkService, times(2)).probe(any(), anyString(), any());
}
/** 失败结果不进新鲜期缓存:余额不足充值后应能立即重检。 */
@Test
void failedCheckIsNotReusedWithinFreshWindow() {
UserApiSecretService service = newService();
when(checkService.probe(any(), anyString()))
when(checkService.probe(any(), anyString(), any()))
.thenReturn(outcome("failed", "insufficient_balance", "代理服务商余额不足,请充值后重试", 80));
service.check(7L, "proxy", "https://api.jikip.com/ip-get?num=1&key=abc");
service.check(7L, "proxy", "https://api.jikip.com/ip-get?num=1&key=abc");
verify(checkService, times(2)).probe(any(), anyString());
verify(checkService, times(2)).probe(any(), anyString(), any());
}
/**
@@ -256,12 +259,12 @@ class UserApiSecretServiceTest {
@Test
void freshCheckIsSharedAcrossAccountsUsingTheSameSecret() {
UserApiSecretService service = newService();
when(checkService.probe(any(), anyString())).thenReturn(outcome("passed", "ok", "代理可用", 120));
when(checkService.probe(any(), anyString(), any())).thenReturn(outcome("passed", "ok", "代理可用", 120));
var first = service.check(7L, "proxy", "https://api.jikip.com/ip-get?num=1&key=abc");
var second = service.check(8L, "proxy", "https://api.jikip.com/ip-get?num=1&key=abc");
verify(checkService, times(1)).probe(any(), anyString());
verify(checkService, times(1)).probe(any(), anyString(), any());
assertThat(second.getCheckStatus()).isEqualTo("passed");
assertThat(second.getCheckStatus()).isEqualTo(first.getCheckStatus());
}
@@ -270,24 +273,24 @@ class UserApiSecretServiceTest {
@Test
void freshCheckDoesNotLeakAcrossDifferentSecretValues() {
UserApiSecretService service = newService();
when(checkService.probe(any(), anyString())).thenReturn(outcome("passed", "ok", "正常", 100));
when(checkService.probe(any(), anyString(), any())).thenReturn(outcome("passed", "ok", "正常", 100));
service.check(7L, "proxy", "https://api.jikip.com/ip-get?num=1&key=aaa");
service.check(8L, "proxy", "https://api.jikip.com/ip-get?num=1&key=bbb");
verify(checkService, times(2)).probe(any(), anyString());
verify(checkService, times(2)).probe(any(), anyString(), any());
}
/** 同一串值在不同模块下结论可能不同(货源/外观各自独立的检测端点),新鲜期按模块隔离。 */
@Test
void freshCheckIsScopedPerModule() {
UserApiSecretService service = newService();
when(checkService.probe(any(), anyString())).thenReturn(outcome("passed", "ok", "连通正常", 100));
when(checkService.probe(any(), anyString(), any())).thenReturn(outcome("passed", "ok", "连通正常", 100));
service.check(7L, "similar-asin", "sk-shared");
service.check(8L, "appearance-patent", "sk-shared");
verify(checkService, times(2)).probe(any(), anyString());
verify(checkService, times(2)).probe(any(), anyString(), any());
}
private UserApiSecretEntity existingRow(String cipher) {
@@ -980,7 +983,7 @@ class UserApiSecretServiceTest {
11L, storedRow(11L, "proxy", sharedCipher),
12L, storedRow(12L, "proxy", sharedCipher),
13L, storedRow(13L, "proxy", sharedCipher)));
when(checkService.probe(any(), anyString())).thenReturn(outcome("passed", "ok", "代理可用", 120));
when(checkService.probe(any(), anyString(), any())).thenReturn(outcome("passed", "ok", "代理可用", 120));
when(mapper.selectList(any())).thenReturn(List.of(
inspectionRow(1L, 11L, "proxy", sharedCipher),
inspectionRow(2L, 12L, "proxy", sharedCipher),
@@ -988,7 +991,7 @@ class UserApiSecretServiceTest {
var summary = service.checkAllForScheduler(500, 20);
verify(checkService, times(1)).probe(any(), anyString());
verify(checkService, times(1)).probe(any(), anyString(), any());
assertThat(summary.checked()).isEqualTo(3);
assertThat(summary.probed()).isEqualTo(1);
assertThat(inspectionWrites.keySet()).containsExactlyInAnyOrder(11L, 12L, 13L);
@@ -1003,7 +1006,7 @@ class UserApiSecretServiceTest {
stubInspectionStore(Map.of(
11L, storedRow(11L, "proxy", sharedCipher),
12L, storedRow(12L, "proxy", sharedCipher)));
when(checkService.probe(any(), anyString())).thenReturn(
when(checkService.probe(any(), anyString(), any())).thenReturn(
outcome("failed", "insufficient_balance", "代理服务商余额不足,请充值后重试", 80));
when(mapper.selectList(any())).thenReturn(List.of(
inspectionRow(1L, 11L, "proxy", sharedCipher),
@@ -1011,7 +1014,7 @@ class UserApiSecretServiceTest {
var summary = service.checkAllForScheduler(500, 20);
verify(checkService, times(1)).probe(any(), anyString());
verify(checkService, times(1)).probe(any(), anyString(), any());
assertThat(summary.probed()).isEqualTo(1);
assertThat(summary.failed()).isEqualTo(2);
assertThat(inspectionWrites.values()).containsOnly("failed");
@@ -1025,14 +1028,14 @@ class UserApiSecretServiceTest {
stubInspectionStore(Map.of(
11L, storedRow(11L, "similar-asin", sharedCipher),
12L, storedRow(12L, "appearance-patent", sharedCipher)));
when(checkService.probe(any(), anyString())).thenReturn(outcome("passed", "ok", "连通正常", 100));
when(checkService.probe(any(), anyString(), any())).thenReturn(outcome("passed", "ok", "连通正常", 100));
when(mapper.selectList(any())).thenReturn(List.of(
inspectionRow(1L, 11L, "similar-asin", sharedCipher),
inspectionRow(2L, 12L, "appearance-patent", sharedCipher)), List.of());
var summary = service.checkAllForScheduler(500, 20);
verify(checkService, times(2)).probe(any(), anyString());
verify(checkService, times(2)).probe(any(), anyString(), any());
assertThat(summary.checked()).isEqualTo(2);
assertThat(summary.probed()).isEqualTo(2);
}
@@ -1049,7 +1052,7 @@ class UserApiSecretServiceTest {
11L, storedRow(11L, "proxy", sharedCipher),
12L, storedRow(12L, "proxy", sharedCipher),
13L, storedRow(13L, "proxy", sharedCipher)));
when(checkService.probe(any(), anyString())).thenReturn(
when(checkService.probe(any(), anyString(), any())).thenReturn(
outcome("failed", "insufficient_balance", "代理服务商余额不足,请充值后重试", 80));
when(mapper.selectList(any())).thenReturn(List.of(
inspectionRow(1L, 11L, "proxy", sharedCipher),
@@ -1081,7 +1084,7 @@ class UserApiSecretServiceTest {
UserApiSecretService service = newService();
String cipher = "enc:sk-solo";
stubInspectionStore(Map.of(11L, storedRow(11L, "similar-asin", cipher)));
when(checkService.probe(any(), anyString())).thenReturn(
when(checkService.probe(any(), anyString(), any())).thenReturn(
outcome("failed", "invalid_key", "密钥无效或已被拒绝", 90));
when(mapper.selectList(any())).thenReturn(List.of(
inspectionRow(1L, 11L, "similar-asin", cipher)), List.of());
@@ -0,0 +1,129 @@
package com.nanri.aiimage.modules.usersecret.support;
import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.config.AppearancePatentProperties;
import com.nanri.aiimage.config.SimilarAsinProperties;
import org.junit.jupiter.api.Test;
import java.util.LinkedHashMap;
import java.util.Map;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
/**
* 密钥「货源」维度:LLM 渠道(龙抬头/贞贞)与代理供应商(青果/极客)的归一化、默认值、主机与模型映射。
*
* <p>口径来源(2026-10-29 用户拍板):新建默认龙抬头 + 青果;存量由迁移 V180 回填贞贞/极客;
* 龙抬头的模型 id 统一用 {@code gemini-3.1-flash-lite}。</p>
*/
class LlmChannelProxyProviderTest {
@Test
void llmChannelDefaultsToT8star() {
// 2026-10-30 用户拍板:默认仍是贞贞,龙抬头只做手动切换
assertThat(LlmChannel.of(null)).isEqualTo(LlmChannel.T8STAR);
assertThat(LlmChannel.of(" ")).isEqualTo(LlmChannel.T8STAR);
assertThat(LlmChannel.of("unknown-channel")).isEqualTo(LlmChannel.T8STAR);
assertThat(LlmChannel.of(" LONGTAITOU ")).isEqualTo(LlmChannel.LONGTAITOU);
}
@Test
void llmChannelParseRejectsUnknownWithoutFallback() {
assertThat(LlmChannel.parse("t8star")).isEqualTo(LlmChannel.T8STAR);
assertThat(LlmChannel.parse("nope")).isNull();
assertThat(LlmChannel.parse("")).isNull();
}
@Test
void longtaitouUsesItsOwnModelId() {
// 贞贞的模型名在龙抬头那边不存在,切渠道必须换 id,否则 model_not_found
assertThat(LlmChannel.LONGTAITOU.modelFor("gemini-3.1-flash-lite-preview-thinking-medium"))
.isEqualTo("gemini-3.1-flash-lite");
// 贞贞保持配置里的模型名原样
assertThat(LlmChannel.T8STAR.modelFor("gemini-3.1-flash-lite-preview-thinking-medium"))
.isEqualTo("gemini-3.1-flash-lite-preview-thinking-medium");
}
@Test
void modelMappingFollowsRequestHost() {
assertThat(LlmChannel.modelForHost("https://longtaitou.token6688.com", "any-model"))
.isEqualTo("gemini-3.1-flash-lite");
assertThat(LlmChannel.modelForHost("https://ai.t8star.org", "any-model")).isEqualTo("any-model");
assertThat(LlmChannel.modelForHost(null, "any-model")).isEqualTo("any-model");
assertThat(LlmChannel.modelForHost("https://unknown.example.com", "any-model")).isEqualTo("any-model");
}
@Test
void proxyProviderDefaultsToQingguoAndDetectsByUrl() {
assertThat(ProxyProvider.of(null)).isEqualTo(ProxyProvider.JIKIP);
assertThat(ProxyProvider.of("jikip")).isEqualTo(ProxyProvider.JIKIP);
assertThat(ProxyProvider.detectFromUrl(
"https://overseas.proxy.qg.net/get?key=abc&num=1&format=json")).isEqualTo(ProxyProvider.QINGGUO);
assertThat(ProxyProvider.detectFromUrl("https://api.jikip.com/ip-get?num=1&minute=3&key=abc"))
.isEqualTo(ProxyProvider.JIKIP);
assertThat(ProxyProvider.detectFromUrl("https://example.com/get")).isNull();
assertThat(ProxyProvider.detectFromUrl(null)).isNull();
}
@Test
void longtaitouNeedsExplicitThinkingParamsLikeT8star() {
// 贞贞把 thinking 写在模型名里(…-thinking-medium),请求体不用额外字段
Map<String, Object> t8 = new LinkedHashMap<>();
LlmChannel.applyThinkingForHost("https://ai.t8star.org", t8);
assertThat(t8).isEmpty();
// 龙抬头的同名模型默认不思考,业务链路要显式补上,才等于现网口径
Map<String, Object> ltt = new LinkedHashMap<>();
LlmChannel.applyThinkingForHost("https://longtaitou.token6688.com", ltt);
assertThat(ltt).containsEntry("enable_thinking", true);
assertThat(ltt).containsEntry("thinking_budget", "medium");
// 未知主机不动请求体(自建中转/本地假上游不要被塞参数)
Map<String, Object> unknown = new LinkedHashMap<>();
LlmChannel.applyThinkingForHost("http://127.0.0.1:8080", unknown);
assertThat(unknown).isEmpty();
}
@Test
void moduleDefaultsAndValidation() {
assertThat(UserSecretModule.APPEARANCE_PATENT.defaultChannel()).isEqualTo("t8star");
assertThat(UserSecretModule.SIMILAR_ASIN.defaultChannel()).isEqualTo("t8star");
assertThat(UserSecretModule.PROXY.defaultChannel()).isEqualTo("jikip");
assertThat(UserSecretModule.APPEARANCE_PATENT.requireChannel(null)).isEqualTo("t8star");
assertThat(UserSecretModule.APPEARANCE_PATENT.requireChannel("T8STAR")).isEqualTo("t8star");
assertThat(UserSecretModule.PROXY.requireChannel("jikip")).isEqualTo("jikip");
assertThatThrownBy(() -> UserSecretModule.APPEARANCE_PATENT.requireChannel("qingguo"))
.isInstanceOf(BusinessException.class)
.hasMessageContaining("未知的 LLM 渠道");
assertThatThrownBy(() -> UserSecretModule.PROXY.requireChannel("longtaitou"))
.isInstanceOf(BusinessException.class)
.hasMessageContaining("未知的代理供应商");
}
@Test
void proxyChannelFallsBackToUrlDetectionForLegacyRows() {
// 老数据只有链接、没有供应商列:按域名识别,别把极客链接当青果解析
assertThat(UserSecretModule.PROXY.channelOrDefault(null, "https://api.jikip.com/ip-get?key=x"))
.isEqualTo("jikip");
assertThat(UserSecretModule.PROXY.channelOrDefault(null, "https://overseas.proxy.qg.net/get?key=x"))
.isEqualTo("qingguo");
assertThat(UserSecretModule.PROXY.channelOrDefault(null, "https://example.com/get"))
.isEqualTo("jikip");
}
@Test
void llmHostResolvesByChannelAndFallsBackToModuleConfig() {
AppearancePatentProperties appearance = new AppearancePatentProperties();
SimilarAsinProperties similar = new SimilarAsinProperties();
assertThat(UserSecretModule.APPEARANCE_PATENT.resolveLlmHost(appearance, similar, "longtaitou"))
.isEqualTo("https://longtaitou.token6688.com");
assertThat(UserSecretModule.APPEARANCE_PATENT.resolveLlmHost(appearance, similar, "t8star"))
.isEqualTo("https://ai.t8star.org");
// 迁移前遗留行没有 channel:沿用模块全局配置,行为不变
assertThat(UserSecretModule.APPEARANCE_PATENT.resolveLlmHost(appearance, similar, null))
.isEqualTo(appearance.getLlmHost());
}
}
@@ -4,6 +4,7 @@ import com.baomidou.mybatisplus.core.MybatisConfiguration;
import com.baomidou.mybatisplus.core.conditions.AbstractWrapper;
import com.baomidou.mybatisplus.core.metadata.TableInfoHelper;
import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.modules.file.service.ChunkedUploadService;
import com.nanri.aiimage.modules.file.service.oss.OssStorageService;
import com.nanri.aiimage.modules.workbenchclient.mapper.WorkbenchClientPackageMapper;
import com.nanri.aiimage.modules.workbenchclient.model.entity.WorkbenchClientPackageEntity;
@@ -61,12 +62,13 @@ class WorkbenchClientPackageServiceTest {
private final WorkbenchClientPackageMapper packageMapper = mock(WorkbenchClientPackageMapper.class);
private final OssStorageService ossStorageService = mock(OssStorageService.class);
private final ChunkedUploadService chunkedUploadService = mock(ChunkedUploadService.class);
@TempDir
Path tempDir;
private WorkbenchClientPackageService service() {
return new WorkbenchClientPackageService(packageMapper, ossStorageService);
return new WorkbenchClientPackageService(packageMapper, ossStorageService, chunkedUploadService);
}
/** 每次调用返回一个新的合法 zip:confirm 扫描后会把临时文件删掉,不能复用同一个路径。 */
@@ -67,7 +67,6 @@ TaskCenterQueryMapper selectShopNames Select CONST <script>SELECT DISTINCT s.sho
TaskCenterQueryMapper selectSnapshots Select PURE <script>SELECT s.task_id AS taskId, s.module_type AS moduleType, s.total_count AS totalCount, s.success_count AS successCount, s.failed_count AS failedCount, s.pending_count AS pendingCount, s.current_scope_key AS currentScopeKey, s.message AS message FROM biz_task_progress_snapshot s WHERE s.task_id IN <foreach collection='taskIds' item='tid' open='(' separator=',' close=')'>#{tid}</foreach></script>
TaskCenterQueryMapper selectTaskByTaskNo Select CONST SELECT t.id AS id, t.task_no AS taskNo, UPPER(t.module_type) AS moduleType, CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END AS status, t.user_id AS userId, t.device_id AS deviceId, t.created_at AS createdAt, t.finished_at AS finishedAt, t.updated_at AS updatedAt, t.source_file_count AS totalCount, t.success_file_count AS successCount, t.failed_file_count AS failedCount, 0 AS brand, t.error_message AS errorMessage FROM biz_file_task t WHERE t.task_no = #{taskNo} AND t.module_type IN ('PUBLISH','DELETE_BRAND','PRODUCT_RISK_RESOLVE','PRICE_TRACK','SHOP_MATCH','PATROL_DELETE','QUERY_ASIN','WITHDRAW','APPEARANCE_PATENT','SIMILAR_ASIN','COLLECT_DATA','SHOP_DATA_CRAWL') LIMIT 1
TaskCenterQueryMapper selectTaskCountries Select CONST <script>SELECT t.id AS taskId, COALESCE(NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.filters.countryCode')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.countryCode')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.country')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.publish_country')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.countryCodes')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.country_codes')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.filters.countryCodes')), ''), 'null')) AS country FROM biz_file_task t WHERE t.id IN <foreach collection='taskIds' item='tid' open='(' separator=',' close=')'>#{tid}</foreach></script>
TaskCenterQueryMapper selectTaskPage Select CONST <script>SELECT m.* FROM (SELECT t.id AS id, t.task_no AS taskNo, UPPER(t.module_type) AS moduleType, CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END AS status, t.user_id AS userId, t.device_id AS deviceId, t.created_at AS createdAt, t.finished_at AS finishedAt, t.updated_at AS updatedAt, t.source_file_count AS totalCount, t.success_file_count AS successCount, t.failed_file_count AS failedCount, 0 AS brand FROM biz_file_task t WHERE t.module_type IN ('PUBLISH','DELETE_BRAND','PRODUCT_RISK_RESOLVE','PRICE_TRACK','SHOP_MATCH','PATROL_DELETE','QUERY_ASIN','WITHDRAW','APPEARANCE_PATENT','SIMILAR_ASIN','COLLECT_DATA','SHOP_DATA_CRAWL')<if test='userIds != null'> AND t.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND UPPER(t.module_type) = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND t.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND t.created_at &lt;= #{endTime}</if><if test='shop != null'> AND (EXISTS (SELECT 1 FROM biz_publish_file p WHERE p.task_id = t.id AND LOWER(TRIM(p.shop_name)) = LOWER(TRIM(#{shop}))) OR (t.module_type IN ('PATROL_DELETE','PRICE_TRACK','PRODUCT_RISK_RESOLVE','QUERY_ASIN','SHOP_DATA_CRAWL','SHOP_MATCH') AND EXISTS (SELECT 1 FROM biz_file_result f WHERE f.task_id = t.id AND LOWER(TRIM(f.source_filename)) = LOWER(TRIM(#{shop})))))</if> UNION ALL SELECT b.id, CONCAT('brand-', b.id), 'BRAND', CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END, b.user_id, b.device_id, b.created_at, CASE WHEN b.status IN ('cancelled','failed','success') THEN b.updated_at END, b.updated_at, b.progress_total, b.progress_current, 0, 1 FROM brand_crawl_tasks b WHERE 1 = 1<if test='userIds != null'> AND b.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND 'BRAND' = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND b.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND b.created_at &lt;= #{endTime}</if><if test='shop != null'> AND 1 = 0</if>) m ORDER BY m.createdAt DESC, m.id DESC LIMIT #{limit} OFFSET #{offset}</script>
TaskCenterQueryMapper selectUserIdsByUsername Select PURE SELECT id FROM users WHERE username LIKE CONCAT('%', #{keyword}, '%') LIMIT 500
TaskCenterQueryMapper selectUsernames Select PURE <script>SELECT id AS userId, username FROM users WHERE id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></script>
TaskResultItemMapper upsertBatch Insert PURE <script>\nINSERT INTO biz_task_result_item\n (task_id, module_type, result_id, scope_key, scope_hash, item_key, asin, status,\n payload_json, payload_hash, created_at, updated_at)\nVALUES\n<foreach collection="rows" item="row" separator=",">\n (#{row.taskId}, #{row.moduleType}, #{row.resultId}, #{row.scopeKey}, #{row.scopeHash},\n #{row.itemKey}, #{row.asin}, #{row.status}, #{row.payloadJson}, #{row.payloadHash},\n #{row.createdAt}, #{row.updatedAt})\n</foreach>\nON DUPLICATE KEY UPDATE\n result_id = VALUES(result_id),\n scope_key = VALUES(scope_key),\n asin = VALUES(asin),\n status = VALUES(status),\n payload_json = VALUES(payload_json),\n payload_hash = VALUES(payload_hash),\n updated_at = VALUES(updated_at)\n</script>
Can't render this file because it contains an unexpected character in line 1 and column 222.
@@ -22,10 +22,14 @@
{{ parsing ? '解析中...' : '解析并创建任务' }}
</button>
<button type="button" class="btn-run btn-queue" :disabled="pushing || !parseResult?.taskId" @click="pushToPythonQueue">
{{ pushing ? '提交中...' : '提交后台执行' }}
{{ pushing ? (localCollectEnabled ? '启动中...' : '提交中...') : (localCollectEnabled ? '启动任务' : '提交后台执行') }}
</button>
</div>
<p class="loading-msg">提交后系统自动分批检测,进度显示在右侧任务区,完成后可下载结果。</p>
<p class="loading-msg">
{{ localCollectEnabled
? '启动后任务推送到本机客户端采集,进度显示在右侧任务区,完成后可下载结果。'
: '提交后系统自动分批检测,进度显示在右侧任务区,完成后可下载结果。' }}
</p>
<div v-if="visibleTaskSummary" class="parse-card">
<div>任务 ID:{{ visibleTaskSummary.taskId }}</div>
@@ -80,17 +84,21 @@ import ModuleTemplateDownload from '@/shared/components/ModuleTemplateDownload.v
import TaskCenterPanel from '@/shared/components/tasks/TaskCenterPanel.vue'
import type { TaskItemView, TaskStatCard } from '@/shared/components/tasks/types'
import {
activateAppearancePatentTask,
deleteAppearancePatentHistory,
deleteAppearancePatentTask,
getAppearancePatentDashboard,
getAppearancePatentHistory,
getAppearancePatentQueuePayload,
getAppearancePatentResultDownloadUrl,
getAppearancePatentTaskProgressBatch,
parseAppearancePatent,
type AppearancePatentHistoryItem,
type AppearancePatentParseVo,
type AppearancePatentParsedPayloadDto,
type UploadedFileRef,
} from '@/shared/api/java-modules'
import { getCollectRouting } from '@/shared/api/types/modules/collect-routing'
import { getPywebviewApi } from '@/shared/bridges/pywebview'
import { useExcelFileUpload } from '@/shared/composables/useExcelFileUpload'
import {
@@ -101,7 +109,7 @@ import {
} from '@/shared/composables/useLiveTaskBoard'
import { getModuleProgressLight } from '@/shared/api/progress-light.ts'
import { createRowsVersionTracker } from '@/shared/api/task-progress-polling.ts'
import { checkParseResult } from '@/shared/dispatch-guard'
import { checkParseResult, checkQueuePayload } from '@/shared/dispatch-guard'
import { passGuard } from '@/shared/dispatch-guard-ui'
import { formatDateTime } from '@/shared/utils/datetime'
import { resultBuildFailed, taskStatusClass, taskStatusText } from '@/shared/utils/task-status-text'
@@ -135,6 +143,12 @@ const queuedTaskSummary = ref<TaskSummary | null>(null)
const aiPrompt = ref('')
const parsing = ref(false)
const pushing = ref(false)
/**
* 本地采集开关(服务端 aiimage.collect-routing.local-enabled,默认开)。
* 先按「本地」渲染,取不到配置也不改口:真正兜底在服务端——开关关着时 activate 会明确拒绝,
* 页面最多多报一次错,不会把任务推给不该跑本地的机器。
*/
const localCollectEnabled = ref(true)
/** 结果行版本跟踪(审查 F5 后续):版本未变的任务跳过行明细拉取 */
const rowsVersionTracker = createRowsVersionTracker()
@@ -459,17 +473,97 @@ async function pushToPythonQueue() {
ElMessage.warning('请先解析文件')
return
}
// 2026-09-30 外观专利已迁移 Linux 后台执行(无浏览器抓取):
// 「启动」不再推送客户端队列,任务保持 PENDING,由服务端内部认领
// (/api/internal/tasks/pull-pending + 主机B maixiang-ap worker)在等待窗口后自动接管。
if (!localCollectEnabled.value) {
// 本地采集关闭:只建单,任务保持 PENDING 由 Linux 后台认领
// (/api/internal/tasks/pull-pending + 主机B maixiang-ap worker)在等待窗口后自动接管。
pushing.value = true
try {
queuedTaskSummary.value = toTaskSummary(currentParseResult)
board.addPollingTask(taskId)
clearParsedTask()
await board.loadDashboard()
await board.loadHistory({ force: true })
ElMessage.success('已提交,后台自动执行中(无需打开浏览器)')
} finally {
pushing.value = false
}
return
}
const api = getPywebviewApi()
if (!api?.enqueue_json) {
ElMessage.error('当前环境未启用 pywebview enqueue_json,请在本地客户端中打开本页后再启动')
return
}
if (!effectiveLlmApiKey()) {
// 本地无明文不再阻断:服务端已保存该用户密钥,任务执行时按 uid 兜底读取
console.log('[appearance-patent] 本地无密钥明文,启动任务由服务端兜底')
}
pushing.value = true
try {
const parsedPayload: AppearancePatentParsedPayloadDto = await getAppearancePatentQueuePayload(taskId)
const groups = parsedPayload.groups || []
const payload = {
type: 'appearance-patent-run',
ts: Date.now(),
data: {
taskId,
prompt: aiPrompt.value.trim(),
api_key: effectiveLlmApiKey(),
sourceFileCount: currentParseResult.sourceFileCount || 0,
totalRows: currentParseResult.totalRows || 0,
acceptedRows: currentParseResult.acceptedRows || 0,
groupCount: currentParseResult.groupCount || 0,
groups: groups.map((group) => ({
sourceFileKey: group.sourceFileKey || '',
sourceFilename: group.sourceFilename || '',
groupKey: group.groupKey || '',
baseId: group.baseId || '',
displayId: group.displayId || '',
items: (group.items || []).map((row) => ({
sourceFileKey: row.sourceFileKey || '',
sourceFilename: row.sourceFilename || '',
rowToken: row.rowToken || '',
groupKey: row.groupKey || '',
id: row.displayId,
asin: row.asin,
country: row.country,
sku: row.sku || '',
url: row.url || '',
title: row.title || '',
})),
})),
},
}
// 后端返回的 groups 为空时推送出去,Python 端没有可执行批次会一直停在执行中
const guard = checkQueuePayload(payload, {
expectedType: 'appearance-patent-run',
// api_key 不列为必填:密钥已服务端化,本地无明文时由服务端按 uid 兜底
requiredDataKeys: ['taskId'],
nonEmptyArrayKeys: ['groups'],
})
if (!(await passGuard(guard))) return
// 任务可能已被客户端「兜底拉取」/Linux 兜底领走:服务端会拒绝 activate,
// 此时必须直接返回,不能再入队,否则同一任务会被执行两次
try {
await activateAppearancePatentTask(taskId)
} catch (activateError) {
ElMessage.warning(activateError instanceof Error ? activateError.message : '任务已在执行中,无需重复启动')
await Promise.all([board.loadDashboard(), board.loadHistory({ force: true })])
return
}
const result = await api.enqueue_json(payload)
if (!result?.success) {
ElMessage.error(userNotice(result?.error, '启动失败'))
return
}
queuedTaskSummary.value = toTaskSummary(currentParseResult)
board.addPollingTask(taskId)
clearParsedTask()
await board.loadDashboard()
await board.loadHistory({ force: true })
ElMessage.success('已提交,后台自动执行中(无需打开浏览器)')
ElMessage.success('已启动任务')
} catch (e) {
ElMessage.error(e instanceof Error ? e.message : '启动失败')
} finally {
pushing.value = false
}
@@ -483,6 +577,14 @@ const { batchDeleteHistory, deleteTaskRecord, downloadResult } = board
onMounted(async () => {
board.loadPollingIds()
// 本地采集开关:读不到就按「本地」渲染,服务端 activate 会兜底拒绝,不至于误推给后台
getCollectRouting()
.then((routing) => {
localCollectEnabled.value = routing.appearancePatentLocal !== false
})
.catch((error) => {
console.warn('[appearance-patent] 读取本地采集开关失败,按本地采集处理', error)
})
await Promise.all([
board.loadDashboard().catch(() => undefined),
board.loadHistory().catch(() => undefined),
@@ -60,10 +60,14 @@
</button>
<button type="button" class="btn-run btn-queue" :disabled="pushing || !parseResult?.taskId"
@click="pushToPythonQueue">
{{ pushing ? '启动中...' : '启动任务' }}
{{ pushing ? (localCollectEnabled ? '启动中...' : '提交中...') : (localCollectEnabled ? '启动任务' : '提交后台执行') }}
</button>
</div>
<p class="loading-msg">提交后系统自动分批检测,进度显示在右侧任务区,完成后可下载结果。</p>
<p class="loading-msg">
{{ localCollectEnabled
? '提交后系统自动分批检测,进度显示在右侧任务区,完成后可下载结果。'
: '提交后由后台自动分批检测,进度显示在右侧任务区,完成后可下载结果。' }}
</p>
<div v-if="visibleTaskSummary" class="parse-card">
<div>任务 ID:{{ visibleTaskSummary.taskId }}</div>
@@ -146,6 +150,7 @@ import {
checkQueuePayload,
} from '@/shared/dispatch-guard'
import { passGuard } from '@/shared/dispatch-guard-ui'
import { getCollectRouting } from '@/shared/api/types/modules/collect-routing'
import { formatDateTime } from '@/shared/utils/datetime'
import { isCompletedTask, resultBuildFailed, taskStatusClass, taskStatusText } from '@/shared/utils/task-status-text'
import { getStoredApiSecret } from '@/shared/utils/api-secret-store'
@@ -182,6 +187,11 @@ type TaskSummary = Pick<SimilarAsinParseVo, 'taskId' | 'totalRows' | 'acceptedRo
const queuedTaskSummary = ref<TaskSummary | null>(null)
const parsing = ref(false)
const pushing = ref(false)
/**
* 本地采集开关(服务端 aiimage.collect-routing.local-enabled,默认开)。
* 先按「本地」渲染,取不到配置也不改口:服务端 activate 在开关关着时会明确拒绝。
*/
const localCollectEnabled = ref(true)
const imgSwitch = ref(false)
const categorySwitch = ref(false)
const alipriceUsername = ref('')
@@ -522,6 +532,21 @@ async function pushToPythonQueue() {
ElMessage.warning('请先解析文件')
return
}
if (!localCollectEnabled.value) {
// 本地采集关闭:只建单,任务保持 PENDING 由 Linux 后台认领
pushing.value = true
try {
queuedTaskSummary.value = toTaskSummary(currentParseResult)
board.addPollingTask(taskId)
clearParsedTask()
await board.loadDashboard()
await board.loadHistory({ force: true })
ElMessage.success('已提交,后台自动执行中')
} finally {
pushing.value = false
}
return
}
if (!api?.enqueue_json) {
ElMessage.error('当前环境未启用 pywebview enqueue_json')
return
@@ -657,6 +682,14 @@ onMounted(async () => {
}
window.addEventListener('pywebviewready', pywebviewReadyHandler)
}
// 本地采集开关:读不到就按「本地」渲染,服务端 activate 会兜底拒绝,不至于误推给后台
getCollectRouting()
.then((routing) => {
localCollectEnabled.value = routing.similarAsinLocal !== false
})
.catch((error) => {
console.warn('[similar-asin] 读取本地采集开关失败,按本地采集处理', error)
})
await Promise.all([
board.loadDashboard().catch(() => undefined),
board.loadHistory().catch(() => undefined),
@@ -815,6 +815,30 @@ function sleep(milliseconds: number) {
return new Promise<void>((resolve) => window.setTimeout(resolve, milliseconds))
}
/**
* 隐藏标签页暂停轮询(P1-F4):页面不可见时不再每 5 秒打一次任务接口,
* 等 visibilitychange 回前台再放行。回到前台后第一次等待照常执行,不改变终态判定。
*/
function waitWhileHidden(): Promise<void> {
if (typeof document === 'undefined' || document.visibilityState !== 'hidden') {
return Promise.resolve()
}
return new Promise<void>((resolve) => {
const onVisible = () => {
if (document.visibilityState !== 'hidden') {
document.removeEventListener('visibilitychange', onVisible)
resolve()
}
}
document.addEventListener('visibilitychange', onVisible)
})
}
async function sleepRespectingVisibility(milliseconds: number) {
await waitWhileHidden()
await sleep(milliseconds)
}
async function waitForImageVideoTask(ticket: ImageVideoAsyncTaskVo): Promise<unknown> {
let task = ticket
for (let pollCount = 0; pollCount < IMAGE_VIDEO_TASK_MAX_POLLS; pollCount += 1) {
@@ -826,7 +850,7 @@ async function waitForImageVideoTask(ticket: ImageVideoAsyncTaskVo): Promise<unk
if (isTerminalImageVideoTask(task)) {
throw new Error(task.errorMessage || 'Coze 任务执行失败')
}
await sleep(IMAGE_VIDEO_TASK_POLL_DELAY_MS)
await sleepRespectingVisibility(IMAGE_VIDEO_TASK_POLL_DELAY_MS)
task = await getImageVideoAsyncTask(task.taskId)
}
throw new Error('Coze 任务查询超时,请稍后重试')
@@ -1133,7 +1157,13 @@ async function pollAssemblyResult(tab: WorkspaceTab, taskId: number) {
assembly.polling = false
return
}
assembly.pollTimer = window.setTimeout(() => {
assembly.pollTimer = window.setTimeout(async () => {
// P1-F4:结果轮询(约 5s/轮)在隐藏标签页里暂停,回前台再续链
await waitWhileHidden()
if (disposed) {
assembly.polling = false
return
}
void pollAssemblyResult(tab, taskId)
}, IMAGE_VIDEO_TASK_POLL_DELAY_MS)
}
+6
View File
@@ -190,14 +190,20 @@ export const API_ENDPOINTS = {
},
appearancePatent: {
parse: '/api/appearance-patent/parse',
queuePayload: '/api/appearance-patent/tasks/{taskId}/queue-payload',
dashboard: '/api/appearance-patent/dashboard',
history: '/api/appearance-patent/history',
historyDelete: '/api/appearance-patent/history/{resultId}',
progressBatch: '/api/appearance-patent/tasks/progress/batch',
progressLight: '/api/appearance-patent/tasks/progress/light',
taskActivate: '/api/appearance-patent/tasks/{taskId}/activate',
taskDelete: '/api/appearance-patent/tasks/{taskId}',
resultDownload: '/api/appearance-patent/results/{resultId}/download',
},
/** 专利/货源的本地采集开关(服务端唯一真源,前端只读)。 */
collectRouting: {
get: '/api/collect-routing',
},
userSecret: {
bundle: '/api/user-secrets',
save: '/api/user-secrets/{moduleKey}',
+29 -5
View File
@@ -83,12 +83,36 @@ export function onResponseFulfilled(response: AxiosResponse): AxiosResponse {
/** 响应失败分支:折叠为 Error 之前先判下线(互踢与登录态失效;后者也可能真以 HTTP 401 回来)。 */
export function onResponseRejected(error: unknown): Promise<never> {
const payload = responsePayloadOf(error)
if (isKickedPayload(payload)) {
handleKicked()
} else if (isUnauthorizedPayload(payload) || responseStatusOf(error) === 401) {
handleSessionExpired(authReasonOf(responseHeadersOf(error)))
const status = responseStatusOf(error)
// S1-F1:把"这是不是鉴权失败"随错误带出去。弱网/5xx 与 401 过去在调用方长得一模一样,
// ensureAuth 那条旁路因此逢错就清令牌;现在清态只认这个标注(HTTP 200+code=401 也算)。
const authFailure = isKickedPayload(payload) || isUnauthorizedPayload(payload) || status === 401
if (authFailure) {
if (isKickedPayload(payload)) {
handleKicked()
} else {
handleSessionExpired(authReasonOf(responseHeadersOf(error)))
}
}
return Promise.reject(new Error(extractErrorMessage(error)))
const friendly = extractErrorMessage(error)
// 纯网络层错误(没有任何响应:断网/DNS/超时)保留底层技术文案——它不是后端返回的英文,
// 不受"只透中文"闸门约束;调用方据此区分"网络断了"与"登录失效",据此决定重试还是登出。
const noResponse = status === undefined && payload === undefined
const rawMessage = (error as { message?: unknown })?.message
const message = (!authFailure && noResponse && typeof rawMessage === 'string' && rawMessage)
? rawMessage
: friendly
const rejected = new Error(message) as Error & { isAuthError?: boolean }
rejected.isAuthError = authFailure
return Promise.reject(rejected)
}
/**
* 这次请求失败是否为鉴权类(401/4011/互踢),由 onResponseRejected 标注。
* 调用方据此决定要不要清登录态——网络抖动与 5xx 一律不是。
*/
export function isAuthError(error: unknown): boolean {
return (error as { isAuthError?: boolean } | null | undefined)?.isAuthError === true
}
function createHttpClient(): AxiosInstance {
@@ -33,6 +33,16 @@ export interface AppearancePatentParsedGroup {
items: AppearancePatentParsedRow[];
}
export interface AppearancePatentParsedPayloadDto {
aiPrompt?: string;
apiKey?: string;
sourceFiles?: UploadedFileRef[];
headers?: string[];
items?: AppearancePatentParsedRow[];
groups?: AppearancePatentParsedGroup[];
allItems?: AppearancePatentParsedRow[];
}
export interface AppearancePatentParseVo {
taskId: number;
sourceFilename?: string;
@@ -180,8 +190,30 @@ export function parseAppearancePatent(files: UploadedFileRef[], aiPrompt: string
);
}
export function getAppearancePatentDashboard() {
/**
* 本地采集模式下推本机客户端队列的载荷:只含 Python 当前需要的分组,
* 由 /queue-payload 按 taskId 现取,避免前端搬运完整解析大数据。
*/
export function getAppearancePatentQueuePayload(taskId: number) {
return unwrapJavaResponse(
get<JavaApiResponse<AppearancePatentParsedPayloadDto>>(
buildJavaUrl(API_ENDPOINTS.appearancePatent.queuePayload.replace('{taskId}', String(taskId))),
{ params: { user_id: getCurrentUserId() } },
),
);
}
/** 本地采集:推送客户端队列前把任务从 PENDING 翻成 RUNNING(与后台认领互斥)。 */
export function activateAppearancePatentTask(taskId: number) {
return unwrapJavaResponse(
post<JavaApiResponse<null>, undefined>(
`${JAVA_API_PREFIX}/appearance-patent/tasks/${taskId}/activate?user_id=${encodeURIComponent(String(getCurrentUserId()))}`,
undefined,
),
);
}
export function getAppearancePatentDashboard() { return unwrapJavaResponse(
get<JavaApiResponse<AppearancePatentDashboardVo>>(
buildJavaUrl(API_ENDPOINTS.appearancePatent.dashboard),
{ params: { user_id: getCurrentUserId() } },
@@ -0,0 +1,21 @@
import { get, type JavaApiResponse, unwrapJavaResponse } from '../../http.ts'
import { buildJavaUrl } from '../../url.ts'
import { API_ENDPOINTS } from '../../endpoints.ts'
/**
* 专利/货源的本地采集开关(服务端唯一真源,前端只读)。
*
* 开着=页面把任务推本机客户端队列;关着=只建单、等 Linux 后台认领。
* 每次进页面现取,不做模块级缓存:运维改开关后用户刷新页面就该看到新口径,
* 一次小 GET 不值得换一个「改了没生效」的排查成本。
*/
export interface CollectRoutingVo {
appearancePatentLocal: boolean;
similarAsinLocal: boolean;
}
export function getCollectRouting() {
return unwrapJavaResponse(
get<JavaApiResponse<CollectRoutingVo>>(buildJavaUrl(API_ENDPOINTS.collectRouting.get)),
)
}
@@ -11,6 +11,8 @@ export type ApiSecretCheckStatus = 'unknown' | 'passed' | 'failed' | 'error'
export interface UserApiSecretItem {
moduleKey: string
moduleLabel: string
/** 货源:LLM 行为渠道(longtaitou/t8star),代理行为供应商(qingguo/jikip);未配置时服务端下发默认值。 */
channel?: string | null
masked: string
/** 完整明文值:仅代理模块被后台代填并锁定时下发,供客户端写入本地配置;其余情况为空。 */
full?: string | null
@@ -65,6 +67,8 @@ export interface UserProxyPlan {
export interface UserApiSecretMigrateItem {
moduleKey: string
value: string
/** 货源(老客户端不上报,服务端按「贞贞/极客」回填存量口径)。 */
channel?: string
}
export function fetchMyApiSecrets() {
@@ -73,11 +77,11 @@ export function fetchMyApiSecrets() {
)
}
export function putMyApiSecret(moduleKey: string, value: string) {
export function putMyApiSecret(moduleKey: string, value: string, channel?: string) {
return unwrapJavaResponse(
put<JavaApiResponse<UserApiSecretItem>, { value: string }>(
put<JavaApiResponse<UserApiSecretItem>, { value: string; channel?: string }>(
buildJavaUrl(API_ENDPOINTS.userSecret.save.replace('{moduleKey}', encodeURIComponent(moduleKey))),
{ value },
channel ? { value, channel } : { value },
),
)
}
@@ -90,11 +94,11 @@ export function deleteMyApiSecret(moduleKey: string) {
/** value 非空时检测输入值(不落库);为空时检测服务端已存密钥并把结果落库。 */
/** 检测连通性:服务端最多「首查 + 传输层失败重试一次」,最坏约 31s,故单独放宽超时。 */
export function checkMyApiSecret(moduleKey: string, value?: string) {
export function checkMyApiSecret(moduleKey: string, value?: string, channel?: string) {
return unwrapJavaResponse(
post<JavaApiResponse<UserApiSecretCheckResult>, { value?: string }>(
post<JavaApiResponse<UserApiSecretCheckResult>, { value?: string; channel?: string }>(
buildJavaUrl(API_ENDPOINTS.userSecret.check.replace('{moduleKey}', encodeURIComponent(moduleKey))),
{ value },
channel ? { value, channel } : { value },
{ timeout: 60000 },
),
)
+114 -8
View File
@@ -24,14 +24,23 @@ export interface UploadTempFileOptions {
moduleType?: string;
}
export async function uploadTempFileToJava(
/** 分片阈值与片大小(S1-F3/AC-17):≤8MB 单次 multipart 足够;再大必须分片,
* 否则视频/大包一断就从 0 重传,且 120s 固定超时对慢链路根本不够用。 */
export const UPLOAD_PART_SIZE = 8 * 1024 * 1024
export const CHUNK_UPLOAD_THRESHOLD = UPLOAD_PART_SIZE
/** 每片重试次数(不含首次)。 */
const PART_RETRY_LIMIT = 2
function uploadPartUrl(uploadId: string, index: number): string {
return `${JAVA_API_PREFIX}/files/chunk/${encodeURIComponent(uploadId)}/part/${index}`
}
/** 单次 multipart 上传(小文件短链路)。超时按体积给,不再固定 120s。 */
async function singleUpload(
file: File,
relativePathOrOptions?: string | UploadTempFileOptions,
) {
const options =
typeof relativePathOrOptions === 'string'
? { relativePath: relativePathOrOptions }
: relativePathOrOptions || {}
options: UploadTempFileOptions,
onProgress?: (percent: number) => void,
): Promise<JavaApiResponse<UploadFileVo>> {
const formData = new FormData()
formData.append('file', file)
if (options.relativePath) {
@@ -50,8 +59,105 @@ export async function uploadTempFileToJava(
headers: {
'Content-Type': 'multipart/form-data',
},
timeout: 120000,
// 30s 保底 + 每 MB 15s:慢公网传 100MB 视频不再被固定 120s 掐死(S1-F3)
timeout: 30_000 + Math.ceil(file.size / (1024 * 1024)) * 15_000,
onUploadProgress: (event) => {
if (!onProgress) return
const total = event.total || file.size
onProgress(total > 0 ? Math.min(Math.round((event.loaded / total) * 100), 100) : 0)
},
},
)
return response.data
}
/**
* 分片上传(AC-17):init → 逐片 POST(每片独立重试)→ complete 服务端拼装。
* 开传前查询已到片号:上一次断掉的会话可跳过已传片(后端保留 24h)。
*/
async function chunkedUpload(
file: File,
options: UploadTempFileOptions,
onProgress?: (percent: number) => void,
): Promise<JavaApiResponse<UploadFileVo>> {
const totalParts = Math.max(1, Math.ceil(file.size / UPLOAD_PART_SIZE))
const init = await http.post<JavaApiResponse<{ upload_id?: string }>>(
`${JAVA_API_PREFIX}/files/chunk/init`,
null,
{ params: { fileName: file.name, totalParts, totalBytes: file.size } },
)
const uploadId = String(init.data?.data?.upload_id ?? '')
if (!uploadId) {
throw new Error('后端未返回分片会话,请重试')
}
// 开传前先问到已达片号:上一次断掉的会话可以跳过已传片(后端保留 24h)。
// 这一问失败不再静默按"全缺"重传:断网/5xx 当场说清楚,别闷头把整包从 0 再传一遍。
const status = await http.get<JavaApiResponse<{ uploaded_parts?: unknown[] }>>(
`${JAVA_API_PREFIX}/files/chunk/${encodeURIComponent(uploadId)}`,
)
const arrived = status.data?.data?.uploaded_parts
const done = new Set<number>(Array.isArray(arrived) ? arrived.map((index) => Number(index)) : [])
const report = (partialBytes: number) => {
if (!onProgress) return
const loaded = done.size * UPLOAD_PART_SIZE + partialBytes
onProgress(Math.min(Math.round((loaded / file.size) * 100), 99))
}
for (let index = 0; index < totalParts; index += 1) {
if (done.has(index)) {
report(0)
continue
}
const blob = file.slice(index * UPLOAD_PART_SIZE, Math.min(file.size, (index + 1) * UPLOAD_PART_SIZE))
let lastError: unknown = null
for (let attempt = 0; attempt <= PART_RETRY_LIMIT; attempt += 1) {
const form = new FormData()
form.append('file', blob, `part-${index}`)
try {
await http.post(uploadPartUrl(uploadId, index), form, {
headers: { 'Content-Type': 'multipart/form-data' },
timeout: 300_000,
onUploadProgress: (event) => report(Math.min(event.loaded || 0, blob.size)),
})
done.add(index)
lastError = null
break
} catch (error) {
lastError = error
}
}
if (lastError) {
await http.delete(`${JAVA_API_PREFIX}/files/chunk/${encodeURIComponent(uploadId)}`).catch(() => undefined)
throw lastError instanceof Error ? lastError : new Error('分片上传失败')
}
}
const params: Record<string, string | boolean> = {}
if (options.relativePath) params.relativePath = options.relativePath
if (options.uploadToOss) params.uploadToOss = true
if (options.moduleType) params.moduleType = options.moduleType
const response = await http.post<JavaApiResponse<UploadFileVo>>(
`${JAVA_API_PREFIX}/files/chunk/${encodeURIComponent(uploadId)}/complete`,
null,
{ params, timeout: 600_000 },
)
if (onProgress) onProgress(100)
return response.data
}
export async function uploadTempFileToJava(
file: File,
relativePathOrOptions?: string | UploadTempFileOptions,
onProgress?: (percent: number) => void,
): Promise<JavaApiResponse<UploadFileVo>> {
const options =
typeof relativePathOrOptions === 'string'
? { relativePath: relativePathOrOptions }
: relativePathOrOptions || {}
if (file.size > CHUNK_UPLOAD_THRESHOLD) {
return chunkedUpload(file, options, onProgress)
}
return singleUpload(file, options, onProgress)
}

Some files were not shown because too many files have changed in this diff Show More