feat(phase-5): AC-19/AC-21 读路径时间窗与 keyset、心跳瘦读、菜单缓存、去 RAND、MDC 与补检限流

AC-19(P1-B1/B2/B3/B4)
- 任务中心(P1-B1):TaskCenterQueryService.tasks() 不带 startTime 时补 90 天默认窗口
  (startOfDefaultWindow),不再把 null 透传给 12 模块 UNION 扫全历史;selectTaskPage 的
  `LIMIT ? OFFSET ?` 改成 (createdAt,id) 游标,深翻页的 skip 挪进新语句
  selectTaskPageAnchor——它只投影 (id,createdAt) 两列。宽/窄两份 UNION 共用同一对筛选片段
  (taskFilterT/taskFilterB),锚点与页查询筛的行集在结构上不可能漂;页码语义、total、
  响应结构与前端全部不变。
- 关键词抽词(P1-B2):selectRandomUnused 不再 ORDER BY RAND()(全表 filesort)。
  服务层先 countHit 数命中集、再把窗口起点掷在 [0, total-limit],按 k.id 有序取一段后
  在内存洗牌。分布口径变了(连续 id 的一段 vs 逐行独立随机),已如实登记。
- 心跳(P1-B3):selectFileTask 只捞 id/moduleType/status/userId/updatedAt 五列;
  需要整实体的模块由新 SPI 默认方法 heartbeatNeedsFullTaskEntity()(默认 true=保守)
  声明后补一次主键读,collect-data/similar-asin/appearance-patent 三个只用
  task.getId() 的显式豁免。半行进 saveTaskCache 会写出坏缓存,比多读一次列难查得多。
  markInterrupted 改走整实体读(要 errorMessage 并同步模块缓存/快照)。
- 权限菜单(P1-B4):loadMenus 加 (companyId,menuType) 短 TTL(30s) 快照缓存,
  五处目录写路径 create/update/delete/reorder/copyTreeFromTemplate 全部写时失效;
  只缓存目录本体、不缓存用户授权行——撤权必须下一请求就生效。

AC-21(审计 §5/PERF §6)
- RequestTraceFilter 把已解析却只进日志参数的 requestId 写入 MDC(缺失写 "-"),
  finally 里 MDC.clear()(tomcat 线程复用,不清就把上一条的 id 带给下一条日志);
  logback-spring.xml 的两条 pattern 默认值 + application.yml 显式设的 file/console
  两条同批改带 [%X{requestId}]——本文件那两条会盖掉默认值,漏一处线上照样看不见。
- AppearancePatentDeliverySupport.requestRecheckForMissingRows 的补检轮 WARN 按任务限流:
  5 分钟窗口内首条 WARN(带 suppressedInLastWindow 汇总),其余降 DEBUG。刻意不吃
  recheckRounds/countRecheckRound 那对轮次计数——那是 scheduleFallbackRecheck 的补检预算,
  从日志路径再计一次会让"限流"吃掉业务预算、任务更早被判补检耗尽。
  "补检排不上队"那条不受限流(要人处置的信号不许压)。

契约激活与判据变更
- PendingPhase5ReadPathTest 5 例、PendingPhase5LogVolumeAndMdcTest 3 例全部去 @Disabled 转绿;
  后者预置的 logback 路径层级写错(mainSourcesRoot() 已是 src/main/java),激活时修正,
  并补一条 application.yml 的 pattern 判据(只增不减)。
- 新增 AppearancePatentRecheckWarnThrottleTest 3 例:钉实跑读数(6 轮只留 1 条 WARN + 5 条
  DEBUG、按任务隔离、排不上队仍出声),形状尺子之外的行为证据。
- TaskCenterQueryMapperSqlTest +2 例(锚点窄投影、游标收窄两分支)、
  TaskCenterQueryServiceTest +3 例(默认窗口、第一页不付锚点钱、页码越界返回空页)。
- 三处既有判据按新契约反转,一支没注释掉:selectTaskPage 的 `LIMIT ? OFFSET ?`→游标、
  分页参数绑定数 2→1/16→18/8→7;tasks() 三处 isNull()→notNull()(默认窗口生效本身成为断言)。
- 棘轮登记:SQL 分层语句总数 143→144(新语句点名),MigratedSqlEquivalenceTest
  XML_SELECT_TOTAL 99→100、LEDGER_ROWS 85→84、LEDGER_SELECTS 60→59——
  selectTaskPage 那一行整条摘出基准账本并点名进 NOT_MIGRATED,因为基准是"搬迁前的注解原文",
  改写它等于让等价性测试替新 SQL 背书。
- application.yml 走 hash-object+update-index 只提交本轮 pattern 那一格(4/2),
  另一路会话在飞的 client-task-pull 注释格原样留在工作树未提交。

读数:后端 mvn -o test 5,021 例/0 Failures/0 Errors/19 skipped(6:21 min,
较 Phase 4 的 5,013 净增 8 例=激活 8 例预置后新增 8 例判据);fat jar 实起,落盘行带出
`[main] INFO  [] c.nanri.aiimage.AiImageApplication - Starting...` 证明 MDC 位真在渲染;
本机无 MySQL ⇒ Hikari 反复 Starting、Tomcat 未监听,请求级 requestId 关联与现网四样本顺延 Phase 6。
This commit is contained in:
2026-10-29 16:32:46 +08:00
parent ef0bc51fa2
commit 217f5eec62
23 changed files with 630 additions and 62 deletions
@@ -7,6 +7,7 @@ import jakarta.servlet.http.HttpServletResponse;
import java.util.Locale;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;
@@ -18,6 +19,13 @@ import java.util.Optional;
@Component
public class RequestTraceFilter extends OncePerRequestFilter {
/**
* AC-21(审计 §5.2):requestId 早在 2026-09 就被解析出来了,却只塞进这一行日志的参数里,
* 全仓没有一处 {@code MDC.put} ⇒ 一个请求里另外 570 处日志互相认不出彼此。
* 入 MDC 之后,logback 的 {@code %X{requestId}} 把整条请求链缝起来(含异步 appender 落盘)。
*/
static final String MDC_REQUEST_ID = "requestId";
private static final Logger log = LoggerFactory.getLogger(RequestTraceFilter.class);
private static final int MIN_REQUEST_BODY_CACHE_LIMIT_BYTES = 1024 * 1024;
@@ -55,6 +63,9 @@ public class RequestTraceFilter extends OncePerRequestFilter {
response.setHeader("X-AIIMAGE-Instance-Source", instanceMetadata.getSource());
response.setHeader("X-AIIMAGE-Instance-Stable", String.valueOf(instanceMetadata.isStable()));
// tomcat 线程是复用的:不清 MDC 的话,下一条请求(哪怕没有 requestId)会带上上一条的 id,
// 按请求关联反而变成按请求串案。缺失也写 "-",让 pattern 里这一位永远存在、格式不抖。
MDC.put(MDC_REQUEST_ID, blankToDash(requestId));
try {
filterChain.doFilter(requestToUse, response);
} finally {
@@ -96,6 +107,7 @@ public class RequestTraceFilter extends OncePerRequestFilter {
costMs
);
}
MDC.clear();
}
}
@@ -50,6 +50,12 @@ public class AppearancePatentTaskHeartbeatSpi implements TaskModuleHeartbeatSpi
task.getId(), moduleType(), jobTouchIntervalMillis());
}
/** 只把 {@code task.getId()} 交给作业续期,不写模块缓存 ⇒ 心跳可以吃瘦读(AC-19/P1-B3)。 */
@Override
public boolean heartbeatNeedsFullTaskEntity() {
return false;
}
private long jobTouchIntervalMillis() {
return Math.max(1_000L, appearancePatentProperties.getDbJobTouchIntervalMillis());
}
@@ -17,6 +17,8 @@ import lombok.extern.slf4j.Slf4j;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicLong;
/**
* 外观专利的交付闸门与补检(E 聚类之一),从 {@code AppearancePatentTaskService} <b>原样搬移</b>
@@ -30,6 +32,20 @@ public class AppearancePatentDeliverySupport {
private static final String STATUS_FAILED = TaskStatuses.FileTask.FAILED;
/**
* AC-21(审计 §5、PERF_REPORT §6):交付缺口补检 WARN 的按任务限流窗口。
*
* <p>现网实录 task 32609/job 23345 三分钟 478 条(≈9,560 条/小时):结果文件作业每 15s
* 被 claim 一轮,同一批缺口行每轮原样再报一次,字段一字不差,日志里没有新信息,只有量。
*
* <p>本类按调用现取(不是 Spring bean,{@code AppearancePatentTaskService#deliverySupport()}
* 每次 new 一只),所以限流态只能挂静态:一个 JVM 一份,两台实例各打各的,不互相吞计数。
* 条目数由 {@link #WARN_TASKS_MAX} 兜住,超了就先把窗口已过期的清出去。
*/
static final long RECHECK_WARN_WINDOW_MILLIS = 300_000L;
static final int WARN_TASKS_MAX = 1024;
private static final ConcurrentHashMap<Long, RecheckWarnWindow> RECHECK_WARN_WINDOWS = new ConcurrentHashMap<>();
private final String moduleType;
private final AppearancePatentProperties properties;
private final AppearancePatentTaskCacheService taskCacheService;
@@ -120,8 +136,7 @@ public class AppearancePatentDeliverySupport {
boolean requeued = taskFileJobService.requeue(job.getId(),
"有 " + facts.missingRows() + " 行未完成检测,补检一轮后重新组装结果文件");
if (requeued) {
log.warn("[appearance-patent] 交付缺口触发补检轮 taskId={} jobId={} missingRows={} keys={} noDataRows={}",
task.getId(), job.getId(), facts.missingRows(), keys.size(), facts.noDataRows());
throttleRecheckWarning(task, job, facts);
return true;
}
log.warn("[appearance-patent] 交付缺口补检排不上队,转失败兜底出口按补检预算处置 taskId={} jobId={} missingRows={}",
@@ -129,6 +144,47 @@ public class AppearancePatentDeliverySupport {
throw new BusinessException("有 " + facts.missingRows() + " 行未完成检测,等待补检轮");
}
/**
* 补检轮 WARN 的限流出口(AC-21):同一任务在一个窗口内只留一条 WARN,窗口里被压掉的条数
* 随下一条 WARN 一起汇总出去({@code suppressedInLastWindow}),窗口内的重复轮次降 DEBUG。
*
* <p>刻意不去碰 {@code taskCacheService.recheckRounds/countRecheckRound} 那对轮次计数:
* 那是 {@link #scheduleFallbackRecheck} 的补检预算({@code recheckMaxRounds})。从这条路径
* 再计一次,会让「日志限流」吃掉业务预算,任务更早被判「补检已耗尽」——那是行为变更,不是治理。
*
* <p>并发口径:读-判-写三步之间有竞争,最坏情况是同窗口多打一条 WARN。限流是为了压日志量,
* 不是为了恰好一条,不值得为它加锁或把整张表串行化。
*/
private void throttleRecheckWarning(FileTaskEntity task, TaskFileJobEntity job,
AppearancePatentRowOutcome.Facts facts) {
long now = System.currentTimeMillis();
RecheckWarnWindow current = RECHECK_WARN_WINDOWS.get(task.getId());
if (current != null && now - current.startedAtMillis < RECHECK_WARN_WINDOW_MILLIS) {
log.debug("[appearance-patent] 交付缺口触发补检轮 taskId={} jobId={} missingRows={} keys={}(限流:窗口内第 {} 条)",
task.getId(), job.getId(), facts.missingRows(), facts.missingKeys().size(),
current.suppressed.incrementAndGet());
return;
}
log.warn("[appearance-patent] 交付缺口触发补检轮 taskId={} jobId={} missingRows={} keys={} noDataRows={} suppressedInLastWindow={}",
task.getId(), job.getId(), facts.missingRows(), facts.missingKeys().size(),
facts.noDataRows(), current == null ? 0L : current.suppressed.get());
if (RECHECK_WARN_WINDOWS.size() >= WARN_TASKS_MAX) {
RECHECK_WARN_WINDOWS.entrySet().removeIf(entry ->
now - entry.getValue().startedAtMillis >= RECHECK_WARN_WINDOW_MILLIS);
}
RECHECK_WARN_WINDOWS.put(task.getId(), new RecheckWarnWindow(now));
}
/** 一个任务在一个限流窗口里的补检告警态:窗口起始时刻 + 窗口内被压到 DEBUG 的条数。 */
private static final class RecheckWarnWindow {
private final long startedAtMillis;
private final AtomicLong suppressed = new AtomicLong();
private RecheckWarnWindow(long startedAtMillis) {
this.startedAtMillis = startedAtMillis;
}
}
/**
* 只量行账、不出表。补检的决定必须做在 {@code assembleResultWorkbook} 之前,而那一跑完就把
* 成品传上 OSS 并挂好了 {@code resultFileUrl}。判据与出表共用
@@ -25,4 +25,13 @@ public class CollectDataTaskHeartbeatSpi implements TaskModuleHeartbeatSpi {
public void touchHeartbeat(Long taskId, TaskHeartbeatRequest request) {
collectDataService.updateProgress(taskId, request);
}
/**
* 进度写走 {@code updateProgress(taskId, request)} 自己按主键取数,心跳不需要整实体
* (AC-19/P1-B3:采集任务的心跳最密,两列大 JSON 每 30s 捞一次纯属白付)。
*/
@Override
public boolean heartbeatNeedsFullTaskEntity() {
return false;
}
}
@@ -41,11 +41,13 @@ import org.springframework.transaction.annotation.Transactional;
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.util.ArrayList;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ThreadLocalRandom;
/**
* 五国关键词共享(桌面 Tkinter 工具的 web 化)。
@@ -200,12 +202,23 @@ public class KeywordShareService {
Long count) {
KeywordShareFilterQuery filter = buildFilter(companyId, viewerUserName, country, category, subCategory,
categoryL3, categoryL4, search, FILTER_UNUSED, null, false);
filter.setOffset(0);
long safeCount = count == null ? DEFAULT_RANDOM_COUNT : Math.min(Math.max(count, 1), MAX_RANDOM_COUNT);
filter.setLimit(safeCount);
return keywordShareMapper.selectRandomUnused(filter).stream()
// AC-19(审计 P1-B2):窗口起点先数命中集再掷,MySQL 就不必给全表打 RAND 后 filesort。
// 掷的是起点而非每一行,所以同一批词的相邻关系仍在(一次抽到的是连续 id 的一段),
// 服务层洗牌补回"散着"的观感;命中集不足一批时起点恒 0,与旧口径同为"全给你"。
filter.setOffset(randomWindowOffset(keywordShareMapper.countHit(filter), safeCount));
List<KeywordShareRowVo> window = new ArrayList<>(keywordShareMapper.selectRandomUnused(filter).stream()
.map(this::normalizeRow)
.toList();
.toList());
Collections.shuffle(window);
return window;
}
/** 随机窗口起点:落在 {@code [0, total - count]},装得下整批才有得掷。 */
static long randomWindowOffset(long total, long count) {
long span = total - count;
return span > 0 ? ThreadLocalRandom.current().nextLong(span + 1) : 0L;
}
// ---------------------------------------------------------------- 写操作
@@ -261,6 +261,7 @@ public class PermissionMenuService {
entity.setCompanyId(companyId);
entity.setSortOrder(resolveSortOrder(companyId, menuType, request.getSortOrder(), null));
permissionMenuMapper.insert(entity);
invalidateMenuTreeCache();
return toItemVo(getMenuById(entity.getId()));
}
@@ -300,6 +301,7 @@ public class PermissionMenuService {
entity.setRoutePath(routePath);
entity.setSortOrder(resolveSortOrder(companyId, menuType, request.getSortOrder(), id));
permissionMenuMapper.updateById(entity);
invalidateMenuTreeCache();
return toItemVo(getMenuById(id));
}
@@ -321,6 +323,7 @@ public class PermissionMenuService {
// installations without the FK are cleaned up as well.
userColumnPermissionMapper.deleteByColumnId(entity.getId());
permissionMenuMapper.deleteById(entity.getId());
invalidateMenuTreeCache();
}
/**
@@ -376,6 +379,7 @@ public class PermissionMenuService {
}
sort += 10;
}
invalidateMenuTreeCache();
}
/**
@@ -457,6 +461,7 @@ public class PermissionMenuService {
}
log.info("[menu-tree-copy] 新公司菜单树已建 companyId={} template={} rows={}",
companyId, templateCompanyId, inserted);
invalidateMenuTreeCache();
return inserted;
}
@@ -1031,6 +1036,29 @@ public class PermissionMenuService {
}
}
/**
* AC-19(审计 P1-B4):目录树的短 TTL 缓存。
*
* <p>侧边栏、每个受管入口的 {@code hasAnyAdminMenu}、可授予全集都从 {@code loadMenus} 取同一棵
* 树,此前每请求重查一遍再在内存里排序展开。目录结构是"配一次、几个月不动"的数据,
* 30 秒的窗口换来热路径少一次全表取数。
*
* <p>只缓存目录本体,**不缓存用户授权行**:撤权必须下一次请求就生效,那是安全语义;
* 目录结构最多陈旧一个 TTL(写路径都同机 {@code invalidateMenuTreeCache},跨实例靠 TTL 收敛)。
* 键是 (companyId, menuType),company 维度来自库里的用户行而非请求参数,条数有界。
*/
private static final long MENU_CACHE_TTL_MILLIS = 30_000L;
private final Map<String, CachedMenuTree> menuTreeCache = new java.util.concurrent.ConcurrentHashMap<>();
/** 一棵已排序的目录树快照与其入缓存的时刻。 */
private record CachedMenuTree(List<PermissionMenuEntity> menus, long cachedAtMillis) {
}
/** 目录写路径共用的失效出口:结构变了就把整份快照丢掉,下一次现取。 */
private void invalidateMenuTreeCache() {
menuTreeCache.clear();
}
/**
* 目录取数的唯一出口(§4):{@code companyId} 非空时只返回那一棵树的行。
* 授权树、有效菜单、可授予全集、越权判定都从这里取数,所以公司收窄改这一处就够一片。
@@ -1038,14 +1066,23 @@ public class PermissionMenuService {
* @param companyId null 只来自「无操作者 / 无归属」的历史与测试路径,此时退回全量目录
*/
private List<PermissionMenuEntity> loadMenus(Long companyId, String menuType) {
List<PermissionMenuEntity> rows = loadRawMenus(companyId, menuType);
if (rows == null || rows.isEmpty()) {
return List.of();
String cacheKey = (companyId == null ? "*" : companyId.toString())
+ '|' + normalizeMenuType(menuType);
long now = System.currentTimeMillis();
CachedMenuTree cached = menuTreeCache.get(cacheKey);
if (cached != null && now - cached.cachedAtMillis() < MENU_CACHE_TTL_MILLIS) {
return cached.menus();
}
return rows.stream()
.filter(row -> row != null && row.getId() != null)
.sorted(menuComparator())
.toList();
List<PermissionMenuEntity> rows = loadRawMenus(companyId, menuType);
// 不可变快照:缓存出去的东西被调用方就地改一笔,下一次请求会读到别人的写入
List<PermissionMenuEntity> sorted = rows == null || rows.isEmpty()
? List.of()
: rows.stream()
.filter(row -> row != null && row.getId() != null)
.sorted(menuComparator())
.toList();
menuTreeCache.put(cacheKey, new CachedMenuTree(sorted, now));
return sorted;
}
/**
@@ -47,6 +47,12 @@ public class SimilarAsinTaskHeartbeatSpi implements TaskModuleHeartbeatSpi {
return Duration.ofMillis(checkpointIntervalMillis());
}
/** 只把 {@code task.getId()} 交给作业续期,不写模块缓存 ⇒ 心跳可以吃瘦读(AC-19/P1-B3)。 */
@Override
public boolean heartbeatNeedsFullTaskEntity() {
return false;
}
private long checkpointIntervalMillis() {
return Math.max(1_000L, similarAsinProperties.getDbTaskTouchIntervalMillis());
}
@@ -79,7 +79,7 @@ public class TaskHeartbeatService {
boolean brandOnly = MODULE_BRAND.equalsIgnoreCase(requestedModuleType);
boolean skipBrandLookup = MODULE_SHOP_DATA_CRAWL.equalsIgnoreCase(requestedModuleType);
TaskHeartbeatVo fileResult = brandOnly ? null : touchFileTaskIfRunning(selectFileTask(taskId), request);
TaskHeartbeatVo fileResult = brandOnly ? null : touchFileTaskIfRunning(taskId, request);
TaskHeartbeatVo brandResult = skipBrandLookup ? null : brandTaskHeartbeatSpi.heartbeat(taskId, request);
// 心跳到达打点:陈旧判死线用它把「心跳时钟」改成从首次心跳起算(见 StaleTaskHeartbeatPolicy)。
@@ -112,7 +112,26 @@ public class TaskHeartbeatService {
return TaskHeartbeatVo.notAlive(null, null, "task not found");
}
/**
* AC-19(审计 P1-B3):心跳热路径的任务读只取活性判定要的五列。
*
* <p>{@code biz_file_task.request_json}/{@code result_json} 能到 MB 级,而这条读是
* 「每单每 30s 一次」的热路径,原实现整行捞回来只为读 id/moduleType/status/userId/updatedAt。
* 需要把实体交给模块 SPI 的({@code saveTaskCache} 会把整个实体序列化进模块缓存),
* 由 {@link TaskModuleHeartbeatSpi#heartbeatNeedsFullTaskEntity()} 声明后走
* {@link #selectFullFileTask}——半行进缓存比多读两列难查得多,所以宁可多一次主键点查。
*/
private FileTaskEntity selectFileTask(Long taskId) {
LambdaQueryWrapper<FileTaskEntity> fileQuery = new LambdaQueryWrapper<FileTaskEntity>()
.select(FileTaskEntity::getId, FileTaskEntity::getModuleType, FileTaskEntity::getStatus,
FileTaskEntity::getUserId, FileTaskEntity::getUpdatedAt)
.eq(FileTaskEntity::getId, taskId)
.last("limit 1");
return fileTaskMapper.selectOne(fileQuery);
}
/** 整实体读:客户端中断恢复(要 errorMessage,并把实体同步进模块缓存与快照)与声明需要整行的模块心跳。 */
private FileTaskEntity selectFullFileTask(Long taskId) {
LambdaQueryWrapper<FileTaskEntity> fileQuery = new LambdaQueryWrapper<FileTaskEntity>()
.eq(FileTaskEntity::getId, taskId)
.last("limit 1");
@@ -132,7 +151,7 @@ public class TaskHeartbeatService {
String safeReason = (reason == null || reason.isBlank())
? "客户端异常中断,任务已自动失败"
: "客户端异常中断: " + reason.trim();
FileTaskEntity fileTask = selectFileTask(taskId);
FileTaskEntity fileTask = selectFullFileTask(taskId);
if (fileTask != null) {
String status = fileTask.getStatus();
int updated = fileTaskMapper.update(null, new LambdaUpdateWrapper<FileTaskEntity>()
@@ -167,12 +186,23 @@ public class TaskHeartbeatService {
return TaskHeartbeatVo.notAlive(null, null, "task not found");
}
private TaskHeartbeatVo touchFileTaskIfRunning(FileTaskEntity task, TaskHeartbeatRequest request) {
private TaskHeartbeatVo touchFileTaskIfRunning(Long taskId, TaskHeartbeatRequest request) {
FileTaskEntity task = selectFileTask(taskId);
if (task == null) {
return null;
}
String moduleType = task.getModuleType() == null ? "" : task.getModuleType();
TaskModuleHeartbeatSpi handler = moduleHandlers.get(moduleType);
if (handler != null && handler.heartbeatNeedsFullTaskEntity()) {
// 归属校验与模块缓存回写都吃整实体(publish/shop-data-crawl 读实例路由列,
// 七个模块的 saveTaskCache 把实体序列化落盘),瘦行给它们会写出半行 ⇒ 补一次主键读。
task = selectFullFileTask(taskId);
if (task == null) {
return null;
}
moduleType = task.getModuleType() == null ? "" : task.getModuleType();
handler = moduleHandlers.get(moduleType);
}
if (handler != null) {
handler.ensureOwnership(task, moduleType + " task heartbeat");
}
@@ -44,6 +44,21 @@ public interface TaskModuleHeartbeatSpi {
default void onHeartbeat(FileTaskEntity task) {
}
/**
* 心跳时是否需要整行 {@code FileTaskEntity}。
*
* <p>AC-19(审计 P1-B3):心跳是「每单每 30s 一次」的热路径,而 {@code biz_file_task} 的
* {@code request_json}/{@code result_json} 能到 MB 级。task 侧默认只捞活性判定要的五列,
* 只有真把实体交出去用的模块才付整行读的钱——即 {@link #saveTaskCache(FileTaskEntity)}
* 会把实体序列化进模块缓存的那些(半行进缓存比多读一次列难查得多)。
*
* <p>默认 {@code true}(保守):新模块不改这儿就维持今天的取整行行为。只用得到
* {@code task.getId()} 的实现才显式改成 false。
*/
default boolean heartbeatNeedsFullTaskEntity() {
return true;
}
/**
* biz_file_task.updated_at 检查点节流间隔。
*
@@ -217,6 +217,27 @@ public interface TaskCenterQueryMapper {
+ USER_SCOPE_B
+ ")";
/**
* 页首游标锚点(AC-19/P1-B1):第 N 页那一行的前一行——按同一套筛选与同一个排序键
* {@code (createdAt DESC, id DESC)} 走到 {@code offset} 处,只取两列。
*
* <p>深翻页的 skip 只在这条窄语句里发生,页查询本身改用游标;两条必须共用同一份 UNION
* (mapper XML 的 {@code taskUnionRows}),否则锚点算的是另一批行,第 N 页会安静地给错。
*/
TaskCenterTaskRow selectTaskPageAnchor(@Param("userIds") List<Long> userIds,
@Param("moduleType") String moduleType,
@Param("statuses") List<String> statuses,
@Param("shop") String shop,
@Param("startTime") LocalDateTime startTime,
@Param("endTime") LocalDateTime endTime,
@Param("offset") long offset);
/**
* 任务列表一页:keyset 游标取数,不再吃 OFFSET。
*
* <p>{@code afterCreatedAt}/{@code afterId} 同时给值才是游标(第一页两个都为 null = 从头取);
* 游标条件能被我方的 MySQL 8.4 压进 UNION 分支,从而用上 created_at 索引。
*/
List<TaskCenterTaskRow> selectTaskPage(@Param("userIds") List<Long> userIds,
@Param("moduleType") String moduleType,
@Param("statuses") List<String> statuses,
@@ -224,7 +245,8 @@ public interface TaskCenterQueryMapper {
@Param("startTime") LocalDateTime startTime,
@Param("endTime") LocalDateTime endTime,
@Param("limit") long limit,
@Param("offset") long offset);
@Param("afterCreatedAt") LocalDateTime afterCreatedAt,
@Param("afterId") Long afterId);
long countTaskPage(@Param("userIds") List<Long> userIds,
@Param("moduleType") String moduleType,
@@ -51,6 +51,11 @@ public class TaskCenterQueryService {
private static final long ACTIVE_TASK_SCAN_CAP = 500;
private static final long DEFAULT_PAGE_SIZE = 20;
private static final long MAX_PAGE_SIZE = 100;
/**
* AC-19(审计 P1-B1):任务中心默认时间窗(天)。请求不带 {@code startTime} 时由服务层补上,
* 页码跳转与总数都只在这个窗口里发生;带 {@code startTime} 就完全按请求走。
*/
private static final int DEFAULT_WINDOW_DAYS = 90;
/**
* 心跳位置读取封顶,与页大小同源:看板一页最多 100 行、30 秒一轮。
* 沿用 light 侧那个 20 的话,第 21 行之后的在跑任务永远读不到进度。
@@ -117,15 +122,34 @@ public class TaskCenterQueryService {
}
List<String> statuses = statusFilter(query.getStatus());
// AC-19(审计 P1-B1):时间窗先补再传,count 与页查询共用同一个窗口口径。
LocalDateTime startTime = startOfDefaultWindow(query.getStartTime());
LocalDateTime endTime = query.getEndTime();
long total = queryMapper.countTaskPage(userIds, query.getModuleType(), statuses,
query.getShop(), query.getStartTime(), query.getEndTime());
query.getShop(), startTime, endTime);
if (total <= 0) {
return TaskCenterTaskPageVo.empty(page, pageSize);
}
// 深翻页的 skip 交给窄锚点语句(只投影 createdAt,id 两列),页查询本身走 keyset 游标:
// 游标条件能被 MySQL 压进 UNION 分支用上 created_at 索引,OFFSET 不能,只能物化整个 UNION。
long offset = (page - 1) * pageSize;
LocalDateTime afterCreatedAt = null;
Long afterId = null;
if (offset > 0) {
TaskCenterTaskRow anchor = queryMapper.selectTaskPageAnchor(userIds, query.getModuleType(),
statuses, query.getShop(), startTime, endTime, offset);
if (anchor == null || anchor.getCreatedAt() == null || anchor.getId() == null) {
// 页码落在窗口之外:与旧实现取到空行同样返回空页,不拿 null 游标去要第一页
return TaskCenterTaskPageVo.empty(page, pageSize);
}
afterCreatedAt = anchor.getCreatedAt();
afterId = anchor.getId();
}
List<TaskCenterTaskRow> rows = queryMapper.selectTaskPage(userIds, query.getModuleType(),
statuses, query.getShop(), query.getStartTime(), query.getEndTime(),
pageSize, (page - 1) * pageSize);
statuses, query.getShop(), startTime, endTime,
pageSize, afterCreatedAt, afterId);
if (rows == null || rows.isEmpty()) {
return TaskCenterTaskPageVo.empty(page, pageSize);
}
@@ -138,6 +162,17 @@ public class TaskCenterQueryService {
return vo;
}
/**
* 默认时间窗起点(AC-19/P1-B1):没带 {@code startTime} 才补 {@code now - 90 天}。
*
* <p>这一条给 UNION 的两个分支各加一条 {@code created_at} 下界,是把「全历史物化再排序」
* 换成「按索引取窗口」的那一刀。代价是对管理员可见的行为收紧:90 天前的任务不再默认出现在
* 列表与总数里,必须显式传 startTime 才能翻旧账——Phase 6 真机总验收要按这条复现一次翻页路径。
*/
static LocalDateTime startOfDefaultWindow(LocalDateTime requested) {
return requested != null ? requested : LocalDateTime.now().minusDays(DEFAULT_WINDOW_DAYS);
}
/**
* 状态筛选展开:空白 = 不裁剪;{@code ACTIVE} = 前端默认的「执行中」,等于三个进行中状态的合并
* (与判卡住用的 {@link #ACTIVE_STATUSES} 同一组,不另定口径);{@code SUCCESS}/{@code FAILED}
@@ -80,8 +80,10 @@ logging:
root: INFO
com.nanri.aiimage: INFO
pattern:
file: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
console: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
# AC-21:与 logback-spring.xml 的默认值同口径(本文件显式设了这两条,默认值会被盖掉),
# %X{requestId} 由 RequestTraceFilter 入 MDC。两处必须一起改,漏一处线上就看不见请求链。
file: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%X{requestId}] %logger{36} - %msg%n"
console: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%X{requestId}] %logger{36} - %msg%n"
mybatis-plus:
mapper-locations: classpath*:mapper/**/*.xml
@@ -4,8 +4,11 @@
<springProperty scope="context" name="LOG_NAME" source="logging.file.name" defaultValue="aiimage-backend.log"/>
<springProperty scope="context" name="ROOT_LOG_LEVEL" source="logging.level.root" defaultValue="INFO"/>
<springProperty scope="context" name="APP_LOG_LEVEL" source="logging.level.com.nanri.aiimage" defaultValue="INFO"/>
<springProperty scope="context" name="FILE_PATTERN" source="logging.pattern.file" defaultValue="%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"/>
<springProperty scope="context" name="CONSOLE_PATTERN" source="logging.pattern.console" defaultValue="%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"/>
<!-- AC-21(审计 §5.2):%X{requestId} 由 RequestTraceFilter 入 MDC。放在 %-5level 之后、
%logger 之前,既保住原有列宽,又能用一行 requestId 把整条请求链的日志串起来。
⚠ 这里是"配置没给时的默认值",application.yml 显式设了 logging.pattern.* 会盖掉它 ⇒ 两边同改。 -->
<springProperty scope="context" name="FILE_PATTERN" source="logging.pattern.file" defaultValue="%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%X{requestId}] %logger{36} - %msg%n"/>
<springProperty scope="context" name="CONSOLE_PATTERN" source="logging.pattern.console" defaultValue="%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%X{requestId}] %logger{36} - %msg%n"/>
<property name="APP_LOG_FILE" value="${LOG_PATH}/${LOG_NAME}"/>
<property name="APP_LOG_ARCHIVE" value="${LOG_PATH}/${LOG_NAME}.%d{yyyy-MM-dd}.%i"/>
@@ -45,13 +45,17 @@
LIMIT #{f.offset}, #{f.limit}
</select>
<!-- AC-19(审计 P1-B2):随机抽词不再 ORDER BY RAND()——那一句给命中集每一行打一个随机数再
filesort 全表,词库越大越慢。改成「按主键升序取一段窗口」:窗口起点由服务层先 count 命中集、
再掷在 [0, total-limit] 上,LIMIT 之前不必排序。服务层拿到窗口后就地洗牌,
页面看到的仍是散着的词。 -->
<select id="selectRandomUnused" resultType="com.nanri.aiimage.modules.keywordshare.model.vo.KeywordShareRowVo">
SELECT k.id, k.country, k.keyword, k.chinese, k.category, k.sub_category, k.category_l3, k.category_l4,
(u.country IS NOT NULL) AS used, u.used_time, u.user_name AS used_user
FROM <include refid="usageJoin"/>
WHERE <include refid="filterConds"/>
ORDER BY RAND()
LIMIT #{f.limit}
ORDER BY k.id
LIMIT #{f.limit} OFFSET #{f.offset}
</select>
<select id="countDupKeywords" resultType="long">
@@ -54,8 +54,28 @@
<select id="selectTaskCountries" resultType="com.nanri.aiimage.modules.taskcenter.model.dto.TaskCenterCountryRow">
SELECT t.id AS taskId, COALESCE(NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.filters.countryCode')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.countryCode')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.country')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.publish_country')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.countryCodes')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.country_codes')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.filters.countryCodes')), ''), 'null')) AS country FROM biz_file_task t WHERE t.id IN <foreach collection='taskIds' item='tid' open='(' separator=',' close=')'>#{tid}</foreach>
</select>
<!-- AC-19(审计 P1-B1):页查询不再吃 OFFSET。深翻页的 skip 挪进 selectTaskPageAnchor,
它只投影 (id,createdAt) 两列;页查询本身按 (createdAt,id) 游标取 limit 行,
MySQL 8.0.22+ 能把游标条件压进 UNION 分支、用上 created_at 索引,OFFSET 压不进去,
只能把整个 UNION 物化再跳过。
下面的筛选片段是宽行 UNION 与窄锚点的唯一公共出处:两份投影的筛选必须逐字相同,
否则锚点算的是另一批行,第 N 页会安静地给错。共用片段让「漂」在结构上不可能发生。 -->
<sql id="taskFilterT">t.module_type IN ('PUBLISH','DELETE_BRAND','PRODUCT_RISK_RESOLVE','PRICE_TRACK','SHOP_MATCH','PATROL_DELETE','QUERY_ASIN','WITHDRAW','APPEARANCE_PATENT','SIMILAR_ASIN','COLLECT_DATA','SHOP_DATA_CRAWL')<if test='userIds != null'> AND t.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND UPPER(t.module_type) = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND t.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND t.created_at &lt;= #{endTime}</if><if test='shop != null'> AND (EXISTS (SELECT 1 FROM biz_publish_file p WHERE p.task_id = t.id AND LOWER(TRIM(p.shop_name)) = LOWER(TRIM(#{shop}))) OR (t.module_type IN ('PATROL_DELETE','PRICE_TRACK','PRODUCT_RISK_RESOLVE','QUERY_ASIN','SHOP_DATA_CRAWL','SHOP_MATCH') AND EXISTS (SELECT 1 FROM biz_file_result f WHERE f.task_id = t.id AND LOWER(TRIM(f.source_filename)) = LOWER(TRIM(#{shop})))))</if></sql>
<sql id="taskFilterB">1 = 1<if test='userIds != null'> AND b.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND 'BRAND' = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND b.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND b.created_at &lt;= #{endTime}</if><if test='shop != null'> AND 1 = 0</if></sql>
<sql id="taskUnionRows">
(SELECT t.id AS id, t.task_no AS taskNo, UPPER(t.module_type) AS moduleType, CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END AS status, t.user_id AS userId, t.device_id AS deviceId, t.created_at AS createdAt, t.finished_at AS finishedAt, t.updated_at AS updatedAt, t.source_file_count AS totalCount, t.success_file_count AS successCount, t.failed_file_count AS failedCount, 0 AS brand FROM biz_file_task t WHERE <include refid="taskFilterT"/> UNION ALL SELECT b.id, CONCAT('brand-', b.id), 'BRAND', CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END, b.user_id, b.device_id, b.created_at, CASE WHEN b.status IN ('cancelled','failed','success') THEN b.updated_at END, b.updated_at, b.progress_total, b.progress_current, 0, 1 FROM brand_crawl_tasks b WHERE <include refid="taskFilterB"/>)
</sql>
<sql id="taskUnionKeys">
(SELECT t.id AS id, t.created_at AS createdAt FROM biz_file_task t WHERE <include refid="taskFilterT"/> UNION ALL SELECT b.id, b.created_at FROM brand_crawl_tasks b WHERE <include refid="taskFilterB"/>)
</sql>
<select id="selectTaskPageAnchor" resultType="com.nanri.aiimage.modules.taskcenter.model.dto.TaskCenterTaskRow">
SELECT m.createdAt, m.id FROM <include refid="taskUnionKeys"/> m
ORDER BY m.createdAt DESC, m.id DESC LIMIT 1 OFFSET #{offset}
</select>
<select id="selectTaskPage" resultType="com.nanri.aiimage.modules.taskcenter.model.dto.TaskCenterTaskRow">
SELECT m.* FROM (SELECT t.id AS id, t.task_no AS taskNo, UPPER(t.module_type) AS moduleType, CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END AS status, t.user_id AS userId, t.device_id AS deviceId, t.created_at AS createdAt, t.finished_at AS finishedAt, t.updated_at AS updatedAt, t.source_file_count AS totalCount, t.success_file_count AS successCount, t.failed_file_count AS failedCount, 0 AS brand FROM biz_file_task t WHERE t.module_type IN ('PUBLISH','DELETE_BRAND','PRODUCT_RISK_RESOLVE','PRICE_TRACK','SHOP_MATCH','PATROL_DELETE','QUERY_ASIN','WITHDRAW','APPEARANCE_PATENT','SIMILAR_ASIN','COLLECT_DATA','SHOP_DATA_CRAWL')<if test='userIds != null'> AND t.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND UPPER(t.module_type) = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND t.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND t.created_at &lt;= #{endTime}</if><if test='shop != null'> AND (EXISTS (SELECT 1 FROM biz_publish_file p WHERE p.task_id = t.id AND LOWER(TRIM(p.shop_name)) = LOWER(TRIM(#{shop}))) OR (t.module_type IN ('PATROL_DELETE','PRICE_TRACK','PRODUCT_RISK_RESOLVE','QUERY_ASIN','SHOP_DATA_CRAWL','SHOP_MATCH') AND EXISTS (SELECT 1 FROM biz_file_result f WHERE f.task_id = t.id AND LOWER(TRIM(f.source_filename)) = LOWER(TRIM(#{shop})))))</if> UNION ALL SELECT b.id, CONCAT('brand-', b.id), 'BRAND', CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END, b.user_id, b.device_id, b.created_at, CASE WHEN b.status IN ('cancelled','failed','success') THEN b.updated_at END, b.updated_at, b.progress_total, b.progress_current, 0, 1 FROM brand_crawl_tasks b WHERE 1 = 1<if test='userIds != null'> AND b.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND 'BRAND' = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND b.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND b.created_at &lt;= #{endTime}</if><if test='shop != null'> AND 1 = 0</if>) m ORDER BY m.createdAt DESC, m.id DESC LIMIT #{limit} OFFSET #{offset}
SELECT m.* FROM <include refid="taskUnionRows"/> m
<if test='afterCreatedAt != null'> WHERE (m.createdAt &lt; #{afterCreatedAt} OR (m.createdAt = #{afterCreatedAt} AND m.id &lt; #{afterId}))</if>
ORDER BY m.createdAt DESC, m.id DESC LIMIT #{limit}
</select>
<select id="selectUserIdsByUsername" resultType="long">
SELECT id FROM users WHERE username LIKE CONCAT('%', #{keyword}, '%') LIMIT 500
@@ -52,8 +52,12 @@ import static org.junit.jupiter.api.Assertions.fail;
class MigratedSqlEquivalenceTest {
private static final String FIXTURE = "sql-layering/migrated-statements.tsv";
private static final int LEDGER_ROWS = 85;
private static final int LEDGER_SELECTS = 60;
// 85 → 84:Phase 5(AC-19/P1-B1)有意把 {@code TaskCenterQueryMapper#selectTaskPage} 从
// OFFSET 改成 keyset 游标。账本记的是「注解时代原文」,改一行基准就等于让等价性测试照着
// 新 SQL 自证 ⇒ 把该行整条摘出账本、点名进下方 NOT_MIGRATED,让基准继续只描述真正未改动的迁移件。
private static final int LEDGER_ROWS = 84;
// 60 → 59:摘出去那行是一条 select。
private static final int LEDGER_SELECTS = 59;
/** 全部 mapper XML 里的 select 总数:账本 60 + TaskImageCache 5 + AsinResultCache 1。 */
/** 66 → 67:会话容忍新增的 {@code AdminUserMapper#countLiveWorkFor} 也是一条 select。 */
// 67 → 78:keywordshare 模块新写 11 条 select,逐条登记进下方 NOT_MIGRATED。
@@ -67,7 +71,10 @@ class MigratedSqlEquivalenceTest {
// (SqlLayeringConventionTest.XML_STATEMENT_TOTAL 同轮 130 → 140)。
// 99 → 99:客户端全量同步新写 1 条 insert(KeywordShareMapper#upsertBatchFromSync),
// 点名进 NOT_MIGRATED 但不是 select ⇒ select 总数不动(SqlLayeringConventionTest 同轮 140 → 141)。
private static final int XML_SELECT_TOTAL = 99;
// 99 → 100:Phase 5(AC-19)为任务中心深翻页新增窄锚点语句 {@code TaskCenterQueryMapper#selectTaskPageAnchor}
// (两列投影算页首游标),同轮点名进 NOT_MIGRATED;同轮的 selectTaskPage 改写让它从账本搬进
// NOT_MIGRATED(select 总数不变,只是记账位置换了)。
private static final int XML_SELECT_TOTAL = 100;
/**
* 不是「从注解搬过来」的语句:注解时代没有原文可对撞,等价性这支只能跳过它们。
@@ -121,7 +128,14 @@ class MigratedSqlEquivalenceTest {
"KeywordUsageAdminMapper#selectDeleteRows",
"KeywordUsageAdminMapper#selectStats",
"KeywordUsageAdminMapper#selectUsers",
"KeywordUsageAdminMapper#selectCountries");
"KeywordUsageAdminMapper#selectCountries",
// Phase 5(AC-19/P1-B1)taskcenter 两条:
// selectTaskPage —— 注解时代原文带 {@code LIMIT ? OFFSET ?},本轮有意改成
// (createdAt,id) 游标。基准是搬迁前的快照,不该被改写来替新 SQL 背书,
// 所以整行从 migrated-statements.tsv 摘掉、点名到这里(LEDGER_ROWS 85→84)。
// selectTaskPageAnchor —— 本轮新写的两列窄投影,从来没有注解原文可对撞。
"TaskCenterQueryMapper#selectTaskPage",
"TaskCenterQueryMapper#selectTaskPageAnchor");
/** 账本 5 列:接口 / 方法 / @Select 种类 / 是否含常量插值 / 注解原文(Java 反转义后那条串)。 */
private static final int LEDGER_COLUMNS = 5;
@@ -142,7 +156,7 @@ class MigratedSqlEquivalenceTest {
List<String[]> rows = readLedger();
assertEquals(LEDGER_ROWS, rows.size(), "fixture 行数变了——它是搬迁前的快照,不该跟着代码漂");
assertEquals(LEDGER_SELECTS, rows.stream().filter(r -> "Select".equals(r[2])).count(),
"fixture 里的 Select 条数与基线不符(读侧账本 = 60)");
"fixture 里的 Select 条数与基线不符(读侧账本 = 59)");
assertEquals(LEDGER_ROWS - LEDGER_SELECTS,
rows.stream().filter(r -> !"Select".equals(r[2])).count(),
"写侧条数对不上(25 = Insert+Update+Delete)");
@@ -88,7 +88,10 @@ class SqlLayeringConventionTest {
// 141 → 143:V179 采集「被过滤行」新表 {@code CollectDataFilteredRowMapper} 的两条写侧语句
// (insertBatch 多值插入 + markResolved 盖回流时间)。该 mapper 没有注解时代,
// 不在 MigratedSqlEquivalenceTest 的基准账本里 ⇒ 不占 NOT_MIGRATED,只在这本总分母上记名。
private static final int XML_STATEMENT_TOTAL = 143;
// 143 → 144:Phase 5(AC-19/P1-B1)为任务中心深翻页新增窄锚点语句
// {@code TaskCenterQueryMapper#selectTaskPageAnchor}(两列投影算页首游标),
// 同轮登记进 MigratedSqlEquivalenceTest.NOT_MIGRATED(该文件的 XML_SELECT_TOTAL 同轮 99 → 100)。
private static final int XML_STATEMENT_TOTAL = 144;
/**
* wrapper 里夹带函数级 SQL 的现状("这不是简单单表"的证据),文件名 → 条数:
@@ -1,7 +1,6 @@
package com.nanri.aiimage.config;
import com.nanri.aiimage.testutil.SourceScan;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
@@ -34,7 +33,6 @@ import static org.assertj.core.api.Assertions.assertThat;
* {@code app_client/tests/test_pending_phase1_日志脱敏与轮转.py}(AC-1,含大小轮转)与
* {@code scripts/tests/test_pending_phase5_worker日志轮转与保留.py}(AC-21 的 worker 侧)。
*/
@Disabled("Pending Phase 5: AC-21 补检 WARN 限流/汇总与 MDC 关联")
@DisplayName("AC-21 日志量级与请求关联契约")
class PendingPhase5LogVolumeAndMdcTest {
@@ -63,13 +61,22 @@ class PendingPhase5LogVolumeAndMdcTest {
@Test
void 日志_pattern_必须带出_MDC_位() throws Exception {
Path moduleRoot = com.nanri.aiimage.testutil.SourceScan.mainSourcesRoot()
.resolve("../../src/main/resources/logback-spring.xml")
// 预置时这条路径写错了层级(mainSourcesRoot() 已经是 src/main/java,../../ 只退到 src/),
// Phase 5 激活时按真实落点修正——判据本身一字未减。
Path logbackXml = com.nanri.aiimage.testutil.SourceScan.mainSourcesRoot()
.resolve("../../../src/main/resources/logback-spring.xml")
.normalize();
String xml = Files.readString(moduleRoot);
String xml = Files.readString(logbackXml);
assertThat(xml)
.as("pattern 里没有 %X{...} 时,MDC 写了也看不见(FILE/CONSOLE 两条都要带)")
.contains("%X{requestId}");
// application.yml 显式设了 logging.pattern.*,会把 logback 里的 defaultValue 整条盖掉 ⇒
// 只改 logback 等于改了一份线上读不到的默认值。两处必须同时带出 MDC 位。
String application = Files.readString(logbackXml.resolveSibling("application.yml"));
assertThat(application)
.as("application.yml 的 pattern 覆盖了 logback 默认值,漏改这一处线上照样看不见 requestId")
.contains("%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%X{requestId}] %logger{36} - %msg%n");
}
}
@@ -0,0 +1,165 @@
package com.nanri.aiimage.modules.appearancepatent.service.support;
import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.Logger;
import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.read.ListAppender;
import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.config.AppearancePatentProperties;
import com.nanri.aiimage.modules.appearancepatent.service.AppearancePatentTaskCacheService;
import com.nanri.aiimage.modules.task.mapper.FileTaskMapper;
import com.nanri.aiimage.modules.task.model.entity.FileTaskEntity;
import com.nanri.aiimage.modules.task.model.entity.TaskFileJobEntity;
import com.nanri.aiimage.modules.task.service.TaskFileJobService;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.slf4j.LoggerFactory;
import java.util.List;
import java.util.concurrent.atomic.AtomicLong;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
/**
* AC-21(审计 §5、PERF_REPORT §6):交付缺口补检 WARN 必须按任务限流。
*
* <p>现网实录 task 32609/job 23345 三分钟 478 条(≈9,560 条/小时):结果文件作业每 15s 被
* claim 一轮,同一批缺口行每轮原样再报一次,字段一字不差。同名预置的
* {@code PendingPhase5LogVolumeAndMdcTest} 钉的是源码形状,本支钉的是实跑读数——
* 跑六轮只留一条 WARN,其余降级但不丢数。
*
* <p>限流态是静态的(本类按调用现取、不是 bean),所以每个用例各拿自己的 taskId,
* 免得先跑的用例把窗口占掉。
*/
class AppearancePatentRecheckWarnThrottleTest {
private static final AtomicLong TASK_SEQ = new AtomicLong(System.nanoTime());
private final Logger logger = (Logger) LoggerFactory.getLogger(AppearancePatentDeliverySupport.class);
private final ListAppender<ILoggingEvent> appender = new ListAppender<>();
private Level originalLevel;
private boolean originalAdditive;
private AppearancePatentTaskCacheService cacheService;
private TaskFileJobService taskFileJobService;
private AppearancePatentDeliverySupport support;
@BeforeEach
void attachAppender() {
originalLevel = logger.getLevel();
originalAdditive = logger.isAdditive();
logger.setLevel(Level.DEBUG);
logger.setAdditive(false);
appender.start();
logger.addAppender(appender);
cacheService = mock(AppearancePatentTaskCacheService.class);
taskFileJobService = mock(TaskFileJobService.class);
support = new AppearancePatentDeliverySupport("APPEARANCE_PATENT", new AppearancePatentProperties(),
cacheService, taskFileJobService, mock(FileTaskMapper.class),
null, null, null, null, null, null);
}
@AfterEach
void detachAppender() {
logger.detachAppender(appender);
appender.stop();
logger.setLevel(originalLevel);
logger.setAdditive(originalAdditive);
}
@Test
@DisplayName("AC-21:同一任务连补六轮只留一条 WARN,其余降 DEBUG")
void 补检轮重复触发时窗口内只留一条_WARN() {
long taskId = TASK_SEQ.incrementAndGet();
when(taskFileJobService.requeue(anyLong(), any())).thenReturn(true);
for (int round = 0; round < 6; round++) {
assertThat(support.requestRecheckForMissingRows(task(taskId), job(700L), gap())).isTrue();
}
assertThat(warns()).hasSize(1);
assertThat(debugs()).hasSize(5);
assertThat(warns().get(0)).contains("交付缺口触发补检轮").contains("missingRows=3");
// 限流不许把「补了几轮」这件事抹掉:窗口内被压掉的条数随下一条 WARN 汇总出去
assertThat(warns().get(0)).contains("suppressedInLastWindow=");
}
@Test
@DisplayName("AC-21:限流按任务隔离,另一个任务的缺口照样告警")
void 限流按任务隔离不许吞掉别的任务() {
when(taskFileJobService.requeue(anyLong(), any())).thenReturn(true);
long first = TASK_SEQ.incrementAndGet();
long second = TASK_SEQ.incrementAndGet();
support.requestRecheckForMissingRows(task(first), job(701L), gap());
support.requestRecheckForMissingRows(task(first), job(701L), gap());
support.requestRecheckForMissingRows(task(second), job(702L), gap());
assertThat(warns()).hasSize(2);
assertThat(warns()).anyMatch(line -> line.contains("taskId=" + first))
.anyMatch(line -> line.contains("taskId=" + second));
}
/**
* 「补检排不上队」是要人处置的那一条(作业就此转失败兜底出口),不能被限流顺手压掉。
*/
@Test
@DisplayName("AC-21:排不上队那条 WARN 不受限流影响")
void 排不上队仍然要出声并抛错() {
long taskId = TASK_SEQ.incrementAndGet();
when(taskFileJobService.requeue(anyLong(), any())).thenReturn(false);
assertThatThrownBy(() -> {
support.requestRecheckForMissingRows(task(taskId), job(703L), gap());
support.requestRecheckForMissingRows(task(taskId), job(703L), gap());
}).isInstanceOf(BusinessException.class);
assertThat(warns()).hasSize(1);
assertThat(warns().get(0)).contains("补检排不上队");
assertThat(debugs()).isEmpty();
}
// ── 造数 helper ────────────────────────────────────────────────────
private static FileTaskEntity task(long id) {
FileTaskEntity task = new FileTaskEntity();
task.setId(id);
task.setModuleType("APPEARANCE_PATENT");
task.setStatus("RUNNING");
return task;
}
private static TaskFileJobEntity job(long id) {
TaskFileJobEntity job = new TaskFileJobEntity();
job.setId(id);
return job;
}
/** expected=5 / judged=2 ⇒ missingRows=3,两条缺口行的 key。 */
private static AppearancePatentRowOutcome.Facts gap() {
return new AppearancePatentRowOutcome.Facts(5, 2, 1, List.of("B0AAAA", "B0BBBB"));
}
private List<String> warns() {
return lines(Level.WARN);
}
private List<String> debugs() {
return lines(Level.DEBUG);
}
private List<String> lines(Level level) {
return appender.list.stream()
.filter(event -> event.getLevel() == level)
.map(ILoggingEvent::getFormattedMessage)
.toList();
}
}
@@ -58,10 +58,12 @@ class TaskCenterQueryMapperSqlTest {
.replaceAll("\\s*,\\s*", ",");
}
/** 任务列表分页的查询参数(状态已改为「一组大写状态」,「在跑」由服务层展开)。 */
private static Map<String, Object> taskPageParams(Object shop, long limit, long offset) {
/** AC-19 之后的页查询参数:游标两个都缺 = 从头取;给值 = keyset 取下一页。 */
private static Map<String, Object> pageParams(Object shop, long limit,
Object afterCreatedAt, Object afterId) {
return params("userIds", null, "moduleType", null, "statuses", null, "shop", shop,
"startTime", null, "endTime", null, "limit", limit, "offset", offset);
"startTime", null, "endTime", null, "limit", limit,
"afterCreatedAt", afterCreatedAt, "afterId", afterId);
}
@Test
@@ -190,7 +192,7 @@ class TaskCenterQueryMapperSqlTest {
@Test
void taskListUnionsBizAndBrandWithoutTouchingRequestJson() {
BoundSql boundSql = render("selectTaskPage", taskPageParams(null, 20L, 0L));
BoundSql boundSql = render("selectTaskPage", pageParams(null, 20L, null, null));
String sql = sqlOf(boundSql);
assertTrue(sql.contains("SELECT m.* FROM(SELECT t.id AS id"));
@@ -203,11 +205,45 @@ class TaskCenterQueryMapperSqlTest {
assertTrue(sql.contains("CASE WHEN b.status IN('cancelled','failed','success') THEN b.updated_at END"),
"品牌表无 finished_at,终态用 updated_at");
assertTrue(sql.contains("ORDER BY m.createdAt DESC,m.id DESC"));
assertTrue(sql.contains("LIMIT ? OFFSET ?"));
assertEquals(2, boundSql.getParameterMappings().size(), "只有分页两个参数");
// AC-19(P1-B1):页查询不再吃 OFFSET——深翻页的 skip 挪到 selectTaskPageAnchor 的窄投影里
assertFalse(sql.contains("OFFSET"), "页查询里再出现 OFFSET 就是深分页回归:" + sql);
assertTrue(sql.contains("LIMIT ?"));
assertEquals(1, boundSql.getParameterMappings().size(), "不带游标时只绑 limit");
assertFalse(sql.contains("request_json"), "列表主查询不拉大 JSON 列");
}
/** AC-19(P1-B1):页首游标锚点必须与页查询共用同一份 UNION,且只做两列窄投影。 */
@Test
void pageAnchorSharesTheUnionAndProjectsOnlyTheCursorColumns() {
BoundSql boundSql = render("selectTaskPageAnchor",
params("userIds", null, "moduleType", null, "statuses", null, "shop", null,
"startTime", LocalDateTime.of(2026, 9, 1, 0, 0), "endTime", null, "offset", 200L));
String sql = sqlOf(boundSql);
assertTrue(sql.startsWith("SELECT m.createdAt,m.id FROM(SELECT t.id AS id"),
"锚点只许投影游标两列,宽行装配是页查询的事:" + sql);
assertTrue(sql.contains("AND t.created_at >= ?"), "锚点必须与页查询同一套筛选,否则算出的是另一批行");
assertTrue(sql.contains("ORDER BY m.createdAt DESC,m.id DESC LIMIT 1 OFFSET ?"),
"排序键与页查询逐字相同,skip 只发生在这条窄语句里");
assertEquals(3, boundSql.getParameterMappings().size(),
"offset + 起止时间下界两个(锚点与页查询共用同一份 UNION,筛选照常绑)");
assertFalse(sql.contains("task_no"), "锚点不许带列表列");
}
/** AC-19(P1-B1):给了游标才渲染 keyset 条件,两个分支都被 created_at 下界收窄。 */
@Test
void cursorNarrowsBothBranchesInsteadOfSkippingMaterialisedRows() {
LocalDateTime anchor = LocalDateTime.of(2026, 10, 1, 12, 0);
BoundSql boundSql = render("selectTaskPage", pageParams(null, 20L, anchor, 8801L));
String sql = sqlOf(boundSql);
assertTrue(sql.contains("WHERE(m.createdAt < ? OR(m.createdAt = ? AND m.id < ?))"),
"游标必须落在 (createdAt,id) 这一对排序键上:" + sql);
assertEquals(4, boundSql.getParameterMappings().size(), "limit + 游标三个占位(afterCreatedAt 用两次)");
assertEquals("afterCreatedAt", boundSql.getParameterMappings().get(1).getProperty(),
"游标紧跟 limit 之后入参,顺序漂了说明绑的不是那对排序键");
}
@Test
void taskListFiltersReachBothBranchesAndBrandDropsShopFilter() {
BoundSql boundSql = render("selectTaskPage",
@@ -215,7 +251,8 @@ class TaskCenterQueryMapperSqlTest {
"statuses", List.of("RUNNING"), "shop", "旗舰店",
"startTime", LocalDateTime.of(2026, 9, 1, 0, 0),
"endTime", LocalDateTime.of(2026, 9, 28, 0, 0),
"limit", 20L, "offset", 20L));
"limit", 20L,
"afterCreatedAt", LocalDateTime.of(2026, 9, 20, 0, 0), "afterId", 4001L));
String sql = sqlOf(boundSql);
assertTrue(sql.contains("AND UPPER(t.module_type) = ?"));
@@ -236,13 +273,14 @@ class TaskCenterQueryMapperSqlTest {
assertFalse(sql.contains("LIKE CONCAT('%',?,'%')"), "店铺筛选按规范化精确匹配,模糊会命中同名前缀");
assertEquals(5L, boundSql.getAdditionalParameter("__frch_uid_0"));
assertEquals(16, boundSql.getParameterMappings().size(),
"biz 8(账号×2/业务/状态/起止/店铺×2)+ 品牌 6(账号×2/业务/状态/起止)+ 分页 2");
assertEquals(18, boundSql.getParameterMappings().size(),
"biz 8(账号×2/业务/状态/起止/店铺×2)+ 品牌 6(账号×2/业务/状态/起止)"
+ "+ AC-19 分页 4(limit + 游标三个占位,afterCreatedAt 用两次)");
}
@Test
void shopFilterDoesNotReadSnapshotScopeOrResultItemGroup() {
String filtered = sqlOf(render("selectTaskPage", taskPageParams("林芳", 20L, 0L)));
String filtered = sqlOf(render("selectTaskPage", pageParams("林芳", 20L, null, null)));
assertFalse(filtered.contains("biz_task_result_item"), "任务结果分组不是店铺来源");
assertFalse(filtered.contains("biz_task_progress_snapshot"), "进度快照不是店铺来源");
@@ -251,14 +289,14 @@ class TaskCenterQueryMapperSqlTest {
@Test
void taskListWithoutFiltersOnlyBindsPaging() {
BoundSql boundSql = render("selectTaskPage", taskPageParams(null, 100L, 0L));
BoundSql boundSql = render("selectTaskPage", pageParams(null, 100L, null, null));
String sql = sqlOf(boundSql);
assertFalse(sql.contains("AND UPPER(t.module_type)"), "未选业务不加条件");
assertFalse(sql.contains("AND 1 = 0"), "未选店铺时品牌分支照常入列");
assertFalse(sql.contains("EXISTS"), "未选店铺不做 EXISTS 子查询");
assertFalse(sql.contains("__frch"), "超管不渲染账号占位符");
assertEquals(2, boundSql.getParameterMappings().size());
assertEquals(1, boundSql.getParameterMappings().size(), "AC-19 之后只剩 limit 一个分页参数");
}
@Test
@@ -286,14 +324,14 @@ class TaskCenterQueryMapperSqlTest {
params("userIds", null, "moduleType", null,
"statuses", TaskCenterQueryMapper.ACTIVE_STATUSES_ORDERED,
"shop", null, "startTime", null, "endTime", null,
"limit", 20L, "offset", 0L));
"limit", 20L, "afterCreatedAt", null, "afterId", null));
String sql = sqlOf(boundSql);
assertTrue(sql.contains("AND " + TaskCenterQueryMapper.STATUS_T + " IN(?,?,?)"),
"biz 分支按展开后的三个状态过滤:" + sql);
assertTrue(sql.contains("AND " + TaskCenterQueryMapper.STATUS_B + " IN(?,?,?)"),
"品牌分支同口径(小写状态已由 STATUS_B 归一为大写)");
assertEquals(8, boundSql.getParameterMappings().size(), "两张表各绑三个状态 + 分页两个");
assertEquals(7, boundSql.getParameterMappings().size(), "两张表各绑三个状态 + AC-19 只剩 limit 一个");
assertEquals("PENDING", boundSql.getAdditionalParameter("__frch_st_0"));
assertEquals("RUNNING", boundSql.getAdditionalParameter("__frch_st_1"));
assertEquals("SCHEDULED", boundSql.getAdditionalParameter("__frch_st_2"));
@@ -12,7 +12,6 @@ import com.baomidou.mybatisplus.core.MybatisConfiguration;
import com.baomidou.mybatisplus.core.metadata.TableInfoHelper;
import org.apache.ibatis.builder.MapperBuilderAssistant;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
@@ -50,7 +49,6 @@ import static org.mockito.Mockito.when;
* <p>p95 不劣化要靠 Phase 5 的 EXPLAIN/基准前后对比(蓝图 §4 Phase 5 准出),
* 本文件钉的是"形状必须改成这样"的可判据部分。
*/
@Disabled("Pending Phase 5: AC-19 读路径时间窗/keyset、去 RAND、心跳瘦字段、菜单缓存")
@DisplayName("AC-19 高频读路径治理契约")
class PendingPhase5ReadPathTest {
@@ -15,6 +15,7 @@ import com.nanri.aiimage.modules.taskcenter.model.vo.TaskCenterOverviewVo;
import com.nanri.aiimage.modules.taskcenter.model.vo.TaskCenterTaskDetailVo;
import com.nanri.aiimage.modules.taskcenter.model.vo.TaskCenterTaskPageVo;
import com.nanri.aiimage.modules.taskcenter.model.vo.TaskCenterTaskRowVo;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.ArgumentCaptor;
@@ -39,6 +40,7 @@ import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.argThat;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.ArgumentMatchers.isNull;
import static org.mockito.ArgumentMatchers.notNull;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
@@ -166,8 +168,8 @@ class TaskCenterQueryServiceTest {
private void stubPage(List<TaskCenterTaskRow> rows) {
when(queryMapper.countTaskPage(any(), any(), any(), any(), any(), any()))
.thenReturn((long) rows.size());
when(queryMapper.selectTaskPage(any(), any(), any(), any(), any(), any(), anyLong(), anyLong()))
.thenReturn(rows);
when(queryMapper.selectTaskPage(any(), any(), any(), any(), any(), any(),
anyLong(), any(), any())).thenReturn(rows);
}
private void stubActiveTasks(List<TaskCenterTaskRow> rows) {
@@ -296,8 +298,9 @@ class TaskCenterQueryServiceTest {
TaskCenterTaskPageVo page = service.tasks(scoped(5L, 11L), query);
assertEquals(0, page.getTotal(), "只有 11 在本组可见集合内,99 越界");
// 第 5 个参数是 startTime:AC-19 之后不带时间参数的请求由服务层补默认窗口,不再是 null
verify(queryMapper).countTaskPage(argThat(ids -> List.of(11L).equals(ids)),
isNull(), isNull(), isNull(), isNull(), isNull());
isNull(), isNull(), isNull(), notNull(), isNull());
}
@Test
@@ -324,16 +327,21 @@ class TaskCenterQueryServiceTest {
service.tasks(superScope(), query);
verify(queryMapper).countTaskPage(argThat(ids -> List.of(11L, 99L).equals(ids)),
isNull(), isNull(), isNull(), isNull(), isNull());
isNull(), isNull(), isNull(), notNull(), isNull());
}
@Test
void filtersPassThroughAndPageSizeIsCapped() {
LocalDateTime start = LocalDateTime.of(2026, 9, 1, 0, 0);
LocalDateTime end = LocalDateTime.of(2026, 9, 28, 23, 59);
// AC-19:第 3 页的 skip 不再交给 SQL 的 OFFSET,而是先取页首游标(上一页最后一行)
LocalDateTime anchorTime = LocalDateTime.of(2026, 9, 20, 8, 30);
long anchorId = 770_003L;
when(queryMapper.countTaskPage(any(), any(), any(), any(), any(), any())).thenReturn(1L);
when(queryMapper.selectTaskPage(any(), any(), any(), any(), any(), any(), anyLong(), anyLong()))
.thenReturn(List.of());
when(queryMapper.selectTaskPageAnchor(any(), any(), any(), any(), any(), any(), anyLong()))
.thenReturn(taskRow(anchorId, "PUBLISH", "SUCCESS", anchorTime, anchorTime));
when(queryMapper.selectTaskPage(any(), any(), any(), any(), any(), any(),
anyLong(), any(), any())).thenReturn(List.of());
TaskCenterTaskQuery query = new TaskCenterTaskQuery();
query.setModuleType("PUBLISH");
@@ -348,8 +356,74 @@ class TaskCenterQueryServiceTest {
verify(queryMapper).countTaskPage(isNull(), eq("PUBLISH"), eq(List.of("RUNNING")),
eq("旗舰店"), eq(start), eq(end));
// (page-1)*pageSize 这条算术还在,只是它的去向从 OFFSET 变成了锚点行号
verify(queryMapper).selectTaskPageAnchor(isNull(), eq("PUBLISH"), eq(List.of("RUNNING")),
eq("旗舰店"), eq(start), eq(end), eq(200L));
verify(queryMapper).selectTaskPage(isNull(), eq("PUBLISH"), eq(List.of("RUNNING")),
eq("旗舰店"), eq(start), eq(end), eq(100L), eq(200L));
eq("旗舰店"), eq(start), eq(end), eq(100L), eq(anchorTime), eq(anchorId));
}
/**
* AC-19(审计 P1-B1):不带时间参数的请求由服务层补默认窗口,不许把 null 透传给全历史 UNION。
* 带了 startTime 就完全按请求走——翻旧账的出口不能被默认窗口吃掉。
*/
@Test
@DisplayName("AC-19:不带时间参数补默认窗口,带了就原样下传")
void missingStartTimeGetsDefaultWindowWhileExplicitRangeSurvives() {
when(queryMapper.countTaskPage(any(), any(), any(), any(), any(), any())).thenReturn(0L);
TaskCenterTaskQuery query = new TaskCenterTaskQuery();
service.tasks(superScope(), query);
ArgumentCaptor<LocalDateTime> window = ArgumentCaptor.forClass(LocalDateTime.class);
verify(queryMapper).countTaskPage(isNull(), isNull(), isNull(), isNull(),
window.capture(), isNull());
LocalDateTime floor = window.getValue();
assertFalse(floor.isAfter(LocalDateTime.now()), "默认窗口起点不许在未来:" + floor);
assertTrue(floor.isBefore(LocalDateTime.now().minusDays(30)),
"默认窗口太短会把在跑的长任务切掉:" + floor);
assertTrue(floor.isAfter(LocalDateTime.now().minusDays(180)),
"默认窗口太长就等于没有:" + floor);
LocalDateTime start = LocalDateTime.of(2025, 1, 1, 0, 0);
TaskCenterTaskQuery explicit = new TaskCenterTaskQuery();
explicit.setStartTime(start);
service.tasks(superScope(), explicit);
verify(queryMapper).countTaskPage(isNull(), isNull(), isNull(), isNull(), eq(start), isNull());
}
/** AC-19:第一页不许多跑一次锚点查询——看板每 30s 就落在第一页。 */
@Test
@DisplayName("AC-19:第一页不付锚点查询的钱")
void firstPageSkipsTheAnchorQuery() {
stubPage(List.of(taskRow(1L, "PUBLISH", "SUCCESS", LocalDateTime.now(), LocalDateTime.now())));
service.tasks(superScope(), new TaskCenterTaskQuery());
verify(queryMapper, never()).selectTaskPageAnchor(any(), any(), any(), any(), any(), any(), anyLong());
verify(queryMapper).selectTaskPage(any(), any(), any(), any(), any(), any(),
eq(20L), isNull(), isNull());
}
/** AC-19:页码越过窗口尾部时返回空页,不许拿 null 游标把第一页当第 N 页发回去。 */
@Test
@DisplayName("AC-19:页码越界返回空页而不是回落到第一页")
void pageBeyondWindowReturnsEmptyInsteadOfFirstPage() {
when(queryMapper.countTaskPage(any(), any(), any(), any(), any(), any())).thenReturn(3L);
when(queryMapper.selectTaskPageAnchor(any(), any(), any(), any(), any(), any(), anyLong()))
.thenReturn(null);
TaskCenterTaskQuery query = new TaskCenterTaskQuery();
query.setPage(999L);
TaskCenterTaskPageVo page = service.tasks(superScope(), query);
assertEquals(0L, page.getTotal(), "总数仍是窗口内的 3");
assertTrue(page.getItems() == null || page.getItems().isEmpty(),
"越界页必须空,把第一页塞进第 999 页是静默错数据");
verify(queryMapper, never()).selectTaskPage(any(), any(), any(), any(), any(), any(),
anyLong(), any(), any());
}
/** 前端默认项「在跑」= 三个进行中状态的合并:下传的是状态序列,不是字面量 ACTIVE。 */
@@ -362,7 +436,7 @@ class TaskCenterQueryServiceTest {
service.tasks(superScope(), query);
verify(queryMapper).countTaskPage(isNull(), isNull(),
eq(TaskCenterQueryMapper.ACTIVE_STATUSES_ORDERED), isNull(), isNull(), isNull());
eq(TaskCenterQueryMapper.ACTIVE_STATUSES_ORDERED), isNull(), notNull(), isNull());
assertEquals(Set.copyOf(TaskCenterQueryMapper.ACTIVE_STATUSES_ORDERED),
TaskCenterQueryMapper.ACTIVE_STATUS_SET, "「在跑」的合并口径必须与判卡住用的那组同源");
}
@@ -67,7 +67,6 @@ TaskCenterQueryMapper selectShopNames Select CONST <script>SELECT DISTINCT s.sho
TaskCenterQueryMapper selectSnapshots Select PURE <script>SELECT s.task_id AS taskId, s.module_type AS moduleType, s.total_count AS totalCount, s.success_count AS successCount, s.failed_count AS failedCount, s.pending_count AS pendingCount, s.current_scope_key AS currentScopeKey, s.message AS message FROM biz_task_progress_snapshot s WHERE s.task_id IN <foreach collection='taskIds' item='tid' open='(' separator=',' close=')'>#{tid}</foreach></script>
TaskCenterQueryMapper selectTaskByTaskNo Select CONST SELECT t.id AS id, t.task_no AS taskNo, UPPER(t.module_type) AS moduleType, CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END AS status, t.user_id AS userId, t.device_id AS deviceId, t.created_at AS createdAt, t.finished_at AS finishedAt, t.updated_at AS updatedAt, t.source_file_count AS totalCount, t.success_file_count AS successCount, t.failed_file_count AS failedCount, 0 AS brand, t.error_message AS errorMessage FROM biz_file_task t WHERE t.task_no = #{taskNo} AND t.module_type IN ('PUBLISH','DELETE_BRAND','PRODUCT_RISK_RESOLVE','PRICE_TRACK','SHOP_MATCH','PATROL_DELETE','QUERY_ASIN','WITHDRAW','APPEARANCE_PATENT','SIMILAR_ASIN','COLLECT_DATA','SHOP_DATA_CRAWL') LIMIT 1
TaskCenterQueryMapper selectTaskCountries Select CONST <script>SELECT t.id AS taskId, COALESCE(NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.filters.countryCode')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.countryCode')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.country')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.publish_country')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.countryCodes')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.country_codes')), ''), 'null'),NULLIF(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(t.request_json, '$.filters.countryCodes')), ''), 'null')) AS country FROM biz_file_task t WHERE t.id IN <foreach collection='taskIds' item='tid' open='(' separator=',' close=')'>#{tid}</foreach></script>
TaskCenterQueryMapper selectTaskPage Select CONST <script>SELECT m.* FROM (SELECT t.id AS id, t.task_no AS taskNo, UPPER(t.module_type) AS moduleType, CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END AS status, t.user_id AS userId, t.device_id AS deviceId, t.created_at AS createdAt, t.finished_at AS finishedAt, t.updated_at AS updatedAt, t.source_file_count AS totalCount, t.success_file_count AS successCount, t.failed_file_count AS failedCount, 0 AS brand FROM biz_file_task t WHERE t.module_type IN ('PUBLISH','DELETE_BRAND','PRODUCT_RISK_RESOLVE','PRICE_TRACK','SHOP_MATCH','PATROL_DELETE','QUERY_ASIN','WITHDRAW','APPEARANCE_PATENT','SIMILAR_ASIN','COLLECT_DATA','SHOP_DATA_CRAWL')<if test='userIds != null'> AND t.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND UPPER(t.module_type) = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(t.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(t.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND t.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND t.created_at &lt;= #{endTime}</if><if test='shop != null'> AND (EXISTS (SELECT 1 FROM biz_publish_file p WHERE p.task_id = t.id AND LOWER(TRIM(p.shop_name)) = LOWER(TRIM(#{shop}))) OR (t.module_type IN ('PATROL_DELETE','PRICE_TRACK','PRODUCT_RISK_RESOLVE','QUERY_ASIN','SHOP_DATA_CRAWL','SHOP_MATCH') AND EXISTS (SELECT 1 FROM biz_file_result f WHERE f.task_id = t.id AND LOWER(TRIM(f.source_filename)) = LOWER(TRIM(#{shop})))))</if> UNION ALL SELECT b.id, CONCAT('brand-', b.id), 'BRAND', CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END, b.user_id, b.device_id, b.created_at, CASE WHEN b.status IN ('cancelled','failed','success') THEN b.updated_at END, b.updated_at, b.progress_total, b.progress_current, 0, 1 FROM brand_crawl_tasks b WHERE 1 = 1<if test='userIds != null'> AND b.user_id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></if><if test='moduleType != null'> AND 'BRAND' = #{moduleType}</if><if test='statuses != null'> AND CASE WHEN UPPER(b.status) = 'CANCELED' THEN 'CANCELLED' ELSE UPPER(b.status) END IN <foreach collection='statuses' item='st' open='(' separator=',' close=')'>#{st}</foreach></if><if test='startTime != null'> AND b.created_at &gt;= #{startTime}</if><if test='endTime != null'> AND b.created_at &lt;= #{endTime}</if><if test='shop != null'> AND 1 = 0</if>) m ORDER BY m.createdAt DESC, m.id DESC LIMIT #{limit} OFFSET #{offset}</script>
TaskCenterQueryMapper selectUserIdsByUsername Select PURE SELECT id FROM users WHERE username LIKE CONCAT('%', #{keyword}, '%') LIMIT 500
TaskCenterQueryMapper selectUsernames Select PURE <script>SELECT id AS userId, username FROM users WHERE id IN <foreach collection='userIds' item='uid' open='(' separator=',' close=')'>#{uid}</foreach></script>
TaskResultItemMapper upsertBatch Insert PURE <script>\nINSERT INTO biz_task_result_item\n (task_id, module_type, result_id, scope_key, scope_hash, item_key, asin, status,\n payload_json, payload_hash, created_at, updated_at)\nVALUES\n<foreach collection="rows" item="row" separator=",">\n (#{row.taskId}, #{row.moduleType}, #{row.resultId}, #{row.scopeKey}, #{row.scopeHash},\n #{row.itemKey}, #{row.asin}, #{row.status}, #{row.payloadJson}, #{row.payloadHash},\n #{row.createdAt}, #{row.updatedAt})\n</foreach>\nON DUPLICATE KEY UPDATE\n result_id = VALUES(result_id),\n scope_key = VALUES(scope_key),\n asin = VALUES(asin),\n status = VALUES(status),\n payload_json = VALUES(payload_json),\n payload_hash = VALUES(payload_hash),\n updated_at = VALUES(updated_at)\n</script>
Can't render this file because it contains an unexpected character in line 1 and column 222.