test(phase-0): 1B 四支巨石现网行为快照补齐——建单/解析/载荷三段打进真服务

重构蓝图 Phase 0 · 1B:§1B 那句「铺到缺的四支」今天补齐——deletebrand/shopmatch/
pricetrack/collectdata 各一支 *SnapshotTest + golden .txt,沿用仓内已在用的约定
(modules/<域>/service/support/*SnapshotTest 读 src/test/resources/<域>/golden/*.txt 整串对撞,
带 output/reproducible/golden_committed/diff_detected 四件套)。

与旧四支的**关键差别**:快照打的是真服务入口(run/createTask/parseAndCreateTask 一路走到各支
私有内联上下文 DeleteBrandSheetContext 等),不是在测试里照抄一遍管线——照抄的那支对 Phase 3
迁移毫无保护(旧代码消灭了,测试还绿)。落库侧用 Mockito doAnswer 捕获 mapper 的 insert/update
入参、主键顺序发固定值,回传载荷侧捕获 saveParsedPayload/saveLatest;时钟、雪花 id、临时目录
绝对路径一律不进快照(后者擦成 <root>/<outside>),所以 golden 跨 JVM 逐字节可比。

钉到的真实形状(举其要):
- deletebrand:解析成功但店铺未匹配 ⇒ 整任务 FAILED+「全部文件解析或匹配失败」;表头行被
  dataStartRowIndex=1 那条分支当数据行吃进去(ASIN="ASIN");BOM/全角归一化、按归一化键去重、
  状态有值而 ASIN 空 ⇒ 丢行;非 Excel 魔数拒绝。
- shopmatch:国家代码只认 ProductRiskCountryCode 五个(**US 不在内**,报「非法国家代码: US」);
  心跳只在 RUNNING 打,SCHEDULED 一支不打;建单校验全在 insert 之前 ⇒ 拒绝时一行库都没碰。
- pricetrack:模式二选一、ASIN 文件三种路径形态与「必须落在上传临时目录内」的穿越校验、
  表头别名(商品ASIN/最低价→asin/minimumPrice)、按 sheet 名认国家、单国家时 sheet 名兜底。
- collectdata:无关键词列则一条不丢、关键词为空丢行计数、尾部空列不生成占位列(2026-09-26
  生产事故那条同源修复)、extraJson 形状、PENDING 建单。

两处当场抓到并改掉的量具缺陷(不改就等于假绿):
1. 替身里写 `String.valueOf(invocation.getArgument(0))` 会因泛型推断挑中 valueOf(char[]) 重载,
   运行时 String 强转 [C 抛 ClassCastException,把五格全打成解析失败——已改成显式 String 变量。
2. Spring 注入的 ObjectMapper 不等于 `new ObjectMapper()`:Boot 会注册 classpath 上的
   jackson-datatype-jsr310。裸 mapper 下 shopmatch 的 scheduleTimes(List<LocalDateTime>)
   序列化失败、被建单段 catch(Exception) 吞成「task 不存在」⇒ 快照会锁在生产不存在的假形状上。

门禁:`mvn -o test` 全量 4417 passed/0 failures/7 skipped(12:42,BUILD SUCCESS);
四支新用例 17 passed;ops `pytest scripts/tests` 500 passed、`pt_worker/tests` 15 passed、
`ruff check .` All checks passed;格式步本项目未纳管 ⇒ 跳过并如实登记。
This commit is contained in:
2026-10-18 11:52:54 +08:00
parent 3e797ab159
commit 660a1b7084
8 changed files with 1632 additions and 0 deletions
@@ -0,0 +1,328 @@
package com.nanri.aiimage.modules.collectdata.service.support;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.nanri.aiimage.modules.collectdata.mapper.CollectDataCountryPrefMapper;
import com.nanri.aiimage.modules.collectdata.mapper.CollectDataItemMapper;
import com.nanri.aiimage.modules.collectdata.model.dto.CollectDataParseRequest;
import com.nanri.aiimage.modules.collectdata.model.dto.CollectDataSourceFileDto;
import com.nanri.aiimage.modules.collectdata.model.entity.CollectDataItemEntity;
import com.nanri.aiimage.modules.collectdata.model.vo.CollectDataParseVo;
import com.nanri.aiimage.modules.collectdata.service.CollectDataBrandRecheckService;
import com.nanri.aiimage.modules.collectdata.service.CollectDataService;
import com.nanri.aiimage.modules.collectdata.util.CollectDataBatchQuery;
import com.nanri.aiimage.modules.collectdata.util.CollectDataBrandBatchFilter;
import com.nanri.aiimage.modules.collectdata.util.CollectDataInvalidAsinBatchWriter;
import com.nanri.aiimage.modules.collectdata.util.CollectDataResultDetailCodec;
import com.nanri.aiimage.modules.collectdata.util.CollectDataResultDetailReader;
import com.nanri.aiimage.modules.collectdata.util.CollectDataResultItemBatchWriter;
import com.nanri.aiimage.modules.file.service.LocalFileStorageService;
import com.nanri.aiimage.modules.invalidasin.mapper.InvalidAsinDataMapper;
import com.nanri.aiimage.modules.task.mapper.FileResultMapper;
import com.nanri.aiimage.modules.task.mapper.FileTaskMapper;
import com.nanri.aiimage.modules.task.mapper.TaskChunkMapper;
import com.nanri.aiimage.modules.task.mapper.TaskResultItemMapper;
import com.nanri.aiimage.modules.task.mapper.TaskScopeStateMapper;
import com.nanri.aiimage.modules.task.model.entity.FileResultEntity;
import com.nanri.aiimage.modules.task.model.entity.FileTaskEntity;
import com.nanri.aiimage.modules.task.service.TaskDistributedLockService;
import com.nanri.aiimage.modules.task.service.TaskFileJobService;
import com.nanri.aiimage.modules.task.service.TaskProgressLightAssembler;
import com.nanri.aiimage.modules.file.service.oss.OssStorageService;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.junit.jupiter.api.Test;
import org.mockito.Answers;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.support.TransactionTemplate;
import java.io.File;
import java.io.FileOutputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyList;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.doAnswer;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
/**
* 集采建单+解析段的现网行为快照(重构蓝图 Phase 0 · 1B,缺快照的四支之四)。
*
* 快照打的是真 {@link CollectDataService#parseAndCreateTask}:表头别名认列(关键词/状态)、
* 无关键词列时「一条都不丢」、关键词为空丢行计数、尾部空列不生成占位列(2026-09-26 生产事故
* 那条同源修复)、非 Excel 魔数拒绝、{@code taskType} 缺省回落、{@code PENDING} 建单形状,
* 以及逐行 {@code CollectDataItemEntity} 批量落库的 extraJson 形状(Phase 3 迁移后必须仍逐字节相等)。
*
* 时钟与雪花 taskNo 不进快照;item 主键由 doAnswer 顺序发固定值。
*
* golden 文件:src/test/resources/collectdata/golden/parse-create-snapshot.txt
*/
class CollectDataSnapshotTest {
private static final File GOLDEN_PARSE =
new File("src/test/resources/collectdata/golden/parse-create-snapshot.txt");
// ---- 夹具 ----
private static File workbook(File dir, String name, List<List<String>> rows) throws Exception {
File file = new File(dir, name);
try (XSSFWorkbook wb = new XSSFWorkbook(); FileOutputStream out = new FileOutputStream(file)) {
Sheet sheet = wb.createSheet("Sheet1");
for (int r = 0; r < rows.size(); r++) {
Row row = sheet.createRow(r);
List<String> cells = rows.get(r);
for (int c = 0; c < cells.size(); c++) {
row.createCell(c).setCellValue(cells.get(c));
}
}
wb.write(out);
}
return file;
}
private static File keywordWorkbook(File dir) throws Exception {
return workbook(dir, "keyword.xlsx", List.of(
List.of("关键词", "asin", "国家", "状态", "价格"),
List.of("wirel less 耳机", "B011", "US", "在售", "19.90"),
List.of("", "B012", "US", "", "9.90"),
List.of("", "", "", "", ""),
List.of(" mouse ", "", "DE", "缺少报价", "")));
}
/** 没有关键词列也没有状态列:一条非空行都不该被丢。 */
private static File noKeywordHeaderWorkbook(File dir) throws Exception {
return workbook(dir, "no-keyword.xlsx", List.of(
List.of("asin", "国家", "价格"),
List.of("B021", "UK", "5.00"),
List.of("B022", "UK", "")));
}
/** 表头整行空白 ⇒「Excel 表头为空」。 */
private static File blankHeaderWorkbook(File dir) throws Exception {
return workbook(dir, "blank-header.xlsx", List.of(
List.of("", "", ""),
List.of("B031", "US", "1.00")));
}
/** 关键词列存在但每行关键词都空 ⇒ 有效行 0 ⇒「未解析到有效数据行」。 */
private static File allDroppedWorkbook(File dir) throws Exception {
return workbook(dir, "all-dropped.xlsx", List.of(
List.of("关键词", "asin"),
List.of("", "B041"),
List.of("", "B042")));
}
private static File notExcel(File dir) throws Exception {
File file = new File(dir, "fake.xlsx");
Files.writeString(file.toPath(), "关键词,asin\nwireless,B0001\n", StandardCharsets.UTF_8);
return file;
}
// ---- 真服务 + 捕获 ----
private static final class Captured {
private final List<FileTaskEntity> taskInserts = new ArrayList<>();
private final List<CollectDataItemEntity> items = new ArrayList<>();
private final List<FileResultEntity> resultInserts = new ArrayList<>();
}
private static CollectDataService newService(LocalFileStorageService storage, Captured captured) {
FileTaskMapper fileTaskMapper = mock(FileTaskMapper.class);
FileResultMapper fileResultMapper = mock(FileResultMapper.class);
CollectDataItemMapper itemMapper = mock(CollectDataItemMapper.class);
ObjectMapper objectMapper = new ObjectMapper();
doAnswer(invocation -> {
FileTaskEntity entity = invocation.getArgument(0);
entity.setId(1001L + captured.taskInserts.size());
captured.taskInserts.add(entity);
return 1;
}).when(fileTaskMapper).insert(any(FileTaskEntity.class));
doAnswer(invocation -> {
List<CollectDataItemEntity> batch = invocation.getArgument(0);
for (CollectDataItemEntity entity : batch) {
entity.setId(3001L + captured.items.size());
captured.items.add(entity);
}
return batch.size();
}).when(itemMapper).insertBatch(anyList());
doAnswer(invocation -> {
FileResultEntity entity = invocation.getArgument(0);
entity.setId(2001L + captured.resultInserts.size());
captured.resultInserts.add(entity);
return 1;
}).when(fileResultMapper).insert(any(FileResultEntity.class));
return new CollectDataService(storage, fileTaskMapper, fileResultMapper, itemMapper,
mock(CollectDataCountryPrefMapper.class), mock(InvalidAsinDataMapper.class),
mock(TaskChunkMapper.class), mock(TaskScopeStateMapper.class), mock(TaskResultItemMapper.class),
mock(TaskDistributedLockService.class), mock(TaskFileJobService.class),
mock(com.nanri.aiimage.modules.task.service.TaskHeartbeatPositionService.class),
mock(com.nanri.aiimage.modules.task.service.TransientPayloadStorageService.class),
mock(com.nanri.aiimage.modules.collectdata.service.CollectDataExcelAssemblyService.class),
mock(OssStorageService.class), objectMapper,
new TransactionTemplate(mock(PlatformTransactionManager.class)),
mock(com.nanri.aiimage.common.service.DistributedJobLockService.class, Answers.RETURNS_MOCKS),
mock(CollectDataBatchQuery.class), mock(CollectDataBrandBatchFilter.class),
mock(CollectDataInvalidAsinBatchWriter.class), new CollectDataResultDetailCodec(objectMapper),
mock(CollectDataResultItemBatchWriter.class), mock(CollectDataResultDetailReader.class),
mock(TaskProgressLightAssembler.class),
mock(com.nanri.aiimage.modules.notification.service.NotificationDispatchService.class),
mock(com.nanri.aiimage.modules.asincache.service.AsinResultCacheService.class),
mock(CollectDataBrandRecheckService.class),
mock(com.nanri.aiimage.modules.task.service.StaleTaskHeartbeatPolicy.class));
}
private static CollectDataParseRequest request(List<CollectDataSourceFileDto> files, String taskType) {
CollectDataParseRequest request = new CollectDataParseRequest();
request.setUserId(7L);
request.setFiles(files);
request.setTaskType(taskType);
return request;
}
private static CollectDataSourceFileDto source(String fileKey, String filename) {
CollectDataSourceFileDto dto = new CollectDataSourceFileDto();
dto.setFileKey(fileKey);
dto.setOriginalFilename(filename);
return dto;
}
private static String nullToDash(Object value) {
String text = value == null ? "" : String.valueOf(value);
return text.isBlank() ? "-" : text;
}
private static String runCase(String caseName, Map<String, File> filesByFileKey, CollectDataParseRequest request) {
Captured captured = new Captured();
LocalFileStorageService storage = mock(LocalFileStorageService.class);
when(storage.findLocalSourceFile(anyString())).thenAnswer(invocation -> filesByFileKey.get(invocation.getArgument(0)));
CollectDataService service = newService(storage, captured);
StringBuilder sb = new StringBuilder();
sb.append("== case=").append(caseName).append('\n');
try {
CollectDataParseVo vo = service.parseAndCreateTask(request, "dev-1");
sb.append("vo: taskId=").append(nullToDash(vo.getTaskId()))
.append(" sourceFilename=").append(nullToDash(vo.getSourceFilename()))
.append(" sourceFileCount=").append(nullToDash(vo.getSourceFileCount()))
.append(" totalRows=").append(nullToDash(vo.getTotalRows()))
.append(" acceptedRows=").append(nullToDash(vo.getAcceptedRows()))
.append(" droppedRows=").append(nullToDash(vo.getDroppedRows()))
.append(" pageSize=").append(nullToDash(vo.getPageSize()))
.append('\n');
} catch (Exception ex) {
sb.append("thrown: ").append(ex.getClass().getSimpleName())
.append(" message=").append(nullToDash(ex.getMessage())).append('\n');
}
for (FileTaskEntity task : captured.taskInserts) {
sb.append("task.insert: id=").append(task.getId())
.append(" moduleType=").append(nullToDash(task.getModuleType()))
.append(" taskMode=").append(nullToDash(task.getTaskMode()))
.append(" status=").append(nullToDash(task.getStatus()))
.append(" sourceFileCount=").append(nullToDash(task.getSourceFileCount()))
.append(" deviceId=").append(nullToDash(task.getDeviceId()))
.append(" requestJsonTaskTypeDefault=").append(task.getRequestJson() != null
&& task.getRequestJson().contains("\"taskType\":\"collect-data\""))
.append(" requestJsonTaskTypeExplicit=").append(task.getRequestJson() != null
&& task.getRequestJson().contains("\"taskType\":\"collect-data-brand\""))
.append(" resultJsonHasFinalRowCount=").append(task.getResultJson() != null
&& task.getResultJson().contains("finalRowCount"))
.append('\n');
}
for (CollectDataItemEntity item : captured.items) {
sb.append("item: id=").append(item.getId())
.append(" taskId=").append(nullToDash(item.getTaskId()))
.append(" rowIndex=").append(nullToDash(item.getRowIndex()))
.append(" sourceFileKey=").append(nullToDash(item.getSourceFileKey()))
.append(" sourceFilename=").append(nullToDash(item.getSourceFilename()))
.append(" keyword=").append(nullToDash(item.getKeyword()))
.append(" statusValue=").append(nullToDash(item.getStatusValue()))
.append(" extraJson=").append(nullToDash(item.getExtraJson()))
.append('\n');
}
for (FileResultEntity result : captured.resultInserts) {
sb.append("result.insert: id=").append(result.getId())
.append(" taskId=").append(nullToDash(result.getTaskId()))
.append(" moduleType=").append(nullToDash(result.getModuleType()))
.append(" sourceFilename=").append(nullToDash(result.getSourceFilename()))
.append(" sourceFileUrl=").append(nullToDash(result.getSourceFileUrl()))
.append(" rowCount=").append(nullToDash(result.getRowCount()))
.append('\n');
}
return sb.toString();
}
public static String runAll() throws Exception {
File dir = Files.createTempDirectory("collect-data-snapshot-").toFile();
dir.deleteOnExit();
File keyword = keywordWorkbook(dir);
File noKeyword = noKeywordHeaderWorkbook(dir);
File blankHeader = blankHeaderWorkbook(dir);
File allDropped = allDroppedWorkbook(dir);
File fake = notExcel(dir);
StringBuilder sb = new StringBuilder();
sb.append(runCase("keywordSheet", Map.of("fk-keyword", keyword),
request(List.of(source("fk-keyword", "keyword.xlsx")), null)));
sb.append(runCase("explicitTaskType", Map.of("fk-keyword", keyword),
request(List.of(source("fk-keyword", "keyword.xlsx")), "collect-data-brand")));
sb.append(runCase("noKeywordHeader", Map.of("fk-no-keyword", noKeyword),
request(List.of(source("fk-no-keyword", "no-keyword.xlsx")), null)));
sb.append(runCase("twoFiles", Map.of("fk-keyword", keyword, "fk-no-keyword", noKeyword),
request(List.of(source("fk-keyword", "keyword.xlsx"), source("fk-no-keyword", "no-keyword.xlsx")), null)));
sb.append(runCase("blankHeader", Map.of("fk-blank", blankHeader),
request(List.of(source("fk-blank", "blank-header.xlsx")), null)));
sb.append(runCase("allRowsDropped", Map.of("fk-dropped", allDropped),
request(List.of(source("fk-dropped", "all-dropped.xlsx")), null)));
sb.append(runCase("notExcel", Map.of("fk-fake", fake),
request(List.of(source("fk-fake", "fake.xlsx")), null)));
sb.append(runCase("missingSourceFile", Map.of(),
request(List.of(source("fk-absent", "absent.xlsx")), null)));
sb.append(runCase("blankFileKeyFiltered", Map.of("fk-keyword", keyword),
request(List.of(source(" ", "blank.xlsx")), null)));
sb.append(runCase("emptyFiles", Map.of(), request(List.of(), null)));
return sb.toString();
}
private static String read(File file) throws Exception {
return Files.readString(file.toPath(), StandardCharsets.UTF_8).replace("\r\n", "\n");
}
@Test
void parse_create_snapshot_matches_golden() throws Exception {
assertEquals(read(GOLDEN_PARSE), runAll(), "建单+解析输出快照与 golden 一致");
}
@Test
void parse_create_snapshot_is_reproducible() throws Exception {
assertEquals(runAll(), runAll(), "同一夹具跑两次结果一致(时钟与雪花 taskNo 不得进快照)");
}
@Test
void parse_create_snapshot_golden_is_committed() {
assertTrue(GOLDEN_PARSE.isFile(), "golden 文件必须存在并入库: " + GOLDEN_PARSE.getAbsolutePath());
assertTrue(GOLDEN_PARSE.length() > 0, "golden 文件非空");
}
@Test
void parse_create_snapshot_tamper_is_detected() throws Exception {
String original = read(GOLDEN_PARSE);
assertTrue(original.contains("== case=keywordSheet"), "golden 内容合法");
try {
Files.writeString(GOLDEN_PARSE.toPath(), original + "\n# tampered", StandardCharsets.UTF_8);
assertTrue(!read(GOLDEN_PARSE).equals(runAll()), "篡改 golden 后断言应失败");
} finally {
Files.writeString(GOLDEN_PARSE.toPath(), original, StandardCharsets.UTF_8);
}
assertEquals(original, read(GOLDEN_PARSE), "恢复原始 golden");
}
}
@@ -0,0 +1,394 @@
package com.nanri.aiimage.modules.deletebrand.service.support;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.nanri.aiimage.config.DeleteBrandProgressProperties;
import com.nanri.aiimage.config.TaskPressureProperties;
import com.nanri.aiimage.modules.deletebrand.model.cache.DeleteBrandParsedFileCacheDto;
import com.nanri.aiimage.modules.deletebrand.model.dto.DeleteBrandRunRequest;
import com.nanri.aiimage.modules.deletebrand.model.dto.DeleteBrandSourceFileDto;
import com.nanri.aiimage.modules.deletebrand.model.vo.DeleteBrandCountryAsinVo;
import com.nanri.aiimage.modules.deletebrand.model.vo.DeleteBrandCountryGroupVo;
import com.nanri.aiimage.modules.deletebrand.model.vo.DeleteBrandPreviewRowVo;
import com.nanri.aiimage.modules.deletebrand.model.vo.DeleteBrandResultItemVo;
import com.nanri.aiimage.modules.deletebrand.model.vo.DeleteBrandRunVo;
import com.nanri.aiimage.modules.deletebrand.service.DeleteBrandRunService;
import com.nanri.aiimage.modules.deletebrand.service.DeleteBrandTaskCacheService;
import com.nanri.aiimage.modules.deletebrand.service.DeleteBrandTaskStorageService;
import com.nanri.aiimage.modules.file.service.LocalFileStorageService;
import com.nanri.aiimage.modules.file.service.oss.OssStorageService;
import com.nanri.aiimage.modules.task.mapper.FileResultMapper;
import com.nanri.aiimage.modules.task.mapper.FileTaskMapper;
import com.nanri.aiimage.modules.task.model.entity.FileResultEntity;
import com.nanri.aiimage.modules.task.model.entity.FileTaskEntity;
import com.nanri.aiimage.modules.task.service.TaskDistributedLockService;
import com.nanri.aiimage.modules.task.service.TaskFileJobService;
import com.nanri.aiimage.modules.task.service.TaskProgressLightAssembler;
import com.nanri.aiimage.modules.ziniao.model.vo.ZiniaoShopMatchResultVo;
import com.nanri.aiimage.modules.ziniao.service.ZiniaoShopSwitchService;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.junit.jupiter.api.Test;
import org.springframework.transaction.PlatformTransactionManager;
import java.io.File;
import java.io.FileOutputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyBoolean;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.doAnswer;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
/**
* 删除品牌建单+解析段的现网行为快照(重构蓝图 Phase 0 · 1B,缺快照的四支之一)。
*
* 与 {@code AppearancePatentSnapshotTest} 那套约定的区别,也是这一支的价值所在:
* 快照打的是**真服务**({@link DeleteBrandRunService#run} 一路走到私有内联上下文
* {@code DeleteBrandSheetContext}),不是在测试里照抄一遍管线——Phase 3 要动的正是这条链,
* 测试里重写的管线会跟着旧代码一起留在原地,对迁移毫无保护。
* 落库侧用 Mockito 捕获 {@code fileTaskMapper}/{@code fileResultMapper} 的 insert/update 入参,
* 回传载荷侧捕获 {@code saveParsedPayload} 的 map(Phase 3 迁移后必须逐字节仍相等)。
*
* 时钟/雪花 id 不进快照:taskNo、createdAt/updatedAt/finishedAt 与 requestJson/resultJson 一律不渲染,
* 主键由 doAnswer 按调用顺序发固定值(1001/2001…),所以整串输出可复现。
*
* golden 文件:src/test/resources/deletebrand/golden/run-snapshot.txt
*/
class DeleteBrandSnapshotTest {
private static final File GOLDEN_RUN =
new File("src/test/resources/deletebrand/golden/run-snapshot.txt");
// ---- 夹具 ----
/** 两行表头(国家标题行 + 字段行)+ 数据行;BOM/全角空格/大小写/去重都在这支里吃到。 */
private static File fixtureMain() throws Exception {
return workbook(List.of(
List.of("美国", "", "德国", ""),
List.of("删除ASIN", "状态", "删除ASIN", "状态"),
List.of("b011 1", "在售", "B021", ""),
List.of("B011 1", "已删除", "", "状态值不该单独成行"),
List.of("B012", "", "B022", "在售"),
List.of("", "", "", "")));
}
/** 只有一行表头(没有字段行)⇒ 与旧实现一致的「Excel 表头不完整」。 */
private static File fixtureHeaderOnly() throws Exception {
return workbook(List.of(List.of("美国", "状态")));
}
/**
* 「店铺名」列被标题字面排除、其余列走 dataStartRowIndex=1 那条分支:
* 表头行自身会被当数据行吃进去(ASIN="ASIN"、状态="状态")——今天真实存在的行为,
* Phase 3 把解析段抽走时这条必须仍然相等,所以进快照而不是修掉。
*/
private static File fixtureSingleHeaderRowShape() throws Exception {
return workbook(List.of(
List.of("店铺名", "英国", ""),
List.of("负责人", "ASIN", "状态"),
List.of("张三", "B031", "在售"),
List.of("李四", "B032", "缺少报价")));
}
/** 后缀是 xlsx、内容却是 CSV 文本 ⇒ 魔数拦下,不许被 EasyExcel 静默当 CSV 解析成功。 */
private static File fixtureNotExcel() throws Exception {
File file = File.createTempFile("delete-brand-snapshot-not-excel-", ".xlsx");
Files.writeString(file.toPath(), "国家,ASIN\nUS,B0001\n", StandardCharsets.UTF_8);
return file;
}
private static File workbook(List<List<String>> rows) throws Exception {
File file = File.createTempFile("delete-brand-snapshot-", ".xlsx");
try (XSSFWorkbook wb = new XSSFWorkbook(); FileOutputStream out = new FileOutputStream(file)) {
Sheet sheet = wb.createSheet("Sheet1");
for (int r = 0; r < rows.size(); r++) {
Row row = sheet.createRow(r);
for (int c = 0; c < rows.get(r).size(); c++) {
row.createCell(c).setCellValue(rows.get(r).get(c));
}
}
wb.write(out);
}
return file;
}
// ---- 真服务 + 捕获 ----
private static final class Captured {
private final List<FileTaskEntity> taskInserts = new ArrayList<>();
private final List<FileTaskEntity> taskUpdates = new ArrayList<>();
private final List<FileResultEntity> resultInserts = new ArrayList<>();
private final Map<String, DeleteBrandParsedFileCacheDto> payload = new LinkedHashMap<>();
}
/** 每个夹具一个「店铺名→紫鸟匹配结果」开关:未匹配的店铺解析成功但不进待跑清单。 */
private static DeleteBrandRunService newService(Map<String, File> filesByFileKey,
Captured captured,
List<String> matchedShops) {
FileTaskMapper fileTaskMapper = mock(FileTaskMapper.class);
FileResultMapper fileResultMapper = mock(FileResultMapper.class);
LocalFileStorageService storage = mock(LocalFileStorageService.class);
DeleteBrandTaskCacheService cacheService = mock(DeleteBrandTaskCacheService.class);
DeleteBrandTaskStorageService taskStorage = mock(DeleteBrandTaskStorageService.class);
ZiniaoShopSwitchService ziniao = mock(ZiniaoShopSwitchService.class);
OssStorageService oss = mock(OssStorageService.class);
// ObjectMapper 用真身:mock 它会让 toCompactTaskResultJsonForDb 静默失败(clone is null),
// 快照就锁在了「resultJson 组装失败也能建单」这个假形状上。
ObjectMapper objectMapper = new ObjectMapper();
DeleteBrandProgressProperties progressProperties = mock(DeleteBrandProgressProperties.class);
TaskPressureProperties pressureProperties = mock(TaskPressureProperties.class);
TaskDistributedLockService lockService = mock(TaskDistributedLockService.class);
TaskFileJobService taskFileJobService = mock(TaskFileJobService.class);
PlatformTransactionManager transactionManager = mock(PlatformTransactionManager.class);
TaskProgressLightAssembler progressLightAssembler = mock(TaskProgressLightAssembler.class);
doAnswer(invocation -> {
FileTaskEntity entity = invocation.getArgument(0);
entity.setId(1001L + captured.taskInserts.size());
captured.taskInserts.add(entity);
return 1;
}).when(fileTaskMapper).insert(any(FileTaskEntity.class));
doAnswer(invocation -> {
captured.taskUpdates.add(invocation.getArgument(0));
return 1;
}).when(fileTaskMapper).updateById(any(FileTaskEntity.class));
doAnswer(invocation -> {
FileResultEntity entity = invocation.getArgument(0);
entity.setId(2001L + captured.resultInserts.size());
captured.resultInserts.add(entity);
return 1;
}).when(fileResultMapper).insert(any(FileResultEntity.class));
when(storage.findLocalSourceFile(anyString()))
.thenAnswer(invocation -> filesByFileKey.get(invocation.getArgument(0)));
// 注意别写 `String.valueOf(invocation.getArgument(0))`:泛型 T 会推断成 char[],
// 编译期挑中 valueOf(char[]) 重载,运行时 String 强转 [C 直接抛 ClassCastException。
when(ziniao.normalizeShopName(any())).thenAnswer(invocation -> {
String shopName = invocation.getArgument(0);
return shopName == null ? "" : shopName;
});
when(ziniao.findIndexedStoreByName(anyString(), anyBoolean())).thenAnswer(invocation -> {
String shopName = invocation.getArgument(0);
ZiniaoShopMatchResultVo result = new ZiniaoShopMatchResultVo();
if (!matchedShops.contains(shopName)) {
result.setMatched(false);
result.setMatchStatus("NOT_FOUND");
result.setMatchMessage("店铺未匹配");
return result;
}
result.setMatched(true);
result.setMatchStatus("MATCHED");
result.setShopId("shop-" + shopName);
result.setCompanyName("公司-" + shopName);
result.setPlatform("amazon");
result.setOpenStoreUrl("https://example.invalid/" + shopName);
return result;
});
doAnswer(invocation -> {
@SuppressWarnings("unchecked")
Map<String, DeleteBrandParsedFileCacheDto> payload = invocation.getArgument(1);
captured.payload.putAll(payload);
return null;
}).when(taskStorage).saveParsedPayload(any(), any());
return new DeleteBrandRunService(fileTaskMapper, fileResultMapper, storage, cacheService,
taskStorage, ziniao, oss, objectMapper, progressProperties, pressureProperties,
lockService, taskFileJobService, transactionManager, progressLightAssembler);
}
private static DeleteBrandRunRequest request(List<String> fileKeys) {
DeleteBrandRunRequest request = new DeleteBrandRunRequest();
request.setUserId(7L);
List<DeleteBrandSourceFileDto> files = new ArrayList<>();
for (String fileKey : fileKeys) {
DeleteBrandSourceFileDto sourceFile = new DeleteBrandSourceFileDto();
sourceFile.setFileKey(fileKey);
sourceFile.setOriginalFilename(fileKey.substring(fileKey.lastIndexOf('/') + 1));
files.add(sourceFile);
}
request.setFiles(files);
return request;
}
// ---- 快照渲染 ----
private static String nullToDash(Object value) {
String text = value == null ? "" : String.valueOf(value);
return text.isBlank() ? "-" : text;
}
private static String runCase(String caseName, Map<String, File> filesByFileKey,
List<String> fileKeys, List<String> matchedShops) {
Captured captured = new Captured();
DeleteBrandRunService service = newService(filesByFileKey, captured, matchedShops);
DeleteBrandRunVo vo = service.run(request(fileKeys), "dev-1");
StringBuilder sb = new StringBuilder();
sb.append("== case=").append(caseName).append(" files=").append(fileKeys.size()).append('\n');
for (FileTaskEntity task : captured.taskInserts) {
sb.append("task.insert: id=").append(task.getId())
.append(" moduleType=").append(nullToDash(task.getModuleType()))
.append(" status=").append(nullToDash(task.getStatus()))
.append(" taskMode=").append(nullToDash(task.getTaskMode()))
.append(" userId=").append(nullToDash(task.getUserId()))
.append(" createdBy=").append(nullToDash(task.getCreatedBy()))
.append(" deviceId=").append(nullToDash(task.getDeviceId()))
.append(" sourceFileCount=").append(nullToDash(task.getSourceFileCount()))
.append('\n');
}
for (FileTaskEntity task : captured.taskUpdates) {
sb.append("task.update: id=").append(task.getId())
.append(" status=").append(nullToDash(task.getStatus()))
.append(" sourceFileCount=").append(nullToDash(task.getSourceFileCount()))
.append(" successFileCount=").append(nullToDash(task.getSuccessFileCount()))
.append(" failedFileCount=").append(nullToDash(task.getFailedFileCount()))
.append(" errorMessage=").append(nullToDash(task.getErrorMessage()))
.append('\n');
}
for (FileResultEntity result : captured.resultInserts) {
sb.append("result.insert: id=").append(result.getId())
.append(" taskId=").append(nullToDash(result.getTaskId()))
.append(" moduleType=").append(nullToDash(result.getModuleType()))
.append(" sourceFileUrl=").append(nullToDash(result.getSourceFileUrl()))
.append(" rowCount=").append(nullToDash(result.getRowCount()))
.append(" success=").append(nullToDash(result.getSuccess()))
.append(" resultFilename=").append(nullToDash(result.getResultFilename()))
.append(" errorMessage=").append(nullToDash(result.getErrorMessage()))
.append('\n');
}
sb.append("vo: total=").append(vo.getTotal())
.append(" successCount=").append(vo.getSuccessCount())
.append(" failedCount=").append(vo.getFailedCount()).append('\n');
for (DeleteBrandResultItemVo item : vo.getItems()) {
sb.append("item: resultId=").append(nullToDash(item.getResultId()))
.append(" fileKey=").append(nullToDash(item.getFileKey()))
.append(" shopName=").append(nullToDash(item.getShopName()))
.append(" matched=").append(item.isMatched())
.append(" matchStatus=").append(nullToDash(item.getMatchStatus()))
.append(" countryCount=").append(nullToDash(item.getCountryCount()))
.append(" totalRows=").append(nullToDash(item.getTotalRows()))
.append(" truncated=").append(item.isTruncated())
.append(" success=").append(item.isSuccess())
.append(" error=").append(nullToDash(item.getError()))
.append(" taskStatus=").append(nullToDash(item.getTaskStatus()))
.append('\n');
renderCountries(sb, " parsed ", item.getCountries());
for (DeleteBrandPreviewRowVo preview : item.getPreviewRows()) {
sb.append(" preview: row=").append(preview.getRowIndex())
.append(" country=").append(nullToDash(preview.getCountry()))
.append(" asin=").append(nullToDash(preview.getAsin()))
.append(" status=").append(nullToDash(preview.getStatus()))
.append('\n');
}
}
sb.append("payload.files=").append(captured.payload.size()).append('\n');
for (DeleteBrandParsedFileCacheDto dto : captured.payload.values()) {
sb.append(" payload: fileKey=").append(nullToDash(dto.getFileKey()))
.append(" shopId=").append(nullToDash(dto.getShopId()))
.append(" companyName=").append(nullToDash(dto.getCompanyName()))
.append(" platform=").append(nullToDash(dto.getPlatform()))
.append(" totalRows=").append(nullToDash(dto.getTotalRows()))
.append('\n');
renderCountries(sb, " ", dto.getCountries());
}
return sb.toString();
}
private static void renderCountries(StringBuilder sb, String indent, List<DeleteBrandCountryGroupVo> countries) {
for (DeleteBrandCountryGroupVo country : countries) {
sb.append(indent).append("country=").append(nullToDash(country.getCountry()))
.append(" asinCount=").append(nullToDash(country.getAsinCount())).append('\n');
for (DeleteBrandCountryAsinVo asin : country.getItems()) {
sb.append(indent).append(" asin: row=").append(asin.getRowIndex())
.append(" value=").append(nullToDash(asin.getAsin()))
.append(" status=").append(nullToDash(asin.getStatus()))
.append('\n');
}
}
}
static String runAll() {
Map<String, File> main = Map.of("uploads/魏振峰.xlsx", fixtureMainOrNull());
StringBuilder sb = new StringBuilder();
sb.append(runCase("main", main, List.of("uploads/魏振峰.xlsx"), List.of("魏振峰")));
sb.append(runCase("unmatched", single("uploads/魏振峰.xlsx"), List.of("uploads/魏振峰.xlsx"), List.of()));
sb.append(runCase("headerOnly", single("uploads/缺字段行.xlsx"), List.of("uploads/缺字段行.xlsx"), List.of("缺字段行")));
sb.append(runCase("shopNameColumnSkipped", single("uploads/英国.xlsx"), List.of("uploads/英国.xlsx"), List.of("英国")));
sb.append(runCase("notExcel", single("uploads/伪装.csv.xlsx"), List.of("uploads/伪装.csv.xlsx"), List.of("伪装")));
return sb.toString();
}
private static File fixtureMainOrNull() {
try {
return fixtureMain();
} catch (Exception ex) {
throw new IllegalStateException(ex);
}
}
private static Map<String, File> single(String fileKey) {
try {
File file = switch (fileKey) {
case "uploads/缺字段行.xlsx" -> fixtureHeaderOnly();
case "uploads/英国.xlsx" -> fixtureSingleHeaderRowShape();
case "uploads/伪装.csv.xlsx" -> fixtureNotExcel();
default -> fixtureMain();
};
return Map.of(fileKey, file);
} catch (Exception ex) {
throw new IllegalStateException(ex);
}
}
private static String read(File file) throws Exception {
return Files.readString(file.toPath(), StandardCharsets.UTF_8).replace("\r\n", "\n");
}
// ---- 用例 ----
@Test
void run_snapshot_matches_golden() throws Exception {
assertEquals(read(GOLDEN_RUN), runAll(), "建单+解析输出快照与 golden 一致");
}
@Test
void run_snapshot_is_reproducible() {
assertEquals(runAll(), runAll(), "同一夹具跑两次结果一致(时钟与雪花 id 不得进快照)");
}
@Test
void run_snapshot_golden_is_committed() {
assertTrue(GOLDEN_RUN.isFile(), "golden 文件必须存在并入库: " + GOLDEN_RUN.getAbsolutePath());
assertTrue(GOLDEN_RUN.length() > 0, "golden 文件非空");
}
@Test
void run_snapshot_tamper_is_detected() throws Exception {
String original = read(GOLDEN_RUN);
assertTrue(original.contains("== case=main"), "golden 内容合法");
try {
Files.writeString(GOLDEN_RUN.toPath(), original + "\n# tampered", StandardCharsets.UTF_8);
assertTrue(assertDoesNotEqualGolden(), "篡改 golden 后断言应失败");
} finally {
Files.writeString(GOLDEN_RUN.toPath(), original, StandardCharsets.UTF_8);
}
assertEquals(original, read(GOLDEN_RUN), "恢复原始 golden");
}
private static boolean assertDoesNotEqualGolden() {
try {
return !read(GOLDEN_RUN).equals(runAll());
} catch (Exception ex) {
throw new IllegalStateException(ex);
}
}
}
@@ -0,0 +1,362 @@
package com.nanri.aiimage.modules.pricetrack.service.support;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.nanri.aiimage.common.service.ResultDownloadResolver;
import com.nanri.aiimage.config.TaskPressureProperties;
import com.nanri.aiimage.modules.file.service.LocalFileStorageService;
import com.nanri.aiimage.modules.file.service.oss.OssStorageService;
import com.nanri.aiimage.modules.pricetrack.mapper.PriceTrackShopCandidateMapper;
import com.nanri.aiimage.modules.pricetrack.model.dto.PriceTrackCreateTaskRequest;
import com.nanri.aiimage.modules.pricetrack.model.vo.PriceTrackCreateTaskVo;
import com.nanri.aiimage.modules.pricetrack.model.vo.PriceTrackMatchShopsVo;
import com.nanri.aiimage.modules.pricetrack.model.vo.PriceTrackResultItemVo;
import com.nanri.aiimage.modules.pricetrack.service.PriceTrackExcelAssemblyService;
import com.nanri.aiimage.modules.pricetrack.service.PriceTrackLoopRunService;
import com.nanri.aiimage.modules.pricetrack.service.PriceTrackTaskCacheService;
import com.nanri.aiimage.modules.pricetrack.service.PriceTrackTaskService;
import com.nanri.aiimage.modules.shopkey.service.SkipPriceAsinService;
import com.nanri.aiimage.modules.task.mapper.FileResultMapper;
import com.nanri.aiimage.modules.task.mapper.FileTaskMapper;
import com.nanri.aiimage.modules.task.model.entity.FileResultEntity;
import com.nanri.aiimage.modules.task.model.entity.FileTaskEntity;
import com.nanri.aiimage.modules.task.service.TaskDistributedLockService;
import com.nanri.aiimage.modules.task.service.TaskFileJobService;
import com.nanri.aiimage.modules.task.service.TaskProgressLightAssembler;
import com.nanri.aiimage.modules.task.service.TaskResultPayloadService;
import com.nanri.aiimage.modules.ziniao.service.ZiniaoShopSwitchService;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.junit.jupiter.api.Test;
import org.springframework.transaction.PlatformTransactionManager;
import java.io.File;
import java.io.FileOutputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.doAnswer;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
/**
* 跟价建单+ASIN 文件解析段的现网行为快照(重构蓝图 Phase 0 · 1B,缺快照的四支之三)。
*
* 快照打的是真 {@link PriceTrackTaskService#createTask}:模式二选一校验、按归一化店名去重、
* 未匹配店铺在建单前就拒、ASIN 文件路径的三种形态与「必须落在上传临时目录内」的穿越校验、
* 表头别名映射(商品ASIN/价格/最低价 → asin/price/minimumPrice)、按 sheet 名认国家、
* 单国家时的 sheet 名兜底、空 ASIN 行丢弃、BOM 与全角空格归一化,以及解析结果落到
* {@code TaskResultPayloadService.saveLatest} 的载荷形状——parse 与 report 两段同时钉住。
* 心跳也在本支钉住:建单即 {@code touchTaskHeartbeat}(蓝图 §1A 那一格,本支从建单侧取证)。
*
* 时钟与临时目录绝对路径不进快照(后者统一擦成 {@code <root>}),否则逐字节对撞无从谈起。
*
* golden 文件:src/test/resources/pricetrack/golden/create-task-snapshot.txt
*/
class PriceTrackSnapshotTest {
private static final File GOLDEN_CREATE =
new File("src/test/resources/pricetrack/golden/create-task-snapshot.txt");
// ---- 夹具 ----
private record SheetCells(String name, List<List<String>> rows) {
}
private static File workbook(File root, String name, List<SheetCells> sheets) throws Exception {
File file = new File(root, name);
try (XSSFWorkbook wb = new XSSFWorkbook(); FileOutputStream out = new FileOutputStream(file)) {
for (SheetCells sheet : sheets) {
Sheet created = wb.createSheet(sheet.name());
for (int r = 0; r < sheet.rows().size(); r++) {
Row row = created.createRow(r);
List<String> cells = sheet.rows().get(r);
for (int c = 0; c < cells.size(); c++) {
row.createCell(c).setCellValue(cells.get(c));
}
}
}
wb.write(out);
}
return file;
}
/** 英国 sheet 用「商品ASIN/价格/最低价/备注」四列别名,德国 sheet 用最小两列;含空 ASIN 行与 BOM/全角 ASIN。 */
private static File asinWorkbookTwoSheets(File root) throws Exception {
return workbook(root, "asin.xlsx", List.of(
new SheetCells("英国", List.of(
List.of("商品ASIN", "价格", "最低价", "备注"),
List.of("B011", "19.90", "18.00", "note"),
List.of("", "9.90", "", ""),
List.of("b012 ", "20.00", "", ""))),
new SheetCells("德国", List.of(
List.of("ASIN", "价格"),
List.of("B021", "5.00")))));
}
/** sheet 名不在国家词表里(葡萄牙)——国家归属只能靠 countryCodes 兜底。 */
private static File unknownSheetWorkbook(File root) throws Exception {
return workbook(root, "unknown-sheet.xlsx", List.of(
new SheetCells("葡萄牙", List.of(
List.of("ASIN", "价格"),
List.of("B099", "1.00")))));
}
// ---- 真服务 + 捕获 ----
private static final class Captured {
private final List<FileTaskEntity> taskInserts = new ArrayList<>();
private final List<FileTaskEntity> taskUpdates = new ArrayList<>();
private final List<FileResultEntity> resultInserts = new ArrayList<>();
private final List<Map<String, List<Map<String, String>>>> payloads = new ArrayList<>();
private int heartbeats;
}
private static PriceTrackTaskService newService(Captured captured, File root) {
FileTaskMapper fileTaskMapper = mock(FileTaskMapper.class);
FileResultMapper fileResultMapper = mock(FileResultMapper.class);
ZiniaoShopSwitchService ziniao = mock(ZiniaoShopSwitchService.class);
PriceTrackTaskCacheService cacheService = mock(PriceTrackTaskCacheService.class);
LocalFileStorageService storage = mock(LocalFileStorageService.class);
TaskResultPayloadService payloadService = mock(TaskResultPayloadService.class);
doAnswer(invocation -> {
FileTaskEntity entity = invocation.getArgument(0);
entity.setId(1001L + captured.taskInserts.size());
captured.taskInserts.add(entity);
return 1;
}).when(fileTaskMapper).insert(any(FileTaskEntity.class));
doAnswer(invocation -> {
captured.taskUpdates.add(invocation.getArgument(0));
return 1;
}).when(fileTaskMapper).updateById(any(FileTaskEntity.class));
doAnswer(invocation -> {
FileResultEntity entity = invocation.getArgument(0);
entity.setId(2001L + captured.resultInserts.size());
captured.resultInserts.add(entity);
return 1;
}).when(fileResultMapper).insert(any(FileResultEntity.class));
// 去重键是「归一化后的店名」,这里给一条去首尾空白的最小归一化(紫鸟归一化本身归它自己的套件)
when(ziniao.normalizeShopName(any())).thenAnswer(invocation -> {
String shopName = invocation.getArgument(0);
return shopName == null ? "" : shopName.trim();
});
when(storage.localTempRoot()).thenReturn(root);
doAnswer(invocation -> {
captured.heartbeats++;
return null;
}).when(cacheService).touchTaskHeartbeat(any());
doAnswer(invocation -> {
@SuppressWarnings("unchecked")
Map<String, List<Map<String, String>>> rows = invocation.getArgument(3);
captured.payloads.add(rows);
return null;
}).when(payloadService).saveLatest(any(), anyString(), anyString(), any());
return new PriceTrackTaskService(fileTaskMapper, fileResultMapper,
mock(PriceTrackShopCandidateMapper.class), ziniao, mock(OssStorageService.class),
mock(ResultDownloadResolver.class), new ObjectMapper(), mock(SkipPriceAsinService.class),
mock(PriceTrackExcelAssemblyService.class), cacheService, mock(PriceTrackLoopRunService.class),
mock(TaskPressureProperties.class), payloadService, mock(TaskFileJobService.class),
mock(TaskDistributedLockService.class), mock(TaskProgressLightAssembler.class),
storage, mock(PlatformTransactionManager.class));
}
private static PriceTrackMatchShopsVo.PriceTrackShopQueueItem shop(String shopName, Boolean matched) {
PriceTrackMatchShopsVo.PriceTrackShopQueueItem item = new PriceTrackMatchShopsVo.PriceTrackShopQueueItem();
item.setShopName(shopName);
item.setMatched(matched);
item.setShopId(Boolean.TRUE.equals(matched) ? "shop-" + shopName : null);
item.setPlatform("amazon");
item.setCompanyName("公司-" + shopName);
item.setMatchStatus(Boolean.TRUE.equals(matched) ? "MATCHED" : "NOT_FOUND");
return item;
}
private static PriceTrackCreateTaskRequest request(List<PriceTrackMatchShopsVo.PriceTrackShopQueueItem> items,
boolean statusMode,
boolean asinMode,
List<String> asinFiles,
List<String> countryCodes) {
PriceTrackCreateTaskRequest request = new PriceTrackCreateTaskRequest();
request.setUserId(7L);
request.setItems(items);
request.setStatusMode(statusMode);
request.setAsinMode(asinMode);
request.setAsinFiles(asinFiles);
request.setCountryCodes(countryCodes);
request.setListingStatuses(List.of("在售", "缺少报价"));
return request;
}
private static String nullToDash(Object value) {
String text = value == null ? "" : String.valueOf(value);
return text.isBlank() ? "-" : text;
}
private static String runCase(String caseName, PriceTrackCreateTaskRequest request, File root, File outsideRoot) {
Captured captured = new Captured();
PriceTrackTaskService service = newService(captured, root);
StringBuilder sb = new StringBuilder();
sb.append("== case=").append(caseName).append('\n');
try {
PriceTrackCreateTaskVo vo = service.createTask(request, "dev-1");
sb.append("vo: taskId=").append(nullToDash(vo.getTaskId()))
.append(" items=").append(vo.getItems().size()).append('\n');
for (PriceTrackResultItemVo item : vo.getItems()) {
sb.append(" item: resultId=").append(nullToDash(item.getResultId()))
.append(" shopName=").append(nullToDash(item.getShopName()))
.append(" matched=").append(nullToDash(item.getMatched()))
.append(" taskStatus=").append(nullToDash(item.getTaskStatus()))
.append(" success=").append(nullToDash(item.getSuccess()))
.append('\n');
}
// 载荷在 parse 段就落库了;VO 上那三个 map 今天恒空(注释掉的那几行),钉住「别再往回加」
sb.append("vo: asinRowsByCountry=").append(vo.getAsinRowsByCountry().size())
.append(" skipAsinsByCountry=").append(vo.getSkipAsinsByCountry().size())
.append(" minimumPriceByCountry=").append(vo.getMinimumPriceByCountryAndAsin().size())
.append('\n');
} catch (Exception ex) {
sb.append("thrown: ").append(ex.getClass().getSimpleName())
.append(" message=").append(nullToDash(ex.getMessage())).append('\n');
}
for (FileTaskEntity task : captured.taskInserts) {
sb.append("task.insert: id=").append(task.getId())
.append(" moduleType=").append(nullToDash(task.getModuleType()))
.append(" taskMode=").append(nullToDash(task.getTaskMode()))
.append(" status=").append(nullToDash(task.getStatus()))
.append(" sourceFileCount=").append(nullToDash(task.getSourceFileCount()))
.append('\n');
}
for (FileTaskEntity task : captured.taskUpdates) {
sb.append("task.update: id=").append(task.getId())
.append(" status=").append(nullToDash(task.getStatus()))
.append(" requestJsonHasAsinFiles=").append(task.getRequestJson() != null
&& task.getRequestJson().contains("asinFiles"))
.append(" requestJsonHasListingStatuses=").append(task.getRequestJson() != null
&& task.getRequestJson().contains("listingStatuses"))
.append(" resultJsonItems=").append(task.getResultJson() == null ? -1
: count(task.getResultJson(), "\"resultId\""))
.append('\n');
}
for (FileResultEntity result : captured.resultInserts) {
sb.append("result.insert: id=").append(result.getId())
.append(" moduleType=").append(nullToDash(result.getModuleType()))
.append(" sourceFilename=").append(nullToDash(result.getSourceFilename()))
.append(" sourceFileUrl=").append(nullToDash(result.getSourceFileUrl()))
.append(" success=").append(nullToDash(result.getSuccess()))
.append('\n');
}
sb.append("payloadSaves=").append(captured.payloads.size()).append('\n');
for (Map<String, List<Map<String, String>>> payload : captured.payloads) {
for (Map.Entry<String, List<Map<String, String>>> entry : payload.entrySet()) {
sb.append(" payload country=").append(entry.getKey())
.append(" rows=").append(entry.getValue().size()).append('\n');
for (Map<String, String> row : entry.getValue()) {
sb.append(" row:");
for (Map.Entry<String, String> cell : row.entrySet()) {
sb.append(' ').append(cell.getKey()).append('=').append(nullToDash(cell.getValue()));
}
sb.append('\n');
}
}
}
sb.append("heartbeatTouches=").append(captured.heartbeats).append('\n');
return scrub(scrub(sb.toString(), root, "<root>"), outsideRoot, "<outside>");
}
private static int count(String text, String needle) {
int occurrences = 0;
int index = text.indexOf(needle);
while (index >= 0) {
occurrences++;
index = text.indexOf(needle, index + needle.length());
}
return occurrences;
}
/** 临时目录绝对路径每次跑都不一样(拒收文案里也带原路径),统一擦成固定记号才谈得上逐字节对撞。 */
private static String scrub(String text, File dir, String token) {
return text.replace(dir.getAbsolutePath(), token);
}
public static String runAll() throws Exception {
File root = Files.createTempDirectory("price-track-snapshot-root-").toFile();
root.deleteOnExit();
File outsideRoot = Files.createTempDirectory("price-track-snapshot-outside-").toFile();
outsideRoot.deleteOnExit();
asinWorkbookTwoSheets(root);
unknownSheetWorkbook(root);
File outside = new File(outsideRoot, "outside.xlsx");
try (FileOutputStream ignored = new FileOutputStream(outside)) {
ignored.write("PK\u0003\u0004-not-inside-temp-root".getBytes(StandardCharsets.UTF_8));
}
List<PriceTrackMatchShopsVo.PriceTrackShopQueueItem> twoShops =
List.of(shop("店铺A", true), shop(" 店铺A ", true), shop("店铺B", true));
List<PriceTrackMatchShopsVo.PriceTrackShopQueueItem> oneShop = List.of(shop("店铺A", true));
StringBuilder sb = new StringBuilder();
sb.append(runCase("asinModeTwoSheets",
request(twoShops, false, true, List.of("asin.xlsx"), List.of("UK", "DE")), root, outsideRoot));
sb.append(runCase("singleSheetFallback",
request(oneShop, false, true, List.of("unknown-sheet.xlsx"), List.of("FR")), root, outsideRoot));
sb.append(runCase("unknownSheetWithoutFallback",
request(oneShop, false, true, List.of("unknown-sheet.xlsx"), List.of("UK", "DE")), root, outsideRoot));
sb.append(runCase("multipleFilesSameCountryMerged",
request(oneShop, false, true, List.of("asin.xlsx", "unknown-sheet.xlsx"), List.of("FR")), root, outsideRoot));
sb.append(runCase("fileOutsideTempRootRejected",
request(oneShop, false, true, List.of(outside.getAbsolutePath()), List.of("UK")), root, outsideRoot));
sb.append(runCase("missingFile",
request(oneShop, false, true, List.of("不存在的.xlsx"), List.of("UK")), root, outsideRoot));
sb.append(runCase("blankPathSkipped",
request(oneShop, false, true, List.of("", "asin.xlsx"), List.of("FR")), root, outsideRoot));
sb.append(runCase("statusMode", request(oneShop, true, false, List.of(), List.of("UK")), root, outsideRoot));
sb.append(runCase("bothModes",
request(oneShop, true, true, List.of("asin.xlsx"), List.of("UK")), root, outsideRoot));
sb.append(runCase("neitherMode", request(oneShop, false, false, List.of(), List.of("UK")), root, outsideRoot));
sb.append(runCase("asinModeWithoutFiles", request(oneShop, false, true, List.of(), List.of("UK")), root, outsideRoot));
sb.append(runCase("unmatchedShop", request(List.of(shop("店铺C", false)), true, false, List.of(), List.of("UK")), root, outsideRoot));
return sb.toString();
}
private static String read(File file) throws Exception {
return Files.readString(file.toPath(), StandardCharsets.UTF_8).replace("\r\n", "\n");
}
@Test
void create_task_snapshot_matches_golden() throws Exception {
assertEquals(read(GOLDEN_CREATE), runAll(), "建单+ASIN 解析输出快照与 golden 一致");
}
@Test
void create_task_snapshot_is_reproducible() throws Exception {
assertEquals(runAll(), runAll(), "同一夹具跑两次结果一致(时钟与临时目录路径不得进快照)");
}
@Test
void create_task_snapshot_golden_is_committed() {
assertTrue(GOLDEN_CREATE.isFile(), "golden 文件必须存在并入库: " + GOLDEN_CREATE.getAbsolutePath());
assertTrue(GOLDEN_CREATE.length() > 0, "golden 文件非空");
}
@Test
void create_task_snapshot_tamper_is_detected() throws Exception {
String original = read(GOLDEN_CREATE);
assertTrue(original.contains("== case=asinModeTwoSheets"), "golden 内容合法");
try {
Files.writeString(GOLDEN_CREATE.toPath(), original + "\n# tampered", StandardCharsets.UTF_8);
assertTrue(!read(GOLDEN_CREATE).equals(runAll()), "篡改 golden 后断言应失败");
} finally {
Files.writeString(GOLDEN_CREATE.toPath(), original, StandardCharsets.UTF_8);
}
assertEquals(original, read(GOLDEN_CREATE), "恢复原始 golden");
}
}
@@ -0,0 +1,292 @@
package com.nanri.aiimage.modules.shopmatch.service.support;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import com.nanri.aiimage.common.service.ResultDownloadResolver;
import com.nanri.aiimage.config.TaskPressureProperties;
import com.nanri.aiimage.modules.file.service.oss.OssStorageService;
import com.nanri.aiimage.modules.shopkey.mapper.QueryAsinMapper;
import com.nanri.aiimage.modules.shopkey.service.SkipPriceAsinService;
import com.nanri.aiimage.modules.shopmatch.mapper.ShopMatchShopCandidateMapper;
import com.nanri.aiimage.modules.shopmatch.model.dto.ShopMatchCreateTaskRequest;
import com.nanri.aiimage.modules.shopmatch.model.dto.ShopMatchTaskItemDto;
import com.nanri.aiimage.modules.shopmatch.model.vo.ShopMatchCreateTaskVo;
import com.nanri.aiimage.modules.shopmatch.service.ShopMatchExcelAssemblyService;
import com.nanri.aiimage.modules.shopmatch.service.ShopMatchTaskCacheService;
import com.nanri.aiimage.modules.shopmatch.service.ShopMatchTaskService;
import com.nanri.aiimage.modules.task.mapper.FileResultMapper;
import com.nanri.aiimage.modules.task.mapper.FileTaskMapper;
import com.nanri.aiimage.modules.task.model.entity.FileResultEntity;
import com.nanri.aiimage.modules.task.model.entity.FileTaskEntity;
import com.nanri.aiimage.modules.task.service.TaskDistributedLockService;
import com.nanri.aiimage.modules.task.service.TaskFileJobService;
import com.nanri.aiimage.modules.task.service.TaskProgressLightAssembler;
import com.nanri.aiimage.modules.task.service.TaskResultPayloadService;
import com.nanri.aiimage.modules.ziniao.service.ZiniaoShopSwitchService;
import org.junit.jupiter.api.Test;
import org.springframework.transaction.PlatformTransactionManager;
import java.io.File;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.time.LocalDateTime;
import java.util.ArrayList;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.doAnswer;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
/**
* 匹配店铺建单段的现网行为快照(重构蓝图 Phase 0 · 1B,缺快照的四支之二)。
*
* 本模块此前只有 1 支测试/264 行,对 3,275 行源码——蓝图把 3C(collectdata+shopmatch)
* 的开工红线钉在「快照补齐之前不得动」,这一支就是那条红线的解。
* 快照打的是真 {@link ShopMatchTaskService#createTask}:建单校验的顺序、按归一化店名去重、
* 国家代码白名单、排期分出的 SCHEDULED/RUNNING 两条状态,以及「心跳只在 RUNNING 时打」
* 这条跨模块契约(蓝图 §1A 要接的心跳,本支在建单侧一并钉住)。
* 落库侧捕获 fileTaskMapper/fileResultMapper 的 insert/update 入参;时钟不进快照——
* scheduleTimes 用「现在 + N 天」构造,scheduledAt 只渲染「有没有」。
*
* 这支没有 Excel 输入:它的「解析」是把 {@code ShopMatchTaskItemDto} 队列归一化成任务项
* ({@code toQueueItems → dedupeQueueItems → toTaskItems}),所以夹具直接喂请求对象。
*
* golden 文件:src/test/resources/shopmatch/golden/create-task-snapshot.txt
*/
class ShopMatchSnapshotTest {
private static final File GOLDEN_CREATE =
new File("src/test/resources/shopmatch/golden/create-task-snapshot.txt");
private static final class Captured {
private final List<FileTaskEntity> taskInserts = new ArrayList<>();
private final List<FileTaskEntity> taskUpdates = new ArrayList<>();
private final List<FileResultEntity> resultInserts = new ArrayList<>();
private int heartbeats;
}
/**
* Spring 注入的那只 ObjectMapper 不是 {@code new ObjectMapper()}:Boot 会把 classpath 上的
* jackson-datatype-jsr310 注册进去(本仓 2.18.11 在),并把 WRITE_DATES_AS_TIMESTAMPS 关掉。
* 用裸 mapper 会让 {@code scheduleTimes}(List&lt;LocalDateTime&gt;)序列化失败,被建单那段
* 的 {@code catch (Exception)} 吞成「task 不存在」——快照就会锁在一个生产里不存在的假形状上。
*/
private static ObjectMapper bootLikeObjectMapper() {
return JsonMapper.builder()
.addModule(new JavaTimeModule())
.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
.build();
}
private static ShopMatchTaskService newService(Captured captured) {
FileTaskMapper fileTaskMapper = mock(FileTaskMapper.class);
FileResultMapper fileResultMapper = mock(FileResultMapper.class);
ZiniaoShopSwitchService ziniao = mock(ZiniaoShopSwitchService.class);
ShopMatchTaskCacheService cacheService = mock(ShopMatchTaskCacheService.class);
doAnswer(invocation -> {
FileTaskEntity entity = invocation.getArgument(0);
entity.setId(1001L + captured.taskInserts.size());
captured.taskInserts.add(entity);
return 1;
}).when(fileTaskMapper).insert(any(FileTaskEntity.class));
doAnswer(invocation -> {
captured.taskUpdates.add(invocation.getArgument(0));
return 1;
}).when(fileTaskMapper).updateById(any(FileTaskEntity.class));
doAnswer(invocation -> {
FileResultEntity entity = invocation.getArgument(0);
entity.setId(2001L + captured.resultInserts.size());
captured.resultInserts.add(entity);
return 1;
}).when(fileResultMapper).insert(any(FileResultEntity.class));
// 去重键是「归一化后的店名」,这里给一条去首尾空白的最小归一化:
// 本支要钉的是「按归一化名去重、不按原始名」这条形状,紫鸟归一化本身归它自己的套件。
when(ziniao.normalizeShopName(any())).thenAnswer(invocation -> {
String shopName = invocation.getArgument(0);
return shopName == null ? "" : shopName.trim();
});
doAnswer(invocation -> {
captured.heartbeats++;
return null;
}).when(cacheService).touchTaskHeartbeat(any());
return new ShopMatchTaskService(fileTaskMapper, fileResultMapper,
mock(ShopMatchShopCandidateMapper.class), ziniao,
mock(ShopMatchExcelAssemblyService.class), mock(OssStorageService.class),
mock(ResultDownloadResolver.class), bootLikeObjectMapper(), cacheService,
mock(TaskPressureProperties.class), mock(TaskResultPayloadService.class),
mock(TaskFileJobService.class), mock(TaskDistributedLockService.class),
mock(SkipPriceAsinService.class), mock(QueryAsinMapper.class),
mock(TaskProgressLightAssembler.class), mock(PlatformTransactionManager.class));
}
private static ShopMatchTaskItemDto shop(String shopName, boolean matched) {
ShopMatchTaskItemDto item = new ShopMatchTaskItemDto();
item.setShopName(shopName);
item.setMatched(matched);
item.setShopId(matched ? "shop-" + shopName.trim() : null);
item.setPlatform("amazon");
item.setCompanyName("公司-" + shopName.trim());
item.setOpenStoreUrl("https://example.invalid/" + shopName.trim());
item.setMatchedUserId(9L);
item.setMatchStatus(matched ? "MATCHED" : "NOT_FOUND");
item.setMatchMessage(matched ? "" : "店铺未匹配");
return item;
}
private static ShopMatchCreateTaskRequest request(List<ShopMatchTaskItemDto> items,
List<String> countryCodes,
List<LocalDateTime> scheduleTimes) {
ShopMatchCreateTaskRequest request = new ShopMatchCreateTaskRequest();
request.setUserId(7L);
request.setItems(items);
request.setCountryCodes(countryCodes);
request.setScheduleTimes(scheduleTimes);
return request;
}
private static String nullToDash(Object value) {
String text = value == null ? "" : String.valueOf(value);
return text.isBlank() ? "-" : text;
}
private static int countOccurrences(String text, String needle) {
int count = 0;
int index = text.indexOf(needle);
while (index >= 0) {
count++;
index = text.indexOf(needle, index + needle.length());
}
return count;
}
private static String runCase(String caseName, ShopMatchCreateTaskRequest request) {
Captured captured = new Captured();
ShopMatchTaskService service = newService(captured);
StringBuilder sb = new StringBuilder();
sb.append("== case=").append(caseName).append('\n');
try {
ShopMatchCreateTaskVo vo = service.createTask(request, "dev-1");
sb.append("vo: taskId=").append(nullToDash(vo.getTaskId()))
.append(" items=").append(vo.getItems().size()).append('\n');
for (ShopMatchCreateTaskVo.Item item : vo.getItems()) {
sb.append(" item: resultId=").append(nullToDash(item.getResultId()))
.append(" shopName=").append(nullToDash(item.getShopName()))
.append(" shopId=").append(nullToDash(item.getShopId()))
.append(" matched=").append(item.isMatched())
.append(" matchStatus=").append(nullToDash(item.getMatchStatus()))
.append(" taskStatus=").append(nullToDash(item.getTaskStatus()))
.append(" success=").append(nullToDash(item.getSuccess()))
.append(" scheduledAtPresent=").append(item.getScheduledAt() != null)
.append('\n');
}
} catch (Exception ex) {
sb.append("thrown: ").append(ex.getClass().getSimpleName())
.append(" message=").append(nullToDash(ex.getMessage())).append('\n');
}
for (FileTaskEntity task : captured.taskInserts) {
sb.append("task.insert: id=").append(task.getId())
.append(" moduleType=").append(nullToDash(task.getModuleType()))
.append(" taskMode=").append(nullToDash(task.getTaskMode()))
.append(" status=").append(nullToDash(task.getStatus()))
.append(" sourceFileCount=").append(nullToDash(task.getSourceFileCount()))
.append(" deviceId=").append(nullToDash(task.getDeviceId()))
.append(" scheduledAtPresent=").append(task.getScheduledAt() != null)
.append('\n');
}
for (FileTaskEntity task : captured.taskUpdates) {
sb.append("task.update: id=").append(task.getId())
.append(" status=").append(nullToDash(task.getStatus()))
.append(" requestJsonStageIndexZero=").append(task.getRequestJson() != null
&& task.getRequestJson().contains("\"current_stage_index\":0"))
.append(" requestJsonStageIndexNull=").append(task.getRequestJson() != null
&& task.getRequestJson().contains("\"current_stage_index\":null"))
.append(" resultJsonResultIds=").append(task.getResultJson() == null ? -1
: countOccurrences(task.getResultJson(), "\"resultId\""))
.append('\n');
}
for (FileResultEntity result : captured.resultInserts) {
sb.append("result.insert: id=").append(result.getId())
.append(" taskId=").append(nullToDash(result.getTaskId()))
.append(" moduleType=").append(nullToDash(result.getModuleType()))
.append(" sourceFilename=").append(nullToDash(result.getSourceFilename()))
.append(" sourceFileUrl=").append(nullToDash(result.getSourceFileUrl()))
.append(" success=").append(nullToDash(result.getSuccess()))
.append('\n');
}
sb.append("heartbeatTouches=").append(captured.heartbeats).append('\n');
return sb.toString();
}
public static String runAll() {
List<ShopMatchTaskItemDto> twoShops = List.of(shop("店铺A", true), shop("店铺B", true));
StringBuilder sb = new StringBuilder();
// 国家代码:去空白+转大写+去重,且只认 ProductRiskCountryCode 那五个(US 不在内)
sb.append(runCase("immediate", request(twoShops, List.of("uk", "UK", " uk ", ""), List.of())));
sb.append(runCase("usNotSupportedCountry", request(twoShops, List.of("US"), List.of())));
sb.append(runCase("dedupedByNormalizedName", request(
List.of(shop("店铺A", true), shop(" 店铺A ", true), shop("店铺B", true)),
List.of("DE"), List.of())));
sb.append(runCase("scheduled", request(twoShops, List.of("UK"),
List.of(LocalDateTime.now().plusDays(1), LocalDateTime.now().plusDays(2)))));
sb.append(runCase("unmatchedShop", request(
List.of(shop("店铺A", true), shop("店铺C", false)), List.of("UK"), List.of())));
sb.append(runCase("badCountryCode", request(twoShops, List.of("CN"), List.of())));
sb.append(runCase("emptyCountryCodes", request(twoShops, List.of(), List.of())));
sb.append(runCase("scheduleInPast", request(twoShops, List.of("UK"),
List.of(LocalDateTime.now().minusDays(1)))));
sb.append(runCase("scheduleNotIncreasing", request(twoShops, List.of("UK"),
List.of(LocalDateTime.now().plusDays(2), LocalDateTime.now().plusDays(1)))));
return sb.toString();
}
private static String read(File file) throws Exception {
return Files.readString(file.toPath(), StandardCharsets.UTF_8).replace("\r\n", "\n");
}
@Test
void create_task_snapshot_matches_golden() throws Exception {
assertEquals(read(GOLDEN_CREATE), runAll(), "建单输出快照与 golden 一致");
}
@Test
void create_task_snapshot_is_reproducible() {
assertEquals(runAll(), runAll(), "同一夹具跑两次结果一致(时钟与雪花 id 不得进快照)");
}
@Test
void create_task_snapshot_golden_is_committed() {
assertTrue(GOLDEN_CREATE.isFile(), "golden 文件必须存在并入库: " + GOLDEN_CREATE.getAbsolutePath());
assertTrue(GOLDEN_CREATE.length() > 0, "golden 文件非空");
}
@Test
void create_task_snapshot_tamper_is_detected() throws Exception {
String original = read(GOLDEN_CREATE);
assertTrue(original.contains("== case=immediate"), "golden 内容合法");
try {
Files.writeString(GOLDEN_CREATE.toPath(), original + "\n# tampered", StandardCharsets.UTF_8);
assertTrue(!read(GOLDEN_CREATE).equals(runAll()), "篡改 golden 后断言应失败");
} finally {
Files.writeString(GOLDEN_CREATE.toPath(), original, StandardCharsets.UTF_8);
}
assertEquals(original, read(GOLDEN_CREATE), "恢复原始 golden");
}
/** 逐行 insert 的自增主键回填是 resultId 的来源——改成批量插入会让快照 resultId 变 null。 */
@Test
void create_task_backfills_result_id_per_row() {
Captured captured = new Captured();
ShopMatchCreateTaskVo vo = newService(captured).createTask(
request(List.of(shop("店铺A", true), shop("店铺B", true)), List.of("UK"), List.of()), "dev-1");
assertEquals(2, vo.getItems().size());
assertEquals(2001L, vo.getItems().get(0).getResultId());
assertEquals(2002L, vo.getItems().get(1).getResultId());
}
}