修复双实例 owner-scoped 结果文件 job 被非归属实例误杀
线上现象:货源查询任务 26402/26400(归属 server-121)在归属机正常推进, 却被 server-110 判为「结果生成失败」,错误为「该任务已绑定到另一台服务实例处理」。 两个缺陷叠加: 1. 两台实例共用一个 RocketMQ consumer group,dispatch 消息被轮询投递; worker 对"PENDING + owner 不匹配"的 job 选择本地抢跑,模块层 ensureTaskOwnedByCurrentInstance 直接抛 TaskOwnerMismatchException, 异常落进通用 catch 被当成生成失败,5 次抢跑即耗尽 retryCount 并终态失败。 2. runPendingJobs 双重 claim:claimRunnableJobsForOwner 已把 job 翻成 RUNNING, 随后又走 claimRunning(条件 PENDING/FAILED)必然失败 → job 被静默丢弃, 归属机的定时兜底从未真正生效。 修复: - worker:非归属实例一律跳过 owner-scoped job;抽出 processClaimed 供已持有 claim 的调用方使用;processInternal 单独捕获 TaskOwnerMismatchException, 只 deferRunning 退回 PENDING,不计入重试。 - TaskFileJobLocalDispatcher 新增 dispatchClaimed。 - dispatch 事件带上 owner,coordinator 按归属路由:本机 owner 直接本地派发不发 MQ, 他机 owner 不发不跑交由归属机轮询接手,owner 为空维持原有 MQ 行为。 - 新增 owner 路由回归测试两组,回滚配合"本地抢跑"的旧断言。 同时提交此前工作区内已随 JAR 上线的 backend-java 改动:SimilarAsin 解析载荷 groups 内嵌 items(Python 旧链路兼容)与 chunk 类型不匹配跳过、 PermissionMenuSchemaInitializer 与 V103 两级菜单分组迁移、相关测试。
This commit is contained in:
+50
@@ -336,6 +336,56 @@ class SimilarAsinTaskServiceGroupRefTest {
|
||||
assertEquals(80, coverage);
|
||||
}
|
||||
|
||||
@Test
|
||||
void test_task_006_group_payload_embeds_items_for_python() throws Exception {
|
||||
// Python 旧链路(normalize_groups 直接返回 groups 并按 gp["items"] 取行):
|
||||
// 写入存储的 payload 中每组必须内嵌完整行,startIndex/endIndex 保留供 Java 内部展开。
|
||||
java.util.concurrent.atomic.AtomicReference<String> captured = new java.util.concurrent.atomic.AtomicReference<>();
|
||||
when(transientPayloadStorageService.storeParsedPayloadFast(
|
||||
eq(SimilarAsinTaskService.MODULE_TYPE), any(), anyString(), anyString(), eq(false)))
|
||||
.thenAnswer(invocation -> {
|
||||
captured.set(invocation.getArgument(3));
|
||||
return "rustfs:task-parsed/similar-asin/30000/payload.json";
|
||||
});
|
||||
File workbook = buildWorkbook(12);
|
||||
SimilarAsinParseVo vo = parse(workbook, "uploads/20260829/gr-python.xlsx");
|
||||
assertEquals(12, vo.getAcceptedRows());
|
||||
SimilarAsinParsedPayloadDto stored = objectMapper.readValue(captured.get(), SimilarAsinParsedPayloadDto.class);
|
||||
assertNotNull(stored.getGroups());
|
||||
assertEquals(12, stored.getGroups().size());
|
||||
// 每组内嵌 items 与引用区间宽度一致,且行内容对应 items[startIndex]
|
||||
for (SimilarAsinParsedGroupVo group : stored.getGroups()) {
|
||||
assertNotNull(group.getItems());
|
||||
assertEquals(group.getEndIndex() - group.getStartIndex(), group.getItems().size(),
|
||||
"Python 兼容:组内嵌 items 必须与引用区间宽度一致");
|
||||
int start = group.getStartIndex();
|
||||
assertEquals(stored.getItems().get(start).getRowToken(), group.getItems().get(0).getRowToken(),
|
||||
"组 items 首行必须是 items[startIndex]");
|
||||
}
|
||||
// Java 内部展开路径不受内嵌 items 影响:仍按引用切片
|
||||
List<SimilarAsinParsedRowVo> expanded = SimilarAsinTaskService.expandGroupRefs(stored);
|
||||
assertEquals(12, expanded.size(), "Java 内部展开仍按引用,不受内嵌 items 影响");
|
||||
}
|
||||
|
||||
@Test
|
||||
void test_task_006_group_hydrate_embeds_items_for_legacy_payload() throws Exception {
|
||||
// 存量任务 payload 只有 startIndex/endIndex 引用(无内嵌 items):
|
||||
// parsedPayload 读取时 hydrate 必须回填组内嵌 items,Python 才能取到行。
|
||||
String legacy = "{\"items\":["
|
||||
+ "{\"rowToken\":\"t1\",\"asin\":\"B0GRP00001\",\"sourceFileKey\":\"f.xlsx\",\"country\":\"英国\"},"
|
||||
+ "{\"rowToken\":\"t2\",\"asin\":\"B0GRP00002\",\"sourceFileKey\":\"f.xlsx\",\"country\":\"英国\"}"
|
||||
+ "],\"groups\":[{\"groupKey\":\"g1\",\"startIndex\":0,\"endIndex\":2,\"itemCount\":2}]}";
|
||||
SimilarAsinParsedPayloadDto payload = objectMapper.readValue(legacy, SimilarAsinParsedPayloadDto.class);
|
||||
SimilarAsinParsedPayloadDto hydrated = SimilarAsinTaskService.hydrateForPythonCompat(payload);
|
||||
assertEquals(1, hydrated.getGroups().size());
|
||||
assertNotNull(hydrated.getGroups().get(0).getItems());
|
||||
assertEquals(2, hydrated.getGroups().get(0).getItems().size(), "hydrate 后组内嵌 items 必须为 2 行");
|
||||
assertEquals("B0GRP00001", hydrated.getGroups().get(0).getItems().get(0).getAsin(),
|
||||
"hydrate 后组内嵌 items 首行必须对应 items[0]");
|
||||
assertEquals("B0GRP00002", hydrated.getGroups().get(0).getItems().get(1).getAsin(),
|
||||
"hydrate 后组内嵌 items 第二行必须对应 items[1]");
|
||||
}
|
||||
|
||||
// ---- helpers ----
|
||||
|
||||
private static String groupRefJson(int groupCount, int[][] ranges) {
|
||||
|
||||
Reference in New Issue
Block a user