test(phase-3): 3D 前置——publish 回传段 golden 落地,27 段流水钉住本支独有的三处形状

只加测试与 golden,`PublishTaskService` 一字未动(现网行为无变更,故不需要行为回归之外的验证)。

钉住的 publish 独有形状(逐段对 golden 复核过,不是推测):
- 取锁在取任务**之前**:`lockBusyRejectedBeforeAnyLoad` 只有 `lock.acquire` 一行、没有任何 `task.load`。这与 collectdata 的回传段方向相反,是「同族不同形」的第一处实证。
- SUCCESS 静默短路不抛:`taskSuccessShortCircuitsWithoutThrowing` = 两次 `task.load`(锁外一次、事务内一次)+ `tx.begin/commit`、零写入、`result=void`。底座 `TerminalRejection` 那三种档位里没有这一种。
- 未完成回传时 `job.enqueue resultId=2001 scopeKey=task:1001:owner:node-A`(键形状带 owner,不是纯 `task:<id>`)。
- 同请求重复文件被拒时,**未提交的 chunk 会被清掉**:`duplicateFileInOneRequestRejected` 走出 `chunk.countReferenced -> 0 → payload.delete pointer=rustfs:chunk-1`,`payloadDelete=1` 而 `files` 仍全 PENDING。

口径与坑(都是老坑的新一次踩法):
- golden 27 段由 3 支执行用例渲染,第 4 支 `@Test` 是写档器、无 `SHUFU_SNAPSHOT_WRITE` 时 skip ⇒ 执行读数 `Tests run: 4, Skipped: 1`。「段数 ≠ 用例数」第七处坑在此复现,登记以免下一个人拿 27 去对 `@Test` 计数。
- 写档过程中有一记**假 golden**:MyBatis-Plus `AbstractWrapper.getParamNameValuePairs()` 是懒填充,不先调 `getSqlSegment()` 就拿到空 Map ⇒ 分片查询替身对任何条件都返回同一行,`lastChunkCompletesMerge` 那格真跑成了「同一 chunk_index 已回传不同内容」的错路。现以 `boundValues(...)` 统一先取 `getSqlSegment()` 再读绑定值。这类坑不会判红,只会把流水记成假的。
- 清理件在 afterCommit 抛异常被吞、调用方仍见成功:`cleanupFailureAfterCommitIsSwallowedAndCallerStillSeesSuccess` 另证 `chunk` 删失败后**不再**删 scope(现网两删除同在一个 try 里)。

验证:全量 `mvn -o test` → `Tests run: 4537, Failures: 0, Errors: 0, Skipped: 15` + BUILD SUCCESS(rc 取自重定向而非管道);非写档模式复跑本类 4/0/skip 1;ops `ruff check .` = All checks passed、readability ratchet 9 passed(该尺只扫 `src/main`,792 行的测试件不构成新 offender)。
This commit is contained in:
2026-10-19 17:51:04 +08:00
parent 311bbadc97
commit f5c64b3b23
2 changed files with 1150 additions and 0 deletions
@@ -0,0 +1,792 @@
package com.nanri.aiimage.modules.publish.service.support;
import com.baomidou.mybatisplus.core.MybatisConfiguration;
import com.baomidou.mybatisplus.core.conditions.AbstractWrapper;
import com.baomidou.mybatisplus.core.metadata.TableInfoHelper;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.nanri.aiimage.common.exception.BusinessException;
import com.nanri.aiimage.common.service.DistributedJobLockService;
import com.nanri.aiimage.config.InstanceMetadata;
import com.nanri.aiimage.modules.file.service.LocalFileStorageService;
import com.nanri.aiimage.modules.file.service.oss.OssStorageService;
import com.nanri.aiimage.modules.publish.mapper.PublishFileMapper;
import com.nanri.aiimage.modules.publish.mapper.PublishItemMapper;
import com.nanri.aiimage.modules.publish.model.dto.PublishResultFileDto;
import com.nanri.aiimage.modules.publish.model.dto.PublishRowDto;
import com.nanri.aiimage.modules.publish.model.dto.PublishSubmitResultRequest;
import com.nanri.aiimage.modules.publish.model.entity.PublishFileEntity;
import com.nanri.aiimage.modules.publish.model.entity.PublishItemEntity;
import com.nanri.aiimage.modules.publish.service.PublishTaskService;
import com.nanri.aiimage.modules.publish.service.PublishWorkbookService;
import com.nanri.aiimage.modules.task.mapper.FileResultMapper;
import com.nanri.aiimage.modules.task.mapper.FileTaskMapper;
import com.nanri.aiimage.modules.task.mapper.TaskChunkMapper;
import com.nanri.aiimage.modules.task.mapper.TaskScopeStateMapper;
import com.nanri.aiimage.modules.task.model.entity.FileResultEntity;
import com.nanri.aiimage.modules.task.model.entity.FileTaskEntity;
import com.nanri.aiimage.modules.task.model.entity.TaskChunkEntity;
import com.nanri.aiimage.modules.task.model.entity.TaskScopeStateEntity;
import com.nanri.aiimage.modules.task.service.TaskDistributedLockService;
import com.nanri.aiimage.modules.task.service.TaskFileJobService;
import com.nanri.aiimage.modules.task.service.TaskHeartbeatPositionService;
import com.nanri.aiimage.modules.task.service.TaskProgressLightAssembler;
import com.nanri.aiimage.modules.task.service.TransientPayloadStorageService;
import com.nanri.aiimage.modules.ziniao.service.ZiniaoShopSwitchService;
import org.apache.ibatis.builder.MapperBuilderAssistant;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
import org.mockito.stubbing.Answer;
import org.springframework.transaction.TransactionStatus;
import org.springframework.transaction.support.TransactionSynchronization;
import org.springframework.transaction.support.TransactionSynchronizationManager;
import org.springframework.transaction.support.TransactionTemplate;
import java.io.File;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Collection;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.function.Consumer;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyInt;
import static org.mockito.ArgumentMatchers.anyList;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.doAnswer;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
/**
* 上架结果回传落库(段 3)的现网行为快照——3D 余下那一支({@code publish})接底座的护身符。
*
* <p><b>为什么先补这一支。</b>{@code publish} 现有唯一快照 {@code batch-loading-snapshot.txt}
* 锁的是建单与批量加载,{@code submitResult} 出结果段一行没打。3D 要动的正是那一段里的准入
* (先取锁后加载、{@code requireTaskId} 同形)、终态判定(本支的 SUCCESS 是<b>静默短路</b>、
* FAILED 抛「任务已失败,拒绝继续回传」,两支都不是底座那两种形状)与组装 job 挂接。
*
* <p><b>这支的形状与前面六支不同的三处,流水里都有对应格子钉着。</b>
* ①<b>任务被加载两次</b>({@code prepareResultSubmission} 与 {@code submitResultLocked} 各一次
* {@code requireTask}),而<b>取锁在两次之前</b>——与 collectdata 的「预检在取锁之前」正好相反;
* ②<b>载荷删除走两条路</b>:prepare 抛异常时删全量,事务之后再删「未提交」那一份(要减掉
* committedPayloads),而 {@code deleteUncommittedPayloads} 删之前还会先查一次
* {@code taskChunkMapper.selectCount} 问「这条指针还被分片行引用吗」——引用中就跳过;
* ③<b>事务提交后的清理本身也在流水里</b>:全部文件失败 ⇒ {@code cleanupAfterCommit=true} ⇒
* 出事务后删 chunk 行与 scope 行再删载荷,而清理自己抛异常是被 catch 吞掉只打 warn 的。
*
* <p><b>事务替身为什么要把 synchronization 开起来。</b>{@code deletePayloadsAfterCommit} 判的是
* {@code TransactionSynchronizationManager.isSynchronizationActive()}:真 {@code TransactionTemplate}
* 在事务里会激活它、把远端删除推到 afterCommit。裸跑回调会让删除早于事务返回落到另一个位置,
* 快照就记成了现网没有的顺序。这里照 Spring 的做法 init→回调→afterCommit→clear,
* 并在 {@code finally} 里清干净(thread-local,漏一次就污染后面所有用例)。
*
* <p><b>替身只实现这条链真吃到的部分</b>:任务行/文件行/分片行/作用域行都是内存活实体
* (重读看得见上一次写入),分片查询按 wrapper 绑定值(scopeHash 与 chunkIndex)匹配而不是
* 「任意入参返回同一只」——后者会把入参校验整格搬进测试(3C 刚踩过)。长内容(分片载荷、
* stateJson)只进长度与 sha256 前 12 位;时钟、雪花 id、OSS 直链、实例 id 不进快照。
*
* golden:src/test/resources/publish/golden/submit-result-snapshot.txt
* 重新生成:{@code SHUFU_SNAPSHOT_WRITE=true mvn -o test -Dtest=PublishSubmitResultSnapshotTest}
*/
class PublishSubmitResultSnapshotTest {
private static final File GOLDEN_SUBMIT =
new File("src/test/resources/publish/golden/submit-result-snapshot.txt");
private static final long TASK_ID = 1001L;
private static final long FILE_ID = 9101L;
private static final long RESULT_ID = 2001L;
private static final long USER_ID = 7L;
@BeforeAll
static void initLambdaCache() {
MapperBuilderAssistant assistant = new MapperBuilderAssistant(new MybatisConfiguration(), "");
TableInfoHelper.initTableInfo(assistant, FileTaskEntity.class);
TableInfoHelper.initTableInfo(assistant, FileResultEntity.class);
TableInfoHelper.initTableInfo(assistant, TaskChunkEntity.class);
TableInfoHelper.initTableInfo(assistant, TaskScopeStateEntity.class);
TableInfoHelper.initTableInfo(assistant, PublishFileEntity.class);
TableInfoHelper.initTableInfo(assistant, PublishItemEntity.class);
}
// ------------------------------------------------------------------
// 现网副作用替身
// ------------------------------------------------------------------
private static final class Fixture {
private final List<String> events = new ArrayList<>();
private final Map<Long, PublishFileEntity> files = new LinkedHashMap<>();
private final List<TaskChunkEntity> chunks = new ArrayList<>();
private final Map<String, String> payloadBody = new LinkedHashMap<>();
private FileTaskEntity task;
private FileResultEntity result;
private TaskScopeStateEntity scope;
private boolean rustfsEnabled = true;
private boolean rustfsPointer = true;
private boolean lockAvailable = true;
private long itemCount = 0L;
private int jobEnqueues = 0;
private int payloadDeletes = 0;
/** 留给「清理自身抛异常」那一格重新打桩用(mock 在 {@code newService} 里就地创建,不存住就够不着)。 */
private TaskChunkMapper chunkMapper;
}
private static FileTaskEntity task(String status) {
FileTaskEntity task = new FileTaskEntity();
task.setId(TASK_ID);
task.setModuleType("PUBLISH");
task.setStatus(status);
task.setTaskMode("PYTHON_QUEUE");
task.setUserId(USER_ID);
task.setTaskNo("PUBLISH-1");
task.setSourceFileCount(1);
task.setSuccessFileCount(0);
task.setFailedFileCount(0);
task.setResultJson("{\"ownerInstanceId\":\"node-A\"}");
return task;
}
private static PublishFileEntity file(Long id, String status) {
PublishFileEntity file = new PublishFileEntity();
file.setId(id);
file.setTaskId(TASK_ID);
file.setFileKey("uploads/key-" + id + ".xlsx");
file.setSourceFilename("listing-" + id + ".xlsx");
file.setStatus(status);
file.setTotalRows(0);
file.setProcessedRows(0);
return file;
}
private static FileResultEntity resultRow() {
FileResultEntity row = new FileResultEntity();
row.setId(RESULT_ID);
row.setTaskId(TASK_ID);
row.setModuleType("PUBLISH");
row.setSourceFilename("PUBLISH-1");
row.setRowCount(0);
row.setSuccess(0);
row.setUserId(USER_ID);
return row;
}
private static TaskChunkEntity storedChunk(int chunkIndex, int chunkTotal, String pointer, String payloadHash) {
TaskChunkEntity chunk = new TaskChunkEntity();
chunk.setId((long) (5000 + chunkIndex));
chunk.setTaskId(TASK_ID);
chunk.setModuleType("PUBLISH");
chunk.setScopeKey("file:" + FILE_ID);
chunk.setScopeHash("scope-hash-" + FILE_ID);
chunk.setChunkIndex(chunkIndex);
chunk.setChunkTotal(chunkTotal);
chunk.setPayloadJson(pointer);
chunk.setPayloadHash(payloadHash);
return chunk;
}
private static PublishRowDto row(String asin, String country) {
PublishRowDto row = new PublishRowDto();
row.setAsin(asin);
row.setCountry(country);
row.setBrand("brand-" + asin);
row.setPrice("19.90");
row.setStatus("ACTIVE");
return row;
}
private static PublishResultFileDto fileDto(Long fileId, Integer chunkIndex, Integer chunkTotal,
String error, PublishRowDto... rows) {
PublishResultFileDto dto = new PublishResultFileDto();
dto.setFileId(fileId);
dto.setChunkIndex(chunkIndex);
dto.setChunkTotal(chunkTotal);
dto.setError(error);
dto.setRows(new ArrayList<>(List.of(rows)));
return dto;
}
private static PublishSubmitResultRequest request(Long userId, PublishResultFileDto... files) {
PublishSubmitResultRequest request = new PublishSubmitResultRequest();
request.setUserId(userId);
request.setFiles(new ArrayList<>(List.of(files)));
return request;
}
// ------------------------------------------------------------------
// 装配
// ------------------------------------------------------------------
private static PublishTaskService newService(Fixture f) {
FileTaskMapper fileTaskMapper = mock(FileTaskMapper.class);
FileResultMapper fileResultMapper = mock(FileResultMapper.class);
PublishFileMapper publishFileMapper = mock(PublishFileMapper.class);
PublishItemMapper publishItemMapper = mock(PublishItemMapper.class);
TaskChunkMapper taskChunkMapper = mock(TaskChunkMapper.class);
TaskScopeStateMapper taskScopeStateMapper = mock(TaskScopeStateMapper.class);
TaskFileJobService taskFileJobService = mock(TaskFileJobService.class);
TaskDistributedLockService lockService = mock(TaskDistributedLockService.class);
TransientPayloadStorageService payloadStorage = mock(TransientPayloadStorageService.class);
TransactionTemplate transactionTemplate = mock(TransactionTemplate.class);
InstanceMetadata instanceMetadata = mock(InstanceMetadata.class);
// 真 ObjectMapper:这条链上有 writeValueAsString(载荷)/readValue(合并分片)/
// readTree(resultJson 里的 ownerInstanceId),拿替身会把三段全短路。
ObjectMapper objectMapper = new ObjectMapper();
when(instanceMetadata.getInstanceId()).thenReturn("node-A");
f.chunkMapper = taskChunkMapper;
when(lockService.acquire(anyString(), anyLong())).thenAnswer(invocation -> {
f.events.add("lock.acquire taskId=" + invocation.getArgument(1));
return f.lockAvailable ? mock(TaskDistributedLockService.LockHandle.class) : null;
});
when(lockService.acquire(anyString(), anyLong(), anyLong()))
.thenAnswer(invocation -> mock(TaskDistributedLockService.LockHandle.class));
when(fileTaskMapper.selectById(any())).thenAnswer(invocation -> {
// 按主键取:写 `return f.task` 会让「taskId 非法/不存在的任务」那两格证的是替身形状。
Object id = invocation.getArgument(0);
FileTaskEntity loaded = Objects.equals(id, TASK_ID) ? f.task : null;
f.events.add("task.load id=" + id + " status=" + dash(loaded == null ? null : loaded.getStatus())
+ " module=" + dash(loaded == null ? null : loaded.getModuleType())
+ " userId=" + dash(loaded == null ? null : loaded.getUserId()));
return loaded;
});
doAnswer(recordTaskUpdate(f)).when(fileTaskMapper).updateById(any(FileTaskEntity.class));
when(publishFileMapper.selectById(any())).thenAnswer(invocation -> {
Object id = invocation.getArgument(0);
PublishFileEntity loaded = f.files.get(id);
f.events.add("file.selectById id=" + id
+ " status=" + dash(loaded == null ? null : loaded.getStatus()));
return loaded;
});
when(publishFileMapper.selectOne(any())).thenAnswer(invocation -> {
Collection<Object> values = boundValues(invocation.getArgument(0));
PublishFileEntity matched = null;
for (PublishFileEntity candidate : f.files.values()) {
if (values.contains(candidate.getFileKey()) || values.contains(candidate.getSourceFilename())) {
matched = candidate;
break;
}
}
f.events.add("file.selectOne -> id=" + dash(matched == null ? null : matched.getId())
+ " status=" + dash(matched == null ? null : matched.getStatus()));
return matched;
});
when(publishFileMapper.selectList(any())).thenAnswer(invocation -> {
List<PublishFileEntity> snapshot = new ArrayList<>(f.files.values());
f.events.add("files.list size=" + snapshot.size() + " " + renderFiles(snapshot));
return snapshot;
});
doAnswer(invocation -> {
PublishFileEntity file = invocation.getArgument(0);
f.events.add("file.update id=" + file.getId() + " status=" + dash(file.getStatus())
+ " totalRows=" + dash(file.getTotalRows()) + " processedRows=" + dash(file.getProcessedRows())
+ " error=" + dash(file.getErrorMessage()));
return 1;
}).when(publishFileMapper).updateById(any(PublishFileEntity.class));
when(publishItemMapper.selectCount(any())).thenAnswer(invocation -> {
f.events.add("item.count -> " + f.itemCount);
return f.itemCount;
});
doAnswer(invocation -> {
f.events.add("item.delete");
return 0;
}).when(publishItemMapper).delete(any());
doAnswer(invocation -> {
List<PublishItemEntity> rows = invocation.getArgument(0);
f.events.add("item.insertBatch n=" + (rows == null ? 0 : rows.size()));
return rows == null ? 0 : rows.size();
}).when(publishItemMapper).insertBatch(anyList());
when(taskChunkMapper.selectOne(any())).thenAnswer(invocation -> {
Collection<Object> values = boundValues(invocation.getArgument(0));
TaskChunkEntity matched = matchChunk(f, values);
f.events.add("chunk.selectOne idx=" + intIn(values) + " -> "
+ (matched == null ? "-" : "id=" + matched.getId() + " hash=" + dash(matched.getPayloadHash())));
return matched;
});
when(taskChunkMapper.selectList(any())).thenAnswer(invocation -> {
List<TaskChunkEntity> snapshot = new ArrayList<>(f.chunks);
snapshot.sort((left, right) -> Integer.compare(
Objects.requireNonNullElse(left.getChunkIndex(), 0),
Objects.requireNonNullElse(right.getChunkIndex(), 0)));
f.events.add("chunk.selectList size=" + snapshot.size() + " idx=" + renderChunkIndexes(snapshot));
return snapshot;
});
when(taskChunkMapper.selectCount(any())).thenAnswer(invocation -> {
Collection<Object> values = boundValues(invocation.getArgument(0));
long referenced = f.chunks.stream().filter(chunk -> values.contains(chunk.getPayloadJson())).count();
f.events.add("chunk.countReferenced -> " + referenced);
return referenced;
});
doAnswer(invocation -> {
TaskChunkEntity chunk = invocation.getArgument(0);
chunk.setId((long) (5000 + Objects.requireNonNullElse(chunk.getChunkIndex(), 0)));
f.chunks.add(chunk);
f.events.add("chunk.insert idx=" + chunk.getChunkIndex() + "/" + dash(chunk.getChunkTotal())
+ " pointer=" + dash(chunk.getPayloadJson()) + " hash=" + dash(chunk.getPayloadHash()));
return 1;
}).when(taskChunkMapper).insert(any(TaskChunkEntity.class));
doAnswer(invocation -> {
f.events.add("chunk.deleteAll");
f.chunks.clear();
return 0;
}).when(taskChunkMapper).delete(any());
when(taskScopeStateMapper.selectOne(any())).thenAnswer(invocation -> {
f.events.add("scope.selectOne -> " + (f.scope == null ? "-" : "id=" + f.scope.getId()
+ " chunkTotal=" + dash(f.scope.getChunkTotal())
+ " stateJson.len=" + length(f.scope.getStateJson())
+ " stateJson.hash=" + hash(f.scope.getStateJson())));
return f.scope;
});
doAnswer(invocation -> {
TaskScopeStateEntity scope = invocation.getArgument(0);
scope.setId(7001L);
f.scope = scope;
f.events.add("scope.insert chunkTotal=" + dash(scope.getChunkTotal())
+ " stateJson.len=" + length(scope.getStateJson())
+ " stateJson.hash=" + hash(scope.getStateJson()));
return 1;
}).when(taskScopeStateMapper).insert(any(TaskScopeStateEntity.class));
doAnswer(invocation -> {
TaskScopeStateEntity scope = invocation.getArgument(0);
f.scope = scope;
f.events.add("scope.update id=" + scope.getId() + " chunkTotal=" + dash(scope.getChunkTotal())
+ " stateJson.len=" + length(scope.getStateJson())
+ " stateJson.hash=" + hash(scope.getStateJson()));
return 1;
}).when(taskScopeStateMapper).updateById(any(TaskScopeStateEntity.class));
doAnswer(invocation -> {
f.events.add("scope.deleteAll");
f.scope = null;
return 0;
}).when(taskScopeStateMapper).delete(any());
when(payloadStorage.isSharedWriteEnabled()).thenAnswer(invocation -> {
f.events.add("storage.ensure sharedWrite=" + f.rustfsEnabled);
return f.rustfsEnabled;
});
when(payloadStorage.extractPointer(any())).thenAnswer(invocation -> invocation.getArgument(0));
when(payloadStorage.storeChunkPayloadVersioned(anyString(), anyLong(), anyString(), anyInt(), anyString()))
.thenAnswer(invocation -> {
String json = invocation.getArgument(4);
String pointer = (f.rustfsPointer ? "rustfs:chunk-" : "local:chunk-") + invocation.getArgument(3);
f.payloadBody.put(pointer, json);
f.events.add("payload.storeChunk idx=" + invocation.getArgument(3)
+ " len=" + json.length() + " hash=" + hash(json));
return pointer;
});
when(payloadStorage.resolvePayload(anyString(), anyString())).thenAnswer(invocation -> {
String pointer = invocation.getArgument(0);
String body = f.payloadBody.get(pointer);
f.events.add("payload.resolve pointer=" + dash(pointer)
+ " -> " + (body == null ? "MISSING" : "len=" + body.length() + " hash=" + hash(body)));
if (body == null) {
throw new BusinessException("读取上架结果分片失败");
}
return body;
});
doAnswer(invocation -> {
f.payloadDeletes++;
f.events.add("payload.delete pointer=" + invocation.getArgument(0));
return null;
}).when(payloadStorage).deletePayloadIfPresent(anyString());
when(fileResultMapper.selectOne(any())).thenAnswer(invocation -> {
f.events.add("result.selectOne -> " + (f.result == null ? "-" : "id=" + f.result.getId()));
return f.result;
});
doAnswer(invocation -> {
FileResultEntity row = invocation.getArgument(0);
row.setId(RESULT_ID);
f.result = row;
f.events.add("result.insert taskId=" + row.getTaskId() + " sourceFilename="
+ dash(row.getSourceFilename()) + " rowCount=" + dash(row.getRowCount())
+ " success=" + dash(row.getSuccess()) + " contentType=" + dash(row.getResultContentType()));
return 1;
}).when(fileResultMapper).insert(any(FileResultEntity.class));
doAnswer(invocation -> {
FileResultEntity row = invocation.getArgument(0);
f.events.add("result.update id=" + row.getId() + " success=" + dash(row.getSuccess())
+ " rowCount=" + dash(row.getRowCount()) + " error=" + dash(row.getErrorMessage()));
return 1;
}).when(fileResultMapper).updateById(any(FileResultEntity.class));
when(taskFileJobService.enqueueAssembleResult(anyLong(), anyString(), any(), anyString()))
.thenAnswer(invocation -> {
f.jobEnqueues++;
f.events.add("job.enqueue resultId=" + invocation.getArgument(2)
+ " scopeKey=" + invocation.getArgument(3));
return null;
});
doAnswer(transactionAnswer(f)).when(transactionTemplate).executeWithoutResult(any());
return new PublishTaskService(mock(LocalFileStorageService.class), mock(ZiniaoShopSwitchService.class),
mock(PublishWorkbookService.class), publishFileMapper, publishItemMapper, fileTaskMapper,
fileResultMapper, taskChunkMapper, taskScopeStateMapper, taskFileJobService, lockService,
payloadStorage, mock(OssStorageService.class), objectMapper, transactionTemplate,
mock(DistributedJobLockService.class), instanceMetadata,
mock(TaskProgressLightAssembler.class), mock(TaskHeartbeatPositionService.class));
}
private static Answer<Integer> recordTaskUpdate(Fixture f) {
return invocation -> {
FileTaskEntity task = invocation.getArgument(0);
f.events.add("task.update id=" + task.getId() + " status=" + dash(task.getStatus())
+ " successFileCount=" + dash(task.getSuccessFileCount())
+ " failedFileCount=" + dash(task.getFailedFileCount())
+ " errorMessage=" + dash(task.getErrorMessage())
+ " finishedAt=" + (task.getFinishedAt() == null ? "-" : "set"));
return 1;
};
}
/**
* 事务替身:照 {@code TransactionTemplate} 的做法把 synchronization 开起来,
* 让 {@code deletePayloadsAfterCommit} 走的分支与现网一致(远端删除排在事务返回之后)。
*/
private static Answer<Void> transactionAnswer(Fixture f) {
return invocation -> {
Consumer<TransactionStatus> callback = invocation.getArgument(0);
f.events.add("tx.begin");
boolean alreadyActive = TransactionSynchronizationManager.isSynchronizationActive();
if (!alreadyActive) {
TransactionSynchronizationManager.initSynchronization();
}
try {
callback.accept(mock(TransactionStatus.class));
f.events.add("tx.commit");
if (!alreadyActive) {
for (TransactionSynchronization synchronization
: TransactionSynchronizationManager.getSynchronizations()) {
synchronization.afterCommit();
}
}
} catch (RuntimeException ex) {
f.events.add("tx.rollback " + ex.getClass().getSimpleName() + " " + dash(ex.getMessage()));
if (!alreadyActive) {
for (TransactionSynchronization synchronization
: TransactionSynchronizationManager.getSynchronizations()) {
synchronization.afterCompletion(TransactionSynchronization.STATUS_ROLLED_BACK);
}
}
throw ex;
} finally {
if (!alreadyActive) {
TransactionSynchronizationManager.clearSynchronization();
}
}
return null;
};
}
// ------------------------------------------------------------------
// 用例
// ------------------------------------------------------------------
@Test
void goldenMatchesCommittedSnapshot() throws Exception {
assumeGoldenExists();
String written = renderAllCases();
assertEquals(Files.readString(GOLDEN_SUBMIT.toPath(), StandardCharsets.UTF_8), written,
"回传段流水与在册 golden 不一致——先确认是行为变了,还是替身/时钟进了快照");
}
@Test
void cleanupFailureAfterCommitIsSwallowedAndCallerStillSeesSuccess() {
Fixture fixture = runningFixture();
PublishTaskService service = newService(fixture);
// 既要真抛(触发吞异常那一格),也要留下事件——不然断言只能证明「替身没被叫到」,证不了现网走到哪
doAnswer(invocation -> {
fixture.events.add("chunk.deleteAll");
throw new IllegalStateException("chunk cleanup exploded");
}).when(fixture.chunkMapper).delete(any());
service.submitResult(TASK_ID, request(USER_ID, fileDto(FILE_ID, null, null, "全部失败")));
assertTrue(fixture.events.stream().anyMatch(line -> line.startsWith("chunk.deleteAll")),
"没走到清理步骤,这一格就没在测「清理自己炸」那条吞异常路径");
assertTrue(fixture.events.stream().noneMatch(line -> line.startsWith("scope.deleteAll")),
"chunk 删除炸了就不该再往下删 scope——现网这两步在同一个方法里顺序执行,"
+ "抛出点后由 submitResult 的 catch 吞掉");
}
@Test
void generatedGoldenIsByteStableAcrossTwoRuns() throws Exception {
assertEquals(renderAllCases(), renderAllCases(),
"同一夹具两次流水不一致——多半是时钟、雪花 id 或 Map 迭代顺序进了快照");
}
// ------------------------------------------------------------------
// 渲染
// ------------------------------------------------------------------
private static String renderAllCases() {
StringBuilder golden = new StringBuilder();
appendCase(golden, "completeSingleChunk", completeCase1(),
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"), row("B002", "US"))));
appendCase(golden, "partialChunkWaitsForRest", runningFixture(),
request(USER_ID, fileDto(FILE_ID, 1, 2, null, row("B001", "US"))));
appendCase(golden, "lastChunkCompletesMerge", lastChunkCase(),
request(USER_ID, fileDto(FILE_ID, 2, 2, null, row("B002", "US"))));
appendCase(golden, "errorFileFailsAllSoCleanupRuns", runningFixture(),
request(USER_ID, fileDto(FILE_ID, null, null, "打开文件失败")));
Fixture alreadySuccess = runningFixture();
alreadySuccess.files.put(FILE_ID, file(FILE_ID, "SUCCESS"));
appendCase(golden, "fileAlreadySuccessSkipped", alreadySuccess,
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
Fixture taskSuccess = runningFixture();
taskSuccess.task = task("SUCCESS");
appendCase(golden, "taskSuccessShortCircuitsWithoutThrowing", taskSuccess,
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
Fixture taskFailed = runningFixture();
taskFailed.task = task("FAILED");
appendCase(golden, "taskFailedRejectedBeforeLockWork", taskFailed,
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
Fixture pending = runningFixture();
pending.task = task("PENDING");
pending.itemCount = 1L;
appendCase(golden, "pendingFlipsToRunningOnFirstChunk", pending,
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
Fixture missingTask = runningFixture();
missingTask.task = null;
appendCase(golden, "taskNotFoundAfterLock", missingTask,
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
appendCase(golden, "illegalTaskIdRejectedAfterLock", runningFixture(),
TASK_ID - 1, request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
Fixture wrongModule = runningFixture();
wrongModule.task.setModuleType("COLLECT_DATA");
appendCase(golden, "wrongModuleTypeRejectedAsNotFound", wrongModule,
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
appendCase(golden, "userIdMismatchRejectedAsNotFound", runningFixture(),
request(USER_ID + 1, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
Fixture nullUserId = runningFixture();
nullUserId.itemCount = 1L;
appendCase(golden, "nullUserIdSkipsOwnershipCheck", nullUserId,
request(null, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
appendCase(golden, "illegalUserIdRejectedBeforeLock", runningFixture(),
request(0L, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
appendCase(golden, "emptyFileListRejectedBeforeLock", runningFixture(), request(USER_ID));
appendCase(golden, "nullRequestRejectedBeforeLock", runningFixture(), null);
PublishSubmitResultRequest withNullFile = new PublishSubmitResultRequest();
withNullFile.setUserId(USER_ID);
// List.of 不收 null(当场 NPE),这里要的就是「列表里有一个 null 元素」这一形
withNullFile.setFiles(new ArrayList<>(java.util.Arrays.asList((PublishResultFileDto) null)));
appendCase(golden, "nullFileObjectRejected", runningFixture(), withNullFile);
appendCase(golden, "duplicateFileInOneRequestRejected", runningFixture(),
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US")),
fileDto(FILE_ID, 1, 1, null, row("B002", "US"))));
Fixture locatedByKey = runningFixture();
PublishResultFileDto byKey = fileDto(null, 1, 1, null, row("B001", "US"));
byKey.setFileKey(locatedByKey.files.get(FILE_ID).getFileKey());
locatedByKey.itemCount = 1L;
appendCase(golden, "locateByFileKeyUsesSelectOne", locatedByKey, request(USER_ID, byKey));
Fixture locatedByName = runningFixture();
PublishResultFileDto byName = fileDto(null, 1, 1, null, row("B001", "US"));
byName.setFileKey(null);
byName.setSourceFilename(locatedByName.files.get(FILE_ID).getSourceFilename());
locatedByName.itemCount = 1L;
appendCase(golden, "locateBySourceFilenameUsesSelectOne", locatedByName, request(USER_ID, byName));
appendCase(golden, "fileWithoutAnyLocatorRejected", runningFixture(),
request(USER_ID, fileDto(null, 1, 1, null, row("B001", "US"))));
Fixture lockBusy = runningFixture();
lockBusy.lockAvailable = false;
appendCase(golden, "lockBusyRejectedBeforeAnyLoad", lockBusy,
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
Fixture rustfsDisabled = runningFixture();
rustfsDisabled.rustfsEnabled = false;
appendCase(golden, "rustfsDisabledRejectsBeforeStoring", rustfsDisabled,
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
Fixture localPointer = runningFixture();
localPointer.rustfsPointer = false;
appendCase(golden, "nonRustfsPointerDeletesThenThrows", localPointer,
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
PublishSubmitResultRequest blankRows = new PublishSubmitResultRequest();
blankRows.setUserId(USER_ID);
PublishResultFileDto blankDto = fileDto(FILE_ID, 1, 1, null);
blankDto.setRows(new ArrayList<>(List.of(new PublishRowDto())));
blankRows.setFiles(new ArrayList<>(List.of(blankDto)));
appendCase(golden, "blankRowRejectedBeforeStoring", runningFixture(), blankRows);
Fixture ownerMismatch = runningFixture();
ownerMismatch.task.setResultJson("{\"ownerInstanceId\":\"node-B\"}");
appendCase(golden, "ownerMismatchRejectedInPrepare", ownerMismatch,
request(USER_ID, fileDto(FILE_ID, 1, 1, null, row("B001", "US"))));
Fixture scopeTotalMismatch = runningFixture();
scopeTotalMismatch.scope = scopeRow(3);
appendCase(golden, "chunkTotalMustMatchScope", scopeTotalMismatch,
request(USER_ID, fileDto(FILE_ID, 1, 2, null, row("B001", "US"))));
return golden.toString();
}
private static Fixture completeCase1() {
Fixture fixture = runningFixture();
fixture.itemCount = 2L;
return fixture;
}
private static Fixture lastChunkCase() {
Fixture fixture = runningFixture();
fixture.chunks.add(storedChunk(1, 2, "rustfs:chunk-1", "hash-chunk-1"));
fixture.payloadBody.put("rustfs:chunk-1", "[{\"asin\":\"B000\",\"country\":\"US\"}]");
fixture.scope = scopeRow(2);
fixture.itemCount = 2L;
return fixture;
}
@EnabledIfEnvironmentVariable(named = "SHUFU_SNAPSHOT_WRITE", matches = "true")
@Test
void writeGolden() throws Exception {
Files.createDirectories(GOLDEN_SUBMIT.toPath().getParent());
Files.writeString(GOLDEN_SUBMIT.toPath(), renderAllCases(), StandardCharsets.UTF_8);
}
private static void appendCase(StringBuilder golden, String name, Fixture fixture,
PublishSubmitResultRequest request) {
appendCase(golden, name, fixture, TASK_ID, request);
}
private static void appendCase(StringBuilder golden, String name, Fixture fixture,
Long taskId, PublishSubmitResultRequest request) {
golden.append("== case=").append(name).append('\n');
PublishTaskService service = newService(fixture);
try {
service.submitResult(taskId, request);
golden.append("result=void\n");
} catch (Exception ex) {
golden.append("throw=").append(ex.getClass().getSimpleName()).append(" msg=")
.append(dash(ex.getMessage())).append('\n');
}
fixture.events.forEach(line -> golden.append(line).append('\n'));
golden.append(counters(fixture)).append('\n');
}
private static String counters(Fixture f) {
return "counters lockAcquire=" + f.events.stream().filter(e -> e.startsWith("lock.acquire")).count()
+ " taskStatus=" + dash(f.task == null ? null : f.task.getStatus())
+ " successFileCount=" + dash(f.task == null ? null : f.task.getSuccessFileCount())
+ " failedFileCount=" + dash(f.task == null ? null : f.task.getFailedFileCount())
+ " errorMessage=" + dash(f.task == null ? null : f.task.getErrorMessage())
+ " files=[" + renderFiles(List.copyOf(f.files.values())) + "]"
+ " chunks=" + f.chunks.size()
+ " scope=" + (f.scope == null ? "-" : f.scope.getId())
+ " result=" + (f.result == null ? "-" : f.result.getId())
+ " jobEnqueue=" + f.jobEnqueues
+ " payloadDelete=" + f.payloadDeletes;
}
private static String renderFiles(List<PublishFileEntity> files) {
StringBuilder rendered = new StringBuilder();
for (PublishFileEntity file : files) {
if (!rendered.isEmpty()) {
rendered.append(", ");
}
rendered.append("id=").append(file.getId())
.append("/").append(dash(file.getStatus()))
.append("/total=").append(dash(file.getTotalRows()))
.append("/processed=").append(dash(file.getProcessedRows()))
.append("/err=").append(dash(file.getErrorMessage()));
}
return rendered.toString();
}
private static String renderChunkIndexes(List<TaskChunkEntity> chunks) {
StringBuilder rendered = new StringBuilder();
for (TaskChunkEntity chunk : chunks) {
if (!rendered.isEmpty()) {
rendered.append(",");
}
rendered.append(chunk.getChunkIndex()).append('/').append(dash(chunk.getChunkTotal()));
}
return rendered.toString();
}
// ------------------------------------------------------------------
// 夹具小工具
// ------------------------------------------------------------------
private static Fixture runningFixture() {
Fixture fixture = new Fixture();
fixture.task = task("RUNNING");
fixture.files.put(FILE_ID, file(FILE_ID, "PENDING"));
return fixture;
}
private static TaskScopeStateEntity scopeRow(int chunkTotal) {
TaskScopeStateEntity scope = new TaskScopeStateEntity();
scope.setId(7001L);
scope.setTaskId(TASK_ID);
scope.setModuleType("PUBLISH");
scope.setScopeKey("file:" + FILE_ID);
scope.setScopeHash("scope-hash-" + FILE_ID);
scope.setChunkTotal(chunkTotal);
scope.setStateJson("{\"receivedRowCount\":0}");
return scope;
}
/**
* 替身必须按 wrapper 的绑定值匹配,否则「非法入参」那些格证的是替身形状而不是现网语义。
*
* <p><b>为什么先调一次 {@code getSqlSegment()}。</b>MyBatis-Plus 3.5.7 的 {@code eq(col, val)}
* 只把「怎么拼」存成 {@code ISqlSegment} 闭包,参数真正落进 {@code paramNameValuePairs} 是在
* 生成 SQL 那一刻。直接读那个 Map 会拿到<b>空表</b>——于是「按 chunkIndex 找已有分片」这种
* 查询在替身里退化成「随便返回第一只」,多分片用例就会走出「同一 chunk_index 已回传不同内容」
* 这条现网根本不会走的路径,而快照照样自洽、测试照样绿(3C 那条「别拿替身的形状当现网语义」
* 的同一失效模式,这次是从量具侧进来的)。
*/
private static Collection<Object> boundValues(Object wrapper) {
if (wrapper instanceof AbstractWrapper<?, ?, ?> abstractWrapper) {
abstractWrapper.getSqlSegment();
return abstractWrapper.getParamNameValuePairs().values();
}
return List.of();
}
private static TaskChunkEntity matchChunk(Fixture f, Collection<Object> values) {
Integer index = intIn(values);
for (TaskChunkEntity chunk : f.chunks) {
if (index == null || Objects.equals(index, chunk.getChunkIndex())) {
return chunk;
}
}
return null;
}
private static Integer intIn(Collection<Object> values) {
for (Object value : values) {
if (value instanceof Integer integer) {
return integer;
}
}
return null;
}
private static String dash(Object value) {
return value == null ? "-" : String.valueOf(value);
}
private static int length(String value) {
return value == null ? 0 : value.length();
}
private static String hash(String value) {
if (value == null) {
return "-";
}
return cn.hutool.crypto.digest.DigestUtil.sha256Hex(value).substring(0, 12);
}
private static void assumeGoldenExists() {
org.junit.jupiter.api.Assumptions.assumeTrue(GOLDEN_SUBMIT.exists(),
"golden 还没生成:先跑 SHUFU_SNAPSHOT_WRITE=true 那一次");
}
}
@@ -0,0 +1,358 @@
== case=completeSingleChunk
result=void
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
storage.ensure sharedWrite=true
payload.storeChunk idx=1 len=271 hash=514c20b21e0f
chunk.selectList size=0 idx=
payload.resolve pointer=rustfs:chunk-1 -> len=271 hash=514c20b21e0f
tx.begin
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
chunk.insert idx=1/1 pointer=rustfs:chunk-1 hash=514c20b21e0fe230bc34f2d44ce54d57c46c0cf804c51e9bfd600a9cf43ae104
chunk.selectList size=1 idx=1/1
scope.selectOne -> -
scope.insert chunkTotal=1 stateJson.len=37 stateJson.hash=ed021c36b95a
item.count -> 2
item.delete
item.insertBatch n=2
file.update id=9101 status=SUCCESS totalRows=2 processedRows=2 error=-
files.list size=1 id=9101/SUCCESS/total=2/processed=2/err=-
result.selectOne -> -
result.insert taskId=1001 sourceFilename=PUBLISH-1 rowCount=- success=0 contentType=application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
task.update id=1001 status=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- finishedAt=-
job.enqueue resultId=2001 scopeKey=task:1001:owner:node-A
tx.commit
counters lockAcquire=1 taskStatus=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- files=[id=9101/SUCCESS/total=2/processed=2/err=-] chunks=1 scope=7001 result=2001 jobEnqueue=1 payloadDelete=0
== case=partialChunkWaitsForRest
result=void
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
storage.ensure sharedWrite=true
payload.storeChunk idx=1 len=136 hash=4528ed738ff3
chunk.selectList size=0 idx=
tx.begin
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
chunk.insert idx=1/2 pointer=rustfs:chunk-1 hash=4528ed738ff38dd22d57966b149083a6b86f9cecbfc7adc3a82273c149c1d968
chunk.selectList size=1 idx=1/2
scope.selectOne -> -
scope.insert chunkTotal=2 stateJson.len=38 stateJson.hash=22e615ebaa75
item.count -> 0
file.update id=9101 status=RUNNING totalRows=0 processedRows=1 error=-
files.list size=1 id=9101/RUNNING/total=0/processed=1/err=-
result.selectOne -> -
result.insert taskId=1001 sourceFilename=PUBLISH-1 rowCount=- success=0 contentType=application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
task.update id=1001 status=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- finishedAt=-
tx.commit
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/RUNNING/total=0/processed=1/err=-] chunks=1 scope=7001 result=2001 jobEnqueue=0 payloadDelete=0
== case=lastChunkCompletesMerge
result=void
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> id=7001 chunkTotal=2 stateJson.len=22 stateJson.hash=505748527ede
chunk.selectOne idx=2 -> -
storage.ensure sharedWrite=true
payload.storeChunk idx=2 len=136 hash=fa72d8e86017
chunk.selectList size=1 idx=1/2
payload.resolve pointer=rustfs:chunk-1 -> len=32 hash=0b7b26170d13
payload.resolve pointer=rustfs:chunk-2 -> len=136 hash=fa72d8e86017
tx.begin
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> id=7001 chunkTotal=2 stateJson.len=22 stateJson.hash=505748527ede
chunk.selectOne idx=2 -> -
chunk.insert idx=2/2 pointer=rustfs:chunk-2 hash=fa72d8e86017a3d74173ce8a5c9d2b83ad7239a39a21e26d4b6effa8f8c2d92c
chunk.selectList size=2 idx=1/2,2/2
scope.selectOne -> id=7001 chunkTotal=2 stateJson.len=22 stateJson.hash=505748527ede
scope.update id=7001 chunkTotal=2 stateJson.len=37 stateJson.hash=ed021c36b95a
item.count -> 2
item.delete
item.insertBatch n=2
file.update id=9101 status=SUCCESS totalRows=2 processedRows=2 error=-
files.list size=1 id=9101/SUCCESS/total=2/processed=2/err=-
result.selectOne -> -
result.insert taskId=1001 sourceFilename=PUBLISH-1 rowCount=- success=0 contentType=application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
task.update id=1001 status=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- finishedAt=-
job.enqueue resultId=2001 scopeKey=task:1001:owner:node-A
tx.commit
counters lockAcquire=1 taskStatus=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- files=[id=9101/SUCCESS/total=2/processed=2/err=-] chunks=2 scope=7001 result=2001 jobEnqueue=1 payloadDelete=0
== case=errorFileFailsAllSoCleanupRuns
result=void
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
tx.begin
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
file.update id=9101 status=FAILED totalRows=0 processedRows=0 error=打开文件失败
files.list size=1 id=9101/FAILED/total=0/processed=0/err=打开文件失败
result.selectOne -> -
result.insert taskId=1001 sourceFilename=PUBLISH-1 rowCount=- success=0 contentType=application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
task.update id=1001 status=FAILED successFileCount=0 failedFileCount=1 errorMessage=全部文件处理失败 finishedAt=set
result.update id=2001 success=0 rowCount=- error=全部文件处理失败
tx.commit
chunk.selectList size=0 idx=
chunk.deleteAll
scope.deleteAll
counters lockAcquire=1 taskStatus=FAILED successFileCount=0 failedFileCount=1 errorMessage=全部文件处理失败 files=[id=9101/FAILED/total=0/processed=0/err=打开文件失败] chunks=0 scope=- result=2001 jobEnqueue=0 payloadDelete=0
== case=fileAlreadySuccessSkipped
result=void
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=SUCCESS
tx.begin
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=SUCCESS
files.list size=1 id=9101/SUCCESS/total=0/processed=0/err=-
result.selectOne -> -
result.insert taskId=1001 sourceFilename=PUBLISH-1 rowCount=- success=0 contentType=application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
task.update id=1001 status=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- finishedAt=-
job.enqueue resultId=2001 scopeKey=task:1001:owner:node-A
tx.commit
counters lockAcquire=1 taskStatus=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- files=[id=9101/SUCCESS/total=0/processed=0/err=-] chunks=0 scope=- result=2001 jobEnqueue=1 payloadDelete=0
== case=taskSuccessShortCircuitsWithoutThrowing
result=void
lock.acquire taskId=1001
task.load id=1001 status=SUCCESS module=PUBLISH userId=7
tx.begin
task.load id=1001 status=SUCCESS module=PUBLISH userId=7
tx.commit
counters lockAcquire=1 taskStatus=SUCCESS successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=taskFailedRejectedBeforeLockWork
throw=BusinessException msg=任务已失败,拒绝继续回传
lock.acquire taskId=1001
task.load id=1001 status=FAILED module=PUBLISH userId=7
counters lockAcquire=1 taskStatus=FAILED successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=pendingFlipsToRunningOnFirstChunk
result=void
lock.acquire taskId=1001
task.load id=1001 status=PENDING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
storage.ensure sharedWrite=true
payload.storeChunk idx=1 len=136 hash=4528ed738ff3
chunk.selectList size=0 idx=
payload.resolve pointer=rustfs:chunk-1 -> len=136 hash=4528ed738ff3
tx.begin
task.load id=1001 status=PENDING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
chunk.insert idx=1/1 pointer=rustfs:chunk-1 hash=4528ed738ff38dd22d57966b149083a6b86f9cecbfc7adc3a82273c149c1d968
chunk.selectList size=1 idx=1/1
scope.selectOne -> -
scope.insert chunkTotal=1 stateJson.len=37 stateJson.hash=bfb9a54b7c1d
item.count -> 1
item.delete
item.insertBatch n=1
file.update id=9101 status=SUCCESS totalRows=1 processedRows=1 error=-
files.list size=1 id=9101/SUCCESS/total=1/processed=1/err=-
result.selectOne -> -
result.insert taskId=1001 sourceFilename=PUBLISH-1 rowCount=- success=0 contentType=application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
task.update id=1001 status=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- finishedAt=-
job.enqueue resultId=2001 scopeKey=task:1001:owner:node-A
tx.commit
counters lockAcquire=1 taskStatus=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- files=[id=9101/SUCCESS/total=1/processed=1/err=-] chunks=1 scope=7001 result=2001 jobEnqueue=1 payloadDelete=0
== case=taskNotFoundAfterLock
throw=BusinessException msg=任务不存在
lock.acquire taskId=1001
task.load id=1001 status=- module=- userId=-
counters lockAcquire=1 taskStatus=- successFileCount=- failedFileCount=- errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=illegalTaskIdRejectedAfterLock
throw=BusinessException msg=任务不存在
lock.acquire taskId=1000
task.load id=1000 status=- module=- userId=-
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=wrongModuleTypeRejectedAsNotFound
throw=BusinessException msg=任务不存在
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=COLLECT_DATA userId=7
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=userIdMismatchRejectedAsNotFound
throw=BusinessException msg=任务不存在
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=nullUserIdSkipsOwnershipCheck
result=void
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
storage.ensure sharedWrite=true
payload.storeChunk idx=1 len=136 hash=4528ed738ff3
chunk.selectList size=0 idx=
payload.resolve pointer=rustfs:chunk-1 -> len=136 hash=4528ed738ff3
tx.begin
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
chunk.insert idx=1/1 pointer=rustfs:chunk-1 hash=4528ed738ff38dd22d57966b149083a6b86f9cecbfc7adc3a82273c149c1d968
chunk.selectList size=1 idx=1/1
scope.selectOne -> -
scope.insert chunkTotal=1 stateJson.len=37 stateJson.hash=bfb9a54b7c1d
item.count -> 1
item.delete
item.insertBatch n=1
file.update id=9101 status=SUCCESS totalRows=1 processedRows=1 error=-
files.list size=1 id=9101/SUCCESS/total=1/processed=1/err=-
result.selectOne -> -
result.insert taskId=1001 sourceFilename=PUBLISH-1 rowCount=- success=0 contentType=application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
task.update id=1001 status=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- finishedAt=-
job.enqueue resultId=2001 scopeKey=task:1001:owner:node-A
tx.commit
counters lockAcquire=1 taskStatus=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- files=[id=9101/SUCCESS/total=1/processed=1/err=-] chunks=1 scope=7001 result=2001 jobEnqueue=1 payloadDelete=0
== case=illegalUserIdRejectedBeforeLock
throw=BusinessException msg=user_id 不合法
counters lockAcquire=0 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=emptyFileListRejectedBeforeLock
throw=BusinessException msg=文件列表不能为空
counters lockAcquire=0 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=nullRequestRejectedBeforeLock
throw=BusinessException msg=文件列表不能为空
counters lockAcquire=0 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=nullFileObjectRejected
throw=BusinessException msg=files 不能包含空对象
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=duplicateFileInOneRequestRejected
throw=BusinessException msg=同一文件不能在一次请求中重复提交
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
storage.ensure sharedWrite=true
payload.storeChunk idx=1 len=136 hash=4528ed738ff3
chunk.selectList size=0 idx=
payload.resolve pointer=rustfs:chunk-1 -> len=136 hash=4528ed738ff3
file.selectById id=9101 status=PENDING
chunk.countReferenced -> 0
payload.delete pointer=rustfs:chunk-1
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=1
== case=locateByFileKeyUsesSelectOne
result=void
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectOne -> id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
storage.ensure sharedWrite=true
payload.storeChunk idx=1 len=136 hash=4528ed738ff3
chunk.selectList size=0 idx=
payload.resolve pointer=rustfs:chunk-1 -> len=136 hash=4528ed738ff3
tx.begin
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
chunk.insert idx=1/1 pointer=rustfs:chunk-1 hash=4528ed738ff38dd22d57966b149083a6b86f9cecbfc7adc3a82273c149c1d968
chunk.selectList size=1 idx=1/1
scope.selectOne -> -
scope.insert chunkTotal=1 stateJson.len=37 stateJson.hash=bfb9a54b7c1d
item.count -> 1
item.delete
item.insertBatch n=1
file.update id=9101 status=SUCCESS totalRows=1 processedRows=1 error=-
files.list size=1 id=9101/SUCCESS/total=1/processed=1/err=-
result.selectOne -> -
result.insert taskId=1001 sourceFilename=PUBLISH-1 rowCount=- success=0 contentType=application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
task.update id=1001 status=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- finishedAt=-
job.enqueue resultId=2001 scopeKey=task:1001:owner:node-A
tx.commit
counters lockAcquire=1 taskStatus=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- files=[id=9101/SUCCESS/total=1/processed=1/err=-] chunks=1 scope=7001 result=2001 jobEnqueue=1 payloadDelete=0
== case=locateBySourceFilenameUsesSelectOne
result=void
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectOne -> id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
storage.ensure sharedWrite=true
payload.storeChunk idx=1 len=136 hash=4528ed738ff3
chunk.selectList size=0 idx=
payload.resolve pointer=rustfs:chunk-1 -> len=136 hash=4528ed738ff3
tx.begin
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
chunk.insert idx=1/1 pointer=rustfs:chunk-1 hash=4528ed738ff38dd22d57966b149083a6b86f9cecbfc7adc3a82273c149c1d968
chunk.selectList size=1 idx=1/1
scope.selectOne -> -
scope.insert chunkTotal=1 stateJson.len=37 stateJson.hash=bfb9a54b7c1d
item.count -> 1
item.delete
item.insertBatch n=1
file.update id=9101 status=SUCCESS totalRows=1 processedRows=1 error=-
files.list size=1 id=9101/SUCCESS/total=1/processed=1/err=-
result.selectOne -> -
result.insert taskId=1001 sourceFilename=PUBLISH-1 rowCount=- success=0 contentType=application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
task.update id=1001 status=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- finishedAt=-
job.enqueue resultId=2001 scopeKey=task:1001:owner:node-A
tx.commit
counters lockAcquire=1 taskStatus=RUNNING successFileCount=1 failedFileCount=0 errorMessage=- files=[id=9101/SUCCESS/total=1/processed=1/err=-] chunks=1 scope=7001 result=2001 jobEnqueue=1 payloadDelete=0
== case=fileWithoutAnyLocatorRejected
throw=BusinessException msg=回传文件缺少 fileId/fileKey/sourceFilename
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=lockBusyRejectedBeforeAnyLoad
throw=BusinessException msg=任务正被占用,请稍后重试
lock.acquire taskId=1001
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=rustfsDisabledRejectsBeforeStoring
throw=BusinessException msg=RustFS 未配置,上架结果分片暂不可接收
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
storage.ensure sharedWrite=false
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=nonRustfsPointerDeletesThenThrows
throw=BusinessException msg=上架结果分片必须写入 RustFS
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
chunk.selectOne idx=1 -> -
storage.ensure sharedWrite=true
payload.storeChunk idx=1 len=136 hash=4528ed738ff3
payload.delete pointer=local:chunk-1
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=1
== case=blankRowRejectedBeforeStoring
throw=BusinessException msg=回传数据不能包含空白对象行
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> -
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=ownerMismatchRejectedInPrepare
throw=TaskOwnerMismatchException msg=该任务已绑定到另一台服务实例处理,请通过原实例继续处理
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=- result=- jobEnqueue=0 payloadDelete=0
== case=chunkTotalMustMatchScope
throw=BusinessException msg=同一文件的 chunk_total 必须保持一致
lock.acquire taskId=1001
task.load id=1001 status=RUNNING module=PUBLISH userId=7
file.selectById id=9101 status=PENDING
scope.selectOne -> id=7001 chunkTotal=3 stateJson.len=22 stateJson.hash=505748527ede
counters lockAcquire=1 taskStatus=RUNNING successFileCount=0 failedFileCount=0 errorMessage=- files=[id=9101/PENDING/total=0/processed=0/err=-] chunks=0 scope=7001 result=- jobEnqueue=0 payloadDelete=0