test(phase-3): 3C 前置——补 collectdata 回传段 golden,钉住「两段准入」的现网步序

CollectDataSnapshotTest 只锁到建单+解析+载荷,submitResult 那一段按 §5「先补再动」
必须先在非源上落一笔。本笔吃真实入口 service.submitResult,golden 270 行/15 段 case,
渲染的是调用顺序流水而非分组清单。

这支与另五支最不一样的是**准入发生两次**:锁外预检(快速失败,不为终态任务白跑
去重与品牌检测)+锁内重读复核。于是流水把四个锁外副作用(filter/brandFilter/
asinCache.write/recheck.enqueue)也记下来,terminalRejectedBeforeLock 与
taskNotFoundBeforeLock 两格读出来是 lockAcquire=0 outsideLock=0 ——
「预检在取锁与全部预处理之前」这条形状从此改不动了。

钉住的其他现网形状:
- 重复分片:锁外预处理照跑、锁内 dup 命中后一条库都不写,vo.currentChunkRows=0
- 结果行按需新建(sourceFilename 空串那一形),scopeKey 用 submissionId 回落 "task:{id}"
- done=true 才挂最终组装;带 error 且已收到分片 ⇒ 标失败也照常组装(28599 那起的账)
- 空 ASIN 的行不进过滤链但仍进 chunk payload(结果文件 sheet 要全量)
- requireRustfsPayload 的形状:指针不是 rustfs: 前缀 ⇒ 先 deletePayloadIfPresent 再抛,
  顺序反了就是留垃圾对象(chunkPointerNotRustfs 那格钉住)
- 锁竞争抛的是 code=40902 +「任务正在处理,请稍后再试」——与底座件的
  「任务正在处理中,请稍后再试」差一个字,本笔只登记不统一
- 这支没有「taskId 不合法」那格:非法 id 一路走到「任务不存在」(invalidTaskId... 那格)

长内容只进长度与 sha256 前 12 位(resultJson/chunk payload),时钟与雪花 id 不进快照;
同一夹具跑两次+跨两个 JVM 复跑逐字节相同(golden 由写模式生成后,非写模式复跑一字未动)。
门禁:collectdata 全面+棘轮/边界/面数见证 定向 320 例/0 红,BUILD SUCCESS。
This commit is contained in:
2026-10-19 14:16:37 +08:00
parent 394145deb5
commit c4cf7bab7f
2 changed files with 851 additions and 0 deletions
@@ -0,0 +1,270 @@
== case=middleChunk
storage.ensure sharedWrite=true
task.load id=1001 status=RUNNING module=COLLECT_DATA
outsideLock filter rows=2 owner=7 dedupe=0 invalid=0
outsideLock brandFilter accepted=2 rejected=0 queryFailed=0
outsideLock asinCache.write country=- taskId=1001 rows=2
outsideLock recheck.enqueue n=0
lock.acquire taskId=1001 waitMillis=5000
task.load id=1001 status=RUNNING module=COLLECT_DATA
result.selectOne -> id=2001
chunk.selectOne -> none
payload.storeResult key=chunk-1 len=253 hash=8640b8fb328e
resultItem.upsert chunk=1 accepted=2 ref=rustfs:result-chunk-1
payload.storeChunk idx=1 len=253 hash=8640b8fb328e
chunk.insert scope=sub-1 idx=1/3 payload=rustfs:chunk-1 payloadHash=8640b8fb328efe229676242654815f1ff673b047ff74be70619e2ef8e5a74783
scope.selectOne -> none
chunk.count -> 0
scope.insert chunkTotal=3 received=0 completed=0 lastError=- stateJson={"phase":"RECEIVED"}
task.update id=1001 status=RUNNING errorMessage=- successFileCount=- failedFileCount=- finishedAt=- resultJson.len=333 resultJson.hash=f635be79fec3
task.load id=1001 status=RUNNING module=COLLECT_DATA
thrown: none
vo: taskId=1001 resultId=2001 chunk=1/3 done=false receivedRows=2 currentChunkRows=2 dedupeFiltered=0 invalidFiltered=0 brandRejected=0 brandQueryFailed=0 finalRows=2 taskStatus=RUNNING
counters lockAcquire=1 outsideLock=4 taskUpdate=1 resultUpdate=0 resultInsert=0 chunkInsert=1 scopeWrite=1 jobEnqueue=0
== case=doneChunkEnqueuesWorkbook
storage.ensure sharedWrite=true
task.load id=1001 status=RUNNING module=COLLECT_DATA
outsideLock filter rows=1 owner=7 dedupe=0 invalid=0
outsideLock brandFilter accepted=1 rejected=0 queryFailed=0
outsideLock asinCache.write country=- taskId=1001 rows=1
outsideLock recheck.enqueue n=0
lock.acquire taskId=1001 waitMillis=5000
task.load id=1001 status=RUNNING module=COLLECT_DATA
result.selectOne -> id=2001
chunk.selectOne -> none
payload.storeResult key=chunk-3 len=127 hash=c3d3dc56797d
resultItem.upsert chunk=3 accepted=1 ref=rustfs:result-chunk-3
payload.storeChunk idx=3 len=127 hash=c3d3dc56797d
chunk.insert scope=sub-1 idx=3/3 payload=rustfs:chunk-3 payloadHash=c3d3dc56797d71f2eb35b6c7a0228fff4135195e8758a2e80e849a36e15f556c
scope.selectOne -> none
chunk.count -> 0
scope.insert chunkTotal=3 received=0 completed=1 lastError=- stateJson={"phase":"RECEIVED"}
result.update id=2001 success=0 rowCount=1 resultFilename=collect-data-1001-result.xlsx resultFileUrl=- fileSize=0 errorMessage=-
task.update id=1001 status=RUNNING errorMessage=- successFileCount=- failedFileCount=- finishedAt=- resultJson.len=333 resultJson.hash=2f13867b410c
task.load id=1001 status=RUNNING module=COLLECT_DATA
thrown: none
vo: taskId=1001 resultId=2001 chunk=3/3 done=true receivedRows=1 currentChunkRows=1 dedupeFiltered=0 invalidFiltered=0 brandRejected=0 brandQueryFailed=0 finalRows=1 taskStatus=RUNNING
counters lockAcquire=1 outsideLock=4 taskUpdate=1 resultUpdate=1 resultInsert=0 chunkInsert=1 scopeWrite=1 jobEnqueue=0
== case=duplicateChunkIgnored
storage.ensure sharedWrite=true
task.load id=1001 status=RUNNING module=COLLECT_DATA
outsideLock filter rows=1 owner=7 dedupe=0 invalid=0
outsideLock brandFilter accepted=1 rejected=0 queryFailed=0
outsideLock asinCache.write country=- taskId=1001 rows=1
outsideLock recheck.enqueue n=0
lock.acquire taskId=1001 waitMillis=5000
task.load id=1001 status=RUNNING module=COLLECT_DATA
result.selectOne -> id=2001
chunk.selectOne -> idx=1
thrown: none
vo: taskId=1001 resultId=2001 chunk=1/3 done=false receivedRows=0 currentChunkRows=0 dedupeFiltered=0 invalidFiltered=0 brandRejected=0 brandQueryFailed=0 finalRows=0 taskStatus=RUNNING
counters lockAcquire=1 outsideLock=4 taskUpdate=0 resultUpdate=0 resultInsert=0 chunkInsert=0 scopeWrite=0 jobEnqueue=0
== case=resultRowCreatedOnDemand
storage.ensure sharedWrite=true
task.load id=1001 status=RUNNING module=COLLECT_DATA
outsideLock filter rows=1 owner=7 dedupe=0 invalid=0
outsideLock brandFilter accepted=1 rejected=0 queryFailed=0
outsideLock asinCache.write country=- taskId=1001 rows=1
outsideLock recheck.enqueue n=0
lock.acquire taskId=1001 waitMillis=5000
task.load id=1001 status=RUNNING module=COLLECT_DATA
result.selectOne -> id=-
result.insert taskId=1001 sourceFilename=- rowCount=0 success=-
chunk.selectOne -> none
payload.storeResult key=chunk-1 len=127 hash=23c6b2cbef4e
resultItem.upsert chunk=1 accepted=1 ref=rustfs:result-chunk-1
payload.storeChunk idx=1 len=127 hash=23c6b2cbef4e
chunk.insert scope=sub-1 idx=1/1 payload=rustfs:chunk-1 payloadHash=23c6b2cbef4e8aec3d03e19cd8ad481749cfd1aea7addf69357cb6ea842f0365
scope.selectOne -> none
chunk.count -> 0
scope.insert chunkTotal=1 received=0 completed=1 lastError=- stateJson={"phase":"RECEIVED"}
result.update id=2001 success=- rowCount=1 resultFilename=collect-data-1001-result.xlsx resultFileUrl=- fileSize=0 errorMessage=-
task.update id=1001 status=RUNNING errorMessage=- successFileCount=- failedFileCount=- finishedAt=- resultJson.len=333 resultJson.hash=2f13867b410c
task.load id=1001 status=RUNNING module=COLLECT_DATA
thrown: none
vo: taskId=1001 resultId=2001 chunk=1/1 done=true receivedRows=1 currentChunkRows=1 dedupeFiltered=0 invalidFiltered=0 brandRejected=0 brandQueryFailed=0 finalRows=1 taskStatus=RUNNING
counters lockAcquire=1 outsideLock=4 taskUpdate=1 resultUpdate=1 resultInsert=1 chunkInsert=1 scopeWrite=1 jobEnqueue=0
== case=errorWithReceivedChunks
storage.ensure sharedWrite=true
task.load id=1001 status=RUNNING module=COLLECT_DATA
outsideLock filter rows=1 owner=7 dedupe=0 invalid=0
outsideLock brandFilter accepted=1 rejected=0 queryFailed=0
outsideLock asinCache.write country=- taskId=1001 rows=1
outsideLock recheck.enqueue n=0
lock.acquire taskId=1001 waitMillis=5000
task.load id=1001 status=RUNNING module=COLLECT_DATA
result.selectOne -> id=2001
chunk.selectOne -> none
payload.storeResult key=chunk-2 len=127 hash=ead7a0034044
resultItem.upsert chunk=2 accepted=1 ref=rustfs:result-chunk-2
payload.storeChunk idx=2 len=127 hash=ead7a0034044
chunk.insert scope=sub-1 idx=2/3 payload=rustfs:chunk-2 payloadHash=ead7a00340449db788bc72ac7726b1c98b2d0bf42c9dd7e3a5f806fea91b32de
scope.selectOne -> none
chunk.count -> 2
scope.insert chunkTotal=3 received=2 completed=0 lastError=Python 采集异常 stateJson={"phase":"RECEIVED"}
result.update id=2001 success=0 rowCount=1 resultFilename=- resultFileUrl=- fileSize=- errorMessage=Python 采集异常
task.update id=1001 status=FAILED errorMessage=Python 采集异常 successFileCount=- failedFileCount=1 finishedAt=set resultJson.len=333 resultJson.hash=2f13867b410c
chunk.count -> 2
result.update id=2001 success=0 rowCount=1 resultFilename=collect-data-1001-result.xlsx resultFileUrl=- fileSize=0 errorMessage=Python 采集异常
task.update id=1001 status=FAILED errorMessage=Python 采集异常 successFileCount=- failedFileCount=1 finishedAt=set resultJson.len=333 resultJson.hash=2f13867b410c
task.load id=1001 status=FAILED module=COLLECT_DATA
thrown: none
vo: taskId=1001 resultId=2001 chunk=2/3 done=false receivedRows=1 currentChunkRows=1 dedupeFiltered=0 invalidFiltered=0 brandRejected=0 brandQueryFailed=0 finalRows=1 taskStatus=FAILED
counters lockAcquire=1 outsideLock=4 taskUpdate=2 resultUpdate=2 resultInsert=0 chunkInsert=1 scopeWrite=1 jobEnqueue=0
== case=errorWithoutChunks
storage.ensure sharedWrite=true
task.load id=1001 status=RUNNING module=COLLECT_DATA
outsideLock filter rows=1 owner=7 dedupe=0 invalid=0
outsideLock brandFilter accepted=1 rejected=0 queryFailed=0
outsideLock asinCache.write country=- taskId=1001 rows=1
outsideLock recheck.enqueue n=0
lock.acquire taskId=1001 waitMillis=5000
task.load id=1001 status=RUNNING module=COLLECT_DATA
result.selectOne -> id=2001
chunk.selectOne -> none
payload.storeResult key=chunk-1 len=127 hash=e1eb2ee676be
resultItem.upsert chunk=1 accepted=1 ref=rustfs:result-chunk-1
payload.storeChunk idx=1 len=127 hash=e1eb2ee676be
chunk.insert scope=sub-1 idx=1/2 payload=rustfs:chunk-1 payloadHash=e1eb2ee676be676c79b78c1ff6103b0a6afea0c4fc1c71be60a6119497e1b33c
scope.selectOne -> none
chunk.count -> 0
scope.insert chunkTotal=2 received=0 completed=0 lastError=Python 采集异常 stateJson={"phase":"RECEIVED"}
result.update id=2001 success=0 rowCount=1 resultFilename=- resultFileUrl=- fileSize=- errorMessage=Python 采集异常
task.update id=1001 status=FAILED errorMessage=Python 采集异常 successFileCount=- failedFileCount=1 finishedAt=set resultJson.len=333 resultJson.hash=2f13867b410c
chunk.count -> 0
task.load id=1001 status=FAILED module=COLLECT_DATA
thrown: none
vo: taskId=1001 resultId=2001 chunk=1/2 done=false receivedRows=1 currentChunkRows=1 dedupeFiltered=0 invalidFiltered=0 brandRejected=0 brandQueryFailed=0 finalRows=1 taskStatus=FAILED
counters lockAcquire=1 outsideLock=4 taskUpdate=1 resultUpdate=1 resultInsert=0 chunkInsert=1 scopeWrite=1 jobEnqueue=0
== case=pendingTaskResetToRunning
storage.ensure sharedWrite=true
task.load id=1001 status=PENDING module=COLLECT_DATA
outsideLock filter rows=1 owner=7 dedupe=0 invalid=0
outsideLock brandFilter accepted=1 rejected=0 queryFailed=0
outsideLock asinCache.write country=- taskId=1001 rows=1
outsideLock recheck.enqueue n=0
lock.acquire taskId=1001 waitMillis=5000
task.load id=1001 status=PENDING module=COLLECT_DATA
task.update id=1001 status=RUNNING errorMessage=- successFileCount=- failedFileCount=- finishedAt=- resultJson.len=50 resultJson.hash=4e7ad4d07deb
result.selectOne -> id=2001
chunk.selectOne -> none
payload.storeResult key=chunk-1 len=127 hash=a55ecd3678de
resultItem.upsert chunk=1 accepted=1 ref=rustfs:result-chunk-1
payload.storeChunk idx=1 len=127 hash=a55ecd3678de
chunk.insert scope=sub-1 idx=1/2 payload=rustfs:chunk-1 payloadHash=a55ecd3678decea68f9e2a7b478fdb4e42575fecb11bd2aa15dd849f42a26e7d
scope.selectOne -> none
chunk.count -> 0
scope.insert chunkTotal=2 received=0 completed=0 lastError=- stateJson={"phase":"RECEIVED"}
task.update id=1001 status=RUNNING errorMessage=- successFileCount=- failedFileCount=- finishedAt=- resultJson.len=333 resultJson.hash=2f13867b410c
task.load id=1001 status=RUNNING module=COLLECT_DATA
thrown: none
vo: taskId=1001 resultId=2001 chunk=1/2 done=false receivedRows=1 currentChunkRows=1 dedupeFiltered=0 invalidFiltered=0 brandRejected=0 brandQueryFailed=0 finalRows=1 taskStatus=RUNNING
counters lockAcquire=1 outsideLock=4 taskUpdate=2 resultUpdate=0 resultInsert=0 chunkInsert=1 scopeWrite=1 jobEnqueue=0
== case=blankAsinRowKeptOutOfFilter
storage.ensure sharedWrite=true
task.load id=1001 status=RUNNING module=COLLECT_DATA
outsideLock filter rows=0 owner=7 dedupe=0 invalid=0
outsideLock brandFilter accepted=0 rejected=0 queryFailed=0
outsideLock recheck.enqueue n=0
lock.acquire taskId=1001 waitMillis=5000
task.load id=1001 status=RUNNING module=COLLECT_DATA
result.selectOne -> id=2001
chunk.selectOne -> none
payload.storeChunk idx=1 len=113 hash=e0365b1c5568
chunk.insert scope=sub-1 idx=1/1 payload=rustfs:chunk-1 payloadHash=e0365b1c556879ffbb03ada23f9b4d53a2cbc9cb025e885bef262febfec13cd5
scope.selectOne -> none
chunk.count -> 0
scope.insert chunkTotal=1 received=0 completed=0 lastError=- stateJson={"phase":"RECEIVED"}
task.update id=1001 status=RUNNING errorMessage=- successFileCount=- failedFileCount=- finishedAt=- resultJson.len=333 resultJson.hash=53ca52c40ab0
task.load id=1001 status=RUNNING module=COLLECT_DATA
thrown: none
vo: taskId=1001 resultId=2001 chunk=1/1 done=false receivedRows=1 currentChunkRows=1 dedupeFiltered=0 invalidFiltered=0 brandRejected=0 brandQueryFailed=0 finalRows=0 taskStatus=RUNNING
counters lockAcquire=1 outsideLock=3 taskUpdate=1 resultUpdate=0 resultInsert=0 chunkInsert=1 scopeWrite=1 jobEnqueue=0
== case=terminalRejectedBeforeLock
storage.ensure sharedWrite=true
task.load id=1001 status=SUCCESS module=COLLECT_DATA
thrown: BusinessException code=40901 message=任务已结束,拒绝重复提交
counters lockAcquire=0 outsideLock=0 taskUpdate=0 resultUpdate=0 resultInsert=0 chunkInsert=0 scopeWrite=0 jobEnqueue=0
== case=taskNotFoundBeforeLock
storage.ensure sharedWrite=true
task.load id=1001 status=- module=-
thrown: BusinessException code=- message=任务不存在
counters lockAcquire=0 outsideLock=0 taskUpdate=0 resultUpdate=0 resultInsert=0 chunkInsert=0 scopeWrite=0 jobEnqueue=0
== case=wrongModuleBeforeLock
storage.ensure sharedWrite=true
task.load id=1001 status=RUNNING module=APPEARANCE_PATENT
thrown: BusinessException code=- message=任务不存在
counters lockAcquire=0 outsideLock=0 taskUpdate=0 resultUpdate=0 resultInsert=0 chunkInsert=0 scopeWrite=0 jobEnqueue=0
== case=nullRequest
thrown: BusinessException code=- message=请求内容为空
counters lockAcquire=0 outsideLock=0 taskUpdate=0 resultUpdate=0 resultInsert=0 chunkInsert=0 scopeWrite=0 jobEnqueue=0
== case=rustfsDisabled
storage.ensure sharedWrite=false
thrown: BusinessException code=- message=RustFS 未配置,采集结果回传暂不可接收
counters lockAcquire=0 outsideLock=0 taskUpdate=0 resultUpdate=0 resultInsert=0 chunkInsert=0 scopeWrite=0 jobEnqueue=0
== case=chunkPointerNotRustfs
storage.ensure sharedWrite=true
task.load id=1001 status=RUNNING module=COLLECT_DATA
outsideLock filter rows=1 owner=7 dedupe=0 invalid=0
outsideLock brandFilter accepted=1 rejected=0 queryFailed=0
outsideLock asinCache.write country=- taskId=1001 rows=1
outsideLock recheck.enqueue n=0
lock.acquire taskId=1001 waitMillis=5000
task.load id=1001 status=RUNNING module=COLLECT_DATA
result.selectOne -> id=2001
chunk.selectOne -> none
payload.storeResult key=chunk-1 len=127 hash=b40fdfc4dc5b
resultItem.upsert chunk=1 accepted=1 ref=rustfs:result-chunk-1
payload.storeChunk idx=1 len=127 hash=b40fdfc4dc5b
payload.delete pointer=local:chunk-1
thrown: BusinessException code=- message=采集结果分片必须写入 RustFS
counters lockAcquire=1 outsideLock=4 taskUpdate=0 resultUpdate=0 resultInsert=0 chunkInsert=0 scopeWrite=0 jobEnqueue=0
== case=lockBusyRejected
storage.ensure sharedWrite=true
task.load id=1001 status=RUNNING module=COLLECT_DATA
outsideLock filter rows=1 owner=7 dedupe=0 invalid=0
outsideLock brandFilter accepted=1 rejected=0 queryFailed=0
outsideLock asinCache.write country=- taskId=1001 rows=1
outsideLock recheck.enqueue n=0
lock.acquire taskId=1001 waitMillis=5000 -> busy
thrown: BusinessException code=40902 message=任务正在处理,请稍后再试
counters lockAcquire=1 outsideLock=4 taskUpdate=0 resultUpdate=0 resultInsert=0 chunkInsert=0 scopeWrite=0 jobEnqueue=0
== case=invalidTaskIdFallsThroughToNotFound
storage.ensure sharedWrite=true
task.load id=0 status=RUNNING module=COLLECT_DATA
outsideLock filter rows=1 owner=7 dedupe=0 invalid=0
outsideLock brandFilter accepted=1 rejected=0 queryFailed=0
outsideLock asinCache.write country=- taskId=0 rows=1
outsideLock recheck.enqueue n=0
lock.acquire taskId=0 waitMillis=5000
task.load id=0 status=RUNNING module=COLLECT_DATA
result.selectOne -> id=2001
chunk.selectOne -> none
payload.storeResult key=chunk-1 len=127 hash=291c0ca80f2c
resultItem.upsert chunk=1 accepted=1 ref=rustfs:result-chunk-1
payload.storeChunk idx=1 len=127 hash=291c0ca80f2c
chunk.insert scope=sub-1 idx=1/2 payload=rustfs:chunk-1 payloadHash=291c0ca80f2cd6a87cff2dad9e7dbf122f6111e9dc930c0f8dcbcc7f74bf8f46
scope.selectOne -> none
chunk.count -> 0
scope.insert chunkTotal=2 received=0 completed=0 lastError=- stateJson={"phase":"RECEIVED"}
task.update id=1001 status=RUNNING errorMessage=- successFileCount=- failedFileCount=- finishedAt=- resultJson.len=333 resultJson.hash=2f13867b410c
task.load id=0 status=RUNNING module=COLLECT_DATA
thrown: none
vo: taskId=1001 resultId=2001 chunk=1/2 done=false receivedRows=1 currentChunkRows=1 dedupeFiltered=0 invalidFiltered=0 brandRejected=0 brandQueryFailed=0 finalRows=1 taskStatus=RUNNING
counters lockAcquire=1 outsideLock=4 taskUpdate=1 resultUpdate=0 resultInsert=0 chunkInsert=1 scopeWrite=1 jobEnqueue=0