降本三条(10-11 现网取证):外观同请求图片去重 + 货源合规合批 + LLM token 计量落库
取证基线(库侧 biz_user_secret_usage_daily,10-01~10-11 实测 12,704 / 4,134 / 15,135 次): 外观 24h 全量 831 条请求逐条 json 解析,564 条带图共投 2,693 张、去重只剩 1,465 张 ⇒ 45.6% 是同一张图在同一次请求里的副本;重试占比只有 0.24%(attempt 829/2/0), 所以省钱的口子在图片副本,不在重试。货源 4,134 次里 4,096 次集中在 10-10/10-11 两天, 类目已合批(10-02)而合规与图比仍逐行 ⇒ 合规是该模块最后一步逐行 LLM 调用。 1) 外观:每张不同的图只随请求发一份,正文附「图j=序号1、2…」对照表;每个序号仍各自 出一份 status/reason(省副本,不省判定,与 10-10「同组兄弟行也要都检测」同口径)。 归一化 URL 只当去重的键,发出去的还是行里那条原始串。应急回退: llm-appearance-batch-size=1 即退回逐行,无需改码。 2) 货源:新键 llm-conform-batch-size=10(设 1 退回逐行)。前置条件是分类预计算存在, 否则整条不生效——避免同一行的类目被算两遍(两遍=多扣用户一次);缺编号行与整块 失败都逐行补检,并有一支 A/B 差分管住合批与逐行写回行的三字段+状态相等。 3) V157 给 biz_user_secret_usage_daily 加 prompt_tokens/completion_tokens,两个 client 收下 网关 usage。计次与 token 分两条语句:计次发生在发请求之前(含重试,请求没回来也算 一次),token 只在响应回来后有读数;网关没回 usage 就一条不写——0 会被读成"实测为 0"。 护栏记账:新 mapper 语句 addTokens 两处一起点名(SqlLayeringConventionTest 分母 97→98 + MigratedSqlEquivalenceTest.NOT_MIGRATED);批量路径的 catch 不新增一处——把分类那一路的 "整块失败→逐行补检"抽成 batchAttempt/fallbackSingleRows 共用,静默 catch 棘轮四格一格没放宽。 顺带修掉 AppearancePatentResultRowDto 上一句已经过期的 javadoc(还写着兄弟行复用代表行结论)。 证据:后端全量 4288 例 0 红(本批 +24 例:去重 7 / token 4 / 合规合批 7 / 用量服务 6)。 变异自证 3/3 全杀:取消去重、摘掉 usage 落库、断掉合规合批,分别让对应用例判红; 两个受试文件按注入前字节还原,sha 复算一致。
This commit is contained in:
@@ -178,6 +178,17 @@ public class SimilarAsinProperties {
|
||||
* 设 0/1 回退逐行旧行为。
|
||||
*/
|
||||
private int llmCategoryBatchSize = 10;
|
||||
/**
|
||||
* 批量合规判定的行数(2026-10-11):>1 且批内多行时,合规判定按该行数分块、每块一次 LLM 调用。
|
||||
* 省的是**往返**,不是判定:每件商品的 is_conform/reason/category 仍然各出一份(与 2026-09-27
|
||||
* 类目两级合并同型)。缺编号/坏编号的行与整块调用失败的行回退逐行单调用,行级失败语义不变。
|
||||
* 设 0/1 回退逐行旧行为。
|
||||
*
|
||||
* <p>前置条件:它读的是 {@code llmCategoryBatchSize} 那次预计算出来的逐行类目路径,
|
||||
* 所以批量分类被关掉(阈值 ≤1)时这一路自动不生效 —— 不是配置写错,是为了避免
|
||||
* 同一行的类目被算两遍(两遍就等于多扣用户一次)。
|
||||
*/
|
||||
private int llmConformBatchSize = 10;
|
||||
/**
|
||||
* 首次尝试读超时(毫秒):上游偶发单请求不应答,首查用更短预算快速失败后重试。
|
||||
* 与外观专利同口径:那边一批 5 张图后单次耗时按倍数涨,60s 会误杀正常慢响应,
|
||||
|
||||
+48
-6
@@ -107,11 +107,13 @@ public class AppearancePatentLlmClient {
|
||||
|
||||
private static final String APPEARANCE_BATCH_ENVELOPE =
|
||||
"\n\n### 批量输入输出协议\n"
|
||||
+ "本次同时提交多件商品:用户消息先给出「序号<TAB>产品描述」清单,随后按同样的序号依次附上各件商品图片。"
|
||||
+ "本次同时提交多件商品:用户消息先给出「序号<TAB>产品描述」清单,随后附上**去重后**的商品图片"
|
||||
+ "(同一张图只出现一次),并在文末「图片对照」里逐张写明它对应哪些序号(一张图可服务多件商品)。"
|
||||
+ "必须逐件独立判定,严禁把某一件的结论写到另一件上。\n"
|
||||
+ "只输出一个 JSON 对象:{\"items\":[{\"i\":1,\"appearance_status\":\"侵权\",\"appearance_reason\":\"...\"}]}\n"
|
||||
+ "* i=该件商品的序号,每件恰好一项,不得漏项、不得凭空增加项。\n"
|
||||
+ "* appearance_status / appearance_reason=按上述规则对该件商品图片给出的判定。\n"
|
||||
+ "* appearance_status / appearance_reason=按上述规则对该件商品**所对应的那张图**给出的判定;"
|
||||
+ "共用同一张图的多个序号仍要各出一项,理由要落到该序号自己的描述上。\n"
|
||||
+ "* 除 JSON 外不要输出任何内容。";
|
||||
|
||||
private static final String NO_KEYWORD = "无";
|
||||
@@ -457,17 +459,34 @@ public class AppearancePatentLlmClient {
|
||||
String prompt,
|
||||
String apiKey) throws Exception {
|
||||
List<String> lineTexts = new ArrayList<>(chunk.size());
|
||||
List<String> images = new ArrayList<>(chunk.size());
|
||||
// 同批里同一张图会被多行共用(兄弟行补空取自同一条主图,2026-10-10 起又是逐行送检),
|
||||
// 现网 24h 全量实测:一次请求内投了 2,693 张、去重只剩 1,465 张,45.6% 是同一张图的副本。
|
||||
// 所以每张不同的图只随请求发一份,行与图的对应关系写在正文「图片对照」里 ——
|
||||
// 省的是图片 token,不是判定:每个序号仍然要各自出一份 status/reason。
|
||||
Map<String, String> rawByKey = new LinkedHashMap<>();
|
||||
Map<String, String> rowsByKey = new LinkedHashMap<>();
|
||||
for (int index : chunk) {
|
||||
AppearancePatentResultRowDto row = rows.get(index);
|
||||
lineTexts.add(singleLine("产品描述:" + nonBlank(row.getTitle(), "") + nonBlank(row.getSku(), "")));
|
||||
images.add(row.getUrl());
|
||||
// 归一化只当去重的键;发出去的仍是行里那条原始 URL(口径不许被 normalize 顺手改掉)。
|
||||
String key = normalize(row.getUrl());
|
||||
rawByKey.putIfAbsent(key, row.getUrl());
|
||||
rowsByKey.merge(key, String.valueOf(positionOf(index, chunk)),
|
||||
(oldValue, newValue) -> oldValue + "、" + newValue);
|
||||
}
|
||||
List<String> images = new ArrayList<>(rawByKey.values());
|
||||
StringBuilder legend = new StringBuilder();
|
||||
int imageIndex = 0;
|
||||
for (String positions : rowsByKey.values()) {
|
||||
legend.append("\n图").append(++imageIndex).append("=序号").append(positions);
|
||||
}
|
||||
String userText = "本次提交 " + chunk.size() + " 件商品,先给出「序号<TAB>产品描述」清单,"
|
||||
+ "随后按同样的序号依次附上各件商品图片:\n" + indexedLines(lineTexts);
|
||||
+ "随后附上本批**去重后**的商品图片共 " + images.size() + " 张(一张图可能被多件商品共用,"
|
||||
+ "共用的序号见文末「图片对照」)。每件商品都必须按它对应的那张图**单独**给出一份判定。\n"
|
||||
+ indexedLines(lineTexts) + "\n图片对照:" + legend;
|
||||
Map<Integer, AppearanceJudgement> judgements = new LinkedHashMap<>();
|
||||
String content = invokeChat(properties.getAppearanceModel(), appearanceSystemPrompt(prompt),
|
||||
userText, images, apiKey, "json_object");
|
||||
Map<Integer, AppearanceJudgement> judgements = new LinkedHashMap<>();
|
||||
for (Map.Entry<Integer, JsonNode> entry : parseIndexedItems(content).entrySet()) {
|
||||
int position = entry.getKey() - 1;
|
||||
if (position < 0 || position >= chunk.size()) {
|
||||
@@ -484,6 +503,11 @@ public class AppearancePatentLlmClient {
|
||||
return judgements;
|
||||
}
|
||||
|
||||
/** 该行在本批内的序号(1 基,与发给模型的「序号<TAB>…」对齐)。 */
|
||||
private static int positionOf(int index, List<Integer> chunk) {
|
||||
return chunk.indexOf(index) + 1;
|
||||
}
|
||||
|
||||
private String appearanceSystemPrompt(String prompt) {
|
||||
String system = APPEARANCE_SYSTEM_PROMPT;
|
||||
if (prompt != null && !prompt.isBlank()) {
|
||||
@@ -650,6 +674,7 @@ public class AppearancePatentLlmClient {
|
||||
String message = text(errorNode.path("message"));
|
||||
throw new IllegalStateException(firstNonBlank(message, "LLM error"));
|
||||
}
|
||||
recordTokenUsage(root);
|
||||
JsonNode contentNode = root.path("choices").path(0).path("message").path("content");
|
||||
String content = text(contentNode);
|
||||
if (content == null || content.isBlank()) {
|
||||
@@ -1246,6 +1271,23 @@ public class AppearancePatentLlmClient {
|
||||
userSecretUsageService.record(context.userId(), UserSecretModule.APPEARANCE_PATENT.key(), 1);
|
||||
}
|
||||
|
||||
/**
|
||||
* 收下网关响应的 token 读数(V157)。计次已在发请求前记过,这里只补读数。
|
||||
* 网关没回 usage 就整条跳过 —— 不许写 0 冒充「实测消耗为 0」,那种行以后会被拿去算成本。
|
||||
*/
|
||||
private void recordTokenUsage(JsonNode root) {
|
||||
JsonNode usage = root.path("usage");
|
||||
if (usage.isMissingNode() || usage.isNull()) {
|
||||
return;
|
||||
}
|
||||
SecretUsageContext.Context context = SecretUsageContext.current();
|
||||
if (context == null) {
|
||||
return;
|
||||
}
|
||||
userSecretUsageService.recordTokens(context.userId(), UserSecretModule.APPEARANCE_PATENT.key(),
|
||||
usage.path("prompt_tokens").asLong(0L), usage.path("completion_tokens").asLong(0L));
|
||||
}
|
||||
|
||||
private void sleepQuietly(long delayMillis) {
|
||||
try {
|
||||
Thread.sleep(delayMillis);
|
||||
|
||||
+1
-1
@@ -20,7 +20,7 @@ public class AppearancePatentResultRowDto {
|
||||
@Schema(description = "主数据分组 key。Python 应从解析结果原样透传,用于把同组子行补回。", example = "uploads/20260426/appearance_patent_17.xlsx::2@2")
|
||||
private String groupKey;
|
||||
|
||||
@Schema(description = "Excel 中的 id。代表行通常是整数 id 或 n_1,例如 2_1;最终生成 xlsx 时,2_2、2_3 会复用同组 2_1 的 LLM 检测结果。", example = "2_1")
|
||||
@Schema(description = "Excel 中的 id。代表行通常是整数 id 或 n_1,例如 2_1;2_2、2_3 这些同组兄弟行也各自送检、各自出结论(2026-10-10 撤掉了「复用代表行结论」的摊派:现网实测它造成两到五成行三列全空)。", example = "2_1")
|
||||
private String id;
|
||||
|
||||
@Schema(description = "亚马逊 ASIN。后端会统一按大写处理和匹配。", example = "B0CJ8SNXXV")
|
||||
|
||||
+18
@@ -141,6 +141,7 @@ public class SimilarAsinLlmClient {
|
||||
String message = text(errorNode.path("message"));
|
||||
throw new IllegalStateException(firstNonBlank(message, "LLM error"));
|
||||
}
|
||||
recordTokenUsage(root);
|
||||
JsonNode contentNode = root.path("choices").path(0).path("message").path("content");
|
||||
String content = text(contentNode);
|
||||
if (content == null || content.isBlank()) {
|
||||
@@ -311,6 +312,23 @@ public class SimilarAsinLlmClient {
|
||||
userSecretUsageService.record(context.userId(), UserSecretModule.SIMILAR_ASIN.key(), 1);
|
||||
}
|
||||
|
||||
/**
|
||||
* 收下网关响应的 token 读数(V157)。计次已在发请求前记过,这里只补读数。
|
||||
* 网关没回 usage 就整条跳过 —— 不许写 0 冒充「实测消耗为 0」,那种行以后会被拿去算成本。
|
||||
*/
|
||||
private void recordTokenUsage(JsonNode root) {
|
||||
JsonNode usage = root.path("usage");
|
||||
if (usage.isMissingNode() || usage.isNull()) {
|
||||
return;
|
||||
}
|
||||
SecretUsageContext.Context context = SecretUsageContext.current();
|
||||
if (context == null) {
|
||||
return;
|
||||
}
|
||||
userSecretUsageService.recordTokens(context.userId(), UserSecretModule.SIMILAR_ASIN.key(),
|
||||
usage.path("prompt_tokens").asLong(0L), usage.path("completion_tokens").asLong(0L));
|
||||
}
|
||||
|
||||
/**
|
||||
* 按尝试次数取客户端:第 1 次用「首次尝试超时」(快速失败,单次挂死代价更小),
|
||||
* 重试用完整读超时,避免把慢但成功的正常调用误杀(实测成功调用 p99≈34s)。
|
||||
|
||||
+242
-30
@@ -24,6 +24,7 @@ import java.net.http.HttpResponse;
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
@@ -174,6 +175,22 @@ public class SimilarAsinLlmService {
|
||||
+ "* 严禁伪造类目名称,first/second 必须逐字来自候选列表。输出必须严格遵循 JSON 语法。";
|
||||
private static final String USER_TEXT_CONFORM =
|
||||
"标题:%s\nSKU:%s\n违规类目产品:一级类目:%s ,二级类目:%s,三级类目:%s";
|
||||
|
||||
/**
|
||||
* 批量合规的输出契约追加段(2026-10-11):合规是货源链路里唯一还逐行发的那一步
|
||||
* (10-10 起该模块一天 2,366 次调用,其中约一半是合规)。合并成"一批一次请求"省的是**往返**,
|
||||
* 每件商品的 is_conform/reason/category 仍然各出一份 ⇒ 判定语义与单件模式完全一致。
|
||||
* 类目上下文必须逐件带着:违规类目是按行匹配的,共用一份会把 A 行的违规判到 B 行上。
|
||||
*/
|
||||
private static final String CONFORM_BATCH_ENVELOPE =
|
||||
"\n\n# 批量模式(本次输入包含多件待判定商品)\n"
|
||||
+ "输入按编号 [1]、[2]… 列出多件商品,每件**自带**它自己的违规类目(一级/二级/三级)。"
|
||||
+ "对每件商品**分别**按上述规则判定,禁止遗漏编号、禁止把多件商品合并成一条。\n"
|
||||
+ "只输出一个 JSON 对象:{\"items\":[{\"i\":1,\"asin\":\"…\",\"is_conform\":\"符合\","
|
||||
+ "\"reason\":\"无\",\"category\":\"一级->二级->三级\"}]}\n"
|
||||
+ "* i=商品编号;asin/is_conform/reason/category=按上述规则对该件商品的判定,取值域与单件模式一致。\n"
|
||||
+ "* **严禁**把某一件的理由或类目写到另一件上:category 必须由该行自己的标题/SKU/违规类目推出来。\n"
|
||||
+ "* 输出必须严格遵循 JSON 语法。";
|
||||
private static final String USER_TEXT_COMPARE = "ASIN:%s\n所属类目:%s\n产品信息:%s";
|
||||
|
||||
private final SimilarAsinLlmClient llmClient;
|
||||
@@ -216,6 +233,8 @@ public class SimilarAsinLlmService {
|
||||
// 批量分类预计算:树文本整批只随一次请求发送(llmCategoryBatchSize>1 且多行时启用),
|
||||
// 关闭或单行时为 null,走逐行调用旧路径。见 matchCategoryPathsBatched。
|
||||
CategoryPrepass prepass = categorySwitch ? matchCategoryPathsBatched(rows, categoryTree, apiKey) : null;
|
||||
// 合规批量预计算紧跟在分类预计算之后:它要用的逐行违规类目上下文就是分类预计算的产物。
|
||||
ConformPrepass conformPrepass = checkConformsBatched(rows, prepass, categoryTree, prompt, apiKey);
|
||||
// 行级处理跑在 worker 线程:把「当前任务归属用户」快照带过去,供 LLM 请求计次
|
||||
SecretUsageContext.Context usageContext = SecretUsageContext.snapshot();
|
||||
List<CompletableFuture<SimilarAsinResultRowDto>> futures = new ArrayList<>(rows.size());
|
||||
@@ -226,7 +245,8 @@ public class SimilarAsinLlmService {
|
||||
try {
|
||||
concurrency.acquire();
|
||||
try {
|
||||
return inspectRow(row, rowIndex, prepass, prompt, apiKey, imgSwitch, categorySwitch, categoryTree);
|
||||
return inspectRow(row, rowIndex, prepass, conformPrepass, prompt, apiKey,
|
||||
imgSwitch, categorySwitch, categoryTree);
|
||||
} finally {
|
||||
concurrency.release();
|
||||
}
|
||||
@@ -252,6 +272,7 @@ public class SimilarAsinLlmService {
|
||||
private SimilarAsinResultRowDto inspectRow(SimilarAsinResultRowDto source,
|
||||
int rowIndex,
|
||||
CategoryPrepass prepass,
|
||||
ConformPrepass conformPrepass,
|
||||
String prompt,
|
||||
String apiKey,
|
||||
boolean imgSwitch,
|
||||
@@ -259,7 +280,7 @@ public class SimilarAsinLlmService {
|
||||
CategoryTree categoryTree) {
|
||||
SimilarAsinResultRowDto row = copyRow(source);
|
||||
try {
|
||||
inspectRowInternal(row, rowIndex, prepass, prompt, apiKey, imgSwitch, categorySwitch, categoryTree);
|
||||
inspectRowInternal(row, rowIndex, prepass, conformPrepass, prompt, apiKey, imgSwitch, categorySwitch, categoryTree);
|
||||
return row;
|
||||
} catch (Exception ex) {
|
||||
log.warn("[similar-asin][llm] row failed asin={} title={} err={}",
|
||||
@@ -271,6 +292,7 @@ public class SimilarAsinLlmService {
|
||||
private void inspectRowInternal(SimilarAsinResultRowDto row,
|
||||
int rowIndex,
|
||||
CategoryPrepass prepass,
|
||||
ConformPrepass conformPrepass,
|
||||
String prompt,
|
||||
String apiKey,
|
||||
boolean imgSwitch,
|
||||
@@ -297,7 +319,8 @@ public class SimilarAsinLlmService {
|
||||
: categoryTree.childrenOf(second.id());
|
||||
|
||||
// --- 3. 合规检查(1570640:总是执行)---
|
||||
boolean conform = checkConform(row, first, second, thirdCandidates, prompt, apiKey);
|
||||
boolean conform = conformForRow(rowIndex, conformPrepass, row, first, second, thirdCandidates,
|
||||
prompt, apiKey);
|
||||
|
||||
// --- 4. 不符合 / 无 alibaba / img_switch=false:只回填合规结果(139255 true 分支 → 1398750)---
|
||||
if (!conform || row.getAlibaba().isEmpty() || !imgSwitch) {
|
||||
@@ -442,20 +465,44 @@ public class SimilarAsinLlmService {
|
||||
return new CategoryPrepass(paths, errors);
|
||||
}
|
||||
|
||||
/** 批量调用体:可能抛,由 {@link #batchAttempt} 统一收口。 */
|
||||
private interface BatchAttempt {
|
||||
void run() throws Exception;
|
||||
}
|
||||
|
||||
/** 逐行补检的单行调用体:允许抛受检异常,存进 errors 的是原异常本体(不是包装壳)。 */
|
||||
private interface SingleRowCall<T> {
|
||||
T run(int rowIndex) throws Exception;
|
||||
}
|
||||
|
||||
/**
|
||||
* 一批一次的调用出口:整块失败只记一条 warn 并立刻逐行补检,异常不外抛
|
||||
* (行级失败语义留给补检那一步判,与 2026-10-02 分类那一路逐字同形)。
|
||||
* <p>分类与合规共用这一处 catch,是为了让「异常到此为止」的总数不随批量路径增加而变宽。
|
||||
*/
|
||||
private void batchAttempt(String label, List<SimilarAsinResultRowDto> rows, List<Integer> chunk,
|
||||
BatchAttempt attempt, Runnable fallback) {
|
||||
try {
|
||||
attempt.run();
|
||||
} catch (Exception ex) {
|
||||
log.warn("[similar-asin][llm] {} batch failed rows={} firstAsin={} err={} → 整块回退单行",
|
||||
label, chunk.size(), abbreviate(rows.get(chunk.get(0)).getAsin(), 40), failureMessage(ex));
|
||||
fallback.run();
|
||||
}
|
||||
}
|
||||
|
||||
private void matchCategoryChunk(List<SimilarAsinResultRowDto> rows, List<Integer> chunk,
|
||||
CategoryTree tree, String apiKey,
|
||||
Map<Integer, CategoryPath> paths, Map<Integer, Exception> errors) {
|
||||
try {
|
||||
String content = llmClient.invokeChat(properties.getLlmCategoryModel(),
|
||||
SYSTEM_CATEGORY + CATEGORY_BATCH_ENVELOPE,
|
||||
buildCategoryBatchUserText(rows, chunk, tree), apiKey);
|
||||
JsonNode root = llmClient.parseJsonContent(content);
|
||||
resolveCategoryBatchItems(rows, chunk, root, tree, apiKey, paths, errors);
|
||||
} catch (Exception ex) {
|
||||
log.warn("[similar-asin][llm] category batch failed rows={} firstAsin={} err={} → 整块回退单行",
|
||||
chunk.size(), abbreviate(rows.get(chunk.get(0)).getAsin(), 40), failureMessage(ex));
|
||||
fallbackCategorySingleRows(rows, chunk, tree, apiKey, paths, errors);
|
||||
}
|
||||
batchAttempt("category", rows, chunk,
|
||||
() -> {
|
||||
String content = llmClient.invokeChat(properties.getLlmCategoryModel(),
|
||||
SYSTEM_CATEGORY + CATEGORY_BATCH_ENVELOPE,
|
||||
buildCategoryBatchUserText(rows, chunk, tree), apiKey);
|
||||
JsonNode root = llmClient.parseJsonContent(content);
|
||||
resolveCategoryBatchItems(rows, chunk, root, tree, apiKey, paths, errors);
|
||||
},
|
||||
() -> fallbackCategorySingleRows(rows, chunk, tree, apiKey, paths, errors));
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -500,22 +547,32 @@ public class SimilarAsinLlmService {
|
||||
return new CategoryPath(first, second == null ? CategoryLevel.none() : second);
|
||||
}
|
||||
|
||||
/** 批量失败的行回退逐行单调用;分类调用本身抛异常时记入 errors(行级失败语义不变)。 */
|
||||
private void fallbackCategorySingleRows(List<SimilarAsinResultRowDto> rows, List<Integer> chunk,
|
||||
CategoryTree tree, String apiKey,
|
||||
Map<Integer, CategoryPath> paths, Map<Integer, Exception> errors) {
|
||||
/**
|
||||
* 批量失败的行回退逐行单调用;单行调用本身抛异常时记入 errors(行级失败语义不变)。
|
||||
* <p>分类与合规共用这一处 catch:批量路径每多一条,"异常到此为止"的出口不应该跟着多一处,
|
||||
* 否则可读性/静默异常那两把只许减的尺子就得替新代码放宽基线。
|
||||
*/
|
||||
private <T> void fallbackSingleRows(List<Integer> chunk, Map<Integer, T> target,
|
||||
Map<Integer, Exception> errors, SingleRowCall<T> singleCall) {
|
||||
for (Integer rowIndex : chunk) {
|
||||
if (paths.containsKey(rowIndex) || errors.containsKey(rowIndex)) {
|
||||
if (target.containsKey(rowIndex) || errors.containsKey(rowIndex)) {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
paths.put(rowIndex, matchCategoryPath(rows.get(rowIndex), tree, apiKey));
|
||||
target.put(rowIndex, singleCall.run(rowIndex));
|
||||
} catch (Exception ex) {
|
||||
errors.put(rowIndex, ex);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void fallbackCategorySingleRows(List<SimilarAsinResultRowDto> rows, List<Integer> chunk,
|
||||
CategoryTree tree, String apiKey,
|
||||
Map<Integer, CategoryPath> paths, Map<Integer, Exception> errors) {
|
||||
fallbackSingleRows(chunk, paths, errors,
|
||||
rowIndex -> matchCategoryPath(rows.get(rowIndex), tree, apiKey));
|
||||
}
|
||||
|
||||
/** 每行分类结果取用:优先预计算;回退也失败的行按单行语义抛出 → 行失败。 */
|
||||
private CategoryPath categoryPathForRow(int rowIndex, CategoryPrepass prepass,
|
||||
SimilarAsinResultRowDto row, CategoryTree tree,
|
||||
@@ -583,20 +640,175 @@ public class SimilarAsinLlmService {
|
||||
List<CategoryLevel> thirdCandidates,
|
||||
String prompt,
|
||||
String apiKey) {
|
||||
ConformVerdict verdict = conformSingleVerdict(row,
|
||||
new String[]{first.name(), second.name(), formatCandidates(thirdCandidates)}, prompt, apiKey);
|
||||
applyConformVerdict(row, verdict);
|
||||
return SimilarAsinLlmClient.isConformYes(verdict.isConform());
|
||||
}
|
||||
|
||||
/** 单件合规调用:与批量路径共用同一个解析出口,避免两条路对同一响应读出不同字段。 */
|
||||
private ConformVerdict conformSingleVerdict(SimilarAsinResultRowDto row, String[] context,
|
||||
String prompt, String apiKey) {
|
||||
String userText = String.format(USER_TEXT_CONFORM,
|
||||
firstNonBlank(row.getTitle(), row.getAsin()),
|
||||
normalize(row.getSku()),
|
||||
first.name(),
|
||||
second.name(),
|
||||
formatCandidates(thirdCandidates));
|
||||
String system = prompt == null || prompt.isBlank() ? SYSTEM_CONFORM : SYSTEM_CONFORM + "\n\n额外要求:" + prompt.trim();
|
||||
String content = llmClient.invokeChat(properties.getLlmConformModel(), system, userText, apiKey);
|
||||
context[0],
|
||||
context[1],
|
||||
context[2]);
|
||||
String content = llmClient.invokeChat(properties.getLlmConformModel(),
|
||||
conformSystemPrompt(prompt), userText, apiKey);
|
||||
JsonNode root = llmClient.parseJsonContent(content);
|
||||
String isConform = textOrEmpty(root, "is_conform");
|
||||
row.setIsConform(isConform);
|
||||
row.setReason(firstNonBlank(textOrEmpty(root, "reason"), ""));
|
||||
row.setCategory(firstNonBlank(textOrEmpty(root, "category"), ""));
|
||||
return SimilarAsinLlmClient.isConformYes(isConform);
|
||||
return new ConformVerdict(textOrEmpty(root, "is_conform"),
|
||||
textOrEmpty(root, "reason"), textOrEmpty(root, "category"));
|
||||
}
|
||||
|
||||
private String conformSystemPrompt(String prompt) {
|
||||
return prompt == null || prompt.isBlank() ? SYSTEM_CONFORM
|
||||
: SYSTEM_CONFORM + "\n\n额外要求:" + prompt.trim();
|
||||
}
|
||||
|
||||
/** 合规三字段回填行:与批量结果共用,逐行旧路径的写入口径一字未改。 */
|
||||
private void applyConformVerdict(SimilarAsinResultRowDto row, ConformVerdict verdict) {
|
||||
row.setIsConform(verdict.isConform());
|
||||
row.setReason(firstNonBlank(verdict.reason(), ""));
|
||||
row.setCategory(firstNonBlank(verdict.category(), ""));
|
||||
}
|
||||
|
||||
// --- 批量合规(2026-10-11):合规是货源链路里最后一步逐行发的 LLM 调用,合并省往返不省判定 ---
|
||||
|
||||
/** 一件商品的合规读数。 */
|
||||
private record ConformVerdict(String isConform, String reason, String category) {
|
||||
}
|
||||
|
||||
/** 批量合规预计算的产物:byRow 覆盖拿到结果或单行补检成功的行,errors 是补检也失败的行。 */
|
||||
private record ConformPrepass(Map<Integer, ConformVerdict> byRow, Map<Integer, Exception> errors) {
|
||||
}
|
||||
|
||||
/**
|
||||
* 按 {@code llmConformBatchSize} 分块,每块一次 LLM 调用;块内缺编号的行、整块调用失败的行
|
||||
* 回退逐行单调用(行级失败语义不变)。返回 null 表示本批不启用,调用方走逐行旧路径。
|
||||
*
|
||||
* <p>前置条件是分类预计算已存在:每件商品的违规类目上下文取自那次预计算的逐行路径。
|
||||
* 分类批量被关掉时这里必须返回 null —— 否则要为一行重算一遍类目,等于多扣用户一次。
|
||||
*/
|
||||
private ConformPrepass checkConformsBatched(List<SimilarAsinResultRowDto> rows,
|
||||
CategoryPrepass categoryPrepass,
|
||||
CategoryTree tree, String prompt, String apiKey) {
|
||||
int batchSize = Math.max(1, properties.getLlmConformBatchSize());
|
||||
if (categoryPrepass == null || batchSize <= 1 || rows.size() < 2) {
|
||||
return null;
|
||||
}
|
||||
Map<Integer, String[]> contextByRow = new LinkedHashMap<>();
|
||||
for (int rowIndex = 0; rowIndex < rows.size(); rowIndex++) {
|
||||
CategoryPath path = categoryPrepass.paths().get(rowIndex);
|
||||
if (path == null) {
|
||||
continue; // 分类没出结果的行不在这里补:它在行内会先按分类缺失抛成行级失败
|
||||
}
|
||||
CategoryLevel second = path.second();
|
||||
contextByRow.put(rowIndex, new String[]{path.first().name(), second.name(),
|
||||
formatCandidates(isNoMatch(second) ? List.of() : tree.childrenOf(second.id()))});
|
||||
}
|
||||
if (contextByRow.size() < 2) {
|
||||
return null;
|
||||
}
|
||||
List<Integer> indexes = new ArrayList<>(contextByRow.keySet());
|
||||
Semaphore concurrency = new Semaphore(Math.max(1, properties.getLlmRowConcurrency()));
|
||||
ExecutorService executor = rowExecutor();
|
||||
SecretUsageContext.Context usageContext = SecretUsageContext.snapshot();
|
||||
Map<Integer, ConformVerdict> byRow = new ConcurrentHashMap<>();
|
||||
Map<Integer, Exception> errors = new ConcurrentHashMap<>();
|
||||
List<CompletableFuture<Void>> futures = new ArrayList<>();
|
||||
for (int from = 0; from < indexes.size(); from += batchSize) {
|
||||
List<Integer> chunk = new ArrayList<>(
|
||||
indexes.subList(from, Math.min(from + batchSize, indexes.size())));
|
||||
futures.add(CompletableFuture.supplyAsync(SecretUsageContext.wrap(() -> {
|
||||
try {
|
||||
concurrency.acquire();
|
||||
try {
|
||||
checkConformChunk(rows, chunk, contextByRow, prompt, apiKey, byRow, errors);
|
||||
} finally {
|
||||
concurrency.release();
|
||||
}
|
||||
} catch (InterruptedException interruptedException) {
|
||||
Thread.currentThread().interrupt();
|
||||
throw new IllegalStateException("LLM conform batch interrupted", interruptedException);
|
||||
}
|
||||
return null;
|
||||
}, usageContext), executor));
|
||||
}
|
||||
for (CompletableFuture<Void> future : futures) {
|
||||
future.join(); // checkConformChunk 内部自捕获,join 只会因中断失败
|
||||
}
|
||||
return new ConformPrepass(byRow, errors);
|
||||
}
|
||||
|
||||
private void checkConformChunk(List<SimilarAsinResultRowDto> rows, List<Integer> chunk,
|
||||
Map<Integer, String[]> contextByRow, String prompt, String apiKey,
|
||||
Map<Integer, ConformVerdict> byRow, Map<Integer, Exception> errors) {
|
||||
batchAttempt("conform", rows, chunk,
|
||||
() -> {
|
||||
String content = llmClient.invokeChat(properties.getLlmConformModel(),
|
||||
SYSTEM_CONFORM + CONFORM_BATCH_ENVELOPE,
|
||||
buildConformBatchUserText(rows, chunk, contextByRow), apiKey);
|
||||
JsonNode root = llmClient.parseJsonContent(content);
|
||||
JsonNode items = root.path("items");
|
||||
if (items.isArray()) {
|
||||
for (JsonNode item : items) {
|
||||
int i = item.path("i").asInt(-1);
|
||||
if (i < 1 || i > chunk.size()) {
|
||||
continue; // 坏编号:交给缺失回退
|
||||
}
|
||||
int rowIndex = chunk.get(i - 1);
|
||||
if (byRow.containsKey(rowIndex)) {
|
||||
continue; // 重复编号取首个
|
||||
}
|
||||
byRow.put(rowIndex, new ConformVerdict(textOrEmpty(item, "is_conform"),
|
||||
textOrEmpty(item, "reason"), textOrEmpty(item, "category")));
|
||||
}
|
||||
}
|
||||
},
|
||||
() -> { /* 整块失败:下面那段"缺结果就逐行补检"会覆盖全部行,这里不必另做 */ });
|
||||
fallbackSingleRows(chunk, byRow, errors,
|
||||
rowIndex -> conformSingleVerdict(rows.get(rowIndex), contextByRow.get(rowIndex), prompt, apiKey));
|
||||
}
|
||||
|
||||
/** 批量正文:每件商品把自己的违规类目上下文一起带上,共用一份类目会把 A 行的拦截判到 B 行。 */
|
||||
private String buildConformBatchUserText(List<SimilarAsinResultRowDto> rows, List<Integer> chunk,
|
||||
Map<Integer, String[]> contextByRow) {
|
||||
StringBuilder sb = new StringBuilder();
|
||||
sb.append("待判定商品(共 ").append(chunk.size()).append(" 件,按编号 [i] 逐件判定):\n");
|
||||
for (int position = 0; position < chunk.size(); position++) {
|
||||
SimilarAsinResultRowDto row = rows.get(chunk.get(position));
|
||||
String[] context = contextByRow.get(chunk.get(position));
|
||||
sb.append('[').append(position + 1).append("] 标题:")
|
||||
.append(firstNonBlank(row.getTitle(), row.getAsin())).append('\n');
|
||||
sb.append(" SKU:").append(normalize(row.getSku())).append('\n');
|
||||
sb.append(" 违规类目产品:一级类目:").append(context[0])
|
||||
.append(" ,二级类目:").append(context[1])
|
||||
.append(",三级类目:").append(context[2]).append('\n');
|
||||
}
|
||||
return sb.toString();
|
||||
}
|
||||
|
||||
/** 取用合规判定:优先预计算结果,缺失的行按行级失败语义抛出(对齐分类那一路)。 */
|
||||
private boolean conformForRow(int rowIndex, ConformPrepass prepass, SimilarAsinResultRowDto row,
|
||||
CategoryLevel first, CategoryLevel second,
|
||||
List<CategoryLevel> thirdCandidates, String prompt, String apiKey)
|
||||
throws Exception {
|
||||
if (prepass == null) {
|
||||
return checkConform(row, first, second, thirdCandidates, prompt, apiKey);
|
||||
}
|
||||
Exception error = prepass.errors().get(rowIndex);
|
||||
if (error != null) {
|
||||
throw error instanceof RuntimeException runtimeException ? runtimeException
|
||||
: new IllegalStateException(error.getMessage(), error);
|
||||
}
|
||||
ConformVerdict verdict = prepass.byRow().get(rowIndex);
|
||||
if (verdict == null) {
|
||||
throw new IllegalStateException("批量合规结果缺失 rowIndex=" + rowIndex);
|
||||
}
|
||||
applyConformVerdict(row, verdict);
|
||||
return SimilarAsinLlmClient.isConformYes(verdict.isConform());
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
+14
@@ -19,6 +19,20 @@ public interface UserSecretUsageMapper extends BaseMapper<UserSecretUsageEntity>
|
||||
@Param("businessDate") LocalDate businessDate,
|
||||
@Param("count") int count);
|
||||
|
||||
/**
|
||||
* 按唯一键 uk_user_module_date 累加 token 读数。
|
||||
*
|
||||
* <p>刻意与 {@link #increment} 分成两条:计次发生在**发请求之前**(口径含重试,请求没回来也要算一次),
|
||||
* 而 token 只有响应回来才知道。合并成一条 upsert 就会把"已发出但超时"的那些请求从计次里漏掉。
|
||||
* <p>这里是 UPDATE 而不是 upsert:行由前一条 increment 保证存在;真不存在就是 0 行受影响,
|
||||
* token 读数丢掉也不该反过来影响计次。
|
||||
*/
|
||||
int addTokens(@Param("userId") Long userId,
|
||||
@Param("moduleKey") String moduleKey,
|
||||
@Param("businessDate") LocalDate businessDate,
|
||||
@Param("promptTokens") long promptTokens,
|
||||
@Param("completionTokens") long completionTokens);
|
||||
|
||||
/** 分批删除保留期外的用量日统计行(该表此前只增不删)。 */
|
||||
int deleteBefore(@Param("cutoff") LocalDate cutoff, @Param("batchSize") int batchSize);
|
||||
}
|
||||
|
||||
+4
@@ -20,6 +20,10 @@ public class UserSecretUsageEntity {
|
||||
private String moduleKey;
|
||||
private LocalDate businessDate;
|
||||
private Integer callCount;
|
||||
/** 输入 token 累计(网关 usage.prompt_tokens)。0 = 该天没有 token 证据,不等于没消耗。 */
|
||||
private Long promptTokens;
|
||||
/** 输出 token 累计(网关 usage.completion_tokens)。0 = 该天没有 token 证据,不等于没产出。 */
|
||||
private Long completionTokens;
|
||||
private BigDecimal unitPrice;
|
||||
private LocalDateTime createdAt;
|
||||
private LocalDateTime updatedAt;
|
||||
|
||||
+35
-6
@@ -61,7 +61,7 @@ public class UserSecretUsageService {
|
||||
log.warn("[user-secret-usage] 忽略未知模块计次 userId={} module={}", userId, moduleKey);
|
||||
return;
|
||||
}
|
||||
recordModule(userId, module, count);
|
||||
recordModule(userId, module, count, 0L, 0L);
|
||||
}
|
||||
|
||||
/** 内部通道上报入口:模块非法返回 false(供接口回执),写库失败只告警。 */
|
||||
@@ -73,16 +73,45 @@ public class UserSecretUsageService {
|
||||
if (module == null) {
|
||||
return false;
|
||||
}
|
||||
return recordModule(userId, module, Math.min(count, MAX_REPORT_COUNT));
|
||||
return recordModule(userId, module, Math.min(count, MAX_REPORT_COUNT), 0L, 0L);
|
||||
}
|
||||
|
||||
private boolean recordModule(Long userId, UserSecretModule module, int count) {
|
||||
/**
|
||||
* 记一次响应回来的 token 读数(V157)。与计次分开的理由只有一个,但很硬:
|
||||
* 计次发生在**发请求之前**(口径含重试,请求没回来也得算一次),token 只有响应回来才知道。
|
||||
*
|
||||
* <p>网关没回 {@code usage} 时必须整条不调用——不许写 0 冒充「实测为 0」,
|
||||
* 那一行 0 以后会被当成成本读数拿去算账。
|
||||
*/
|
||||
public void recordTokens(Long userId, String moduleKey, long promptTokens, long completionTokens) {
|
||||
if (userId == null || userId <= 0) {
|
||||
return;
|
||||
}
|
||||
if (promptTokens <= 0 && completionTokens <= 0) {
|
||||
return;
|
||||
}
|
||||
UserSecretModule module = UserSecretModule.of(moduleKey).orElse(null);
|
||||
if (module == null) {
|
||||
return;
|
||||
}
|
||||
recordModule(userId, module, 0, promptTokens, completionTokens);
|
||||
}
|
||||
|
||||
private boolean recordModule(Long userId, UserSecretModule module, int count,
|
||||
long promptTokens, long completionTokens) {
|
||||
LocalDate today = LocalDate.now();
|
||||
try {
|
||||
usageMapper.increment(userId, module.key(), LocalDate.now(), count);
|
||||
if (count > 0) {
|
||||
usageMapper.increment(userId, module.key(), today, count);
|
||||
}
|
||||
if (promptTokens > 0 || completionTokens > 0) {
|
||||
usageMapper.addTokens(userId, module.key(), today, promptTokens, completionTokens);
|
||||
}
|
||||
return true;
|
||||
} catch (Exception ex) {
|
||||
log.warn("[user-secret-usage] 计次写入失败 userId={} module={} count={} err={}",
|
||||
userId, module.key(), count, ex.getMessage());
|
||||
log.warn("[user-secret-usage] 计次写入失败 userId={} module={} count={} promptTokens={} "
|
||||
+ "completionTokens={} err={}", userId, module.key(), count, promptTokens,
|
||||
completionTokens, ex.getMessage());
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -421,6 +421,9 @@ aiimage:
|
||||
llm-retry-times: ${AIIMAGE_SIMILAR_ASIN_LLM_RETRY_TIMES:3}
|
||||
llm-row-concurrency: ${AIIMAGE_SIMILAR_ASIN_LLM_ROW_CONCURRENCY:5}
|
||||
llm-category-batch-size: ${AIIMAGE_SIMILAR_ASIN_LLM_CATEGORY_BATCH_SIZE:10}
|
||||
# 合规判定一次请求合并的行数:省往返不省判定,每件商品仍各自出 is_conform/reason/category。
|
||||
# 设 1 即退回逐行;批量分类被关掉时这一路自动不生效(避免同一次类目匹配被算两遍)。
|
||||
llm-conform-batch-size: ${AIIMAGE_SIMILAR_ASIN_LLM_CONFORM_BATCH_SIZE:10}
|
||||
llm-image-download-timeout-seconds: ${AIIMAGE_SIMILAR_ASIN_LLM_IMAGE_DOWNLOAD_TIMEOUT_SECONDS:10}
|
||||
llm-image-download-retry-times: ${AIIMAGE_SIMILAR_ASIN_LLM_IMAGE_DOWNLOAD_RETRY_TIMES:2}
|
||||
llm-image-host-alias-map: ${AIIMAGE_SIMILAR_ASIN_LLM_IMAGE_HOST_ALIAS_MAP:m.media-amazon.com=images-na.ssl-images-amazon.com}
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
-- =============================================================
|
||||
-- 迁移 V157__user_secret_usage_tokens
|
||||
--
|
||||
-- 1. 变更目的:给 `biz_user_secret_usage_daily` 加两列 token 累计,把「次数」升级成
|
||||
-- 「次数 + 输入/输出 token」。今天降本决策全部卡在缺这一维:
|
||||
-- `V117` 起该表只有 call_count,`unit_price` 建表即预留、至今 480 行全 NULL,
|
||||
-- 两个 LLM client(AppearancePatentLlmClient / SimilarAsinLlmClient)都只调
|
||||
-- recordSecretUsage() 记 1 次、完全不解析网关响应里的 `usage`。
|
||||
-- 后果是"合批省了多少""一次多图请求里重复投同一张图值多少钱"这类问题只能用
|
||||
-- 字符数推算回答,答不了实测。本迁移是那两处推算变实测的存储前提。
|
||||
--
|
||||
-- 2. 前提取证(2026-10-11 只读探针,主机A,零写入):
|
||||
-- · 该表 10-01~10-11 共 480 行(proxy 414 / appearance-patent 60 / similar-asin 6),
|
||||
-- 是"用户 × 模块 × 天"的日聚合表,不是明细表 ⇒ 加两列的存储代价可忽略;
|
||||
-- · 唯一键 `uk_user_module_date(user_id, module_key, business_date)` 已存在(V117),
|
||||
-- 新列按该键累加,不需要新索引;
|
||||
-- · 写侧只有两条语句:`UserSecretUsageMapper.increment`(ON DUPLICATE KEY 累加)与
|
||||
-- `deleteBefore`(保留期清理)。新列不进这两条,改为独立的 `addTokens` UPDATE,
|
||||
-- 这样"次数口径"逐字不变(口径含重试、由请求前那一次 increment 决定)。
|
||||
--
|
||||
-- 3. 影响表与数据量:只加两列,不写数据、不加索引、不删列、不碰其他表。
|
||||
-- 存量行取 0(=该天没有 token 读数),行为与加列前逐字一致。
|
||||
-- 0 与 NULL 的区分留给了未来的"网关没回 usage":那种情况本迁移的列不写,
|
||||
-- 读侧把 0 读成"没有 token 证据",不许读成"这次消耗为 0"。
|
||||
--
|
||||
-- 4. 锁表风险:极低。表为日聚合、480 行/11 天的量级,即便走 COPY 也是毫秒级;
|
||||
-- MySQL 8.4 下 ADD COLUMN ... NOT NULL DEFAULT 0 走 INSTANT。
|
||||
--
|
||||
-- 5. 验证 SQL(迁移后跑,不许只看启动日志的 Successfully applied):
|
||||
-- SHOW COLUMNS FROM `biz_user_secret_usage_daily` LIKE '%tokens%';
|
||||
-- -> 期望 2 行:prompt_tokens / completion_tokens,Null = NO,Default = 0
|
||||
-- SELECT COUNT(*) AS total, SUM(prompt_tokens) AS p, SUM(completion_tokens) AS c
|
||||
-- FROM `biz_user_secret_usage_daily`;
|
||||
-- -> 期望 p = 0、c = 0(本迁移只加列,不回填)
|
||||
-- 应用侧:UserSecretUsageTokenTest 全绿(钉"计次仍先于响应"、"usage 缺失不写列"、
|
||||
-- "写库失败只告警不影响任务"三条口径)。
|
||||
--
|
||||
-- 6. 上线窗口:随 jar 双节点同窗口发布。加列对旧 jar 无影响(旧代码不认识新列);
|
||||
-- 新代码写这两列前 Flyway 已保证列存在 ⇒ 迁移必须与读它的 jar 同版,
|
||||
-- 只发 jar 不发迁移会在第一次写 token 时撞 1054 Unknown column。
|
||||
--
|
||||
-- 7. 可重跑性:照 V146/V155 惯例,加列前先查 information_schema——迁移中途失败时
|
||||
-- flyway repair + 重启会从第一条语句重跑整个文件,裸 ADD COLUMN 会撞 1060。
|
||||
--
|
||||
-- 8. 回滚(人工补丁,写成注释以免落在 Flyway 扫描路径里被当迁移执行):
|
||||
-- 触发场景是"这两列不该加"。顺序:停两台应用 → 跑下面两条 DROP →
|
||||
-- 删 flyway_schema_history 的 version='157' 行 → 回滚 jar → 起应用。
|
||||
-- ALTER TABLE `biz_user_secret_usage_daily` DROP COLUMN `prompt_tokens`;
|
||||
-- ALTER TABLE `biz_user_secret_usage_daily` DROP COLUMN `completion_tokens`;
|
||||
-- 仅回滚 jar(代码问题、与列无关)不需要 DROP:旧 jar 不认识这两列,留着零影响。
|
||||
-- =============================================================
|
||||
|
||||
SET @tbl := (
|
||||
SELECT COUNT(*) FROM information_schema.COLUMNS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'biz_user_secret_usage_daily'
|
||||
AND COLUMN_NAME = 'prompt_tokens'
|
||||
);
|
||||
SET @ddl := IF(@tbl = 0,
|
||||
'ALTER TABLE `biz_user_secret_usage_daily` ADD COLUMN `prompt_tokens` bigint NOT NULL DEFAULT 0 COMMENT ''输入 token 累计(网关 usage.prompt_tokens;0=该天没有 token 证据,不等于没消耗)''',
|
||||
'SELECT ''biz_user_secret_usage_daily.prompt_tokens already exists, skipped'' AS note');
|
||||
PREPARE stmt FROM @ddl;
|
||||
EXECUTE stmt;
|
||||
DEALLOCATE PREPARE stmt;
|
||||
|
||||
SET @tbl2 := (
|
||||
SELECT COUNT(*) FROM information_schema.COLUMNS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'biz_user_secret_usage_daily'
|
||||
AND COLUMN_NAME = 'completion_tokens'
|
||||
);
|
||||
SET @ddl2 := IF(@tbl2 = 0,
|
||||
'ALTER TABLE `biz_user_secret_usage_daily` ADD COLUMN `completion_tokens` bigint NOT NULL DEFAULT 0 COMMENT ''输出 token 累计(网关 usage.completion_tokens;0=该天没有 token 证据,不等于没产出)''',
|
||||
'SELECT ''biz_user_secret_usage_daily.completion_tokens already exists, skipped'' AS note');
|
||||
PREPARE stmt FROM @ddl2;
|
||||
EXECUTE stmt;
|
||||
DEALLOCATE PREPARE stmt;
|
||||
@@ -13,4 +13,13 @@
|
||||
VALUES (#{userId}, #{moduleKey}, #{businessDate}, #{count}, NOW(), NOW())
|
||||
ON DUPLICATE KEY UPDATE call_count = call_count + #{count}, updated_at = NOW()
|
||||
</insert>
|
||||
<!-- 不是「从注解搬过来」的语句:V157 加 token 两列时新写的,登记在 MigratedSqlEquivalenceTest.NOT_MIGRATED。
|
||||
与上面的 increment 分两条是口径要求:计次先于响应,token 只在响应回来后有读数。 -->
|
||||
<update id="addTokens">
|
||||
UPDATE biz_user_secret_usage_daily
|
||||
SET prompt_tokens = prompt_tokens + #{promptTokens},
|
||||
completion_tokens = completion_tokens + #{completionTokens},
|
||||
updated_at = NOW()
|
||||
WHERE user_id = #{userId} AND module_key = #{moduleKey} AND business_date = #{businessDate}
|
||||
</update>
|
||||
</mapper>
|
||||
|
||||
@@ -63,7 +63,10 @@ class MigratedSqlEquivalenceTest {
|
||||
* 登记在此而不是让它们红,是为了让「谁新写了语句」这件事留痕;新增一条不记名的,照样红。
|
||||
*/
|
||||
private static final java.util.Set<String> NOT_MIGRATED =
|
||||
java.util.Set.of("AdminUserMapper#countLiveWorkFor");
|
||||
java.util.Set.of("AdminUserMapper#countLiveWorkFor",
|
||||
// V157 加 token 两列时新写的累加语句;注解时代没有原文可对撞,等价性这支只能跳过它。
|
||||
// 与 increment 分两条是口径要求(计次先于响应、token 只在响应后有读数),不是随手加的第二条 SQL。
|
||||
"UserSecretUsageMapper#addTokens");
|
||||
|
||||
/** 账本 5 列:接口 / 方法 / @Select 种类 / 是否含常量插值 / 注解原文(Java 反转义后那条串)。 */
|
||||
private static final int LEDGER_COLUMNS = 5;
|
||||
|
||||
@@ -66,7 +66,11 @@ class SqlLayeringConventionTest {
|
||||
* 它不是「从注解搬进 XML」而是新写的,所以只在 XML 侧留痕——同一轮
|
||||
* {@code MigratedSqlEquivalenceTest} 里它被登记成「非迁移语句」,两处一起改才配得上。</p>
|
||||
*/
|
||||
private static final int XML_STATEMENT_TOTAL = 97;
|
||||
// 97 → 98:V157 token 计量新增 1 条写侧语句 {@code UserSecretUsageMapper#addTokens}。
|
||||
// 它同样是「新写」而非「从注解搬进 XML」,所以两处一起改:本格的总数 +
|
||||
// MigratedSqlEquivalenceTest.NOT_MIGRATED 的点名。与 increment 分两条是口径要求——
|
||||
// 计次发生在发请求之前(含重试),token 只有响应回来才有读数,合成一条 upsert 就会漏计超时的那次。
|
||||
private static final int XML_STATEMENT_TOTAL = 98;
|
||||
|
||||
/**
|
||||
* wrapper 里夹带函数级 SQL 的现状("这不是简单单表"的证据),文件名 → 条数:
|
||||
|
||||
+339
@@ -0,0 +1,339 @@
|
||||
package com.nanri.aiimage.modules.appearancepatent.client;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.nanri.aiimage.config.AppearancePatentProperties;
|
||||
import com.nanri.aiimage.config.BrandCheckProperties;
|
||||
import com.nanri.aiimage.modules.appearancepatent.model.dto.AppearancePatentResultRowDto;
|
||||
import com.nanri.aiimage.modules.brand.client.BrandCheckClient;
|
||||
import com.nanri.aiimage.modules.usersecret.service.UserSecretUsageService;
|
||||
import com.sun.net.httpserver.HttpExchange;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
import org.junit.jupiter.api.AfterEach;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.DisplayName;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.io.OutputStream;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collections;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.mockito.Mockito.mock;
|
||||
|
||||
/**
|
||||
* 外观专利「同一次请求内,同一张图只随请求发一份」的用例。
|
||||
*
|
||||
* <p>为什么要这条:2026-10-11 现网取证,主机B 24h 全量 831 条外观请求逐条解析,564 条带图、
|
||||
* 共投 2,693 张、去重只剩 1,465 张 —— 45.6% 是同一张图在同一批里的副本(兄弟行共用父号主图,
|
||||
* 2026-10-10 起改成逐行送检后更频繁)。图片是外观判定的输入大头,副本直接计费。
|
||||
*
|
||||
* <p>这批用例钉两件事,缺一不可:
|
||||
* ① 副本被去掉(image_url 段数 = 去重后的张数);
|
||||
* ② 判定一份没少(每个序号仍各出一份 status/reason)——省的是副本,不是判定。
|
||||
*
|
||||
* <p>本文件的中文一律写成 \\u 转义:断言要比对请求正文里的中文标记,而读文件工具会给裸中文
|
||||
* 行多塞一个字(项目内既有用例同此写法,不是洁癖)。
|
||||
*/
|
||||
class AppearancePatentLlmClientImageDedupeTest {
|
||||
|
||||
/** 「、」=图片对照里序号之间的分隔符。 */
|
||||
private static final String IDEOGRAPHIC_COMMA = "、";
|
||||
/** 「图」与「=序号」=对照表一行的前后缀。 */
|
||||
private static final String FIGURE = "\u56fe";
|
||||
private static final String LEGEND_EQ = "\uff1d\u5e8f\u53f7";
|
||||
|
||||
private HttpServer server;
|
||||
private final ObjectMapper objectMapper = new ObjectMapper();
|
||||
private final List<String> bodies = Collections.synchronizedList(new ArrayList<>());
|
||||
private final AtomicInteger textRequests = new AtomicInteger();
|
||||
private final AtomicInteger imageRequests = new AtomicInteger();
|
||||
/** 外观批量响应改写钩子:模拟模型只回了前 K 项(漏行形状)。 */
|
||||
private volatile int appearanceItemsToReturn = 0;
|
||||
private AppearancePatentLlmClient client;
|
||||
|
||||
@BeforeEach
|
||||
void setUp() throws IOException {
|
||||
server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
server.createContext("/v1/chat/completions", this::handleChat);
|
||||
server.createContext("/brand_check",
|
||||
exchange -> respond(exchange, 200, "{\"faild_data\":[],\"query_faild_data\":[]}"));
|
||||
server.start();
|
||||
AppearancePatentProperties properties = new AppearancePatentProperties();
|
||||
properties.setLlmHost("http://127.0.0.1:" + server.getAddress().getPort());
|
||||
properties.setLlmRetryTimes(1);
|
||||
client = new AppearancePatentLlmClient(properties, objectMapper, null, localBrandCheckClient(),
|
||||
mock(UserSecretUsageService.class));
|
||||
}
|
||||
|
||||
private BrandCheckClient localBrandCheckClient() {
|
||||
BrandCheckProperties brand = new BrandCheckProperties();
|
||||
brand.setBaseUrl("http://127.0.0.1:" + server.getAddress().getPort());
|
||||
brand.setRetryTimes(1);
|
||||
brand.setRetryIntervalMillis(0);
|
||||
brand.setTotalTimeoutMillis(5000);
|
||||
return new BrandCheckClient(brand, null);
|
||||
}
|
||||
|
||||
@AfterEach
|
||||
void tearDown() {
|
||||
server.stop(0);
|
||||
}
|
||||
|
||||
private void handleChat(HttpExchange exchange) throws IOException {
|
||||
String body = new String(exchange.getRequestBody().readAllBytes(), StandardCharsets.UTF_8);
|
||||
bodies.add(body);
|
||||
boolean withImage = body.contains("\"type\":\"image_url\"");
|
||||
if (withImage) {
|
||||
imageRequests.incrementAndGet();
|
||||
} else {
|
||||
textRequests.incrementAndGet();
|
||||
}
|
||||
List<Integer> indexes = new ArrayList<>();
|
||||
List<String> texts = new ArrayList<>();
|
||||
parseIndexedLines(textPartOf(body), indexes, texts);
|
||||
List<Map<String, Object>> items = new ArrayList<>();
|
||||
for (int i = 0; i < indexes.size(); i++) {
|
||||
if (withImage && appearanceItemsToReturn > 0 && i >= appearanceItemsToReturn) {
|
||||
continue;
|
||||
}
|
||||
if (withImage) {
|
||||
items.add(Map.of("i", indexes.get(i),
|
||||
"appearance_status", "\u65e0\u4fb5\u6743",
|
||||
"appearance_reason", "\u3010\u89c6\u89c9\u62c6\u89e3\u3011\uff1a\u901a\u7528\u5f62\u72b6"));
|
||||
} else {
|
||||
items.add(Map.of("i", indexes.get(i), "keywords", "\u65e0"));
|
||||
}
|
||||
}
|
||||
try {
|
||||
String content = objectMapper.writeValueAsString(Map.of("items", items));
|
||||
respond(exchange, 200, objectMapper.writeValueAsString(Map.of(
|
||||
"choices", List.of(Map.of("message", Map.of("content", content))))));
|
||||
} catch (Exception ex) {
|
||||
respond(exchange, 500, "{\"error\":{\"message\":\"" + ex + "\"}}");
|
||||
}
|
||||
}
|
||||
|
||||
private void respond(HttpExchange exchange, int status, String body) throws IOException {
|
||||
byte[] bytes = body.getBytes(StandardCharsets.UTF_8);
|
||||
exchange.getResponseHeaders().set("Content-Type", "application/json; charset=utf-8");
|
||||
exchange.sendResponseHeaders(status, bytes.length);
|
||||
try (OutputStream os = exchange.getResponseBody()) {
|
||||
os.write(bytes);
|
||||
}
|
||||
}
|
||||
|
||||
/** 请求正文里那一段文本(多模态请求的其余段是 image_url)。 */
|
||||
@SuppressWarnings("unchecked")
|
||||
private String textPartOf(String body) {
|
||||
try {
|
||||
Map<String, Object> request = objectMapper.readValue(body, Map.class);
|
||||
List<Object> messages = (List<Object>) request.get("messages");
|
||||
Map<String, Object> last = (Map<String, Object>) messages.get(messages.size() - 1);
|
||||
Object content = last.get("content");
|
||||
if (content instanceof List<?> parts) {
|
||||
for (Object part : parts) {
|
||||
if (part instanceof Map<?, ?> map && "text".equals(map.get("type"))) {
|
||||
return String.valueOf(map.get("text"));
|
||||
}
|
||||
}
|
||||
return "";
|
||||
}
|
||||
return content == null ? "" : String.valueOf(content);
|
||||
} catch (Exception ex) {
|
||||
throw new IllegalStateException(ex);
|
||||
}
|
||||
}
|
||||
|
||||
private void parseIndexedLines(String raw, List<Integer> indexes, List<String> texts) {
|
||||
for (String line : raw.split("\n")) {
|
||||
int tab = line.indexOf('\t');
|
||||
if (tab <= 0) {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
indexes.add(Integer.parseInt(line.substring(0, tab).trim()));
|
||||
} catch (NumberFormatException ignored) {
|
||||
continue;
|
||||
}
|
||||
texts.add(line.substring(tab + 1));
|
||||
}
|
||||
}
|
||||
|
||||
private static AppearancePatentResultRowDto row(String asin, String title, String url) {
|
||||
AppearancePatentResultRowDto row = new AppearancePatentResultRowDto();
|
||||
row.setId(asin);
|
||||
row.setAsin(asin);
|
||||
row.setTitle(title);
|
||||
row.setUrl(url);
|
||||
return row;
|
||||
}
|
||||
|
||||
private static String urlOf(char letter) {
|
||||
return "https://m.media-amazon.com/images/I/" + letter + "L._AC_SX679_.jpg";
|
||||
}
|
||||
|
||||
/** 所有请求正文里 image_url 段的总数。 */
|
||||
private int sentImageParts() {
|
||||
return bodies.stream().mapToInt(AppearancePatentLlmClientImageDedupeTest::imagePartsIn).sum();
|
||||
}
|
||||
|
||||
private static int imagePartsIn(String body) {
|
||||
return splitOccurrences(body).size();
|
||||
}
|
||||
|
||||
private static List<Integer> splitOccurrences(String body) {
|
||||
List<Integer> marks = new ArrayList<>();
|
||||
for (int i = body.indexOf("\"type\":\"image_url\""); i >= 0; i = body.indexOf("\"type\":\"image_url\"", i + 1)) {
|
||||
marks.add(i);
|
||||
}
|
||||
return marks;
|
||||
}
|
||||
|
||||
private int imagePartsOfLastImageRequest() {
|
||||
for (int i = bodies.size() - 1; i >= 0; i--) {
|
||||
String body = bodies.get(i);
|
||||
if (body.contains("\"type\":\"image_url\"")) {
|
||||
return splitOccurrences(body).size();
|
||||
}
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
private String lastImageRequestText() {
|
||||
for (int i = bodies.size() - 1; i >= 0; i--) {
|
||||
String body = bodies.get(i);
|
||||
if (body.contains("\"type\":\"image_url\"")) {
|
||||
return textPartOf(body);
|
||||
}
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("五行共用一张图:只发一份图片,但五行各出一份判定")
|
||||
void fiveRowsSharingOneImageSendOneCopyButFiveJudgements() {
|
||||
List<AppearancePatentResultRowDto> rows = new ArrayList<>();
|
||||
for (int i = 1; i <= 5; i++) {
|
||||
rows.add(row("B0" + i, "防滑罩 尺寸" + i, urlOf('X')));
|
||||
}
|
||||
|
||||
List<AppearancePatentResultRowDto> result = client.inspectRows(rows, "", "k");
|
||||
|
||||
assertThat(imagePartsOfLastImageRequest()).isEqualTo(1);
|
||||
String text = lastImageRequestText();
|
||||
assertThat(text).contains(FIGURE + "1" + LEGEND_EQ + "1" + IDEOGRAPHIC_COMMA + "2"
|
||||
+ IDEOGRAPHIC_COMMA + "3" + IDEOGRAPHIC_COMMA + "4" + IDEOGRAPHIC_COMMA + "5");
|
||||
assertThat(result).hasSize(5);
|
||||
for (AppearancePatentResultRowDto row : result) {
|
||||
assertThat(row.getAppearanceRisk()).contains("\u65e0\u4fb5\u6743");
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("五行五张不同图:仍发五份,对照表逐行一条(去重不许误伤不同的图)")
|
||||
void fiveDistinctImagesStillSendFiveCopies() {
|
||||
List<AppearancePatentResultRowDto> rows = new ArrayList<>();
|
||||
for (int i = 0; i < 5; i++) {
|
||||
rows.add(row("B0" + i, "杯子 " + i, urlOf((char) ('A' + i))));
|
||||
}
|
||||
|
||||
client.inspectRows(rows, "", "k");
|
||||
|
||||
assertThat(imagePartsOfLastImageRequest()).isEqualTo(5);
|
||||
String text = lastImageRequestText();
|
||||
for (int i = 1; i <= 5; i++) {
|
||||
assertThat(text).contains(FIGURE + i + LEGEND_EQ + i);
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("混合形状(A,A,B,B,C):发三份,图1=1、2,图2=3、4,图3=5")
|
||||
void mixedDuplicatesSendThreeCopiesWithCorrectRowMapping() {
|
||||
List<AppearancePatentResultRowDto> rows = List.of(
|
||||
row("B01", "a", urlOf('A')), row("B02", "b", urlOf('A')),
|
||||
row("B03", "c", urlOf('B')), row("B04", "d", urlOf('B')),
|
||||
row("B05", "e", urlOf('C')));
|
||||
|
||||
client.inspectRows(rows, "", "k");
|
||||
|
||||
assertThat(imagePartsOfLastImageRequest()).isEqualTo(3);
|
||||
String text = lastImageRequestText();
|
||||
assertThat(text).contains(FIGURE + "1" + LEGEND_EQ + "1" + IDEOGRAPHIC_COMMA + "2");
|
||||
assertThat(text).contains(FIGURE + "2" + LEGEND_EQ + "3" + IDEOGRAPHIC_COMMA + "4");
|
||||
assertThat(text).contains(FIGURE + "3" + LEGEND_EQ + "5");
|
||||
assertThat(text).doesNotContain(FIGURE + "4");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("去重后图片份数不等于行数时,判定条数仍按行数收(省副本不是省判定)")
|
||||
void judgementCountFollowsRowsNotImages() {
|
||||
List<AppearancePatentResultRowDto> rows = List.of(
|
||||
row("B01", "a", urlOf('A')), row("B02", "b", urlOf('A')),
|
||||
row("B03", "c", urlOf('A')), row("B04", "d", urlOf('A')),
|
||||
row("B05", "e", urlOf('A')));
|
||||
|
||||
List<AppearancePatentResultRowDto> result = client.inspectRows(rows, "", "k");
|
||||
|
||||
assertThat(imagePartsOfLastImageRequest()).isEqualTo(1);
|
||||
assertThat(result).hasSize(5);
|
||||
assertThat(result).allSatisfy(item -> assertThat(item.getError()).isNullOrEmpty());
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("模型漏回序号:缺失的行仍走单行补检,不静默出空结论")
|
||||
void missingIndexedItemStillFallsBackToSingleRow() {
|
||||
appearanceItemsToReturn = 2;
|
||||
List<AppearancePatentResultRowDto> rows = new ArrayList<>();
|
||||
for (int i = 1; i <= 5; i++) {
|
||||
rows.add(row("B0" + i, "防滑罩 " + i, urlOf('X')));
|
||||
}
|
||||
|
||||
client.inspectRows(rows, "", "k");
|
||||
|
||||
// 1 次批量外观 + 3 行补检:补检必须发生,否则漏掉的行就空着出文件了
|
||||
assertThat(imageRequests.get()).isGreaterThanOrEqualTo(4);
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("同一张图只是两端多了空格:也算同一张,且发出去的还是行里那条原始串")
|
||||
void dedupeKeysOnNormalizedUrlButSendsRawUrl() {
|
||||
String bare = urlOf('Z');
|
||||
List<AppearancePatentResultRowDto> rows = List.of(
|
||||
row("B01", "a", " " + bare), row("B02", "b", bare), row("B03", "c", bare + " "));
|
||||
|
||||
client.inspectRows(rows, "", "k");
|
||||
|
||||
assertThat(imagePartsOfLastImageRequest()).isEqualTo(1);
|
||||
String body = lastImageRequestBody();
|
||||
// 归一化只当键:发送内容仍是首次出现的那条原始串(带前导空格),不改写行的数据。
|
||||
assertThat(body).contains("\"url\":\" " + bare + "\"");
|
||||
assertThat(body).doesNotContain("\"url\":\"" + bare + "\"");
|
||||
}
|
||||
|
||||
private String lastImageRequestBody() {
|
||||
for (int i = bodies.size() - 1; i >= 0; i--) {
|
||||
if (bodies.get(i).contains("\"type\":\"image_url\"")) {
|
||||
return bodies.get(i);
|
||||
}
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("标题提取请求不带图:去重只作用在外观那一路")
|
||||
void titlePassStaysTextOnly() {
|
||||
List<AppearancePatentResultRowDto> rows = List.of(
|
||||
row("B01", "a", urlOf('A')), row("B02", "b", urlOf('A')));
|
||||
|
||||
client.inspectRows(rows, "", "k");
|
||||
|
||||
assertThat(textRequests.get()).isEqualTo(1);
|
||||
assertThat(sentImageParts()).isEqualTo(1);
|
||||
}
|
||||
}
|
||||
+181
@@ -0,0 +1,181 @@
|
||||
package com.nanri.aiimage.modules.appearancepatent.client;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.nanri.aiimage.config.AppearancePatentProperties;
|
||||
import com.nanri.aiimage.config.BrandCheckProperties;
|
||||
import com.nanri.aiimage.modules.appearancepatent.model.dto.AppearancePatentResultRowDto;
|
||||
import com.nanri.aiimage.modules.brand.client.BrandCheckClient;
|
||||
import com.nanri.aiimage.modules.usersecret.service.UserSecretUsageService;
|
||||
import com.nanri.aiimage.modules.usersecret.support.SecretUsageContext;
|
||||
import com.sun.net.httpserver.HttpExchange;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
import org.junit.jupiter.api.AfterEach;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.DisplayName;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.io.OutputStream;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.List;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.mockito.ArgumentMatchers.any;
|
||||
import static org.mockito.ArgumentMatchers.anyInt;
|
||||
import static org.mockito.ArgumentMatchers.anyLong;
|
||||
import static org.mockito.ArgumentMatchers.anyString;
|
||||
import static org.mockito.ArgumentMatchers.eq;
|
||||
import static org.mockito.Mockito.mock;
|
||||
import static org.mockito.Mockito.never;
|
||||
import static org.mockito.Mockito.times;
|
||||
import static org.mockito.Mockito.verify;
|
||||
|
||||
/**
|
||||
* 外观专利把网关 {@code usage} 落库的用例(V157 的两列 token 读数)。
|
||||
*
|
||||
* <p>为什么要这条:降本决策一直只能用「次数」和字符数推算,因为两个 LLM client 都只记 1 次、
|
||||
* 从不读响应里的 usage。这批用例钉三条口径,一条都不能漂:
|
||||
* ① 计次仍然发生在**发请求之前**(含重试,请求没回来也算一次);
|
||||
* ② token 读数只在响应回来之后补写;
|
||||
* ③ 网关没回 usage 就一条都不写 —— 不许写 0 冒充「实测消耗为 0」,那种行以后会被拿去算成本。
|
||||
*/
|
||||
class AppearancePatentLlmClientTokenUsageTest {
|
||||
|
||||
private HttpServer server;
|
||||
private final ObjectMapper objectMapper = new ObjectMapper();
|
||||
private final UserSecretUsageService usageService = mock(UserSecretUsageService.class);
|
||||
/** 响应的 usage 形状:full=两个都有、none=整个 usage 缺失、prompt_only=只有输入侧。 */
|
||||
private volatile String usageShape = "full";
|
||||
private AppearancePatentLlmClient client;
|
||||
|
||||
@BeforeEach
|
||||
void setUp() throws IOException {
|
||||
server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
server.createContext("/v1/chat/completions", this::handleChat);
|
||||
server.createContext("/brand_check",
|
||||
exchange -> respond(exchange, "{\"faild_data\":[],\"query_faild_data\":[]}"));
|
||||
server.start();
|
||||
AppearancePatentProperties properties = new AppearancePatentProperties();
|
||||
properties.setLlmHost("http://127.0.0.1:" + server.getAddress().getPort());
|
||||
properties.setLlmRetryTimes(1);
|
||||
client = new AppearancePatentLlmClient(properties, objectMapper, null, localBrandCheckClient(), usageService);
|
||||
}
|
||||
|
||||
private BrandCheckClient localBrandCheckClient() {
|
||||
BrandCheckProperties brand = new BrandCheckProperties();
|
||||
brand.setBaseUrl("http://127.0.0.1:" + server.getAddress().getPort());
|
||||
brand.setRetryTimes(1);
|
||||
brand.setRetryIntervalMillis(0);
|
||||
brand.setTotalTimeoutMillis(5000);
|
||||
return new BrandCheckClient(brand, null);
|
||||
}
|
||||
|
||||
@AfterEach
|
||||
void tearDown() {
|
||||
server.stop(0);
|
||||
}
|
||||
|
||||
private void handleChat(HttpExchange exchange) throws IOException {
|
||||
String body = new String(exchange.getRequestBody().readAllBytes(), StandardCharsets.UTF_8);
|
||||
boolean withImage = body.contains("\"type\":\"image_url\"");
|
||||
String keywords = "\u65e0";
|
||||
String item = withImage
|
||||
? "{\"i\":1,\"appearance_status\":\"\u65e0\u4fb5\u6743\","
|
||||
+ "\"appearance_reason\":\"\u3010\u89c6\u89c9\u62c6\u89e3\u3011\u901a\u7528\"}"
|
||||
: "{\"i\":1,\"keywords\":\"" + keywords + "\"}";
|
||||
String content = "{\"items\":[" + item + "]}";
|
||||
String escaped = content.replace("\"", "\\\"");
|
||||
String usage = switch (usageShape) {
|
||||
case "none" -> "";
|
||||
case "prompt_only" -> ",\"usage\":{\"prompt_tokens\":1234}";
|
||||
default -> ",\"usage\":{\"prompt_tokens\":1234,\"completion_tokens\":56}";
|
||||
};
|
||||
String response = "{\"choices\":[{\"message\":{\"content\":\"" + escaped + "\"}}]" + usage + "}";
|
||||
// 测试桩自证:拼出来的必须是合法 JSON 且 usage 在根对象**里面**。
|
||||
// 上一版把 usage 接在根对象的收尾括号之后,Jackson 默认容忍尾随内容 ⇒
|
||||
// "网关没回 usage" 那条用例其实是恒绿的,这里先把这个形状钉死再发出去。
|
||||
try {
|
||||
var parsed = objectMapper.readTree(response);
|
||||
if (!"none".equals(usageShape) && parsed.path("usage").isMissingNode()) {
|
||||
throw new IllegalStateException("harness built a response without usable usage node");
|
||||
}
|
||||
if ("none".equals(usageShape) && !parsed.path("usage").isMissingNode()) {
|
||||
throw new IllegalStateException("harness leaked a usage node into the none shape");
|
||||
}
|
||||
} catch (Exception ex) {
|
||||
respond(exchange, "{\"error\":{\"message\":\"harness produced invalid json: " + ex + "\"}}");
|
||||
return;
|
||||
}
|
||||
respond(exchange, response);
|
||||
}
|
||||
|
||||
private void respond(HttpExchange exchange, String body) throws IOException {
|
||||
byte[] bytes = body.getBytes(StandardCharsets.UTF_8);
|
||||
exchange.getResponseHeaders().set("Content-Type", "application/json; charset=utf-8");
|
||||
exchange.sendResponseHeaders(200, bytes.length);
|
||||
try (OutputStream os = exchange.getResponseBody()) {
|
||||
os.write(bytes);
|
||||
}
|
||||
}
|
||||
|
||||
private List<AppearancePatentResultRowDto> inspectOneRow() {
|
||||
AppearancePatentResultRowDto row = new AppearancePatentResultRowDto();
|
||||
row.setId("1");
|
||||
row.setAsin("B001");
|
||||
row.setTitle("\u9632\u6ed1\u7f69");
|
||||
row.setUrl("https://m.media-amazon.com/images/I/71L._AC_SX679_.jpg");
|
||||
// 计次与 token 都要归属到任务用户:没有这层上下文时两条读数都会静默跳过(无归属不记账)。
|
||||
return SecretUsageContext.call(7L, "appearance-patent",
|
||||
() -> client.inspectRows(List.of(row), "", "k"));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("网关回了 usage:两次请求各补一条 token 读数,计次一条不少")
|
||||
void usageIsRecordedAfterResponseWithoutLosingTheCallCount() {
|
||||
List<AppearancePatentResultRowDto> result = inspectOneRow();
|
||||
|
||||
assertThat(result).hasSize(1);
|
||||
// 标题提取 + 外观判定 = 2 次对外请求
|
||||
verify(usageService, times(2)).record(eq(7L), eq("appearance-patent"), eq(1));
|
||||
verify(usageService, times(2))
|
||||
.recordTokens(eq(7L), eq("appearance-patent"), eq(1234L), eq(56L));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("网关没回 usage:一条 token 都不写,计次照常")
|
||||
void missingUsageWritesNoTokensInsteadOfZero() {
|
||||
usageShape = "none";
|
||||
|
||||
inspectOneRow();
|
||||
|
||||
verify(usageService, times(2)).record(eq(7L), eq("appearance-patent"), eq(1));
|
||||
verify(usageService, never()).recordTokens(anyLong(), anyString(), anyLong(), anyLong());
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("usage 只回了一半:有读数的那侧照写,缺的那侧写 0,整条不能被丢掉")
|
||||
void partialUsageStillStored() {
|
||||
usageShape = "prompt_only";
|
||||
|
||||
inspectOneRow();
|
||||
|
||||
verify(usageService, times(2))
|
||||
.recordTokens(eq(7L), eq("appearance-patent"), eq(1234L), eq(0L));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("无任务归属(context 为空):计次与 token 都不写,绝不记到别人账上")
|
||||
void noContextRecordsNothing() {
|
||||
AppearancePatentResultRowDto row = new AppearancePatentResultRowDto();
|
||||
row.setId("1");
|
||||
row.setAsin("B001");
|
||||
row.setTitle("\u9632\u6ed1\u7f69");
|
||||
row.setUrl("https://m.media-amazon.com/images/I/71L._AC_SX679_.jpg");
|
||||
|
||||
client.inspectRows(List.of(row), "", "k");
|
||||
|
||||
verify(usageService, never()).record(any(), anyString(), anyInt());
|
||||
verify(usageService, never()).recordTokens(any(), anyString(), anyLong(), anyLong());
|
||||
}
|
||||
}
|
||||
+211
-9
@@ -10,6 +10,7 @@ import com.nanri.aiimage.modules.similarasin.client.SimilarAsinLlmClient;
|
||||
import com.nanri.aiimage.modules.similarasin.model.dto.SimilarAsinResultRowDto;
|
||||
import com.nanri.aiimage.modules.similarasin.util.PuzzleImageMerger;
|
||||
import com.nanri.aiimage.modules.similarasin.util.SimilarAsinImageEmbedder;
|
||||
import org.junit.jupiter.api.DisplayName;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
@@ -352,6 +353,16 @@ class SimilarAsinLlmServiceTest {
|
||||
/** 同上,另支持批量分类文本(含"待分类商品"标记)——batchCategoryJson 为 null 时批量调用视为意外。 */
|
||||
private SimilarAsinLlmClient recordingClient(List<String> chatUserTexts, String batchCategoryJson,
|
||||
String categoryJson, String conformJson) {
|
||||
return recordingClientWithConformBatch(chatUserTexts, batchCategoryJson, null, categoryJson, conformJson);
|
||||
}
|
||||
|
||||
/** 批量调用的响应桩:batchConformJson 为 null 时把 conformJson 按行数复制成 items(让合批走得通)。 */
|
||||
private static final String CONFORM_BATCH_THROWS = "<<throw>>";
|
||||
|
||||
private SimilarAsinLlmClient recordingClientWithConformBatch(List<String> chatUserTexts,
|
||||
String batchCategoryJson,
|
||||
String batchConformJson,
|
||||
String categoryJson, String conformJson) {
|
||||
SimilarAsinLlmClient client = mock(SimilarAsinLlmClient.class);
|
||||
when(client.resolveApiKey(anyString())).thenReturn("k");
|
||||
when(client.hasApiKey(anyString())).thenReturn(true);
|
||||
@@ -365,6 +376,13 @@ class SimilarAsinLlmServiceTest {
|
||||
}
|
||||
return batchCategoryJson;
|
||||
}
|
||||
if (userText.contains("待判定商品")) {
|
||||
if (CONFORM_BATCH_THROWS.equals(batchConformJson)) {
|
||||
throw new IllegalStateException("LLM gateway 502");
|
||||
}
|
||||
return batchConformJson != null
|
||||
? batchConformJson : conformBatchJsonFor(userText, conformJson);
|
||||
}
|
||||
return userText.startsWith("产品标题:") ? categoryJson : conformJson;
|
||||
});
|
||||
when(client.invokeChatWithImages(anyString(), anyString(), anyString(), any(), anyString()))
|
||||
@@ -374,6 +392,34 @@ class SimilarAsinLlmServiceTest {
|
||||
return client;
|
||||
}
|
||||
|
||||
/** 批量正文里有几件 [n] 商品,就把这份单件 JSON 复制成几个 item:只比调用形状时用,不改变判定内容。 */
|
||||
private static String conformBatchJsonFor(String batchUserText, String conformJson) {
|
||||
int rows = 0;
|
||||
for (int i = 1; i < 500; i++) {
|
||||
if (!batchUserText.contains("[" + i + "] 标题:")) {
|
||||
break;
|
||||
}
|
||||
rows = i;
|
||||
}
|
||||
StringBuilder items = new StringBuilder();
|
||||
String fields = conformJson.substring(1, conformJson.length() - 1);
|
||||
for (int i = 1; i <= rows; i++) {
|
||||
if (i > 1) {
|
||||
items.append(',');
|
||||
}
|
||||
items.append("{\"i\":").append(i).append(',').append(fields).append('}');
|
||||
}
|
||||
return "{\"items\":[" + items + "]}";
|
||||
}
|
||||
|
||||
/** 取批量合规正文里第 n 件商品那一段(到下一个 [m] 为止),用于逐件断言类目上下文。 */
|
||||
private static String conformRowBlock(String batchUserText, int n) {
|
||||
int start = batchUserText.indexOf("[" + n + "] 标题:");
|
||||
assertTrue(start >= 0, "批量合规正文缺第 " + n + " 件商品:" + batchUserText);
|
||||
int next = batchUserText.indexOf("\n[", start + 1);
|
||||
return next < 0 ? batchUserText.substring(start) : batchUserText.substring(start, next);
|
||||
}
|
||||
|
||||
private static SimilarAsinResultRowDto rowOf(String asin, String title) {
|
||||
SimilarAsinResultRowDto row = new SimilarAsinResultRowDto();
|
||||
row.setAsin(asin);
|
||||
@@ -516,19 +562,18 @@ class SimilarAsinLlmServiceTest {
|
||||
List.of(rowOf("B01", "布质防滑罩"), rowOf("B02", "亚麻沙发套"), rowOf("B03", "手机支架")),
|
||||
"", "k", false, true);
|
||||
|
||||
// 1 次批量分类 + 3 次合规;树文本只在批量请求里出现一次(逐行模式会出现 3 次)
|
||||
assertEquals(4, chats.size(), "实际调用:" + chats);
|
||||
// 1 次批量分类 + 1 次批量合规(2026-10-11 合规也合批:省的是往返,不是判定)
|
||||
assertEquals(2, chats.size(), "实际调用:" + chats);
|
||||
assertEquals(1, countTreeTexts(chats));
|
||||
String batchText = chats.get(0);
|
||||
assertTrue(batchText.contains("布质防滑罩") && batchText.contains("亚麻沙发套") && batchText.contains("手机支架"),
|
||||
"批量请求应包含整块全部行:" + batchText);
|
||||
// 批量判定的行级语义与单行一致:i=2 second=无 → 合规入参二级为"无"且无三级候选
|
||||
// 批量合规必须逐件带自己的违规类目上下文:i=2 的二级是"无",那一行不许出现别的分支的三级候选
|
||||
String rowTwo = conformRowBlock(chats.get(1), 2);
|
||||
assertTrue(rowTwo.contains("一级类目:家具") && rowTwo.contains("二级类目:无"),
|
||||
"i=2 的判定应保留一级、二级为无,实际块:" + rowTwo);
|
||||
assertFalse(rowTwo.contains("手机壳"), "二级为无时该行不该有三级候选:" + rowTwo);
|
||||
assertTrue(result.get(1).getIsConform() != null, "每行都要完成合规判定");
|
||||
String conformTexts = chats.stream().filter(text -> text.contains("三级类目:"))
|
||||
.filter(text -> text.contains("一级类目:家具") && text.contains("二级类目:无"))
|
||||
.findFirst().orElse("");
|
||||
assertTrue(!conformTexts.isEmpty(), "i=2 的判定应保留一级、二级为无:" + chats);
|
||||
assertFalse(conformTexts.contains("手机壳"), "二级为无时不应给出三级候选");
|
||||
}
|
||||
|
||||
@Test
|
||||
@@ -551,7 +596,8 @@ class SimilarAsinLlmServiceTest {
|
||||
long singleCategoryCalls = chats.stream().filter(text -> text.startsWith("产品标题:")).count();
|
||||
assertEquals(1, batchCalls);
|
||||
assertEquals(1, singleCategoryCalls, "缺失行要回退单行分类调用:" + chats);
|
||||
assertEquals(3, chats.size() - batchCalls - singleCategoryCalls, "3 行都要有合规调用:" + chats);
|
||||
assertEquals(1, chats.size() - batchCalls - singleCategoryCalls,
|
||||
"3 行合规合并成一次批量请求(2026-10-11 合批),不该再逐行发:" + chats);
|
||||
}
|
||||
|
||||
@Test
|
||||
@@ -605,4 +651,160 @@ class SimilarAsinLlmServiceTest {
|
||||
assertEquals(3, chats.stream().filter(text -> text.contains("ID:100")).count(), "3 次分类各带一棵树");
|
||||
assertEquals(6, chats.size(), "3 次分类 + 3 次合规");
|
||||
}
|
||||
|
||||
// --- 批量合规(2026-10-11):货源最后一步逐行 LLM 调用合并成一批,省往返不省判定 ---
|
||||
|
||||
private static final String CATEGORY_BATCH_JSON =
|
||||
"{\"items\":[{\"i\":1,\"first\":\"家具\",\"second\":\"配件\"},"
|
||||
+ "{\"i\":2,\"first\":\"数码\",\"second\":\"配件\"},"
|
||||
+ "{\"i\":3,\"first\":\"家具\",\"second\":\"无\"}]}";
|
||||
private static final String SINGLE_CATEGORY_JSON = "{\"first\":\"家具\",\"second\":\"配件\"}";
|
||||
|
||||
private List<SimilarAsinResultRowDto> threeRows() {
|
||||
return List.of(rowOf("B01", "布质防滑罩"), rowOf("B02", "手机支架"), rowOf("B03", "亚麻沙发套"));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("批量合规:每件商品带自己的违规类目上下文,禁止共用一份把 A 行的拦截判到 B 行")
|
||||
void conformBatchCarriesEachRowOwnViolationContext() {
|
||||
List<String> chats = new CopyOnWriteArrayList<>();
|
||||
SimilarAsinLlmService service = service(
|
||||
recordingClient(chats, CATEGORY_BATCH_JSON, SINGLE_CATEGORY_JSON, CONFORM_JSON),
|
||||
duplicateTreeCategoryService());
|
||||
|
||||
service.inspectRows(threeRows(), "", "k", false, true);
|
||||
|
||||
String batch = onlyConformBatch(chats);
|
||||
assertTrue(conformRowBlock(batch, 1).contains("一级类目:家具"), "第 1 件的上下文:" + conformRowBlock(batch, 1));
|
||||
assertTrue(conformRowBlock(batch, 1).contains("沙发套"), "第 1 件二级=配件 ⇒ 三级候选取家具分支");
|
||||
assertTrue(conformRowBlock(batch, 2).contains("一级类目:数码"), "第 2 件的上下文:" + conformRowBlock(batch, 2));
|
||||
assertTrue(conformRowBlock(batch, 2).contains("手机壳"), "第 2 件二级=配件 ⇒ 三级候选取数码分支");
|
||||
assertTrue(conformRowBlock(batch, 3).contains("二级类目:无"), "第 3 件的上下文:" + conformRowBlock(batch, 3));
|
||||
assertFalse(conformRowBlock(batch, 3).contains("手机壳"), "二级为无的那件不许拿到别的分支的三级候选");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("批量合规:整块只发一次请求,且不顺手多算一次类目")
|
||||
void conformBatchIsOneCallAndNeverRecalculatesCategory() {
|
||||
List<String> chats = new CopyOnWriteArrayList<>();
|
||||
SimilarAsinLlmService service = service(
|
||||
recordingClient(chats, CATEGORY_BATCH_JSON, SINGLE_CATEGORY_JSON, CONFORM_JSON),
|
||||
duplicateTreeCategoryService());
|
||||
|
||||
List<SimilarAsinResultRowDto> result = service.inspectRows(threeRows(), "", "k", false, true);
|
||||
|
||||
assertEquals(1, chats.stream().filter(text -> text.contains("待分类商品")).count());
|
||||
assertEquals(0, chats.stream().filter(text -> text.startsWith("产品标题:")).count(),
|
||||
"类目已由批量预计算给出,合规合批不得回头逐行重算(重算=多扣用户一次):" + chats);
|
||||
assertEquals(1, chats.stream().filter(text -> text.contains("待判定商品")).count(), "3 行合规合并成一次");
|
||||
assertEquals(2, chats.size(), "批量类目 + 批量合规,共 2 次对外请求:" + chats);
|
||||
assertEquals(3, result.size());
|
||||
for (SimilarAsinResultRowDto row : result) {
|
||||
assertEquals("不符合", row.getIsConform(), "每行都要有自己的合规结果");
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("批量合规:模型漏回编号的行回退逐行单调用,不静默出空结论")
|
||||
void conformBatchMissingItemFallsBackToSingleRowCall() {
|
||||
List<String> chats = new CopyOnWriteArrayList<>();
|
||||
SimilarAsinLlmService service = service(
|
||||
recordingClientWithConformBatch(chats, CATEGORY_BATCH_JSON,
|
||||
"{\"items\":[{\"i\":1,\"is_conform\":\"符合\",\"reason\":\"无\",\"category\":\"家具\"}]}",
|
||||
SINGLE_CATEGORY_JSON, CONFORM_JSON),
|
||||
duplicateTreeCategoryService());
|
||||
|
||||
List<SimilarAsinResultRowDto> result = service.inspectRows(threeRows(), "", "k", false, true);
|
||||
|
||||
assertEquals(1, chats.stream().filter(text -> text.contains("待判定商品")).count());
|
||||
assertEquals(2, chats.stream().filter(text -> text.startsWith("标题:")).count(),
|
||||
"缺 i=2、i=3 ⇒ 这两行各回退一次单行合规:" + chats);
|
||||
assertEquals("符合", result.get(0).getIsConform(), "批量给的结果");
|
||||
assertEquals("不符合", result.get(1).getIsConform(), "回退单行给的结果");
|
||||
assertEquals("不符合", result.get(2).getIsConform(), "回退单行给的结果");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("批量合规:整块传输失败 ⇒ 全块回退逐行,行级结果照常产出")
|
||||
void conformBatchTransportErrorFallsBackForWholeChunk() {
|
||||
List<String> chats = new CopyOnWriteArrayList<>();
|
||||
SimilarAsinLlmService service = service(
|
||||
recordingClientWithConformBatch(chats, CATEGORY_BATCH_JSON, CONFORM_BATCH_THROWS,
|
||||
SINGLE_CATEGORY_JSON, CONFORM_JSON),
|
||||
duplicateTreeCategoryService());
|
||||
|
||||
List<SimilarAsinResultRowDto> result = service.inspectRows(threeRows(), "", "k", false, true);
|
||||
|
||||
assertEquals(1, chats.stream().filter(text -> text.contains("待判定商品")).count(), "批量尝试过一次并失败");
|
||||
assertEquals(3, chats.stream().filter(text -> text.startsWith("标题:")).count(), "整块回退逐行");
|
||||
for (SimilarAsinResultRowDto row : result) {
|
||||
assertEquals("不符合", row.getIsConform(), "回退后不得有空结论");
|
||||
assertEquals("成功", row.getStatus());
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("批量合规:阈值设 1 即退回逐行(应急开关,且不发批量请求)")
|
||||
void conformBatchDisabledWhenBatchSizeOne() {
|
||||
List<String> chats = new CopyOnWriteArrayList<>();
|
||||
SimilarAsinProperties properties = properties();
|
||||
properties.setLlmConformBatchSize(1);
|
||||
SimilarAsinLlmService service = service(
|
||||
recordingClient(chats, CATEGORY_BATCH_JSON, SINGLE_CATEGORY_JSON, CONFORM_JSON),
|
||||
duplicateTreeCategoryService(), properties);
|
||||
|
||||
service.inspectRows(threeRows(), "", "k", false, true);
|
||||
|
||||
assertEquals(0, chats.stream().filter(text -> text.contains("待判定商品")).count(),
|
||||
"批量关闭时不得发起批量合规调用");
|
||||
assertEquals(3, chats.stream().filter(text -> text.startsWith("标题:")).count(), "逐行 3 次合规");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("批量合规:分类批量被关掉时合规合批必须不生效(否则同一行类目被算两遍=多扣一次)")
|
||||
void conformBatchStaysOffWhenCategoryBatchIsOff() {
|
||||
List<String> chats = new CopyOnWriteArrayList<>();
|
||||
SimilarAsinProperties properties = properties();
|
||||
properties.setLlmCategoryBatchSize(1);
|
||||
SimilarAsinLlmService service = service(
|
||||
recordingClient(chats, null, SINGLE_CATEGORY_JSON, CONFORM_JSON),
|
||||
duplicateTreeCategoryService(), properties);
|
||||
|
||||
service.inspectRows(threeRows(), "", "k", false, true);
|
||||
|
||||
assertEquals(0, chats.stream().filter(text -> text.contains("待判定商品")).count(),
|
||||
"没有逐行类目路径可用时不得发起批量合规:" + chats);
|
||||
assertEquals(3, chats.stream().filter(text -> text.startsWith("标题:")).count(), "逐行 3 次合规");
|
||||
assertEquals(6, chats.size(), "3 次分类 + 3 次合规,一次都不许多:" + chats);
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("批量合规口径差分:合批与逐行写回行的三字段+状态逐行相等")
|
||||
void conformBatchAndSingleRowPathsAgreeOnRowFields() {
|
||||
List<String> batchedChats = new CopyOnWriteArrayList<>();
|
||||
List<SimilarAsinResultRowDto> batched = service(
|
||||
recordingClient(batchedChats, CATEGORY_BATCH_JSON, SINGLE_CATEGORY_JSON, CONFORM_JSON),
|
||||
duplicateTreeCategoryService()).inspectRows(threeRows(), "", "k", false, true);
|
||||
|
||||
List<String> singleChats = new CopyOnWriteArrayList<>();
|
||||
SimilarAsinProperties off = properties();
|
||||
off.setLlmConformBatchSize(1);
|
||||
List<SimilarAsinResultRowDto> single = service(
|
||||
recordingClient(singleChats, CATEGORY_BATCH_JSON, SINGLE_CATEGORY_JSON, CONFORM_JSON),
|
||||
duplicateTreeCategoryService(), off).inspectRows(threeRows(), "", "k", false, true);
|
||||
|
||||
for (int i = 0; i < batched.size(); i++) {
|
||||
assertEquals(single.get(i).getIsConform(), batched.get(i).getIsConform(), "行 " + i + " is_conform");
|
||||
assertEquals(single.get(i).getReason(), batched.get(i).getReason(), "行 " + i + " reason");
|
||||
assertEquals(single.get(i).getCategory(), batched.get(i).getCategory(), "行 " + i + " category");
|
||||
assertEquals(single.get(i).getStatus(), batched.get(i).getStatus(), "行 " + i + " status");
|
||||
}
|
||||
}
|
||||
|
||||
/** 从记录下来的调用里取那一次批量合规正文(必须恰好一次,否则用例自己在测别的东西)。 */
|
||||
private static String onlyConformBatch(List<String> chats) {
|
||||
List<String> batches = chats.stream().filter(text -> text.contains("待判定商品")).toList();
|
||||
assertEquals(1, batches.size(), "批量合规应恰好发生一次:" + chats);
|
||||
return batches.get(0);
|
||||
}
|
||||
}
|
||||
|
||||
+71
@@ -31,6 +31,7 @@ import java.util.List;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.mockito.ArgumentMatchers.any;
|
||||
import static org.mockito.ArgumentMatchers.anyLong;
|
||||
import static org.mockito.ArgumentMatchers.eq;
|
||||
import static org.mockito.Mockito.mock;
|
||||
import static org.mockito.Mockito.never;
|
||||
@@ -360,4 +361,74 @@ class UserSecretUsageServiceTest {
|
||||
assertThat(wrapper.getSqlSegment()).doesNotContain("user_id");
|
||||
}
|
||||
|
||||
// --- token 读数(V157):计次与 token 是两次写入,各自口径不许互相带跑 ---
|
||||
|
||||
@Test
|
||||
@DisplayName("recordTokens:只补 token 读数,一次计次都不许多写")
|
||||
void recordTokensAddsTokensWithoutTouchingCallCount() {
|
||||
UserSecretUsageService service = newService();
|
||||
|
||||
service.recordTokens(7L, "appearance-patent", 1234L, 56L);
|
||||
|
||||
verify(usageMapper).addTokens(eq(7L), eq("appearance-patent"), eq(LocalDate.now()), eq(1234L), eq(56L));
|
||||
verify(usageMapper, never()).increment(any(), any(), any(), any(Integer.class));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("recordTokens:两侧都是 0 就不写——0 会被读成「实测消耗为 0」,那是假账")
|
||||
void recordTokensSkipsWhenBothSidesAreZero() {
|
||||
UserSecretUsageService service = newService();
|
||||
|
||||
service.recordTokens(7L, "appearance-patent", 0L, 0L);
|
||||
service.recordTokens(7L, "appearance-patent", -5L, 0L);
|
||||
|
||||
verify(usageMapper, never()).addTokens(any(), any(), any(), anyLong(), anyLong());
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("recordTokens:单侧有读数也要写,缺的那侧留 0")
|
||||
void recordTokensWritesWhenOneSideHasReading() {
|
||||
UserSecretUsageService service = newService();
|
||||
|
||||
service.recordTokens(7L, "similar-asin", 900L, 0L);
|
||||
|
||||
verify(usageMapper).addTokens(eq(7L), eq("similar-asin"), eq(LocalDate.now()), eq(900L), eq(0L));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("recordTokens:用户非法或模块不认识时整条跳过,不落到别人的账上")
|
||||
void recordTokensSkipsInvalidUserOrUnknownModule() {
|
||||
UserSecretUsageService service = newService();
|
||||
|
||||
service.recordTokens(null, "proxy", 10L, 10L);
|
||||
service.recordTokens(0L, "proxy", 10L, 10L);
|
||||
service.recordTokens(7L, "not-a-module", 10L, 10L);
|
||||
|
||||
verify(usageMapper, never()).addTokens(any(), any(), any(), anyLong(), anyLong());
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("recordTokens:写库失败只告警,绝不把任务带崩")
|
||||
void recordTokensSwallowsMapperFailure() {
|
||||
UserSecretUsageService service = newService();
|
||||
when(usageMapper.addTokens(any(), any(), any(), anyLong(), anyLong()))
|
||||
.thenThrow(new IllegalStateException("db down"));
|
||||
|
||||
service.recordTokens(7L, "proxy", 10L, 2L);
|
||||
|
||||
verify(usageMapper).addTokens(eq(7L), eq("proxy"), eq(LocalDate.now()), eq(10L), eq(2L));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("计次那一路不许顺手写 token:两条语句分开是口径要求(计次先于响应)")
|
||||
void countPathNeverWritesTokens() {
|
||||
UserSecretUsageService service = newService();
|
||||
when(usageMapper.increment(any(), any(), any(), eq(1))).thenReturn(1);
|
||||
|
||||
service.record(7L, "appearance-patent", 1);
|
||||
service.recordReported(7L, "proxy", 3);
|
||||
|
||||
verify(usageMapper, never()).addTokens(any(), any(), any(), anyLong(), anyLong());
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user