降本三条(10-11 现网取证):外观同请求图片去重 + 货源合规合批 + LLM token 计量落库

取证基线(库侧 biz_user_secret_usage_daily,10-01~10-11 实测 12,704 / 4,134 / 15,135 次):
外观 24h 全量 831 条请求逐条 json 解析,564 条带图共投 2,693 张、去重只剩 1,465 张
⇒ 45.6% 是同一张图在同一次请求里的副本;重试占比只有 0.24%(attempt 829/2/0),
所以省钱的口子在图片副本,不在重试。货源 4,134 次里 4,096 次集中在 10-10/10-11 两天,
类目已合批(10-02)而合规与图比仍逐行 ⇒ 合规是该模块最后一步逐行 LLM 调用。

1) 外观:每张不同的图只随请求发一份,正文附「图j=序号1、2…」对照表;每个序号仍各自
   出一份 status/reason(省副本,不省判定,与 10-10「同组兄弟行也要都检测」同口径)。
   归一化 URL 只当去重的键,发出去的还是行里那条原始串。应急回退:
   llm-appearance-batch-size=1 即退回逐行,无需改码。
2) 货源:新键 llm-conform-batch-size=10(设 1 退回逐行)。前置条件是分类预计算存在,
   否则整条不生效——避免同一行的类目被算两遍(两遍=多扣用户一次);缺编号行与整块
   失败都逐行补检,并有一支 A/B 差分管住合批与逐行写回行的三字段+状态相等。
3) V157 给 biz_user_secret_usage_daily 加 prompt_tokens/completion_tokens,两个 client 收下
   网关 usage。计次与 token 分两条语句:计次发生在发请求之前(含重试,请求没回来也算
   一次),token 只在响应回来后有读数;网关没回 usage 就一条不写——0 会被读成"实测为 0"。

护栏记账:新 mapper 语句 addTokens 两处一起点名(SqlLayeringConventionTest 分母 97→98 +
MigratedSqlEquivalenceTest.NOT_MIGRATED);批量路径的 catch 不新增一处——把分类那一路的
"整块失败→逐行补检"抽成 batchAttempt/fallbackSingleRows 共用,静默 catch 棘轮四格一格没放宽。
顺带修掉 AppearancePatentResultRowDto 上一句已经过期的 javadoc(还写着兄弟行复用代表行结论)。

证据:后端全量 4288 例 0 红(本批 +24 例:去重 7 / token 4 / 合规合批 7 / 用量服务 6)。
变异自证 3/3 全杀:取消去重、摘掉 usage 落库、断掉合规合批,分别让对应用例判红;
两个受试文件按注入前字节还原,sha 复算一致。
This commit is contained in:
2026-10-11 23:52:19 +08:00
parent c4d2349cda
commit b0829db81e
17 changed files with 1273 additions and 54 deletions
@@ -0,0 +1,339 @@
package com.nanri.aiimage.modules.appearancepatent.client;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.nanri.aiimage.config.AppearancePatentProperties;
import com.nanri.aiimage.config.BrandCheckProperties;
import com.nanri.aiimage.modules.appearancepatent.model.dto.AppearancePatentResultRowDto;
import com.nanri.aiimage.modules.brand.client.BrandCheckClient;
import com.nanri.aiimage.modules.usersecret.service.UserSecretUsageService;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpServer;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.mock;
/**
* 外观专利「同一次请求内,同一张图只随请求发一份」的用例。
*
* <p>为什么要这条:2026-10-11 现网取证,主机B 24h 全量 831 条外观请求逐条解析,564 条带图、
* 共投 2,693 张、去重只剩 1,465 张 —— 45.6% 是同一张图在同一批里的副本(兄弟行共用父号主图,
* 2026-10-10 起改成逐行送检后更频繁)。图片是外观判定的输入大头,副本直接计费。
*
* <p>这批用例钉两件事,缺一不可:
* ① 副本被去掉(image_url 段数 = 去重后的张数);
* ② 判定一份没少(每个序号仍各出一份 status/reason)——省的是副本,不是判定。
*
* <p>本文件的中文一律写成 \\u 转义:断言要比对请求正文里的中文标记,而读文件工具会给裸中文
* 行多塞一个字(项目内既有用例同此写法,不是洁癖)。
*/
class AppearancePatentLlmClientImageDedupeTest {
/** 「、」=图片对照里序号之间的分隔符。 */
private static final String IDEOGRAPHIC_COMMA = "、";
/** 「图」与「=序号」=对照表一行的前后缀。 */
private static final String FIGURE = "\u56fe";
private static final String LEGEND_EQ = "\uff1d\u5e8f\u53f7";
private HttpServer server;
private final ObjectMapper objectMapper = new ObjectMapper();
private final List<String> bodies = Collections.synchronizedList(new ArrayList<>());
private final AtomicInteger textRequests = new AtomicInteger();
private final AtomicInteger imageRequests = new AtomicInteger();
/** 外观批量响应改写钩子:模拟模型只回了前 K 项(漏行形状)。 */
private volatile int appearanceItemsToReturn = 0;
private AppearancePatentLlmClient client;
@BeforeEach
void setUp() throws IOException {
server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
server.createContext("/v1/chat/completions", this::handleChat);
server.createContext("/brand_check",
exchange -> respond(exchange, 200, "{\"faild_data\":[],\"query_faild_data\":[]}"));
server.start();
AppearancePatentProperties properties = new AppearancePatentProperties();
properties.setLlmHost("http://127.0.0.1:" + server.getAddress().getPort());
properties.setLlmRetryTimes(1);
client = new AppearancePatentLlmClient(properties, objectMapper, null, localBrandCheckClient(),
mock(UserSecretUsageService.class));
}
private BrandCheckClient localBrandCheckClient() {
BrandCheckProperties brand = new BrandCheckProperties();
brand.setBaseUrl("http://127.0.0.1:" + server.getAddress().getPort());
brand.setRetryTimes(1);
brand.setRetryIntervalMillis(0);
brand.setTotalTimeoutMillis(5000);
return new BrandCheckClient(brand, null);
}
@AfterEach
void tearDown() {
server.stop(0);
}
private void handleChat(HttpExchange exchange) throws IOException {
String body = new String(exchange.getRequestBody().readAllBytes(), StandardCharsets.UTF_8);
bodies.add(body);
boolean withImage = body.contains("\"type\":\"image_url\"");
if (withImage) {
imageRequests.incrementAndGet();
} else {
textRequests.incrementAndGet();
}
List<Integer> indexes = new ArrayList<>();
List<String> texts = new ArrayList<>();
parseIndexedLines(textPartOf(body), indexes, texts);
List<Map<String, Object>> items = new ArrayList<>();
for (int i = 0; i < indexes.size(); i++) {
if (withImage && appearanceItemsToReturn > 0 && i >= appearanceItemsToReturn) {
continue;
}
if (withImage) {
items.add(Map.of("i", indexes.get(i),
"appearance_status", "\u65e0\u4fb5\u6743",
"appearance_reason", "\u3010\u89c6\u89c9\u62c6\u89e3\u3011\uff1a\u901a\u7528\u5f62\u72b6"));
} else {
items.add(Map.of("i", indexes.get(i), "keywords", "\u65e0"));
}
}
try {
String content = objectMapper.writeValueAsString(Map.of("items", items));
respond(exchange, 200, objectMapper.writeValueAsString(Map.of(
"choices", List.of(Map.of("message", Map.of("content", content))))));
} catch (Exception ex) {
respond(exchange, 500, "{\"error\":{\"message\":\"" + ex + "\"}}");
}
}
private void respond(HttpExchange exchange, int status, String body) throws IOException {
byte[] bytes = body.getBytes(StandardCharsets.UTF_8);
exchange.getResponseHeaders().set("Content-Type", "application/json; charset=utf-8");
exchange.sendResponseHeaders(status, bytes.length);
try (OutputStream os = exchange.getResponseBody()) {
os.write(bytes);
}
}
/** 请求正文里那一段文本(多模态请求的其余段是 image_url)。 */
@SuppressWarnings("unchecked")
private String textPartOf(String body) {
try {
Map<String, Object> request = objectMapper.readValue(body, Map.class);
List<Object> messages = (List<Object>) request.get("messages");
Map<String, Object> last = (Map<String, Object>) messages.get(messages.size() - 1);
Object content = last.get("content");
if (content instanceof List<?> parts) {
for (Object part : parts) {
if (part instanceof Map<?, ?> map && "text".equals(map.get("type"))) {
return String.valueOf(map.get("text"));
}
}
return "";
}
return content == null ? "" : String.valueOf(content);
} catch (Exception ex) {
throw new IllegalStateException(ex);
}
}
private void parseIndexedLines(String raw, List<Integer> indexes, List<String> texts) {
for (String line : raw.split("\n")) {
int tab = line.indexOf('\t');
if (tab <= 0) {
continue;
}
try {
indexes.add(Integer.parseInt(line.substring(0, tab).trim()));
} catch (NumberFormatException ignored) {
continue;
}
texts.add(line.substring(tab + 1));
}
}
private static AppearancePatentResultRowDto row(String asin, String title, String url) {
AppearancePatentResultRowDto row = new AppearancePatentResultRowDto();
row.setId(asin);
row.setAsin(asin);
row.setTitle(title);
row.setUrl(url);
return row;
}
private static String urlOf(char letter) {
return "https://m.media-amazon.com/images/I/" + letter + "L._AC_SX679_.jpg";
}
/** 所有请求正文里 image_url 段的总数。 */
private int sentImageParts() {
return bodies.stream().mapToInt(AppearancePatentLlmClientImageDedupeTest::imagePartsIn).sum();
}
private static int imagePartsIn(String body) {
return splitOccurrences(body).size();
}
private static List<Integer> splitOccurrences(String body) {
List<Integer> marks = new ArrayList<>();
for (int i = body.indexOf("\"type\":\"image_url\""); i >= 0; i = body.indexOf("\"type\":\"image_url\"", i + 1)) {
marks.add(i);
}
return marks;
}
private int imagePartsOfLastImageRequest() {
for (int i = bodies.size() - 1; i >= 0; i--) {
String body = bodies.get(i);
if (body.contains("\"type\":\"image_url\"")) {
return splitOccurrences(body).size();
}
}
return -1;
}
private String lastImageRequestText() {
for (int i = bodies.size() - 1; i >= 0; i--) {
String body = bodies.get(i);
if (body.contains("\"type\":\"image_url\"")) {
return textPartOf(body);
}
}
return "";
}
@Test
@DisplayName("五行共用一张图:只发一份图片,但五行各出一份判定")
void fiveRowsSharingOneImageSendOneCopyButFiveJudgements() {
List<AppearancePatentResultRowDto> rows = new ArrayList<>();
for (int i = 1; i <= 5; i++) {
rows.add(row("B0" + i, "防滑罩 尺寸" + i, urlOf('X')));
}
List<AppearancePatentResultRowDto> result = client.inspectRows(rows, "", "k");
assertThat(imagePartsOfLastImageRequest()).isEqualTo(1);
String text = lastImageRequestText();
assertThat(text).contains(FIGURE + "1" + LEGEND_EQ + "1" + IDEOGRAPHIC_COMMA + "2"
+ IDEOGRAPHIC_COMMA + "3" + IDEOGRAPHIC_COMMA + "4" + IDEOGRAPHIC_COMMA + "5");
assertThat(result).hasSize(5);
for (AppearancePatentResultRowDto row : result) {
assertThat(row.getAppearanceRisk()).contains("\u65e0\u4fb5\u6743");
}
}
@Test
@DisplayName("五行五张不同图:仍发五份,对照表逐行一条(去重不许误伤不同的图)")
void fiveDistinctImagesStillSendFiveCopies() {
List<AppearancePatentResultRowDto> rows = new ArrayList<>();
for (int i = 0; i < 5; i++) {
rows.add(row("B0" + i, "杯子 " + i, urlOf((char) ('A' + i))));
}
client.inspectRows(rows, "", "k");
assertThat(imagePartsOfLastImageRequest()).isEqualTo(5);
String text = lastImageRequestText();
for (int i = 1; i <= 5; i++) {
assertThat(text).contains(FIGURE + i + LEGEND_EQ + i);
}
}
@Test
@DisplayName("混合形状(A,A,B,B,C):发三份,图1=1、2,图2=3、4,图3=5")
void mixedDuplicatesSendThreeCopiesWithCorrectRowMapping() {
List<AppearancePatentResultRowDto> rows = List.of(
row("B01", "a", urlOf('A')), row("B02", "b", urlOf('A')),
row("B03", "c", urlOf('B')), row("B04", "d", urlOf('B')),
row("B05", "e", urlOf('C')));
client.inspectRows(rows, "", "k");
assertThat(imagePartsOfLastImageRequest()).isEqualTo(3);
String text = lastImageRequestText();
assertThat(text).contains(FIGURE + "1" + LEGEND_EQ + "1" + IDEOGRAPHIC_COMMA + "2");
assertThat(text).contains(FIGURE + "2" + LEGEND_EQ + "3" + IDEOGRAPHIC_COMMA + "4");
assertThat(text).contains(FIGURE + "3" + LEGEND_EQ + "5");
assertThat(text).doesNotContain(FIGURE + "4");
}
@Test
@DisplayName("去重后图片份数不等于行数时,判定条数仍按行数收(省副本不是省判定)")
void judgementCountFollowsRowsNotImages() {
List<AppearancePatentResultRowDto> rows = List.of(
row("B01", "a", urlOf('A')), row("B02", "b", urlOf('A')),
row("B03", "c", urlOf('A')), row("B04", "d", urlOf('A')),
row("B05", "e", urlOf('A')));
List<AppearancePatentResultRowDto> result = client.inspectRows(rows, "", "k");
assertThat(imagePartsOfLastImageRequest()).isEqualTo(1);
assertThat(result).hasSize(5);
assertThat(result).allSatisfy(item -> assertThat(item.getError()).isNullOrEmpty());
}
@Test
@DisplayName("模型漏回序号:缺失的行仍走单行补检,不静默出空结论")
void missingIndexedItemStillFallsBackToSingleRow() {
appearanceItemsToReturn = 2;
List<AppearancePatentResultRowDto> rows = new ArrayList<>();
for (int i = 1; i <= 5; i++) {
rows.add(row("B0" + i, "防滑罩 " + i, urlOf('X')));
}
client.inspectRows(rows, "", "k");
// 1 次批量外观 + 3 行补检:补检必须发生,否则漏掉的行就空着出文件了
assertThat(imageRequests.get()).isGreaterThanOrEqualTo(4);
}
@Test
@DisplayName("同一张图只是两端多了空格:也算同一张,且发出去的还是行里那条原始串")
void dedupeKeysOnNormalizedUrlButSendsRawUrl() {
String bare = urlOf('Z');
List<AppearancePatentResultRowDto> rows = List.of(
row("B01", "a", " " + bare), row("B02", "b", bare), row("B03", "c", bare + " "));
client.inspectRows(rows, "", "k");
assertThat(imagePartsOfLastImageRequest()).isEqualTo(1);
String body = lastImageRequestBody();
// 归一化只当键:发送内容仍是首次出现的那条原始串(带前导空格),不改写行的数据。
assertThat(body).contains("\"url\":\" " + bare + "\"");
assertThat(body).doesNotContain("\"url\":\"" + bare + "\"");
}
private String lastImageRequestBody() {
for (int i = bodies.size() - 1; i >= 0; i--) {
if (bodies.get(i).contains("\"type\":\"image_url\"")) {
return bodies.get(i);
}
}
return "";
}
@Test
@DisplayName("标题提取请求不带图:去重只作用在外观那一路")
void titlePassStaysTextOnly() {
List<AppearancePatentResultRowDto> rows = List.of(
row("B01", "a", urlOf('A')), row("B02", "b", urlOf('A')));
client.inspectRows(rows, "", "k");
assertThat(textRequests.get()).isEqualTo(1);
assertThat(sentImageParts()).isEqualTo(1);
}
}
@@ -0,0 +1,181 @@
package com.nanri.aiimage.modules.appearancepatent.client;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.nanri.aiimage.config.AppearancePatentProperties;
import com.nanri.aiimage.config.BrandCheckProperties;
import com.nanri.aiimage.modules.appearancepatent.model.dto.AppearancePatentResultRowDto;
import com.nanri.aiimage.modules.brand.client.BrandCheckClient;
import com.nanri.aiimage.modules.usersecret.service.UserSecretUsageService;
import com.nanri.aiimage.modules.usersecret.support.SecretUsageContext;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpServer;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.List;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyInt;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
/**
* 外观专利把网关 {@code usage} 落库的用例(V157 的两列 token 读数)。
*
* <p>为什么要这条:降本决策一直只能用「次数」和字符数推算,因为两个 LLM client 都只记 1 次、
* 从不读响应里的 usage。这批用例钉三条口径,一条都不能漂:
* ① 计次仍然发生在**发请求之前**(含重试,请求没回来也算一次);
* ② token 读数只在响应回来之后补写;
* ③ 网关没回 usage 就一条都不写 —— 不许写 0 冒充「实测消耗为 0」,那种行以后会被拿去算成本。
*/
class AppearancePatentLlmClientTokenUsageTest {
private HttpServer server;
private final ObjectMapper objectMapper = new ObjectMapper();
private final UserSecretUsageService usageService = mock(UserSecretUsageService.class);
/** 响应的 usage 形状:full=两个都有、none=整个 usage 缺失、prompt_only=只有输入侧。 */
private volatile String usageShape = "full";
private AppearancePatentLlmClient client;
@BeforeEach
void setUp() throws IOException {
server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
server.createContext("/v1/chat/completions", this::handleChat);
server.createContext("/brand_check",
exchange -> respond(exchange, "{\"faild_data\":[],\"query_faild_data\":[]}"));
server.start();
AppearancePatentProperties properties = new AppearancePatentProperties();
properties.setLlmHost("http://127.0.0.1:" + server.getAddress().getPort());
properties.setLlmRetryTimes(1);
client = new AppearancePatentLlmClient(properties, objectMapper, null, localBrandCheckClient(), usageService);
}
private BrandCheckClient localBrandCheckClient() {
BrandCheckProperties brand = new BrandCheckProperties();
brand.setBaseUrl("http://127.0.0.1:" + server.getAddress().getPort());
brand.setRetryTimes(1);
brand.setRetryIntervalMillis(0);
brand.setTotalTimeoutMillis(5000);
return new BrandCheckClient(brand, null);
}
@AfterEach
void tearDown() {
server.stop(0);
}
private void handleChat(HttpExchange exchange) throws IOException {
String body = new String(exchange.getRequestBody().readAllBytes(), StandardCharsets.UTF_8);
boolean withImage = body.contains("\"type\":\"image_url\"");
String keywords = "\u65e0";
String item = withImage
? "{\"i\":1,\"appearance_status\":\"\u65e0\u4fb5\u6743\","
+ "\"appearance_reason\":\"\u3010\u89c6\u89c9\u62c6\u89e3\u3011\u901a\u7528\"}"
: "{\"i\":1,\"keywords\":\"" + keywords + "\"}";
String content = "{\"items\":[" + item + "]}";
String escaped = content.replace("\"", "\\\"");
String usage = switch (usageShape) {
case "none" -> "";
case "prompt_only" -> ",\"usage\":{\"prompt_tokens\":1234}";
default -> ",\"usage\":{\"prompt_tokens\":1234,\"completion_tokens\":56}";
};
String response = "{\"choices\":[{\"message\":{\"content\":\"" + escaped + "\"}}]" + usage + "}";
// 测试桩自证:拼出来的必须是合法 JSON 且 usage 在根对象**里面**。
// 上一版把 usage 接在根对象的收尾括号之后,Jackson 默认容忍尾随内容 ⇒
// "网关没回 usage" 那条用例其实是恒绿的,这里先把这个形状钉死再发出去。
try {
var parsed = objectMapper.readTree(response);
if (!"none".equals(usageShape) && parsed.path("usage").isMissingNode()) {
throw new IllegalStateException("harness built a response without usable usage node");
}
if ("none".equals(usageShape) && !parsed.path("usage").isMissingNode()) {
throw new IllegalStateException("harness leaked a usage node into the none shape");
}
} catch (Exception ex) {
respond(exchange, "{\"error\":{\"message\":\"harness produced invalid json: " + ex + "\"}}");
return;
}
respond(exchange, response);
}
private void respond(HttpExchange exchange, String body) throws IOException {
byte[] bytes = body.getBytes(StandardCharsets.UTF_8);
exchange.getResponseHeaders().set("Content-Type", "application/json; charset=utf-8");
exchange.sendResponseHeaders(200, bytes.length);
try (OutputStream os = exchange.getResponseBody()) {
os.write(bytes);
}
}
private List<AppearancePatentResultRowDto> inspectOneRow() {
AppearancePatentResultRowDto row = new AppearancePatentResultRowDto();
row.setId("1");
row.setAsin("B001");
row.setTitle("\u9632\u6ed1\u7f69");
row.setUrl("https://m.media-amazon.com/images/I/71L._AC_SX679_.jpg");
// 计次与 token 都要归属到任务用户:没有这层上下文时两条读数都会静默跳过(无归属不记账)。
return SecretUsageContext.call(7L, "appearance-patent",
() -> client.inspectRows(List.of(row), "", "k"));
}
@Test
@DisplayName("网关回了 usage:两次请求各补一条 token 读数,计次一条不少")
void usageIsRecordedAfterResponseWithoutLosingTheCallCount() {
List<AppearancePatentResultRowDto> result = inspectOneRow();
assertThat(result).hasSize(1);
// 标题提取 + 外观判定 = 2 次对外请求
verify(usageService, times(2)).record(eq(7L), eq("appearance-patent"), eq(1));
verify(usageService, times(2))
.recordTokens(eq(7L), eq("appearance-patent"), eq(1234L), eq(56L));
}
@Test
@DisplayName("网关没回 usage:一条 token 都不写,计次照常")
void missingUsageWritesNoTokensInsteadOfZero() {
usageShape = "none";
inspectOneRow();
verify(usageService, times(2)).record(eq(7L), eq("appearance-patent"), eq(1));
verify(usageService, never()).recordTokens(anyLong(), anyString(), anyLong(), anyLong());
}
@Test
@DisplayName("usage 只回了一半:有读数的那侧照写,缺的那侧写 0,整条不能被丢掉")
void partialUsageStillStored() {
usageShape = "prompt_only";
inspectOneRow();
verify(usageService, times(2))
.recordTokens(eq(7L), eq("appearance-patent"), eq(1234L), eq(0L));
}
@Test
@DisplayName("无任务归属(context 为空):计次与 token 都不写,绝不记到别人账上")
void noContextRecordsNothing() {
AppearancePatentResultRowDto row = new AppearancePatentResultRowDto();
row.setId("1");
row.setAsin("B001");
row.setTitle("\u9632\u6ed1\u7f69");
row.setUrl("https://m.media-amazon.com/images/I/71L._AC_SX679_.jpg");
client.inspectRows(List.of(row), "", "k");
verify(usageService, never()).record(any(), anyString(), anyInt());
verify(usageService, never()).recordTokens(any(), anyString(), anyLong(), anyLong());
}
}
@@ -10,6 +10,7 @@ import com.nanri.aiimage.modules.similarasin.client.SimilarAsinLlmClient;
import com.nanri.aiimage.modules.similarasin.model.dto.SimilarAsinResultRowDto;
import com.nanri.aiimage.modules.similarasin.util.PuzzleImageMerger;
import com.nanri.aiimage.modules.similarasin.util.SimilarAsinImageEmbedder;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import javax.imageio.ImageIO;
@@ -352,6 +353,16 @@ class SimilarAsinLlmServiceTest {
/** 同上,另支持批量分类文本(含"待分类商品"标记)——batchCategoryJson 为 null 时批量调用视为意外。 */
private SimilarAsinLlmClient recordingClient(List<String> chatUserTexts, String batchCategoryJson,
String categoryJson, String conformJson) {
return recordingClientWithConformBatch(chatUserTexts, batchCategoryJson, null, categoryJson, conformJson);
}
/** 批量调用的响应桩:batchConformJson 为 null 时把 conformJson 按行数复制成 items(让合批走得通)。 */
private static final String CONFORM_BATCH_THROWS = "<<throw>>";
private SimilarAsinLlmClient recordingClientWithConformBatch(List<String> chatUserTexts,
String batchCategoryJson,
String batchConformJson,
String categoryJson, String conformJson) {
SimilarAsinLlmClient client = mock(SimilarAsinLlmClient.class);
when(client.resolveApiKey(anyString())).thenReturn("k");
when(client.hasApiKey(anyString())).thenReturn(true);
@@ -365,6 +376,13 @@ class SimilarAsinLlmServiceTest {
}
return batchCategoryJson;
}
if (userText.contains("待判定商品")) {
if (CONFORM_BATCH_THROWS.equals(batchConformJson)) {
throw new IllegalStateException("LLM gateway 502");
}
return batchConformJson != null
? batchConformJson : conformBatchJsonFor(userText, conformJson);
}
return userText.startsWith("产品标题:") ? categoryJson : conformJson;
});
when(client.invokeChatWithImages(anyString(), anyString(), anyString(), any(), anyString()))
@@ -374,6 +392,34 @@ class SimilarAsinLlmServiceTest {
return client;
}
/** 批量正文里有几件 [n] 商品,就把这份单件 JSON 复制成几个 item:只比调用形状时用,不改变判定内容。 */
private static String conformBatchJsonFor(String batchUserText, String conformJson) {
int rows = 0;
for (int i = 1; i < 500; i++) {
if (!batchUserText.contains("[" + i + "] 标题:")) {
break;
}
rows = i;
}
StringBuilder items = new StringBuilder();
String fields = conformJson.substring(1, conformJson.length() - 1);
for (int i = 1; i <= rows; i++) {
if (i > 1) {
items.append(',');
}
items.append("{\"i\":").append(i).append(',').append(fields).append('}');
}
return "{\"items\":[" + items + "]}";
}
/** 取批量合规正文里第 n 件商品那一段(到下一个 [m] 为止),用于逐件断言类目上下文。 */
private static String conformRowBlock(String batchUserText, int n) {
int start = batchUserText.indexOf("[" + n + "] 标题:");
assertTrue(start >= 0, "批量合规正文缺第 " + n + " 件商品:" + batchUserText);
int next = batchUserText.indexOf("\n[", start + 1);
return next < 0 ? batchUserText.substring(start) : batchUserText.substring(start, next);
}
private static SimilarAsinResultRowDto rowOf(String asin, String title) {
SimilarAsinResultRowDto row = new SimilarAsinResultRowDto();
row.setAsin(asin);
@@ -516,19 +562,18 @@ class SimilarAsinLlmServiceTest {
List.of(rowOf("B01", "布质防滑罩"), rowOf("B02", "亚麻沙发套"), rowOf("B03", "手机支架")),
"", "k", false, true);
// 1 次批量分类 + 3 次合规;树文本只在批量请求里出现一次(逐行模式会出现 3 次)
assertEquals(4, chats.size(), "实际调用:" + chats);
// 1 次批量分类 + 1 次批量合规(2026-10-11 合规也合批:省的是往返,不是判定)
assertEquals(2, chats.size(), "实际调用:" + chats);
assertEquals(1, countTreeTexts(chats));
String batchText = chats.get(0);
assertTrue(batchText.contains("布质防滑罩") && batchText.contains("亚麻沙发套") && batchText.contains("手机支架"),
"批量请求应包含整块全部行:" + batchText);
// 批量判定的行级语义与单行一致:i=2 second=无 → 合规入参二级为"无"且无三级候选
// 批量合规必须逐件带自己的违规类目上下文:i=2 的二级是"无",那一行不许出现别的分支的三级候选
String rowTwo = conformRowBlock(chats.get(1), 2);
assertTrue(rowTwo.contains("一级类目:家具") && rowTwo.contains("二级类目:无"),
"i=2 的判定应保留一级、二级为无,实际块:" + rowTwo);
assertFalse(rowTwo.contains("手机壳"), "二级为无时该行不该有三级候选:" + rowTwo);
assertTrue(result.get(1).getIsConform() != null, "每行都要完成合规判定");
String conformTexts = chats.stream().filter(text -> text.contains("三级类目:"))
.filter(text -> text.contains("一级类目:家具") && text.contains("二级类目:无"))
.findFirst().orElse("");
assertTrue(!conformTexts.isEmpty(), "i=2 的判定应保留一级、二级为无:" + chats);
assertFalse(conformTexts.contains("手机壳"), "二级为无时不应给出三级候选");
}
@Test
@@ -551,7 +596,8 @@ class SimilarAsinLlmServiceTest {
long singleCategoryCalls = chats.stream().filter(text -> text.startsWith("产品标题:")).count();
assertEquals(1, batchCalls);
assertEquals(1, singleCategoryCalls, "缺失行要回退单行分类调用:" + chats);
assertEquals(3, chats.size() - batchCalls - singleCategoryCalls, "3 行都要有合规调用:" + chats);
assertEquals(1, chats.size() - batchCalls - singleCategoryCalls,
"3 行合规合并成一次批量请求(2026-10-11 合批),不该再逐行发:" + chats);
}
@Test
@@ -605,4 +651,160 @@ class SimilarAsinLlmServiceTest {
assertEquals(3, chats.stream().filter(text -> text.contains("ID:100")).count(), "3 次分类各带一棵树");
assertEquals(6, chats.size(), "3 次分类 + 3 次合规");
}
// --- 批量合规(2026-10-11):货源最后一步逐行 LLM 调用合并成一批,省往返不省判定 ---
private static final String CATEGORY_BATCH_JSON =
"{\"items\":[{\"i\":1,\"first\":\"家具\",\"second\":\"配件\"},"
+ "{\"i\":2,\"first\":\"数码\",\"second\":\"配件\"},"
+ "{\"i\":3,\"first\":\"家具\",\"second\":\"无\"}]}";
private static final String SINGLE_CATEGORY_JSON = "{\"first\":\"家具\",\"second\":\"配件\"}";
private List<SimilarAsinResultRowDto> threeRows() {
return List.of(rowOf("B01", "布质防滑罩"), rowOf("B02", "手机支架"), rowOf("B03", "亚麻沙发套"));
}
@Test
@DisplayName("批量合规:每件商品带自己的违规类目上下文,禁止共用一份把 A 行的拦截判到 B 行")
void conformBatchCarriesEachRowOwnViolationContext() {
List<String> chats = new CopyOnWriteArrayList<>();
SimilarAsinLlmService service = service(
recordingClient(chats, CATEGORY_BATCH_JSON, SINGLE_CATEGORY_JSON, CONFORM_JSON),
duplicateTreeCategoryService());
service.inspectRows(threeRows(), "", "k", false, true);
String batch = onlyConformBatch(chats);
assertTrue(conformRowBlock(batch, 1).contains("一级类目:家具"), "第 1 件的上下文:" + conformRowBlock(batch, 1));
assertTrue(conformRowBlock(batch, 1).contains("沙发套"), "第 1 件二级=配件 ⇒ 三级候选取家具分支");
assertTrue(conformRowBlock(batch, 2).contains("一级类目:数码"), "第 2 件的上下文:" + conformRowBlock(batch, 2));
assertTrue(conformRowBlock(batch, 2).contains("手机壳"), "第 2 件二级=配件 ⇒ 三级候选取数码分支");
assertTrue(conformRowBlock(batch, 3).contains("二级类目:无"), "第 3 件的上下文:" + conformRowBlock(batch, 3));
assertFalse(conformRowBlock(batch, 3).contains("手机壳"), "二级为无的那件不许拿到别的分支的三级候选");
}
@Test
@DisplayName("批量合规:整块只发一次请求,且不顺手多算一次类目")
void conformBatchIsOneCallAndNeverRecalculatesCategory() {
List<String> chats = new CopyOnWriteArrayList<>();
SimilarAsinLlmService service = service(
recordingClient(chats, CATEGORY_BATCH_JSON, SINGLE_CATEGORY_JSON, CONFORM_JSON),
duplicateTreeCategoryService());
List<SimilarAsinResultRowDto> result = service.inspectRows(threeRows(), "", "k", false, true);
assertEquals(1, chats.stream().filter(text -> text.contains("待分类商品")).count());
assertEquals(0, chats.stream().filter(text -> text.startsWith("产品标题:")).count(),
"类目已由批量预计算给出,合规合批不得回头逐行重算(重算=多扣用户一次):" + chats);
assertEquals(1, chats.stream().filter(text -> text.contains("待判定商品")).count(), "3 行合规合并成一次");
assertEquals(2, chats.size(), "批量类目 + 批量合规,共 2 次对外请求:" + chats);
assertEquals(3, result.size());
for (SimilarAsinResultRowDto row : result) {
assertEquals("不符合", row.getIsConform(), "每行都要有自己的合规结果");
}
}
@Test
@DisplayName("批量合规:模型漏回编号的行回退逐行单调用,不静默出空结论")
void conformBatchMissingItemFallsBackToSingleRowCall() {
List<String> chats = new CopyOnWriteArrayList<>();
SimilarAsinLlmService service = service(
recordingClientWithConformBatch(chats, CATEGORY_BATCH_JSON,
"{\"items\":[{\"i\":1,\"is_conform\":\"符合\",\"reason\":\"无\",\"category\":\"家具\"}]}",
SINGLE_CATEGORY_JSON, CONFORM_JSON),
duplicateTreeCategoryService());
List<SimilarAsinResultRowDto> result = service.inspectRows(threeRows(), "", "k", false, true);
assertEquals(1, chats.stream().filter(text -> text.contains("待判定商品")).count());
assertEquals(2, chats.stream().filter(text -> text.startsWith("标题:")).count(),
"缺 i=2、i=3 ⇒ 这两行各回退一次单行合规:" + chats);
assertEquals("符合", result.get(0).getIsConform(), "批量给的结果");
assertEquals("不符合", result.get(1).getIsConform(), "回退单行给的结果");
assertEquals("不符合", result.get(2).getIsConform(), "回退单行给的结果");
}
@Test
@DisplayName("批量合规:整块传输失败 ⇒ 全块回退逐行,行级结果照常产出")
void conformBatchTransportErrorFallsBackForWholeChunk() {
List<String> chats = new CopyOnWriteArrayList<>();
SimilarAsinLlmService service = service(
recordingClientWithConformBatch(chats, CATEGORY_BATCH_JSON, CONFORM_BATCH_THROWS,
SINGLE_CATEGORY_JSON, CONFORM_JSON),
duplicateTreeCategoryService());
List<SimilarAsinResultRowDto> result = service.inspectRows(threeRows(), "", "k", false, true);
assertEquals(1, chats.stream().filter(text -> text.contains("待判定商品")).count(), "批量尝试过一次并失败");
assertEquals(3, chats.stream().filter(text -> text.startsWith("标题:")).count(), "整块回退逐行");
for (SimilarAsinResultRowDto row : result) {
assertEquals("不符合", row.getIsConform(), "回退后不得有空结论");
assertEquals("成功", row.getStatus());
}
}
@Test
@DisplayName("批量合规:阈值设 1 即退回逐行(应急开关,且不发批量请求)")
void conformBatchDisabledWhenBatchSizeOne() {
List<String> chats = new CopyOnWriteArrayList<>();
SimilarAsinProperties properties = properties();
properties.setLlmConformBatchSize(1);
SimilarAsinLlmService service = service(
recordingClient(chats, CATEGORY_BATCH_JSON, SINGLE_CATEGORY_JSON, CONFORM_JSON),
duplicateTreeCategoryService(), properties);
service.inspectRows(threeRows(), "", "k", false, true);
assertEquals(0, chats.stream().filter(text -> text.contains("待判定商品")).count(),
"批量关闭时不得发起批量合规调用");
assertEquals(3, chats.stream().filter(text -> text.startsWith("标题:")).count(), "逐行 3 次合规");
}
@Test
@DisplayName("批量合规:分类批量被关掉时合规合批必须不生效(否则同一行类目被算两遍=多扣一次)")
void conformBatchStaysOffWhenCategoryBatchIsOff() {
List<String> chats = new CopyOnWriteArrayList<>();
SimilarAsinProperties properties = properties();
properties.setLlmCategoryBatchSize(1);
SimilarAsinLlmService service = service(
recordingClient(chats, null, SINGLE_CATEGORY_JSON, CONFORM_JSON),
duplicateTreeCategoryService(), properties);
service.inspectRows(threeRows(), "", "k", false, true);
assertEquals(0, chats.stream().filter(text -> text.contains("待判定商品")).count(),
"没有逐行类目路径可用时不得发起批量合规:" + chats);
assertEquals(3, chats.stream().filter(text -> text.startsWith("标题:")).count(), "逐行 3 次合规");
assertEquals(6, chats.size(), "3 次分类 + 3 次合规,一次都不许多:" + chats);
}
@Test
@DisplayName("批量合规口径差分:合批与逐行写回行的三字段+状态逐行相等")
void conformBatchAndSingleRowPathsAgreeOnRowFields() {
List<String> batchedChats = new CopyOnWriteArrayList<>();
List<SimilarAsinResultRowDto> batched = service(
recordingClient(batchedChats, CATEGORY_BATCH_JSON, SINGLE_CATEGORY_JSON, CONFORM_JSON),
duplicateTreeCategoryService()).inspectRows(threeRows(), "", "k", false, true);
List<String> singleChats = new CopyOnWriteArrayList<>();
SimilarAsinProperties off = properties();
off.setLlmConformBatchSize(1);
List<SimilarAsinResultRowDto> single = service(
recordingClient(singleChats, CATEGORY_BATCH_JSON, SINGLE_CATEGORY_JSON, CONFORM_JSON),
duplicateTreeCategoryService(), off).inspectRows(threeRows(), "", "k", false, true);
for (int i = 0; i < batched.size(); i++) {
assertEquals(single.get(i).getIsConform(), batched.get(i).getIsConform(), "行 " + i + " is_conform");
assertEquals(single.get(i).getReason(), batched.get(i).getReason(), "行 " + i + " reason");
assertEquals(single.get(i).getCategory(), batched.get(i).getCategory(), "行 " + i + " category");
assertEquals(single.get(i).getStatus(), batched.get(i).getStatus(), "行 " + i + " status");
}
}
/** 从记录下来的调用里取那一次批量合规正文(必须恰好一次,否则用例自己在测别的东西)。 */
private static String onlyConformBatch(List<String> chats) {
List<String> batches = chats.stream().filter(text -> text.contains("待判定商品")).toList();
assertEquals(1, batches.size(), "批量合规应恰好发生一次:" + chats);
return batches.get(0);
}
}
@@ -31,6 +31,7 @@ import java.util.List;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
@@ -360,4 +361,74 @@ class UserSecretUsageServiceTest {
assertThat(wrapper.getSqlSegment()).doesNotContain("user_id");
}
// --- token 读数(V157):计次与 token 是两次写入,各自口径不许互相带跑 ---
@Test
@DisplayName("recordTokens:只补 token 读数,一次计次都不许多写")
void recordTokensAddsTokensWithoutTouchingCallCount() {
UserSecretUsageService service = newService();
service.recordTokens(7L, "appearance-patent", 1234L, 56L);
verify(usageMapper).addTokens(eq(7L), eq("appearance-patent"), eq(LocalDate.now()), eq(1234L), eq(56L));
verify(usageMapper, never()).increment(any(), any(), any(), any(Integer.class));
}
@Test
@DisplayName("recordTokens:两侧都是 0 就不写——0 会被读成「实测消耗为 0」,那是假账")
void recordTokensSkipsWhenBothSidesAreZero() {
UserSecretUsageService service = newService();
service.recordTokens(7L, "appearance-patent", 0L, 0L);
service.recordTokens(7L, "appearance-patent", -5L, 0L);
verify(usageMapper, never()).addTokens(any(), any(), any(), anyLong(), anyLong());
}
@Test
@DisplayName("recordTokens:单侧有读数也要写,缺的那侧留 0")
void recordTokensWritesWhenOneSideHasReading() {
UserSecretUsageService service = newService();
service.recordTokens(7L, "similar-asin", 900L, 0L);
verify(usageMapper).addTokens(eq(7L), eq("similar-asin"), eq(LocalDate.now()), eq(900L), eq(0L));
}
@Test
@DisplayName("recordTokens:用户非法或模块不认识时整条跳过,不落到别人的账上")
void recordTokensSkipsInvalidUserOrUnknownModule() {
UserSecretUsageService service = newService();
service.recordTokens(null, "proxy", 10L, 10L);
service.recordTokens(0L, "proxy", 10L, 10L);
service.recordTokens(7L, "not-a-module", 10L, 10L);
verify(usageMapper, never()).addTokens(any(), any(), any(), anyLong(), anyLong());
}
@Test
@DisplayName("recordTokens:写库失败只告警,绝不把任务带崩")
void recordTokensSwallowsMapperFailure() {
UserSecretUsageService service = newService();
when(usageMapper.addTokens(any(), any(), any(), anyLong(), anyLong()))
.thenThrow(new IllegalStateException("db down"));
service.recordTokens(7L, "proxy", 10L, 2L);
verify(usageMapper).addTokens(eq(7L), eq("proxy"), eq(LocalDate.now()), eq(10L), eq(2L));
}
@Test
@DisplayName("计次那一路不许顺手写 token:两条语句分开是口径要求(计次先于响应)")
void countPathNeverWritesTokens() {
UserSecretUsageService service = newService();
when(usageMapper.increment(any(), any(), any(), eq(1))).thenReturn(1);
service.record(7L, "appearance-patent", 1);
service.recordReported(7L, "proxy", 3);
verify(usageMapper, never()).addTokens(any(), any(), any(), anyLong(), anyLong());
}
}