task-121: N+1 扫描审计（循环内单查扫描器 + 10 条测试 + 报告快照，不改生产代码）
This commit is contained in:
@@ -0,0 +1,317 @@
|
||||
{
|
||||
"scanned_files": 640,
|
||||
"modules": [
|
||||
"admin",
|
||||
"appearancepatent",
|
||||
"auth",
|
||||
"brand",
|
||||
"collectdata",
|
||||
"convert",
|
||||
"debug",
|
||||
"dedupe",
|
||||
"deletebrand",
|
||||
"digitalhuman",
|
||||
"file",
|
||||
"filetemplate",
|
||||
"imagehistory",
|
||||
"imagevideo",
|
||||
"invalidasin",
|
||||
"patroldelete",
|
||||
"permission",
|
||||
"pricetrack",
|
||||
"productcategory",
|
||||
"productrisk",
|
||||
"publish",
|
||||
"queryasin",
|
||||
"shopdatacrawl",
|
||||
"shopkey",
|
||||
"shopmatch",
|
||||
"similarasin",
|
||||
"split",
|
||||
"task",
|
||||
"withdraw",
|
||||
"ziniao"
|
||||
],
|
||||
"findings": [
|
||||
{
|
||||
"file": "deletebrand\\service\\DeleteBrandStaleTaskService.java",
|
||||
"line": 209,
|
||||
"statement": "FileTaskEntity refreshed = fileTaskMapper.selectById(task.getId());",
|
||||
"loop_type": "for",
|
||||
"impact": "HIGH",
|
||||
"module": "deletebrand"
|
||||
},
|
||||
{
|
||||
"file": "pricetrack\\service\\PriceTrackTaskService.java",
|
||||
"line": 354,
|
||||
"statement": "FileResultEntity lockedResult = fileResultMapper.selectById(fr.getId());",
|
||||
"loop_type": "for",
|
||||
"impact": "HIGH",
|
||||
"module": "pricetrack"
|
||||
},
|
||||
{
|
||||
"file": "productrisk\\service\\ProductRiskTaskService.java",
|
||||
"line": 339,
|
||||
"statement": "FileResultEntity lockedResult = fileResultMapper.selectById(fr.getId());",
|
||||
"loop_type": "for",
|
||||
"impact": "HIGH",
|
||||
"module": "productrisk"
|
||||
},
|
||||
{
|
||||
"file": "task\\service\\TaskFileJobService.java",
|
||||
"line": 282,
|
||||
"statement": "publishDispatchEvent(taskFileJobMapper.selectById(job.getId()));",
|
||||
"loop_type": "for",
|
||||
"impact": "HIGH",
|
||||
"module": "task"
|
||||
},
|
||||
{
|
||||
"file": "task\\service\\TaskFileJobService.java",
|
||||
"line": 303,
|
||||
"statement": "TaskFileJobEntity exhausted = taskFileJobMapper.selectById(job.getId());",
|
||||
"loop_type": "for",
|
||||
"impact": "HIGH",
|
||||
"module": "task"
|
||||
},
|
||||
{
|
||||
"file": "task\\service\\TaskFileJobService.java",
|
||||
"line": 315,
|
||||
"statement": "publishDispatchEvent(taskFileJobMapper.selectById(job.getId()));",
|
||||
"loop_type": "for",
|
||||
"impact": "HIGH",
|
||||
"module": "task"
|
||||
},
|
||||
{
|
||||
"file": "appearancepatent\\service\\AppearancePatentTaskService.java",
|
||||
"line": 1094,
|
||||
"statement": "TaskChunkEntity chunk = taskChunkMapper.selectOne(new LambdaQueryWrapper<TaskChunkEntity>()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "appearancepatent"
|
||||
},
|
||||
{
|
||||
"file": "appearancepatent\\service\\AppearancePatentTaskService.java",
|
||||
"line": 2112,
|
||||
"statement": "List<TaskChunkEntity> page = taskChunkMapper.selectList(query);",
|
||||
"loop_type": "while",
|
||||
"impact": "LOW",
|
||||
"module": "appearancepatent"
|
||||
},
|
||||
{
|
||||
"file": "dedupe\\service\\DedupeTotalDataService.java",
|
||||
"line": 279,
|
||||
"statement": "List<DedupeTotalDataEntity> rows = dedupeTotalDataMapper.selectList(",
|
||||
"loop_type": "while",
|
||||
"impact": "LOW",
|
||||
"module": "dedupe"
|
||||
},
|
||||
{
|
||||
"file": "deletebrand\\service\\DeleteBrandRunService.java",
|
||||
"line": 1234,
|
||||
"statement": "tasks.addAll(fileTaskMapper.selectList(new LambdaQueryWrapper<FileTaskEntity>()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "deletebrand"
|
||||
},
|
||||
{
|
||||
"file": "deletebrand\\service\\DeleteBrandRunService.java",
|
||||
"line": 1252,
|
||||
"statement": "rows.addAll(fileResultMapper.selectList(historyResultQuery()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "deletebrand"
|
||||
},
|
||||
{
|
||||
"file": "deletebrand\\service\\DeleteBrandRunService.java",
|
||||
"line": 1268,
|
||||
"statement": "List<FileResultEntity> rows = fileResultMapper.selectList(historyResultQuery()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "deletebrand"
|
||||
},
|
||||
{
|
||||
"file": "patroldelete\\service\\PatrolDeleteTaskService.java",
|
||||
"line": 638,
|
||||
"statement": "List<FileTaskEntity> tasks = fileTaskMapper.selectList(new LambdaQueryWrapper<FileTaskEntity>()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "patroldelete"
|
||||
},
|
||||
{
|
||||
"file": "permission\\service\\PermissionMenuService.java",
|
||||
"line": 676,
|
||||
"statement": "cursor = permissionMenuMapper.selectById(ancestorId);",
|
||||
"loop_type": "while",
|
||||
"impact": "LOW",
|
||||
"module": "permission"
|
||||
},
|
||||
{
|
||||
"file": "pricetrack\\service\\PriceTrackTaskService.java",
|
||||
"line": 153,
|
||||
"statement": "tasks.addAll(fileTaskMapper.selectList(new LambdaQueryWrapper<FileTaskEntity>()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "pricetrack"
|
||||
},
|
||||
{
|
||||
"file": "pricetrack\\service\\PriceTrackTaskService.java",
|
||||
"line": 418,
|
||||
"statement": "List<FileTaskEntity> tasks = fileTaskMapper.selectList(new LambdaQueryWrapper<FileTaskEntity>()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "pricetrack"
|
||||
},
|
||||
{
|
||||
"file": "productcategory\\service\\ProductCategoryService.java",
|
||||
"line": 465,
|
||||
"statement": "ProductCategoryEntity parent = productCategoryMapper.selectById(cursor.getParentId());",
|
||||
"loop_type": "while",
|
||||
"impact": "LOW",
|
||||
"module": "productcategory"
|
||||
},
|
||||
{
|
||||
"file": "productcategory\\service\\ProductCategoryService.java",
|
||||
"line": 481,
|
||||
"statement": "cursor = cursor.getParentId() == null ? null : productCategoryMapper.selectById(cursor.getParentId());",
|
||||
"loop_type": "while",
|
||||
"impact": "LOW",
|
||||
"module": "productcategory"
|
||||
},
|
||||
{
|
||||
"file": "productcategory\\service\\ProductCategoryService.java",
|
||||
"line": 518,
|
||||
"statement": "ProductCategoryEntity row = productCategoryMapper.selectById(cursor);",
|
||||
"loop_type": "while",
|
||||
"impact": "LOW",
|
||||
"module": "productcategory"
|
||||
},
|
||||
{
|
||||
"file": "productrisk\\service\\ProductRiskTaskService.java",
|
||||
"line": 144,
|
||||
"statement": "tasks.addAll(fileTaskMapper.selectList(new LambdaQueryWrapper<FileTaskEntity>()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "productrisk"
|
||||
},
|
||||
{
|
||||
"file": "queryasin\\service\\QueryAsinTaskService.java",
|
||||
"line": 587,
|
||||
"statement": "List<FileTaskEntity> tasks = fileTaskMapper.selectList(new LambdaQueryWrapper<FileTaskEntity>()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "queryasin"
|
||||
},
|
||||
{
|
||||
"file": "shopdatacrawl\\service\\ShopDataCrawlTaskService.java",
|
||||
"line": 1085,
|
||||
"statement": "List<FileTaskEntity> tasks = fileTaskMapper.selectList(new LambdaQueryWrapper<FileTaskEntity>()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "shopdatacrawl"
|
||||
},
|
||||
{
|
||||
"file": "shopdatacrawl\\service\\ShopDataCrawlTaskService.java",
|
||||
"line": 1940,
|
||||
"statement": "FileResultEntity result = fileResultMapper.selectById(member.getResultId());",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "shopdatacrawl"
|
||||
},
|
||||
{
|
||||
"file": "shopdatacrawl\\service\\ShopDataCrawlTaskService.java",
|
||||
"line": 2497,
|
||||
"statement": "FileResultEntity result = fileResultMapper.selectById(member.getResultId());",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "shopdatacrawl"
|
||||
},
|
||||
{
|
||||
"file": "shopdatacrawl\\service\\ShopDataCrawlTaskService.java",
|
||||
"line": 2575,
|
||||
"statement": "FileTaskEntity task = fileTaskMapper.selectById(member.getTaskId());",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "shopdatacrawl"
|
||||
},
|
||||
{
|
||||
"file": "shopmatch\\service\\ShopMatchTaskService.java",
|
||||
"line": 179,
|
||||
"statement": "tasks.addAll(fileTaskMapper.selectList(new LambdaQueryWrapper<FileTaskEntity>()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "shopmatch"
|
||||
},
|
||||
{
|
||||
"file": "similarasin\\service\\SimilarAsinTaskService.java",
|
||||
"line": 1499,
|
||||
"statement": "page = mapper.selectList(new com.baomidou.mybatisplus.core.conditions.query.QueryWrapper<TaskChunkEntity>()",
|
||||
"loop_type": "while",
|
||||
"impact": "LOW",
|
||||
"module": "similarasin"
|
||||
},
|
||||
{
|
||||
"file": "similarasin\\service\\SimilarAsinTaskService.java",
|
||||
"line": 1646,
|
||||
"statement": "TaskChunkEntity chunk = taskChunkMapper.selectOne(new LambdaQueryWrapper<TaskChunkEntity>()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "similarasin"
|
||||
},
|
||||
{
|
||||
"file": "similarasin\\service\\SimilarAsinTaskService.java",
|
||||
"line": 2346,
|
||||
"statement": "FileTaskEntity task = fileTaskMapper.selectById(taskId);",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "similarasin"
|
||||
},
|
||||
{
|
||||
"file": "similarasin\\service\\SimilarAsinTaskService.java",
|
||||
"line": 2356,
|
||||
"statement": "List<TaskChunkEntity> chunks = taskChunkMapper.selectList(new LambdaQueryWrapper<TaskChunkEntity>()",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "similarasin"
|
||||
},
|
||||
{
|
||||
"file": "similarasin\\service\\SimilarAsinTaskService.java",
|
||||
"line": 2379,
|
||||
"statement": "TaskScopeStateEntity state = taskScopeStateMapper.selectById(stateId);",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "similarasin"
|
||||
},
|
||||
{
|
||||
"file": "similarasin\\service\\SimilarAsinTaskService.java",
|
||||
"line": 4364,
|
||||
"statement": "List<TaskChunkEntity> page = taskChunkMapper.selectList(query);",
|
||||
"loop_type": "while",
|
||||
"impact": "LOW",
|
||||
"module": "similarasin"
|
||||
},
|
||||
{
|
||||
"file": "similarasin\\service\\SimilarAsinTaskService.java",
|
||||
"line": 4437,
|
||||
"statement": "List<TaskScopeStateEntity> page = taskScopeStateMapper.selectList(query);",
|
||||
"loop_type": "while",
|
||||
"impact": "LOW",
|
||||
"module": "similarasin"
|
||||
},
|
||||
{
|
||||
"file": "task\\service\\ModuleHistoryCleanupService.java",
|
||||
"line": 131,
|
||||
"statement": "List<FileTaskEntity> page = fileTaskMapper.selectList(new LambdaQueryWrapper<FileTaskEntity>()",
|
||||
"loop_type": "while",
|
||||
"impact": "LOW",
|
||||
"module": "task"
|
||||
},
|
||||
{
|
||||
"file": "task\\service\\TaskFileJobService.java",
|
||||
"line": 153,
|
||||
"statement": "TaskFileJobEntity claim = taskFileJobMapper.selectById(candidate.getId());",
|
||||
"loop_type": "for",
|
||||
"impact": "LOW",
|
||||
"module": "task"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,313 @@
|
||||
"""task-121 N+1 扫描审计工具。
|
||||
|
||||
对 backend-java modules 下的 Java Service 源码做机械化扫描：
|
||||
识别循环（for/while/forEach/map）内对 MyBatis-Plus 单行查询
|
||||
（selectById/selectOne/selectList/selectCount/selectObjs/selectMaps/getById/getOne）
|
||||
的调用，产出 N+1 候选清单（位置/对象/影响），供人工审计与 task-122 修复。
|
||||
本工具只读，不改任何代码。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
HIGH = "HIGH"
|
||||
MEDIUM = "MEDIUM"
|
||||
LOW = "LOW"
|
||||
IMPACT_ORDER = {HIGH: 0, MEDIUM: 1, LOW: 2}
|
||||
|
||||
MODULES_ROOT = (
|
||||
Path(__file__).resolve().parent.parent / "src/main/java/com/nanri/aiimage/modules"
|
||||
)
|
||||
|
||||
# 单行查询调用（selectBatchIds/selectByIds 天然批量，不匹配此正则）
|
||||
QUERY_RE = re.compile(
|
||||
r"\.(selectById|selectOne|selectList|selectCount|selectObjs|selectMaps|getById|getOne)\("
|
||||
)
|
||||
# 循环头：for/while 关键字 + 圆括号跨度，或 .forEach( / .map( 方法调用
|
||||
LOOP_RE = re.compile(r"\b(for|while)\s*\(|\.(forEach|map)\s*\(")
|
||||
# 返回集合的查询调用（循环遍历这类结果集 → 影响 HIGH）
|
||||
COLLECTION_QUERY_RE = re.compile(
|
||||
r"(selectList|selectObjs|selectMaps|selectBatchIds|selectByIds|\.list\()"
|
||||
)
|
||||
ASSIGN_RE = re.compile(r"(\w+)\s*=\s*([^;{}=]+);")
|
||||
FIELD_RE = re.compile(r"\b[A-Za-z_]\w*\b")
|
||||
|
||||
|
||||
def _strip_comments(text: str) -> str:
|
||||
"""用空格替换注释（保留换行与偏移），简化后续扫描。"""
|
||||
out = list(text)
|
||||
i, n = 0, len(text)
|
||||
while i < n:
|
||||
if text.startswith("/*", i):
|
||||
j = text.find("*/", i + 2)
|
||||
j = n if j < 0 else j + 2
|
||||
for k in range(i, j):
|
||||
if text[k] != "\n":
|
||||
out[k] = " "
|
||||
i = j
|
||||
elif text.startswith("//", i):
|
||||
j = text.find("\n", i)
|
||||
j = n if j < 0 else j
|
||||
for k in range(i, j):
|
||||
out[k] = " "
|
||||
i = j
|
||||
else:
|
||||
i += 1
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def _match_span(text: str, open_pos: int) -> int | None:
|
||||
"""从 text[open_pos] 的 ( 出发，返回匹配的 ) 偏移（含），不匹配返回 None。"""
|
||||
depth = 0
|
||||
for i in range(open_pos, len(text)):
|
||||
c = text[i]
|
||||
if c == "(":
|
||||
depth += 1
|
||||
elif c == ")":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return i
|
||||
return None
|
||||
|
||||
|
||||
def _line_of(text: str, offset: int) -> int:
|
||||
return text.count("\n", 0, offset) + 1
|
||||
|
||||
|
||||
class _Frame:
|
||||
__slots__ = ("body_depth", "expr", "loop_type", "start")
|
||||
|
||||
def __init__(self, start: int, body_depth: int, loop_type: str, expr: str):
|
||||
self.start = start
|
||||
self.body_depth = body_depth
|
||||
self.loop_type = loop_type
|
||||
self.expr = expr
|
||||
|
||||
|
||||
def _query_results(stripped: str) -> set[str]:
|
||||
"""收集被赋值给集合查询结果（selectList 等）的变量名。"""
|
||||
names = set()
|
||||
for lhs, rhs in ASSIGN_RE.findall(stripped):
|
||||
if COLLECTION_QUERY_RE.search(rhs):
|
||||
names.add(lhs)
|
||||
return names
|
||||
|
||||
|
||||
def _loop_headers(stripped: str) -> list[dict]:
|
||||
"""预取全部循环头：(start, end, type, expr)（start 为 ( 偏移，end 为匹配的 ) 偏移）。"""
|
||||
headers: list[dict] = []
|
||||
pos = 0
|
||||
while True:
|
||||
m = LOOP_RE.search(stripped, pos)
|
||||
if not m:
|
||||
break
|
||||
paren_pos = m.end() - 1
|
||||
if stripped[paren_pos] == "(":
|
||||
close = _match_span(stripped, paren_pos)
|
||||
if close is not None:
|
||||
headers.append(
|
||||
{
|
||||
"start": paren_pos,
|
||||
"end": close,
|
||||
"type": m.group(1) or m.group(2),
|
||||
"expr": stripped[paren_pos + 1 : close].strip(),
|
||||
}
|
||||
)
|
||||
pos = m.end()
|
||||
return headers
|
||||
|
||||
|
||||
def _header_context(stripped: str, h: dict, offset: int) -> tuple[bool, str, str]:
|
||||
"""查询落在循环头括号跨度内时的归属判定。
|
||||
|
||||
返回 (是否循环内, 循环类型, 循环表达式)：
|
||||
- lambda 箭头之后 → 循环内（无花括号 lambda 体）
|
||||
- 增强 for 的 ` : ` 之后（迭代源 selectList 等）→ 集合装载，非循环内
|
||||
- for/while 初始化或条件 → 视为循环内（条件逐次求值）
|
||||
"""
|
||||
expr = h["expr"]
|
||||
if "->" in stripped[h["start"] : h["end"]]:
|
||||
arrow = stripped.find("->", h["start"], h["end"])
|
||||
if arrow != -1 and offset > arrow:
|
||||
return True, h["type"], expr
|
||||
return False, h["type"], expr
|
||||
if " : " in expr:
|
||||
colon = stripped.find(" : ", h["start"], h["end"])
|
||||
if colon != -1 and offset > colon:
|
||||
return False, h["type"], expr
|
||||
return True, h["type"], expr
|
||||
return True, h["type"], expr
|
||||
|
||||
|
||||
def scan_text(text: str, filename: str = "fixture.java") -> list[dict]:
|
||||
"""扫描单份 Java 源码，返回 N+1 候选清单（按影响降序、文件行号升序）。"""
|
||||
stripped = _strip_comments(text)
|
||||
query_results = _query_results(stripped)
|
||||
headers = _loop_headers(stripped)
|
||||
|
||||
depth = 0
|
||||
frames: list[_Frame] = []
|
||||
single: tuple[str, str, int, int] | None = None # (loop_type, expr, start, end)
|
||||
findings: list[dict] = []
|
||||
|
||||
def in_loop(offset: int) -> bool:
|
||||
if single is not None and single[2] <= offset < single[3]:
|
||||
return True
|
||||
if any(f.start <= offset and depth >= f.body_depth for f in frames):
|
||||
return True
|
||||
for h in reversed(headers):
|
||||
if h["start"] <= offset <= h["end"]:
|
||||
return _header_context(stripped, h, offset)[0]
|
||||
return False
|
||||
|
||||
def enclosing(offset: int) -> tuple[str, str]:
|
||||
if single is not None and single[2] <= offset < single[3]:
|
||||
return single[0], single[1]
|
||||
for h in reversed(headers):
|
||||
if h["start"] <= offset <= h["end"]:
|
||||
in_loop, loop_type, expr = _header_context(stripped, h, offset)
|
||||
if in_loop:
|
||||
return loop_type, expr
|
||||
for f in reversed(frames):
|
||||
if f.start <= offset:
|
||||
return f.loop_type, f.expr
|
||||
return "loop", ""
|
||||
|
||||
i = 0
|
||||
n = len(stripped)
|
||||
while i < n:
|
||||
c = stripped[i]
|
||||
if c == "{":
|
||||
depth += 1
|
||||
i += 1
|
||||
continue
|
||||
if c == "}":
|
||||
depth -= 1
|
||||
if frames and depth == frames[-1].body_depth - 1:
|
||||
frames.pop()
|
||||
i += 1
|
||||
continue
|
||||
|
||||
loop_match = LOOP_RE.match(stripped, i)
|
||||
if loop_match:
|
||||
loop_type = loop_match.group(1) or loop_match.group(2)
|
||||
paren_pos = loop_match.end() - 1
|
||||
close = (
|
||||
_match_span(stripped, paren_pos) if stripped[paren_pos] == "(" else None
|
||||
)
|
||||
if close is not None:
|
||||
expr = stripped[paren_pos + 1 : close].strip()
|
||||
body_start = close + 1
|
||||
while body_start < n and stripped[body_start].isspace():
|
||||
body_start += 1
|
||||
if body_start < n and stripped[body_start] == "{":
|
||||
frames.append(_Frame(body_start + 1, depth + 1, loop_type, expr))
|
||||
else:
|
||||
end = stripped.find(";", body_start)
|
||||
single = (loop_type, expr, body_start, n if end < 0 else end)
|
||||
# 继续扫描括号跨度内容（无花括号 lambda 体的查询落在其中）
|
||||
i = paren_pos + 1
|
||||
continue
|
||||
i = loop_match.end()
|
||||
continue
|
||||
|
||||
query_match = QUERY_RE.match(stripped, i)
|
||||
if query_match:
|
||||
if in_loop(i):
|
||||
line_start = stripped.rfind("\n", 0, i) + 1
|
||||
line_end = stripped.find("\n", i)
|
||||
line_end = n if line_end < 0 else line_end
|
||||
statement = stripped[line_start:line_end].strip()
|
||||
loop_type, expr = enclosing(i)
|
||||
impact = _classify(loop_type, [expr], query_results)
|
||||
findings.append(
|
||||
{
|
||||
"file": filename,
|
||||
"line": _line_of(text, i),
|
||||
"statement": statement,
|
||||
"loop_type": loop_type,
|
||||
"impact": impact,
|
||||
}
|
||||
)
|
||||
i = query_match.end()
|
||||
continue
|
||||
|
||||
i += 1
|
||||
|
||||
findings.sort(key=lambda f: (IMPACT_ORDER[f["impact"]], f["line"]))
|
||||
return findings
|
||||
|
||||
|
||||
def _classify(loop_type: str, exprs: list[str], query_results: set[str]) -> str:
|
||||
"""影响分级：遍历查询结果集 → HIGH；forEach/map lambda → MEDIUM；其余 → LOW。"""
|
||||
for expr in exprs:
|
||||
if expr and COLLECTION_QUERY_RE.search(expr):
|
||||
return HIGH
|
||||
if expr and any(t in query_results for t in FIELD_RE.findall(expr)):
|
||||
return HIGH
|
||||
if loop_type in ("forEach", "map"):
|
||||
return MEDIUM
|
||||
return LOW
|
||||
|
||||
|
||||
@dataclass
|
||||
class Report:
|
||||
scanned_files: int = 0
|
||||
modules: list[str] = field(default_factory=list)
|
||||
findings: list[dict] = field(default_factory=list)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"scanned_files": self.scanned_files,
|
||||
"modules": self.modules,
|
||||
"findings": self.findings,
|
||||
}
|
||||
|
||||
|
||||
def scan_modules(root: Path) -> Report:
|
||||
"""扫描 modules 根目录下全部 Java 文件，产出 N+1 候选报告。"""
|
||||
report = Report()
|
||||
module_set: set[str] = set()
|
||||
for java_file in sorted(root.rglob("*.java")):
|
||||
report.scanned_files += 1
|
||||
rel = java_file.relative_to(root)
|
||||
module_set.add(rel.parts[0])
|
||||
for finding in scan_text(java_file.read_text(encoding="utf-8"), str(rel)):
|
||||
finding["module"] = rel.parts[0]
|
||||
report.findings.append(finding)
|
||||
report.findings.sort(
|
||||
key=lambda f: (IMPACT_ORDER[f["impact"]], f["file"], f["line"])
|
||||
)
|
||||
report.modules = sorted(module_set)
|
||||
return report
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="N+1 扫描审计")
|
||||
parser.add_argument("--json", help="输出 JSON 报告路径（默认 stdout）")
|
||||
args = parser.parse_args()
|
||||
|
||||
report = scan_modules(MODULES_ROOT)
|
||||
payload = report.to_dict()
|
||||
if args.json:
|
||||
Path(args.json).write_text(
|
||||
json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8"
|
||||
)
|
||||
else:
|
||||
print(json.dumps(payload, ensure_ascii=False, indent=2))
|
||||
print(
|
||||
f"scanned {report.scanned_files} files, "
|
||||
f"{len(report.findings)} N+1 candidates "
|
||||
f"across {len(report.modules)} modules",
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,199 @@
|
||||
"""task-121 N+1 扫描审计测试。
|
||||
|
||||
对应 plan 06 任务 121 的 8 条用例 + 2 条补强：
|
||||
1. test_scan_select_in_loop 识别循环内单查
|
||||
2. test_scan_output_listed 清单产出（位置/对象/影响字段齐全）
|
||||
3. test_scan_priority 按影响排序
|
||||
4. test_scan_module_coverage 覆盖主要模块（真实代码扫描）
|
||||
5. test_scan_false_positive_checked 排除误报（已批量/循环外）
|
||||
6. test_scan_repeatable 可重复（同输入同输出）
|
||||
7. test_scan_documented 审计文档产出
|
||||
8. test_scan_no_edit 本任务不改生产代码
|
||||
9. test_scan_lambda_brace_less 无花括号 lambda 内单查识别
|
||||
10. test_scan_iterating_query_result 遍历查询结果集判 HIGH
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from n1_scan import (
|
||||
HIGH,
|
||||
LOW,
|
||||
MEDIUM,
|
||||
MODULES_ROOT,
|
||||
scan_modules,
|
||||
scan_text,
|
||||
)
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent
|
||||
REPO_ROOT = SCRIPTS_DIR.parent.parent
|
||||
DOC_PATH = REPO_ROOT / "backend-java" / "docs" / "n1-scan-audit.md"
|
||||
|
||||
FIXTURE_LOOP = """package com.example;
|
||||
public class DemoService {
|
||||
public void batchByIds(List<Long> ids) {
|
||||
for (Long id : ids) {
|
||||
FileTaskEntity task = fileTaskMapper.selectById(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_QUERY_RESULT_LOOP = """package com.example;
|
||||
public class DemoService {
|
||||
public List<Vo> listVos() {
|
||||
List<FileTaskEntity> tasks = fileTaskMapper.selectList(new LambdaQueryWrapper<>());
|
||||
List<Vo> vos = new ArrayList<>();
|
||||
for (FileTaskEntity t : tasks) {
|
||||
FileResultEntity r = fileResultMapper.selectById(t.getResultId());
|
||||
vos.add(toVo(r));
|
||||
}
|
||||
return vos;
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_LAMBDA_BRACELESS = """package com.example;
|
||||
public class DemoService {
|
||||
public void touch(List<Long> ids) {
|
||||
ids.forEach(id -> fileTaskMapper.selectOne(
|
||||
new LambdaQueryWrapper<FileTaskEntity>().eq(FileTaskEntity::getId, id)));
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_BATCH_SAFE = """package com.example;
|
||||
public class DemoService {
|
||||
public List<FileTaskEntity> load(List<Long> ids) {
|
||||
List<FileTaskEntity> tasks = new ArrayList<>();
|
||||
for (Long id : ids) {
|
||||
tasks.addAll(fileTaskMapper.selectBatchIds(Collections.singleton(id)));
|
||||
}
|
||||
for (Long id : ids) {
|
||||
tasks.addAll(fileTaskMapper.selectByIds(Collections.singleton(id)));
|
||||
}
|
||||
return tasks;
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_NO_LOOP = """package com.example;
|
||||
public class DemoService {
|
||||
public FileTaskEntity one(Long id) {
|
||||
return fileTaskMapper.selectById(id);
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_LOOP_NO_QUERY = """package com.example;
|
||||
public class DemoService {
|
||||
public int sum(List<Integer> nums) {
|
||||
int s = 0;
|
||||
for (Integer n : nums) {
|
||||
s += n;
|
||||
}
|
||||
return s;
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_INDEX_LOOP = """package com.example;
|
||||
public class DemoService {
|
||||
public List<Vo> listVos(List<Long> ids) {
|
||||
List<Vo> vos = new ArrayList<>();
|
||||
for (int i = 0; i < ids.size(); i++) {
|
||||
FileTaskEntity t = fileTaskMapper.selectById(ids.get(i));
|
||||
vos.add(toVo(t));
|
||||
}
|
||||
return vos;
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_MIXED = (
|
||||
FIXTURE_QUERY_RESULT_LOOP
|
||||
+ "\n"
|
||||
+ FIXTURE_LAMBDA_BRACELESS
|
||||
+ "\n"
|
||||
+ FIXTURE_INDEX_LOOP
|
||||
)
|
||||
|
||||
|
||||
class ScanDetectTest(unittest.TestCase):
|
||||
def test_scan_select_in_loop(self):
|
||||
findings = scan_text(FIXTURE_LOOP, "DemoService.java")
|
||||
self.assertEqual(len(findings), 1)
|
||||
self.assertIn("selectById", findings[0]["statement"])
|
||||
self.assertEqual(findings[0]["loop_type"], "for")
|
||||
self.assertEqual(findings[0]["line"], 5)
|
||||
|
||||
def test_scan_output_listed(self):
|
||||
for finding in scan_text(FIXTURE_MIXED, "DemoService.java"):
|
||||
for key in ("file", "line", "statement", "loop_type", "impact"):
|
||||
self.assertIn(key, finding)
|
||||
self.assertTrue(finding[key] is not None and finding[key] != "")
|
||||
|
||||
def test_scan_priority(self):
|
||||
findings = scan_text(FIXTURE_MIXED, "DemoService.java")
|
||||
order = {HIGH: 0, MEDIUM: 1, LOW: 2}
|
||||
impacts = [order[f["impact"]] for f in findings]
|
||||
self.assertEqual(impacts, sorted(impacts))
|
||||
|
||||
def test_scan_false_positive_checked(self):
|
||||
self.assertEqual(len(scan_text(FIXTURE_BATCH_SAFE, "Batch.java")), 0)
|
||||
self.assertEqual(len(scan_text(FIXTURE_NO_LOOP, "NoLoop.java")), 0)
|
||||
self.assertEqual(len(scan_text(FIXTURE_LOOP_NO_QUERY, "NoQuery.java")), 0)
|
||||
|
||||
def test_scan_repeatable(self):
|
||||
first = scan_text(FIXTURE_MIXED, "DemoService.java")
|
||||
second = scan_text(FIXTURE_MIXED, "DemoService.java")
|
||||
self.assertEqual(
|
||||
json.dumps(first, sort_keys=True), json.dumps(second, sort_keys=True)
|
||||
)
|
||||
|
||||
def test_scan_lambda_brace_less(self):
|
||||
findings = scan_text(FIXTURE_LAMBDA_BRACELESS, "DemoService.java")
|
||||
self.assertEqual(len(findings), 1)
|
||||
self.assertEqual(findings[0]["loop_type"], "forEach")
|
||||
self.assertEqual(findings[0]["impact"], MEDIUM)
|
||||
|
||||
def test_scan_iterating_query_result(self):
|
||||
findings = scan_text(FIXTURE_QUERY_RESULT_LOOP, "DemoService.java")
|
||||
self.assertEqual(len(findings), 1)
|
||||
self.assertEqual(findings[0]["impact"], HIGH)
|
||||
|
||||
|
||||
class ScanCoverageTest(unittest.TestCase):
|
||||
def test_scan_module_coverage(self):
|
||||
report = scan_modules(MODULES_ROOT)
|
||||
self.assertGreaterEqual(report.scanned_files, 600)
|
||||
self.assertGreater(len(report.findings), 0)
|
||||
modules = {f["module"] for f in report.findings}
|
||||
self.assertGreaterEqual(len(modules), 3)
|
||||
order = {HIGH: 0, MEDIUM: 1, LOW: 2}
|
||||
impacts = [order[f["impact"]] for f in report.findings]
|
||||
self.assertEqual(impacts, sorted(impacts))
|
||||
|
||||
def test_scan_documented(self):
|
||||
self.assertTrue(DOC_PATH.is_file(), f"audit doc missing: {DOC_PATH}")
|
||||
text = DOC_PATH.read_text(encoding="utf-8")
|
||||
self.assertIn("N+1", text)
|
||||
self.assertIn("影响", text)
|
||||
|
||||
def test_scan_no_edit(self):
|
||||
changed = subprocess.run(
|
||||
["git", "diff", "--name-only", "HEAD", "--", "backend-java/src"],
|
||||
cwd=REPO_ROOT,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
).stdout.strip()
|
||||
self.assertEqual(changed, "", f"生产代码被改动: {changed}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user