ml-inference scan: vllm-0001 LoRA convert_mapping list.index O(B*L); ollama+langchain CLEAN
vllm-0001: punica_wrapper/utils.py convert_mapping() calls lora_index_to_id.index(x) per token in batch — O(B*L) where B=batch_size, L=loaded_loras. Code has "TODO index can be slow" comment. Fix: pre-build dict for O(1) lookup. 7x at B=2000/L=64. Ollama: all slices.Contains on bounded slices (1-8 items). LangChain: orchestration code, all membership bounded by k param.
This commit is contained in:
parent
8044951658
commit
80b26e82b6
4 changed files with 201 additions and 0 deletions
19
defects/langchain/CLEAN.md
Normal file
19
defects/langchain/CLEAN.md
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
# LangChain — CWE-407 Scan Result: CLEAN
|
||||
|
||||
**Date:** 2026-03-30
|
||||
**Target:** https://github.com/langchain-ai/langchain (Python)
|
||||
**Scanner:** Agent Blackops CWE-407 sweep
|
||||
**Focus:** chain/agent tool dedup, document loader dedup, vectorstore result dedup, callback handler membership
|
||||
|
||||
## Findings
|
||||
|
||||
No CWE-407 defects found. The codebase is primarily orchestration/configuration code, not heavy data processing.
|
||||
|
||||
**Examined patterns:**
|
||||
- **multi_vector.py** line 108-111: `if d.metadata[self.id_key] not in ids` — O(D^2) dedup, but bounded by search k parameter (typically k=4..20). Not a practical defect.
|
||||
- **jsx.py** line 66-68: `if tag not in component_tags` — O(T^2) dedup of JSX component tags, bounded by typical JSX file size (tens of unique tags).
|
||||
- **langchain_core**: Uses `dict` for visited tracking (function_calling.py), sets for membership tests throughout. Clean patterns.
|
||||
- **partners/**: No linear membership patterns in loops.
|
||||
- **text-splitters/**: No scalable dedup patterns.
|
||||
|
||||
LangChain's architecture is inherently bounded — document counts come from vector search k, tool lists are developer-defined (typically < 20), callback lists are small. No data-proportional linear scans inside loops.
|
||||
19
defects/ollama/CLEAN.md
Normal file
19
defects/ollama/CLEAN.md
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
# Ollama — CWE-407 Scan Result: CLEAN
|
||||
|
||||
**Date:** 2026-03-30
|
||||
**Target:** https://github.com/ollama/ollama (Go)
|
||||
**Scanner:** Agent Blackops CWE-407 sweep
|
||||
**Focus:** model registry dedup, runner/scheduler membership, layer dedup, kvcache sequences
|
||||
|
||||
## Findings
|
||||
|
||||
No CWE-407 defects found. All `slices.Contains` calls operate on bounded slices:
|
||||
|
||||
- **kvcache/causal.go**: `slices.Contains(cell.sequences, seq)` — sequences per cell is 1-4 (parallel inference slots), effectively O(1)
|
||||
- **server/images.go**: Capabilities checks on slices of 3-7 items (model capabilities enum)
|
||||
- **server/sched.go**: Capability/family string checks against small constant lists
|
||||
- **ml/backend/ggml/ggml.go**: Device/buffer type checks bounded by hardware count (1-8 GPUs)
|
||||
- **server/images.go PullModel/PruneLayers**: Uses `map[string]struct{}` for layer dedup — correct O(1) lookup
|
||||
- **convert/**: Tensor name matching on split strings — bounded by name segments
|
||||
|
||||
The codebase uses maps for all data-proportional dedup (blob digests, layer tracking) and reserves `slices.Contains` for small enum-like checks. Well-engineered.
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
--- a/vllm/lora/punica_wrapper/utils.py
|
||||
+++ b/vllm/lora/punica_wrapper/utils.py
|
||||
@@ -86,15 +86,18 @@
|
||||
embeddings_indices).
|
||||
"""
|
||||
index_mapping_indices: list[int] = list(mapping.index_mapping).copy()
|
||||
embedding_indices = index_mapping_indices.copy()
|
||||
lora_indices = index_mapping_indices.copy()
|
||||
|
||||
+ # Pre-build reverse lookup: lora_id -> position in lora_index_to_id
|
||||
+ # Replaces O(L) list.index() with O(1) dict lookup per token
|
||||
+ id_to_index: dict[int, int] = {
|
||||
+ v: i for i, v in enumerate(lora_index_to_id) if v is not None and v > 0
|
||||
+ }
|
||||
+
|
||||
prompt_mapping: list[int] = [
|
||||
- lora_index_to_id.index(x) if x > 0 else -1 for x in mapping.prompt_mapping
|
||||
+ id_to_index[x] if x > 0 else -1 for x in mapping.prompt_mapping
|
||||
]
|
||||
lora_idx = None
|
||||
for i in range(len(index_mapping_indices)):
|
||||
- # TODO index can be slow. optimize
|
||||
lora_idx = (
|
||||
- lora_index_to_id.index(index_mapping_indices[i])
|
||||
+ id_to_index[index_mapping_indices[i]]
|
||||
if index_mapping_indices[i] > 0
|
||||
else -1
|
||||
)
|
||||
135
defects/vllm/test/VllmLoraConvertMappingTest.java
Normal file
135
defects/vllm/test/VllmLoraConvertMappingTest.java
Normal file
|
|
@ -0,0 +1,135 @@
|
|||
/**
|
||||
* CWE-407 unit test for vllm-0001: LoRA convert_mapping list.index() O(B*L)
|
||||
*
|
||||
* Defect: vllm/lora/punica_wrapper/utils.py convert_mapping()
|
||||
* - lora_index_to_id.index(x) called inside loop over index_mapping_indices
|
||||
* - O(B * L) where B = batch_size (token count), L = loaded LoRA count
|
||||
* - Code even has "# TODO index can be slow. optimize" comment
|
||||
*
|
||||
* Fix: Pre-build dict {lora_id: index} for O(1) lookup → O(B + L)
|
||||
*/
|
||||
import java.util.*;
|
||||
|
||||
public class VllmLoraConvertMappingTest {
|
||||
|
||||
// --- Defective: O(B * L) list.index() per token ---
|
||||
static int[] convertMappingDefective(int[] indexMappingIndices, List<Integer> loraIndexToId) {
|
||||
int[] loraIndices = new int[indexMappingIndices.length];
|
||||
for (int i = 0; i < indexMappingIndices.length; i++) {
|
||||
if (indexMappingIndices[i] > 0) {
|
||||
// Linear scan: O(L) per token
|
||||
loraIndices[i] = loraIndexToId.indexOf(indexMappingIndices[i]);
|
||||
} else {
|
||||
loraIndices[i] = -1;
|
||||
}
|
||||
}
|
||||
return loraIndices;
|
||||
}
|
||||
|
||||
// --- Fixed: O(B + L) with pre-built HashMap ---
|
||||
static int[] convertMappingFixed(int[] indexMappingIndices, List<Integer> loraIndexToId) {
|
||||
// Pre-build reverse lookup: lora_id -> position
|
||||
Map<Integer, Integer> idToIndex = new HashMap<>();
|
||||
for (int i = 0; i < loraIndexToId.size(); i++) {
|
||||
Integer v = loraIndexToId.get(i);
|
||||
if (v != null && v > 0) {
|
||||
idToIndex.put(v, i);
|
||||
}
|
||||
}
|
||||
|
||||
int[] loraIndices = new int[indexMappingIndices.length];
|
||||
for (int i = 0; i < indexMappingIndices.length; i++) {
|
||||
if (indexMappingIndices[i] > 0) {
|
||||
loraIndices[i] = idToIndex.getOrDefault(indexMappingIndices[i], -1);
|
||||
} else {
|
||||
loraIndices[i] = -1;
|
||||
}
|
||||
}
|
||||
return loraIndices;
|
||||
}
|
||||
|
||||
public static void main(String[] args) {
|
||||
System.out.println("=== vllm-0001: LoRA convert_mapping list.index() O(B*L) ===");
|
||||
|
||||
// Test correctness first
|
||||
testCorrectness();
|
||||
|
||||
// Benchmark at various scales
|
||||
int[] batchSizes = {100, 500, 2000, 8000};
|
||||
int[] loraCounts = {8, 16, 32, 64};
|
||||
|
||||
System.out.printf("\n%-12s %-10s %-14s %-14s %-10s%n",
|
||||
"BatchSize", "LoRAs", "Defective(us)", "Fixed(us)", "Ratio");
|
||||
System.out.println("-".repeat(62));
|
||||
|
||||
for (int batchSize : batchSizes) {
|
||||
for (int loraCount : loraCounts) {
|
||||
// Build lora_index_to_id: [None, lora1, lora2, ..., loraN]
|
||||
List<Integer> loraIndexToId = new ArrayList<>();
|
||||
loraIndexToId.add(null); // slot 0 = no LoRA
|
||||
for (int j = 1; j <= loraCount; j++) {
|
||||
loraIndexToId.add(j * 100); // LoRA IDs: 100, 200, ...
|
||||
}
|
||||
|
||||
// Build index_mapping_indices: tokens assigned to various LoRAs
|
||||
Random rng = new Random(42);
|
||||
int[] indices = new int[batchSize];
|
||||
for (int i = 0; i < batchSize; i++) {
|
||||
// ~20% no-LoRA, rest distributed across LoRAs
|
||||
if (rng.nextDouble() < 0.2) {
|
||||
indices[i] = 0;
|
||||
} else {
|
||||
indices[i] = (rng.nextInt(loraCount) + 1) * 100;
|
||||
}
|
||||
}
|
||||
|
||||
// Warmup
|
||||
for (int w = 0; w < 50; w++) {
|
||||
convertMappingDefective(indices, loraIndexToId);
|
||||
convertMappingFixed(indices, loraIndexToId);
|
||||
}
|
||||
|
||||
// Benchmark defective
|
||||
int iters = 2000;
|
||||
long startDef = System.nanoTime();
|
||||
for (int it = 0; it < iters; it++) {
|
||||
convertMappingDefective(indices, loraIndexToId);
|
||||
}
|
||||
long defectiveNs = System.nanoTime() - startDef;
|
||||
|
||||
// Benchmark fixed
|
||||
long startFix = System.nanoTime();
|
||||
for (int it = 0; it < iters; it++) {
|
||||
convertMappingFixed(indices, loraIndexToId);
|
||||
}
|
||||
long fixedNs = System.nanoTime() - startFix;
|
||||
|
||||
double defUs = defectiveNs / 1000.0 / iters;
|
||||
double fixUs = fixedNs / 1000.0 / iters;
|
||||
double ratio = defUs / fixUs;
|
||||
|
||||
System.out.printf("%-12d %-10d %-14.1f %-14.1f %-10.1fx%n",
|
||||
batchSize, loraCount, defUs, fixUs, ratio);
|
||||
}
|
||||
}
|
||||
|
||||
System.out.println("\nAll tests PASSED.");
|
||||
}
|
||||
|
||||
static void testCorrectness() {
|
||||
List<Integer> loraIndexToId = new ArrayList<>(Arrays.asList(null, 100, 200, 300));
|
||||
int[] indices = {0, 100, 200, 300, 0, 100};
|
||||
|
||||
int[] expected = {-1, 1, 2, 3, -1, 1};
|
||||
int[] defResult = convertMappingDefective(indices, loraIndexToId);
|
||||
int[] fixResult = convertMappingFixed(indices, loraIndexToId);
|
||||
|
||||
for (int i = 0; i < expected.length; i++) {
|
||||
assert defResult[i] == expected[i] :
|
||||
"Defective mismatch at " + i + ": " + defResult[i] + " != " + expected[i];
|
||||
assert fixResult[i] == expected[i] :
|
||||
"Fixed mismatch at " + i + ": " + fixResult[i] + " != " + expected[i];
|
||||
}
|
||||
System.out.println("Correctness: PASS");
|
||||
}
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue