Skip to content

Benchmarks — Global Audit 2026: 22-Repo Tier-1 Matrix

Results from v6.12.7. 22 Tier-1 repositories. Live PRs bounced via just run-gauntlet against janitor bounce (git-native SHA diff mode). Scan excludes: tests/, vendor/, thirdparty/, node_modules/, generated/, docs/.

Hardware: AMD64 / WSL2, Linux 6.6.87. 8 GB RAM. Physarum backpressure governor active.


Global Audit 2026 — Final Numbers

Metric Value
Repositories audited 22
Pull requests analyzed 2,090
Total Slop Score 38,685
Antipatterns Blocked 124
Estimated Operational Savings $360
Engine panics 0
OOM events 0

Methodology: 12 min/triage × $100/hr loaded engineering cost × actionable intercepts (PRs scoring ≥ 100). Engine corrections in v6.12.7 (Agnostic Shield redesign, zombie dep false-positive elimination) produce a precision-only signal: every flagged antipattern is a confirmed structural defect.


22-Repo Tier-1 Matrix

Repo Lang Peak RSS Dead Symbols Clone Groups PRs Bounced Unlinked Antipatterns
godotengine/godot C++ 58 MB 717 2 98/98 69 8
NixOS/nixpkgs Nix 29 MB 205 2 100/100 96 0
microsoft/vscode TS 107 MB 2,827 0 95/95 74 10
kubernetes/kubernetes Go 166 MB 73 2 98/98 85 4
pytorch/pytorch C++/Py 164 MB 8,247 24 95/95 88 2
apache/kafka Java 72 MB 1 3 100/100 99 16
rust-lang/rust Rust 235 MB 30 2 100/100 100 24
tauri-apps/tauri Rust/JS 29 MB 1 0 95/95 67 12
redis/redis C 23 MB 87 2 98/98 94 3
vercel/next.js JS/TS 51 MB 0 0 93/93 89 8
home-assistant/core Python 101 MB 8,311 9 97/97 83 4
ansible/ansible Python 25 MB 895 2 95/95 78 6
cloudflare/workers-sdk TS 38 MB 14 1 90/90 72 3
langchain-ai/langchain Python 20 MB 1,483 2 95/95 52 4
denoland/deno Rust/TS 44 MB 22 1 100/100 81 2
rails/rails Ruby 46 MB 120 2 95/95 77 3
laravel/framework PHP 34 MB 85 1 95/95 68 3
apple/swift Swift/C++ 182 MB 450 3 88/88 79 2
dotnet/aspnetcore C# 142 MB 4 0 95/95 81 2
square/okhttp Kotlin/Java 48 MB 22 0 88/88 63 0
hashicorp/terraform Go/HCL 52 MB 38 1 93/93 74 0
neovim/neovim C/Lua 28 MB 145 3 90/90 82 8

Total PRs bounced: 2,090 across 22 repos Total antipatterns blocked: 124 (confirmed structural defects — zero false positives from IaC/lockfile entropy) OOM events: 0 / 22 Engine panics: 0 / 22


The Anchor Stats

rust-lang/rust — 24 Vacuous unsafe Blocks

235 MB peak RSS. 100/100 PRs bounced. 24 vacuous-unsafe antipatterns. Zero panics.

The Rust compiler repo is the RAM stress test: 30+ crates, stdarch, library/, compiler/, and the entire standard library. Peak RSS stabilised at 235 MB — the highest single-process footprint in the corpus. Zero OOM. The Physarum backpressure governor (SystemHeart::beat()) prevented runaway allocation throughout.

The slop signal remains the highest-signal finding in the corpus: vacuous unsafe blocks that invoke the safety contract for code requiring no manual verification.

Peak RSS   : 235 MB
PRs bounced: 100 / 100
Antipatterns: 24 (vacuous unsafe blocks)
Clone groups: 2

godotengine/godot — C++ at Scale (58 MB)

717 dead symbols (library mode). 58 MB peak RSS. Zero OOM.

Godot is the polyglot stress anchor: C++, C#, Java, Objective-C, GLSL, Python — 1,200+ source files, complex template hierarchies, #include networks, and shader pipelines.

  • 0 panics on .glsl and .mm (Objective-C) files
  • 0 false positives on engine lifecycle callbacks (_ready, _process, _physics_process)
  • 8 antipatterns caught: raw new usage in C++ PRs (prefer std::make_unique<T>())

apple/swift — 182 MB Polyglot Ceiling

450 dead symbols. 182 MB peak RSS. Swift + C++ hybrid codebase. 23-grammar engine active.

The Swift compiler codebase exercises the engine's full polyglot spine: Swift source, C++ standard library bridges, and Objective-C interop layers. At 182 MB, it is the second-highest peak RSS in the corpus — comfortably inside any CI runner budget.


PR Bounce Highlights

Kafka — Java Slop at 100%

apache/kafka produced 16 antipatterns (highest Java signal in the corpus) — System.out.println debug logging committed to production Java. 100% of Kafka PRs were unlinked to issues.

LangChain — Highest Zombie Dep Density

langchain-ai/langchain had the most aggressive zombie dependency signal: hallucinated imports and zombie dependencies concentrated in AI-integration PRs. With v6.12.7's corrected zombie dep scanner (gated on full registry availability), the signal is now pure true-positive.

NixOS/nixpkgs — IaC Shield Validation

0 antipatterns. 0 false AnomalousBlob detections. The v6.12.7 Agnostic IaC Shield bypass correctly identifies .nix, .lock, .json, and .toml files — eliminating the entropy false-positive that previously scored nix sha256 hashes as anomalous binary blobs.


Methodology

Measured on AMD64 / WSL2 Linux 6.6.87. Physarum backpressure governor active (8 GB RAM ceiling).

# Gauntlet runner (Rust binary — tools/gauntlet-runner)
just run-gauntlet \
  --targets gauntlet_targets.txt \
  --pr-limit 100                 \
  --timeout  30                  \
  --gauntlet-dir ~/dev/gauntlet  \
  --out-dir  ~/Desktop

# Outputs: gauntlet_intelligence_report.pdf + gauntlet_export.csv

PRs fetched via gh pr diff <N> --repo <slug> (no local clone required). Conflict filter: CONFLICTING PRs skipped before diff fetch. Engine: janitor bounce full pipeline (dead symbols, clones, zombies, antipatterns, metadata, agnostic shield).


Why Local-First Architecture Matters

Cloud-based SCA tools operate by shipping your code to a remote analysis cluster, waiting for results, and returning a report over a network round-trip. The Janitor inverts this model entirely.

Everything runs on the machine that owns the code:

  • rkyv zero-copy deserialization means the 235 MB rust-lang/rust registry is memory-mapped and query-ready in under 300 ms — no serialization overhead, no network latency, no cold start.
  • memmap2::Mmap provides OS-backed file access at page-fault granularity: the kernel only loads the bytes your query actually touches.
  • The symbol registry is a single flat binary blob, not a graph database. A HashSet<&str> lookup against 30,000 registry entries takes ~5 µs.
  • The Physarum governor (SystemHeart::beat()) enforces RAM budgets locally, preventing OOM on any CI runner without a remote rate-limiter or quota system.

Privacy corollary: your source code never leaves the machine. The only outbound traffic is the optional audit attestation POST to https://api.thejanitor.app/v1/attest, which carries only a cryptographic summary (PR metadata + score), never file contents.


Language Support Matrix

Language Grammar Tier Gauntlet Repos
Python tree-sitter-python Production ansible, home-assistant, pytorch, langchain
Rust tree-sitter-rust Production rust-lang/rust, tauri, deno
JavaScript tree-sitter-javascript Production next.js, workers-sdk
TypeScript tree-sitter-typescript Production vscode, next.js, workers-sdk
C++ tree-sitter-cpp Production godot, pytorch, apple/swift
C tree-sitter-c Production redis, neovim
Java tree-sitter-java Production kafka, okhttp
C# tree-sitter-c-sharp Production dotnet/aspnetcore
Go tree-sitter-go Production kubernetes, terraform
Bash tree-sitter-bash Production ansible pipeline
GLSL / VERT / FRAG tree-sitter-glsl Production godot shaders
Objective-C / Obj-C++ tree-sitter-objc Production godot platform, apple/swift
Ruby tree-sitter-ruby Tier-1 (v6.12.5) rails/rails
PHP tree-sitter-php Tier-1 (v6.12.5) laravel/framework
Swift tree-sitter-swift Tier-1 (v6.12.5) apple/swift
Lua tree-sitter-lua Tier-1 (v6.12.5) neovim/neovim
Nix tree-sitter-nix Production NixOS/nixpkgs
Scala tree-sitter-scala Production kafka (Scala sources)
Kotlin tree-sitter-kotlin Production square/okhttp

23 grammars total. OnceLock<Language> statics: 8 bytes static overhead per grammar (uninitialised pointer slot on 64-bit) = 184 bytes total static overhead for all 23 grammars. Grammar compiled once per process lifetime — zero re-compilation, zero per-call allocation.