Every number here is reproducible. The relationships Synaptic extracts are validated against hand-labeled ground truth, and where the data is unflattering, it stays in.
tree-sitter parses 30+ languages into concrete syntax, and Synaptic resolves call, import, and reference edges from it, then clusters the graph into communities so god nodes, surprising connections, and import cycles surface on their own.
Every inferred relationship is tagged EXTRACTED, INFERRED, or AMBIGUOUS. A "0 dependents" answer carries a caveat when a symbol is reachable only through dynamic dispatch, so absence of evidence is never dressed up as proof.
A code-only corpus never makes a network call. The only feature that needs an API key is the optional semantic pass over docs and papers. Everything structural runs locally.
Querying a compact graph costs a fraction of feeding raw files to an LLM. Measured on Synaptic's own source: 199 Rust files, 56,408 lines, 510,966 cl100k tokens.
fewer tokens overall, and a query response stays small no matter how big the repo gets, so the ratio grows with the codebase.
| Question | Query response | Read the files | Fewer tokens |
|---|---|---|---|
| http request handling | 1,804 | 48,803 | 27× |
| session create / reap | 1,974 | 65,578 | 33× |
| query_graph subgraph | 2,011 | 53,759 | 27× |
| extraction walker | 1,977 | 70,443 | 36× |
| PR fetch / rank | 1,926 | 73,231 | 38× |
| incremental merge | 2,010 | 53,440 | 27× |
Exact cl100k_base counts via cargo run -p synaptic-server --example tokcount. The baseline is the unique source files the result's nodes live in, the conservative grep-then-read case. The graph.json index itself is large because it encodes every symbol and edge; you never load it into context, you query it and get back only the slice.
A corpus of mini-repos whose true call edges, test linkages, blast radii (including distractors that must not be flagged), and cross-language couplings are written out by hand. A preflight fails the run if any labeled symbol doesn't resolve, so a dropped node becomes a loud failure, not a quietly smaller denominator.
| Fixture | Family | Call P/R/F1 | Aff-test rec | Blast rec/excl/size | Cross P/R/F1 |
|---|---|---|---|---|---|
| systems-rust | systems-rust | 100/50/66 | – | 100% / 100% / 1.0 | – |
| scripting-python | scripting-python | 100/100/100 | 100% | 100% / 100% / 2.0 | – |
| web-ts | web-ts | 100/100/100 | – | 100% / 100% / 1.0 | – |
| oo-java | oo-java | 100/100/100 | – | 100% / 100% / 1.0 | – |
| systems-go | systems-go | 100/100/100 | – | 100% / 100% / 1.0 | – |
| deep-python | scripting-python | 100/100/100 | 100% | 100% / 100% / 3.0 | – |
| cross-lang-ts-rust | cross-lang | – | – | – | 100/100/100 |
Across 7 fixtures / 6 language families / 26 labeled symbols (all resolved): pooled call edges precision 100% / recall 93% / F1 96%; blast-radius recall 100% with 0 distractors leaked; affected-test recall 100% with the one labeled unrelated test correctly not selected; cross-language precision 100% / recall 100%.
Reading it honestly: the 50% recall on Rust is real and expected. Rust call resolution is intra-file, so a module-qualified cross-file call is a true miss. Cross-file reachability is still preserved through imports edges, which is why blast-radius recall stays 100%. The corpus is intentionally small and hand-verified: it validates extraction correctness on representative shapes, not internet-scale coverage.
Extraction throughput across real OSS repos, each cloned at a pinned SHA, median of 3 reps. Cold clears the AST cache; warm is cache-hot; incr re-extracts a single file. Measured on Windows / x86_64 / 16 logical CPUs.
| Repo | Tier | Files | Nodes | Edges | Cold (s) | Warm (s) | Incr (s) |
|---|---|---|---|---|---|---|---|
| memchr | small | 75 | 3,849 | 13,592 | 12.5 | 7.5 | 4.3 |
| click | medium | 112 | 2,189 | 3,475 | 2.4 | 1.7 | 0.8 |
| p-map | small | 10 | 85 | 83 | 0.07 | 0.04 | 0.04 |
| cobra | medium | 55 | 846 | 2,362 | 1.1 | 0.7 | 0.4 |
| axum | large | 348 | 3,656 | 9,510 | 4.7 | 3.6 | 3.5 |
Absolute times are machine-dependent; the reproducible signals are the cold→warm ratio (about 1.4 to 2×, the Rust AST cache removing re-parsing) and that throughput tracks repo content rather than collapsing on the large tier. memchr is slow per-file on purpose to show it: macro-heavy and edge-dense (13.6k edges over 75 files), which the benchmark surfaces rather than hides.
The change-forecast layer attaches a confidence to each predicted co-change. synaptic eval calibrate measures whether that confidence is meaningful: a reliability table, a Brier score, the Brier skill score against an always-guess-the-base-rate baseline, and expected calibration error.
On this repo's own squash-heavy, synthetic history the skill score is negative: co-change prediction there is worse than guessing the base rate, because squashed commits touch many files at once and inflate apparent co-change. That is the metric working: it refuses to dress up a predictor that is miscalibrated on this history. Confidence is a per-repo property, so run it on yours.
