living document · updated by the quality loop

The boost roadmap

Every finding from the autonomous quality loop lands here — what shipped, what's mid-flight, and what's queued — scored by complexity, impact, and a little bit of wow.

See what's next → Back to the guide
Shipped merged & released In flight a PR is open Next triaged, starting soon Planned on the list
20Shipped
1Next up
54Planned
457Loop finds

Shipped

// merged to main & published
ShippedLicensing

The copyleft protected nothing and cost the one thing boost needs

Write-up · relicence-apache-2.md

Complexity M Impact High Wow ★★★★ GPL-3.0-only → Apache-2.0, while it was still a one-person decision
ShippedAgents · Parity

bmad on knew about a second host and wrote to one anyway

Write-up · bmad-hooks-all-hosts.md

Complexity S Impact Med Wow ★★★ the autopilot wrote hooks to Claude and nowhere else, long after boost knew a second host
ShippedAgents · Parity

A Gemini user got the heuristic fallback from every AI command

Write-up · ai-backend-fanout.md

Complexity S Impact High Wow ★★★★ the last surface that only spoke Claude — and the one a user can see
ShippedQuality · CLI

All 80 boost commands audited against a disposable HOME — four defects fixed

Write-up · cli-command-audit.md

Complexity M Impact Med Wow ★★★ audited all 80 COMMANDS entries against a disposable HOME; 4 real defects fixed with tests

Next up

// triaged findings, starting soon

Planned

// on the list
DeclinedTesting · Gap

Bring commands/ under mutation testing

The ~8,100-line command layer has zero mutation coverage: mutmut is scoped to core/ only. A blocking 80% floor was attempted and the baseline has now actually been measured — it does not clear the bar, and two of the three constraints recorded here earlier were wrong. What a sample run of commands/taps.py (the best-covered module in the package, 95.8% lines) against tests/unit/ + tests/functional/ reports through this repo's own gate: 570/793 killed — 71.9%, under the 80% floor, with all 223 survivors spread across every one of the module's 7 functions rather than concentrated in one testable gap. Mutant density is 4.2 per statement in taps.py but 2.46 across core/ (9,909 mutants over 4,033 statements), so commands/'s 5,314 statements imply somewhere between 13,000 and 22,300 mutants; at the measured 1.42 mutations/sec — roughly 6.5× slower per mutant than the core/ job, which matches the functional-vs-unit suite cost — that is 2.5 to 4.5 hours on one runner, against an ~18-minute job today. The job sets no timeout-minutes, so it inherits GitHub's 360-minute cap rather than failing fast. Two blockers sit underneath that number. Selecting only tests/unit/, the way core/ does, leaves commands/ at 17.9% line coverage versus 91.5% with functional included — and "no tests" mutants count against the score, so that route floors out near 18%. Selecting tests/functional/ instead crashes the run: mutmut's record_trampoline_hit calls p.resolve(strict=True) on the relative source path, so any test that chdirs into a temp project dies with FileNotFoundError: <tmp>/boost_cli — confirmed on test_verify_sees_a_project_skill. Corrections to the earlier note: pytest_add_cli_args_test_selection is not single-valued, it is a list that configparser splits on newlines (a space-separated line is what errors), and a sibling pytest_add_cli_args takes extra pytest flags. One prerequisite is already fixed: --no-mcp leaked through os.environ and made the suite order-dependent, which mutmut exposes because it runs whichever test subset covers each mutant. Declined 2026-07-29. Not "hard" — blocked, and the block is upstream. mutmut 3.6.0 is still the latest release, and src/mutmut/__main__.py:120 still reads source_paths = [p.resolve(strict=True) for p in Config.get().source_paths] — unconditionally, before the max_stack_depth guard — on paths that configuration.py:102 builds as plain relative Paths. Every chdir-ing functional test therefore kills the run, and the functional suite is precisely what takes commands/ from 17.9% to 91.5% line coverage. So the two candidate configurations are "floors out near 18%" and "crashes"; there is no third. Even granting a fix, the measured numbers already refuse the proposal on their own terms: 71.9% against an 80% blocking floor, with survivors spread evenly rather than pooled in one testable gap, at 2.5–4.5 hours per run against an ~18-minute job. A gate that is 8 points red on the day it lands does not gate anything; it just makes main red. The lead, if anyone reopens this: the crash is a relative-path bug, and source_paths is not required to be relative — an absolute path survives resolve(strict=True) from any working directory. Whether the rest of mutmut's mutants/ copy machinery tolerates one is untested. That, plus a non-blocking scheduled job that rotates one module per week, is the shape worth trying; a blocking 80% floor is not. What is not lost by declining. The architecture already puts behaviour in core/ — which is mutation-gated at 80% — and keeps commands/ as thin CLI glue. The uncovered layer is the one deliberately designed to hold the least logic, and it still carries 91.5% line coverage from the functional suite.

Complexity XL Impact High Wow ★★★ declined 2026-07-29 — 71.9% measured, and the run cannot complete at all
In flightQuality · Eval

a Tier 3 eval for tool-call behaviour, floored in both directions

boost's required gate floors four retrieval metrics — recall@k ≥ 0.78, hit@1 ≥ 0.40, MRR ≥ 0.52, nDCG@k ≥ 0.58 — over a 91-query golden set and a 10,152-entry corpus, every row pinned to a commit SHA. All of it measures what boost returns once it is asked. Nothing measures whether an agent asks. **The call itself is unmeasured**, and it is the step everything downstream depends on. The miss that exposed it. A Gemini CLI session was asked to "create a new, simplified app demonstrating RAG implementation in Python3 using langGraph, langChain, and langSmith" — a new project, an architecture decision and a dependency choice, which is three of the triggers boost_search's description names explicitly. It activated two already-installed skills, built the app, and never called boost. Asked why, it paraphrased boost's own lock-in trigger list back verbatim, so the text was read and was not persuasive. A gate that floors recall@k at 0.78 reported nothing, because retrieval was never invoked. Every claim in the MCP surface is argued, not measured. The triggers, the 10-15s stated cost, the skip list, the three-kind framing, the "already covered is not already checked" defeater — each survived a careful review and none has a number behind it. That is a high-variance lever tuned blind: Tool Preferences in Agentic LLMs are Unreliable (EMNLP 2025, arxiv 2505.18135) measures description-only edits swinging call rate by more than 10×. boost currently ships those edits on reasoning alone. The design constraint that decides whether this is worth building: floor both directions. A tier that measures call rate alone rewards making boost maximally assertive, which is precisely the capture the surface is written to avoid — and boost has already learned this exact lesson one tier down. Flooring recall alone was a hole rather than a simplification: a ranker that finds the right answer every time and never ranks it first scores recall@10 1.000 with hit@1 0.000, and passed. So the prompt set needs two halves — a should-call set (multi-file work, a new subsystem, a config or CI job that outlives the session) and a should-not-call set drawn from the shipped skip list (a question, a one-line edit, a command the user just handed over) — with a false-call ceiling as binding as the call-rate floor. One number without the other is an incentive to ship the thing boost refuses to be. Per host, never averaged. The two registered hosts do not see the same boost text. Claude Code puts server instructions in the system prompt; Gemini CLI never delivers them in interactive mode at all — Config.initialize() does not await mcpInitializationPromise, so getMcpInstructions() returns "", startChat stamps the context entry once with a stable id, and the later refreshMcpContext() re-renders Tier 1 only. A single averaged score would hide a host where 1,786 characters of guidance are simply absent, and would credit or blame wording for a delivery failure. Shape. An opt-in make eval-tools beside eval-ai / eval-rec / eval-explain — real hosts and real LLM calls, so it is non-deterministic and key-gated and must not join the required check gate; same degrade-cleanly contract as the other Tier 2 evals. Because the outcome is stochastic, report N runs per prompt with an interval rather than a single pass/fail, the way golden-set-statistical-power established for retrieval — a one-shot replay cannot tell a wording regression from a sampling wobble. Where this stands (2026-08-31), and a correction. The Claude Code arm shipped in #616: scripts/eval_tools.py, a 16-prompt set halved into should-call and should-NOT-call, Wilson intervals over N runs, and a verdict that floors call rate and ceilings false calls. Its probe was broken, and the finding this card recorded was an artifact of it. An earlier revision of this card reported “3/3 false calls — boost's tools fired on What is the difference between a Python list and a tuple?”. They did not fire. called_boost() substring-scanned the raw event stream, and claude -p --output-format stream-json --verbose opens with a system/init event enumerating every tool available to the session — which on any machine where boost is registered contains mcp__boost__boost_search and the other three CONSULT names. So the check returned true on every run, including runs with no tool call at all. Measured directly: Say OK and nothing else. produced zero tool_use blocks and scored as a boost consult. Two consequences shipped with it. make eval-tools could never pass — eight no-call rows × three runs is 24 forced trues, so the false-call rate's lower bound sat at 1.00 against a 0.20 ceiling, red on every machine forever. And the tier built to retire unfalsifiable claims had produced one. The lesson is the tier's own: the existing tests passed because they fed hand-written one-line fragments with no init event — a fixture the author invented could not catch the author's wrong model of the input. The probe now parses the NDJSON and counts only tool_use blocks inside assistant events, and the regression test drives a captured real stream. The second host arm is still unwritten, and should stay that way until the fixed probe is re-run. Building arm two on a probe that cannot tell an offer from a call would produce two hosts scoring an identical, meaningless 1.00. When it is built, the candidate is Gemini CLI proper, not Antigravity CLI: the delivery claim below is about Gemini's Node bundle, and agy is a third mode again — it receives boost's instructions and writes them to ~/.gemini/antigravity-cli/mcp/boost/instructions.md, pointing the agent at the file rather than inlining it. Substituting it would measure a different mechanism than the one this card argues about. Cost, now measured. Two trivial runs on a real host reported $0.657 and $0.682 of total_cost_usd, so 16 prompts × 3 runs is roughly $30–50 per host per invocation on a machine with a crowded tool surface. --strict-mcp-config with a boost-only config cuts that sharply and controls the surface confound in the same move. 2026-08-31: --strict-mcp-config shipped. eval_tools.py now takes a --strict-mcp-config flag: it writes a boost-only mcpServers config (the same <launcher> mcp --stdio invocation and fork-safety env core.mcphost.register_argv uses for a real registration — confirmed against an actual claude mcp add-json write, not guessed at the schema) to a temp file and passes --strict-mcp-config --mcp-config <path> to every claude -p call, cleaning the file up afterward. This session's sandbox had no network path to PyPI, so the pinned toolchain (pytest, ruff, mypy, …) could not be installed and make check could not be run here; the change was verified by hand instead — direct python3.12 import of the module, the new unit tests executed by eye against the interpreter, py_compile, a manual line-length check against ruff's 88-column default, and an end-to-end dry run with subprocess.run mocked that confirms the flags land on the argv and the temp file is created and removed. CI runs the real gate on the PR. Still unwritten: the second host arm (Gemini CLI). No gemini CLI was reachable in this sandbox to capture a real stream from, and building that arm on an invented model of Gemini's non-interactive output format is the exact mistake this card's own probe fix (2026-08-30) already paid for once — "a fixture the author invented cannot catch the author's wrong model of the input." That arm stays a placeholder until it can be built against a captured real stream, on a machine with the gemini CLI installed. What it unlocks. The first honest answer to "did that description edit help", a baseline the next surface change can regress against, and a way to retire claims that survive only because nobody can check them.

Complexity L Impact High Wow ★★★★ probe fixed; --strict-mcp-config shipped for the surface confound; second host (Gemini CLI) still unwritten
PlannedTech-debt

unpin the [eval] langchain stack when ragas ships its fix

The [eval] extra pins langchain-core<0.4, langchain-community<0.4 and langchain-openai<1 because ragas hard-imports ChatVertexAI from a langchain_community chat-models path that 0.4.x deleted. The LangChain integration card originally made this unpin its phase 0 and was corrected in place: ragas 0.4.3 still carries the import (measured 2026-08-04 — declared bounds are open, but import ragas crashes beside langchain 1.x), while upstream main already has the removal merged. So the unpin is one release of someone else's package away. What to do when it lands. Check pip index versions ragas (or the PyPI JSON) for a release after 0.4.3; verify in a throwaway venv that import ragas succeeds beside langchain>=1; then move [eval] to that floor, delete the three langchain pins, and adapt scripts/eval_explain.py if the 0.4 scoring API moved (its evaluate/to_pandas surface is what test_eval_faithfulness.py stubs in the unit suite). The eval-explain workflow is the live proof — it must stay green with real keys. Re-checked 2026-08-30. pip index versions ragas still reports 0.4.3 as the newest release, so nothing has changed and this card is still not claimable. Recorded here rather than left implicit: a card that says "check before starting" gives a reader no way to tell a check that came back negative from a check nobody ran. Why it stays its own card. The shipped integration card documents the block but will not be re-read; an unpin nobody remembers is how a workaround pin outlives its reason by years. This card is the reminder, and it is deliberately not claimable until the upstream release exists.

Complexity S Impact Low Wow ★ still blocked — re-checked 2026-08-30, PyPI's newest ragas is still 0.4.3
DeclinedRelease

give boost-langchain a release path to PyPI

Declined, deliberately. Both missing pieces below were owner-only or upstream-blocked, and the research they prompted dissolved the premise: a second PyPI project bought a separate release cadence nobody needed (boost releases more often than langchain), while the ecosystem evidence — langchain-community sunset, non-langchain-* names in LangChain's own integrations listing, in-host precedents from ragatouille to mlflow — showed the standalone distribution was never required. The integration now ships inside the boost-skill-cli wheel behind a [langchain] extra instead; see langchain-in-the-wheel. The original card follows for the record. The boost-langchain distribution shipped under integrations/langchain/ with its whole point being a separate release cadence from boost-skill-cli — langchain majors move faster than boost does, and the conformance workflow already builds the sdist/wheel and runs twine check on every touching PR. What does not exist is any way for those artifacts to reach PyPI: the name 404s there, and nothing publishes on any trigger. Two pieces, one of which only the repo owner can do. First, create the PyPI project and configure a Trusted Publisher for it — pending-publisher registration works before the first upload, and the filename-matching rule that pinned boost's own workflow name applies here too. Second, a publish workflow with a deliberate trigger: not boost's every-merge cadence (publish.yml releases boost-skill-cli on every push to main, which is exactly the coupling the separate distribution exists to avoid) — a tag like boost-langchain-v0.1.0 or a manual dispatch that bumps the static version, builds from integrations/langchain/, and publishes with the OIDC token. Remember the repo's own lesson: a release:-triggered workflow can never fire here (GITHUB_TOKEN events do not chain), so trigger on the tag push or dispatch directly. The floor is already honest. The package requires boost-skill-cli>=1.0.320 — measured against the actual API it calls, verified by an adversarial install — so the first published version works against PyPI as it stands today.

Complexity S Impact Med Wow ★★ declined — superseded by shipping the integration inside the boost-skill-cli wheel (see langchain-in-the-wheel)
In flightSearch · Performance

publish the keyword index the way vectors are published

Dense vectors are built once in CI and downloaded. The BM25 index is not: core/rag.py has no export or import function at all, and shards.yml / scripts/publish_shards.py are dense-only end to end. Every install rebuilds the same index from the same registries, at the same pinned commits, to produce the same bytes. Measured, on a real 458-tap machine. The on-disk index is rag_index.json 43.7 MB plus rag_postings.sqlite 653.0 MB — 696.7 MB for 18,619,658 postings. Build cost, timed over a 9,306-entry / 69-tap slice: 4.54 s reading bodies and tokenizing, 3.83 s writing postings, 8.4 s total — about 0.9 ms per entry, so roughly 65 s and ~900 MB extrapolated to the full 71,700-entry catalogue. Which user actually pays it. Not the default one: boost quickstart taps the 7 starter registries and indexes them in about a second. The cost lands on boost quickstart --catalog — 463 registries, 2 min 10 s of parallel cloning and then a minute of indexing on top — and on anyone who taps their way there gradually. The bug that makes this worth doing is not speed. boost catalog --import already exists and already looks like the answer: shareable-catalogue-bundle advertises 10.9 MB replacing a 12 GB clone and "59,972 searchable items in 4 seconds". That 4 seconds is fast for a reason the card does not state. rag.read_body degrades silently to name + description when the item's clone is absent (rag.py: "Missing files degrade to just the catalog metadata"), and a bundle import restores catalogues with zero repositories cloned. So the index it builds is not the full-content index the evals gate floors — it is a frontmatter index wearing the same file name. Measured directly over 3,015 real entries, indexing them with and then without their clones: 3,041,326 tokens versus 182,507. A bundle-only index carries 6.0% of the searchable text, and nothing in the output says so. That is the same failure shape as an unpinned eval corpus — a number that still renders confidently while measuring something else. Why this is easier than the dense shards, not harder. BM25 looks like it needs global statistics, and it does — but none of them are frozen at build time. _bm25 derives n = len(docs) and df = len(plist) on every query, so IDF is computed from whatever corpus is loaded. A per-registry shard therefore merges by offsetting doc_id, unioning the postings, and recomputing avg_len from per-shard totals — arithmetic, not re-derivation. And unlike vectors there is no embedding space to match and no API key to hold, so shards.incompatible() has no analogue here: a published keyword index is importable by everyone, including the keyless user who cannot use vectors at all. Shape. rag.export_shard / rag.import_shard mirroring dense's pair, per-registry assets on the existing shards-latest release, rows carried in the same manifest.json with the same commit pin and sha256 — the carry-forward machinery in publish_shards.py manifest --carry-forward applies unchanged, because a registry whose commit did not move has an index that did not change either. Three invariants transfer verbatim from the dense side and each is load-bearing: verify before replacing, refuse a shard whose commit is not the tap's commit, and never treat a missing digest as a match. The open question is payload size, and it is large enough to be its own decision — see shrink-the-published-index. This card should not ship until that one has an answer, because publishing 697 MB per refresh to save 65 s of CPU is not obviously the right trade, and at the compressed sizes measured there it clearly is. Partly landed, and deliberately still inflight — 2026-09-10. What shipped is the half that needed no size decision: the index now records what it is. read_body_full returns the text and whether it contains the item's body, build() reports metadata_only over every document written (reused ones included, or an incremental build reports zero on the run after a bundle import), index_completeness() reads the share back off disk, and boost reindex says it out loud instead of reporting the same confident count for a 6% index. The share is of tokens, not documents: a bodyless entry still produces a document, so a document share sits at 1.0 until it drops to 0.0. INDEX_VERSION moved to 9, because the flag is written only when a body is missing and absence may only be read as "complete" once no older document can survive. What did NOT land: rag.export_shard / rag.import_shard, the per-registry assets, and the manifest.json rows — the publishing pipeline itself. That half is what the payload-size question governs, and shrink-the-published-index still has no answer: its claim is stale, not active — branch loop/shrink-postings-index was last touched 2026-09-02, carries one commit, has no pull request, and is 394 commits behind main. Someone should un-claim it. One structural finding for whoever takes it: doc ids are positional (_save does enumerate(docs)), so the card's "merge by offsetting doc_id" is sound as written — and the shard format should serialize logical postings (digest → term → tf) rather than the SQLite layout, so the interning that branch was attempting cannot invalidate a published shard.

Complexity L Impact High Wow ★★★★ every machine rebuilds 697 MB of index for a corpus that is byte-identical on all of them
In flightPerformance · Storage

shrink the keyword index before publishing it — structure first, then compression

publish-the-keyword-index is worth doing only if the artifact is small enough to ship weekly. This card is the measurement that decides it, and the first answer is that compression is the second lever, not the first. What the format actually stores. _write_postings creates postings (term TEXT, doc INTEGER, tf INTEGER) and inserts one row per posting, so the term string is repeated in every row. Measured on the real 458-tap store: 18,619,658 rows over 210,422 distinct terms averaging 6.6 characters. That is ~123 MB of term text to carry 1.4 MB of distinct term text — 88× redundancy, before the per-row and B-tree overhead that turns it into a 653 MB file (page_size 4096, 167,174 pages, freelist 0, so it is not slack space). Compression measured on that file, as it stands: rag_index.json 43.7 MB raw · gzip -6 10.6 MB (4.12×).
rag_postings.sqlite 653.0 MB raw · gzip -6 201.7 MB (3.23×) · zstd -3 169.9 MB (3.84×) · zstd -19 106.7 MB (6.11×). So even with no format change, zstd -19 puts the whole index near 117 MB — under half the ~300 MB of dense vectors already published weekly. The trade is already good; the point of this card is that it can be much better, and that the two levers compose. Structure first, and it is the bigger win. Interning terms into terms(id, term) with postings(term_id, doc, tf) removes ~123 MB of duplicated strings and shrinks the postings_term index from a text key to an integer one. Beyond that, the classic inverted-index encodings apply directly because doc ids within a term are ascending: delta-encode them, varint or bitpack the deltas, and store one blob per term rather than one row per posting. Both shrink the file on disk, not just in transit, which is the half a compressed download never gives back — the user still ends up with 653 MB resident after import. What must not regress. read_postings exists precisely so a query touches a handful of terms instead of materialising the whole map — the change that took cold search from 8-13 s and multiple GB resident to 31-70 ms of scoring. A blob-per-term layout keeps that property (one row read per query term, decoded on the spot); a scheme that requires decoding neighbouring terms to find one does not. _bm25 must stay byte-identical, as it did through the SQLite move, and TestBm25Math is what says so. Decompression cost is the thing to measure, not assume. zstd -19 is slow to compress and fast to decompress, which is the right asymmetry for a weekly build feeding many imports — but "fast" needs a number on the import path before it is a claim, next to the 0.12 s that importing dense rows costs today. A zstd dictionary trained across shards is the obvious follow-on for the many-small-registries case, where per-shard compression has little context to work with. Deliverable. A measured comparison — raw, interned, delta+varint, each × none/gzip/zstd — on the real store, with import-side decode time beside each. That table is what tells publish-the-keyword-index what to ship, and it is worth having even if publishing is declined: the on-disk win applies to every install today. Progress — PR 688, merged as f003fa03 in train 691. The structural half shipped: _write_postings now interns terms into their own terms(id, term, df) table, with postings carrying an integer term_id instead of repeating the term string on every row — exactly the "structure first" change this card calls the bigger win, and it bumps INDEX_VERSION so every store picks it up on its next rebuild. stem_expansions now reads the precomputed df column directly instead of a GROUP BY COUNT(*) over postings on every prefix lookup. Not done: the delta/varint doc-id encoding, and the full raw/interned/delta+varint × none/gzip/zstd comparison table with import-side decode times on a real multi-hundred-MB store — this sandbox has no such store to measure against, only a small synthetic one (interning alone cut a 1.8M-posting/20k-term synthetic store from 63.6 MB to 47.9 MB, directionally consistent with the real-store estimate above but not a substitute for it). Left as follow-on work before this card can be called shipped.

Complexity M Impact Med Wow ★★★ 653 MB of postings holds 1.4 MB of distinct terms — the term string is stored 88 times over
ShippedHealth · Robustness

A best-effort log handler prints a traceback over every command's output

Write-up · delay-true-defeats-the-log-suppress.md

Complexity S Impact Medium Wow ★★ handleError overridden on the file handler, scoped to emit (#637); doctor probes open() rather than mode bits (#638)
In flightSearch · Ranking

Near-identical copies survive content-hash dedup and take the whole result page

Content-hash dedup shipped and worked: rag.dedupe_by_content took duplicate result slots from 4.94 to 0.60 per query over a 77-tap corpus. That card closed naming one thing still open — near-identical rather than byte-identical clustering, where core/typosquat.py's confusion machinery would apply — and buried it under a shipped status where nobody would claim it. This card is that remainder, with a measurement that makes it look considerably worse than “refinement”. Observed on a real 466-tap install with hybrid RRF serving (658,131 chunks): for the query exa search, every one of the top ten rows is exa-search, and the descriptions are what give the shape away — one Japanese (Exa MCPによるウェブ、コード、企業調査), two Chinese (通过Exa MCP进行神经搜索), five English variants of Neural search via Exa MCP, plus Use Exa MCP for current web… and AI-powered web search…. All ten are ★ curated. The footer reads 51 matches · ranked by hybrid RRF (BM25 + dense). Every one of those passed dedup correctly. They are not byte-identical: they are the same skill in Japanese, in Chinese, and in five English phrasings across different registries. The body digest differs, so dedupe_by_content keeps them all — which is exactly the behaviour #366 proved must be preserved, since two entries sharing a name can be genuinely different rules. The shipped fix is not misbehaving. It simply does not reach this shape. What the 0.60 residual actually was. The prior card described its leftover as “entries sharing a name whose bodies genuinely differ, which must stay separate” — true as stated, and it reads as a rounding error. At 466 taps the same residual is a full result page. The gap between 0.60 and 10.0 is worth understanding before designing anything: the 77-tap measurement used 50 natural-language queries averaged, and an average hides the shape here. Duplicate pressure was already known to be a step function of which registries are tapped rather than how many; near-identical pressure looks like a step function of which query — harmless across a query set, total on any query that lands on a widely-mirrored skill. Re-measure per-query maxima, not means. The hard part is the safety proof, not the clustering. Content hashing was adoptable because one count settled it: of 14,153 distinct bodies, clusters spanning more than one name numbered zero, so collapsing could not merge two different skills. Near-identical clustering has no such free proof — any similarity threshold loose enough to merge a Japanese translation with its English original is loose enough to merge two genuinely different skills that share boilerplate. Establish the equivalent bound first (over a real corpus, at the chosen threshold, count clusters spanning more than one meaning) or the fix trades a visible problem for a silent one. Three things to get right. Translations are the motivating case and the hardest: they share almost no tokens with the original, so token-overlap similarity will not find them while an embedding will — and the vectors are already on disk, which makes this cheaper here than it would be anywhere else. Collapse before k, and at both the retrieve and retrieve_any seams, for the reason the shipped dedup already documents: fusion reintroduces copies either engine dropped, because the copies are distinct (tap, skill_md) keys and RRF has no reason to treat them as one. The existing quality prior carries over unchanged — rag.source_rank orders on the user's curated flag first and shipped confidence second, and choosing among near-identical copies is the same question as choosing among identical ones: where should the user install from. Not to be confused with #629, which deduplicated vector storage (one row per distinct embedding, 39.7% repeats reclaimed). That is a disk-size fix beneath the index and changes no ranking; this is about which rows reach the user's screen. What shipped, and what did not. rag.collapse_near_duplicate_hits is the same "keep the earliest rank slot, promote a better source" contract as dedupe_by_content, run over cosine similarity of the entries' first-chunk embeddings (dense.entry_vectors, an index probe through chunks_entry on a quantized store) instead of a body hash, at the retrieve_any seam before k is applied. It is covered by unit tests down to the arithmetic (_cosine's dimension- mismatch and zero-vector guards), the clustering contract (rank order, quality-prior promotion, limit-after-collapse), the dense.entry_vectors lookup against a real quantized sqlite-vec store, and the retrieve_any/boost search --collapse-near-duplicates wiring in both directions (on and off). It ships opt-in and off by default — retrieve_any(..., collapse_near_duplicates=True) or boost search --collapse-near-duplicates — rather than replacing dedupe_by_content's output on the default path. Two things this card asks for are still open, and both need a real embedding backend (a built dense index, over a real multi-tap corpus) that the environment this was implemented in cannot reach — no network path to an embeddings provider or to the local ONNX model download, confirmed rather than assumed: huggingface.co and pypi.org both refuse at the network policy layer. First, the safety proof this card itself demands before defaulting the mechanism on — “over a real corpus, at the chosen threshold, count clusters spanning more than one meaning” — has not been run; NEAR_DUPLICATE_THRESHOLD = 0.97 is a starting point, not a validated floor. Second, re-measuring the exa search case (and per-query maxima generally) against the fix needs that same corpus and index. Whoever runs that measurement should flip the CLI flag's default, fold the corpus count into this card's evidence, and only then consider this shipped. The bound has now been measured, and it says the acceptance test in this card is the wrong one. scripts/measure_near_duplicate_bound.py runs the count this card asks for against the pinned 20-repo eval corpus (10,152 entries, 104,271 chunks, BAAI/bge-small-en-v1.5 at 384-d). Those entries reduce to 5,714 distinct chunk-0 vectors — 44% of entries already share a chunk-0 embedding byte for byte — and at NEAR_DUPLICATE_THRESHOLD = 0.97, 162 pairs clear the threshold and 56 clusters span more than one name. Sweeping the threshold moves that number but never to zero: 0.96 → 91, 0.97 → 56, 0.98 → 28, 0.99 → 13, 0.995 → 8, 0.999 → 4. Four of those 56 are not the threshold's doing at all. They are clusters of a single vector shared by several names, so they cluster at any threshold, which is why the sweep bottoms out at 4 rather than 0. The largest is the same at every threshold and is worth naming: 28 differently-named agents from one tap (affaan-m/ECC — architect, code-reviewer, chief-of-staff, database-reviewer, e2e-runner, …) whose chunk 0 is the same Spanish preamble (No cambiar rol, persona ni identidad…) in every file. Chunk 0 is name + description + opening of body, and where a registry opens every file with identical boilerplate, the name does not move the vector enough to separate them. A floor exists that no threshold can reach under, so “count must be zero” was never achievable. Worse for the test: most of the other 52 are the feature working. Hand-classifying all 56 at 0.97, roughly two-thirds are genuinely one skill under two names — twelve are pure hyphen-versus-underscore renderings of one integration (zoho-mail / zoho_mail, google_maps / google-maps, anthropic_administrator / anthropic-administrator), and the rest are suffix variants of one document (tdd / tdd-guide, rust-review / rust-reviewer, testing-patterns / code-showcase-testing-patterns). Collapsing those is precisely what this card exists to do. A metric that counts them as violations would reject every threshold that works. The dangerous merges have a shape, and this card already named it. The ~20 clusters that are real false merges are dominated by near-miss brand names: coinmarketcal with coinmarketcap, bugbug with bugsnag, parsehub with parseur, linkhut with linkup, mx-technologies with mx-toolbox, salesforce-marketing-cloud with salesforce-service-cloud. These are distinct products whose descriptions are boilerplate around a swapped word. That is the core/typosquat.py confusion shape this card's opening paragraph pointed at, arrived at independently from the other end: the guard this needs is not a tighter cosine floor but a name-confusability veto — refuse to collapse two entries whose names are a confusable edit apart, however close their vectors sit. So the default stays off, for a better-supported reason than before. The measurement does not say 0.97 is too loose; it says similarity alone cannot separate tdd/tdd-guide (collapse) from coinmarketcal/coinmarketcap (never collapse), because both pairs sit in the same cosine band. Flipping the default needs the confusability veto first, and a re-count with it applied. And this bound is space-specific: it was measured in bge-small 384-d, while a keyed production install is voyage-4 at 1024-d. Cosine thresholds do not transfer between embedding spaces — rerun the script against each space before trusting a number in it.

Complexity M Impact High Wow ★★★★ shipped opt-in (#639); bound measured (#645) and it refutes the zero-clusters test -- default-on now waits on a name-confusability veto
PlannedQuality · Retrieval eval

The monthly corpus refresh rewrote the pins and the baseline but left every documented number stale — taps.txt now contradicts its own header, and nothing checks it

Measured. tests/eval/taps.txt contradicts itself inside one file: its header at line 25 states the corpus is "10,152 entries", while its own twenty data rows fifty lines below sum to 10,731 (awk '!/^#/ && NF {s+=$3} END {print s}'), and an independent materialisation at exactly those pinned SHAs measures 10,731 entries across 20 taps with sickn33 at 61.8% - verified at HEAD 5ec7ed75. Reproduce it. cd <repo>
git show --stat cbc0a58b
grep -n '10,152\|6,309\|62.1\|1,616\|0.852' tests/eval/taps.txt scripts/eval_corpus.py CLAUDE.md Makefile .github/workflows/eval-scale.yml tests/unit/test_eval_corpus.py
sed -n '48,70p' tests/eval/taps.txt # data rows: sickn33 ... 6634, ECC ... 1621
awk '!/^#/ && NF {s+=$3} END {print "pinned rows sum to", s}' tests/eval/taps.txt # -> 10731
.venv/bin/python -c "import json;d=json.load(open('tests/eval/baseline.json'));print(json.dumps(d['sets']['golden.jsonl@a0617183f8c9']['engines']['BM25 full-content'],indent=1))" What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Only one statement in the finding is false, and it is narrow: 1. "the same stale figures propagate to ... the Makefile" is WRONG. The Makefile contains no "10,152" (grep -c '10,152' Makefile -> 0) and Makefile:126 quotes a DIFFERENT, older stale set: "0.863 / 0.473 / 0.607 / 0.662", which matches neither the pre-refresh baseline (0.8516/0.4725/0.6048/0.6575) nor the post-refresh one (0.8407/0.4835/0.6065/0.6552). Makefile:126 is stale prose, but it went stale before this refresh, not because of it. (Makefile:132-133 is separately stale in the other direction: it says the corpus "tracks upstream HEAD rather than pinned commits", untrue since #410 pinned it.) The card must not claim the refresh caused the Makefile drift. 2. Under-count of call sites. The finding lists six files; there are at least four more quoting "10,152", one of which is ACTIVE rather than merely left behind: - scripts/build_scale_corpus.py:189 emits the literal string "# 10,152 entries and a real install carries ~71,655..." into tests/eval/taps-scale.txt on regeneration, so a build step re-writes the stale number. - scripts/build_scale_corpus.py:6, tests/eval/taps-scale.txt:11, tests/unit/test_scale_corpus.py:5. - Also scripts/eval_corpus.py:10 ("affaan-m/ECC alone is 1,616"), which the finding's line list omits. 3. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO. I materialised the corpus at the shipped pins in my own disposable HOME (network reachable, 20 blobless sparse clones) and scored it with the exact make eval invocation, so the 10,731 / 61.8% / 76.9% / 0.841-0.484-0.607-0.655 figures are my own measurements, not the finder's. I did not touch the shared read-only eval-home except to read its caches; that corpus is still at the PRE-refresh pins (sickn33 @ d43065e) and totals 10,152 with sickn33 at 6,309 - which is what let me confirm both sides of the move rather than only the new one. My runs wrote nothing into the repo (git status --short shows only a peer session's four modified boost_cli/tests files, none under tests/eval/ or scripts/). CORRECTED, not REFUTED, and the correction is small: sizes, shares, line numbers, the baseline match, the monthly cron and "no test covers it" all reproduced exactly. Only the Makefile attribution is wrong, plus an under-count of sites. Do not let CORRECTED read as doubt about the defect. DO NOT list roadmap item bodies as stale. eval-corpus-was-not-actually-pinned.md, eval-corpus-is-one-strangers-repo.md, eval-corpus-is-96x-smaller-than-a-real-install.md, BOOST-D12.md and the CLI-audit cards all quote 10,152 / 6,309 / 0.852, but they are dated records of a measurement that was true when written. The live surfaces are the ones that must be fixed. NOT A DUPLICATE, but cross-link it. Why it is worth doing. taps.txt's header is the primary explanation of what the corpus is and why the floors sit where they do; it now states a size, a concentration and a four-metric score that its own data rows contradict, and CLAUDE.md repeats them as the project's ground truth. The next person reasoning about the corpus (or the next agent re-deriving the floors) reads numbers that are one refresh out of date, and the drift widens every month because the job that causes it is scheduled and nothing fails when the prose and the rows disagree. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity S Impact Low Wow ★★ .github/workflows/eval-corpus-refresh.yml regenerates taps.txt's data rows and baseli…
PlannedOnboarding · Bug

With a corrupt config.json, doctor reports "no registries tapped" and verdicts "● ready to set up" exit 0 while search is dead; heal says "nothing to heal"

Measured. With a corrupt ~/.boost/config.json on a machine holding 1 tap clone (repos/verify-30-fix) and its catalog cache (cache/verify-30-fix.json), boost doctor --json returns {"issues": 0, "ok": true, "verdict": "ready to set up — tap a registry to make boost searchable"} with exit 0 and no corruption entry anywhere in its checks array, on the same machine where boost search brainstorm exits 1 with "no taps configured — nothing to search" — because the only notice of the corruption is emitted by config.load() before report.Report exists and is routed to stderr (config.py:205-210), so it can never be counted, never reach --json, and never reach boost doctor > health.log. Reproduce it. cd <repo> || exit 1
export HOME=$TMPDIR/audit-doctor-verify3; export BOOST_HOME=$HOME/.boost; mkdir -p "$HOME"
python3 tests/make_fixture.py $TMPDIR/audit-doctor-fix3 >/dev/null
./boost tap $TMPDIR/audit-doctor-fix3 >/dev/null 2>&1
BOOST_ASSUME_YES=1 ./boost install brainstorming >/dev/null 2>&1
printf '{"taps": [' > "$BOOST_HOME/config.json"
ls -1 "$BOOST_HOME/repos" | wc -l # 1 clone still on disk
./boost doctor; echo "DOCTOR_EXIT=$?" # "no registries tapped" ... "● ready to set up", 0
./boost heal; echo "HEAL_EXIT=$?" # "✓ nothing to heal", 0
./boost search brainstorm; echo "SEARCH_EXIT=$?" # "no taps configured", 1 What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The BEHAVIOUR is exactly as claimed. Every source citation in the finding is wrong. 1. FILE PATH. The finding says "quality.py" with no directory and implies core. The file is boost_cli/commands/quality.py. There is no boost_cli/core/quality.py (sed on that path errors: No such file or directory). 2. VERDICT BRANCH. Finding says quality.py:744-750. Actual: the block is lines 727-736, with if issues == 0 and not taps: on line 728 and rep.verdict(True, "ready to set up — tap a registry to make boost searchable") on 729-730. 3. ORPHANED-STORE-DIR CHECK. Finding says quality.py:646-651. Actual: 612-618 (orphans = [c.name for c in sorted(root.iterdir()) ...] at 613, bad("orphans", ...) at 617-618). 4. These line numbers were never right. I checked HEAD, HEAD~1, HEAD~2, HEAD~3, HEAD~5 and HEAD~10 (verdict line 728/728/723/723/720/720; orphan line 617/617/614/614/611/611) and the three other copies in the tree (.claude/worktrees/bmad-autopilot: 631/537; build/lib: -/510; mutants/: the block is absent). Nothing anywhere has 744 or 646, so this is not stale line numbers from a recent move — it is a miscount. 5. WORDING OF THE FINDING'S "printed above doctor's own heading". What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO — do not let a card overstate it: - I measured 1 clone + 1 catalog cache on disk against "0 taps synced". The finding's "a user who has been using boost for months" is extrapolation, not measured. State the disparity as "N clones on disk vs 0 taps reported" and cite N=1 as what was actually verified. - I did NOT test boost tap --defaults (needs network). The claim that following doctor's advice re-taps defaults over the real list is unverified; what IS verified is that any tap write quarantines the old file to config.json.corrupt first, so the list is recoverable. That mitigation is why Med holds rather than High. - Fixture-tap repro only; no claim about a starter/default registry set. THREE ROADMAP PRECEDENTS — none is a duplicate, all must be cited by the author: - docs/roadmap/items/audit-doctor-findings.md (shipped, PR #656) fixed THIS EXACT CLASS — an "!" line that wears the issue glyph but is never counted, under a "● healthy" exit-0 verdict — but only for the crash-report notice. The config-corrupt warning is a second instance that fix did not reach, and is harder because it originates outside doctor's own code. - docs/roadmap/items/audit-verify-drift-say-nothing-installed-exit-0-and-doctor-says-lo.md (shipped, PR #674) is the lock-file sibling and ships the FIX PATTERN to copy: lockfile.check() returning ok/missing/corrupt instead of the empty skeleton read() collapses to, plus store.has_content() to tell a genuinely empty install from one whose record vanished. Why it is worth doing. This is the worst wording doctor can choose for the state: a user who has been using boost for months is told they are a new user who has not set anything up yet, with a zero exit code, and the repair command agrees there is nothing wrong. Following the printed advice (boost tap --defaults) re-taps the defaults over their real registry list — recoverable only because a .corrupt sidecar happens to be written, which neither doctor nor heal mentions. Doctor already knows how to compare on-disk state against a record (the orphaned-store-dir check); the same comparison against repos/ would turn this into one accurate line. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ Doctor has no config-integrity check. When ~/.boost/config.json fails to parse, confi…
PlannedQuality · Retrieval eval

BOOST_NO_EMBED has no state in the reason ladder: doctor calls a deliberate kill switch a degraded fault (exit 1) and hands advice that is a measured no-op in both branches

Measured. With BOOST_NO_EMBED=1 and a 5-chunk voyage-4 store, dense.fix_hint tells the user "set the key it was built with: export VOYAGE_API_KEY=..."; exporting that key produces a byte-identical status (reason='no-key', degraded=True, ready=False) and the byte-identical hint — the kill switch is read in provider() before any key, so boost's own remedy is a measured no-op, on both boost doctor (exit 1) and boost search. Reproduce it. cd <repo>
export HOME=$TMPDIR/verify-dense-f3; export BOOST_HOME=$HOME/.boost; mkdir -p "$HOME"
unset VOYAGE_API_KEY OPENAI_API_KEY BOOST_NO_EMBED
python3 tests/make_fixture.py $TMPDIR/verify-dense-f3-fix >/dev/null
./boost tap $TMPDIR/verify-dense-f3-fix >/dev/null
# ./boost is system python3 (no sqlite_vec) -> reports no-backend. Use .venv below.
build () { # $1 provider $2 model $3 dim
.venv/bin/python - "$1" "$2" "$3" <<'PY'
import hashlib, sys; sys.path.insert(0, ".")
from boost_cli.core import catalog, dense, embed
P, M, DIM = sys.argv[1], sys.argv[2], int(sys.argv[3])
def fake(texts, input_type=None, timeout=60):
out = []
for t in texts:
h = hashlib.sha256(t.encode()).digest()
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Every card-worthy measurement is exact. Only citation ranges drift: 1. "eight lines away in the same module" (why_it_matters) is FALSE as written and is the one non-trivial correction. embed.fallback_note() is in boost_cli/core/embed.py:170-179; the inert _FIX table is in boost_cli/core/dense.py:520-534. Different files, ~340 lines and one module apart. Do not put "eight lines away" on the card — it misstates where the fix lives. 2. "dense.py:513-516 states the invariant" -> the quoted sentence spans dense.py:512-514 (512 is "# Why dense retrieval isn't serving...", 513 is "# Each names the ONE next action..."). 3. "_FIX (dense.py:522-533)" -> _FIX is dense.py:520-534; the entries span 521-533. The 8-key count is correct. 4. "tests/unit/test_dense_status.py:100-101" -> the comment and the assert st["degraded"] is False are at 101-102 (line 100 is assert st["model"] is None). 5. "embed.py:134" -> the guard if not enabled(): is 133 and return None is 134. Accurate enough as cited. Confirmed exact: reason/degraded/built_provider in both branches; both hint strings verbatim; doctor rc=1; the fallback_note string; 8 _FIX keys; test_dense_status.py:264 and its lack of a store; 432 roadmap items; 0 BOOST_NO_EMBED hits in roadmap items; embed.py:32 kill-switch doc line. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. 1. BLAST RADIUS IS TWO SURFACES, NOT ONE. The finding only measured boost doctor. I also measured boost search under the kill switch: it prints the same inert "set the key it was built with" line (both read the one dense.fix_hint table, by design — CLAUDE.md's "doctor and search cannot give contradictory advice" rule). The card should say both. 2. AN EXISTING CARD PROPOSES A FIX THAT WOULD MAKE THIS WORSE. docs/roadmap/items/audit-quickstart-findings.md line 18 ("The [rag] install hint has three different wordings") proposes: "have embed.fallback_note() and both quickstart paths call dense.fix_hint()". Executed as written that deletes the only correct kill-switch sentence in the codebase (embed.py:178). This card's fix must go the other direction — add a disabled reason to the ladder in status() and give _FIX that key — or the two cards collide. This is not a duplicate (that card is about zsh-unsafe unquoted pip install boost-skill-cli[rag], not the kill switch), but a card author must be told. 3. THE ONLY ESCAPE IS DELETING THE VECTORS. I renamed the store away and degraded flipped to False (reason stays 'no-key' but degraded = store_exists and reason is not None). So the exit code is not literally unfixable — but the only remedy is discarding every vector the user paid to embed, and no hint mentions it. "red forever" in why_it_matters should be phrased as "red until you delete the store". 4. PRECONDITION IS NARROW — do not overstate reach. You must have BUILT a dense store and THEN set the kill switch. Why it is worth doing. BOOST_NO_EMBED is documented as a hard kill switch (embed.py:32) and is what a user or a CI job sets to opt out deliberately — including anyone who hits the previous two findings and wants to stop paying for a broken embedder. Doing so turns boost doctor red forever, which breaks it as a CI gate, and the only remedies boost offers are provably inert: reinstalling an installed package, or exporting a key the kill switch never reads. The correct sentence already exists eight lines away in the same module. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CONFIRMED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ embed.enabled() short-circuits provider() to None (embed.py:134), so BOOST_NO_EMBED=1…
PlannedQuality · Retrieval eval

fix_hint's no-key guard has been unreachable since the day it was written; a missing API key now prescribes the full re-embed the guard exists to prevent

Measured. On a complete [rag] extra with a voyage-4-built vector store and no API key exported, dense.status() returns reason='provider-changed' (not no-key), so all three status-passing surfaces print rebuild it: \boost reindex --dense --force\` and boost doctor exits 1 — measured verbatim, including doctor's "live key is local; searches are using BM25" line — while the guard written to prevent exactly that (21f28223, #444) fires only under BOOST_NO_EMBED=1` or a partial install, both confirmed by direct probe. Reproduce it. cd <repo>
export HOME=$TMPDIR/verify-dense-f1; export BOOST_HOME=$HOME/.boost; mkdir -p "$HOME"
unset VOYAGE_API_KEY OPENAI_API_KEY BOOST_NO_EMBED
python3 tests/make_fixture.py $TMPDIR/verify-dense-f1-fix >/dev/null
./boost tap $TMPDIR/verify-dense-f1-fix >/dev/null
# NOTE: ./boost execs system python3, which has no sqlite_vec, so it always says
# "no-backend". Every dense command below MUST run under .venv.
.venv/bin/python - <<'PY'
import hashlib, sys; sys.path.insert(0, ".")
from boost_cli.core import catalog, dense, embed
DIM = 1024
def fake(texts, input_type=None, timeout=60): # no network, no API spend
out = []
for t in texts:
h = hashlib.sha256(t.encode()).digest()
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The defect is real and reproduced verbatim, but five stated details are wrong: 1. SOURCE ATTRIBUTION OF THE COST FIGURE. why_it_matters says "which CLAUDE.md prices at ~1.2 s per chunk on CPU". grep -n 'per chunk\|1\.2 s' CLAUDE.md returns nothing. The figure is real but lives in the roadmap: docs/roadmap/items/keyless-semantic-search-for-everyone.md:298 ("4,431 s — 74 minutes, about 1.2 s per chunk") and :308, plus the-shard-job-that-could-not-finish.md:17 and keyless-dense-tier-local-static-embeddings.md:75. Cite the roadmap, not CLAUDE.md. 2. quality.py:769 is wrong -> the fix_hint call is boost_cli/commands/quality.py:780 (fix = dense.fix_hint(st["reason"], st)). Line 769 sits inside the unrelated search-quantization warning. 3. pyproject.toml:55-57 is wrong -> the three pins are at lines 56, 67 and 68 (sqlite-vec>=0.1.6, onnxruntime>=1.17, tokenizers>=0.15), split by a 10-line comment justifying the fastembed rejection. They are in one extra, which is the load-bearing part, but the range is not 55-57. 4. THE CAUSAL-ORDER COMMIT PAIR NAMES THE WRONG COMMIT FOR THE GUARD. 86163e03 (2026-07-31, #364) created test_dense_fix_hint.py and the _FIX table, but not the guard. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. PRECONDITION — state it on the card so nobody reads it as unconditional. The misroute needs (a) a store built with an API provider, (b) the complete [rag] extra so provider() falls through to local, and (c) the key absent from the *process* environment. #444's own commit message documents that shape as recurring in the wild: "the shell exports one; the spawned server does not inherit it." THE REAL MACHINE IS NOT CURRENTLY IN THE BUG STATE. ~/.boost/cache/rag_vectors.sqlite reads provider="local", model="BAAI/bge-small-en-v1.5", dim=384 — so provider() matches built_provider and its reason is None. The 645,592-chunk / 1.20 GB / 440-tap figure is cost-if-the-hint-is-followed, not present harm. Do not let the card imply the machine is broken today. (The 750,416-chunk voyage-4 store in the #444 message and the test docstring is a past state of the same machine.) THE MISDIAGNOSIS WAS REFUTABLE FROM THE SAME FILE. The _FIX comment — "This reason means 'no key AND no local backend', which in practice is a partial install or BOOST_NO_EMBED" — was already in dense.py from 86163e03 (07-31) when #444 (08-03) added a guard whose docstring asserts the opposite: "an unfinished install with no store and a complete install whose key merely went missing both land here." Two claims about no-key, 30 lines apart, that contradict each other. Why it is worth doing. A user who opens a new shell without exporting their key is told by doctor, search and the MCP server to re-embed their whole store. On the real machine in this repo that is 645,592 chunks / 1.20 GB, which CLAUDE.md prices at ~1.2 s per chunk on CPU — hours of compute, or a real API bill — to fix a problem whose actual remedy is one export. The repo already decided this was unacceptable and wrote six tests plus a paragraph of docstring to prevent it; the guard has simply never been able to fire. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity L Impact High Wow ★★★★ dense.status() checks prov is None before it looks at the store (dense.py:618-621), b…
PlannedQuality · Retrieval eval

status() has no state for "ready but the embedder does not work": doctor green-ticks a tier that never ran, the search hint is suppressed, and every search re-pays the failed model fetch

Measured. In a store built as provider=local/BAAI/bge-small-en-v1.5/384-d with the weights absent, dense.ready() returns True and status() returns reason=None, degraded=False, while dense.retrieve() returns None on every query — so boost search prints "1 match · ranked by full-content BM25" with no hint (guard: if st.get("ready"): return, discovery.py:339) and boost doctor prints "✓ semantic search active — local BAAI/bge-small-en-v1.5 (384-d), 5 chunks across 1 tap" and exits 0 (guard: if st["ready"]:, quality.py:759); each such search makes exactly one un-cached 133,093,490-byte model fetch (counted in-process: 1 urlopen, 3.72 s, versus a 0.12 s BOOST_NO_EMBED baseline, of which only 0.06 s is the onnxruntime/tokenizers import). Reproduce it. cd <repo>
export HOME=$TMPDIR/verify-dense-f2; export BOOST_HOME=$HOME/.boost; mkdir -p "$HOME"
unset VOYAGE_API_KEY OPENAI_API_KEY BOOST_NO_EMBED
python3 tests/make_fixture.py $TMPDIR/verify-dense-f2-fix >/dev/null
./boost tap $TMPDIR/verify-dense-f2-fix >/dev/null
# ./boost is system python3 (no sqlite_vec) -> reports no-backend. Use .venv below.
# PRECONDITION: huggingface.co must be unreachable (blocked here by the proxy).
.venv/bin/python - <<'PY'
import hashlib, sys; sys.path.insert(0, ".")
from boost_cli.core import catalog, dense, embed
DIM = 384
def fake(texts, input_type=None, timeout=60):
out = []
for t in texts:
h = hashlib.sha256(t.encode()).digest()
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. "retries BOTH entries of FILES ... up to 1,200 s" is WRONG. ensure_model (localembed.py:141-150) does if not _fetch(...): return None on the FIRST failure, and FILES orders onnx/model.onnx before tokenizer.json, so tokenizer.json is never attempted on a failing network. Measured: exactly 1 nethttp.urlopen call per process, counted inside a real boost search. Correct statement: ONE fetch per search process, socket timeout 600 s. Do not write "up to 1,200 s"; also do not write "up to 600 s" as a wall-clock ceiling — timeout=600 is a per-socket-operation timeout, so a slow-drip server is not bounded by it. 2. Line numbers are off. _hint_semantic_search's guard is discovery.py:339-340 (if st.get("ready"): return), not 341-343. _report_search_engine's ready branch is quality.py:759, not 751-755. _load spans localembed.py:153-176 with the ensure_model() call at :164, not 152-166. (localembed.py:85 available() and :124 timeout=600 are correct, as are the 133,093,490 B and tokenizer.json byte counts.) 3. The BOOST_NO_EMBED=1 baseline is wrong: measured 0.12/0.12/0.12/0.13/0.14 s, not 1.11/0.80/1.50 s. Delta is 3.5-4.1 s. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO — read before writing a card. (a) The failing precondition here is a proxy that truncates the huggingface.co stream, not one that blocks it; the pin in FILES is correct (HF's content-length matches 133,093,490 exactly), so this is not a wrong-hash finding and _verified's length check is doing its job. (b) All timings are from this sandbox on a 5-entry fixture tap under a disposable HOME; only the structural facts (one fetch per process, no negative cache, both guards keyed on ready, no model probe on either surface) are environment-independent. (c) I could not test the positive case — with the model present dense works and none of this fires — so the finding is strictly about the store-present/model-absent state. PRIOR DISCLOSURE (the finder declared it, and I confirmed it): docs/roadmap/items/keyless-semantic-search-for-everyone.md, status shipped, body: "retrieval returned zero hits until the model was present — dense.status() reported ready the whole time, because the store genuinely was ready." That card frames it as a limit of what a shard can carry and commits to no work. Nothing in any item body claims doctor's green tick, the search hint's blindness, exit code 0, or the per-search retry cost; grepped all item bodies for localembed / ensure_model / bge-small / weights / "negative cache" / "model fetch" / "133 MB" and read the two candidate cards in full (the other being prerequisites-and-semantic-search-setup, the shipped hint whose blind spot this is). Why it is worth doing. This is the exact silent-BM25 failure prerequisites-and-semantic-search-setup shipped a hint to close, and the hint is blind to it: the user is told dense is active by doctor, told nothing by search, and pays several seconds per query for a download that can never succeed. A user who runs boost reindex --dense on a laptop with the model cached and then searches from a locked-down network or CI runner gets a slower search than if they had never enabled dense at all, with every surface reporting health. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ localembed.available() (localembed.py:85) tests whether onnxruntime and tokenizers *i…
PlannedUX · Bug

out.err's multi-line hint is coloured as one span, so line 1 ends with no RESET and lines 2+ carry no start code

Measured. With colour forced, a three-line hint from err() emits an opening ESC[2m on hint line 1 that is never terminated, two bare continuation lines carrying no SGR at all, and one closing ESC[0m at visible column 98 of the last line — so boost index 2>&1 | head -2 under CLICOLOR_FORCE hands the consumer a stream ending inside an unterminated dim span; output.py:252 wraps the newline-joined body in a single c(..., DIM), colouring first and splitting after. Reproduce it. cd <repo>
BOOST_COLOR=always .venv/bin/python -c "
from boost_cli.core import output as out
out.err('gh api failed', 'HTTP 401: Bad credentials\nTry authenticating with: gh auth login\ngh: To use GitHub CLI in a GitHub Actions workflow, set the GH_TOKEN environment variable.')
" 2>&1 | /bin/cat -v
# and the live call path (single-line tail in this sandbox, but proves err() is reached):
export HOME=$TMPDIR/audit-output-formatting; export BOOST_HOME=$HOME/.boost; mkdir -p "$HOME"
GH_TOKEN=deadbeefdeadbeefdeadbeefdeadbeefdeadbeef BOOST_COLOR=always ./boost index 2>&1 | head -3 | /bin/cat -v What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The core defect and its byte shape reproduce exactly. Five stated details are wrong: 1. "empty_state._block (output.py:559-561)" — WRONG LINES. 559-561 is docstring prose. _block is defined at output.py:569 and its per-line colouring is at 571-572. 2. "the lone ^[[0m is 90 columns into line 4" — WRONG. It sits at visible column 98. 90 is the length of the message text alone; the line also carries the 8-column hint: alignment indent. 3. "err is the one emitter that does not [colour per line]" — REFUTED. Handed a string containing a literal newline with wrap off, dim(), warn() and empty_state() all emit the identical straddling span (measured above). empty_state._block only colours per line of its *wrap* output; with wrap=False body = [text], so the newline sits inside one c(..., DIM). What is actually unique about err is that it is the only emitter whose docstring documents accepting embedded newlines and the only one with a live multi-line caller (gh_failure_hint); I grepped every out.warn( / out.dim( / out.empty_state( call site and none passes a \n-bearing string (all joins use ", " or "; "). 4. "boost index 2>&1 | head -2 hands the shell an opening SGR with no close" — WRONG without a stated precondition. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Not carded — I re-checked independently. Grepped all ^title: lines for colour/ANSI/reset/escape/SGR/dim/hint terms and read the four plausible cards in full: long-hints-overflow-narrow-panes.md (line width, not escape spans), BOOST-D04.md (semantic colour roles), BOOST-D18.md (empty-state/hint styling consistency), BOOST-D27.md (its three named remaining wrap-law gaps are dim()'s embedded indent, empty_state() adoption, and truncate()/search_layout() wide-char width — err() is not among them). grep -ril 'unterminated|RESET|SGR|escape code' docs/roadmap/items/ returned 11 files, none about err(). Scope of my repro, and what a card must not overclaim: - No corpus was needed; this is a pure library-call repro plus one live CLI path. The 20-tap eval corpus at $TMPDIR/eval-home is irrelevant here and I did not touch it. The ./boost index run used its own disposable HOME. - I could not produce the 3-line gh tail live either: the sandbox proxy's TLS failure is a single line, so the live run only proves err() is reached with a gh-sourced hint, not that a 3-line tail occurred. The multi-line shape is proven by the direct call, which is what the finder also did. - A card must state the forced-colour precondition. In an ordinary interactive run the bytes ARE emitted (stdout is a TTY), but every terminal I know carries DIM across a newline, so it renders correctly — the finding concedes this. The only consumer that sees breakage is a line-oriented reader under BOOST_COLOR=always / CLICOLOR_FORCE (CI). Why it is worth doing. Most terminals carry DIM across a newline, so it usually *looks* right — the harm is to anything reading a line at a time. boost index 2>&1 | head -2 hands the shell an opening SGR with no close, dimming everything printed afterwards; and because the hint goes to stderr, any stdout written between hint lines lands inside the unterminated dim run. The module already knows the correct shape and uses it twice — empty_state._block (output.py:559-561) and table's header cells (output.py:906-908, with the comment "a whole-line wrap would be cancelled at the first separator's RESET") both colour per line. err is the one emitter that does not. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity S Impact Low Wow ★★ err() joins a multi-line hint's lines and then wraps the whole joined string in one c…
PlannedQuality · Retrieval eval

The eval corpus's size and its concentration ceiling are counted with len(scan_dir) — the measure measure_registry.py exists to say is wrong — so 44.7% of the gate's corpus is vendored …

Measured. Two tools in this repo, run on the same clone at the same pinned SHA, give item counts 3.16x apart — scripts/measure_registry.py says est_items=2100 while scripts/eval_corpus.py --ensure records 6634 entries — and the tool that is right is the one the eval gate does not use: measure_registry.py's own docstring (lines 6-11) states that len(catalog.scan_dir(repo)) is not the measurement, "the same rule the eval gate's ranked list uses". eval_corpus.py:299 is still counts[repo] = len(entries), and MAX_SHARE = 0.65 (line 113) ratchets on that number inside CI's required lint job — so 976 extra vendored copies from one stranger's repository, containing zero new content, turn every open pull request red. Reproduce it. cd <repo>
export HOME=$TMPDIR/audit-corpus-verify && mkdir -p "$HOME" && export BOOST_HOME=$HOME/.boost
.venv/bin/python scripts/eval_corpus.py --ensure # -> "corpus: 10731 entries"; "concentration: ... 61.8%"
.venv/bin/python scripts/measure_registry.py "$BOOST_HOME/repos/sickn33__antigravity-awesome-skills" # -> est_items=2100
.venv/bin/python - <<'PY'
import json, pathlib, collections, os
home=pathlib.Path(os.environ["BOOST_HOME"]); ents=[]
for f in sorted((home/"cache").glob("*.json")):
if f.stem.startswith("rag_"): continue
d=json.loads(f.read_text())
for e in d["skills"]: e["_tap"]=d["tap"]; ents.append(e)
alld={e["content"] for e in ents}
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. MATERIALLY WRONG — "The vendoring is 60 plugins/agentic-bundle-*/ directories". There are 58 agentic-bundle-* dirs (60 plugins/* dirs total), and they hold only 449 of the 6,634 entries (6.8%; median 8 entries each). The 3.13x inflation is two FULL CATALOG MIRRORS: plugins/agentic-awesome-skills-claude (2,049 entries) and plugins/agentic-awesome-skills (2,027) = 4,076 entries, digest-identical to skills/ (2,107 entries, all distinct). This reframes the ceiling scenario: "one more bundle render... adds ~2,117 entries" is impossible — a bundle is ~8-10 entries. The correct unit is "one more per-agent mirror", ~2,049 entries -> 67.9%, still over 65%. Sharper still: only 976 extra vendored sickn33 entries are needed to cross the ceiling (< half a mirror). The conclusion survives; the unit does not — and the correct unit is exactly the pattern the est-items card already documents. 2. "Read all 318 titles from grep -h '^title:' docs/roadmap/items/*.md" — there are 432 items / 432 title lines today (verified identical to origin/main except one unrelated file). The not-carded sweep was run over a smaller set than exists. 3. "Every one of them... none measures distinct content" is wrong. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO — what I actually settled, and what I did not. - I materialised the corpus at the CURRENT taps.txt pins in my own disposable HOME and reproduced every headline number independently. The pre-refresh figures (10,152 / 5,790 / 62.1% / 34.5% / 3.16x) I took from the SHARED read-only eval-home at $TMPDIR/eval-home, which is still at the older pins — a read-only census, no writes. - The repo checkout was on a peer session's branch loop/missing-json, not main. I diffed every file this finding touches (scripts/eval_corpus.py, tests/eval/taps.txt, boost_cli/core/rag.py, .github/workflows/eval-corpus-refresh.yml, all of docs/roadmap/items/) against origin/main: identical except one unrelated roadmap item. The verification holds against main (origin/main af4bdbfd). - I could NOT isolate the causal mechanism of the 22 rank changes. The finding asserts "BM25 statistics (N and document frequency), not slot consumption". My data is CONSISTENT with that — dedupe_by_content does run over the full pool before k (rag.py:1053, confirmed), and recall@k is bit-identical while the rank-sensitive metrics move — but I did not separate idf from avgdl, nor rule out that a different byte-identical copy survives dedupe and carries a different grade_key. Write it as "consistent with", not "confirmed". WHY HIGH RATHER THAN MEDIUM. The metric movement alone cannot fail anything: hit@1 +0.011, MRR +0.010, nDCG +0.008 against 0.06-0.10 of headroom over the floors. The severity is the ratchet's teeth, which the finder never traced. Why it is worth doing. MAX_SHARE is the only shipped guard against the gate's corpus becoming one publisher's house style, and it is measured on a quantity a third party can inflate by ~3x without publishing a single new skill — so it can fire on a repo contributing a third of the content, and cannot fire on a repo that dominates the content without vendoring. The same raw count is the headline everywhere (10,152 entries / 62%) and is what the monthly refresh PR reports as growth, overstating real content growth 4x. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity L Impact High Wow ★★★★ scripts/eval_corpus.py counts a repo's contribution as len(entries) from a raw scan (…
PlannedQuality · Retrieval eval

The corpus's 65.6% duplication is 99.93% inside a single tap, so the required gate never once exercises the cross-tap trust ordering dedup exists for — 0 swaps in 264,735 comparisons

Measured. Over the 91 required golden queries on the current pins, 264,544 of 264,735 collapse comparisons inside rag.dedupe_by_content (99.93%) were between two copies in the SAME tap and the source-preference branch executed 0 times; and forcing it to fire — marking one tap curated produces 191 swaps — leaves recall@k / hit@1 / MRR / nDCG@k at 0.8407 / 0.4835 / 0.6065 / 0.6552 both before and after, with all 91 graded ranked-key lists identical. Reproduce it. cd <repo>
export HOME=$TMPDIR/audit-corpus-verify && export BOOST_HOME=$HOME/.boost # after eval_corpus.py --ensure (see other finding)
.venv/bin/python - <<'PY'
import sys, json
sys.path.insert(0,"."); sys.path.insert(0,"scripts")
from boost_cli.core import rag
from boost_cli.core.rag import source_rank
stats={"collapses":0,"same_tap":0,"diff_tap":0,"swaps":0,"ties":0}
def patched(hits, limit):
best={}; out=[]
for hit in hits:
d=hit.get("content")
if not d: out.append(hit); continue
s=best.get(d)
if s is None: best[d]=len(out); out.append(hit); continue
stats["collapses"]+=1; kept=out[s]
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. THE REMEDY CLAIM IS WRONG, and this is the correction that matters. The finding's why_it_matters says "adding a single mirror registry to taps.txt would be enough to make it visible." It would not. eval_retrieval.grade_key (scripts/eval_retrieval.py:197-209) keys every ranked slot on the content digest (body:<digest>), the exemplar class (cls:...), or the entry name — and a swap only ever replaces hit["entry"] INSIDE a content cluster, where the digest is identical by construction and, because catalog._content_digest hashes name + description + body, so is the name. Every return branch of grade_key is therefore invariant under a swap; the nohash:tap::skill_md branch at :209 is unreachable for a swapped hit because dedupe never collapses a hit with no digest. Proven, not argued: marking composio-community/awesome-codex-skills curated on the same pinned corpus fires 191 swaps (every cross-tap collapse becomes a swap — the composio copy arrives second in all 191) and leaves recall@k / hit@1 / MRR / nDCG@k at 0.840659 / 0.483516 / 0.606517 / 0.655233 before AND after, with all 91 graded ranked-key lists byte-identical. So a trust-ordering regression is invisible to the required gate BY CONSTRUCTION, regardless of corpus shape — the gap is that the harness never grades on source, not that the corpus lacks mirrors. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO — read this before re-verifying. The finder's numbers reproduce ONLY on the CURRENT pins (10,731 entries). The shared read-only corpus at $TMPDIR/eval-home is PRE-REFRESH: 10 of its 20 clones are not at the SHAs in tests/eval/taps.txt (anthropics/skills, NeoLabHQ, LessUp, affaan-m/ECC, first-fluke, langchain-ai, minio, OneWave-AI, sickn33, anthropics/claude-agent-sdk-python), and it holds 10,152 entries. There the same instrumentation gives 1,986 clusters / 6,348 entries (62.5%) / 240,646 collapses / 240,455 same_tap / 191 diff_tap / 0 swaps. I materialised the current pins into a private copy (eval_corpus.py --ensure, network fetch works despite a harmless failed to store: 100001 commit-graph warning) to get the finder's exact figures. Anyone re-checking against $TMPDIR/eval-home will get the smaller set and should not read that as refuting the finding — the qualitative result (5 cross-tap clusters, all high/high, 0 swaps, ~99.9% same-tap) is identical on both. DOC TRAP, not this finding's fault: tests/eval/taps.txt's own header prose says "10,152 entries" and "sickn33 … is 6,309", and CLAUDE.md repeats 10,152 — but the file's own per-repo rows sum to 10,731 with sickn33 at 6,634. The pins were moved in commit cbc0a58b ("test(eval): refresh the pinned retrieval corpus") and the header prose was not updated. A card author quoting corpus size must take the row sum, not the header. NOT ALREADY CARDED, but read the existing card first. Why it is worth doing. The required corpus contains essentially none of the duplicate shape that dominates a real install. A user's duplicates arrive as mirror registries republishing each other's skills across taps, which is what source_rank decides between and what determines where a user is told to install from; the gate's duplicates are one publisher re-vendoring itself into 60 plugin bundles, where every candidate has the same tap and the tie-break is a no-op. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity S Impact Low Wow ★★ rag.dedupe_by_content is documented against a real install where "registries mirror e…
PlannedQuality · Retrieval eval

The 2026-09-01 pin refresh moved the corpus and re-baselined it, but nothing re-derives the floors — CLAUDE.md's "~10% under measured" is now 7.2%–17.3%

Measured. The required gate's recall floor has 5.52 queries of headroom out of 91 ((0.8407 - 0.78) x 91), against the 6.55 that CLAUDE.md's published 0.852 implies — one golden query of margin the documentation says exists and does not, because commit cbc0a58b re-baselined the corpus to 10,731 entries and left CLAUDE.md, taps.txt and eval_corpus.py all stating 10,152 / 0.852. Reproduce it. cd <repo>
# 1. the refresh commit moved pins + baseline and nothing else
git show cbc0a58b --stat --format='%h %ad %s' --date=short
git show cbc0a58b -- tests/eval/baseline.json | grep -E '^[-+].*(recall|hit@1|MRR|nDCG)'
# 2. what taps.txt records NOW vs what the docs claim
awk '!/^#/ && NF>=3 {s+=$3; n++} END {print "rows:", n, " recorded total entries:", s}' tests/eval/taps.txt
grep -n '10,152\|0\.852' CLAUDE.md tests/eval/taps.txt scripts/eval_corpus.py
# 3. recompute the floor gaps against the committed post-refresh baseline
python3 -c "
import json
b=json.load(open('tests/eval/baseline.json'))['sets']['golden.jsonl@a0617183f8c9']['engines']['BM25 full-content']
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. CAUSAL FRAMING IS WRONG (the headline correction). The title and claim say the 2026-09-01 pin refresh is what made "~10% under" untrue. It was untrue the day it was written. git log -S'~10% under' traces the sentence to 170d52c0 (2026-07-31), whose own commit message says the floors were "re-derived against the 20-tap numbers at the same ~10% relative headroom" against measured 0.863 / 0.473 / 0.607 / 0.662 — at which point the gaps were 9.6% / 15.4% / 14.3% / 12.4%, a 1.60x spread. The refresh WIDENED a spread (1.60x -> 1.82x -> 2.39x); it did not create one. Correct statement: "the floors were never uniformly ~10% under, and the refresh widened the spread from 1.82x to 2.39x." 2. "not the 6.60 the documented number implies" — 6.60 matches nothing. CLAUDE.md's published 0.852 gives (0.852-0.78) x 91 = 6.55 queries; the unrounded pre-refresh baseline 0.8406... gives 6.52. Use 6.55. (The optimism is therefore ~1.03 queries, not 1.1.) 3. "2.4x spread" -> 2.39x (rounding, harmless, but state it as 2.39x on a card). 4. DIRECTION OF DRIFT IS NOT UNIFORM. "every future passing refresh decays the calibration further" is true only of the SPREAD. This refresh moved two floors tighter (recall 8.4%->7.2%, nDCG 11.8%->11.5%) and two LOOSER (hit@1 15.3%->17.3%, MRR 14.0%->14.3%). A card must not say "the floors tightened." 5. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE THE REMEDY CAREFULLY — the finding's phrase "nothing re-derives the floors" invites exactly the fix 170d52c0 warns against: auto-moving floors on every refresh ("Lowering a threshold deserves suspicion... a floor calibrated on an unrepresentative corpus measures the corpus"). The repo's design is deliberately fixed absolute floors with a relaxed regression-vs-baseline. The CONFIRMED defect is narrower and should be carded as such: the measured values and the "~10%" characterisation are stated in seven places across four files (CLAUDE.md:67,84; tests/eval/taps.txt:25,43; scripts/eval_corpus.py:30,108 and the runtime-printed operator message at :331-334), nothing re-states or guards them after a PASSING refresh, and no test asserts any of them. SCOPE LIMIT OF MY REPRO #1: I did not re-measure 0.8407 / 0.4835 / 0.6065 / 0.6552. Those are the committed baseline written by cbc0a58b — which is also the finder's only source. Verifying them empirically needs a re-tap at the new pins, which is forbidden here (shared read-only corpus) and needs network. Every derived percentage in this verification inherits that. SCOPE LIMIT #2: the shared $TMPDIR/eval-home is materialised at the PRE-refresh pins — live total 10,152 vs taps.txt's recorded 10,731, with the same 10 taps PIN-DIFFERS. make eval against it would trip eval_corpus.py's CORPUS DRIFT check, not run the gate. Why it is worth doing. The "~10% under" statement is the entire published justification for where the four floors sit — CLAUDE.md calls it "loose enough that upstream drift can't flake the build, tight enough to catch a collapse." That is a claim about a margin, and the margin now differs 2.4x across the four metrics with recall the tightest at 5.52 queries out of 91. Because the monthly refresh re-baselines but never re-derives or re-states the floors, and prompts a human only when the refreshed corpus FAILS, every future passing refresh decays the calibration further with no signal at all. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ CLAUDE.md states the gate's four floors sit "~10% under" measured values of 0.852 / 0…
PlannedQuality · Retrieval eval

make eval scores a corpus with every SKILL.md body missing, reports all four floors PASS, and scores HIGHER than the real corpus

Measured. With $BOOST_HOME/repos deleted but the digest sentinel and catalog cache intact, make eval's two commands print "eval corpus already tapped for this taps.txt — skipping", index the identical 10,152 entries across the identical 20 taps under the identical "BM25 full-content" label, and exit 0 with all four floors PASS — while the index's mean document length falls from 814.836780929866 to 40.5795902285264 tokens (95.0% of the scored text absent) and hit@1 RISES from 0.473 to 0.593, clearing its 0.40 floor by 48% instead of 18%. Reproduce it. # Form A — on a machine that can tap (what a developer hits):
cd <repo>
make eval # builds .eval-home/{repos,cache}, sentinel, scores 0.852/0.473/0.605/0.657
rm -rf .eval-home/repos # clones gone; cache + sentinel survive
make eval # "already tapped — skipping"; all four floors PASS at 0.852/0.593/0.692/0.723
.venv/bin/python -c "import json;print(json.load(open('.eval-home/cache/rag_index.json'))['stats']['avg_len'])"
# Form B — exactly what I ran, against the read-only shared corpus (no network needed):
cd <repo>
H=$TMPDIR/audit-eval-tiers-final; rm -rf "$H"; mkdir -p "$H/.boost"
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. One detail in the claim is wrong and must not be copied into a card: "The index already computes the number that would catch it (stats.avg_len, rag.py:542) and nothing reads it." avg_len IS read, twice. grep -rn avg_len boost_cli scripts returns three hits: boost_cli/core/rag.py:542 (write) boost_cli/core/rag.py:677 avg = raw.get("stats", {}).get("avg_len") or 1.0 <- read, on every query scripts/build_demo_index.py:64 "avg_len": raw.get("stats", {}).get("avg_len") or 1.0 <- read rag.py:677 is BM25's own length-normalisation term (avgdl), consumed by the scorer on every search. Correct phrasing: *avg_len is read by the BM25 scorer (rag.py:677) and by build_demo_index.py:64; what nothing does is CHECK it — no gate, guard, doctor line or test compares it against an expected range or a previous build.* That distinction is also mechanically load-bearing and explains the finding's own strongest observation: because avgdl normalises uniformly, a corpus that shrinks ~20x in every document keeps its rank SET intact — which is exactly why recall@10 is bit-identical at 0.852 while only the ordering moves. Every other number in the finding is correct as stated: 40.5795902285264 vs 814.836780929866, 95.0%, +0.120/+0.087/+0.066, identical 0.852, 48%/33%/25%, 10,152 entries / 20 taps, 265 MB, and all five file:line references. Severity is … What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Defect is real and the mechanism is exactly as described; only the "nothing reads avg_len" clause and the severity need fixing. WHY Med, NOT High — two facts I verified that cap the blast radius: 1. CI cannot be greened by this. ci.yml:275 passes FORCE=1, so the sentinel is bypassed and eval_corpus.py --ensure runs; pin_clone (eval_corpus.py:244-261) raises CorpusError(UNAVAILABLE, ...) when the commit is not reachable, and _materialise collects it (line 293), so a missing clone tree exits 75. No merge can ship behind this. 2. Nothing in the repo PRODUCES the state. I grepped the Makefile and .gitignore: no target removes .eval-home/repos selectively (.eval-home/ is gitignored wholesale, so git clean -xdf takes the sentinel with it, which is a cache miss and therefore safe). It takes an out-of-band rm -rf — manual disk reclamation. A local-only false green behind a manual step that CI re-verifies is Med. It would be High if a boost command produced the state or if CI were exposed. SCOPE LIMITS OF MY REPRO — a card author must not blur these: - I measured the FULLY-missing case. The "half-materialised tap tree" phrasing in why_it_matters was NOT measured by me; degradation is presumably proportional but is unverified. Why it is worth doing. The gate whose whole job is to say "retrieval still works over a real corpus" returns a confident, better-than-baseline PASS over a corpus that is name+description only — 95% of its text absent. A developer who reclaims the 265 MB repos/ tree, or whose tap tree is half-materialised, gets four green floors that attest to nothing, and the greener number makes the loss look like an improvement rather than a defect. Every downstream floor calibration ("each floor sits ~10% under its measured value") is stated against 0.473/0.605/0.657, which the body-less run clears by 48%/33%/25% instead. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ The Tier 1 required gate has no check that the corpus it scores actually contains the…
PlannedUX · Bug

boost explain's heuristic fallback prints every heading in the file — 541 lines for one skill — while the sibling list in the same function caps at 12

Measured. BOOST_NO_AI=1 boost explain fpf-agent prints 541 lines / 27,699 bytes, 521 of them an uncapped outline emitted by the unsliced for hashes, title in headings at info.py:844, while for rule in rules[:12] nineteen lines below at info.py:859 caps the sibling list in the same function — and 2,861 of 10,152 eval-corpus entries (28.2%) exceed 25 outline lines. Reproduce it. cd <repo>
export BOOST_HOME=$TMPDIR/eval-home BOOST_NO_AI=1
./boost explain fpf-agent | wc -l # 541
./boost explain fpf-agent | wc -c # 27699
./boost explain fpf-agent | awk 'length>80' | wc -l # 42
grep -n 'rules\[:12\]\|headings = re.findall' boost_cli/commands/info.py # 840 vs 859
python3 - <<'PY'
import json,glob,os,re
home=os.environ["BOOST_HOME"]; counts=[]
for f in glob.glob(home+"/cache/*.json"):
if os.path.basename(f)=="rag_index.json": continue
d=json.load(open(f)); td=os.path.join(home,"repos",d["tap"].replace("/","__"))
for e in d.get("skills",[]):
try: t=open(os.path.join(td,e["skill_md"]),encoding="utf-8",errors="replace").read()
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Two stated details are wrong; every number in the finding is right. 1. "541 lines for one skill" (title) — fpf-agent is a WORKFLOW, not a skill: boost info fpf-agent reports kind workflow, source plugins/fpf/agents/fpf-agent.md. The card must not say "skill" while citing 541. The worst real SKILL is git-worktrees: 237 headings -> 248 output lines. Use both, or say "one catalog entry". 2. The --json rider's "the only name-taking reporting command in the info group with no --json" is contradicted by the finding's own parenthetical. Measured across the info group: list/info/log/deps/tag HAVE --json; cat, edit, preview, home AND explain all take a name and LACK it — explain is one of five, not the only one. (The six commands the finding lists as having --json — search/discover/recommend/trending/stats/count — do have it, but they are in the find group, not info.) Reword to "there is no structured form of the outline either", or drop the rider. Everything else re-derived and exact: 541 / 27,699 bytes / 42 lines >80 cols / longest 147; info.py lines 801, 840, 844, 859; p50 16, p75 27, p90 47, p95 64, p99 106, max 521, mean 22.0; 2,861 of 10,152 (28.2%) >25; 120 >100; 432 roadmap items. The finder's approximation regex agrees with the real code path exactly. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Independently confirmed not carded. 432 items; the three nearest cards are all different defects: audit-explain-findings (shipped, PR 774) is the AI path's faithfulness scoring; audit-info-stats-explain-render-a-different-smaller-shape-for-rule (shipped, PR 789) is where the outline STARTS for rules, not its length; BOOST-D27 (proposed) is the wrap-law rollout and names three gaps, none of them this. Also checked single-imperative-rule-extractor (the card about the sibling rule extractor) — it does not mention a cap. SCOPE LIMIT — the prevalence numbers are corpus-specific. p50/p75/p90/p95/p99 and the 28.2% figure describe the 20-tap, 10,152-entry shared eval corpus at $TMPDIR/eval-home. They are NOT measurements of the ~71,700-item live catalogue or of any default/starter tap set, and the card must attribute them. The code defect itself (a missing slice) is corpus-independent and needs no corpus. This is a TAIL defect, which is what justifies Med rather than High: at p50 (16 headings) output is fine — brainstorming prints 29 lines — and it only becomes unscannable in the upper quartile. Why it is worth doing. explain exists to answer "what does this do?" without reading the file, and the AI path is opt-in — the heuristic is the default answer for anyone without claude on PATH or an API key. On 28% of the catalogue it returns something no one can scan, and at the tail it returns a 541-line, 27 KB wall for a single item, which is not a summary of the file so much as a reformatting of it. The 12-cap on the rules list shows the ceiling was already understood to be necessary here. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ cmd_explain asks the model for "4-6 sentences, no markdown" (info.py:800-803), but th…
PlannedQuality · Retrieval eval

The CI-vs-Makefile floor-parity test compares only the floor VALUES, so changing -k in ci.yml turns a PASS into a FAIL with the test still green

Measured. Changing -k 10 to -k 5 in ci.yml alone leaves TestTheGateIsDefinedOnce at "2 passed" while the required gate flips from exit 0 to exit 1 — recall@k drops 0.852 to 0.753 against the 0.780 floor — and make eval stays green; the same test catches a floor-VALUE edit (hit@1 0.40 -> 0.10) with "1 failed", proving the guard runs and simply cannot see -k. Reproduce it. cd <repo>
# 1. the k=5 vs k=10 measurement, against the shared read-only corpus:
H=$TMPDIR/audit-k; rm -rf "$H"; mkdir -p "$H/.boost"
cp -R "$TMPDIR/eval-home/cache" "$H/.boost/cache"; cp "$TMPDIR/eval-home/config.json" "$H/.boost/config.json"
export HOME=$H BOOST_HOME=$H/.boost BOOST_NO_AI=1
.venv/bin/python scripts/eval_retrieval.py -k 10 --fail-under 0.78 --floor hit@1=0.40 --floor MRR=0.52 --floor nDCG@k=0.58 --regression-eps 1 | tail -8
.venv/bin/python scripts/eval_retrieval.py -k 5 --fail-under 0.78 --floor hit@1=0.40 --floor MRR=0.52 --floor nDCG@k=0.58 --regression-eps 1 | tail -8
# 2. the parity-test blindness, in a scratch copy (never edit the repo):
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Four stated details are wrong. The defect itself reproduces exactly. 1. why_it_matters says a -k edit is "a silent redefinition of three of the four floored metrics". Measured, it is TWO: recall@k (0.852 -> 0.753) and nDCG@k (0.657 -> 0.624). hit@1 and MRR are k-independent by construction — scripts/eval_retrieval.py:111-112 reads "hit@1": lambda r, rel, k: hit_at_1(r, rel) and "MRR": lambda r, rel, k: reciprocal_rank(r, rel), both discarding k — and both measured identical at 0.473 / 0.605 at k=10 and k=5. (The claim field says "recall@k and nDCG@k", which is right; only why_it_matters contradicts it.) 2. "The comment block in ci.yml (twelve lines at :232-243)". The FLAGS-MUST-MATCH paragraph is EIGHT lines, .github/workflows/ci.yml:232-239. Line :240 is a bare # and :241 onward is the unrelated "THE CLONES ARE CACHED" block about cache availability. 3. C's stated mechanism is stale. "dropping --regression-eps 1 restores the 0.02 default, so upstream corpus drift reddens every open PR" — tests/eval/taps.txt now pins a 40-char commit SHA per row, so upstream HEAD can no longer move the corpus. Measured: dropping the flag today exits 0 with no REGRESSION lines. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SOURCE VERIFIED, NOT ALREADY_FIXED. tests/unit/test_eval_corpus.py:404-412 still reads exactly as claimed: _flags does re.findall(r"--floor\s+([\w@]+)=([\d.]+)", text) plus re.search(r"--fail-under\s+([\d.]+)", text) and returns only that dict. .github/workflows/ci.yml:232 still says "THE FLAGS MUST MATCH make eval — tests/unit/test_eval_corpus.py fails the build if they drift." Both file:line references in the finding are accurate. NO SECOND GUARD EXISTS. _flags is the only comparison of ci.yml against the Makefile anywhere in the file (used at lines 423 and 439, the two tests in the class), and grepping tests/unit + tests/functional for any -k parity assertion found only unrelated dense.retrieve(k=10) and boost chat -k arg-validation tests. NOT A DUPLICATE, BUT NAME THE ORIGIN CARD. docs/roadmap/items/eval-corpus-was-not-actually-pinned.md (status: shipped) is the card that INTRODUCED this parity test, and its body overclaims: "a unit test now compares the flags in Makefile and ci.yml and fails the build when they disagree — the drift was invisible precisely because two files each looked right on their own." That sentence is what this finding falsifies for three of the four flags. A card author should link it as the contradicted claim, not merge into it. I re-checked all 432 titles and grepped every item body for TestTheGateIsDefinedOnce, test_eval_corpus, FLAGS MUST MATCH, _flags, regression-eps, golden-natural, and for files mentioning both ci.yml and Makefile — the only hit is that origin card. Why it is worth doing. The comment block in ci.yml (twelve lines at :232-243) and the test's own docstring both promise that the required check and the documented gate cannot diverge — that promise is why nobody re-reads the two invocations. It holds for one of the four flags that decide the gate. A -k edit in either file is a silent redefinition of three of the four floored metrics, and the measured gap at k=5 (0.753 vs a 0.780 floor) is on the failing side, so the first symptom is a red required check on an unrelated PR with a green make eval locally. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ ci.yml:232 says "THE FLAGS MUST MATCH make eval — tests/unit/test_eval_corpus.py fail…
PlannedQuality · Retrieval eval

The exemplar mechanism was applied to the ungated set only: golden.jsonl is 0/91 pinned, and 10 of its 43 hit@1 credits are on names the metric cannot adjudicate

Measured. On the 10,731-entry corpus that committed taps.txt materializes today, tests/eval/golden.jsonl — the only set the required eval gate floors — is 0/91 exemplar-pinned while the ungated golden-natural.jsonl is 50/50; 10 of its 44 hit@1 credits (22.7%) are awarded on a name that resolves to more than one distinct body, and the hit@1 floor's entire headroom is 7.60 queries, smaller than the 10 credits the metric cannot adjudicate. Reproduce it. cd <repo>
export BOOST_HOME=$TMPDIR/eval-home
# 0. confirm which corpus you are on (expect 10152)
.venv/bin/python -c "import sys;sys.path.insert(0,'.');from boost_cli.core import catalog;print(len(catalog.all_entries()))"; echo EXIT=$?
# 1. exemplar counts per set
python3 -c "
import json
for f in ['tests/eval/golden.jsonl','tests/eval/golden-natural.jsonl']:
r=[json.loads(l) for l in open(f) if l.strip() and not l.startswith('#')]
print(f,'rows:',len(r),'with exemplar:',sum(1 for x in r if x.get('exemplar')))"; echo EXIT=$?
# 2. the harness's own list of undecided gated rows
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The defect is real and the spine is exact, but five figures are stale by one corpus refresh. The finding measured on the 10,152-entry corpus CLAUDE.md documents; committed tests/eval/taps.txt moved on 2026-09-01 (cbc0a58b) and now materializes 10,731 entries, which is what make eval/CI builds today. Corrected values, all measured by me on that corpus: - hit@1 credits: 43/91 = 0.473 -> 44/91 = 0.484 - un-adjudicated share of credits: 23.3% -> 22.7% (the count stays 10) - floor headroom: 6.60 queries -> 7.60 queries - worst-case bound stripping all 10: 33/91 = 0.363 -> 34/91 = 0.374 (still below the 0.400 floor) - BM25 four-metric line: 0.852 / 0.473 / 0.605 / 0.657 -> 0.841 / 0.484 / 0.607 / 0.655 The finding's numbers are correct for the corpus it names and it disclosed the provenance honestly; they are nonetheless the wrong numbers to print about "the set the required eval gate floors", because the gate no longer builds that corpus. A card must publish the 10,731 figures and name the corpus. Nothing else in the finding is wrong. 0/91 vs 50/50, the 27 undecided rows / 62 candidate bodies, the same 10 names and 10 queries, the skill-creator worked example, the not-carded check, and the "realistic shift is 1-2 queries, the floor still passes" bound all hold on BOTH corpora. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. 1. WHAT IS CORPUS-INDEPENDENT (identical on 10,152 and 10,731): exemplar counts 0/91 and 50/50; 27 undecided worksheet rows over 62 candidate bodies; the same 10 ambiguous-credit names and the same 10 queries; the skill-creator example. Only the credit count and the two ratios moved. Build the card on the invariant half and quote the 10,731 figures for the rest. 2. TRAP FOR THE NEXT VERIFIER: the shared read-only $TMPDIR/eval-home is a PRE-REFRESH corpus — its .eval-corpus-ready sentinel digest does not match sha256(tests/eval/taps.txt), and 10 of its 20 clones sit at a different commit. Anyone who measures only there reproduces the finding's old numbers verbatim and marks it CONFIRMED without noticing. Rebuilding at the committed pins in a disposable HOME took ~3 minutes and is what produced the correction. 3. DO NOT COPY Makefile:126, which still comments "over twenty it scores 0.863 / 0.473 / 0.607 / 0.662". That matches neither corpus (10,152 gives 0.852/0.473/0.605/0.657; 10,731 gives 0.841/0.484/0.607/0.655). CLAUDE.md:67's figures are right for the old corpus and now also stale. 4. SCOPE OF MY REPRO: I verified the BM25 engine only (--engines bm25), which is what the gate floors; I did not build a dense store (impossible here). I did not run make eval end to end, only its second command against a corpus I materialised with its first. 5. NOT A DEFECT IN THE HARNESS: exemplar_worksheet (scripts/eval_retrieval.py:219) and exemplar grading work correctly and fail loudly on a bad pin. Why it is worth doing. golden.jsonl is the set the required eval gate floors, so it is the only one that can block a merge. 23.3% of its hit@1 credits are awarded on a name that maps to several genuinely different skills, and the floor's whole margin (6.60 queries) is narrower than the un-adjudicated credit count (10). That does not mean retrieval is worse than reported — the realistic shift is 1-2 queries — it means the published margin cannot be read as precision about the intended skill, so anyone tuning blend weights or pool depth against this gate inherits an unquantified slack. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ CLAUDE.md presents exemplar pinning as a live migration — "relevance is still decided…
PlannedUX · Bug

boost info/deps on a not-installed rule or workflow widens the tap's sparse cone for a directory source_dir_for immediately rejects

Measured. Across all 20 taps of the eval corpus, 0 of the 188 distinct (tap, rel_dir) directories that hold a rule or workflow contains a SKILL.md, and 0 of them have rel_dir == "." — so the existence check at store.py:179 rejects every single one of the 1,493 rule/workflow entries (14.71% of 10,152), and it does so only after store.py:178 has already written a new pattern into the tap's .git/info/sparse-checkout. Two read-only commands, boost info and boost deps, each provably widen a tap's sparse cone and then discard the directory they widened it for. Reproduce it. cd <repo>
S=$TMPDIR/eval-home; D=$TMPDIR/verify-f3; rm -rf $D; mkdir -p $D/.boost/repos $D/.boost/cache
cp -R "$S/repos/Aaronontheweb__dotnet-cursor-rules" $D/.boost/repos/
cp "$S/cache/Aaronontheweb__dotnet-cursor-rules.json" $D/.boost/cache/
python3 -c "import json;s=json.load(open('$S/config.json'));s['taps']=[t for t in s['taps'] if t['name']=='Aaronontheweb/dotnet-cursor-rules'];json.dump(s,open('$D/.boost/config.json','w'))"
export HOME=$D BOOST_HOME=$D/.boost BOOST_NO_AI=1
git -C $BOOST_HOME/repos/Aaronontheweb__dotnet-cursor-rules sparse-checkout list
./boost info meta
git -C $BOOST_HOME/repos/Aaronontheweb__dotnet-cursor-rules sparse-checkout list
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The defect is real and both repros land, but four statements in the finding are wrong and must not ship in a card: 1. TIMING — "~0.3s, roughly half the command" (runs 0.701 / 0.400 / 0.325s) is wrong by ~7x. The finder measured three consecutive runs against ONE copy, which conflates Python/page-cache warm-up with the git write: their own run2->run3 drops 0.075s with no write occurring at all. My controlled A/B (5 FRESH copies per arm, same tap, same entry) measures 0.166s mean with the widening vs 0.122s mean pre-widened. The correct figure is a delta of 0.044s, ~27% of the command — not 0.3s and not "roughly half". 2. "mutate the tap clone on EVERY invocation" is wrong. It is once per (tap, rel_dir). gitutil._sparse_list (gitutil.py:292-304) is a plain file read, and materialize returns at gitutil.py:341-342 once the pattern is present — so after the first run there is no git subprocess and no write. source_dir_for is still called and the raise still swallowed on every invocation, but nothing is written. (The finder's own why_it_matters says "per first-look", so it is the claim field that overstates.) 3. "silently re-inflates exactly what boost compact exists to shrink (177 MB -> 93 MB)" is not supported. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Not carded — I re-did the check independently rather than trusting the finder's. I grepped all ^title: lines and all bodies for source_dir_for|materialize|sparse-checkout|sparse cone (22 cards) and read the four plausible ones. taps-check-out-freight-they-never-index.md documents source_dir_for materializing as the deliberate chokepoint for consumers of a tap's real files and says nothing about a consumer that discards the result. audit-info-stats-explain-render-a-different-smaller-shape-for-rule.md (status: shipped, PR 789) is the nearest miss and is worth naming in the card: its shipped fix folded the kind != skill branch of cmd_info INTO the skill path, which is what routes a not-installed rule/workflow into this source_dir_for call in the first place — so this is a side effect of that fix, not an independent old bug. Its source line fix also shipped (my run prints .cursor/rules/meta.mdc, not .cursor/rules). Neither audit-deps-findings.md nor audit-dry-runs-disagree-... touches the materialize call. SCOPE LIMITS of my repro, all of which a card author must respect: - The 188/3/46 and 14.71% figures are measured against the 20-tap, 10,152-entry eval corpus ONLY. They are not the shipped default/starter registry set and not the user's real ~445-tap install. The RATIO could differ on a registry that ships scripts or JSON beside its commands; the direction (the check is guaranteed to fail) will not, since it follows from the classifier. - I could NOT exercise the network path. Why it is worth doing. An information command performs a git write (and, on a tap that has not fetched those blobs, a network fetch) for 14.7% of the catalogue, gains nothing from it, and silently re-inflates exactly what boost compact exists to shrink (CLAUDE.md records compact as 177 MB -> 93 MB). Browsing the catalogue therefore costs disk, ~0.3s per first-look, and — offline or behind a proxy — can fail or hang a command that only needed a cache read. CLAUDE.md's own rule is that source_dir_for is the chokepoint for "anything reading a tap's real files"; here nothing reads them. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ cmd_info (info.py:482-486) and _skill_dir_for_deps (info.py:1058-1066) call store.sou…
PlannedInterop

boost_install's description enumerates four of the five enabled agent targets, and gives Antigravity the one mechanism it does not use

Measured. The enumeration is provably stale rather than deliberately partial: the string was written in PR #442 on 2026-08-03, when boost had four agent targets, and Antigravity CLI landed as a FIFTH -- and a LINKING -- target in PR #602 on 2026-08-30 (git merge-base --is-ancestor confirms #442 precedes #602), and the string has not been touched since; today enabled_agents() returns 5 and linking_agents() returns 4 while the description names 4 and 3 of them respectively, and boost_install is the only one of the seven MCP tool descriptions that names any agent at all -- so the fix is a single string. Reproduce it. cd <repo>
rm -rf $TMPDIR/audit-mcp-f4 $TMPDIR/audit-mcp-f4-fix
export HOME=$TMPDIR/audit-mcp-f4 && export BOOST_HOME=$HOME/.boost
export BOOST_NO_AI=1 && export BOOST_ASSUME_YES=1 && mkdir -p "$HOME"
.venv/bin/python tests/make_fixture.py $TMPDIR/audit-mcp-f4-fix >/dev/null
./boost tap $TMPDIR/audit-mcp-f4-fix >/dev/null 2>&1
.venv/bin/python -c '
from boost_cli.commands import configuration as c
from boost_cli.core import agents
print("linking_agents ->", list(agents.linking_agents()))
print("enabled_agents ->", list(agents.enabled_agents()))
txt,_ = c.REGISTRY.call("boost_install", {"name":"brainstorming"})
print(txt.splitlines()[1])
d={s["name"]:s["description"] for s in c.REGISTRY.specs()}
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Two stated details are wrong; the defect itself is real. 1. THE TITLE OVER-CLAIMS. "gives Antigravity the one mechanism it does not use" states something about the text that the text does not do. The description does not ASSIGN Antigravity any mechanism -- it omits Antigravity entirely (verified: antigravity in any description: False, and 0 hits for 'antigravity' in both mcp.py and configuration.py). The finder's BODY is careful and correct ("the only claim an Antigravity reader can map onto itself"), but that is an inference about how a reader would map itself, not a measured property of the string. A card title ships literally. Correct title: "boost_install's description enumerates four of the five enabled agent targets and omits Antigravity CLI, the fourth linking agent." 2. THE FINDER'S GREP WAS MALFORMED, so its cited "no matches" is not reproducible as meant. grep -n 'Windsurf|Cursor|symlink|gemini' tests/unit/test_mcp.py is BASIC grep -- the pipes are literal characters, so it could only ever match a line containing that exact string. With grep -inE the file has FIVE matches (lines 493, 571, 623, 687, 738). The CONCLUSION survives: I read all five and none pins boost_install's agent enumeration (they are Gemini-CLI-instructions-delivery comments and one tap name). What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO: fully in scope, no caveat needed. The defect is a static source string plus two pure functions over config defaults, so the 20-tap eval corpus is irrelevant here and I never touched it. Antigravity's enabled comes from config defaults (agents.py:25, bool(spec.get("enabled", True))), not from ~/.gemini existing, so the repro does not depend on any Antigravity install being present -- I got all five agents from a bare disposable HOME with only the test fixture tapped. BONUS STALENESS THE FINDER MISSED, same cause and same PR: the explanatory comment at boost_cli/commands/configuration.py:1467-1468 quotes the output as only "linked agents: claude-code, windsurf, cursor" -- a three-agent list. The real line now emits four (linked agents: claude-code, windsurf, cursor, antigravity, per my repro). If the card ships a fix, that comment should be corrected in the same edit or it becomes a second stale artifact of #602. WHY THIS IS THE HOST WHERE IT BITES (supports relevance, already established in-repo, not something I had to infer): core/mcphost.py:13,86,94 registers boost with agy -- Antigravity CLI is its third MCP host -- so an Antigravity agent genuinely reads this exact description. And the design comment at configuration.py:1575-1582 states that on Gemini-family hosts the function declarations are "the only boost text reliably in context at the moment an agent chooses a tool" (server instructions land in the trust-gated GEMINI.md memory tier). Why it is worth doing. The harm is the wrong mechanism, not the missing name. The install reply already tells Gemini-family users that a skill is usable without linking, and the comment above that line (configuration.py:1466-1470) says why it exists: an agent that does not see itself in the linked list "concludes the skill did not reach *it*, and goes back to reconstructing the work by hand." An Antigravity CLI agent reading only the description gets the inverse error — the sole Gemini-family claim on offer says the store is read directly, which for Antigravity is false, so a pre-install answer to "will this reach me" is wrong in the direction the surface was written to avoid. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity S Impact Low Wow ★★ boost_install's description says installs are "wired into every agent you have enable…
PlannedInterop

boost_search advertises "10-15 seconds" unconditionally; with no AI configured it is 0.013 s median and the rerank never runs

Measured. With no AI backend available, boost_search over 20 distinct queries against the 10,152-entry eval corpus returned in a median of 0.0134 s (min 0.0057 s, max 0.1206 s, and 0.075 s for the first search in a fresh process) while both MCP surfaces state an unconditional "10-15 seconds — an LLM reranks every match" — because rag.py:1129 returns the retrieval order untouched when ai.available() is False, so no LLM call is ever made. Reproduce it. cd <repo>
export HOME=$TMPDIR/audit-mcp-surface && mkdir -p "$HOME"
export BOOST_HOME=$TMPDIR/eval-home && export BOOST_NO_AI=1
.venv/bin/python -c '
import time, statistics
from boost_cli.commands import configuration as c
from boost_cli.core import ai
print("ai.available() ->", ai.available(), "(rag.py:1129 returns the retrieval order unreranked when False)")
Q=["set up code review","add commit conventions","write a migration","debug flaky tests","python testing","react hooks","terraform module","docker compose","ci pipeline","security audit","api documentation","database schema","kubernetes deploy","git hooks","typescript lint","rust async","llm prompt","data pipeline","monorepo build","code"]
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. THE HEADLINE MULTIPLIER IS MISLABELED. "790x-2,240x over-stated at the median" mixes two statistics: 2,240x is 15s / 0.0067s, the finder's MIN, not its median. At the finder's own median (0.0126s) the range is 794x-1,190x; at my reproduced median (0.0134s) it is 746x-1,119x. A card must say ~750x-1,200x at the median, or drop the point multiplier entirely (see notes on corpus scope). 2. configuration.py:1271-1290 is wrong for _ranking_note. It is at lines 1275-1294 (def at 1275). 3. mcp.py:428-447 for engine_note — the def is at line 427 (body 427-447). Everything else in the finding checked out exactly: rag.py:1129 is verbatim if not ai.available(): followed by return hits[:limit], engine; tests/unit/test_mcp.py:200-201 and :545 are the exact assertions quoted; the quote from mcp-search-hid-which-ranking-ran.md ("A boost installed with pipx has neither inside its venv unless the key is exported, so the silent path is the common one rather than the edge case") is verbatim; the captured reply line is verbatim; and both surfaces do carry "10-15 seconds" as an unconditional literal with no hedge for machine state. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE LIMITS OF MY REPRO — a card author must not overstate any of these: 1. CORPUS-DEPENDENCE OF THE MULTIPLIER. My timings are against the 20-tap / 10,152-entry eval corpus. The repo's OWN shipped card mcp-search-cost-was-understated.md measured the same rerank-off path at 0.10 s — ~8x slower than my median — which yields 117x-170x, not three orders of magnitude. The real user install is ~71,700 items (7x the eval corpus again). The defect is unaffected, but the headline number is not robust: write it as "two to three orders of magnitude, corpus-dependent" and name the corpus, never a bare "2,240x". 2. THE 10-15 s NUMERATOR IS INHERITED, NOT RE-MEASURED. I could not verify it here. With BOOST_NO_AI unset, ai.available() returned True (the claude shim is on PATH) yet the rerank still fell through to BM25 at 0.80-0.94 s and the reply carried the "did NOT run" note — a sandbox CLI-shim artifact, not evidence about real machines. The 11.7-17.0 s figure comes from mcp-search-cost-was-understated.md, not from me. 3. BOOST_NO_AI=1 IS A PROXY, AND THE "COMMON CASE" PREMISE IS INHERITED. On this machine claude is on PATH, so ai.available() is naturally True and the keyless state was simulated. The proxy is fair — available() is enabled() and (has_cli() or ANTHROPIC_API_KEY), and both routes hit the identical rag.py:1129 branch — but I did NOT observe a keyless machine. The "pipx has neither inside its venv" premise is the sibling card's assertion restated, and it is loose reasoning: has_cli() calls shutil.which, which reads PATH, not the venv. Why it is worth doing. The stated cost is not decoration — mcp.py's own comment calls it load-bearing ("The stated COST kills the hesitation over an unknown-price call") and it is the sole justification for the "WORTH THE SECONDS" gate that tells an agent to search only when the request touches more than one file or outlives the session. On the machine this repo calls the common one there are no seconds, so the gate suppresses a free call on a false price — the mirror of the defect the mcp-search-cost-was-understated card fixed in the other direction, and with the same consequence: "a wrong cost in the one paragraph whose job is to make the tool worth reaching for discredits everything around it." An agent that … Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ boost_search's description and INSTRUCTIONS both state the cost as a flat "10-15 seco…
PlannedInterop

The skip list — the one bound on boost's triggers — ships in INSTRUCTIONS only, in zero of the seven tool descriptions

Measured. Six of the seven load-bearing elements of mcp.INSTRUCTIONS are duplicated into boost_search's description and the seventh — the bound — appears in zero of the seven descriptions: verified both by substring probe and by reading all seven descriptions in full, where the only occurrence of the word "skip" in any description is the rerank-cache sentence ("repeating an identical search skips the LLM"), and "not for", "do not call", "trivial", "too small" and "overkill" return NONE across all seven. Reproduce it. cd <repo>
export HOME=$TMPDIR/audit-mcp-surface && mkdir -p "$HOME"
export BOOST_HOME=$TMPDIR/eval-home && export BOOST_NO_AI=1
.venv/bin/python -c '
from boost_cli.core import mcp
from boost_cli.commands import configuration as c
d={s["name"]:s["description"] for s in c.REGISTRY.specs()}
SKIP="Skip it for a question, a one-line edit, or a command you were just handed"
print("tools:", list(d))
print("skip list in INSTRUCTIONS:", SKIP in mcp.INSTRUCTIONS)
for p in ("one-line edit","just handed","Skip it for"):
print(repr(p),"-> descriptions containing it:",[n for n,v in d.items() if p.lower() in v.lower()] or "NONE")
for p in ("has a name","10-15 seconds","the task stays yours","one kind of three"):
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The DEFECT reproduces exactly as stated — the seven-row table, the substring probes, the mcp.py:224-225 location, the three cited tests, and the single-arm eval_tools.py are all correct. Three stated details in the write-up are wrong and must not be copied onto a card: 1. "grep for the literal across the repo returns only mcp.py:224, test_mcp.py:467 (a comment), tests/eval/tool_calls.jsonl, and boost_cli/data/rules/boost-first.mdc:54." INCOMPLETE. My grep also returns tests/unit/test_builtin.py:253 and :399, boost_cli/core/mcp.py:144 and :225, docs/roadmap/items/mcp-check-skills-before-starting-a-task.md:37, docs/roadmap/items/tool-call-eval-tier.md:43, docs/roadmap/items/mcp-zero-setup-and-three-kinds.md:42, and docs/roadmap.html (3 lines). The test_builtin.py hits matter: they are real skip-list assertions, but over the builtin boost-first RULE body, not over any tool description — so they reinforce rather than weaken the finding. 2. not_carded_check says "grepped grep -rn 'one-line edit|just handed|Skip it for' across docs/roadmap/items: no hits." FALSE — there are three (listed above). The finder almost certainly ran an alternation without -E, so it searched for the literal string containing pipe characters. I read all three cards: none covers the description gap, so the conclusion survives, but the stated evidence for it does not. 3. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Scope of my repro: this is a pure source/text audit, settled entirely against the repo at HEAD (main, 9b70fc8e). I measured NO behavioral effect. No gemini CLI is reachable here and the repo's Tier 3 has no Gemini arm, so "the trigger fires on the excluded prompts" is argued, not measured — the finding asserts it from the description's wording, and I did not and could not test it. The eval-corpus BOOST_HOME was irrelevant to this finding and I used it read-only. Why Med rather than the finder's High. The asymmetry is unambiguous and violates a rule the repo wrote down twice, which argues for High — and the repo did ship mcp-already-covered-defeater (#479) at impact High on the identical argument (descriptions are the only carrier on Gemini). But the harm direction here is OVER-calling, milder than the under-calling failure #479 fixed, and it is unmeasured. High is defensible on the repo's own precedent; I would not let the card assert a measured behavioral effect either way. THE ONE THING A CARD AUTHOR MUST NOT GET WRONG — there is a recorded counter-precedent for the obvious remedy. docs/roadmap/items/mcp-one-benefit-nameable-task.md (#355) says the previous pass "declared three triggers bounded by a proportionality note, and the bound beat the triggers every time — judging work 'non-trivial' takes judgement, while 'this turn looks small' is free, and every turn looks small when it opens." So a bound in the agent-facing text has already been measured-by-observation to over-suppress once. Why it is worth doing. mcp.py's own comments say the bound is what buys the rest of the guidance its credibility — "an unbounded 'check first' gets ignored wholesale" and "a surface that captures work it cannot do gets routed around permanently the first time it misses". On the host where this repo already documented and card-fixed a real miss (the Gemini LangGraph session, PR #479), what actually ships is every persuasive element and none of the restraint. That is the capture the whole surface is written to avoid, delivered by omission on the one host it was rewritten for. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ boost's agent-facing surface duplicates six of its seven load-bearing elements from I…
PlannedUX · Bug

out.panel still fits its content to term_width(), so boost count | … clips the line to an assumed 80 columns

Measured. On this machine ./boost count | cat renders the 110-character inventory summary into an 80-column box, clipping the content to 76 columns and dropping the 34-character tail "0 taps) · discovery index not built", while COLUMNS=300 ./boost count prints all 110. Reproduce it. cd <repo>
F=$TMPDIR/panel-fixture; rm -rf $F; mkdir -p $F/skills/alpha-skill $F/commands $F/.cursor/rules
printf -- '---\nname: alpha-skill\ndescription: A demo skill.\n---\n# Alpha\n' > $F/skills/alpha-skill/SKILL.md
printf -- '---\nname: beta-workflow\ndescription: A demo workflow.\n---\nRun it.\n' > $F/commands/beta-workflow.md
printf -- '---\ndescription: A demo rule.\n---\nAlways do it.\n' > $F/.cursor/rules/gamma-rule.mdc
(cd $F && git init -q . && git add -A && git -c user.email=a@b -c user.name=a commit -qm init)
export HOME=$TMPDIR/panel-home; rm -rf $HOME; mkdir -p $HOME; export BOOST_HOME=$HOME/.boost BOOST_NO_AI=1 BOOST_ASSUME_YES=1
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Three stated widths are wrong; the defect itself reproduces exactly. 1. "The piped box is 78 columns wide" → it is 80. room = term_width()-4 = 76, _clip_visible fills to exactly 76 display columns (ellipsis included), box = inner + 4 = 80. 80 is the only value reachable once clipping fires; 78 is not. 2. "piped 81 (clipped at 76 + ellipsis)" → the piped box row is 80 columns and the content is clipped to 76 columns TOTAL, i.e. 75 chars plus the ellipsis. "76 + ellipsis" would be 77 and is not what _clip_visible does. 3. "full line 114 chars" → 114 is the rendered box ROW in display columns; the summary TEXT is 110 chars. State which one. The lost tail is 34 chars, ~31% of the line. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Scope of my repro, and things a card author must not get wrong: 1. Only count loses UNIQUE data. I verified the install panel clips too (→ claude-code · win…), but the ✓ linked → claude-code · windsurf · cursor · antigravity status line printed immediately above it is unclipped, so nothing is actually lost there. panel is a third EMIT site of the root cause but boost count is the only DATA-LOSS site. Do not inflate install/uninstall into a second loss site. 2. boost count --json is an unclipped machine-readable path, so a script has an escape hatch. That plus (1) is why Low is right. 3. The installed 47 (40 skills · 1 rule · 6 workflows) figure comes from the REAL HOME's lockfile, not the eval corpus — BOOST_HOME does not move the skills store. Only 10152 / 20 taps are the corpus's. Do not attribute 47 to the corpus in a card. 4. I did not run boost count against the shared $TMPDIR/eval-home. I copied config.json + cache/ to a private BOOST_HOME with no repos/, so load_tap served the stale caches as-is and wrote nothing to the shared corpus. 5. A fix that routes panel through pane_width() will turn 7 tests red: tests/unit/test_output.py::TestPanelFitsTerminal (lines 539-586) monkeypatches output.term_width, and under pytest stdout is not a TTY, so pane_width() returns None in all of them. They need to patch pane_width/isatty instead, the way PR 739 reworked the table tests. Whoever fixes this will otherwise think the fix is wrong. 6. Why it is worth doing. boost count is the one-line inventory a script or a shell prompt would capture, and piping it silently drops the discovery-index state and (on a real machine) the tap count. It is small in isolation, but it is the last unconverted emit site of a root cause the repo already diagnosed, wrote a helper for, and pinned tests around — leaving it means the invariant "a pipe has no pane" is true of tables and --help and false of boxes, which is the kind of split that grows back. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity S Impact Low Wow ★★ output.panel computes room = term_width() - 4 (output.py:527) and clips its content t…
PlannedOnboarding · Bug

quickstart silently discards the incompatible shard status, so a user with any API key is never told why zero vectors arrived

Measured. On one machine with VOYAGE_API_KEY set, shards.sync returned 7 rows all reading status: "incompatible", detail: "published shards are local, this machine embeds with voyage" and boost quickstart rendered zero of them — while boost quickstart --dry-run on that same machine promised "would build the keyword index, then import 5 shard(s)". Reproduce it. cd <repo>
curl -sSL -o $TMPDIR/mf.json https://github.com/jonnyeclectic/boost/releases/download/shards-latest/manifest.json
export HOME=$TMPDIR/audit-qs-f1 ; export BOOST_HOME=$HOME/.boost ; mkdir -p "$HOME"
env -u OPENAI_API_KEY VOYAGE_API_KEY=x HOME="$HOME" BOOST_HOME="$BOOST_HOME" \
BOOST_SHARD_MANIFEST="file://$TMPDIR/mf.json" \
.venv/bin/python -c "import sys;sys.path.insert(0,'.');from boost_cli.cli import main;sys.exit(main(sys.argv[1:]))" quickstart
# -> no line mentions 'voyage', 'incompatible', or 'embedding space'.
# Now show what sync actually returned and quickstart threw away:
env -u OPENAI_API_KEY VOYAGE_API_KEY=x HOME="$HOME" BOOST_HOME="$BOOST_HOME" \
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Three details are wrong or understated; the defect itself is real and reproduced verbatim. 1. "17.1 MB for the seven defaults" — 17.1 MB (16.3 MiB) is the total across the 5 of 7 defaults that have a manifest row. expo/skills and K-Dense-AI/scientific-agent-skills have no published shard at all (which is also why they tapped without an @ sha). Per-row: qdhenry/Claude-Command-Suite 7.9 MB, trailofbits/skills 4.5 MB, PatrickJS/awesome-cursorrules 3.6 MB, anthropics/skills 0.7 MB, obra/superpowers 0.4 MB. The claim "all seven come back incompatible" is still exactly right, because incompatible() short-circuits before the row lookup. (459 rows and 1,604.8 MB total re-derive correctly.) 2. "points at hours of CPU" — wrong mechanism. For the affected user (a Voyage/OpenAI key is exported, which is *why* the shards are refused), boost reindex --dense embeds through the paid API, not the local model: the cost is money plus network time, not local CPU hours. Same conclusion, different bill. 3. "The fix is one tuple entry plus adding incompatible to the status comment" — understated, because there is a second defect on the same path that the finding does not name. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Verified against current source, not the finder's text: _report (quickstart.py:105-135) renders imported/current then loops exactly ("unpublished","refused","failed"); incompatible is produced at shards.py:357-360 and is absent from the status-vocabulary comment at shards.py:326-330. _report has exactly one caller (quickstart.py:211) — the sibling surfaces commands/discovery.py:401-403 (reindex --fetch-shards) and commands/pkg.py:984-986 (update --shards) both raise BoostError("published shards cannot serve this machine — %s") and exit 1, so quickstart is the only surface that swallows it. Test claim confirmed: tests/functional/test_cli_quickstart.py:254 lives in class TestFetchShards, and the quickstart class's test_shard_commits_are_passed_as_pins (line 227) stubs shards.sync to []. CARD-AUTHOR WARNINGS: - Do NOT fix this by appending ("incompatible", …) to the existing tuple as-is: sync stamps the *same machine-level* detail on every tap, so that emits seven identical muted lines. Render it once, and name both remedies — unset the key to take the free ~17 MB download, or keep the key and pay for reindex --dense through the API. - Affected population is narrower than "any machine with a key": it is key-exported AND the [rag] extra installed. Why it is worth doing. A Voyage or OpenAI key is a configuration boost explicitly supports ("a key is a quality upgrade rather than the entry fee" — CLAUDE.md), and it is exactly what a serious first user is most likely to have already exported. For that user the headline feature of quickstart — 1,604.8 MB of published vectors across 459 rows, or 17.1 MB for the seven defaults — is declined with no reason given, and the one line they do get (embed the rest locally when you want to: boost reindex --dense) points at hours of CPU without mentioning that unsetting one env var would make the free download work instead. The fix is one tuple entry plus adding incompatible to the status comment at shards.py:327. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ shards.sync() returns status: "incompatible" with a fully-formed reason when the publ…
PlannedOnboarding · Bug

A local manifest read error is reported as "no published shards", and the BoostError's hint — the only actionable line — is discarded

Measured. 4 of the 9 BoostError raises reachable from shards.fetch_manifest() carry a hint, and all three transport-shaped failures are among them — scheme refusal, "cannot reach" (hint: "shards are optional — boost reindex --dense embeds locally instead"), and truncation (hint: "a proxy or a dropped connection cut the stream — retry") — so the one line quickstart.py:164 throws away is precisely the line that fires on the real-world failures, while a live run with an unreadable manifest still ends ✓ ready at exit 0 with all seven taps unpinned and no word about vectors. Reproduce it. cd <repo>
export HOME=$TMPDIR/audit-qs-f4 ; export BOOST_HOME=$HOME/.boost ; mkdir -p "$HOME"
# 1. the error object carries a hint:
env HOME="$HOME" BOOST_HOME="$BOOST_HOME" BOOST_SHARD_MANIFEST="file:///nonexistent-manifest.json" \
.venv/bin/python -c "
import sys; sys.path.insert(0,'.')
from boost_cli.core import shards
from boost_cli.errors import BoostError
try: shards.fetch_manifest()
except BoostError as e: print('message:', e.message); print('hint :', e.hint)"
# 2. what quickstart prints instead (dry-run: no clones, no writes):
env HOME="$HOME" BOOST_HOME="$BOOST_HOME" BOOST_SHARD_MANIFEST="file:///nonexistent-manifest.json" \
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Three stated details are wrong; the defect itself is real and reproduces exactly. 1. "grep -n 'hint' boost_cli/commands/quickstart.py returns only argparse help= strings" — it returns ZERO lines. No help= string in the file contains the substring "hint". The underlying claim (exc.hint is never read in quickstart.py) is correct. 2. "at quickstart.py:212-217 the chain is if manifest is not None: … elif args.no_vectors: … elif not dense.have_backend():" — the chain spans lines 200-217. if manifest is not None: is at line 200; 212-217 covers only the two elif arms (212 elif args.no_vectors:, 214 elif not dense.have_backend():). Cite 200-217, not 212-217. 3. "Read all 318 ^title: lines" — grep -h '^title:' docs/roadmap/items/*.md | wc -l returns 432 today (432 item files). Likely concurrent-loop growth since the finder ran, but the stated count is stale. Every other figure re-derived and matched: quickstart.py:163-165 (the catch) exact; :173-176 (the "the live path already explains both cases" comment) exact; :186-189 (the dry-run explanatory branch) exact; the live manifest is 169,851 bytes and publishes 459 shards; fetch_manifest is one GET with no retry loop; live run exits 0 with 962 items and no @ sha pins. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Scope of my repro: I forced the failure with BOOST_SHARD_MANIFEST=file:///nonexistent-manifest.json, which takes the _open URLError branch (hint "shards are optional…"). The truncation hint at shards.py:152-155 is verified by reading source, NOT by reproducing a truncated download — the finder's 113778 of 169851 bytes capture is theirs. I corroborated only that 169,851 is the real Content-Length today (curl, http=200). Things a card author must not get wrong: - _report() (~quickstart.py:120) uses the SINGULAR "no published shard" per-tap, and that wording is correct in its context. The fix targets only the plural at line 164. Do not sweep both. - shards.rows() never raises (it skips malformed rows by design, shards.py:222-237), so fetch_manifest is the only BoostError source inside the try at 160-165. Every error that reaches the catch is a read/transport/format failure — "no published shards" is wrong in every reachable case, not merely usually. - 5 of the 9 raises carry NO hint (implausibly large, not-valid-JSON, not-an-object, missing provider/model/dim, no shards list), so "render exc.hint" improves a subset; the wording fix and the missing live-path else-branch are the parts that help universally. - The convention the catch bypasses is real: boost_cli/cli.py:368 renders out.err(e.message, hint=e.hint) at top level. - No test pins the current wording — grep -rn 'no published shards' across the repo has exactly one hit, quickstart.py:164 itself. Why it is worth doing. The manifest fetch is one 170 KB GET that decides whether the whole run pins taps and imports vectors, and it is attempted exactly once with no retry. When it fails for any reason — a proxy, a dropped connection, a BOOST_SHARD_MANIFEST typo, an air-gapped mirror path — the user is told the wrong thing ("no published shards", i.e. the project has none) and is not told the right thing (the hint boost already wrote for this case). On the live path they are then told nothing further, so a run that quietly skipped the entire vector step ends in a green tick. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ cmd_quickstart catches every BoostError from shards.fetch_manifest() and prints exc.m…
PlannedOnboarding · Bug

With every registry unreachable, quickstart prints "✓ indexed 0 items" and "✓ ready", exits 0 — and the command it recommends exits 1

Measured. On a machine that cannot reach any registry, boost quickstart fails 0-for-7 clones, prints "✓ indexed 0 items for keyword search" and "✓ ready — try boost search brainstorming", and exits 0 — while the very next line of README's own install snippet (README.md:22-23 is boost quickstart followed by boost search) exits 1 with "Error: no taps configured — nothing to search"; cmd_quickstart has no failure counter at all, so out.ok at quickstart.py:197 and :220 and return 0 at :221 are unreachable-by-failure. Reproduce it. cd <repo>
export HOME=$TMPDIR/audit-qs-f2 ; export BOOST_HOME=$HOME/.boost ; mkdir -p "$HOME"
printf '[url "https://no-such-host.invalid/"]\n\tinsteadOf = https://github.com/\n' > "$HOME/.gitconfig"
BOOST() { .venv/bin/python -c "import sys;sys.path.insert(0,'<repo>');from boost_cli.cli import main;sys.exit(main(sys.argv[1:]))" "$@"; }
BOOST quickstart ; echo "QUICKSTART_EXIT=$?"
BOOST search brainstorming ; echo "SEARCH_EXIT=$?"
BOOST doctor | tail -6 What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Two process nits in the finding's own prose; neither is a defect number and neither belongs on a card. 1. not_carded_check says "Read all 318 ^title: lines". The real count is 432: ls docs/roadmap/items/*.md | wc -l = 432 and grep -h '^title:' docs/roadmap/items/*.md | wc -l = 432. The conclusion still holds — I re-ran the dedupe check myself over all 432. 2. evidence says "lines 195-220 run unconditionally and end return 0". return 0 is line 221; the file is 221 lines long. Lines 195-220 are correct for the unconditional block; the return is one line past it. Every defect-bearing figure is right: quickstart.py:91 is the out.warn("could not tap %s: %s"...) + continue; :197 is out.ok("indexed %s items for keyword search"); :220 is out.ok("ready — try \boost search brainstorming\"); :3 is the "one command from empty machine to working search" docstring; config.DEFAULT_TAPS is exactly 7 registries in the order printed; tests/functional/test_cli_quickstart.py:217 is test_it_reports_each_outcome_and_survives_a_bad_registry with assert both.count("tapped ") >= 2 at :224 and assert "ready" in both at :225, and it passes today. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Dedupe check I ran myself: only 5 of the 432 item files mention quickstart. audit-quickstart-findings.md is unpinned-taps-without-[rag] plus the three [rag] hint wordings — no exit codes, no success lines. shard-refresh-skips-processed-commits.md is rerun re-downloads. published-shards-have-no-consumer.md, publish-the-keyword-index.md and the stale-prose card are the 463/464 count and shard reachability. I also read the exit-0 family (audit-verify-drift-…exit-0, audit-clean-counts-failed-removals-…exits 0, audit-tap-findings, one-dead-tap-broke-every-update) — all per-command, none quickstart. Not carded. Two things a card author must not get wrong: 1. The fix condition is "no taps configured afterwards", not "zero clones succeeded". registry.add_many returns skipped: True for a registry already tapped, and _tap_defaults returns names (every selected registry, tapped here or not) — it discards outcomes entirely. So on a clean rerun where all 7 are already tapped, *zero* results carry ok: True. A naive count(ok) == 0 -> return 1 would make every successful rerun exit 1. Gate on registry.list_taps() being empty (or stats["entries"] == 0) instead. I ran test_it_reports_each_outcome_and_survives_a_bad_registry and it passes today; that test asserts both.count("tapped ") >= 2 and "ready" in both, so a fix that only fires at total failure keeps it green, as the finding says. 2. Why it is worth doing. This is the first command a new user runs and the first one a Dockerfile, CI job or setup script runs. Exit 0 means every automated caller records a successful install of a boost that cannot search anything; a human sees seven warnings scroll past and two green ticks at the bottom, which is the shape that reads as success. The command's own docstring promises "working search" and delivers zero items while claiming ready. A fix is a failure counter: when 0 taps succeeded, say so and return non-zero — leaving the partial-failure behaviour (and its existing test) untouched. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CONFIRMED. No fix is prescribed here — the measurement is the contribution.

Complexity L Impact High Wow ★★★★ cmd_quickstart warns per failed clone but never tracks failures: it unconditionally r…
PlannedOnboarding · Bug

The shard download is invisible both before and during: --catalog --dry-run never names the 1,604.8 MB, and the live fetch passes no progress callback and has no spinner

Measured. On a virgin HOME, boost quickstart --catalog --dry-run prints exactly two lines — "would tap 464 registries (459 pinned to a published shard's commit)" and "would build the keyword index, then import 459 shard(s)" — and never names the 1,604.8 MB (1,604,753,775 bytes) those same 459 manifest rows sum to, although the dry-run has already read the manifest that carries every row's bytes and shards._size_label() exists to format it. Reproduce it. cd <repo>
grep -n 'on_event' boost_cli/commands/quickstart.py # -> no matches
grep -n 'on_event' boost_cli/commands/discovery.py boost_cli/commands/pkg.py
sed -n '195,215p' boost_cli/commands/quickstart.py # Spinner at 195, bare sync at 209
curl -sSL -o $TMPDIR/mf.json https://github.com/jonnyeclectic/boost/releases/download/shards-latest/manifest.json
python3 -c "import json;d=json.load(open('$TMPDIR/mf.json'));print(len(d['shards']),'rows', '%.1f MB' % (sum(r.get('bytes',0) for r in d['shards'])/1e6))"
export HOME=$TMPDIR/audit-qs-f3 ; export BOOST_HOME=$HOME/.boost ; mkdir -p "$HOME"
env HOME="$HOME" BOOST_HOME="$BOOST_HOME" BOOST_SHARD_MANIFEST="file://$TMPDIR/mf.json" \
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. One stated detail is wrong: "shards.sync() … both other callers pass one." shards.sync has THREE call sites, and only ONE passes on_event: - boost_cli/commands/discovery.py:409 (sync) — passes on_event=None if args.as_json else _shard_event at :414. ✓ - boost_cli/commands/quickstart.py:209 (sync) — passes none, no Spinner. ✗ - boost_cli/commands/pkg.py:941 (_resync_vectors) — shards.sync(list(by_name), by_name, manifest=manifest), passes NO on_event and is inside no Spinner either. ✗ The finding's second citation, pkg.py:997, is the on_event= line of a shards.ingest( call that starts at pkg.py:993 — not a sync caller. Both quoted line numbers (414, 997) are accurate as lines; the framing "both other [sync] callers pass one" is not, and so is the implied "quickstart is the lone outlier". Everything else re-derived and correct to the digit: 459 manifest rows; 1,604,753,775 bytes = 1604.8 MB (1530.4 MiB); the five per-default byte counts and chunk counts verbatim; expo/skills and K-Dense-AI/scientific-agent-skills have NO ROW; 17,088,844 = 17.1 MB for the seven defaults; the two dry-run lines verbatim including "464 registries" / "459 shard(s)"; the --catalog help text verbatim. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE LIMITS OF MY REPRO — a card must not overstate these: 1. The live silence was measured on the SEVEN DEFAULTS only, and the sandbox proxy truncates response bodies (every shard ended "failed verification" — bytes were received and hashed, then rejected). My 2.19 s gap ([3.93s] "indexed 962 items" -> [6.12s] first shard line) is a LOWER BOUND; a complete 17.1 MB fetch takes longer. The finder measured 2.53 s and 1.98 s — same shape. 2. --catalog was NEVER exercised live by me (it would pull 1.6 GB, and the proxy truncates). 1,604.8 MB is manifest arithmetic — the sum of bytes over all 459 rows — not a stopwatched transfer. It is the right number for a virgin HOME, where no row is "current" and all 459 download, but write it as a sum, not as a measurement. 3. The dry-run repro requires dense.have_backend() True (the repo .venv has [rag]). Without the extra the manifest is never fetched and the dry-run prints a different "0 because …" line — a different defect, already carded in audit-quickstart-findings.md. 4. The live run needs the REAL manifest URL. With BOOST_SHARD_MANIFEST=file://… the host check refuses every shard ("shard URL … is not on the manifest's own host") and no download is attempted, so that override is fine for the dry-run repro but useless for timing the live path. FIX SCOPE IS WIDER THAN THE FINDING SAYS: because pkg.py:941 (_resync_vectors) is also a silent shards.sync caller, a fix that only touches quickstart leaves two of three sync sites inconsistent. Why it is worth doing. --catalog is the one expensive, mostly-irreversible decision quickstart offers a brand-new user, and both surfaces that exist to describe it — --help and --dry-run — omit its dominant cost. A user on a laptop tether or a metered link is asked to approve "459 shard(s)" with no way to learn that means 1.6 GB, and once it starts there is no spinner, no per-shard line and no byte counter to tell them how far along it is or that anything is happening at all. Every ingredient of the fix is already in the file: pass _shard_event-style on_event, and sum row['bytes'] through the existing _size_label in the dry-run line. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ shards.sync() takes an on_event progress callback and both other callers pass one; qu…
PlannedQuality · Docs

README's "81 commands" table enumerates only 80 — the missing one is quickstart, the README's own first command

Measured. Commit dd2fba07 (PR #594, "publish prebuilt vectors where a new user can reach them") added ("quickstart", "cfg", ...) to cli.py's COMMANDS and changed README.md:389 from "## 80 commands" to "## 81 commands" — and left README.md:404's Configuration row at its 13 names, so the README has advertised 81 commands while listing 80 ever since, with the whole test suite green, because test_marketing_counts.py asserts the integer and test_docsite_chrome.py:63-71 pins the enumeration only for docs/index.html. Reproduce it. cd <repo>
.venv/bin/python - <<'PY'
import re, sys
sys.path.insert(0,'.')
from boost_cli.cli import COMMANDS
names = {n for n,_,_,_ in COMMANDS}
readme = open('README.md').read()
rows = re.findall(r'^\| ([A-Z][^|]*?) \| (.+?) \|$', readme, re.M)
tbl = {g.strip(): [c.strip() for c in cmds.split('·')] for g, cmds in rows if '·' in cmds}
listed = {c for v in tbl.values() for c in v}
print("README table total:", sum(len(v) for v in tbl.values()), "COMMANDS:", len(names))
print("missing from README table:", sorted(names - listed))
PY
grep -n '81 commands' README.md
./boost --help | grep -n 'quickstart' What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The defect itself reproduces exactly as stated — README.md:389 heads "81 commands", the eight rows at README.md:398-405 enumerate 80, and the set difference against cli.py's COMMANDS is exactly {'quickstart'}. Two stated facts are wrong: 1. LOAD-BEARING — why_it_matters says "The README table is the only place a reader browses the command surface without running the CLI." False. docs/commands.html (4 quickstart hits) and docs/index.html (3 hits, including the pinned ["quickstart","cfg",...] row) both enumerate the full surface, and README.md:391-393 links both of them two lines ABOVE the table. A card must not ship that sentence. The honest framing: the README table is the surface a reader browses in-file, and the one enumeration of the three that nothing pins. 2. SOFT — "the one docs/semantic-search.md points at three times". Three is only right if you count fenced-code invocations (lines 30, 63, 64). The string boost quickstart appears 4 times (30, 33, 63, 64) and quickstart appears on 7 lines (+1, 83, 89). Say "four boost quickstart references across seven mentions" or drop the number. 3. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO: entirely static — README.md, boost_cli/cli.py, docs/*.html, the two test files, and ./boost --help from the repo checkout. No corpus, tap set or network was needed, so nothing here is contingent on the 20-tap eval corpus or on which registries ship by default. Fully settled. WHY Low, not Med. The finder's Med overstates it. quickstart is named in README.md five other times (line 23 hero, 142, 145, 151, 176), in ./boost --help, and in both generated doc pages that the README links immediately above the table. The user-visible cost is one missing name out of 81 in one browsable list; nothing breaks and no one is misinformed about whether the command exists. The finder's line "a user who reads the hero sees a command the table says does not exist" is rhetorical — the table omits, it does not deny. WHY IT IS STILL WORTH A CARD. The value is structural, not the symptom: of the three enumerations of the command surface, docs/index.html is pinned name-by-name against COMMANDS (test_docsite_chrome.py:63-71), docs/commands.html is generated from COMMANDS and gated by build_command_reference.py --check, and README's table is hand-maintained with only its integer asserted. PR #594 is the proof that the gap is live rather than theoretical. Why it is worth doing. The README table is the only place a reader browses the command surface without running the CLI, and the one command it drops is the one the same file's hero block makes step 2 of onboarding. A user who reads the table sees no quickstart, and a user who reads the hero sees a command the table says does not exist. The count and the list also contradict each other in the same heading, which is exactly the class of drift test_marketing_counts.py was written for — it just guards the integer and not the enumeration. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity S Impact Low Wow ★★ README.md:389 heads a table "## 81 commands, organized into 8 groups", but the eight …
PlannedQuality · Docs

README's hero block ends in exit 1: boost install tdd-workflow names a skill no starter registry ships

Measured. In a virgin HOME after boost tap --defaults (the identical 7 registries boost quickstart taps, 962 items), the README's fourth hero line boost install tdd-workflow exits 1 with "no skill named 'tdd-workflow' in any tap" while boost install test-driven-development exits 0 from those same 7 taps — the README is wrong by one word, and its error's three close-match hints (eas-workflows, python-llm-ml-workflow-cursorrules-prompt-file, secure-workflow-guide) all steer away from the skill that would have worked. Reproduce it. export HOME=$TMPDIR/audit-docs-onboarding && export BOOST_HOME=$HOME/.boost && mkdir -p "$HOME"
cd <repo>
./boost tap --defaults # == the set boost quickstart taps (quickstart.py:44)
./boost taps | tail -2 # 7 taps · 962 items
./boost search tdd # 11 matches, no tdd-workflow row
./boost install tdd-workflow ; echo "EXIT=$?" # exit 1, "no skill named 'tdd-workflow' in any tap"
# network-free confirmation that the name is not unique anywhere either:
BOOST_HOME=$TMPDIR/eval-home ./boost info tdd-workflow ; echo "EXIT=$?" What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Two stated details are wrong; the defect itself is not. 1. "The name comes from boost's own test fixture ..., not from any real registry" and "No tap set makes the line work" are both too strong, and the finding's own eval-corpus evidence contradicts them: two real registries ship a tdd-workflow (affaan-m/ECC and sickn33/antigravity-awesome-skills), so a tap set containing exactly one of them would make the bare line succeed. Accurate wording: no starter/default tap set resolves the name, and in a wider corpus the bare name is ambiguous (exit 1) rather than absent. 2. docs/index.html:988-1002 is off. The step-3 block is <pre id="install-code"> at line 989 through </pre> at line 1004; boost search brainstorming is line 995 and the "step 3" label is line 986. Use 989-1004. Nothing else needed correcting: 7 starter registries, 962 items, 11 matches, the exact error string and its three close-match hints, and every cited file:line all reproduced verbatim. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. - Provenance is proven, not inferred. git log -S 'boost install tdd-workflow' -- README.md returns exactly one commit: 6ffbf14a, "docs(marketing): correct the front-page numbers, pin them, and automate the demo (#245)" — the shipped card docs/roadmap/items/refresh-the-marketing-surface.md, whose body says "The README hero now leads with ... a four-command block". That commit message also states every command in demo.tape was verified "against a fixture", and docs/demo.tape indeed runs python3 tests/make_fixture.py then boost search tdd / boost install brainstorming. The fixture is where tdd-workflow is defined (make_fixture.py:90). So refresh-the-marketing-surface is the ORIGIN of this defect, not a card covering it — duplicate_of stays empty. - Not carded. I read all 432 titles and grepped item bodies myself: tdd-workflow appears only in browse-could-not-search-two-words.md:31 and cli-output-ignored-the-terminal.md:21, both incidental. "hero" appears only in BOOST-D03.md:16 (terminal gradient) and docsite-chrome-and-content-audit.md:38 (roadmap.html header). audit-quickstart-findings.md is about unpinned taps + three [rag] hint wordings; prerequisites-and-semantic-search-setup.md (shipped, PR 364) is about README's semantic-search section, search's engine note and MCP instructions. Neither names README.md:21-26. - ADDITION the finder missed: line 23's comment "loads prebuilt vectors" is false in the same block for the same reason as line 24 — boost quickstart --dry-run prints "would import 0 shard(s) / 0 because semantic search needs the extra". Why it is worth doing. This is the very first thing a new user copies out of the README, immediately under pipx install boost-skill-cli. Three of the four lines work and the fourth prints an error with three irrelevant close-match suggestions, so the reader's first impression of boost is that install is broken or that the docs are untrustworthy. It is also self-inflicted: the name is a test-fixture identifier that leaked into the marketing surface, and index.html already demonstrates the correct pattern (brainstorming, which resolves against the same starter set). Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity L Impact High Wow ★★★★ The first code block in README.md (lines 21-26) is the four-command get-started seque…
PlannedUX · Bug

boost recommend sizes the description cell per row from that row's because: text, so neither column lines up

Measured. At COLUMNS=100 the eight boost recommend rows truncate their descriptions at eight per-row widths spanning 34 to 54 columns — a 20-column swing, the widest cell 59% wider than the narrowest — because desc_w is recomputed inside the row loop from that row's own because: tag length (boost_cli/commands/discovery.py:1062); every rendered cell matched the predicted desc_w to the character and every description ended in an ellipsis, so this is column geometry, not natural description length. Reproduce it. cd <repo>
export HOME=$TMPDIR/audit-dbi-sweep; mkdir -p "$HOME"; export BOOST_HOME=$TMPDIR/eval-home BOOST_NO_AI=1
COLUMNS=100 ./boost recommend > $TMPDIR/rec.txt 2>&1; cat $TMPDIR/rec.txt
python3 - <<'PY'
import re,os
for line in open(os.environ["TMPDIR"]+"/rec.txt").read().splitlines()[1:]:
m=re.match(r"^ (\S+)\s+(.*?)\s\sbecause: (.*)$", line)
if m: print("desc_cell=%3d because_len=%2d row_len=%d %s"%(len(m.group(2)),len("because: "+m.group(3)),len(line),m.group(1)))
PY
sed -n '1061,1065p' boost_cli/commands/discovery.py
sed -n '638,644p' boost_cli/core/output.py What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The geometry numbers are all CORRECT — I re-derived every one (cells 46/38/46/42/54/34/46/54, because-lengths 19/27/19/23/11/31/19/11, spread 34..54 = 20 columns, row_len 100 on all 8 rows). Four wording/detail claims are wrong and must not ship as written: 1. FALSE AS STATED: "recommend is also the one result table in the find group that does not go through the width-aware shared out.table()." cmd_search does not call out.table() either — it calls out.search_layout() at boost_cli/commands/discovery.py:219 and renders through SearchLayout. out.table() in the find group is called by cmd_index (:857), cmd_discover (:954), _browse_plain (:1091, used by browse) and cmd_trending (:1925, :1949). The correct claim: recommend is the only find-group result renderer that uses neither out.table() nor a frozen SearchLayout. 2. TITLE OVERSTATES: "so neither column lines up" — and the evidence line "row_len 100 on all 8 — every row is right-flush, nothing is a column" contradicts itself. The because: column's RIGHT edge IS flush (col 100 at COLUMNS=100, col 120 at COLUMNS=120, on all 8 rows). What wanders is (a) the because: column's LEFT edge, 69..89, and (b) the description truncation point, 34..54. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. PROVENANCE (I checked; it partly softens the framing and the card should say so). git blame puts line 1062 in ff61dfeb, "fix(search): width-clamp & truncate results — the D05 overflow fix (#33)", whose message states the intent: "clip each description to the remaining terminal width, so every result stays one scannable line with its trailing tag (★ curated / because:) intact." So preserving the trailing tag was deliberate; the per-row recompute is what that intent produced. The defect is drift, not carelessness: search was later rewritten onto the frozen SearchLayout (7f1a9a8d, "feat(ui): one design system across search and browse", #540) whose docstring states the opposite rule, and recommend was left on the 2026-07 scheme. Neither commit message nor any comment claims right-alignment of the reason tag as a design goal for recommend, so nothing in the tree defends the current shape — but a card that calls it an oversight without naming #33's stated intent is overclaiming. SCOPE OF MY REPRO. Measured only from the boost repo (detected stack: javascript, python · frameworks: pytest · also: ci) against the shared read-only 20-tap eval corpus at $TMPDIR/eval-home, with BOOST_NO_AI=1 (which suppresses the "AI picks" block — unrelated to the geometry). Which eight rows appear, and therefore the exact cell widths, depend on that corpus and that stack; the geometry itself is deterministic from :1062 and appears whenever the shown rows carry different because: sets. Why it is worth doing. recommend is the surface a new user hits before they know any skill names, and its output is a ragged block rather than a scannable list — the description truncation point jumps 20 columns row to row for reasons the reader cannot see (it is the length of the reason tag on the right). Two shipped design items (BOOST-D08, BOOST-D24) made out.table() width-aware for ~28 call sites; this one never adopted it, and the frozen-plan rule that search states in its own docstring is inverted here. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity S Impact Low Wow ★★ cmd_recommend hand-rolls its rows and computes desc_w = max(cols - 2 - width - 2 - (l…
PlannedQuality · Retrieval eval

The corpus refresh re-baselines only golden.jsonl, so golden-natural.jsonl's baseline silently describes a corpus that no longer exists

Measured. On the corpus tests/eval/taps.txt pins today (10,731 entries, 20 taps), eval_retrieval.py --golden tests/eval/golden-natural.jsonl -k 10 exits 1 and prints "REGRESSION vs baseline: catalog.search recall@k: 0.080 -> 0.060 (-0.020)" — while the keyword set on that identical corpus exits 0 and reproduces its baseline to four decimals (0.841/0.484/0.607/0.655), because the September refresh commit cbc0a58b moved golden.jsonl's baseline and left golden-natural.jsonl's describing the 10,152-entry corpus it replaced. Reproduce it. cd <repo>
# 1. the refresh re-baselines with no --golden, and the default is golden.jsonl
sed -n '112,119p' .github/workflows/eval-corpus-refresh.yml
grep -n 'DEFAULT_GOLDEN\|--golden' scripts/eval_retrieval.py
# 2. the refresh commit changed ONLY the golden.jsonl set
git show cbc0a58b -- tests/eval/baseline.json | grep -nE 'golden|^[-+] *"(recall|hit)'
# 3. the natural baseline still holds pre-refresh numbers
python3 -c "
import json;s=json.load(open('tests/eval/baseline.json'))['sets']
for k,v in s.items(): print(k, v['engines']['BM25 full-content'])"
# 4. and they reproduce exactly on the PRE-refresh corpus
export BOOST_HOME=$TMPDIR/eval-home
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The mechanism and every load-bearing measurement are correct. Four stated details are wrong: 1. LINE NUMBER. if drop > eps: is at scripts/eval_retrieval.py:545, not 546. (The other three cited lines are exact: DEFAULT_GOLDEN at 66, --golden default at 656, --regression-eps default 0.02 at 668, and the workflow's bare --save-baseline at eval-corpus-refresh.yml:119.) 2. ROADMAP FILENAME. The finding's not_carded_check names nothing-refreshes-the-eval-corpus-pins; no such file exists. That string is the card's TITLE. The file is docs/roadmap/items/eval-corpus-pins-have-no-refresh-path.md (status: shipped, pr: 431). Its conclusion still stands — I read it, and its only baseline sentence is "regenerate baseline.json", singular, with nothing about which sets a refresh maintains. 3. TITLE COUNT. "Checked all 318 titles" — there are 432 item files (405 board: code, 27 board: design). I re-ran the not-carded check over all 432 and reached the same conclusion. 4. UNDERSTATED, NOT OVERSTATED — the important one. The finding frames the false regression as a future risk ("a 2-query shift (0.040) reports a confident REGRESSION"). It is not future. On the corpus the current taps.txt pins, the natural set ALREADY exits 1 today with REGRESSION vs baseline: catalog.search recall@k: 0.080 -> 0.060 (-0.020) — the drop computes to 0.020000000000000004, which clears eps=0.02. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO — what it does and does not settle. - It settles the mechanism completely: I read the current workflow and argparse default, walked every commit that ever touched baseline.json, and confirmed cbc0a58b is the first to move one set of two. - It settles the consequence empirically, which the finding did not: I built the post-refresh corpus (network fetch of the 8 moved pins onto copies of the shared clones, in HOME=$TMPDIR/verify-5, since deleted) and got a real exit-1 false regression on the natural set with a clean exit-0 control on the keyword set. That pair is the card's evidence — one command, two sets, same corpus, opposite results. - Measurement noise seen: BM25 MRR on the natural set came out 0.235 on the --build run and 0.237 on the next run against the same index. Do not quote the natural set's post-refresh MRR to three decimals in a card. recall@k (0.360), hit@1 (0.160) and the catalog.search regression line were stable across both runs. BLAST RADIUS — do not overstate it in the card. The natural set is NOT wired into any gate. make eval and CI run golden.jsonl only, and both pass --regression-eps 1, so the required gate is untouched and stays green. grep for golden-natural across *.py/*.yml/Makefile/*.sh finds it only in tests/unit/test_eval_baseline.py and test_eval_grading.py (which test the harness, not the corpus) plus CLAUDE.md and roadmap prose. Why it is worth doing. CLAUDE.md documents baseline keying as the fix for exactly this class of bug: "Before that, running the natural-language set printed eight confident 'REGRESSION vs baseline' lines that were only the gap between two different question sets." The key was made query-set-aware but not corpus-aware, and the monthly refresh only maintains one of the two sets. The natural set is the only instrument the project has for measuring whether a user's plain-English question finds the right skill (it scores BM25 at recall 0.360 / hit@1 0.160, and 0.077 / 0.000 on its 13 workflow rows) — so it is the set that matters most for judging the keyless-dense work, and its reference point is now silently wrong. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ eval-corpus-refresh.yml:119 runs eval_retrieval.py --save-baseline -k 10 with no --go…
PlannedQuality · Docs

The two roadmap boards' install footers still advertise Python 3.9+, four minor versions under the real floor

Measured. On stock macOS python3 (3.9.6), the exact command in both roadmap footers succeeds silently and installs the wrong software: pip download --no-deps boost-skill-cli exits 0 with no Requires-Python notice and selects boost_skill_cli-1.0.392, because 390 of 557 published releases still declare requires_python: >=3.9 — 167 releases behind the current 1.2.103. Reproduce it. cd <repo>
for f in docs/*.html; do printf '%-22s %s\n' "$(basename $f)" "$(grep -m1 -n 'foot-note' $f)"; done | grep -o 'Python 3\.[0-9]*+' | sort | uniq -c
grep -n 'requires Python 3.9' docs/roadmap.html docs/design-roadmap.html
sed -n '206p' docs/design-roadmap.html
grep -n 'requires-python' pyproject.toml
.venv/bin/python scripts/build_roadmap.py --check ; echo "EXIT=$?" # green despite the drift What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The documented defect (claim + evidence) reproduces byte-for-byte — every file, line number and count is right: 9 of 11 footers say 3.12+, roadmap.html:17141 and design-roadmap.html:582 say 3.9+, design-roadmap.html:206 says "targets Python ≥ 3.9", pyproject.toml:22 is >=3.12, README.md:54 says 3.12+, and build_roadmap.py --check exits 0. The correction is confined entirely to why_it_matters, which is wrong three times: 1. WRONG: "the launcher on that interpreter dies with a SyntaxError out of core/workflows.py's match statements." The ./boost launcher no longer gates at 3.9 — grep -n '3\.9' boost returns nothing (exit 1), and boost:5, boost:27 and boost:37 all say 3.12. Observed on a PATH holding only Python 3.9.6, the launcher prints boost: Python 3.12+ is required but was not found on PATH / hint: brew install python3. It never selects the 3.9 interpreter. 2. WRONG even if the gate is bypassed: importing the package on 3.9 does not raise SyntaxError from core/workflows.py's match statements. It raises ImportError: cannot import name 'UTC' from 'datetime' at boost_cli/core/util.py:15, which fires before workflows.py is reached. 3. WRONG about the failure being visible at all. The footer's advertised path is pip install boost-skill-cli, not the launcher. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO: everything above was measured on this checkout at the session's HEAD (main, 9b70fc8e) and against live PyPI. No eval corpus or BOOST_HOME was involved — this is a static docs/metadata finding, so the 20-tap corpus caveat does not apply. I did not run a real pip install (only pip download, which exercises the same resolver and version selection without mutating anything). NOT CARDED — confirmed independently. I grepped all roadmap item titles, then every item body containing 3.9 (21 files) and every item body containing footer/foot-note (16 files). None names the docs/*.html install footers or their Python version. Three near misses, all genuinely distinct: - audit-root-findings.md covers the *launcher's* three 3.9 sites and explicitly closes "README already says 3.12+ — no doc change needed"; it never mentions a docs page. - python-floor-moves-to-312.md enumerates pyproject, the CI matrix and telemetry; no docs page. - promote-nav-footer-into-the-shared-style-system.md is about nav/footer *presence and styling*, not their text. TRAP FOR THE CARD AUTHOR — do not re-import the SyntaxError claim. The finder took it from docs/roadmap/items/audit-root-findings.md:36-43, which is still status: planned but whose premise is already fixed on disk: that card says the launcher "still gates at Python 3.9" at boost:5/:27/:37, and all three now say 3.12. Anyone who reads that card while writing this one will copy a consequence that no longer happens. Why it is worth doing. Both boards are linked from the README ("Roadmap" section, README.md:490-494) and from the Visual Guide nav, so they are landing surfaces a reader can arrive on first. Their footer is the only install instruction on the page, and it tells the reader a Python 3.9 will do. It will not: stock macOS python3 IS 3.9, and the launcher on that interpreter dies with a SyntaxError out of core/workflows.py's match statements. The design board's "targets Python ≥ 3.9" also mis-sets the floor for anyone writing a proposal against it. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity S Impact Low Wow ★★ docs/roadmap.html:17141 and docs/design-roadmap.html:582 both close with Install with…
PlannedUX · Bug

search caps the name column at 32 and the tap column at 20 at every terminal width, while the description column grows without limit

Measured. At COLUMNS=300 the same two doc-coauthoring rows render their tap as anthropics/skills and sickn33/antigravity… in search while browse prints both taps in full on the identical terminal — and ./boost info 'sickn33/antigravity…:doc-coauthoring' returns "Error: no tap named 'sickn33/antigravity…'", so the qualifier info demands cannot be built from the row search shows. Across the 10,152-entry corpus, 714 rows (7.03%) have at least one clipped copy target — a name over 32 cells, or a required tap qualifier over 20 cells — and because name_w and tap_w are min(..., 32) and min(..., 20) with no width term, that 7.03% does not shrink at any terminal width, while the description column grows from 27 cells at 84 columns to 443 at 500. Reproduce it. cd <repo>
export BOOST_HOME=$TMPDIR/eval-home
python3 -c "import sys;sys.path.insert(0,'.');from boost_cli.core import output as o;[print('cols=%-4d name_w=%d tap_w=%d desc_w=%d'%(c,(l:=o.search_layout(c,['doc-coauthoring'],['skill'],['sickn33/antigravity-awesome-skills'])).name_w,l.tap_w,l.desc_w)) for c in (84,140,300,500)]"
COLUMNS=300 ./boost search "wordpress seo blogwriting" | head -1
./boost info 'wordpress-centric-high-seo-optim…'
COLUMNS=300 ./boost search doc-coauthoring | sed -n '2,3p'
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Three cosmetic errors; none touches the defect or any headline number. 1. The finder's layout table is labelled as evidence for BOTH caps but shows name_w=15 at every width, because it passes the 15-char name doc-coauthoring. That table demonstrates only the tap cap. The name cap needs a >32-char name; my second table (name_w=32 at cols=84/140/300/500) is the missing half. 2. The finder's repro line pastes wordpress-centric-high-seo-optim… (an extra m); the cell search actually renders is wordpress-centric-high-seo-opti… (31 chars + ellipsis = 32 cells). Both error, but the repro string is not what is on screen. Use the 31+ellipsis form. 3. "browse's plain table prints it in full" — the BEHAVIOR is correct and I verified it, but there is no --plain flag on browse (Error: unrecognized arguments: --plain). The plain table is what browse falls back to when stdout is not a TTY. Also add boost list and boost taps, which likewise print full tap names via the width-aware out.table. Everything else re-derived exactly: 487 / 379 / 77.8% / 25 into 10 / 10,152 / 162 / 333 / 3.28% / 102 / 17 of 20 (85%) / real name 54 chars. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE LIMIT ON THE 25-into-10 COLLISION — the most important caveat for a card author. That figure is a property of the NAME LIST in boost_cli/data/registries.json, not an observed screen. I measured the 20-tap eval corpus and found ZERO colliding tap cells ({}). Nobody has seen two identical anthropics/claude-c… cells side by side; it requires a machine that has tapped both repos and a query that hits both. So the finding's why_it_matters sentence "two rows from different repos are indistinguishable on screen" is an INFERENCE, not a measurement — a card must word it as "25 of the 487 shipped registries would render as one of 10 identical cells if co-tapped", never as an observed fact. The DEMONSTRATED defect is truncation-breaks-paste (the doc-coauthoring chain above), which is fully reproduced. MITIGATIONS (why Med and not High): every failure path recovers in one command. The name miss prints closest matches, which included the real 54-char name. The ambiguity error prints BOTH full tap names. search --json carries the full tap verbatim (verified: ('doc-coauthoring', 'sickn33/antigravity-awesome-skills')), and browse/list/taps print both identifiers in full. Why it is worth doing. search is the front door and the row is what a user copies. At real catalogue scale its provenance column stops being an identifier: 25 shipped registries render as one of 10 identical cells, so two rows from different repos are indistinguishable on screen — and the tap is precisely what disambiguates the 102 names that live in more than one tap. The information exists (search --json carries the full tap, browse's plain table prints it in full, and the ambiguity error names both taps), so the claim is that search's human path withholds the copy target, not that it is unreachable — but the user only learns that after the command they typed has failed. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CONFIRMED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ output.search_layout (output.py:668-690) hard-caps name_w at 32 and tap_w at 20 and h…
PlannedUX · Bug

The relevance meter and its "one gradient moment" are constant on the default result page: 138/150 rows full bars, 150/150 the same colour

Measured. Over 10 real queries at the default --limit 15 against the 10,152-entry eval corpus, 138 of 150 rendered rows draw an identical full ▰▰▰▰ bar and 150 of 150 land in the same cyan band — 8 of the 10 queries render a byte-identical meter on every single row — because frac = score / max(shown score) needs frac < 0.875 to drop one of four bars while BM25's top-15 spread is only 2.2%-19.8% (min/top ratio 0.802-0.978). Reproduce it. cd <repo>
export BOOST_HOME=$TMPDIR/eval-home
./boost search quarkus # every row draws ▰▰▰▰
./boost search 'code review' # every row draws ▰▰▰▰
.venv/bin/python - <<'PY'
import json,subprocess,collections,os
QS=["code review","quarkus","python testing","docker kubernetes deploy","react component",
"security audit","database migration","git commit message","terraform aws","documentation writing"]
bars=collections.Counter(); hues=collections.Counter(); rows=0
for q in QS:
d=json.loads(subprocess.run(["./boost","search",*q.split(),"--limit","15","--json"],
capture_output=True,text=True,env=dict(os.environ)).stdout)
top=d[0]["score"]
for e in d:
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Every primary measurement reproduced to 3 decimal places (150 rows, bars {3:12, 4:138}, hues {cyan:150}, 8/10 queries uniform, the 580-row limit-60 histogram, all ten minratios). Three secondary claims are wrong: 1. "The meter costs 7 fixed columns of every search row (output._SEARCH_FIXED = 7)" and "At COLUMNS=40 those 7 cells are 17.5% of the pane spent on a constant." WRONG. output.py:626-630's own comment defines the 7: "the 4-glyph meter, a space, the 1-column installed mark, a space." Two of the seven are the installed ● and its space — and the mark is NOT constant; it varies per row and is the one glyph in that block that carries information. It would survive dropping the meter. Correct figures: the meter is 4 glyphs = 10.0% of a 40-column pane, or 12.5% counting its separator space. Not 17.5%. 2. "the widest single element after the name". WRONG at every width where the kind column survives (cols >= 48). At the default 80 columns the real plan is SearchLayout(name_w=21, kind_w=10, tap_w=0, desc_w=36) — both the kind column (10) and the description (36) are wider than the whole 7-cell fixed block, let alone the meter's 4. At 100+ cols the tap column is 20. 3. The "Visible, unaided" excerpt pairs the LAST five rendered rows with the FIRST five scores. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO — a card author must not overstate this: 1. Measured BM25-only. The eval home has no dense store; its own footer says "semantic search is off — install the extra". That IS the default install path (a plain pip install boost-skill-cli is BM25-only), so the finding's framing is fair. But on a machine with the [rag] extra AND a built or imported dense store, retrieve_any fuses and the score reaching the meter is rag.rrf_fuse's sum 1/(60 + rank), not a BM25 score. By reasoning (NOT measured — a dense build is impossible in this sandbox, the BAAI model cannot be downloaded): a hit at rank 15 in one engine only scores ~0.41 of a two-engine rank-1 hit, which is bar 2 and violet, so the meter would show real variation there. The card should say "on a BM25-only install", not "always". 2. Corpus direction, unmeasured but worth stating honestly: the live catalogue is ~71k items with many mirror copies (the quarkus run above already shows 4 near-identical quarkus-security rows), so BM25's top-15 spread on a real install would compress further, not less. Nothing here suggests the effect is an artifact of the small eval corpus, but I did not measure a 71k install — I deliberately did not run against the user's real ~/.boost. 3. The finding's test-coverage claim is correct and can be stated for BOTH helpers, not just meter. tests/unit/test_output.py TestMeter (lines 691-714) and the meter_hue tests (lines ~1400-1421) exercise only synthetic fractions — 1.0, 0.66, 0.659, 0.33, 0.329, 0.0, and clamps at 1.5/2.5/7.5/-1.0/-3.0. Why it is worth doing. The meter costs 7 fixed columns of every search row (output._SEARCH_FIXED = 7) — the widest single element after the name — and on the default page it carries no information: it is a proportional bar over a quantity that is near-constant by construction, so it reports "top hit" for the 15th result as loudly as for the first. That is precisely the problem BOOST-D07 was raised to solve ("the ranking is invisible — every row looks equal ... Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ format_search_row renders aurora(meter(frac), meter_hue(frac)) with frac = score / ma…
PlannedUX · Bug

boost search rows fit to term_width(), not pane_width(), so a piped search silently loses the TAP column entirely

Measured. Piped, boost search orchestrator | grep -c sickn33/antigravity returns 0, while COLUMNS=200 on the identical pipe prints that tap in the row — and pane_width has 0 callers in boost_cli/commands/discovery.py against 6 out.term_width() calls, so the search row plan (discovery.py:219) is built from an assumed 80 columns that search_layout (output.py:674-676) then uses to drop the tap column entirely. Reproduce it. cd <repo>
export BOOST_HOME=$TMPDIR/eval-home
unset COLUMNS
echo '--- piped, no COLUMNS:'; ./boost search "orchestrator" 2>&1 | head -3
echo '--- COLUMNS=200:'; COLUMNS=200 ./boost search "orchestrator" 2>&1 | head -3
grep -n 'pane_width\|term_width' boost_cli/commands/discovery.py What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Four things in the finding are wrong; the defect itself is real. 1. boost_cli/core/output.py:696-699 is WRONG for the tap_w = 0; if cols >= 84: gate. The real location is output.py:674-676. Lines 696-699 are inside format_search_row's docstring. 2. Three of the six out.term_width() line numbers are wrong. Claimed "lines 219, 260, 352, 1043, 1894, 1907"; actual is 219, 260, 352, 1058, 1924, 1948. (The count of 6, and discovery.py:219 itself, are correct.) 3. pane_width's docstring is cited as output.py:325-336; the def is at 326 and the docstring runs 327-336. The quoted docstring sentence is verbatim correct. 4. "the provenance … prints fine on a wide TTY" is FALSE for most taps. search_layout caps tap_w at 20 cells (output.py:676), so at COLUMNS=200 every tap in the repro is still ellipsised — sickn33/antigravity…, first-fluke/oh-my-a…, OneWave-AI/claude-s…. 17 of the 20 eval-corpus taps exceed 20 characters. Separately, the column is present only at cols >= 84, so a real 80-column TTY drops it too — the trigger is "pane under 84 columns", not "stdout is a pipe". What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. LOAD-BEARING for a card author: switching discovery.py:219 to out.pane_width() or _UNPANED (the cli.py:178 pattern, _UNPANED = 10**6) does NOT make boost search … | grep <owner/repo> work. The 20-cell tap cap at output.py:676 still ellipses 17 of the 20 eval-corpus taps, so grep sickn33/antigravity-awesome-skills stays at 0 hits after the implied fix. The fix needs BOTH: fit to pane_width(), and lift the tap cap when there is no pane. The description half IS fully fixed by the pane change alone (descriptions currently clip to an assumed 80 in a pipe and print in full at COLUMNS=200) — a card should lead with that half, which is unambiguous. SCOPE of my numbers: "17 of 20 taps exceed 20 chars" is measured over the 20-tap eval corpus at $TMPDIR/eval-home only, not over the real 460-tap install. State it with that scope. Everything else is measured against current main source. NOT ALREADY CARDED — I re-checked independently. The PR 739 card (audit-out-table-clips-data-columns-to-an-assumed-80-columns-when-s.md, shipped) names exactly two emit sites: out.table -> _fit_widths (11 of 12 findings) and cli.print_help (cli.py:169,199-209). Search rows go through out.search_layout + out.format_search_row + out.info and are not among them. cli-output-ignored-the-terminal.md (PR 552) mentions search's tap-drop but describes it as *correct* narrow-TTY adaptation, not the pipe case. audit-search-findings.md touches search_layout only for CJK cell-width. Why it is worth doing. The tap is the identifier boost install --tap and boost untap take, and it is the only thing that distinguishes the 13 real skills named code-reviewer. A user scripting over boost search (| grep, | awk '{print $4}') sees a column that exists on their screen and vanishes in their pipeline, with no error. --json exists as an escape hatch, but the same escape hatch existed when PR 739 fixed out.table, and the | grep argument it accepted applies unchanged here. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ commands/discovery.py:219 builds the search column plan from out.term_width(), which …
PlannedQuality · Docs

The contributor-onboarding gate tables state three wrong suite sizes, and README and CONTRIBUTING disagree with each other

Measured. One command (make smoke is literally bash tests/smoke.sh, Makefile:48-49) is documented as three different sizes in three files — 176 in README.md:515, 170 in CONTRIBUTING.md:123 and 170 again in docs/openssf-badge.md:123 — while it actually runs 183 checks, a figure that is deterministic and statically derivable as 102 top-level run lines plus 81 --help entries in the CMDS heredoc. Reproduce it. cd <repo>
sed -n '48,49p' Makefile # make smoke == bash tests/smoke.sh
bash tests/smoke.sh 2>&1 | tail -1 # == results: 183 passed, 0 failed
sed -n '515p' README.md # "176 checks"
sed -n '123,124p' CONTRIBUTING.md # "170 end-to-end checks" ; "11 features, 47 scenarios"
HOME=$TMPDIR/audit-bdd BOOST_HOME=$TMPDIR/audit-bdd/.boost .venv/bin/behave tests/bdd/features --dry-run 2>&1 | tail -2 Verification found nothing to correct. Every number, file:line and command output above was independently re-derived and matched exactly. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. 1. THE FINDING UNDERCOUNTS THE BLAST RADIUS. It names two files; there are three. docs/openssf-badge.md:123 carries both stale numbers ("170 end-to-end checks" and "11 features, 47 scenarios") in the row that answers the OpenSSF test MUST criterion. That makes five stale statements across three files, not three across two. The finder's not_carded_check grepped roadmap bodies but never grepped docs/, which is how it missed this. A card that fixes only README and CONTRIBUTING leaves a stale count in the file that argues boost's test coverage to an external badge audit. 2. TWO ARE DRIFT, ONE WAS WRONG ON DAY ONE — do not write the card as uniform "drift". Re-deriving the count at each blame commit: README's 176 was CORRECT at 3aaf4ba4 (2026-08-28, PR #579) and has since drifted to 183; CONTRIBUTING's 170 was CORRECT at c043bb3f (2026-07-24, PR #233) and was never touched again; CONTRIBUTING's "47 scenarios" was CORRECT at 17bd5003 (2026-07-28, PR #292). But docs/openssf-badge.md's 170 was WRONG WHEN IT WAS WRITTEN — at 560652b1 (2026-08-28, PR #563) the suite already ran 176. It copied CONTRIBUTING's already-stale number rather than measuring. That is the sharper story: the count is not just drifting, it is being propagated by copy from whichever file the author happened to read, which is exactly the failure mode the finding's "no way to tell which is authoritative" line predicts. 3. 183 IS DETERMINISTIC — a card author must not "correct" it. Why it is worth doing. These two tables are the contributor's map of what make check will do to their PR, and a first-time contributor reads whichever file they landed on. Seeing 183 where the doc promised 176 (or 170) is a signal the gate changed under them; worse, the two files disagree, so there is no way to tell which is authoritative without running the suite. The BDD row is the same failure in a suite that is not even in make check, where a wrong size is the only signal a contributor has that scenarios were added. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CONFIRMED. No fix is prescribed here — the measurement is the contribution.

Complexity S Impact Low Wow ★★ README.md:515 says make smoke is "176 checks" and CONTRIBUTING.md:123 says bash tests…
PlannedQuality · Retrieval eval

rag.surface's de-hyphenated name copy is justified by two claims that are both false, and its real effect — an undocumented 3x name / 2x description field weight — is guarded by a test …

Measured. Deleting the "obviously redundant" de-hyphenated copy passes the ENTIRE required gate silently: zero new failures across the full unit + functional suite (the 10 functional failures are a pre-existing dashboard-design CWD leak, identical with and without the ablation), and all four eval floors clear with room to spare (ablated 0.8407 / 0.4725 / 0.6021 / 0.6523 against floors 0.78 / 0.40 / 0.52 / 0.58) — while it silently moves 23 of 141 golden rankings, flips one top-1 result, and drops go-backend-scalability out of the top 10 for the golden query "go backend scalability best practices for microservices and apis", costing exactly 1/91 = 0.011 recall@10. Reproduce it. cd <repo>
export BOOST_HOME=$TMPDIR/eval-home BOOST_NO_AI=1
# 1. the claim is false, corpus-wide
.venv/bin/python -c "
import re
from boost_cli.core import rag, catalog
print(re.split(r'[^a-z0-9]+', 'code-reviewer'))
es = catalog.all_entries()
d = sum(1 for e in es if rag.tokenize(e['name']) != rag.tokenize(e['name'].replace('-',' ').replace('_',' ')))
print('entries where de-hyphenation changes the token list:', d, 'of', len(es))
tp = rag._tap_paths()
e = [x for x in es if x['name']=='docker-expert'][0]
print('surface:', rag.surface(e)[:110])
print(\"tokenize(surface).count('docker') =\", rag.tokenize(rag.surface(e)).count('docker'))
print(\"tf['docker'] =\", rag._make_docs([e], tp)[0]['tf']['docker'])
"
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Four numeric/citation corrections. The defect itself is real and every qualitative claim reproduced. 1. LIVE recall@10 is 0.8516 (0.8516483516483516), NOT 0.8571. ABLATED recall@10 is 0.8407 (0.8406593406593407), NOT 0.8462. Both of the finding's absolutes are 0.0055 (= 0.5/91) too high. The DELTA the finding quotes ("0.011 recall@10") is correct: 0.8516 - 0.8407 = 0.010989 = exactly 1/91. Re-derive with scripts/eval_retrieval.py --golden tests/eval/golden.jsonl -k 10 --engines bm25 [--build] against the 20-tap corpus, n=91. My live number reproduces CLAUDE.md's stated 20-tap figures (0.852 / 0.473) exactly. 2. "(hit@1 matches tests/eval/baseline.json exactly)" is FALSE. baseline.json's BM25 row is recall@k 0.8406593407 / hit@1 0.4835164835 (44/91); measured live is 0.8516483516 / 0.4725274725 (43/91). I confirmed the baseline is keyed to the CURRENT query set (golden_key returns golden.jsonl@a0617183f8c9, which is the committed key), so the disagreement is corpus state, not a stale query set. A card must not cite baseline.json as corroboration. Separately: the ablated recall@10 I measured is byte-identical to baseline.json's recall@k (0.8406593406593407) while hit@1 differs — an unexplained coincidence; do not write it up as a claim in either direction. 3. Line citations drifted. surface() is rag.py:202-212 (docstring 203-209, body 210-212), NOT 200-208. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Scope and method limits a card author must not overstate: - RANKING DIFF METHOD: "top-1 changed 1/141" and "top-10 order changed 23/141" are over RAW rag.retrieve(q, k=10) name lists across 91 golden + 50 golden-natural queries. CLAUDE.md says the eval's GRADED ranked list de-duplicates on the content hash, not the name, so these raw counts are not graded ranks. Do not conflate them. Membership (as opposed to order) changed in only 3/141. - The ablation I ran drops ONLY the de-hyphenated copy (surface returns name + description). It does not remove surface() entirely; a card claiming "removing surface()" would be a different, larger experiment. - CORPUS SCOPE: everything is measured against the shared 20-tap / 10,152-entry eval corpus at $TMPDIR/eval-home, which is the corpus the eval gate itself floors against. It is NOT the user's real ~445-tap install, so the 2.64% surface-token share and the per-query ranking flips are properties of the gate corpus. I did not and could not measure the real install. - I verified the live prebuilt index was not stale by rebuilding an unablated index from the same 20 caches in a third throwaway HOME; it reproduced 0.8516483516483516 / 0.4725274725274725 bit-for-bit, so the 0.011 delta is purely the ablation and not index drift. - BOTH docstring claims were false AT THE COMMIT THAT WROTE THEM (678bbc26, PR #367): tokenize's [^a-z0-9]+ dates from 329c6eb6 and never changed, and read_body already prepended name\ndescription at 678bbc26. Why it is worth doing. The stated invariant and the code disagree, which is the failure mode this repo cards as highest-value. A maintainer reading either the docstring or the shipped roadmap card believes the tokenizer does not split hyphens — a false premise that leads either to deleting the "obviously redundant" duplicate (silently moving 23 of 141 golden rankings and costing 0.011 recall@10 on the gate's own query set, with no test failing) or to building hyphen handling that already exists. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ surface() (rag.py:200-208) indexes the name, a de-hyphenated copy of the name, and th…
PlannedUX · Bug

_fit_widths shrinks data columns to a bare "…" and still overflows: boost taps is 54 columns wide on every terminal narrower than 54

Measured. At COLUMNS=80 — a default terminal width, not a narrow split — boost hooks list renders 5 of its 6 columns (host, scope, event, name, matcher) as a bare "…" for every row INCLUDING the header, so the table no longer says what its own columns are, and the row is still 89 columns wide (94 on a color TTY) against a natural 142: five columns of data destroyed, 15 columns of ink spent on placeholders, and the fit still not achieved — dropping those five columns outright would have measured 74 and fit. Reproduce it. cd <repo>
export BOOST_HOME=$TMPDIR/eval-home
for w in 53 54 55 56 58; do printf "COLUMNS=%s: " $w; COLUMNS=$w ./boost taps 2>&1 | sed -n '2p'; done
# and the width measurement:
for w in 20 40 50 55; do printf "COLUMNS=%s widest_row=" $w; COLUMNS=$w ./boost taps 2>&1 | python3 -c "
import sys,re,unicodedata
A=re.compile(r'\033\[[0-9;]*m')
def vl(s):
s=A.sub('',s); return sum(0 if unicodedata.combining(c) else (2 if unicodedata.east_asian_width(c) in ('W','F') else 1) for c in s)
ls=sys.stdin.read().splitlines(); print(max(vl(l) for l in ls), 'over=', max(vl(l) for l in ls)-int('$w'))
"; done What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. NAME column is 39 columns wide, not 40. The finding says "NAME (40 cols) is therefore unshrinkable". Longest tap name in the corpus is composio-community/awesome-codex-skills = 39. The constant 54 decomposes as 39 + 2 + 5(ITEMS header) + 2 + 1 + 2 + 0(curated ★, empty here) + 2 + 1. 2. EVERY figure in the finding is a PIPED (no-color, sep=2) figure, and the "why it matters" is about real terminals, where use_color makes sep " │ " = 3 columns. On a color TTY the same rows are: taps floor row = 58 (not 54), and the two dead cells + separators cost 8 (not 6); hooks list floor row = 94 (not 89). Verified with BOOST_COLOR=always. A card that ships "54" describes a pane nobody is looking at. 3. Line numbers drifted. Actual, on current main (9b70fc8e): _fit_widths is output.py:809-831 (claimed 801-823); _clip_visible is 781-806 and its keep = width - len(ellipsis) if width > len(ellipsis) else 0 is line 788 (claimed 762-765); table()'s "widest text column is shrunk" docstring sentence is 856-858 (claimed 846-849). Correct as claimed: the fmt clip at 901 (claimed 900-903) and taps.py:351 (out.table at 351, keep=("NAME",) at 352). 4. The COLUMNS=20 over=34 and COLUMNS=40 over=14 numbers are NOT evidence of the floor bug and must not be used as such. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO - taps figures are the 20-tap READ-ONLY eval corpus at $TMPDIR/eval-home (longest name 39). On the real ~445-tap install the NAME column is wider (BOOST-D24's card cites K-Dense-AI/claude-scientific-skills = 44), which shifts the whole band upward. Do not ship "54" as a universal constant — ship the mechanism plus the corpus it was measured on. - hooks list figures are this machine's own ~/.claude/settings.json (2 hooks, longest command 74 chars). boost hooks list reads HOME, not BOOST_HOME, so those numbers are install-specific too. - The other four keep= call sites (quality.py:1007/1569/1596, pkg.py:1750) were not exercised — they need attestations / fingerprints / snapshots I did not create. - boost list (no keep=) degrades correctly: at COLUMNS=20 and 40 its table rows fit; the only over-wide lines are chrome hint lines, which is a different, separate issue. THREE THINGS A CARD AUTHOR MUST NOT GET WRONG 1. floor=0 is NOT the fix, and the card must not propose it. For taps the dead columns are TRAILING, so .rstrip() eats their separators and floor=0 would give 46 — a fit. For hooks list the dead columns are LEADING: 0+2+0+2+0+2+0+2+0+2+74 = 84, still over 80. Only dropping the column AND its separator (the shape search_layout already implements, output.py:651-689, with a stated drop order and an explicit "every row measures within cols for any terminal 40 cells wide or more" guarantee) reaches 74. 2. Why it is worth doing. A 50-column pane (a vertical split, a phone SSH session, a narrow tmux pane) is a realistic terminal. On it boost taps spends 6 columns printing two ellipses that say nothing, hides both the last-update date and the URL for all 20 taps, and *still* wraps every row — so the user gets neither the data nor the fit. The right answer for a column squeezed below a readable floor is to drop it (the shape search_layout already implements, with a stated drop order), not to render a placeholder that costs ink and carries nothing. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ out.table's _fit_widths has floor=1, and _clip_visible(cell, 1) returns just the elli…
PlannedQuality · Retrieval eval

Tier 3's false-call ceiling is unreachable at its own default N, and tolerates zero false calls at the N make eval-tools uses

Measured. At the argparse default --runs 1 (scripts/eval_tools.py:388), a flawless host — 8/8 should-call, 0/8 false-call — is reported FAIL: false-call rate 0.00 [0.00-0.32] over ceiling 0.20 (0/8), because the Wilson upper bound at k=0 is z²/(n+z²) = 3.8416/(8+3.8416) = 0.3244 and cannot fall under the 0.20 ceiling until n ≥ 16, i.e. --runs ≥ 2 over this 8-row half. Reproduce it. cd <repo>
.venv/bin/python - <<'EOF'
import importlib.util
spec=importlib.util.spec_from_file_location("et","scripts/eval_tools.py")
m=importlib.util.module_from_spec(spec); spec.loader.exec_module(m)
rows=m.load_set(m.DEFAULT_SET); call,nocall=m.halves(rows)
print("set: %d call rows, %d no-call rows"%(len(call),len(nocall)))
for runs in (1,2,3):
obs={r["id"]:[True]*runs for r in call}
obs.update({r["id"]:[False]*runs for r in nocall}) # a PERFECT host
met=m.score_host(rows,obs); v=m.verdict(met,0.60,0.20)
print("runs=%d call %d/%d lo=%.4f | false %d/%d hi=%.4f -> %s"%(runs,
met["call_rate"]["k"],met["call_rate"]["n"],met["call_rate"]["lo"],
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. No defect figure is wrong. Every operative number re-derives exactly: 8 call / 8 no-call rows; hi=0.3244 (n=8), 0.1936 (n=16), 0.1380 (n=24), 0.2024 (n=24,k=1); n>=16 required for k=0 to clear 0.20; ceiling tolerates 0 of 24 while the floor tolerates 4 of 24 (fails at 5 = 16.7% miss). Cited locations all correct: scripts/eval_tools.py:388 --runs default=1; docstring lines 53 and 54 pass no --runs (line 51 is --dry-run, 52 carries --runs 3, so "two of four Usage lines" is right); Makefile:176 is the eval-tools: target with --runs 3 on 177. One non-defect number is wrong: the not_carded_check says "Read all 318 ^title: lines" — there are 432 item files and 432 title lines. That is the finder's audit-process count, not a figure that would ship on a card. I redid the carded check independently over all 432 and its conclusion holds. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Scope of my repro: I drove the shipped load_set/halves/score_host/verdict directly and re-derived the Wilson algebra from scratch. I did NOT drive a real host (no claude run, no tokens spent) — that is not needed, because the defect is in the scoring arithmetic, which is fully deterministic and host-independent. Scoping the finding's one overstatement (a wording nuance, not a wrong number): "can only ever print FAIL" is conditional. main() exits 0 *before* reaching verdict() when claude is not on PATH or BOOST_NO_AI is set, and the ceiling clause is skipped entirely if f["n"]==0 (every no-call row unreachable). The precise, proven claim is: whenever the ceiling is actually evaluated at n ≤ 8, FAIL is guaranteed for every possible k — I minimised the upper bound over all k at each n from 1 to 8 and the best case is 0.3244 at n=8. Three things the finder did not report that a card author will want: 1. The test suite corroborates it by omission. tests/unit/test_eval_tools.py::test_a_perfect_host_passes asserts only the two *rates* and never calls verdict(). On its own 2-call/2-no-call fixture, verdict() returns TWO failures — including false-call rate 0.00 [0.00-0.66] over ceiling 0.20 (0/2). A test named "a perfect host passes" would fail if it asserted the thing its name claims. The only test that does clear the ceiling, test_enough_runs_do_clear_it, uses 30 runs (n=30, hi=0.1135), and its comment explains the FLOOR, not the ceiling — so the reachable-N constraint is nowhere stated in the suite either. 2. Why it is worth doing. The card that introduced this tier argues its whole value is having two numbers rather than one — a floor on should-call AND a ceiling on should-not-call — because scoring call rate alone rewards an assertive MCP surface. But the ceiling half is either unsatisfiable (N=8) or a zero-tolerance trip-wire (N=24), while the floor half absorbs a 16.7% miss rate. Anyone running the documented python3 scripts/eval_tools.py … invocation sees a red result on a host that behaved perfectly, learns the tier is broken, and stops running it; anyone running make eval-tools spends the card's own estimate of $30-50 per invocation to discover a ceiling that only ever reports 0-or-fail. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CONFIRMED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ eval_tools.py judges the should-NOT-call ceiling against the Wilson UPPER bound, but …
PlannedOnboarding · Bug

An unreadable tap cache is invisible to doctor ("✓ 1 tap cloned & cached", exit 0) while search, browse and heal all exit 70 with a crash report

Measured. A tap cache file that is fully READABLE but not writable (mode 400 — exactly what one sudo boost run leaves behind) turns the next routine CACHE_FORMAT bump into exit 70 on search, browse, info, update and heal, while boost doctor prints "✓ 1 tap cloned & cached" and verdicts "● healthy" at exit 0 — and no boost command repairs it, because boost update <tap>, the remedy doctor would name, crashes with the same PermissionError at exit 70. Reproduce it. # run as a normal (non-root) user; root ignores mode bits
cd <repo> || exit 1
export HOME=$TMPDIR/audit-doctor-verify2; export BOOST_HOME=$HOME/.boost; mkdir -p "$HOME"
python3 tests/make_fixture.py $TMPDIR/audit-doctor-fix2 >/dev/null
./boost tap $TMPDIR/audit-doctor-fix2 >/dev/null 2>&1
BOOST_ASSUME_YES=1 ./boost install brainstorming >/dev/null 2>&1
chmod 000 "$BOOST_HOME/cache/audit-doctor-fix2.json"
./boost doctor; echo "DOCTOR_EXIT=$?" # "✓ 1 tap cloned & cached" ... "● healthy", 0
./boost heal; echo "HEAL_EXIT=$?" # PermissionError, crash report, 70
./boost search brainstorm >/dev/null 2>&1; echo "SEARCH_EXIT=$?" # 70
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Three corrections; the first two make the finding STRONGER, so the card must not repeat its own understatement. 1. WRONG REMEDY. The finding says "The real fix is one line (boost update <tap>, or deleting the cache file)". boost update <tap> does NOT fix it — it crashes with the identical PermissionError, UPDATE_EXIT=70 (measured unpiped; the finder's own piped run would have shown tail's status). boost clean exits 0 and leaves the file in place (cache/*.json whose stem IS a configured tap is deliberately kept). So there is NO CLI command that repairs this state: the only remedy is manually deleting or chmod-ing the file, which no boost surface names. 2. BLAST RADIUS UNDERSTATED. The finding lists search, browse, heal. boost info <skill> also exits 70 (INFO_EXIT=70). Unaffected: list (0), clean (0), heal --dry-run (0). 3. TWO LINE NUMBERS WRONG (secondary refs only). complete.refresh_names() is at quality.py:1152, NOT :1157. Journal rotation is quality.py:1154-1160, NOT :1159-1165. All four PRIMARY line numbers are exact: quality.py:420, quality.py:1140, quality.py:1146, catalog.py:253. The claim they encode — that both post-rebuild steps sit after the crash point — is correct. 4. TRIGGER IS WIDER THAN chmod 000 (in the finding's favour, and this is the number the card should lead with). The file need only be NON-WRITABLE, not unreadable. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE LIMITS OF MY REPRO — a card author must not overstate these: - A read-only cache DIRECTORY does NOT trip this. I measured chmod 500 on ~/.boost/cache with a stale format: search exited 0. POSIX only needs directory write permission to create or unlink, not to rewrite an existing file, and rebuild_tap overwrites in place. Do NOT describe the trigger as "a read-only ~/.boost" or "a read-only mount" — the trigger is specifically the cache FILE being non-writable by the process. - I could not test the root-owned-file case (no sudo in this sandbox). The sudo boost provenance story is the plausible real-world cause of a non-writable cache file, but it is reasoning from the mode bits I set by hand, not a measured provenance. State it as "what a root-owned cache file looks like", not as an observed user report. - Disk-full (ENOSPC) would hit the same unguarded write_text and is the other realistic trigger, but I could not simulate it. Do not put a number on it. - All exit codes in reproduced_output are unpiped. My first contrast run piped through tail, so those exit codes measured tail; I re-ran every one of them bare. The finder's evidence has the same piping hazard, which is how the boost update remedy claim survived unchecked. - Python here is 3.14.7 (Homebrew). Why it is worth doing. A user whose search has stopped working runs the two commands the CLI offers for exactly that: doctor tells them the machine is healthy, and heal — the self-repair command that owns cache rebuilding — crashes with a stack-trace-shaped error and an invitation to file a GitHub issue. The real fix is one line (boost update <tap>, or deleting the cache file) and nothing names it. Because rebuild_tap runs unconditionally for every tap, one bad cache file among hundreds takes the whole heal run down and silently skips journal rotation and completion refresh. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ Doctor's tap check is elif not tap.cache_file.exists(): (quality.py:420) — existence …
PlannedOnboarding · Bug

"! agent dir ~/.cursor/skills is not writable" is the one doctor issue with no next action — heal has no path for it and the next install crashes at exit 70

Measured. With chmod 500 ~/.cursor/skills, doctor prints the bare line ! agent dir ~/.cursor/skills is not writable and exits 1, boost heal answers ✓ nothing to heal and exits 0, and the very next boost install tdd-workflow exits 70 with a crash report from the unguarded link.symlink_to(target) at boost_cli/core/store.py:209 — leaving a store dir and two live agent links (claude-code, windsurf) that the lock file does not record, after which doctor's only prescription is boost sync, which deletes both links with no confirmation prompt. Reproduce it. cd <repo> || exit 1
export HOME=$TMPDIR/audit-doctor-verify4; export BOOST_HOME=$HOME/.boost; mkdir -p "$HOME"
python3 tests/make_fixture.py $TMPDIR/audit-doctor-fix4 >/dev/null
./boost tap $TMPDIR/audit-doctor-fix4 >/dev/null 2>&1
BOOST_ASSUME_YES=1 ./boost install brainstorming >/dev/null 2>&1
chmod 500 "$HOME/.cursor/skills"
./boost doctor | grep 'agent dir' # bare symptom, no next action
./boost heal; echo "HEAL_EXIT=$?" # "✓ nothing to heal", 0
BOOST_ASSUME_YES=1 ./boost install tdd-workflow; echo "INSTALL_EXIT=$?" # 70
ls -1 "$HOME/.agents/skills"; ls -1 "$HOME/.claude/skills"
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. THE HEADLINE CLAIM IS FALSE. "the one doctor issue with no next action" / "Every other issue doctor raises names a command ... or a concrete action" is wrong. bad("lockfile", "lock file schema is v%s, expected v%d") at boost_cli/commands/quality.py:459-460 is also a bare symptom with no command and no action. I produced it live: ! lock file schema is v2, expected v3. Correct figure: cmd_doctor (HEAD lines 394-811) has 21 bad() call sites; 2 of the 21 carry no remedy — agent-dir (662) and lockfile-schema (459-460). The title must say "one of the two doctor issues with no next action", not "the one". 2. "silently removes the two live tdd-workflow links" overstates. boost sync prints ✓ removed stale link ~/.claude/skills/tdd-workflow and the same for windsurf — it names each deletion. What is true is narrower and I verified it with stdin closed and no BOOST_ASSUME_YES: sync deletes with NO confirmation prompt, and doctor's run \boost sync\` line does not say deletion is what will happen. Reword to that; the sync behaviour itself belongs to the separate doctor-prescribes-sync-that-deletes-live-links finding, as the finder noted. 3. FILE PATH. The finding writes "quality.py:661-662" and "quality.py:695-697" bare. The file is boost_cli/commands/quality.py. There is no boost_cli/core/quality.py` — I looked there first and it does not exist. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE LIMITS OF MY REPRO — things a card author must not get wrong: - THE DOCTOR CHECK AND THE CRASH ITERATE DIFFERENT AGENT SETS. quality.py:660 walks agents.enabled_agents() (5 dirs here: claude-code, windsurf, cursor, gemini, antigravity); store.link_agents walks agents.linking_agents() (4 — gemini is a native-store agent boost never links into). So an unwritable ~/.gemini/skills raises the doctor line but can NEVER crash install. "The next install crashes" holds for 4 of the 5 dirs doctor can flag, not all 5. - "TWO LINKS" IS NOT A FIXED NUMBER. linking_agents() iterates claude-code, windsurf, cursor, antigravity in that order. Cursor is third, so claude-code and windsurf land first and antigravity never does — I confirmed ~/.gemini/antigravity-cli/skills holds only brainstorming. An unwritable ~/.claude/skills (first in the order) would leave the store dir and ZERO links. Card the shape ("every link before the bad dir survives, none after"), not the count. - MY POST-CRASH ls/lock measurement was taken after two consecutive crashed installs, not one (I re-ran to capture the exit code cleanly). I then ran a third crash after sync and got the identical shape — store dir stays, the two links are recreated, lock still ['brainstorming'] — so the state is idempotent across repeats. Say that rather than let the card imply one run was measured. - NOT CARDED, and I checked properly. Why it is worth doing. The lens question for doctor is whether each issue names one concrete next action; this is the line that fails it, and it fails while sitting on top of a real, reachable exit-70 crash — the exact shape a stuck new user hits after copying a dotfiles tree or restoring a backup with wrong ownership. One chmod u+w ~/.cursor/skills clause on the line, matching the log-file line eight lines above it in the same function, converts a dead end into a fix; guarding the symlink call would additionally turn the install crash into the res.conflicts message store.py already has for the neighbouring 'something is in the way' case. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ Every other issue doctor raises names a command (boost heal, boost sync, boost update…
PlannedOnboarding · Bug

Nothing boost prints ever names an entry point: bare ./boost is byte-identical on a virgin machine and a working one, and the one command its failure-hints route you to is the only setup …

Measured. Bare ./boost is md5-identical (dd3c84b8d6510df1871bea5f1518fff4) on a machine with 0 taps and on the same machine with 7 taps and 1 installed skill, because print_help (cli.py:165-223) reads no state at all — and the command every newcomer-facing hint routes to, boost tap --defaults, produces exactly 7 lines whose last is a ✓ tapped … row with no summary, while boost install through the same pipe closes with a framed "next: boost info brainstorming" box. Reproduce it. cd <repo>
export HOME=$TMPDIR/lens2-ep ; export BOOST_HOME=$HOME/.boost ; rm -rf "$HOME" ; mkdir -p "$HOME"
# 1. the first screen: 103 lines, 81 commands, quickstart at line 80, no "start here"
./boost > "$TMPDIR/h0.txt" 2>&1
wc -l < "$TMPDIR/h0.txt" # 103
grep -n quickstart "$TMPDIR/h0.txt" # 80: quickstart Tap the starter registries...
grep -ci 'start here\|new to boost\|first, run\|get started' "$TMPDIR/h0.txt" # 0
head -4 "$TMPDIR/h0.txt"
# 2. the only entry-point pointers are failure hints, and they name tap --defaults, never quickstart
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. Three sentences in the claim are false as written; the underlying defect survives all three. 1. "you must run a command that exits 1 to learn where setup lives" — WRONG. boost doctor exits 0 and carries the identical boost tap --defaults hint plus the verdict line "● ready to set up — tap a registry to make boost searchable". I measured DOCTOR_EXIT=0 (the claim's own evidence block also shows EXIT=0, contradicting its prose). The pointer is not gated behind a non-zero exit; it is gated behind running a diagnostic command, which is a weaker statement. 2. "every CLI hint routes to tap --defaults … never mentions that quickstart exists" — OVERBROAD. Two user-facing hints do name boost quickstart: boost_cli/commands/discovery.py:398 ("boost quickstart taps the defaults and fetches their vectors in one pass", raised by reindex --fetch-shards) and boost_cli/commands/pkg.py:981 ("boost quickstart taps the starters and loads their vectors in one pass", raised by update --shards). Correct figures: ~15 source sites name boost tap --defaults vs 2 that name boost quickstart, and 0 of the newcomer-plausible surfaces (search, doctor, taps, browse, list, catalogbundle, mcp, serve) name quickstart. The substantive routing gap is real; the words "every" and "never" are not. 3. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Not carded — I re-checked independently over all 473 items. audit-root-findings.md (the bare-boost card) covers EPIPE exit 120, help --help/help version alias routing, and the launcher's 3.9 Python floor; its one print_help ask ("add one dim Options line under Usage") has shipped — I see the Options line. Nothing about state-awareness or a first-run pointer. BOOST-D13 "Framed success summary with next step" is status:done, ref: core/output.py:41 · commands/pkg.py, and its body scopes itself to install/uninstall. audit-tap-findings.md (inflight, PR 772) is about path-shaped SPECs going to the network — zero hits for next/defaults/quickstart/summary. audit-taps-findings.md — zero hits for the same terms. audit-quickstart-findings.md covers unpinned-at-HEAD taps without [rag] and three divergent [rag] hint wordings, not routing. Scope limits of my repro: - boost quickstart's final line was verified by source read (quickstart.py:220, unconditional, immediately before return 0), not by running it — a live run fetches the shard manifest and would download vectors. I did not re-run the claim's quickstart | tail -1 control. - core/bootstrap.py shows boost mcp register auto-seeds the catalog (taps the defaults unless BOOST_NO_SEED/--no-seed), and its docstring states "boost mcp is the only command a new user is told to run after installing". So an MCP-first newcomer is not stranded; this finding is specifically the CLI-first path. Why it is worth doing. This is the whole first-value path and it costs a newcomer their first two commands. The productive path is 3 commands / 10.6s once you know it; the tool makes you discover it by failing. Worse, the two setup paths are not equivalent — README leads with quickstart, which pins taps to shard commits and can import prebuilt vectors, while every CLI hint routes to tap --defaults, which does neither and never mentions that quickstart exists. Both fixes are one line each: make print_help add a first-run line when registry.list_taps() is empty (it already resolves config at startup), and give the tap/registry success path the same next-step emitter pkg.py and quickstart.py already use. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ A newcomer's first screen is 103 lines of 81 commands that is byte-for-byte the same …
PlannedTech-debt

golden-natural.jsonl — the only fully exemplar-graded query set — is invoked by no make target and no workflow, so its numbers can only be produced by a human typing the command

Measured. Running the natural set today against the same 20-tap eval corpus prints "REGRESSION vs baseline: catalog.search recall@k: 0.080 -> 0.060 (-0.020)" (BM25 MRR 0.2447 -> 0.237, nDCG 0.2640 -> 0.259) — the snapshot has already drifted, and no Makefile target or workflow passes --golden, so nothing in the repo can ever emit that line. Reproduce it. cd <repo>
grep -n natural Makefile ; echo "makefile grep exit $?"
grep -rn natural .github/workflows/ ; echo "workflow grep exit $?"
grep -rn 'golden-natural' Makefile .github/workflows/ scripts/ tests/ | sed -n '1,20p'
sed -n '1,23p' tests/unit/test_eval_baseline.py
python3 -c "import json,hashlib,pathlib; b=json.load(open('tests/eval/baseline.json')); print('baseline keys:',list(b['sets'])); print('current digests:',[p+'@'+hashlib.sha256(pathlib.Path('tests/eval/'+p).read_bytes()).hexdigest()[:12] for p in ('golden.jsonl','golden-natural.jsonl')])" What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. BASELINE NUMBERS ARE WRONG. The claim says the golden-natural row carries "0.350 / 0.160 / 0.245 / 0.259". It does not. tests/eval/baseline.json's golden-natural.jsonl@0d91b0cd8e41 BM25 row is 0.360 / 0.160 / 0.2447 / 0.2640. The quoted 0.350/0.160/0.245/0.259 are the *pre-migration* ("name-graded (before)") figures at docs/roadmap/items/golden-set-grades-by-name-not-by-skill.md:125; the baseline holds the *post*-migration row from line 126-127 (0.360 / 0.160 / 0.245 / 0.264). The claim read the wrong line of the card. 2. "the only tests that touch it are explicitly synthetic" IS WRONG. tests/unit/test_eval_grading.py reads the REAL shipped file in three live assertions: test_the_shipped_set_has_nothing_left_to_decide (line 217-228, asserts len(rows) == 50 and no unpinned row) and class TestTheMigrationIsFinished (line 247+): test_every_row_pins_an_exemplar, test_every_exemplar_is_well_formed, test_no_exemplar_is_a_localised_copy. Only tests/unit/test_eval_baseline.py is synthetic. The defect survives the correction, because those tests grade the file's SHAPE (row count, exemplar presence and syntax) and never run retrieval — so they cannot move or invalidate a single number in the baseline row. The accurate statement is: automation pins the query set's shape, nothing re-measures its scores. 3. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Scope and limits of my repro: - The structural half (no --golden in any Makefile target or workflow; --golden defaults to golden.jsonl at scripts/eval_retrieval.py:66/:656) is exhaustive over Makefile, .github/workflows/ and scripts/ — read, not inferred. - The drift measurement was run against a PRIVATE COPY of the shared 20-tap eval corpus (I copied cache/ + config.json into $TMPDIR/lens2-exemplar and never wrote to the shared home). I did not verify that this corpus is byte-identical to the one that produced the committed baseline row: Makefile:130-133 says the corpus "tracks upstream HEAD rather than pinned commits", while CLAUDE.md says every taps.txt row pins a SHA — the repo disagrees with itself there and I did not resolve it. So the -0.020 delta could be corpus movement rather than ranker movement. That distinction does not affect the finding: corpus movement is exactly the thing a re-run is supposed to report, and no re-run happens. - I did not run make eval, make check, or any workflow; I read their recipes. - I did not measure how long a make eval-natural would add (the eval corpus was already materialised for me, so I never paid the tap cost). - catalog.search's numbers here are near the floor of resolution (3 of 50 queries vs 4 of 50), so its -0.020 is one query; the BM25 MRR/nDCG drift is below the default 0.02 eps and would not have been flagged even if someone ran it. Why it is worth doing. CLAUDE.md documents the two-set design ("Baselines are keyed by query set (name@content-digest), so one file holds both golden.jsonl and golden-natural.jsonl without either overwriting the other") as if both sets are exercised. Only one is. The set that received 50 hand-made relevance judgments — the project's best available measurement of retrieval quality, and the one whose grading key actually identifies a skill — contributes to no gate and no scheduled monitor, so the investment decays silently. Adding a make eval-natural target (or a non-blocking scheduled run alongside eval-stats.yml) would cost one recipe and turn a frozen snapshot back into a signal. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ The 50-row natural-language golden set is the one place where the project's content-c…
PlannedTech-debt

The exemplar migration never reached the query set the required gate runs: 0 of 91 rows, and 27 of them have an ambiguous target on the gate's own corpus

Measured. Both invocations of the merge-blocking retrieval gate — the Makefile eval target and ci.yml's "retrieval quality gate" step — run scripts/eval_retrieval.py with no --golden, so both take DEFAULT_GOLDEN = tests/eval/golden.jsonl; 0 of that file's 91 rows carry an exemplar, and on the gate's own 20-tap corpus 27 of the 91 rows have at least one target name that resolves to more than one content digest (21 of 77 distinct target names are ambiguous). Reproduce it. cd <repo>
python3 -c "import json; rows=[json.loads(l) for l in open('tests/eval/golden.jsonl') if l.strip() and not l.startswith('#')]; print(len(rows),'rows;',sum(1 for r in rows if r.get('exemplar')),'with exemplar')"
python3 -c "import json; rows=[json.loads(l) for l in open('tests/eval/golden-natural.jsonl') if l.strip() and not l.startswith('#')]; print(len(rows),'rows;',sum(1 for r in rows if r.get('exemplar')),'with exemplar')"
grep -n 'DEFAULT_GOLDEN' scripts/eval_retrieval.py
sed -n '134,139p' Makefile
sed -n '269,279p' .github/workflows/ci.yml
export BOOST_HOME=$TMPDIR/eval-home
python3 -c "
import json,glob,collections,os
n2c=collections.defaultdict(set)
… Verification found nothing to correct. Every number, file:line and command output above was independently re-derived and matched exactly. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Severity lowered from the claim's High to Medium on a measurement the claim did not make. I inspected the bodies behind the ambiguous names and they are two different things. Genuine homonyms exist — brand-guidelines is Anthropic brand colors vs OpenAI brand colors vs Sentry *copy writing* (three different jobs), and pdf is a general PDF skill vs a workflow that converts PDF to Markdown with one specific tool — but they are a minority, and neither appears among the rank-1 hits. The names that actually score hit@1 through an ambiguous target (skill-creator, slack-gif-creator, theme-factory, prompt-engineering, mcp-builder) are re-publications of one upstream skill across anthropics/skills, composio-community/awesome-codex-skills and sickn33/antigravity-awesome-skills, differing by a few bytes. A "wrong body" there is still a correct answer to the user, so the looseness is much smaller in practice than in principle. I did not adopt the shipped card's "the looseness was latent" finding as a reason: that was measured at hit@1 0.160, where there was nothing for ambiguity to inflate, whereas this set measures 0.484 — a different regime, so it does not transfer. Floor-margin exposure, with that qualification attached: hit@1 measures 44/91 = 0.484 against a floor needing 37 rows (margin 7 rows) with 10 rank-1 hits landing on an ambiguous name; recall@10 measures 77/91 = 0.846 against a floor needing 71 rows (margin 6 rows) with 20 rows matched only via ambiguous names. Why it is worth doing. This is the only gate in make check that can fail a merge on retrieval quality, and every retrieval decision validated against it (RRF fusion vs preferring dense, pool depth, whether the LLM rerank earns its keep) inherits the looseness. The project already established the fix and built the harness for it; the required set simply never got migrated, and because the card is closed as shipped nothing tracks the remaining work on the set that matters most. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CONFIRMED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ CLAUDE.md states "Relevance is still decided by name (or by content class when a gold…
PlannedTech-debt

The "taps last refreshed N days ago" hint can never fire on a machine that tapped and never ran boost update — the only writer of the marker is update itself

Measured. The maintainer's own production install has 458 taps configured and no marker: ~/.boost/state/ is fully populated (pulse.jsonl 95k, lock-history/, snapshots/, and last-shard-sync stamped 8 Sep) while last-tap-refresh is absent — so the directory was not wiped, the file was simply never written across 458 taps, and the stale-tap hint CLAUDE.md describes as search's drift-reporting mechanism has never been able to fire there. The 20-tap eval corpus is the same: no marker. Two real installs, zero markers. Reproduce it. export HOME=$TMPDIR/lens2-du2 ; export BOOST_HOME=$HOME/.boost ; mkdir -p "$HOME"
cd <repo>
python3 tests/make_fixture.py $TMPDIR/lens2-fix2
./boost tap $TMPDIR/lens2-fix2
python3 -c "from boost_cli.core import paths, registry; m=paths.tap_refresh_marker(); print('marker:',m); print('exists:',m.exists()); print('refresh_age_days():',registry.refresh_age_days()); print('STALE_TAPS_DAYS:',registry.STALE_TAPS_DAYS)"
# now prove update is the only writer:
grep -rn 'mark_refreshed()' boost_cli | grep -v 'def mark_refreshed'
./boost update
ls -la $BOOST_HOME/state/last-tap-refresh What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The mechanism reproduces exactly, but three of the claim's supporting statements are wrong: 1. "the open card audit-audit-findings.md:36" — that card is status: shipped (docs/roadmap/items/audit-audit-findings.md, frontmatter line 5), not open. Its proposal is not pending; it landed. 2. "would make two more surfaces silent for the same population" — false for boost health, which has ALREADY moved onto this marker and does NOT go silent. boost_cli/commands/quality.py:1417-1425 reads registry.last_refresh_at() and I measured the tap-only install printing last tap sync never. That is an honest degradation, visible to the user — the opposite of the silent failure the claim predicts. 3. "would make two more surfaces silent" — also false for boost audit --skills, which never moved onto the marker. boost_cli/commands/safety.py:233 _tap_age_days still shells out to git log -1 --format=%ct on the clone, so it is unaffected by the marker either way. Zero additional surfaces are made silent, not two. 4. "unreachable for the entire population it is aimed at" — over-broad. The hint is aimed at users whose taps are stale, and the long-ago-updated half of that population IS served correctly (my control run B proves it fires at 200 days). What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Scope limits of my repro: - My A/B used a 5-item tests/make_fixture.py tap, not the real default set. The population claim about boost tap --defaults and boost quickstart is settled by code inspection, not by running them: quickstart.py touches only registry.list_taps and registry.add_many, and pkg.py:1077 is the sole registry.update call in all of boost_cli/commands/. No command other than update can reach mark_refreshed. - I aged install A by backdating every file under BOOST_HOME and install B by backdating the marker; neither is a genuinely 200-day-old machine, but the marker's mtime is the only input refresh_age_days reads (registry.py:435-440), so the emulation is exact. - The real install was inspected read-only with ls and a json.load of config.json. I did NOT run boost health there — it writes caches — so the "last tap sync never" line is measured on my disposable HOME, and the real machine's marker absence is measured directly. - I did not weigh the mitigation heavily: boost health does report "never", but it reaches the wrong population — the hint lives in search precisely because that is the command a never-update user runs, and nothing routes them to health. - Noticed but out of scope, not chased: the if results: guard at registry.py:680 means an update sweep where every tap FAILED also skips the stamp, and test_a_sweep_with_no_taps_stamps_nothing pins the no-taps case deliberately. - Not carded: grep -h '^title:' docs/roadmap/items/*.md plus body greps for mark_refreshed / tap_refresh_marker / refresh_age_days / "never … Why it is worth doing. Tap clones are load-bearing for both BM25 freshness and dense vector validity, and boost deliberately refuses to refresh them in the background — which makes this one muted line the entire mechanism by which a user learns their catalogue has drifted. It is switched off for precisely the users who need it (onboard via quickstart or tap --defaults, search for months, never update), and switched on only for users who already demonstrated they run boost update. Stamping the marker on a successful tap would cost one write_text and make the hint reachable. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ CLAUDE.md's rule is that "search must never refresh taps in the background: _hint_sta…

Engine & command internals

// concrete file:line findings from the code scan
ShippedRetrieval · Architecture

Semantic search is gated behind an API key it does not need

Write-up · keyless-semantic-search-for-everyone.md

Complexity L Impact High Wow ★★★★★ the vector store was never the problem — only turning text into vectors needs a key
ShippedSearch · Performance

Every dense search re-scanned all 3.08 GB of vectors — vec0 has no ANN index

Write-up · dense-search-rescans-every-vector.md

Complexity M Impact High Wow ★★★★★ 33.9s cold search; 28.2s of it was one brute-force scan of 3.08 GB
ShippedInstall engine · Scope

Workspace scope — boost install --local into the project

Write-up · workspace-scope-install.md

Complexity L Impact High Wow ★★★★ --local, committable repo lock
ShippedInstall engine · UX

boost list shows installed rules and workflows

Write-up · list-rules-and-workflows.md

Complexity S Impact Med Wow ★★ list was skill-only after rule/workflow install landed
ShippedEval · Correctness

The golden set grades by name, and 35 of 53 names are ambiguous

Write-up · golden-set-grades-by-name-not-by-skill.md

Complexity M Impact High Wow ★★★★ all 50 rows pinned; the 22 judgment calls were measured to move the published number by zero
ShippedEval · Correctness

The “pinned” eval corpus pinned names, not commits

Write-up · eval-corpus-was-not-actually-pinned.md

Complexity M Impact High Wow ★★★★ 62% of the required gate's corpus was one unpinned third-party repo, against a 1.15-query margin
ShippedEval · Correctness

The eval de-duplicated its ranked list by name, so homonyms shared a rank

Write-up · eval-deduped-ranked-lists-by-name.md

Complexity M Impact High Wow ★★★★ 13 different skills named code-reviewer shared one rank slot — where "recall is 1.000" came from
ShippedEval · Correctness

62% of the required gate's corpus is a single third-party repository

Write-up · eval-corpus-is-one-strangers-repo.md

Complexity S Impact High Wow ★★★ one third-party repo is 62% of the gate's corpus, and if it disappears every PR goes red
ShippedEval · Correctness

Nothing refreshes the eval corpus pins, so the gate measures one frozen day

Write-up · eval-corpus-pins-have-no-refresh-path.md

Complexity S Impact Medium Wow ★★★ the corpus is now frozen at one August 2026 snapshot, and nothing will ever move it
ShippedCompat · Python

the Python floor moves from 3.9 to 3.12

Write-up · python-floor-moves-to-312.md

Complexity M Impact High Wow ★★★ the floor was blocking a CVE fix and a dependency major at once
ShippedTech-debt

the typing.List → list sweep the floor now allows

Write-up · pep585-604-modernization-sweep.md

Complexity M Impact Low Wow ★ unblocked by the 3.12 floor; deferred only because the diff touches ~60 modules
ShippedCorrectness

audit the 16 zip() calls the 3.12 floor made checkable

Write-up · zip-strict-audit.md

Complexity S Impact Med Wow ★★ 16 call sites, each a judgement — a silent truncation or a new raise, never a mechanical fix
ShippedBug

boost search never noticed a tap added after the first search

Write-up · search-never-notices-a-new-tap.md

Complexity S Impact High Wow ★★★★ search said "no matches" for a skill `boost info` described from the same machine
ShippedBug

one deleted upstream stopped boost update for every other tap

Write-up · one-dead-tap-broke-every-update.md

Complexity S Impact High Wow ★★★★ found on the maintainer's own machine — 80+ taps, one deleted upstream, no updates for any of them
ShippedSecurity · Correctness

The update path skipped the scan the install path runs

Write-up · update-refreshed-rules-without-a-gate.md

Complexity M Impact High Wow ★★★★★ install scanned rule content; update did not — and the scanner could not see concealment anyway
ShippedTech-debt

bring scripts/ under the ruff gate

Write-up · scripts-under-the-lint-gate.md

Complexity M Impact Med Wow ★ 291 findings across 28 files, measured — most are the same UP sweep the package just finished
ShippedCorrectness · MCP

boost_search never said which ranking produced its answer

Write-up · mcp-search-hid-which-ranking-ran.md

Complexity S Impact Medium Wow ★★★★ the degraded order was byte-for-byte the shape of the promised one
ShippedCatalog · Correctness

est_items counted one skill fourteen times once registries went multi-agent

Write-up · est-items-counts-agent-mirrors-as-items.md

Complexity S Impact Med Wow ★★★ registries now ship one rendered copy per agent, so a raw walk credits pbakaus/impeccable with 40 items for the 9 it has
ShippedCatalog · Curation

The catalog was missing the two most-starred token-efficiency registries

Write-up · efficiency-registries-ponytail-caveman.md

Complexity S Impact Med Wow ★★★ two ~97k-star repos the catalog was missing; both advertise savings their own benchmarks contradict
ShippedBuild · Bug

The shards workflow has never once produced a shard

Write-up · shards-pipeline-never-produced-a-shard.md

Complexity M Impact High Wow ★★★★ two scheduled runs, 0 artifacts — 40 of 60 jobs tapped a bare SHA, and the other 20 hit an export that cannot work
ShippedBug

The repair command could not repair the thing two commands sent you to it for

Write-up · sync-could-not-repair-a-gutted-skill.md

Complexity S Impact Med Wow ★★★★ two commands named `boost sync` as the repair; sync answered "everything in sync" and changed nothing
ShippedBug

Path.exists() looks total, and is not

Write-up · pathlib-exists-is-not-total.md

Complexity S Impact Medium Wow ★★★★ the weekly fuzzer has now found two crashes in one function, three weeks apart, and been ignored both times
ShippedCI · Correctness

a publisher that could not publish, and an alert that could not stand down

Write-up · two-ci-controls-that-could-not-act.md

Complexity S Impact Medium Wow ★★★★ one publisher could never publish and one alert could never stand down — both reported success
ShippedCI · Performance

the shard job that had never once finished, and the timeout that could not be raised

Write-up · the-shard-job-that-could-not-finish.md

Complexity S Impact High Wow ★★★★★ 5h30m cancelled at the ceiling, then 2h07m green — the same job, with duplicate embeddings collapsed
ShippedCI · Evaluation

the scheduled re-pin that refreshed the twenty rows it must not touch, and none of the hundred and sixty-five it existed to pin

Write-up · the-scale-corpus-refreshed-the-wrong-rows.md

Complexity S Impact High Wow ★★★★★ 185 rows, 20 pinned — the monthly job moved exactly the 20 it did not own and pinned none of the 165 it existed to pin
ShippedCatalog · Curation

--category marketing matched nothing, while four marketing registries sat in the catalog under other names

Write-up · marketing-crm-domain-had-no-category.md

Complexity M Impact High Wow ★★★ the four marketing registries already carried were filed under writing and general, so the category returned nothing
ShippedSearch · Performance

Dense reuse is per tap, so one changed file re-embeds the whole registry

Write-up · dense-reindex-reuses-whole-taps.md

Complexity M Impact High Wow ★★★ one changed file re-embeds 29 min for 1.9 s of new text
ShippedCLI · Bug

boost create: CLI audit findings (2026-08)

Write-up · audit-create-findings.md

Complexity M Impact Med Wow ★★ parse(dump(meta)) != meta — evolve rewrites untouched lines, create writes broken YAML
ShippedCLI · Bug

boost reindex: CLI audit findings (2026-08)

Write-up · audit-reindex-findings.md

Complexity S Impact Med Wow ★★ reindex --json names taps two ways in one object (reused vs reindexed)
ShippedSearch · Performance

quickstart reruns re-download every shard; shards.sync never asks what is built

Write-up · shard-refresh-skips-processed-commits.md

Complexity S Impact Med Wow ★★ a quickstart rerun re-downloads every shard the store already holds at that commit
ShippedReliability · Bug

Two concurrent rag.build() runs delete each other's temp index

Write-up · concurrent-rag-builds-delete-each-others-temp-index.md

Complexity S Impact High Wow ★★★ two overlapping rag.build() runs share one .tmp name; the second unlinks the first's finished 644 MB file and the first dies on replace()
PlannedCLI · Bug

compact <tap> answers a question about one tap with a global all-clear

cmd_compact resolves its named taps and then filters (taps = [t for t in taps if t.is_cloned]). A tap that is configured but has no clone falls out of that list silently, so the empty-list branch fires and prints “✓ no cloned taps to compact” with exit 0 — a green, plural, global-sounding all-clear in answer to a question about one specific tap, which it never names. Reproduced on a scratch BOOST_HOME: tap a repo, delete its clone directory, then boost compact <that-tap> → rc=0, “✓ no cloned taps to compact”. Meanwhile boost doctor exits 1 on the same machine and says “! tap <x> not cloned — run boost update”, so the information exists and this command is the one that withholds it. This contradicts the convention registry.update states in its own docstring: “A named tap still raises. boost update sometap is a request about that one tap, so its failure is the answer to the question asked.” The sweep form is right to stay quiet; the named form is not. Fix: when args.tap named taps and the filter dropped some, say which and why, and point at boost update the way doctor does — and do not report success for a tap nothing was done to.

Complexity S Impact Medium Wow ★★ boost compact <tap> on an uncloned tap prints a green "no cloned taps to compact" and exits 0
PlannedCLI · Bug

boost taps vouches for a clone that is not there

boost taps reads the per-tap catalog cache, and catalog.load_tap deliberately serves a stale cache when the clone is gone (rescanning is only possible while the clone exists — making the missing clone an error there would trade a missing field for a missing catalogue). The consequence reaches the readout unlabelled: a tap whose clone has been deleted still prints its NAME, an ITEMS count and an UPDATED date, and the footer still counts it in “N taps · N items”. Nothing on the line says the clone is missing. Measured on a scratch BOOST_HOME: after deleting the clone directory, boost taps printed probe-src  1  2026-09-10 and “1 taps · 1 items” — while boost doctor exited 1 with “! tap <x> not cloned”. Two readouts of the same machine, one of which is wrong, and the wrong one is the command whose entire job is to describe the taps. It is the same shape as the pin bugs in audit-pinned-taps-silently-moved-to-head-by-update-re-clone-and-co: a readout that keeps vouching for state the clone no longer has. The precedent is in the tree: boost list carries a FLAGS column for exactly this reason, and info.py:239-241 states it — “so a quarantined or pinned rule/workflow doesn't render byte-identical to a healthy one … a reader needs to see that here, not just in --json”. Its values are pinned and quarantined; a tap with no clone is the same class of fact about a different table. Fix: taps already has tap.is_cloned in hand, so mark the row and keep the footer counts honest, rather than leaving doctor as the only command that knows.

Complexity S Impact Medium Wow ★★★ boost taps prints item counts and an UPDATED date for a tap whose clone was deleted

Code health & security

// planned · free tooling to catch vulns, smells & bugs
ShippedSecurity · Posture

OpenSSF Best Practices — all 67 passing criteria answered

Write-up · openssf-best-practices-badge.md

Complexity M Impact Med Wow ★★★ 66 Met/N-A · 1 justified Unmet · registration is the only step left
ShippedSecurity · Posture

OSPS Baseline — three levels, audited rather than assumed

Write-up · osps-baseline-levels.md

Complexity M Impact Med Wow ★★★ L1 + L2 earned · L3 blocked · audit found a merged regression
ShippedSecurity · Posture

OpenSSF silver — reachable solo, and mostly already true

Write-up · openssf-silver.md

Complexity S Impact Med Wow ★★★ 55 criteria, 31% → the gap was two documents, not fifty
ShippedSecurity · Posture

OpenSSF gold — how far it goes without a second human

Write-up · openssf-gold-ceiling.md

Complexity M Impact Med Wow ★★★ 26% → 61% live; the parser that said "21 criteria" had missed two
ShippedDocs · Onboarding

The README read like a machine wrote it — measurably

Write-up · readme-prune-and-search-guide.md

Complexity S Impact Med Wow ★★ 609 → 399 lines; 75 em dashes → 2; a real semantic-search guide
ShippedCI · Release safety

main went red because "require branches up to date" was never actually on

Write-up · branch-current-gate.md

Complexity S Impact High Wow ★★★★ two protection mechanisms disagreed; the binding one had the safety off
ShippedInterop · Bug

Gemini logged a skill conflict every session and boost doctor called the machine healthy

Write-up · gemini-duplicate-skill-discovery.md

Complexity S Impact Med Wow ★★★ Gemini warned once per skill per session and no boost surface could see it
ShippedSecurity · Posture

A machine-readable VEX feed, sourced from findings that already existed

Write-up · vex-openvex-feed.md

Complexity S Impact Low Wow ★★ OSPS-VM-04.02 — 4 statements, zero fabricated CVEs
ShippedSafety · Bug

boost audit: CLI audit findings (2026-08)

Write-up · audit-audit-findings.md

Complexity M Impact Med Wow ★★ rm -rf ~/ passes clean, hidden.js is never scanned, and a missing dir counts as scanned
ShippedCLI · UX

boost doctor: CLI audit findings (2026-08)

Write-up · audit-doctor-findings.md

Complexity S Impact Low Wow ★ crash line wears "!" but verdicts "● healthy" exit 0; and "1 issue need attention"
ShippedCLI · Bug

boost fingerprint: CLI audit findings (2026-08)

Write-up · audit-fingerprint-findings.md

Complexity S Impact Low Wow ★ quarantine doesn't change the fingerprint; an uncloned tap hashes as empty, silently
ShippedCLI · Bug

boost health: CLI audit findings (2026-08)

Write-up · audit-health-findings.md

Complexity S Impact Med Wow ★ health calls a drifted machine "● healthy" and scores gemini 1/1 with the store dir gone
ShippedCLI · Bug

boost lint: CLI audit findings (2026-08)

Write-up · audit-lint-findings.md

Complexity M Impact Med Wow ★★ lint can't lint a path on disk, counts 6307 "skills" for 1997 names, misreads one missing ---
ShippedCLI · Bug

boost policy: CLI audit findings (2026-08)

Write-up · audit-policy-findings.md

Complexity M Impact Med Wow ★★ policy check evaluates 4 of the 7 rules install enforces — a refused env "passes"
ShippedSafety · Bug

boost quarantine --release: CLI audit findings (2026-08)

Write-up · audit-quarantine-findings.md

Complexity S Impact Med Wow ★★ a quarantine/release round trip widens 1 agent to 4 and turns doctor red
ShippedCLI · Bug

boost test: CLI audit findings (2026-08)

Write-up · audit-test-findings.md

Complexity S Impact Med Wow ★★ a skill boost lint fails (exit 1) passes boost test outright (exit 0)
ShippedSafety · Bug

boost verify: CLI audit findings (2026-08)

Write-up · audit-verify-findings.md

Complexity S Impact Low Wow ★ a row counted among the N failed still renders the green ok token

Pipeline & supply-chain integrity

// planned · free tooling to secure the CI/CD path itself
ShippedRetrieval · Onboarding

Prebuilt vectors are published where no new user can reach them

Write-up · published-shards-have-no-consumer.md

Complexity L Impact High Wow ★★★★★ the vectors were being built and then nobody could get them
ShippedSupply chain

OpenSSF Scorecard's findings, triaged into three piles

Write-up · scorecard-findings-triage.md

Complexity M Impact Med Wow ★★ 7 open · 3 now hash-pinned, 3 decided, 1 clears itself
ShippedRelease safety

main has no branch protection, so the release rules are honour-system

Write-up · main-has-no-branch-protection.md

Complexity S Impact High Wow ★★★ decided, applied, and now gated against deadlock
ShippedBug

markdownlint linted the fuzzer's corpus, so shipping a crash reproducer would redden a prose gate

Write-up · markdownlint-lints-the-fuzz-corpus.md

Complexity S Impact Med Wow ★★★ 12 of 21 linted files were malformed on purpose
ShippedCI speed

The mutation gate's floor is a single file — shard 0 is store.py

Write-up · mutation-shard-floor-is-one-file.md

Complexity M Impact Med Wow ★★★ shard 0 is store.py alone — 9.3-11.7 min against a 6.1 even split
ShippedCI speed

A third of CI job time is spent waiting for a runner, not running

Write-up · ci-time-is-now-runner-queueing.md

Complexity M Impact High Wow ★★★★ 104 job-min queued vs 236 executing — 31% of CI is waiting for a runner
ShippedSecurity · CI/CD

The required lint job pins zizmor==1.27.0 — a yanked release

Write-up · zizmor-pin-is-a-yanked-release.md

Complexity S Impact Med Wow ★★★★ already fixed by a4450f76 twelve hours before this card was filed
ShippedCI reporting

demo.yml still fails on every push — and the fix is not in the workflow

Write-up · demo-cannot-open-its-own-pr.md

Complexity S Impact Med Wow ★★★★ a repo setting blocks it — no edit to demo.yml can fix this one
ShippedRelease safety

One commit can cut two releases, and the naive guard against it breaks retries

Write-up · one-commit-can-cut-two-releases.md

Complexity M Impact Med Wow ★★★ guarded on PyPI, not on the tag — so retries still work
ShippedSearch · Index

The BM25 index is one JSON blob, and it stops working between 10k and 50k items

Write-up · bm25-index-is-one-json-blob.md

Complexity L Impact High Wow ★★★★★ 2.5 GB RSS per search at 50k items
DeclinedSearch · Retrieval

Semantic search for users who will never set an API key

Dense retrieval today needs the [rag] extra and a VOYAGE_API_KEY/OPENAI_API_KEY and a built store. Most users will do none of that, so the default experience is BM25 forever. The keyless path is a local static embedding model — potion-retrieval-32M class, MIT, model2vec family — which is not a transformer: the entire weight file is one lookup table, so inference is tokenize → gather rows → mean-pool → L2-normalize. Measured locally, pure stdlib: ~1 ms to embed a query (mmap + bisect over sorted keys), 12.8 MB of int8 vectors for 50k items at 256-d, and ~20 ms to rerank BM25's top-200. No numpy, no sqlite-vec, no ANN index, no new runtime dependency. import numpy alone costs 180–390 ms cold, which disqualifies it from a one-shot CLI query path; the BM25 prefilter is what makes the stdlib version viable, since a full 50k brute-force scan is 1.5 s in pure Python. Pool depth is justified by measurement: BM25 recall saturates at 0.890 by depth 200 and gains nothing at 400, so reranking the top-200 gives up essentially nothing versus scanning everything. Why this and not a shipped Voyage index. A precomputed Voyage index is inert without a Voyage query vector, and the only keyless way to get one is a maintainer-run anonymous embedding endpoint — an unauthenticated free embeddings API backed by the maintainer's card, which also ships every user query off-machine and breaks offline. A local model is deterministic, so the artifact becomes a cache rather than a correctness dependency: a newly tapped repo can be embedded on the user's own machine. Doc-side is the asymmetry worth shipping for — 29 ms/doc in pure Python is ~24 min for 50k single-core, versus seconds in CI with numpy. Do not ship this before the eval and dedup items. The headline claim (+11.0 recall / +15.9 hit@1) did not survive verification: its baseline used the kind oracle the real search path lacks, and both the blend weight (w_dense=0.7) and the pool depth were argmax'd on the same 82 queries they were reported on, by 2-query margins. On a binary metric at n=82 the smallest net win reaching p<0.05 is 6 queries; hit@1 (+13 net queries) holds up, recall (+9) sits at the resolution floor. And the structural risk is real: the name is only ~10.5% of a mean-pooled surface vector while 106 description clusters are shared across 270 distinct names, so the lift may shrink toward 50k rather than hold. Sequence: fix the gate, dedup, fix the index format, then re-measure with McNemar and a held-out blend weight, leading with hit@1. Test entry-level dense alone before the blend, ship 512-d not 256-d (the whole case for 256-d was one query), keep Voyage/OpenAI as the opt-in ceiling, and keep BM25 as the floor. The model table cannot go in the default wheel — the shipped runtime is 0.79 MB and every merge to main cuts a release, so +17.4 MB × ~24 releases/day exhausts PyPI's 10 GB project quota in under a month; it needs a separate, rarely-released data package behind an extra. Related, and partly overtaken: [[keyless-semantic-search-for-everyone]] shipped a keyless path using a transformer (BGE via ONNX Runtime, in the [rag] extra) while this item was open. That does not settle the question this card asks — a static lookup table is still far cheaper, and this card's discipline about not shipping a retrieval claim before the eval still stands. What it does change is the baseline: "keyless" is no longer the differentiator, so the case for a static model now rests on cost (~1 ms and no runtime dependency, against a measured 233 ms cold and 34 MB of wheels) rather than on availability. Status left alone deliberately — this is another loop's item to own. Unblocked, and the case for it got stronger. This card says “do not ship this before the eval and dedup items”. Both have now landed: the eval gate floors four metrics over a realistic-sized corpus with baselines keyed to their query set, and content-hash dedup has merged. The new evidence is a timing measurement. Building the shipped ONNX keyless store over 743 entries (3,740 chunks, bge-small-en-v1.5 on CPU) took 4,431 s — 74 minutes, about 1.2 s per chunk. This card's static-embedding proposal claims ~29 ms/doc in pure Python. If that holds it is a difference of more than an order of magnitude on the doc side, which is exactly the cost that makes prebuilt shards mandatory today. Worth measuring the model2vec path directly before committing — but the gap it claims to close is now a measured number rather than an estimate. Spike done — the prerequisites this card set have all shipped, so the measurement it asked for was finally runnable. It says “do not ship before the eval and dedup items”; dedup landed in #370, the index format in #367/#371, the published eval in #373. What follows is potion-retrieval-32M (MIT, 63,091 × 512 F32 lookup table — confirmed a single tensor, no transformer) driven by a hand-written pure-stdlib loader: WordPiece → gather rows → mean-pool → L2, mmap'd, no numpy. The two unverified performance claims were not just right, they were conservative. Query embedding measured 0.16 ms against the card's ~1 ms. Document embedding measured 1.34 ms on a synthetic 105-token doc and 3.27 ms on 300 real catalogue entries (median 61 tokens), against the card's ~29 ms. That reverses one of this card's design arguments. The doc-side cost was the reason prebuilt artifacts looked mandatory: “29 ms/doc in pure Python is ~24 min for 50k single-core”. At the measured 3.27 ms it is 2.7 min — roughly the time a first boost tap --defaults already takes. Local embedding is therefore viable on the user's own machine, and shipped shards become a genuine optimisation rather than a requirement. (Not to be confused with the ONNX bge-small path measured at ~1.2 s/chunk in keyless-semantic-search-for-everyone; that number stands, and the gap between them is the case for the static model.) But reranking bought nothing at real scale, which is the result that matters. Over the 50 natural-language golden queries against a real 71,655-entry catalogue, reranking BM25's top-200 by cosine scored hit@1 2/50 — identical to BM25's own 2/50, a net change of +0 queries where this card's own statistics note says 6 net queries is the smallest win reaching p<0.05. On two hand-checked pairs the ordering was right but the margin was thin (related 0.154 vs unrelated 0.097). Stated limits, because this does not settle the question. The document vector was built from name + description truncated to 1,500 characters, not the full body the real dense path indexes, so this measures a weaker representation than the one being proposed. No blend was tried — pure rerank, no w_dense — and this card explicitly asks for a held-out blend weight and McNemar. What it does establish is that the cheap version of the idea does not pay for itself, so the remaining work is representation and blending, not inference speed. An unrelated finding fell out of it, and it is the more important one. BM25 scored hit@1 0.040 here against the 0.340 published in #373. Both are correct: the published figure is measured over the pinned 6-tap eval corpus of 743 entries, and this run used a real 77-tap install — 96× larger. Golden targets are all present and rank 7th, 8th, 38th, 163rd rather than 1st. The eval corpus is not a scale model of a real install, and the gate's floors describe a catalogue two orders of magnitude smaller than the one users have. Tracked separately in [[eval-corpus-is-96x-smaller-than-a-real-install]]. Declined on measurement, after a second model was tried specifically to avoid declining on one data point. The card's premise is that a local static model buys keyless semantic search. Tested against the 50 natural-language golden queries over the pinned 20-tap corpus (3,843 entries as those registries stand today), with BM25 at hit@1 0.260 as the baseline in every run: potion-retrieval-32M (retrieval-tuned, 63,091×512 F32) scored dense 0.220, hybrid RRF 0.260. potion-code-16M-v2 (code-domain, 63,457×256 F16 — chosen because this catalogue is coding-agent skills, which is the strongest hypothesis for why a general retrieval model would underperform here) scored dense 0.240, hybrid 0.260. A third representation — name+description only, over the full 71,655-entry catalogue — reranked BM25's top-200 to +0 net queries. So: two models, three representations, no measurable gain, and fusion never beats BM25 alone. The code model is one query better than the retrieval model, which at n=50 is inside the noise (±0.02 per query) and should not be read as a trend. The contrast is what makes this a decline rather than a shrug. The published eval measures a real embedding model at hybrid 0.440 against BM25 0.340 — a genuine +0.100. Static embeddings reproduce the cost profile that made the keyless tier attractive (0.16 ms/query, 3.3 ms/doc, no dependency) but not the quality that made it worth having. Cheap and no better than what ships today is not a tier; it is a second code path to maintain for nothing. What survives. The performance findings stand on their own and are already recorded above: doc-side embedding is ~24× faster than the card assumed, which is why prebuilt shards are an optimisation rather than a requirement for the real models in [[keyless-semantic-search-for-everyone]]. The pure-stdlib loader (WordPiece → gather → mean-pool → L2, mmap'd, F32 and F16) is proven workable if a future model justifies it. What would reopen this. A static model that actually separates on this task — the bar is beating 0.260 as a reranker, not merely producing plausible cosines. The two hand pairs looked fine for both models (0.224 related vs -0.001 unrelated for the code model), which is exactly why plausible similarity was not accepted as evidence.

Complexity L Impact High Wow ★★★★★ two models, three representations — no measurable gain over BM25, and fusion only ties
ShippedBuild · Bug

One global concurrency group let any PR cancel any other PR's check

Write-up · demo-workflow-cancels-other-prs.md

Complexity S Impact High Wow ★★★★ two PRs deadlocked each other for hours — re-running the job just moved the cancellation to the other one
ShippedBuild · Bug

The fuzzer found a real crash and nobody was listening

Write-up · unwatched-crons-hid-a-real-fuzz-crash.md

Complexity M Impact High Wow ★★★★★ the fuzzer found a real crash and was right for three weeks — 24 unattended workflows, 2 watched
ShippedBuild · Bug

The published metrics could never be published

Write-up · eval-stats-could-never-publish.md

Complexity S Impact Med Wow ★★★★ the published-metrics file has ONE commit in its whole history — the one that created it
ShippedPerformance · Search

A cold search materialises 71,600 entries to print five

Write-up · cold-search-reads-the-whole-catalogue.md

Complexity M Impact High Wow ★★★★ 0.94 s cold search at 71.6k entries — ~0.5 s spent materialising data the top hits never use
ShippedSearch · Retrieval

boost search brainstorm finds nothing, and brainstorming finds it

Write-up · bm25-has-no-stemming.md

Complexity M Impact High Wow ★★★★ a term with no postings is now replaced by the commonest term it prefixes; a term that has postings is never touched, which is what keeps the eval floors still
ShippedSecurity · Reproducibility

Reproducible release builds — the sdist half nobody's setuptools does for you

Write-up · reproducible-release-artifacts.md

Complexity M Impact Med Wow ★★★ wheel + sdist now bit-identical; setuptools#2133 has no native fix
ShippedRetrieval · Onboarding

The weekly republish reached the machines that had never been set up

Write-up · weekly-vectors-had-no-ingestion-path.md

Complexity M Impact High Wow ★★★★ the vectors were republished weekly and no command could take delivery
ShippedQuality · Retrieval eval

A query made only of characters tokenize drops returns zero results, and the documented catalog.search fallback is unreachable — boost search "C++" finds nothing on a machine holding …

Write-up · bm25-empty-tokenization-kills-catalog-fallback.md

Complexity L Impact High Wow ★★★★ the four symbol-bearing language names now tokenize, and a query that still tokenizes to nothing says so instead of reporting a miss; the catalog.search fallback stays unreachable on purpose — substring 'R' returns 10,092 of 10,152 entries
ShippedUX · Bug

boost search 'C++' returns zero and blames the catalogue: tokenize drops every 1-char token, and nothing ever says a term was discarded

Write-up · search-drops-symbol-language-queries-silently.md

Complexity L Impact High Wow ★★★★ c++/c#/f#/objective-c now alias to indexable tokens on BOTH the index and query side (INDEX_VERSION 7 -> 8); a term that is still dropped is named on stdout, on stderr under --json, and in the MCP reply
ShippedCI · Bug

The sweep gate died at a page, not at launch — and took six unrun checks with it

Write-up · sweep-died-at-a-page-not-at-launch.md

Complexity S Impact High Wow ★★★★ the gate kept reporting failure after it had stopped checking a page and a half

Developer experience & maintainability

// planned · free tooling to catch issues earlier & keep the code legible
ShippedCI · Reproducibility

Pin the lint toolchain so a release can't redden the gate

Write-up · pin-the-lint-toolchain.md

Complexity S Impact High Wow ★★★ unpinned ruff 0.16 reddened every PR
ShippedFeature

make boost mcp the whole setup, and put all three kinds behind it

Write-up · mcp-zero-setup-and-three-kinds.md

Complexity M Impact High Wow ★★★★ measured — a fresh install answers every MCP search with "no skills match", which reads as "boost is empty" rather than "nothing is tapped"
ShippedBug · MCP

A project-scoped install registers its MCP servers machine-wide

Write-up · mcp-servers-ignore-install-scope.md

Complexity M Impact High Wow ★★★★ --scope project installs the skill into the repo and its MCP server machine-wide
ShippedInterop · MCP

MCP-aware skills — declare and wire an .mcp.json on install

Write-up · mcp-aware-skills.md

Complexity L Impact Med Wow ★★★★ skills that need a server
ShippedDX

roadmap.html goes stale on every rebase, so a card and a merge race redden the whole matrix

Write-up · roadmap-html-goes-stale-on-every-rebase.md

Complexity M Impact Med Wow ★★★ two of the three fixes measured dead; one survivor, and it needs a Pages change
ShippedTesting · Bug

make lint reports success when actionlint fails — and says it wasn't installed

Write-up · make-lint-masks-actionlint-failures.md

Complexity S Impact Med Wow ★★★★ fixed — a failing actionlint now fails make, in all three targets
ShippedTesting · Bug

The second silent skip — actionlint runs, and checks no run: block at all

Write-up · actionlint-skips-shellcheck-silently.md

Complexity S Impact Med Wow ★★★★ fixed — shellcheck is pinned in lint-tools and both gates now assert it is there
ShippedInterop · Adoption

MCP — answer the veto that overruled the trigger ("a skill already matched")

Write-up · mcp-already-covered-defeater.md

Complexity M Impact High Wow ★★★★ every trigger was a predicate over the request; the veto was a predicate over the agent's own context
ShippedInterop · Adoption

boost-first — the one rule boost authors, offered opt-in at boost mcp register

Write-up · boost-first-rule.md

Complexity M Impact High Wow ★★★★ the tool descriptions only help on hosts that deliver them — this is the surface that survives when none do
ShippedCatalog · DX

install dead-ends on a registry that vendors its own skills

Write-up · resolve-vendored-duplicate-copies.md

Complexity S Impact Med Wow ★★★★ found by dogfooding — the fix hint re-raised the error it was fixing
ShippedCLI · Install

install refused an ambiguous name and offered no way to answer it

Write-up · install-path-disambiguation.md

Complexity S Impact High Wow ★★★★ the ambiguity error named the paths and no flag could act on them — a dead end
ShippedDX · Feature

boost discover <query> asks GitHub, instead of filtering whatever boost index happened to sample

Write-up · discover-searches-github-not-a-stale-sample.md

Complexity M Impact High Wow ★★★★ an adversarial review of the first draft confirmed 21 defects, two of which would have reddened CI
ShippedFeature

Share the catalogue instead of making everyone re-tap it

Write-up · shareable-catalogue-bundle.md

Complexity M Impact High Wow ★★★★★ 10.9 MB replaces a 12 GB clone — a fresh machine reaches 59,972 searchable items in 4 seconds
ShippedInterop · Adoption

boost-first carried the trigger that had already fired and lost — and could never be updated

Write-up · boost-first-carried-the-losing-trigger.md

Complexity M Impact High Wow ★★★★ the rule shipped the one trigger boost had already measured as losing, and no revision of it could ever reach a machine that had installed it
ShippedDX · Feature

boost serve becomes a searchable, faceted catalogue with a graph of the taps

Write-up · serve-is-a-searchable-catalogue.md

Complexity L Impact High Wow ★★★★★ the old page listed only what was installed — 147 rows out of 71,695 — with no search, no tags and no way to see what a tap actually is
ShippedBug · UX

browse could not search for two words, and the fix reshaped the whole browser

Write-up · browse-could-not-search-two-words.md

Complexity M Impact High Wow ★★★★ space was bound to select, so two words could never be searched for
ShippedUX · Design

One design system across search and browse

Write-up · search-and-browse-visual-refresh.md

Complexity M Impact High Wow ★★★★ search rows learned kind/tap/installed with a stated drop order; browse got its gradient, an empty state, a badge rail, a list scrollbar and a session chip
ShippedPerformance · MCP

The smart rerank pays the LLM again for a search it already answered

Write-up · smart-rerank-pays-per-novel-query.md

Complexity S Impact High Wow ★★★ MCP boost_search measured 11.7-17 s per call — every call, even a repeat of the last one
ShippedCLI · Output

The box drew 108 columns into an 80-column pane, and --help never asked how wide the pane was

Write-up · cli-output-ignored-the-terminal.md

Complexity S Impact Med Wow ★★★ an 80-command sweep found a box that drew wider than the pane, a help screen that never measured it, and a literal %% on screen
ShippedCLI · Output

The hints still run past the pane, and the worst one is pinned by six test files

Write-up · long-hints-overflow-narrow-panes.md

Complexity M Impact Low Wow ★★ a code span is one atomic token, so a hint folds to the pane without splitting the command it tells you to run
ShippedCLI · Bug

install --dry-run promises agents the real install never writes (antigravity-cli copy, antigravity materialize) and omits the MCP plan

Write-up · audit-install-dry-run-promises-agents-the-real-install-never-write.md

Complexity S Impact High Wow ★★ fixed — dry run now reads agents_for_scope/materializing_agents and plans the MCP action; PR awaiting CI (PyPI unreachable in the authoring sandbox, so make check needs CI's confirmation)
ShippedCLI · UX

AI degrade note blames PATH/API keys regardless of cause; several commands fall back with no note at all

Write-up · audit-ai-degrade-note-blames-path-api-keys-regardless-of-cause-sev.md

Complexity M Impact Med Wow ★★ fix implemented and unit/functional/smoke-tested locally; mutation/eval/full-lint gates unverified — this session had no PyPI/network egress to run them, PR relies on CI
PlannedCLI · Bug

Dry-runs disagree with the real run: compact, heal and onboard previews mispredict

A dry-run's one job is to say what the real run will do, and five previews demonstrably don't. Sharpest case: with an untracked 1 MiB scripts/junk.bin planted in a tap clone, compact minio/skills --dry-run prints “would free 1.0MB” — then compact minio/skills prints “✓ every tap is already compact” and the file is still there. _freight_bytes (boost_cli/commands/configuration.py:268-275) counts by rglob, but git sparse-checkout reapply only drops tracked paths outside the cone, so untracked bytes are promised and never freed. compact --dry-run --reclone compounds it: identical output with or without --reclone, although a reclone would drop the clone's whole .git. heal mispredicts its own branch: with a store copy deleted, heal --dry-run says “would restore brainstorming … (or drop it from the lock)” while the live run on the same state prints “✓ reinstalled missing brainstorming from sickn33/…” — the “drop” alternative never fired. And on a fresh HOME it says only “would create 4 missing directories”, the one heal action that never names its paths, so nothing in the preview says ~/.agents/skills and the agent skill dirs are what get written. onboard --dry-run truncates each file preview at 24 lines with no marker (configuration.py:630 is splitlines()[:24]) — the lock preview ends mid-object — and --dry-run --pr on a directory that is not a git repository exits 0 with no PR plan and no precondition failure, because the dry-run early return at configuration.py:624 sits before the git-repo check at :636. The repo already treats preview/apply divergence as a correctness defect: the shipped item dry-run-promised-a-link-nobody-makes (PR 460) fixed the same class for install --dry-run. Fix per the verified recommendation: compute _freight_bytes from the git ls-files intersection so the dry run predicts what reapply removes, and estimate .git bytes when --reclone is given; word heal's restore line from the branch sync_apply will take and name the directories; have onboard print “… N more lines” past 24 lines and run the read-only --pr precondition checks before the dry-run return. Docs: README.md lines 268-274 (boost compact --dry-run). Found by the 2026-08 CLI audit (cluster dry-run-fidelity); repro in the audit log.

Complexity M Impact Med Wow ★★ compact --dry-run promises "would free 1.0MB"; the live run frees nothing
ShippedCLI · Bug

--json accepted but ignored: cohort/config/policy set, focus, profile, replay rollback, who empty state

Write-up · audit-json-accepted-but-ignored-on-many-branches-cohort-config-pol.md

Complexity M Impact Med Wow ★ shipped in PR 804 — all seven sites now emit JSON (or reject --json as a usage error, for `update` without `--shards`); CI's full check suite (lint, evals, tests, smoke, mutation) is green on the merge-ready head
PlannedCLI · Bug

Project scope seams: uninstall/verify/list/info/reinstall disagree with what install --local wrote

The project-scope-across-every-command item shipped, and the 2026-08 CLI audit found its seams: the writers and readers resolve "the project" differently. install --local uses scopes.resolve_base, which falls back to the cwd (scopes.py:83-104), while uninstall's project fallback (store.py:1249) and verify/doctor/list all go through scopes.project_root, which requires a VCS marker (scopes.py:45). So from a plain directory, install anthropics/skills:pdf --local writes .boost/skill-lock.json and .claude/skills/pdf and reports success — then verify pdf answers “Error: not installed: pdf”, plain uninstall answers “brainstorming is not installed”, and doctor/list show no project row. After mkdir .git the same commands find everything. All six findings reproduced. Four more seams, each confirmed in source. The already-installed error hints “boost reinstall brainstorming --local to force” — a flag reinstall does not have; following the hint exits 2 with “unrecognized arguments: --local” (store.py:615-617). verify <project-only name> ignores the filter and grades every user-scope item — cmd_verify already passes [] for project-only names (safety.py:355-356) but _iter_installed_all treats [] as “everything” (_common.py:66, if names:), so the run can fail on a rule the user never named. list --local --kind rule prints “○ no rules installed” although project scope holds skills only. And info on a project-scoped skill shows the not-installed card — no version, installed date, commit or agents rows — though the project lock records them all and --json returns them under project. The verified fix, one follow-up card: make _iter_installed_all treat [] as nothing (_common.py:66) · change the hint to boost install NAME --local --force (matching README ~328) or add --local to reinstall · unify the uninstall fallback on resolve_base or hint --local (store.py:1249) · have install --local warn outside a VCS root, or teach project_root to accept .boost/skill-lock.json as a marker · refuse list --local --kind rule|workflow the way the existing --tag guard does (info.py:274-281) · render the plock identity rows in cmd_info. Docs: README ~304–328 (uninstall/reinstall routes), a follow-up note on docs/roadmap/items/project-scope-across-every-command.md, and regenerate docs/commands.html if reinstall gains --local. Found by the 2026-08 CLI audit (cluster project-scope-readers); repro in the audit log.

Complexity M Impact Med Wow ★★ install --local writes a lock that uninstall, verify, doctor and list then cannot find
PlannedCLI · UX

Sweep: positionals/--json lack help strings and no command help shows examples (~30 cmds)

Across roughly thirty commands the help screens end at the options table with undocumented arguments. Observed verbatim: help cohort prints {list,create,delete,status,apply} with no help string for the action; test --help shows positional arguments: then NAME with nothing after it; conflict --help and attest --help each list --json with an empty help line; edit/explain/home give name no text (so docs/commands.html renders <code>name</code><span></span>). No help screen in the audit shows an Examples block. The gaps hide real contracts: run never mentions its SDK/key prerequisites (only the runtime error does), discover --help never says a query hits GitHub live while bare/--local read the cache, conflict's exit-1-on-findings is undocumented, cohort status is an unadvertised alias of list, and policy's 11 valid keys appear only in the error hint after a wrong set. Verification confirmed this is omission, not style: the mechanism works and is used exactly once — the only epilog= in all of boost_cli is cohort's membership-hash paragraph (team.py:74), and team.py:77 adds the action positional with no help=. Nothing in CLAUDE.md declares terse help deliberate, and the content gap is unchanged at COLUMNS=60 under a TTY, so it is not a rendering artifact. Fix as one sweep PR: add help= to every bare positional (action choices, NAME, --json) and an Examples epilog per parser — cliparse.parser forwards **kwargs to argparse, so no plumbing is needed. Extend scripts/build_command_reference.py to render parser.epilog (lines 119/137 render description and per-arg help but currently drop the epilog) and fail --check on empty help strings, then regenerate docs/commands.html. Document the defaults (cohort/profile default action = list) and the cohort status alias while there; the list summary in cli.py COMMANDS should also say "skills, rules and workflows" to match its own description. Found by the 2026-08 CLI audit (cluster help-examples-sweep); repro in the audit log.

Complexity M Impact Med Wow ★ exactly one epilog= exists in all of boost_cli; ~30 commands ship bare positionals
PlannedDocs · Drift

Stale prose after shipped changes: catalog --export, live discover, 464 count, Apache-2.0

Four patches of prose describe the repo as it was before a shipped change, each verified against the current tree. A documented command that does not run: docs/security-design.md:35 says a boost catalog export tarball; running it prints Error: one of the arguments --export --import --show is required and exits 2 — the flag is --export. Pre-live-search discover: docs/index.html:877 (boost discover indexes 10,000+ skills across GitHub) and :940 (build the GitHub-wide index … browse it with boost discover) predate the shipped change that made a query hit GitHub Code Search live; the index now only backs bare discover and --local, exactly as the command's own help says. A hard-coded count, off by one and spreading: README:151, semantic-search.md:64 and quickstart.py:39 say --catalog taps 463 registries; registries.json holds 464 non-list_only rows of 487, and quickstart --dry-run --catalog on the 20-tap fixture prints would tap 445 registries (445 + 19 already tapped = 464). Verification found more copies than the auditor: README:211, quickstart.py:65 and CLAUDE.md:183–184 and :500. A licence rule contradicting every file: CLAUDE.md:276 still says headers open with SPDX-License-Identifier: GPL-3.0-only, but every source file greps to Apache-2.0, LICENSE is the Apache License 2.0, and scripts/add_spdx_headers.py already writes Apache-2.0 — the CLAUDE.md paragraph (and its -only→-or-later example) is the sole stale statement. Fix as one doc-only PR: security-design.md:35 → boost catalog --export; rewrite the two index.html sentences to live-GitHub-search plus boost index/--local; replace the literal 463 with every catalogued registry (or 464) in README.md:151/211, semantic-search.md:64, quickstart.py:39/65 and CLAUDE.md:183/500 — a number scripts/build_registries.py changes should not be hard-coded in prose; and correct the CLAUDE.md licence paragraph to Apache-2.0. security-design.md and semantic-search.md are already in prose-lint.yml's vale list, so no lint wiring is needed; docs/commands.html is untouched (no parser changes). Found by the 2026-08 CLI audit (cluster stale-prose-docs); repro in the audit log.

Complexity S Impact Low Wow ★ a documented command that exits 2, 463 vs 464 in eight places, a GPL rule over Apache files
PlannedCLI · Bug

boost ROOT: CLI audit findings (2026-08)

EPIPE leaks past main's own handler. boost --help | (exec 0<&-; sleep 0.3) exits 120 with “Exception ignored on flushing sys.stdout: BrokenPipeError” — 3/3 runs, and count, taps and --version leak the same way. cli.py:327-331 deliberately catches BrokenPipeError and returns 0, but stdout is never flushed inside the try, so any output that fits the stdio buffer raises at interpreter exit — and the --help/--version early returns (cli.py:292-302) sit before the try with no handler at all. Fix: cover the early returns, flush sys.stdout inside the try, and os.dup2 a devnull fd over stdout in the handler so the exit flush cannot raise.

Help routing rejects main's own aliases. boost help --help → “unknown command: --help / hint: did you mean: heal?” exit 2; boost help version suggests verify while boost version works; even boost help help fails. The aliases live only in cli.main (cli.py:292-302) and print_command_help resolves against COMMANDS alone, then difflib-guesses any token — dash-prefixed ones included. And boost --help documents none of the working global flags (-V/-v/--debug/-q). Fix: short-circuit the aliases in print_command_help before difflib, say unknown option for dash tokens in _unknown (cli.py:231-236), and add one dim Options line under Usage in print_help.

The ./boost launcher still gates at Python 3.9 — tuple (boost:27), hint text (boost:37) and header comment (boost:5) all say 3.9+ while pyproject.toml:22 requires >=3.12. On a stock macOS whose only python3 is 3.9 the launcher selects it and the user gets a SyntaxError from core/workflows.py's match statements instead of the friendly hint. The shipped python-floor-moves-to-312 item enumerated every floor touchpoint and missed this one. Fix: bump all three sites to 3.12 and add a unit test pinning the launcher's floor to requires-python. README already says 3.12+ — no doc change needed; none of the three fixes touches a COMMANDS row, so docs/commands.html is unaffected.

Found by the 2026-08 CLI audit (clusters broken-pipe-exit, help-routing-aliases, launcher-python-floor); repro in the audit log.

Complexity S Impact Low Wow ★ EPIPE exits 120 with stderr noise; `help version` suggests verify; launcher gates at 3.9
ShippedCLI · Consistency

boost absorb: CLI audit findings (2026-08)

Write-up · audit-absorb-findings.md

Complexity S Impact Low Wow ★ absorb is the only generated-skill command with no journal.log call at all
In flightCLI · Bug

boost adapt: CLI audit findings (2026-08)

A subagent named after a declared tool renders modules that compile but cannot run. With a subagent grep and tool Grep: crewai emits @tool("grep") def grep then grep = Agent(...), so reviewer_1 = Agent(tools=[read, grep]) hands the Agent where a tool belongs (stub run: TypeError); langgraph assigns grep = create_react_agent(...) inside build_mycrew, making it local, so the earlier tools=[read, grep] raises UnboundLocalError. _unique_idents (core/adapters.py:188-202) dedups only among agent specs and _unique_tools (:205-212) allocates stub names independently. Fix: pass the tool ident set into _unique_idents as pre-reserved names (or prefix stubs tool_<name>) in render_crew/render_graph, plus a golden test that executes a colliding render against stubs.

docs/commands.html brackets required options as optional. Line 369 shows boost adapt [--to FRAMEWORK] … while adapt --help prints an unbracketed --to FRAMEWORK and omitting it exits 2 — and the verify pass found it is broader than adapt: evolve's required --feedback and catalog's required mutually-exclusive group render all-optional too. Pure generator bug: scripts/build_command_reference.py:122-126 brackets every option unconditionally. Emit required options unbracketed (a required group as (--a | --b)), prefer the short flag like argparse, then make generate — the --check gate holds it after that.

Colon-form model ids get double-prefixed for the LiteLLM targets. --model anthropic:claude-x — the form langgraph accepts and emits — renders llm=LLM(model="anthropic/anthropic:claude-x") for crewai and the same for agents-sdk; multi-agent crews inherit it via adapters.py:376. _litellm_model's docstring says a provider-qualified value passes through, but the code checks only /. Fix: replace the first : with / before deciding to prefix (mirror of _langchain_model); document accepted syntaxes in docs/adapters.html's --model paragraph (~line 344, slash form only today), and regenerate docs/commands.html only if the argparse help changes.

adapt -o and run --print -o write generated source mode 0600 — and re-rendering over an existing 0644 file silently downgrades it, unlike a shell redirect (-rw------- vs -rw-r--r-- under umask 022). util.atomic_write_text (core/util.py:91-116) inherits mkstemp's 0600, right for the lock/config it was written for, wrong for source the user asked boost to write. Fix: add an optional mode parameter (fchmod the temp fd before os.replace), keep 0600 the default, and have cmd_adapt (pkg.py:1753-1761) and cmd_run (run.py:62) pass the umask default. Still open — an implementation of exactly this shape (path-based os.chmod, not os.fchmod, learned the hard way: the latter raises on Windows) shipped and then was reverted from PR #728 after three rounds of Windows-only windows-latest CI failures in the "unit + functional (90% coverage gate)" step that neither pytest-cov nor a temporary diagnostic artifact-upload commit could surface a cause for — the session driving that PR could not read Windows job logs (capped and consumed by harden-runner's own diagnostic noise) or download the diagnostic artifact (productionresultssa*.blob.core.windows.net blocked by that session's network egress policy) to see the actual failure. The other three findings landed clean on every platform. Whoever picks this back up needs either a session with working Windows CI log access, or to reproduce locally on a real Windows box.

Found by the 2026-08 CLI audit (clusters adapt-ident-collision, docs-required-flag-synopsis, adapt-model-id-syntax, generated-file-mode); repro in the audit log.

Complexity M Impact Med Wow ★★ PR
ShippedCLI · UX

boost bmad: CLI audit findings (2026-08)

Write-up · audit-bmad-findings.md

Complexity M Impact Med Wow ★ an edited persona reads "not installed"; each no-op bmad on burns 4 of 50 history slots
ShippedCLI · Bug

boost browse: CLI audit findings (2026-08)

Write-up · audit-browse-findings.md

Complexity S Impact Med Wow ★ both findings fixed in PR; make check unrunnable in the sandbox (no PyPI/apt egress) — CI is the real gate
PlannedCLI · UX

boost bundle: CLI audit findings (2026-08)

Local skills vanish into comments with no console notice (cluster bundle-dump-local-notice, med). With 2 imported (tap=local) skills plus one tap skill installed, bundle dump prints “✓ wrote Boostfile.local (1 tap, 1 skill)” and warns about rules/workflows — but says nothing about the local skills, which appear only as # local skill (no tap source): ab-testing comments in the file. Fires on any dump with local skills, not just the all-local edge. Fix in _bundle_dump (boost_cli/commands/pkg.py:1211-1234): count the local entries and out.warn “N local skills have no tap source and were written as comments” on both paths, parallel to the existing rules/workflows notice.

The present check ignores tap and version (cluster bundle-present-check, med). With brainstorming installed from sickn33/antigravity-awesome-skills v0.0.0, the lines skill nosuch/tap:brainstorming and skill sickn33/…:brainstorming@9.9.9 both yield exactly “Installed 0 skills, 2 already present”, exit 0, no warning — presence is have_installed.get(sname) on the bare name and tapq/sver are parsed then discarded (pkg.py:1281-1290). The code's own comment says a Boostfile is “meant to be reproducible” (:1298), and the “Boostfile wants @X, tap has Y” warning already exists on the fresh-install path (:1306-1308). Fix: compare the lock entry's tap/version against the Boostfile line before counting present; warn on mismatch, don't reinstall.

The dump omission notice is styled differently on the two paths (cluster bundle-dump-warn-style, low). TTY bundle dump prints the notice as a bare uncoloured print (pkg.py:1219) while bundle dump Boostfile sends the same text through out.warn, yellow (:1233). One-line fix: out.warn(msg, stream=sys.stderr) on both, keeping stdout a clean artifact.

Two small install-message gaps (cluster bundle-install-messages, low). bundle install with no Boostfile reports the tautology “Error: no Boostfile at Boostfile” — pathlib normalises ./Boostfile to the bare name (pkg.py:1241-1244), and ./nosuch/Boostfile likewise loses its ./. And bundle install - < /dev/null (or a comment-only file) reports “Installed 0 skills”, exit 0, with no hint that nothing was parsed. Fix: display the resolved path through the existing _tilde(), and warn when zero tap/skill lines were read (keeping exit 0).

Found by the 2026-08 CLI audit; repro in the audit log. All behaviour-only — regenerate docs/commands.html only if the bundle summary in cli.py COMMANDS changes.

Complexity M Impact Med Wow ★ mismatched tap/version lines count "already present" — the Boostfile stops being reproducible
ShippedCLI · Bug

boost catalog: CLI audit findings (2026-08)

Write-up · audit-catalog-findings.md

Complexity S Impact Med Wow ★ --show says "22 taps" then tables exactly 20; --export ships sender-machine paths as URLs
PlannedCLI · Bug

boost changelog: CLI audit findings (2026-08)

The shallow-clone hint fires on complete clones (cluster changelog-shallow-hint, med). changelog cowboy-coding on a full local fixture clone prints its one commit and then “(shallow clone: run git -C ~/.boost/repos/fixture-tap fetch --unshallow for full history)” — but the clone has no .git/shallow file, and that command fails on a complete clone. boost_cli/commands/quality.py:1159 gates the hint on len(lines) < 3 — output length, not clone shape — so any short history triggers it; it is only accidentally true for default remote taps, which really are shallow. Fix: gate on (tap.path/'.git'/'shallow').exists() as well, plus a unit test with a full fixture clone.

Rules and workflows are addressed by their directory, and changelog ignores the lock entry (cluster rule-file-vs-directory, med; also hits home). changelog csharp-reviewer — an installed workflow — fails with “Error: 'csharp-reviewer' matches 3 different skills in affaan-m/ECC”, because quality.py:1140-1153 resolves via lockfile.get_skill (skills only) then catalog ambiguity, although the lock already records "source_file": "ci-cd/dotnet-build.mdc"-style entries and lockfile.find_any (lockfile.py:195, whose docstring names this exact failure class) sits unused. When resolution does succeed, the log runs over rel_dir — the containing directory (catalog.py:121-122) — so git log -- ci-cd covers every sibling rule, masked today only by the depth-1 clone; following the command's own unshallow hint would surface it. home --print dependency-management has the same shape: it links …/tree/HEAD/dotnet-sdk (a folder of 11 rules) and actix-expert links a folder of 138 workflows (info.py:865-891). One fix covers all three findings: resolve installed names with lockfile.find_any, and for kind != skill pass entry['source_file'] (catalog: entry['skill_md']) to log_for_path and build /blob/HEAD/<file> URLs in cmd_home, keeping rel_dir for skills; add a unit test with a two-commit repo touching two rules in one directory.

Found by the 2026-08 CLI audit (clusters changelog-shallow-hint, rule-file-vs-directory); repro in the audit log. Regenerate docs/commands.html only if the home/changelog summaries change.

Complexity M Impact Med Wow ★ fetch --unshallow advised on complete clones; rules/workflows logged at directory granularity
PlannedCLI · Bug

boost chat: CLI audit findings (2026-08)

Referential follow-ups retrieve unrelated skills — including the suggestions chat itself prints. Turn 1 "how do I review a diff?" ranks orch-review; chat then suggests "what does orch-review actually do?", and typing that ranks orch-refine-code above orch-review. Its other suggestion "which of these should I install first?" returns teach, mercury-mcp, write-concisely — nothing from the previous turn, and at 7 words it never even hits expand_query's ≤6-word gate (core/chat.py:111-128). Not AI-dependent: retrieval bounds any answer. Fix: resolve referential follow-ups ("the second one", "which of these") against the previous reply's retrieved skills instead of re-querying; on the no-AI path boost the previous hit set or stop printing suggest_followups() questions the extractive path cannot answer (core/chat.py:266-288, :380); rank an exactly-named skill first. Found by the 2026-08 CLI audit (cluster chat-followup-retrieval); repro in the audit log.

The interactive "> " prompt is written to stdout when stdin is piped. Verified with streams separated: three "> " lines in the stdout capture, stderr empty, and chat < /dev/null ends "…Ctrl-D to exit\n\n> \n" — so a script capturing answers gets prompt chrome mixed in. _chat_session calls input("\n> ") unconditionally (boost_cli/commands/intelligence.py:1211); gate it on sys.stdin.isatty() and add a functional test asserting no "> " in piped stdout. Found by the 2026-08 CLI audit (cluster chat-prompt-echo); repro in the audit log.

chat is the only command that accepts -k. search "…" -k 5 fails with "Error: unrecognized arguments: -k 5" while chat -k 5 works (intelligence.py:1157); every other retrieval-limit sibling takes --limit only. Add -k as an alias of --limit to cmd_search (discovery.py:97) — and optionally the other limit commands — then regenerate docs/commands.html and update docs/chat.html. Found by the 2026-08 CLI audit (cluster search-k-alias); repro in the audit log.

Complexity M Impact Med Wow ★★ chat's own printed suggestion retrieves the wrong skill; its "> " prompt leaks into piped stdout
ShippedCLI · Bug

boost cohort: CLI audit findings (2026-08)

Write-up · audit-cohort-findings.md

Complexity S Impact Low Wow ★ fixed in PR #767 — all 41 CI checks green, including mutation, evals, and the full test matrix
ShippedCLI · UX

boost completions: CLI audit findings (2026-08)

Write-up · audit-completions-findings.md

Complexity M Impact Low Wow ★ CI ran the full pinned toolchain (lint, all mutation shards + gate, evals, full OS/version test matrix, DCO) and came back green with no merge conflict and no open review threads — the local sandbox couldn't run mutmut/vulture/etc. (PyPI blocked), so CI is what actually confirms this landed clean
ShippedCLI · Bug

boost config: CLI audit findings (2026-08)

Write-up · audit-config-findings.md

Complexity S Impact Low Wow ★★ `config unset` on a pristine HOME creates config.json and freezes all defaults into it
ShippedCLI · Bug

boost context: CLI audit findings (2026-08)

Write-up · audit-context-findings.md

Complexity S Impact Med Wow ★ git missing from PATH is reported as "not in a git repository", even inside a repo
ShippedCLI · Bug

boost deps: CLI audit findings (2026-08)

Write-up · audit-deps-findings.md

Complexity S Impact Med Wow ★★ deps shows ✗ not installed yet exits 0; the only real-world requires: shape reads as (none)
ShippedCLI · UX

boost discover: CLI audit findings (2026-08)

Write-up · audit-discover-findings.md

Complexity S Impact Low Wow ★ footer blames the network when gh is missing; --json prints [] silently with no index
ShippedCLI · Bug

boost edit: CLI audit findings (2026-08)

Write-up · audit-edit-findings.md

Complexity S Impact Med Wow ★ editor fails → green "✓ no changes", exit 0; drift calls a local edit upstream-moved
PlannedCLI · Bug

boost evolve: CLI audit findings (2026-08)

evolve accepts empty --feedback and has no stdin/file form. evolve brainstorming --feedback "" exits 0 and diffs in +## Feedback (2026-08-31) followed by nothing, plus a bump to version: 0.0.1 — with --apply that empty section lands in the store and the lock. --feedback - becomes the literal bullet +- -. and --feedback @/dev/null becomes +- @/dev/null.. In cmd_evolve (intelligence.py:663-713) raise BoostError when args.feedback.strip() is empty before calling the AI or heuristic; treat - as read-from-stdin and @path as read-from-file, documented in --help (then regenerate docs/commands.html).

evolve --apply leaves the revision unpinned, so a later boost update can silently overwrite it. After --apply the lock holds the evolved sha and pinned: false, and evolve prints only “✓ evolved brainstorming”; pkg.py's update loop (pkg.py:1035-1039) skips only pinned/quarantined/local entries, so once the tap moves, store.install(entry, force=True) (pkg.py:1063-1067) replaces the revision with no warning. On --apply set entry["pinned"] = True (or a local_revision flag the update loop honours) — at minimum print “boost pin <name> to keep this across boost update” after the success line.

After evolve, info claims an update to a lower version while outdated says up to date. With the lock at 0.0.1 and the tap at 0.0.0, info prints “[update available] … version 0.0.1 / latest 0.0.0 (update available)” — info.py:477-479 and 498-501 test latest != inst_v (string inequality) where cmd_outdated (taps.py:285, 340) correctly uses util.semver_gt for the same decision. Replace both checks with util.semver_gt(latest, inst_v) and label the locally-ahead case (e.g. local revision, tap has an older 0.0.0).

Found by the 2026-08 CLI audit (clusters evolve-feedback-input, evolve-revision-unpinned, naive-version-comparison); repro in the audit log.

Complexity M Impact Med Wow ★ empty --feedback writes an empty section + version bump; the revision is left unpinned
ShippedSafety · Bug

boost explain: CLI audit findings (2026-08)

Write-up · audit-explain-findings.md

Complexity M Impact Med Wow ★★ fabricated Kubernetes/Flyway summary scores faithfulness 1.0 and prints verbatim
ShippedCLI · Bug

boost export: CLI audit findings (2026-08)

Write-up · audit-export-findings.md

Complexity M Impact Med Wow ★ -o x.zip writes a gzip tarball; the repair hint drops the skill from the lock instead
PlannedCLI · Bug

boost import: CLI audit findings (2026-08)

import loses provenance, both ways (med). Importing over a tap-installed skill prints the normal four ✓ lines and rewrites the lock to tap='local', commit='' with no notice that the skill just lost its update source. And a URL import records the temp clone as source_dir — cmd_import (pkg.py:1350-1367) clones to a mkdtemp and rmtrees it in finally — so boost info shows a dead path and boost reinstall fails: “local source … is gone — skipped / Reinstalled 0 skills”, exit 1. The URL and cloned HEAD commit are known at import time and simply dropped. Fix: pass them into store.install_from_path, teach reinstall's local branch (pkg.py:1154-1162) to re-clone when source_dir is gone but a URL is recorded, and warn when a non-local lock entry is replaced. Regenerate docs/commands.html only if the import help changes. (Cluster import-provenance-loss.)

--agent narrows the declaration but leaves the links (low). Re-importing an installed skill with --agent cursor prints ✓ linked → cursor and sets only_agents=['cursor'], but the other three symlinks stay; the very next sync --diff reports “linked outside declared scope (3)”. Emit one warn after link_agents naming the out-of-scope links and the boost sync remedy — pruning can stay sync's job (pkg.py:1372-1376). (Cluster import-agent-scope-links.)

The multi-skill table pre-cuts descriptions at 60 chars (low). At COLUMNS=200 rows end mid-word — “implement an A/B tes” — with ~120 spare columns unused, because pkg.py:1413 slices (e["description"] or "")[:60] before out.table gets to fit and ellipsise the column (output.py:746-777). Drop the pre-slice; one line. (Cluster import-desc-truncation.)

Errors print above the tables they refer to (low, shared with policy check). out.err (output.py:240-244) writes to stderr without flushing block-buffered stdout, so any piped capture shows “Error: multiple skills found” before the listing it refers to. One central fix: sys.stdout.flush() at the top of out.err/warn, covering cli.py:317-321, configuration.py:497-503 and every future caller. Found by the 2026-08 CLI audit (cluster stderr-stdout-ordering); repro in the audit log.

Complexity M Impact Med Wow ★ import turns a tap install into "local" silently; a URL import records a deleted temp path
ShippedCLI · Bug

boost index: CLI audit findings (2026-08)

Write-up · audit-index-findings.md

Complexity S Impact Med Wow ★ PR #802: progress bar clears before raise/warn, 0 results keeps the previous index, gh rate-limit/403 gets a native hint — CI green (lint/eval/test/smoke/mutation), awaiting human merge
In flightCLI · Bug

boost install: CLI audit findings (2026-08)

Rule/workflow lock entries are name-keyed across scopes (med). With the benchmarking rule at user scope, install benchmarking --local in a project fails “Error: benchmarking is already installed / hint: boost reinstall benchmarking to force” — the project has no copy, and the hint would reinstall the user one. The reverse direction blocks too, and skills coexist fine (separate project lock). Worse, the --force escape overwrites the user-scope lock entry with the project one, orphaning the user materializations so uninstall can no longer clean them. _install_rule (store.py:836-839) and _install_workflow (store.py:1074) gate on a name-only lookup with no scope/base comparison. Fix: key entries by scope (or compare existing scope/base before raising), word the error “already installed at user scope”, and refuse a cross-scope --force overwrite without cleanup. Docs: README's install-scope section (~301-328) and docs/roadmap/items/install-scope-user-or-project.md. (Cluster cross-scope-name-block.)

--path says “under path” but matches suffix-only (low). --path plugins/tdd/skills is refused while the error's own hint lists plugins/tdd/skills/test-driven-development — a path that is under it. Suffix matching is the shipped design (install-path-disambiguation, PR 483); the wording is the defect. Reword the raise in catalog.py:~502 to “no copy of X whose path ends with Y” and hint “pass a trailing segment of one of: …”. (Cluster install-path-prefix-match.)

The MCP offer never shows the runnable command (low). The server row prints only demo-echo  npx though the sidecar declares npx -y @example/demo-echo-mcp plus env, and on decline the hint is a literal elided claude mcp add …; the full argv only prints when the host CLI is missing. _offer_mcp renders how from spec['command'] alone (pkg.py:161-164) and mcpdecl.register_argv already exists (pkg.py:201) — render command+args, print the joined argv on decline, and indent the confirm prompt to match its neighbours. (Cluster mcp-offer-command-detail.)

The typosquat warning prints three times (low). install NeoLabHQ/context-engineering-kit:test-driven-development --dry-run prints the identical “closely resembles test-driven-development (sickn33/antigravity-awesome-skills)” warning 3×, one per mirror copy in the look-alike tap. De-duplicate find_confusions on (name.lower(), tap) (typosquat.py:79-87) so the [:3] slice in _warn_confusions covers three distinct look-alikes. Found by the 2026-08 CLI audit (cluster typosquat-warning-dupes); repro in the audit log. Status (2026-09). Three of the four clusters shipped as described above: install-path-prefix-match (catalog.py wording), mcp-offer-command-detail (mcpdecl.command_line renders the full command+args, the decline path prints the real argv), and typosquat-warning-dupes (find_confusions dedupes on (name.lower(), tap)). cross-scope-name-block got the narrower of the fix's own two options: store._check_scope_conflict now refuses a rule/workflow install whose name collides with an existing lock entry recorded under a different scope/base — naming the real location ("already installed at user scope") and refusing even under --force, which closes the silent-corruption half of the bug (a forced cross-scope install used to overwrite the other scope's lock entry, orphaning its materializations). What is still missing is the other half: rules and workflows still cannot coexist across scopes the way skills do, because they share one lock keyed by bare name with no per-location table — skills got a separate projectlock.py when project scope was added, rules/workflows never did. Giving them the same treatment (a rules/workflows section in projectlock.py, wiring _install_rule/_install_workflow and their uninstall/sync counterparts through it for project scope) is real coexistence but is its own, larger change, and belongs in its own card rather than folded into a bugfix PR.

Complexity M Impact Med Wow ★ 3 of 4 clusters landed; cross-scope-name-block gets a safe refusal, not true coexistence
ShippedCLI · Bug

boost log: CLI audit findings (2026-08)

Write-up · audit-log-findings.md

Complexity S Impact Med Wow ★ fixed — cli.main() now catches SystemExit and journals the real rc for --help/usage exits
ShippedCLI · Bug

boost mcp: CLI audit findings (2026-08)

Write-up · audit-mcp-findings.md

Complexity M Impact Med Wow ★ no --dry-run; a named missing host exits 0; unregister claims success Gemini denies
ShippedSafety · Bug

boost onboard: CLI audit findings (2026-08)

Write-up · audit-onboard-findings.md

Complexity S Impact Med Wow ★ onboard --pr pushes absolute /Users/… paths from the global lock file to GitHub
ShippedCLI · Bug

boost outdated: CLI audit findings (2026-08)

Write-up · audit-outdated-findings.md

Complexity S Impact Med Wow ★ an untapped skill vanishes from outdated; the same untapped rule shows "source missing"
ShippedCLI · UX

boost preview: CLI audit findings (2026-08)

Write-up · audit-preview-findings.md

Complexity S Impact Med Wow ★ piped preview strips ** with no substitute; at 60 cols 10 lines leak raw markers
ShippedCLI · Bug

boost profile use: CLI audit findings (2026-08)

Write-up · audit-profile-findings.md

Complexity S Impact Low Wow ★ declined --prune leaves extras fully linked yet still prints "✓ switched"
ShippedCLI · Bug

boost protocol: CLI audit findings (2026-08)

Write-up · audit-protocol-findings.md

Complexity S Impact Med Wow ★ macOS status reads "registered" though register never calls Launch Services
ShippedCLI · UX

boost pulse: CLI audit findings (2026-08)

Write-up · audit-pulse-findings.md

Complexity S Impact Med Wow ★ fix implemented + tested; make check's eval/mutation/smoke gates could not run locally (no PyPI egress) — see PR for what did run
PlannedCLI · Bug

boost quickstart: CLI audit findings (2026-08)

Without the [rag] extra, quickstart taps unpinned at HEAD — and the rerun it promises cannot fix it. cmd_quickstart only fetches the manifest (the source of pins) when want_vectors is true (boost_cli/commands/quickstart.py:145-152), so on a machine without a dense backend the six new taps land with pin: null while the output ends “…install the extra…, then boost quickstart again”. The second run prints <tap> already tapped for all seven (registry.add_many skips existing taps, never re-pins), and once the extra is present shards.sync refuses every mismatched commit: refused (tap is at X, shard is for Y). That contradicts the module's own docstring — “Pinning is the whole point”. Fix: fetch the manifest and pin regardless of dense.have_backend() (pinning is a network-and-config operation, not an embedding one), and on rerun retarget already-tapped registries via shards.ingest instead of skipping them. Update README.md (quickstart section, ~line 145) and docs/semantic-search.md (~line 63). The [rag] install hint has three different wordings. quickstart says pipx inject boost-skill-cli "boost-skill-cli[rag]" (hard-coded at quickstart.py:175-177 and 202-204); reindex's embed.fallback_note (boost_cli/core/embed.py:170-179) says unquoted pip install boost-skill-cli[rag], which fails in zsh (no matches found); doctor and search say quoted pip install 'boost-skill-cli[rag]' via dense.fix_hint(). CLAUDE.md's rule is that doctor and search read one table so they cannot contradict — these two surfaces bypass it. Fix: have embed.fallback_note() and both quickstart paths call dense.fix_hint(); if pipx wording is wanted, put install-method detection inside fix_hint so every caller inherits it. docs/semantic-search.md is already quoted — keep it as the reference. Found by the 2026-08 CLI audit (clusters quickstart-pinning, rag-hint-drift); repro in the audit log.

Complexity M Impact Med Wow ★★ without [rag] quickstart taps unpinned at HEAD, and a rerun can never pin them
ShippedCLI · Bug

boost recommend: CLI audit findings (2026-08)

Write-up · audit-recommend-findings.md

Complexity M Impact Med Wow ★★ curated picks repeat one name 6 of 8 rows; --json returns [] while text prints them
ShippedCLI · Bug

boost replay: CLI audit findings (2026-08)

Write-up · audit-replay-findings.md

Complexity S Impact Low Wow ★ rollback says "complete" (exit 0) with a skill unrestored, and replans it forever
ShippedCLI · Bug

boost run: CLI audit findings (2026-08)

Write-up · audit-run-findings.md

Complexity S Impact Low Wow ★ the 55-line runner's banner names a command that produces a 9-line file
ShippedCLI · Bug

boost schedule: CLI audit findings (2026-08)

Write-up · audit-schedule-findings.md

Complexity S Impact Med Wow ★★ 'interval every None' without StartInterval — and StartInterval 0 hangs status forever
ShippedCLI · Bug

boost search: CLI audit findings (2026-08)

Write-up · audit-search-findings.md

Complexity M Impact Med Wow ★★ CJK rows 72 cells in a 60 pane; --json drops --smart; '60 matches' is the cap, not the count
ShippedCLI · UX

boost simulate: CLI audit findings (2026-08)

Write-up · audit-simulate-findings.md

Complexity S Impact Low Wow ★ fixed — norm_rule lowercases the whole modal; trigger desc clips on a word boundary
ShippedCLI · Bug

boost sync: CLI audit findings (2026-08)

Write-up · audit-sync-findings.md

Complexity M Impact Med Wow ★★ First sync run hides a blocked link; --diff prints it as a raw Python tuple
In flightCLI · Bug

boost tag: CLI audit findings (2026-08)

boost tag swallows unknown flags and misreads them as operands. tag brainstorming --verbose prints the current tags and exits 0 — the flag is consumed as a removal of the tag -verbose; tag --verbose gives "Error: --verbose is not installed" (the flag becomes a skill name); verification found a third hole: tag brainstorming --list silently discards the skill-name operand and lists all tags. Cause: cmd_tag's manual split (boost_cli/commands/info.py:988-993) whitelists only --list/--json/-h/--help; every other --x token falls through as an operand. Every sibling command rejects unknown options with "unrecognized arguments" exit 2. And the mutation path has no before/after check. tag brainstorming -nosuch removes a tag that was never present — silent, exit 0; tag brainstorming +x -x prints ✓ and writes the lock plus a journal event for a net no-op (changed is set per-token at info.py:1027-1041, never compared to the before set); "+with space" is accepted as #with space; +Design and #design coexist. The shipped roadmap item robust-tag-argument-parsing (PR 94) built this manual split — these are residual holes in it, not a duplicate. Fix in cmd_tag: hand any token starting with -- (or -letter that is not a tag operand) to argparse so it errors; compute changed = sorted(tags) != sorted(before); print a one-line notice for removing an absent tag; reject whitespace in tags; document or fold case; error when a name is given with --list. Regenerate docs/commands.html if the help text gains the tag grammar. Found by the 2026-08 CLI audit (cluster tag-arg-parsing); repro in the audit log. Partly landed — PR 735. The correctness half shipped: any unrecognized -- token now reaches argparse (unrecognized arguments, exit 2), --list with a skill name is a named error, whitespace in a tag is rejected, and changed is a before/after set comparison in the new lockfile.apply_tag_mods, so +x -x no longer writes the lock and a journal event for a net no-op. Still open, and why this card stays inflight: the one-line notice when -tag removes a tag that was never present (the remove branch is still a silent no-op), and documenting or folding tag case (+Design and #design still coexist). Both are UX asks rather than correctness bugs, which is why the PR left them.

Complexity S Impact Med Wow ★★ tag brainstorming --verbose exits 0 as a remove of '-verbose'; +x -x writes lock + journal
ShippedCLI · Bug

boost tap: CLI audit findings (2026-08)

Write-up · audit-tap-findings.md

Complexity S Impact Med Wow ★ a missing local dir is cloned as https://github.com//private/tmp/… before any check
ShippedCLI · UX

boost taps: CLI audit findings (2026-08)

Write-up · audit-taps-findings.md

Complexity S Impact Low Wow ★ one UPDATED column shows "@b29e7cf", "2026-07-24" and "11h ago" with no legend
ShippedCLI · UX

boost unpin: CLI audit findings (2026-08)

Write-up · audit-unpin-findings.md

Complexity S Impact Low Wow ★ unpin prints 'released the commit pin too' before the unpinned line it qualifies
ShippedCLI · UX

boost untap: CLI audit findings (2026-08)

Write-up · audit-untap-findings.md

Complexity S Impact Low Wow ★ tap takes several SPECs in parallel; untap still errors on a second name
ShippedCLI · Performance

boost update: CLI audit findings (2026-08)

Write-up · audit-update-findings.md

Complexity M Impact Med Wow ★ a no-op update over 20 taps takes ~14 s serial; --force drops 20 pins without a word
ShippedCLI · Bug

boost who: CLI audit findings (2026-08)

Write-up · audit-who-findings.md

Complexity S Impact Med Wow ★ 29 "skills" = 20 tap names + "10152 passages" + 5 cohort names + 3 real items
ShippedCLI · Audit

August 2026 full-CLI audit: every one of the 81 boost commands exercised and verified

Write-up · cli-audit-2026-08-full-sweep.md

Complexity L Impact High Wow ★★ 81 commands, ~2,000+ invocations, 367 findings, 161 clusters, 107 cards on the board
In flightCatalog · UX

Per-item categories in search/browse/info — not just a ★ curated bool

From a user request: “proper categories for skills (can't have all of them listed as just curated)”. They are right about the item level: the only taxonomy a catalog entry carries is a boolean. A boost search row shows name, kind, tap, description and at most a ★; boost recommend's no-match fallback is literally headed “curated picks”; boost info prints no category at all. Across a real install of tens of thousands of items, “starred or not” is the entire classification a user can see or filter by. What the code confirms. catalog._make_entry stamps "curated": curated onto every entry (boost_cli/core/catalog.py:119, signature at 105–106) — and that bool is per-tap, from Tap.curated (core/registry.py:23), set by tap --defaults or by anyone passing --curated (commands/taps.py:126) — a trust star, not a classification. Category-like data does exist, but only per tap: data/registries.json rows carry one (487 registries, 21 values; general alone covers 127), and exactly two surfaces read it — browse's row badge via _tap_categories (commands/discovery.py:936–941, whose own docstring says “catalog entries themselves carry no category, only their tap does”; badge appended last in _row_badges, discovery.py:961–963, so narrow panes drop it first, and taps outside the bundled 487 get none) — and boost serve's web facets (core/serve.py:65). cmd_search renders only the star (discovery.py:179) and takes no filter flag; info shows frontmatter tags when present (commands/info.py, the meta.get("tags") kv) but no category, and its --json has no such field. An item's own frontmatter category/tags ride along invisibly in entry["meta"] and the substring search_blob (catalog.py:131, 621–627), so they can match a query yet can never be displayed or filtered. Proposed fix. Stamp a first-class category on each entry at scan time in _make_entry (catalog.py:105–132): the item's frontmatter category (or first tag) when declared, else inherited from its tap's registry category — and bump catalog.CACHE_FORMAT so hundreds of existing tap caches backfill without a re-tap, per the versioned-cache rule. Then surface it where a category would live: a badge in search rows and a --category filter on search/browse/recommend, a kv row plus JSON field in info, and browse's existing badge switched from tap-level to the entry field (which also gives un-bundled taps' items a label for the first time). ★ keeps meaning curation/trust only. Consumers must degrade cleanly when category is absent (old caches, synthesised entries), same as the content digest rule. Docs: regenerate docs/commands.html for the new flags; no other doc names categories. Found by the 2026-08 CLI audit (cluster catalog-categories-beyond-curated, filed from the user's request); repro in the audit log. Verified against source 2026-08-31. Status (2026-09-01). Landed: the category stamp at scan time (catalog._entry_category, own frontmatter category → first tags entry → tap's registry category), CACHE_FORMAT bumped to 2 so existing caches backfill on next scan, a --category filter on search/browse/recommend (catalog.matches_category/filter_by_category), info's kv row and --json field, and browse's row badge switched from the tap-level lookup to the entry's own field (falling back to the tap lookup for a cache not yet rescanned). Not done: the badge in plain boost search rows. That row's column widths (out.search_layout/format_search_row) are a tuned, heavily-pinned budget system (drop order, per-cap name shrinking, a reserved curated tail) — working it out safely needs its own pass rather than a bolt-on inside this PR. Left inflight rather than shipped for that reason; the next claim on this item is scoped to exactly that piece.

Complexity M Impact Med Wow ★★ landed everywhere except the search-row badge — see PR for what remains

Docs-site & content quality

// planned · free tooling for the Pages site, README & prose
ShippedDocs

The OpenSSF badge playbook

Write-up · openssf-playbook.md

Complexity S Impact Med Wow ★★★ the method, not the answers — so another repo can repeat it
ShippedDocs · Onboarding

The engine had no architecture diagram — and the one written rule was documented backwards

Write-up · internal-architecture-diagrams.md

Complexity M Impact Med Wow ★★★ 45 core modules and an enforced layering rule, with no diagram of either
ShippedDocs · Design system

A GIF carousel touring one flagship command per group

Write-up · command-carousel-page.md

Complexity M Impact Med Wow ★★★★ 8 GIFs · one flagship per group
ShippedDocs · Performance

The Lighthouse budget passes on noise, not on margin

Write-up · roadmap-perf-budget-has-no-local-guard.md

Complexity S Impact Medium Wow ★★★★ main passes this budget on run-to-run luck — its own three runs are 0.810, 0.840, 0.850
ShippedDocs · Interop

an explainer page for the LangChain / LangGraph / LangSmith integration

Write-up · langchain-integration-explainer-page.md

Complexity M Impact Med Wow ★★★★ the eval.html genre, pointed at the LangChain stack — and it can only document what has shipped
ShippedDocsite · Bug

Expanded card bodies overflow the roadmap board sideways

Write-up · expanded-card-bodies-overflow.md

Complexity S Impact Med Wow ★★★ the closed <details> was added for paint cost and is quietly also the only thing keeping long code tokens on the page
PlannedCI · Bug

The performance gate flips on byte-identical input

lighthouse asserts categories.performance ≥ 0.80 on docs/roadmap.html. That page now scores 0.76–0.78, so the assertion is decided by whichever way runner timing falls, not by the page. Two commits on one branch, whose docs/roadmap.html is byte-identical (sha256 42bdc4f7…, 784,560 bytes both), got opposite verdicts — c75f4a9f failed, 037a2468 passed, 60 seconds apart. The same oscillation is visible on other branches in the run history: loop/missing-json failed at 2a5231cb and passed at 1c949b7d; loop/audit-retrieval-search-onboarding failed twice then passed. Three earlier cards closed real findings here — lighthouse-ci-on-the-pages-site installed the budgets, lighthouse-scored-a-page-nobody-is-served fixed a harness sending 3.27x the bytes Pages sends, and roadmap-page-weight-grows-without-bound found it was never the bytes but 705 ms of styleLayout, and bought headroom by letting 55.5% of elements skip it. That headroom has since been spent: the board was 407.6 KB when that card was written and is 787 KB now. So this is not a re-file of any of them — it is what happens after their fix, and the symptom is different in kind: a gate that cannot decide rather than a page that is slow. Why it matters more than two hundredths of a score: a check that fails on input it just passed is a check people learn to re-run rather than read, and the next real regression arrives looking exactly like the last false one. lighthouse is advisory, not required, which is what has let it drift this far unnoticed. Fix — decide which guarantee is wanted, then make the gate express it: assert the median of the three runs rather than the worst, or floor at a value the page actually clears and put a separate bound on the growth that is eating the margin, so the number that moves is the one being regressed. Lowering the floor alone just relocates the coin flip.

Complexity M Impact High Wow ★★★★ the same roadmap.html scored 0.78 and passed, then 0.78 and failed — the gate now decides on runner noise

Compatibility & install integrity

// planned · free tooling to prove boost installs & runs everywhere it claims
ShippedCompat

What Gemini actually receives from boost, audited

Write-up · gemini-mcp-parity.md

Complexity M Impact Med Wow ★★★ audited what Gemini actually receives — two of three findings were our own wrong claims
ShippedCompat

boost hooks learns a second host — and finds two bugs upstream

Write-up · gemini-hooks-host-aware.md

Complexity M Impact Med Wow ★★★★ two upstream Gemini bugs found while establishing the schema
ShippedRelease

ship the LangChain integration inside the wheel, behind a [langchain] extra

Write-up · langchain-in-the-wheel.md

Complexity M Impact High Wow ★★★ same import, same tests, zero new infrastructure — the wheel that already ships on every merge carries the integration too
ShippedBug

sanitize agent frontmatter for Gemini instead of copying it verbatim

Write-up · gemini-agent-frontmatter-sanitizer.md

Complexity S Impact Med Wow ★★ measured on Gemini CLI 0.53.1 — a boost-installed agent fails Zod validation at startup, and hand-fixes regress on the next sync
ShippedInterop · Registry

garrytan/gstack — tap it first, then learn to coexist with it

Write-up · gstack-tap-first-then-coexistence.md

Complexity M Impact Med Wow ★★★ 130k stars of SKILL.md that boost can index today, and a second installer writing into the same dotdirs

Skill-content trust & safety

// planned · boost's core threat model — the third-party skills it installs run inside an agent
ShippedSecurity · Content

The MCP boost_install tool skipped the injection scan the CLI runs

Write-up · mcp-install-skips-the-injection-scan.md

Complexity S Impact High Wow ★★★★ the one install path with no human watching was the one not scanning
ShippedSupply chain

sbom.yml has never run — it waits for an event GITHUB_TOKEN cannot emit

Write-up · sbom-release-event-never-fires.md

Complexity S Impact High Wow ★★★★ 253 releases shipped with no SBOM; fixed with workflow_run
ShippedRelease safety

The code_scanning ruleset rule can go back on — but only scoped to CodeQL

Write-up · code-scanning-rule-can-be-restored.md

Complexity S Impact Med Wow ★★★★ safe scoped to CodeQL (0 open) — adding Scorecard deadlocks every merge
ShippedSupply chain · Bug

The SBOM can declare a different version than the release it is attached to

Write-up · sbom-declares-the-wrong-version.md

Complexity S Impact High Wow ★★★★★ fixed — release v1.0.278's SBOM declared 1.0.277; the version is now pinned to the tag
ShippedSecurity · Correctness

rules and workflows install, then cannot be governed

Write-up · rules-install-but-cannot-be-governed.md

Complexity L Impact High Wow ★★★★★ an upstream push rewrote CLAUDE.md silently; pin and quarantine both answered "not installed"
ShippedSecurity · Bug

boost serve echoed the request path back into its 404 body

Write-up · serve-404-reflected-the-request.md

Complexity S Impact Med Wow ★★★ the test suite pinned the echo in place as if it were the contract
ShippedTrust · UX

boost attest: CLI audit findings (2026-08)

Write-up · audit-attest-findings.md

Complexity S Impact Low Wow ★ a deleted store dir is reported as a sha mismatch; drift names the same state correctly
ShippedCLI · Bug

boost trust: CLI audit findings (2026-08)

Write-up · audit-trust-findings.md

Complexity S Impact Med Wow ★ trust add of a missing .pub path blames "invalid base64 in minisign data"