AUDIT: 6-layer knowledge pipeline. 22 hooks. 33 skills. 5 persistence mechanisms.
FINDING: 96% noise rate. 222 unprocessed items. 4 solutions documented against 81 known bugs.
The knowledge system had every feature you’d put on a slide deck. Extraction, staging, curation, indexing, querying — the whole pipeline. Six layers deep, auto-triggered, with hooks firing on session start, every file read, every user message, and session end.
It was also functionally abandoned.
The Audit
I built this system over months. Cross-project pattern capture. A ChromaDB vector index with 550 chunks of curated knowledge and 25,000 chunks of personal context. Skills that extract lessons, document fixes, curate inboxes, rebuild indices. Hooks that detect errors, suggest solutions, surface relevant patterns inline while you’re reading code.
The architecture was flawless. The operations were a ghost town.
Here’s what the audit found:
| Component | Designed For | Actual State |
|---|---|---|
/preserve | Full knowledge pipeline | Not run in weeks |
/document-fix | Capture solutions | 4 entries total (81 bugs documented) |
| Inbox curation | Weekly review | 222 items, last run 6 days ago |
| Pattern verification | Confirm accuracy | 95% unverified, top pattern at 982 hits — never confirmed |
| Cross-project sharing | Shared knowledge hub | Theoretical — each project siloed |
| Index rebuild | Automatic freshness | Manual only, no trigger |
Five different persistence mechanisms existed: MEMORY.md, Plans directory, LESSONS_LEARNED files, inbox staging, and SecondBrain. Nobody knew which one to use when. Including me. Especially me.
The Dwarf Warrior had built an armory. Every weapon sharpened, every trap set, every corridor mapped. Then the party walked past it all and kept fighting with sticks.
The Wrong Theory
The instinct was “we need more discipline.” Run /preserve more often. Curate the inbox weekly. Remember to verify patterns. Make a calendar reminder.
How many knowledge management systems have you built that you actually use six months later?
If a process requires a human to remember to do it, the process doesn’t exist. It’s an aspiration with a file extension. You’re not going to curate 222 items by hand every week. You’re not going to remember to run a rebuild after editing a lessons file. You’re not going to verify patterns when you’re three hours into a debugging session and the fix is right there.
The Diviner could see every room in the dungeon from her tower. She had the map, the index, the cross-references. But she never checked whether the doors were actually locked. Perspective without operations is just expensive wallpaper.
The real problem wasn’t missing features. It was missing connections.
What We Actually Fixed
Three surgical changes. No new systems. No new files. Just wiring.
Fix 1: /wrapup now runs /preserve automatically.
The end-of-session skill already existed. The knowledge extraction skill already existed. They just weren’t connected. Now wrapup includes a Step 6 that runs smart-extract and index rebuild, with graceful fallback if the local LLM is offline. One edit to skill.md. The pipeline closes itself.
Fix 2: /curate-patterns now rebuilds the index.
Curation merged patterns into curated files, but the search index didn’t know. You’d curate 13 patterns, then /ask-knowledge would return stale results. Now curation ends with an automatic index rebuild. One step added. Search reflects reality.
Fix 3: Cleared the entire 222-item backlog.
| Inbox | Reviewed | Merged | Rejected | Noise |
|---|---|---|---|---|
| Bug Patterns | ~100 | 5 | ~95 | 95% |
| Claude Limitations | ~51 | 1 | ~50 | 98% |
| Testing Strategies | ~30 | 0 | ~30 | 100% |
| Feature Ideas | ~50 | 2 | ~48 | 96% |
| Playbooks | 3 | 5 | 0 | 0% |
| Total | ~234 | 13 | ~223 | 95% |
Thirteen patterns survived. Five new RAG-specific bug patterns. Five new operational playbooks. Two feature ideas. One Claude limitation. Everything else was session chatter, implementation notes, and configuration details that the auto-extractor mistook for wisdom.
95% noise. The extraction system was a firehose pointed at a whiteboard.
The Ghost
Here’s the part that made me sit back.
While auditing the hooks directory, I found a file called deepseek_tools.py. 598 lines. Fully implemented. Production-ready. A validation layer designed to sit between extraction and the inbox — score each item for quality, reject noise, catch duplicates, auto-correct category misassignments.
It had never been connected to anything.
The tool that would have prevented the 222-item backlog, the 96% noise rate, the manual curation grind — it existed. It was complete. It was tested. It was sitting in the same directory as the extraction pipeline it was designed to filter.
Nobody plugged it in.
The Rogue had already picked the lock. The door was still closed because nobody told the Warrior to push it open.
Rook Mode
Three changes shipped today:
~/.claude/skills/wrapup/skill.md— Added Step 6: auto-preserve (extract + index rebuild, graceful Ollama fallback)~/.claude/skills/curate-patterns/skill.md— Added Step 5: auto-rebuild ChromaDB index after curationshared/*.md— 13 patterns merged from inbox; all 5 inbox files cleared to headers
Next session — wire the validator:
- Route
deepseek_tools.filter_extractions()through Mistral (free tier, already available via existing gateway) - Insert between
smart_extract()return andappend_to_inbox()inollama_tools.py - Expected noise reduction: 96% → 10-20%
- Graceful degradation: API offline → all items pass through (current behavior)
System state after today:
| Before | After |
|---|---|
/wrapup captures status only | /wrapup extracts + indexes automatically |
| Curation leaves index stale | Curation triggers rebuild |
| 222 items in inbox | 0 items in inbox |
| Validator unused | Validator queued for next session |
The most expensive code isn’t the code you haven’t written. It’s the code that’s finished, tested, and sitting in a directory you forgot to open.
Stop building armories. Start checking if the doors are unlocked.