Directory Structure
core/
Processing EngineTasks, workflows, protocols, schemas, and intelligence scripts that power the knowledge pipeline.
agents/
AI AgentsMind clones, cargo roles, conclave deliberation agents, and agent templates.
.claude/
Claude IntegrationHooks, skills, slash commands, and rules for Claude Code integration.
knowledge/
Knowledge BasePlaybooks, dossiers, DNA schemas, and sources with full traceability.
artifacts/
Processing ArtifactsIntermediate pipeline outputs: chunks, insights, narratives, and extractions.
inbox/
Input DirectoryRaw materials for processing: videos, PDFs, transcriptions, courses.
Core Components
Processing Engine (core/)
The core engine contains everything needed to transform raw materials into structured knowledge:
core/tasks/
core/tasks/
Task definitions for atomic operations:
extract-dna.md- Extract 5-layer DNA from sourcesanalyze-themes.md- Identify themes and patternsnormalize-entities.md- Resolve entity namesprocess-batch.md- Batch processing orchestrationvalidate-cascade.md- Validation and integrity checks
core/workflows/
core/workflows/
YAML workflow definitions:
wf-pipeline-full.yaml- Complete 5-phase pipelinewf-ingest.yaml- Material ingestionwf-extract-dna.yaml- DNA extraction workflowwf-conclave.yaml- Multi-agent deliberation
core/schemas/
core/schemas/
JSON Schema definitions for all state files:
chunks-state.schema.jsoninsights-state.schema.jsonnarratives-state.schema.jsoncanonical-map.schema.jsondecisions-registry.schema.json
core/intelligence/
core/intelligence/
Python intelligence scripts:
audit_layers.py- Layer system validation- RAG and semantic processing utilities
- Quality control and validation scripts
Agent System (agents/)
Mega Brain includes a hierarchical agent system with four main types:
Claude Code Integration (.claude/)
The .claude/ directory contains all Claude Code integration files:
- Hooks
- Commands
- Skills
20+ lifecycle hooks for automated operations:
agent_index_updater.py- Auto-update agent catalogcontinuous_save.py- Session auto-saveenforce_dual_location.py- Dual-location loggingmemory_persister.py- Agent memory updatesinbox_age_alert.py- Unprocessed material alertscreation_validator.py- File creation validation
Knowledge Base (knowledge/)
All extracted and structured knowledge lives here:
Every piece of knowledge traces back to source material with
chunk_id, file path, and original context.Pipeline Artifacts (artifacts/)
Intermediate processing stages store data here:
State Management
Mega Brain maintains several critical state files:| State File | Purpose | Location |
|---|---|---|
CHUNKS-STATE.json | All content chunks | artifacts/chunks/ |
CANONICAL-MAP.json | Entity name resolution | artifacts/canonical/ |
INSIGHTS-STATE.json | Extracted insights | artifacts/insights/ |
NARRATIVES-STATE.json | Synthesized narratives | artifacts/narratives/ |
AGENT-INDEX.yaml | Agent registry | agents/ |
file-registry.json | Processed files tracking | system/REGISTRY/ |
Binaries and CLI (bin/)
Command-line tools for setup and operations:
setup.js- Interactive setup wizardcli.js- Main CLI entry pointpre-publish-gate.js- Layer validation before publishingvalidate-layers.js- Layer system integrity check
Development Workflow
Next Steps
Knowledge Pipeline
Learn about the 8-phase processing pipeline
DNA Schema
Understand the 5-layer knowledge extraction
Layer System
Explore the L1/L2/L3 distribution layers
Agents
Deep dive into the agent system
