Index File Format Specification V2.0

Complete guide to V2.0 index.codex.yaml format - proper codex format with metadata, children, and NO path fields

Last updated: 2025-11-02

Index.codex.yaml Format Specification V2.0

The index.codex.yaml file follows the ChapterWise Codex Format V1.0 specification, making it a true codex document. It uses metadata, recursive children arrays, and derives paths from the hierarchy.

What's New in V2.0

✨ Proper Codex Format: Index is now a real codex document with metadata + children ✨ NO Path Fields: All paths computed from hierarchy (name fields) ✨ Emoji & Color Support: Visual customization via attributes array ✨ Name/Title Logic: name = actual filename, title = display name (at all levels) ✨ DB Sync: Index changes auto-update Project database fields

Overview

The V2.0 index file:

  • Codex V1.0 Format: Follows standard codex specification
  • Metadata: Standard codex metadata object
  • Children Array: Single recursive structure (no separate structure/files)
  • NO Paths: Paths derived from hierarchy by concatenating name fields
  • Attributes: Emoji, color stored in attributes array
  • Auto-Sync: Changes synced to database on git pull

Basic Structure

# index.codex.yaml - V2.0 Format

metadata:
  formatVersion: "1.0"
  documentVersion: "1.0.0"
  created: "2025-11-02T00:00:00Z"
  author: "Your Name"

id: "index-root"
type: "index"
name: "MyProject"
title: "My Project Title"
summary: "Project description"

attributes:
  - key: "emoji"
    value: "📚"
  - key: "color"
    value: "#10B981"
  - key: "mainFile"
    value: "README.md"

children:
  - id: "folder-characters"
    type: "folder"
    name: "Characters"
    title: "Character Codex"
    order: 1
    expanded: true
    attributes:
      - key: "emoji"
        value: "👤"

    children:
      - id: "file-hero"
        type: "codex"
        name: "hero.codex.yaml"
        title: "Hero"
        order: 1
        # Computed path: Characters/hero.codex.yaml

  - id: "file-readme"
    type: "markdown"
    name: "README.md"
    title: "Overview"
    order: 2
    # Computed path: README.md

Field Reference

Root Level Fields

Field Type Required Description
codexVersion string No Schema version (default: "1.0")
name string No Project display name
description string No Project description
mainFile string No Main file to display (e.g., "README.md")
default_format string No Preferred file format for project: "yaml" or "json" (default: "yaml")
structure array No Folder structure definitions
files array No File list with metadata
patterns object No Include/exclude patterns
display object No UI display preferences
metadata object No Additional project metadata

Note on default_format: This field specifies the preferred file format for the project. When set to "yaml" (default), ChapterWise will use YAML format for auto-generated files and when regenerating the index. Set to "json" if you prefer JSON format throughout the project. This setting helps maintain consistency across all codex files in your repository.

Structure Fields (Folders)

Define folders in your project:

structure:
  - path: "Characters/"              # Required: folder path (must end with /)
    title: "All Characters"          # Optional: display name
    description: "Character codex"   # Optional: folder description
    expanded: true                   # Optional: initially open? (default: false)
    order: 1                         # Optional: display order (lower = earlier)
    hidden: false                    # Optional: hide from view (default: false)
    icon: "ph-users"                 # Optional: custom Phosphor icon

Folder Field Reference:

Field Type Default Description
path string Required Folder path relative to root (must end with /)
title string Folder name Display name in UI
description string None Folder description for tooltips
expanded boolean false Initially expanded in tree view
order number 999 Display order (ascending)
hidden boolean false Hide folder from display
icon string "ph-folder" Phosphor icon name

File Fields

Define individual files:

files:
  - path: "README.md"                         # Required: file path
    title: "Introduction"                     # Optional: display name
    description: "Project overview"           # Optional: file description
    order: 1                                  # Optional: display order
    hidden: false                             # Optional: hide from view
    icon: "ph-star"                           # Optional: custom icon

File Field Reference:

Field Type Default Description
path string Required File path relative to root
title string Filename Display name in UI
description string None File description for tooltips
order number 999 Display order (ascending)
hidden boolean false Hide file from display
icon string Auto-detected Custom Phosphor icon
url_path string Auto-generated Custom URL path (see URL Path Mapping)

Important: No type field needed! File types are automatically detected from extensions.

URL Path Mapping - NEW! ⭐

Create cleaner, more memorable URLs for your files that differ from the actual file paths! Perfect for branding, SEO, and sharing.

Automatic URL Generation

By default, ChapterWise automatically generates clean URLs by stripping file extensions:

files:
  # File path: E02/characters/Aya.codex.yaml
  # Auto-generated URL: /E02/characters/Aya
  - path: "E02/characters/Aya.codex.yaml"
    title: "Aya"

URL Transformation Rules: - Strips .codex.yaml, .codex.json, .md, .txt extensions - Keeps directory structure - Adds leading / - Result: Clean, shareable URLs

Custom URL Paths

Override the default with url_path for complete control:

files:
  # Traditional URL: /projects/{id}/file/E02/characters/Aya.codex.yaml
  # Custom URL:     /projects/{id}/11L02/characters/Aya
  - path: "E02/characters/Aya.codex.yaml"
    title: "Aya"
    url_path: "/11L02/characters/Aya"

  # Simplify complex paths
  - path: "MANUSCRIPT/2025-09-07 11L02 BOOK 1G.html"
    title: "Book 1G Draft"
    url_path: "/11L02/book-1g"

  # Create branded URLs
  - path: "E02/Concepts.codex.yaml"
    title: "Core Concepts"
    url_path: "/11L02/concepts"

Benefits

✅ Cleaner URLs: Remove file extensions and technical paths
✅ Branding: Use project-specific naming (e.g., "11L02")
✅ Flexibility: Change file structure without breaking links
✅ SEO-friendly: More readable URLs for sharing
✅ Backwards Compatible: Works seamlessly with access keys

URL Format Requirements

  • Must start with /
  • Cannot end with /
  • Max length: 500 characters
  • Must be unique within project
  • Avoid invalid characters: < > " { } | \ ^ [ ]`

Example: 11 Lives Project

files:
  # Core files with branded URLs
  - path: "E02/Concepts.codex.yaml"
    url_path: "/11L02/concepts"

  - path: "E02/The Metatron - Characters.codex.yaml"
    url_path: "/11L02/metatron"

  # Character files with clean paths
  - path: "E02/characters/Aya.codex.yaml"
    title: "Aya"
    url_path: "/11L02/characters/Aya"

  - path: "E02/characters/Maya.codex.yaml"
    title: "Maya"
    url_path: "/11L02/characters/Maya"

  # Files without url_path use auto-generated URLs
  - path: "E02/Factions.codex.yaml"
    # Auto: /E02/Factions

Result URLs: - Custom: https://codex.chapterwise.app/projects/{id}/11L02/characters/Aya - Auto: https://codex.chapterwise.app/projects/{id}/E02/Factions

Mixed Usage

You can mix custom and auto-generated URLs in the same project:

files:
  # Custom URLs for important/shared files
  - path: "hero.codex.yaml"
    url_path: "/characters/hero"

  # Auto URLs for internal files (just omit url_path)
  - path: "drafts/notes.md"
    # Auto: /drafts/notes

  - path: "research/references.md"
    # Auto: /research/references

Reserved Paths

Avoid these reserved paths as the first segment: - /file/ - Traditional file route - /files/ - Static file serving - /api/ - API endpoints - /index/ - Index management - /change-branch, /sync, /install-webhook - Git operations

Validation

ChapterWise validates URL paths to ensure: - No duplicate URLs mapping to different files - No invalid characters - Proper normalization (leading /, no trailing /) - No conflicts with reserved paths

Access validation API: GET /projects/{project_id}/index/validate

Pattern Fields

Control which files are included:

patterns:
  include:
    - "*.codex.yaml"      # Include all .codex.yaml files
    - "*.codex.json"      # Include all .codex.json files
    - "*.md"              # Include all markdown files
    - "docs/**/*.yaml"    # Include YAML files in docs/ subdirectories

  exclude:
    - "_ARCHIVE/**"       # Exclude everything in _ARCHIVE/
    - "**/node_modules/**" # Exclude node_modules directories
    - "**/.git/**"        # Exclude .git directories
    - "**/temp/**"        # Exclude temp directories

Default Patterns (if not specified):

patterns:
  include:
    - "*.codex.yaml"
    - "*.codex.json"
    - "*.md"
  exclude:
    - "**/node_modules/**"
    - "**/.git/**"
    - "**/__pycache__/**"
    - "**/venv/**"

Display Fields

Control UI appearance:

display:
  defaultView: "tree"      # View mode: "tree", "list", or "grid"
  sortBy: "order"          # Sort by: "order", "name", "modified", or "type"
  groupBy: "folder"        # Group by: "folder", "type", or "none"
  showHidden: false        # Show hidden files/folders
Field Type Options Default Description
defaultView string tree, list, grid "tree" Initial view mode
sortBy string order, name, modified, type "order" Sort criteria
groupBy string folder, type, custom, none "folder" Grouping method
showHidden boolean true, false false Show hidden items

Groups (Virtual Folders) - NEW! ⭐

Create virtual folders that don't match your filesystem! Perfect for organizing flat files or creating multiple views of the same content.

# Set groupBy to "custom" to use groups
display:
  groupBy: "custom"

# Define virtual groups
groups:
  - name: "Main Characters"
    description: "Primary protagonists"
    icon: "ph-users"
    order: 1
    expanded: true
    files:
      - "hero.codex.yaml"
      - "sidekick.codex.yaml"

  - name: "Antagonists"
    icon: "ph-skull"
    order: 2
    files:
      - "villain.codex.yaml"
      - "dark-lord.codex.yaml"

  - name: "Magic System"
    order: 3
    files:
      - "magic-rules.md"
      - "spells.codex.yaml"

Groups Field Reference:

Field Type Required Description
name string Yes Group display name
files array Yes Array of file paths to include in this group
description string No Group description for tooltips
icon string No Custom Phosphor icon (default: ph-folder)
order number No Display order (ascending, default: 999)
expanded boolean No Initially expanded in tree view? (default: false)

Key Benefits:

  • 📁 Virtual Organization: Files stay in place, but appear in custom groups
  • 🔄 Multiple Views: Same file can appear in multiple groups
  • 🎯 Flat File Friendly: Perfect for projects with files in root directory
  • 🎨 Flexible: Mix grouped and ungrouped files

Example Use Cases:

  1. Flat Directory Organization: ```yaml # All files in root: char-hero.yaml, char-villain.yaml, loc-city.md groups:

    • name: "Characters" files: ["char-hero.yaml", "char-villain.yaml"]
    • name: "Locations" files: ["loc-city.md"] ```

  2. Cross-Cutting Concerns: ```yaml # Same character appears in "Book 1" and "Magic Users" groups groups:

    • name: "Book 1 Cast" files: ["hero.yaml", "wizard.yaml"]
    • name: "Magic Users" files: ["wizard.yaml", "sorceress.yaml"] # wizard appears in both! ```

  3. Series Organization: ```yaml groups:

    • name: "Book 1: The Beginning" files: ["book1/characters.yaml", "book1/plot.md"]
    • name: "Book 2: The Journey" files: ["book2/characters.yaml", "book2/plot.md"] ```

Auto-Type Detection

File types are automatically detected from extensions. You never need to specify a type field!

Supported Extensions

Extension Detected Type Viewer Icon
.codex.yaml codex Full Codex Viewer (structured data) ph-file-code
.codex.json codex Full Codex Viewer (structured data) ph-file-code
.md markdown Markdown Viewer (rendered) ph-file-text
.txt text Plain Text Viewer (monospace) ph-file-text
.json json Code Viewer ph-brackets-curly
.yaml, .yml yaml Code Viewer ph-file-code

Example: Auto-Detection in Action

files:
  - path: "characters.codex.yaml"    # Auto-detected as type: "codex"
  - path: "README.md"                # Auto-detected as type: "markdown"
  - path: "data.json"                # Auto-detected as type: "json"
  - path: "config.yaml"              # Auto-detected as type: "yaml"

Hierarchical Structure

Basic Hierarchy

structure:
  # Top-level folders
  - path: "Characters/"
    order: 1

  - path: "Factions/"
    order: 2

  - path: "Concepts/"
    order: 3

Nested Hierarchy

structure:
  # Parent folder
  - path: "Characters/"
    title: "All Characters"
    expanded: true
    order: 1

  # Child folders (determined by path)
  - path: "Characters/Main/"
    title: "Main Characters"
    expanded: true
    order: 1

  - path: "Characters/Supporting/"
    title: "Supporting Cast"
    order: 2

  - path: "Characters/Antagonists/"
    title: "Villains"
    order: 3

Multi-Level Organization

structure:
  - path: "Worldbuilding/"
    expanded: true

  - path: "Worldbuilding/Geography/"
    title: "Locations & Geography"

  - path: "Worldbuilding/Geography/Continents/"

  - path: "Worldbuilding/Geography/Cities/"

  - path: "Worldbuilding/Culture/"

  - path: "Worldbuilding/Culture/Languages/"

  - path: "Worldbuilding/Culture/Customs/"

Minimal Examples

Absolute Minimum

# Simplest possible index
files:
  - path: "README.md"
  - path: "main.codex.yaml"

Just Patterns

# Only customize what's included
patterns:
  include:
    - "docs/*.codex.yaml"
  exclude:
    - "_ARCHIVE/**"

Just Structure

# Define folder order only
structure:
  - path: "01-Introduction/"
    order: 1
  - path: "02-Characters/"
    order: 2
  - path: "03-Worldbuilding/"
    order: 3

Just File Order

# Control file display order
files:
  - path: "README.md"
    order: 1
  - path: "overview.md"
    order: 2
  - path: "characters.codex.yaml"
    order: 3

Complete Example

Here's a comprehensive example showing all features:

# index.codex.yaml - Complete Example

# Project Metadata
codexVersion: "1.0"
name: "Epic Fantasy Series - World Bible"
description: "Comprehensive codex for a multi-book fantasy series"
mainFile: "README.md"
default_format: "yaml"

# Hierarchical Structure
structure:
  # Main Categories
  - path: "Characters/"
    title: "Characters"
    description: "All character profiles and development notes"
    expanded: true
    order: 1
    icon: "ph-users"

  - path: "Characters/Protagonists/"
    title: "Main Characters"
    description: "Primary POV characters"
    expanded: true
    order: 1

  - path: "Characters/Antagonists/"
    title: "Villains & Antagonists"
    order: 2

  - path: "Characters/Supporting/"
    title: "Supporting Cast"
    order: 3

  - path: "Worldbuilding/"
    title: "World & Setting"
    expanded: false
    order: 2
    icon: "ph-globe"

  - path: "Worldbuilding/Geography/"
    title: "Locations & Maps"

  - path: "Worldbuilding/Magic/"
    title: "Magic System"

  - path: "Factions/"
    title: "Factions & Organizations"
    order: 3
    icon: "ph-shield"

  - path: "Timeline/"
    title: "Historical Timeline"
    order: 4
    icon: "ph-clock"

  - path: "_ARCHIVE/"
    title: "Archived Content"
    hidden: true

# File Definitions
files:
  # Root files
  - path: "README.md"
    title: "Project Overview"
    order: 1

  - path: "QUICKSTART.md"
    title: "Quick Start Guide"
    order: 2

  # Characters
  - path: "Characters/Protagonists/Hero.codex.yaml"
    title: "The Hero"
    description: "Reluctant hero destined to save the world"
    order: 1

  - path: "Characters/Protagonists/Mentor.codex.yaml"
    title: "The Mentor"
    description: "Wise guide with mysterious past"
    order: 2

  - path: "Characters/Antagonists/DarkLord.codex.yaml"
    title: "The Dark Lord"
    description: "Primary antagonist"
    order: 1

  # Worldbuilding
  - path: "Worldbuilding/Geography/WorldMap.md"
    title: "World Map & Geography"

  - path: "Worldbuilding/Magic/MagicSystem.codex.yaml"
    title: "Magic System Rules"

  # Factions
  - path: "Factions/Guilds.codex.yaml"
    title: "Guilds & Organizations"

  # Timeline
  - path: "Timeline/AncientHistory.md"
    title: "Ancient History"

  - path: "Timeline/ModernEra.md"
    title: "Current Era Events"

  # Hidden files
  - path: "NOTES.md"
    hidden: true

  - path: "_ARCHIVE/old-draft.codex.yaml"
    hidden: true

# Include/Exclude Patterns
patterns:
  include:
    - "*.codex.yaml"
    - "*.codex.json"
    - "*.md"
  exclude:
    - "_ARCHIVE/**"
    - "**/node_modules/**"
    - "**/.git/**"
    - "**/temp/**"
    - "**/*.backup"

# Display Preferences
display:
  defaultView: "tree"
  sortBy: "order"
  groupBy: "folder"
  showHidden: false

# Project Metadata
metadata:
  version: "2.1.0"
  authors:
    - "Primary Author Name"
    - "Co-Author Name"
  tags:
    - "fantasy"
    - "epic"
    - "series"
  created: "2024-01-15"
  updated: "2025-10-15"
  copyright: "© 2025 Author Name"

Progressive Enhancement

The index system works at multiple levels of sophistication:

Level 0: No Index

(No index file)

System automatically: - Scans all supported files - Creates folder structure from filesystem - Sorts alphabetically - Groups by folder

Level 1: Minimal Index

patterns:
  exclude:
    - "_ARCHIVE/**"

System uses patterns + auto-generates everything else.

Level 2: Partial Index

structure:
  - path: "Characters/"
    order: 1
  - path: "World/"
    order: 2

patterns:
  exclude:
    - "_ARCHIVE/**"

System uses structure + patterns + auto-generates file list.

Level 3: Full Index

name: "Project Name"
structure: [...]
files: [...]
patterns: [...]
display: [...]
metadata: [...]

Complete control over every aspect.

AI Assistant Guidelines

When creating or modifying index files, AI assistants should:

Analysis

  1. Scan Repository Structure: Understand the file organization
  2. Identify File Types: Count .codex.yaml, .md, and other files
  3. Detect Patterns: Find common folder structures
  4. Understand Purpose: Determine project type (characters, worldbuilding, etc.)

Generation

  1. Start Minimal: Only specify what's needed
  2. Use Clear Naming: Descriptive titles for folders and files
  3. Logical Ordering: Group related content together
  4. Progressive Detail: Add complexity only when beneficial

Best Practices

  1. Never Add type Field: It's auto-detected!
  2. Use Trailing Slashes: "Characters/" for folders
  3. Order Deliberately: Use order field for logical flow
  4. Hide Thoughtfully: Use hidden: true for work-in-progress
  5. Document Structure: Add description fields for clarity
  6. Set Format Preference: Use default_format: "yaml" or "json" to maintain consistency

Example AI Workflow

User: "Create an index for my fantasy novel characters"

AI Analysis:
- Found 15 .codex.yaml files
- Structure: Characters/Protagonists/, Characters/Villains/, Characters/Supporting/
- Main file: README.md exists

AI Generated Index:
```yaml
codexVersion: "1.0"
name: "Fantasy Novel - Character Codex"
mainFile: "README.md"
default_format: "yaml"

structure:
  - path: "Characters/Protagonists/"
    title: "Main Characters"
    expanded: true
    order: 1

  - path: "Characters/Villains/"
    title: "Antagonists"
    order: 2

  - path: "Characters/Supporting/"
    title: "Supporting Cast"
    order: 3

display:
  sortBy: "order"
  groupBy: "folder"

Index File Priority & Caching - NEW! ⭐

ChapterWise now uses a smart caching system for better performance and cleaner repos:

Priority Order

  1. index.codex.yaml (committed, canonical) - Highest priority
  2. index.codex.json (committed, JSON format)
  3. .index.codex.yaml (hidden cache, auto-generated)
  4. .index.codex.json (hidden cache, JSON)

How It Works

📁 Your Repository
├── index.codex.yaml      ← If this exists, ChapterWise uses it (committed)
├── .index.codex.yaml     ← Auto-generated cache (hidden, not committed)
├── Characters/
│   └── hero.codex.yaml
└── README.md

Behavior:

  • No index file: ChapterWise auto-generates and saves as .index.codex.yaml (hidden cache)
  • Hidden cache exists: Uses cached version for speed
  • Committed index exists: Always uses committed version (canonical)
  • Corrupt index: Falls back to auto-generation with detailed error report

Benefits:

  • ✅ Clean repos (hidden cache not committed)
  • ✅ Fast loading (cached index)
  • ✅ Easy customization (download, edit, commit)
  • ✅ Team sharing (committed index shared with team)

.gitignore Recommendation

Add to your repository's .gitignore:

# ChapterWise auto-generated index cache
.index.codex.yaml
.index.codex.json

This prevents the hidden cache from being committed while allowing the committed index.codex.yaml if you create one.

Validation & Error Handling - ENHANCED! ⭐

ChapterWise now provides detailed error reports with line numbers and suggestions:

Error Types

1. YAML Syntax Errors

❌ ERROR (line 12, col 5):
YAML syntax error: mapping values are not allowed here

💡 Suggestion: Check YAML indentation and ensure colons are followed by spaces

2. Missing Files

⚠️  WARNING:
File not found: Characters/deleted-character.codex.yaml

💡 Suggestion: Remove this entry or create the file at Characters/deleted-character.codex.yaml

3. Schema Validation

❌ ERROR:
File 3: 'path' field is required

💡 Suggestion: Every file and folder entry must have a 'path' field

4. Deprecated Fields

⚠️  WARNING:
File 1: 'type' field is deprecated

💡 Suggestion: Remove 'type' field - file types are auto-detected from extension

Validation API

Check your index file programmatically:

GET /projects/{project_id}/index/validate

Response:
{
  "valid": false,
  "has_index": true,
  "index_path": "index.codex.yaml",
  "is_committed": true,
  "error_count": 2,
  "warning_count": 1,
  "errors": [
    {
      "type": "yaml_syntax",
      "line": 12,
      "column": 5,
      "message": "YAML syntax error: ...",
      "suggestion": "Check YAML indentation...",
      "severity": "error"
    }
  ],
  "warnings": [...]
}

Fallback Behavior

If your committed index is corrupt:

  1. ChapterWise logs detailed errors with line numbers
  2. Falls back to auto-generated index
  3. Continues working - your project still loads!
  4. Displays warnings in UI to alert you

Example Log:

ERROR: Index file is corrupt with 2 error(s)
  - Line 12: YAML syntax error
  - Line 25: Missing required 'path' field
INFO: Falling back to auto-generated index
INFO: Saved auto-generated index as .index.codex.yaml (cache)

Auto-Generation

If no index exists, ChapterWise automatically generates one and saves it as a hidden cache:

Generated Structure

codexVersion: "1.0"
name: "Repository Name"
description: "Auto-generated index"
generated: "2025-10-15T10:30:00Z"
mainFile: "README.md"  # If found
default_format: "yaml"

structure:
  # All folders detected
  - path: "Characters/"
  - path: "Factions/"
  - path: "Concepts/"

files:
  # All supported files found
  - path: "README.md"
  - path: "Characters/hero.codex.yaml"
  - path: "Factions/guilds.codex.yaml"

patterns:
  include:
    - "*.codex.yaml"
    - "*.codex.json"
    - "*.md"
  exclude:
    - "**/node_modules/**"
    - "**/.git/**"
    - "**/__pycache__/**"

Security & Performance

Path Security

  • All paths validated (no traversal attacks)
  • Files outside repository blocked
  • Gitignore patterns respected

Performance Limits

  • Index file max size: 1MB
  • Max files to scan: 10,000
  • Scan timeout: 30 seconds
  • Cache TTL: 1 hour

Best Practices

  • Keep index files under 100KB
  • Use patterns to exclude large directories
  • Don't specify every file individually
  • Let auto-detection handle most files

Next Steps

Now that you understand the index format:

  1. See Examples - Real-world index files
  2. Read Best Practices - Tips for organizing
  3. Create Your Index - Start building

Related Documentation: - Git Codex Projects Overview - Best Practices Guide - Example Index Files - Troubleshooting