Vibe Compiler (vibec)

A self-compiling tool to process vibe-coded projects using prompt stacks and LLM generation.

Overview

vibec transforms stacks of prompts into code and tests, supporting static .md and dynamic .js plugins. It outputs staged artifacts (output/stages/) for Git history and a current runtime version (output/current/) aggregated from all stages with a "Last-Wins" strategy. It can compile itself using its own structure.

Project Structure

stacks/: Prompt stacks (e.g., core/, generation/, tests/).
- Numbered prompts: 001_cli.md (processed sequentially).
- Multi-file output syntax:
```
# CLI Parser
Generate a CLI parser for a Node.js tool.
## Output: core/cli.js
## Output: core/cli_utils.js
```
- plugins/: Included in every LLM request for the stack.
  - .md: Static text (e.g., "Use ES6 syntax").
  - .js: Dynamic async plugins (e.g., async (context) => { ... }).
output/: Generated files.
- stages/: Numbered dirs (e.g., 001/core/cli.js, 001/core/cli_utils.js).
- current/: Latest files (e.g., core/cli.js), merged with "Last-Wins" (later stages overwrite earlier ones).
.vibec_hashes.json: Tracks prompt hashes and test results.
bootstrap.js: Runs self-compilation.
vibec.json: Optional config.
bin/vibec-prebuilt.js: Prebuilt minimal vibec.

Compilation Architecture

Processing Order

vibec processes prompts in a specific order:

Numerical Order First: All prompts with the same numerical prefix (e.g., 001_*.md) are processed before moving to the next number (002_*.md).
Stack Order Second: Within each numerical stage, stacks are processed in the order specified (e.g., if --stacks core,generation,tests, then core/001_*.md → generation/001_*.md → tests/001_*.md).

This ordering ensures that:

Each numerical stage represents a logical progression in your codebase
Dependencies between stacks within the same stage are respected
Later numerical stages can build upon earlier ones

Stage Processing Flow

flowchart TD
    subgraph "Stage_001"
        S1C[core] --> S1G[generation]
        S1G --> S1T[tests]
    end
    
    subgraph "Stage_002"
        S2C[core] --> S2G[generation]
        S2G --> S2T[tests]
    end
    
    Stage_001 --> Stage_002

Merging Strategy

After all stages are processed, generated files are merged into output/current/ using a "Last-Wins" strategy:

Files from later numerical stages overwrite earlier ones
Within the same numerical stage, files from later stacks in the processing order overwrite earlier ones

CLI

vibec --stacks core,generation,tests --test-cmd "npm test" --retries 2 --plugin-timeout 5000 --no-overwrite

--stacks: Comma-separated stack list.
--test-cmd: Test command.
--retries: Retry failed tests (default: 0).
--plugin-timeout: Max ms for .js plugins (default: 10000).
--no-overwrite: Fail if output/current/ files would be overwritten.

Configuration

vibec.json:

{
  "stacks": ["core", "generation", "tests"],
  "testCmd": "npm test",
  "retries": 2,
  "pluginTimeout": 5000,
  "pluginParams": {
    "dump_files": { "files": ["src/main.js", "README.md"] }
  }
}

Env vars: VIBEC_STACKS, VIBEC_TEST_CMD, etc.

Self-Compilation

Run node bootstrap.js to compile vibec from stacks/. It:

Checks output/current/core/vibec.js.
Falls back to bin/vibec-prebuilt.js.
Runs vibec --stacks core,generation,tests --test-cmd "npm test" --retries 2.

Plugin Context

.js plugins receive:

{
  config: { /* vibec.json, including pluginParams */ },
  stack: "core",
  promptNumber: 1,
  promptContent: "# CLI Parser\nGenerate a ... \n## Context: src/main.js\n## Output: core/cli.js",
  workingDir: "/path/to/output/current",
  testCmd: "npm test",
  testResult: { errorCode: 1, stdout: "...", stderr: "..." }
}

Prompt Processing Lifecycle

Each prompt goes through the following lifecycle:

Loading: Prompt content is loaded from the .md file
Plugin Integration:
- Static plugins (.md) are appended to the prompt
- Dynamic plugins (.js) are executed and their output is integrated
LLM Generation: The combined prompt is sent to an LLM service
Output Parsing: Generated content is parsed and written to files
Testing: The test command is run to validate the output
Retry Loop (optional): If tests fail and retries > 0, the process repeats with test results

flowchart LR
    A[Load Prompt] --> B[Apply Plugins]
    B --> C[LLM Generate]
    C --> D[Write Files]
    D --> E[Run Tests]
    E -->|Success| F[Mark as Success]
    E -->|Failure| G{Retries?}
    G -->|Yes| C
    G -->|No| H[Update Hash File]
    F --> I[Process Next Stack]
    H --> I

Example Workflow

Edit stacks/core/001_cli.md:

# CLI Parser
Generate a command-line interface parser for a Node.js tool. It should handle flags like `--help` and `--version`, and support subcommands. Use existing generated code as context.
## Context: src/main.js, README.md
## Output: core/cli.js
## Output: core/cli_utils.js

Run node bootstrap.js.
Use output/current/core/vibec.js.

Initial Setup

Clone repo.
Ensure bin/vibec-prebuilt.js exists.
Run node bootstrap.js.

Advanced Usage

Incremental Development

For iterative development:

Edit prompts in a specific numerical stage
Run node bootstrap.js
Check output/stages/NNN/ for immediate results
Look at output/current/ for the final merged result

Testing Strategy

Tests validate each stage's output. Consider:

Unit tests for individual components
Integration tests for the complete stage
End-to-end tests that validate the final output/current/

Plugin Development Tips

When creating .js plugins:

Keep them focused on a single responsibility
Handle errors gracefully
Return structured content when possible
Use async/await for all asynchronous operations
Add debug logging to aid troubleshooting

Dynamic Plugins Example

dump_files.js:

const fs = require("fs").promises;
const path = require("path");

module.exports = async (context) => {
  const output = [];

  // External files from ## Context:
  const contextMatch = context.promptContent.match(/## Context: (.+)/);
  const externalFiles = contextMatch 
    ? contextMatch[1].split(",").map(f => f.trim())
    : context.config.pluginParams.dump_files?.files || [];

  if (externalFiles.length) {
    const contents = await Promise.all(
      externalFiles.map(async (file) => {
        try {
          const content = await fs.readFile(file, "utf8");
          return "```javascript " + file + "\n" + content + "\n```";
        } catch (e) {
          return "```javascript " + file + "\n// File not found\n```";
        }
      })
    );
    output.push(...contents);
  }

  // Aggregate files from output/current/<stack>/
  const stackDir = path.join(context.workingDir, context.stack);
  let generatedFiles = [];
  try {
    generatedFiles = await fs.readdir(stackDir);
  } catch (e) {
    // Dir might not exist yet
  }

  if (generatedFiles.length) {
    const contents = await Promise.all(
      generatedFiles.map(async (file) => {
        const fullPath = path.join(stackDir, file);
        const content = await fs.readFile(fullPath, "utf8");
        return "```javascript " + path.join(context.stack, file) + "\n" + content + "\n```";
      })
    );
    output.push("Generated files in current stack:\n", ...contents);
  }

  return output.length ? output.join("\n") : "No context files available.";
};

Project Structure Visualization

graph TD
    subgraph ProjectStructure["Project Structure"]
        A[stacks/] --> A1[core/]
        A[stacks/] --> A2[generation/]
        A[stacks/] --> A3[tests/]
        A1 --> A1_1[001_cli.md]
        A1 --> A1_2[002_parser.md]
        A2 --> A2_1[001_generator.md]
        A3 --> A3_1[001_cli_tests.md]
        
        P[plugins/] --> P1[static.md]
        P[plugins/] --> P2[dynamic.js]
        
        O[output/] --> O1[stages/]
        O[output/] --> O2[current/]
        O1 --> O1_1[001/]
        O1 --> O1_2[002/]
        O1_1 --> O1_1_1[core/]
        O1_1 --> O1_1_2[generation/]
        O1_1 --> O1_1_3[tests/]
        
        C[.vibec_hashes.json]
        B[bootstrap.js]
        V[vibec.json]
        BIN[bin/vibec-prebuilt.js]
    end

Compilation Process Flow

flowchart TD
    Start([Start]) --> Bootstrap[Run bootstrap.js]
    Bootstrap --> CheckCurrent{Check output/current exists?}
    CheckCurrent -->|Yes| RunVibec[Run vibec from output/current]
    CheckCurrent -->|No| UsePrebuilt[Use bin/vibec-prebuilt.js]
    UsePrebuilt --> RunVibec
    
    RunVibec --> ProcessStages[Process stages numerically]
    
    subgraph NumericalStageProcessing["Numerical Stage Processing"]
        ProcessStages --> Stage1[Process Stage 001]
        Stage1 --> Stage2[Process Stage 002]
        Stage2 --> StageN[Process Stage N...]
        
        Stage1 --> ProcessStacks[Process specified stacks]
        
        subgraph StackProcessing["Stack Processing within Stage"]
            ProcessStacks --> Stack1[Process core/]
            Stack1 --> Stack2[Process generation/]
            Stack2 --> Stack3[Process tests/]
            
            Stack1 --> GenFiles[Generate output files]
            GenFiles --> RunTests[Run test command]
            RunTests --> CheckTests{Tests Pass?}
            CheckTests -->|Yes| NextStack[Move to next stack]
            CheckTests -->|No, retries left| Retry[Retry]
            Retry --> GenFiles
            CheckTests -->|No, no retries| FailStage[Fail]
        end
    end
    
    StageN --> MergeOutput[Merge all stages to output/current/]
    MergeOutput --> End([End])

Troubleshooting

Common Issues

Plugin timeouts: Increase --plugin-timeout for complex plugins
Test failures: Use --retries to give the LLM more chances with test output
Overwritten files: Use --no-overwrite to prevent accidental overwrites

Debugging Tips

Check .vibec_hashes.json for test result history
Examine output/stages/ to see intermediate results
Look at test output for specific error messages
Try running with a smaller subset of stacks for focused debugging

vgrichina/vibec.md