agent-pipeline-builder

Files (4)

• keywords.json · .triggers/
• SKILL.md
• ralph-pipeline.py
• validate-pipeline.ts · scripts/

SKILL.md (first lines)

---
name: agent-pipeline-builder
description: Build multi-agent pipelines with structured data flow between agents. Use when creating workflows where each agent has a specialized role and passes output to the next agent.
allowed-tools: Write, Edit, Read, Bash, WebSearch
license: MIT
---

# Agent Pipeline Builder

Build reliable multi-agent workflows where each agent has a single, focused responsibility and outputs structured data that the next agent consumes.

## When to Use This Skill

Use this skill when:
- Building complex workflows that need multiple specialized agents
- Creating content pipelines (research → analysis → writing)
- Designing data processing flows with validation at each stage
- Implementing "funnel" patterns where broad input becomes focused output

## Pipeline Pattern

A pipeline consists of:
1. **Stage 1: Researcher/Gatherer** - Fetches raw data (WebSearch, file reading, API calls)
2. **Stage 2: Analyzer/Filter** - Processes and selects best options
3. **Stage 3: Creator/Writer** - Produces final output

Each stage:
- Has ONE job
- Outputs structured JSON (or YAML)
- Wraps output in markers (e.g., `<<<stage>>>...<<<end-stage>>>`)
- Passes data to next stage via stdin or file

## RalphLoop "Tackle Until Solved" Integration

For complex pipelines (3+ stages or complexity >= 5), agent-pipeline-builder automatically delegates to Ralph Orchestrator for autonomous pipeline construction and testing.

### When Ralph is Triggered

Ralph mode activates for pipelines with:
- 3 or more stages
- Complex stage patterns (external APIs, complex processing, conditional logic)
- Parallel stage execution
- User opt-in via `RALPH_AUTO=true` or `PIPELINE_USE_RALPH=true`

### Using Ralph Integration

When a complex pipeline is detected:

1. Check for Python integration module:
   ```bash
   python3 /home/uroma/.claude/skills/agent-pipeline-builder/ralph-pipeline.py --test-complexity
   ```

2. If complex, delegate to Ralph:
   ```bash
   /home/uroma/obsidian-web-interface/bin/ralphloop -i .ralph/PIPELINE.md
   ```

3. Monitor Ralph's progress in `.ralph/state.json`

4. On completion, use generated pipeline from `.ralph/iterations/pipeline.md`

### Manual Ralph Invocation

For explicit Ralph mode on any pipeline:
```bash
export PIPELINE_USE_RALPH=true
# or
export RALPH_AUTO=true
```

Then invoke `/agent-pipeline-builder` as normal.

### Ralph-Generated Pipeline Structure

When Ralph builds the pipeline autonomously, it creates:

```
.claude/agents/[pipeline-name]/
├── researcher.md    # Agent definition
├── analyzer.md      # Agent definition
└── writer.md        # Agent definition

scripts/
└── run-[pipeline-name].ts  # Orchestration script

.ralph/
├── PIPELINE.md      # Manifest
├── state.json       # Progress tracking
└── iterations/
    └── pipeline.md  # Final generated pipeline
```

## Creating a Pipeline

### Step 1: Define Pipeline Manifest

Create a `pipeline.md` file:

```markdown
# Pipeline: [Name]

## Stages
1. researcher - Finds/fetches raw data
2. analyzer - Processes and selects
3. writer - Creates final output

## Data Format
All stages use JSON with markers: `<<<stage-name>>>...<<<end-stage-name>>>`
```

### Step 2: Create Agent Definitions

For each stage, create an agent file `.claude/agents/[pipeline-name]/[stage-name].md`:

```markdown
---
name: researcher
description: What this agent does
model: haiku  # or sonnet, opus

From romangalaxys10-spec/SuperCharged-Claude-Code-Upgrade, MIT.