ai-multi-provider

Files (15)

• README.md
• SKILL.md
• client.ts · files/lib/ai/
• config.ts · files/lib/ai/
• fallback.ts · files/lib/ai/
• parse-claude-json.ts · files/lib/ai/helpers/
• anthropic.ts · files/lib/ai/providers/
• base.ts · files/lib/ai/providers/
• factory.ts · files/lib/ai/providers/
• index.ts · files/lib/ai/providers/
• mistral.ts · files/lib/ai/providers/
• openai.ts · files/lib/ai/providers/
• tracer.ts · files/lib/ai/
• install.sh
• client.test.ts · tests/

SKILL.md (first lines)

# AI Multi-Provider Skill

**Version:** 1.0.0
**Category:** Core
**Extracted from:** Production-tested in a real-world SaaS
**Production-ready:** ✅ Yes

## Description

Multi-provider LLM orchestration system with intelligent routing, fallback chains, cost optimization, and automatic observability.

**Key Features:**
- 🔄 **Multi-Provider Support**: Anthropic Claude, OpenAI GPT, Mistral AI
- 💰 **Cost Optimization**: Task-based routing (Haiku for cheap tasks, Sonnet for quality)
- 🛡️ **Resilience**: Automatic fallback chain with circuit breaker
- 📊 **Observability**: Built-in cost tracking, latency monitoring, trace persistence
- ⚙️ **Configurable**: Environment variable overrides for all settings
- 🎯 **Type-Safe**: Full TypeScript support with Zod schema validation

## What This Skill Provides

### Core Components

1. **LLMClient** (`lib/ai/client.ts`)
   - Unified interface for all LLM providers
   - Task-based routing for cost optimization
   - Automatic fallback chain
   - Circuit breaker to prevent infinite retries
   - Timeout handling (30s default)

2. **Multi-Provider Support** (`lib/ai/providers/`)
   - **Anthropic**: Claude Opus/Sonnet/Haiku with prompt caching (90% cost savings)
   - **OpenAI**: GPT-4o/mini with native structured outputs
   - **Mistral**: Mistral Large/Small models
   - Extensible factory pattern for adding new providers

3. **Configuration System** (`lib/ai/config.ts`)
   - Task-to-provider mapping
   - Environment variable overrides
   - Fallback chain configuration
   - Retry and timeout settings

4. **Observability** (`lib/ai/tracer.ts`)
   - Automatic cost calculation (tokens × pricing)
   - Latency tracking
   - Trace persistence to Supabase (optional)
   - Error type detection

5. **Utilities**
   - JSON parsing with Claude-specific handling
   - Zod schema validation for structured outputs
   - Type-safe interfaces

## Installation

### 1. Run Installation Script

```bash
cd skills/core/ai-multi-provider
bash install.sh
```

This will:
- Copy all AI files to your project
- Install npm dependencies
- Create environment variable template
- Set up optional Supabase integration

### 2. Install Dependencies

```bash
npm install @anthropic-ai/sdk openai zod
# Optional providers
npm install @mistralai/mistralai
# Optional observability
npm install @supabase/supabase-js
```

### 3. Configure Environment Variables

Create or update your `.env.local`:

```bash
# Required: At least one provider
ANTHROPIC_API_KEY=sk-ant-xxx
OPENAI_API_KEY=sk-xxx  # Optional
MISTRAL_API_KEY=xxx    # Optional

# Optional: Override default provider
LLM_PROVIDER=anthropic

# Optional: Override per-task providers
LLM_FAST_PROVIDER=anthropic/claude-haiku-4-5
LLM_QUALITY_PROVIDER=anthropic/claude-sonnet-4-5
LLM_EXTRACTION_PROVIDER=anthropic/claude-haiku-4-5
LLM_GENERATION_PROVIDER=anthropic/claude-sonnet-4-5

# Optional: Supabase for trace persistence
NEXT_PUBLIC_SUPABASE_URL=https://xxx.supabase.co
SUPABASE_SERVICE_ROLE_KEY=xxx
```

### 4. Optional: Set Up Supabase Tracing

If you want automatic cost/latency tracking:

```sql
-- Run this migration in your Supabase project
CREATE TABLE llm_traces (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  trace_id TEXT NOT NULL,
  operation TEXT NOT NULL,
  provider TEXT NOT NULL,
  model TEXT NOT NULL,
  tokens_input INTEGER NOT NULL,
  tokens_output INTEGER NOT NULL,
  cost DECIMAL(10, 6) NOT NULL,
  latency_ms INTEGER NOT NULL,
  status TEXT NOT NULL,

From JimmyBlanquet/project-forge, MIT.

More from JimmyBlanquet/project-forge