ivx-avatar-studio

Files (1)

• SKILL.md

SKILL.md (first lines)

# IntelliVerseX Avatar Studio

## Overview

The Avatar Studio creates expressive 3D avatars from a single photo using GAGAvatar (Gaussian Avatars with Geometric-aware Appearance Normalization). It supports real-time facial reenactment, expression-driven animation, and interactive conversations with TTS — enabling custom player avatars, NPC portraits, streamers, and virtual hosts.

```
Single Photo
     │
     ▼
┌── Content-Factory / GAGAvatar ──────────────────────┐
│  3DGS Reconstruction → Appearance Normalization     │
│  → Expression Transfer → Real-time Rendering        │
│  → TTS Integration → Interactive Chat               │
└──────────────────────┬──────────────────────────────┘
                       │
           ┌───────────┼───────────┐
           ▼           ▼           ▼
      3D Avatar    Animated     Interactive
      Model        Sequences    Conversation
      (3DGS)       (MP4/GIF)   (WebSocket)
```

## Content-Factory API

### Create Avatar from Photo

```bash
curl -X POST http://localhost:8001/pipelines/avatar_create \
  -H "Content-Type: application/json" \
  -d '{
    "source_photo": "https://example.com/photo.jpg",
    "avatar_name": "PlayerAvatar",
    "style": "realistic",
    "generate_expressions": true,
    "ivx_export": true
  }'
```

### Drive Avatar with Expressions

```bash
curl -X POST http://localhost:8001/pipelines/avatar_animate \
  -d '{
    "avatar_id": "PlayerAvatar",
    "driving_video": "driving_expressions.mp4",
    "output_format": "mp4"
  }'
```

### Interactive Conversation

```bash
curl -X POST http://localhost:8001/pipelines/avatar_chat \
  -d '{
    "avatar_id": "PlayerAvatar",
    "message": "Welcome to Brain Quest! Ready for a challenge?",
    "voice_id": "elli_friendly",
    "emotion": "excited"
  }'
```

## Generated Output

### Avatar Package

```
avatars/PlayerAvatar/
├── model/
│   ├── avatar.ply                    # 3D Gaussian Splatting model
│   ├── appearance_volume.npz         # Tri-plane appearance features
│   └── metadata.json                 # Generation parameters
├── expressions/
│   ├── neutral.png                   # Rendered expression frames
│   ├── happy.png
│   ├── sad.png
│   ├── angry.png
│   ├── surprised.png
│   └── thinking.png
├── animations/
│   ├── greeting.mp4                  # Pre-rendered animation sequences
│   ├── nodding.mp4
│   ├── laughing.mp4
│   └── talking_idle.mp4
├── interactive/
│   ├── chat_config.json              # TTS + expression mapping config
│   └── emotion_map.json              # Text emotion → facial expression mapping
└── ivx/
    ├── avatar_meta.json              # SDK-compliant avatar metadata
    ├── expression_sheet.png          # 3×2 grid (matches character expression format)
    └── thumbnail.png                 # 512×512 avatar portrait
```

### Avatar Metadata (ivx/avatar_meta.json)

```json
{
  "id": "PlayerAvatar",
  "display_name": "Player Avatar",
  "type": "3dgs_avatar",
  "source": "single_photo",
  "model": {
    "file": "model/avatar.ply",
    "format": "3dgs",
    "gaussian_count": 50000,
    "appearance_volume": "model/appearance_volume.npz"
  },
  "expressions": {
    "neutral":   { "file": "expressions/neutral.png",   "blendshape_weights": {} },
    "happy":     { "file": "expressions/happy.png",     "blendshape_weights": {"mouthSmile": 0.8, "cheekPuff": 0.3} },
    "sad":       { "file": "expressions/sad.png",       "blendshape_weights": {"mouthFrown": 0.7, "browDown": 0.5} },
    "angry":     { "file": "expressions/angry.png",     "blendshape_weights": {"browDown": 0.9, "jawClench": 0.6} },
    "surprised": { "file": "expressions/surprised.png", "blendshape_weights": {"mouthOpen": 0.8, "browUp": 0.9} },
    "thinking":  { "file": "expressions/thinking.png",  "blendshape_weights": {"browUp": 0.3, "eyeSquint": 0.4} }
  },
  "animations": {
    "greeting":     { "file": "animations/greeting.mp4",     "duration_sec": 2.0, "loop": false },
    "nodding":      { "file": "animations/nodding.mp4",      "duration_sec": 1.5, "loop": true },
    "laughing":     { "file": "animations/laughing.mp4",     "duration_sec": 2.5, "loop": false },

From intelli-verse-x/Intelli-verse-X-SDK, MIT.

More from intelli-verse-x/Intelli-verse-X-SDK