audio-to-music-video

Files (1)

• SKILL.md

SKILL.md (first lines)

# Audio to Music Video

Single-pass ffmpeg synthesis of a music video from an audio file plus a chosen template. The audio is pristine (stream-copied where the container allows) and the visual is built entirely with ffmpeg's `-filter_complex` graph — no external editor needed.

## Templates

Templates are stored in `preferences.json` under `audio_to_video_templates` and selected by name. Each template defines: visualizer type, layout, colour palette, optional cover-art treatment, and resolution. Built-in defaults ship in this skill's prose; users add their own via `onboard` or by editing `preferences.json` directly.

### Built-in templates

| Name | Layout | Visualizer |
|---|---|---|
| `waveform-bottom` | full-frame blurred cover art bg + crisp cover art centered + linear waveform strip across bottom 15% | `showwaves` (mode=cline) |
| `spectrum-bars` | solid colour bg + cover art top-center + spectrum bars bottom 40% | `showfreqs` (mode=bar, ascale=log) |
| `circular-cqt-cover` | blurred cover bg + circular spectrum (showcqt) ring around centered cover | `showcqt` |
| `vector-scope` | dark bg + cover art top-left + Lissajous vector scope center | `avectorscope` |
| `volume-meter` | minimal — solid bg + cover art + horizontal volume meter | `showvolume` |

A template entry:

```json
{
  "name": "waveform-bottom",
  "resolution": "1920x1080",
  "fps": 30,
  "bg": { "type": "cover-blur", "blur": 40, "darken": 0.3 },
  "cover": { "size": "60%", "position": "center", "shadow": true },
  "viz": {
    "filter": "showwaves",
    "mode": "cline",
    "size": "1920x162",
    "color": "white",
    "position": "bottom",
    "opacity": 0.9
  }
}
```

## Procedure

### 1. Inputs

| Field | Default |
|-------|---------|
| Audio | required (`.mp3`, `.wav`, `.flac`, `.m4a`) |
| Cover art | optional path; if omitted and the audio has embedded cover art, extract it; else use `bg.solid` colour |
| Template | name from `preferences.audio_to_video_templates`; default `waveform-bottom` |
| Title / Artist | optional `drawtext` overlay |
| Output | `<basename>.mv.mp4` |

### 2. Resolve cover art

If not supplied:

```bash
ffmpeg -hide_banner -y -i "$AUDIO" -an -vcodec copy "$WORK/cover.jpg" 2>/dev/null \
  || COVER_MISSING=1
```

If still missing, use a plain colour from `bg.solid` (or fall back to `#111111`).

### 3. Build the filtergraph (template = `waveform-bottom`)

```bash
WIDTH=1920; HEIGHT=1080; FPS=30
COVER_W=$((WIDTH * 60 / 100))   # 60% width centered

ffmpeg -hide_banner -y \
  -loop 1 -i "$COVER" \
  -i "$AUDIO" \
  -filter_complex "
    [0:v]scale=${WIDTH}:${HEIGHT}:force_original_aspect_ratio=increase,
         crop=${WIDTH}:${HEIGHT},
         boxblur=40:1,
         eq=brightness=-0.3[bg];
    [0:v]scale=${COVER_W}:-1[cov];
    [bg][cov]overlay=x=(W-w)/2:y=(H-h)/2[bgcov];
    [1:a]showwaves=mode=cline:s=${WIDTH}x162:colors=white:rate=${FPS}[wf];
    [bgcov][wf]overlay=x=0:y=H-h-40:format=auto[v]
  " \
  -map "[v]" -map 1:a \
  -c:v libx264 -crf 20 -preset medium -pix_fmt yuv420p \
  -c:a aac -b:a 192k \
  -shortest \
  -movflags +faststart \
  "$OUT"
```

### 4. Spectrum-bars template

Replace the visualizer chain:

```
[1:a]showfreqs=s=${WIDTH}x432:mode=bar:ascale=log:fscale=log:colors=cyan|magenta:rate=${FPS}[viz];
[bgcov][viz]overlay=x=0:y=H-h-60:format=auto[v]
```

### 5. Circular CQT template

```
[1:a]showcqt=s=${WIDTH}x${HEIGHT}:fps=${FPS}:bar_h=300:axis_h=0:cscheme=1|0.5|0|0|0.5|1[viz];
[bgcov][viz]overlay=0:0:format=auto:enable='gte(t,0)'[v]
```

`showcqt` is the most musical-looking visualizer — it maps frequency on a constant-Q logarithmic scale, so each octave gets equal screen real estate. Heavier on CPU than `showwaves`.

### 6. Title / artist overlay (optional)

Add a `drawtext` after the visualizer overlay:

```
[v]drawtext=fontfile='/abs/path/Inter-Bold.ttf':text='${TITLE}':fontsize=64:fontcolor=white:x=80:y=80:shadowcolor=black@0.6:shadowx=2:shadowy=2,
    drawtext=fontfile='/abs/path/Inter.ttf':text='${ARTIST}':fontsize=36:fontcolor=white@0.85:x=80:y=160:shadowcolor=black@0.6:shadowx=2:shadowy=2[v]
```

### 7. Encoder choice

For stable colour and broad compatibility, default to libx264 + `pix_fmt yuv420p`. The visualizer filters generate full-range RGB — without the `pix_fmt` clamp some players show washed-out colours.

From danielrosehill/Claude-Video-Editor-Plugin, MIT.

More from danielrosehill/Claude-Video-Editor-Plugin