synthetic-ui-demo

Files (1)

• SKILL.md

SKILL.md (first lines)

# Synthetic UI Demos

Nothing is screen-recorded. Claude Code reads the product's repo, rebuilds only the screens a scene shows as static
React, and a scripted cursor and camera perform the flow in Remotion. Every click is a frame number, so a scene
re-renders identically, never needs a seeded account or a live environment, and is updated by editing code when the
product changes.

**Reference builds** (read them alongside this skill; both build without the product repos):

- `examples/ds-free-schedule-demo`: one 26s flow (schedule a call) with a narrator tile. The simplest complete case.
  Its `src/Demo.tsx` is a single timeline with every cursor keyframe, click and camera move.
- `examples/ds-release-649`: an 80s release-notes video, eight scenes across three apps. `templates/ds-release-notes` is the
  reusable shape of it (ships as release 650). `src/Release.tsx` sequences
  the scenes, `src/scenes/` has one file per release item, and `src/room/` and `src/ac/` are the screen kits.

## When to use this vs. Playwright

| Synthetic (this skill) | Playwright recording (`/record-demo`) |
|---|---|
| You can read the app's source | You only have the running app |
| Release notes and feature tours that get redone every release | One-off capture of real behaviour or real data |
| The flow needs data you don't have (a full call, other users, an admin view) | The real thing is easy to reach and looks right as is |
| Camera moves, exact click timing, re-renders after a voice change | Long, complex interactions that would be costly to rebuild |

## Building blocks (`lib/components`)

```tsx
import { Stage, typed, cursorAt, near } from '../../../../lib/components';

<Stage
  url="app.example.com/settings"             // address bar; host darker than path
  camera={CAMERA}                             // [{ f, z, x, y }] in canvas px, eased, clamped to the window
  path={PATH}                                 // [{ f, x, y }] cursor keyframes in canvas px
  clicks={CLICKS}                             // frames that squash the cursor and throw a ripple
  aiNote                                      // "People shown are AI-generated" (or pass your own text)
  canvasWidth={1440} canvasHeight={810}       // the defaults
>
  <SettingsScreen field={typed(frame, 'Q4 Product Review', 128)} />
</Stage>
```

- `Stage` draws browser chrome around a fixed-size app canvas, applies the camera, and puts the cursor on top at
  zIndex 100. At zoom 1 the window is `fit` (0.8625) of the frame width. Keyframes repeat a value to hold it.
- `Cursor` and `cursorAt` work on their own for a non-browser scene; `near(cursor, target)` drives hover states.
- Product-specific screen kits (theme, icons, layouts) stay in the project. They are not lib material.

## Method

### 1. Pull a visual spec from the app repo, with subagents

Send one subagent per app or area, in parallel, and have each return a compact spec rather than file dumps:

- **Theme:** palette, font stack, radii, spacing, shadows (from the theme file or CSS variables).
- **Strings:** the exact English locale strings for every label, menu item, dialog and toast the scene shows
  (`public/locales/en/*.json`, `src/i18n/locales/en/*`). Never paraphrase UI text.
- **Icons:** if the app uses an icon font, its icomoon/fontello `selection.json` holds each glyph's SVG paths. Extract
  the ones you need into a small JSON and render them as `<svg><path d=...>`. Otherwise copy the SVG components.
- **Layout and behaviour:** component tree for the screens shown, toolbar order, defaults, what each control does.

Read the default branch from the remote (`git show origin/main:path`, or `git fetch` then read `origin/<default>`).
**Local checkouts are often stale** and will give you last quarter's UI.

For release notes, trace each item to the commit and component that ship it first. That gives you the exact strings
and the default values to show.

### 2. Rebuild only what a scene shows, on a fixed canvas

- Lay every element out with absolute positions on a fixed canvas (1440x810 is a good default). Export the rects
  that matter as constants (`SAVE_BTN = { x, y, w, h }`), so click targets are arithmetic, not guesses.
- Build only what the camera sees. A panel that is never in shot can be a flat colour.
- Keep it static React: no state, no effects. Every visual change is a function of `frame`.
- Put the kit in its own folder (`src/ui/`, `src/room/`) with a `theme.ts` for constants.

### 3. Voice first, then put the timeline in frames

- Write one line per scene, **split at sentences** (one `.txt` per phrase), and generate the audio first
  (`voiceover.py --scene-dir ...`). Three takes and `tools/pick_takes.py` beat a single take plus `--max-wpm`: a
  clone copies its reference clip's pace and flatness, so if the lines come out rushed or monotone fix the clip with
  `tools/voice_prosody.py expand` rather than stretching the output (see "Clone Prosody" in CLAUDE.md). Read the
  measured durations, and `tools/captions.py` for word timings.
- Size each scene to its line plus a beat, and keep a header comment listing each line's start and length. That
  comment is the source of truth (see "Audio-Anchored Timelines" in CLAUDE.md).
- Everything is a frame constant at the top of the scene file: cursor keyframes, `CLICKS`, camera keyframes,
  UI state switches (`frame >= C.save` shows the dialog). Export the click frames so the parent can place click SFX.
- Time each click to land on the word that names it.

### 4. Split dense list lines at their pauses

A line that lists several things ("scheduling, contacts, recordings and transcripts") is too fast for one shot.
Split the audio at its pauses into phrase files (`07a`, `07b`, ...), give each phrase its own shot, and place each
phrase on its own beat. `ds-release-649`'s `moreVo` field does this.

### 5. Measure click targets from a contact sheet before the full render

A guessed coordinate is usually 20-50px off once the camera zooms. Render stills at every click frame and a few
frames before it, tile them, and check the cursor tip sits on the target:

```bash
i=1; for f in 46 78 122 212; do npx remotion still MyComp out/stills/s$i.png --frame=$f; i=$((i+1)); done
ffmpeg -start_number 1 -i out/stills/s%d.png -vf "scale=640:-1,tile=2x2" -frames:v 1 out/contact.png
```

Fix the constants, re-render the stills, and only then render the video.

### 6. Label AI-generated people on screen

Any person who is not real (FLUX.2 portraits in tiles, avatars, a narrator) gets a visible label: `aiNote` on the
`Stage`, or an "AI" badge on a narrator tile. Names, contacts and call data are invented too, so keep them plainly
fictional.

### 7. Render, then normalise

`tools/qc.py --project ...` before rendering, then `qc.py --render out/video.mp4 --fix` for -14 LUFS.
`qc.py --project ... --layout` checks the video's own text (captions, cards, the AI note) for size and contrast, and
lists the product UI's separately. `Stage` marks its window `data-qc="product"`; put the same attribute on any phone
or tablet mockup you build outside it, or its UI text is judged as the video's.

## Gotchas

From digitalsamba/claude-code-video-toolkit, MIT.

More from digitalsamba/claude-code-video-toolkit