compare-kernels

Files (2)

• SKILL.md
• find_references.py · scripts/

SKILL.md (first lines)

# Compare Kernels

Side-by-side kernel comparison between two trace analysis Excel files, grouped by nn.Module for prefill/extend traces.

## Usage

```
/compare-kernels <file_A.xlsx> <file_B.xlsx>
/compare-kernels --budget <file_A.xlsx> <file_B.xlsx>   # phase-level logical-op budget diff (works on decode)
/compare-kernels --refs <keyword>                       # research only: find upstream+Jira prior art for a kernel
```

If the user doesn't provide two paths, ask for them using `AskUserQuestion`.

**Research mode (`--refs`):** skip the trace diff and jump straight to Step 8 — given a
kernel/op keyword, find prior art in aiter + sglang upstream + Jira. Useful before
starting an optimization ("has anyone already done this?"). Sources resolve through
[`_shared/data-sources.md`](../_shared/data-sources.md).

**Two views:** the default module-grouped comparison (Steps 1–4) needs a module tree
→ **prefill/extend only**. The `--budget` view (Step 4.5) is a category-based
phase-level budget that **also works on decode** traces (which collapse to
`CudaGraphReplay`). When comparing decode traces, use `--budget`.

## Step 0: Recategorize both files

Before comparing, run `--recategorize` on both files to ensure they use the latest `kernel_categories.csv`:

```bash
python3 $HOME/agent-box/profile/trace_module_analyzer.py --recategorize <file_A.xlsx> <file_B.xlsx>
```

## Step 1: Read detail sheets from both files

Use Python with openpyxl to read both Excel files. For each file:

1. List all sheet names
2. Identify **detail sheets** — any sheet that is NOT "Summary", "Overview", "Module Tree", or "GPU Kernels"
3. For each detail sheet, read:
   - Row 1: title (module name, phase, kernel count)
   - The kernel detail table (columns: Module, Input Dims, Kernel Name, Duration (us), % of wall time, Category, ...)
   - Extract every kernel's **module**, **name**, **duration**, and **current category**

```python
import openpyxl

def read_detail_kernels(xlsx_path):
    """Returns list of (sheet_name, title, [(module, kernel_name, duration, category), ...])"""
    wb = openpyxl.load_workbook(xlsx_path, data_only=True)
    skip = {"Summary", "Overview", "Module Tree", "GPU Kernels"}
    results = []
    for sname in wb.sheetnames:
        if sname in skip:
            continue
        ws = wb[sname]
        title = ws.cell(1, 1).value or sname
        kernels = []
        header_row = None
        for r in range(1, min(ws.max_row + 1, 30)):
            for c in range(1, ws.max_column + 1):
                if ws.cell(r, c).value == "Kernel Name":
                    header_row = r
                    break
            if header_row:
                break
        if not header_row:
            continue
        col_map = {}
        for c in range(1, ws.max_column + 1):
            val = ws.cell(header_row, c).value
            if val:
                col_map[val] = c
        mod_col = col_map.get("Module")
        name_col = col_map.get("Kernel Name")
        dur_col = col_map.get("Duration (us)")
        cat_col = col_map.get("Category")
        if not name_col:
            continue
        for r in range(header_row + 1, ws.max_row + 1):
            kname = ws.cell(r, name_col).value
            if not kname or "truncated" in str(kname):
                break
            mod = ws.cell(r, mod_col).value if mod_col else ""
            dur = ws.cell(r, dur_col).value if dur_col else 0
            cat = ws.cell(r, cat_col).value if cat_col else ""
            kernels.append((str(mod or ""), str(kname), float(dur or 0), str(cat or "")))
        results.append((sname, title, kernels))
    return results
```

## Step 2: Re-classify every kernel

Load the current `kernel_categories.csv` from the profile directory and re-classify every kernel name. Compare with the category stored in the Excel file.

```python
import re, csv, os

csv_path = os.path.expanduser("~/agent-box/profile/kernel_categories.csv")
categories = []
with open(csv_path) as f:
    for row in csv.DictReader(f):
        categories.append((row["category"].strip(),
                           re.compile(row["pattern"].strip(), re.IGNORECASE)))

def classify(name):
    for cat, pat in categories:
        if pat.search(name):
            return cat
    return "other"
```

Flag any kernel where the Excel category differs from the re-classified category.

## Step 3: Group kernels by Module and compare side-by-side

This is the **core analysis**. For each pair of matching detail sheets:

### 3a. Extract module groups

From yichiche/agent-box, MIT.

More from yichiche/agent-box