autoreview

Files (7)

• SKILL.md
• autoreview · scripts/
• post-review-evidence.sh · scripts/
• test-review-harness · scripts/
• test-review-harness.ps1 · scripts/
• test-review-harness.py · scripts/
• test_post_review_evidence.sh · tests/

SKILL.md (first lines)

# Auto Review

Run the bundled structured review helper as a closeout check. This is code review, not Guardian `auto_review` approval routing.

Codex review is the default when no engine is set. It usually delivers the best review results and should remain the normal final closeout engine.

Use when:

- user asks for Codex review / Claude review / autoreview / second-model review
- after non-trivial code edits, before final/commit/ship
- reviewing a local branch or PR branch after fixes

## Contract

- Treat review output as advisory. Never blindly apply it.
- Verify every finding by reading the real code path and adjacent files.
- Read dependency docs/source/types when the finding depends on external behavior.
- Reject unrealistic edge cases, speculative risks, broad rewrites, and fixes that over-complicate the codebase.
- Prefer small fixes at the right ownership boundary; no refactor unless it clearly improves the bug class.
- When an accepted finding shows a bug class or repeated pattern, inspect the current PR scope for sibling instances before fixing.
- Fix the scoped bug class at once when practical; stop at touched surfaces, owner boundaries, and clear follow-up territory.
- Run one substantive review, verifying its findings against real code. Fix accepted defects and check the affected behavior; do not chase a cosmetically clean verdict or repeat the full review for informational observations. Preserve unresolved-HIGH merge gates and explicitly record any remaining accepted findings.
- **Contradiction rule:** when a round's finding contradicts an earlier round's finding on the same claim — including a finding that would reverse a fix you already applied — do not apply either side on reviewer authority alone. Pin the claim to a checkable primary source first (tool `--help` text, official docs, a reproduction command you run yourself); the reviewer asserting it "is documented" is not a source until you verify the citation. If no such source settles it, record the claim as unverified and shape the code/doc to be safe under both readings. (Origin: a 2026-07 review loop flip-flopped 4 rounds on whether gh-stack stack/PR numbers can collide; it converged only when a round cited `gh stack link --help` verbatim — "stack and PR numbers never overlap" — and the orchestrator verified it.)
- **Carry rejections forward:** when you reject a finding on primary evidence, record it in a rejected-findings file and pass that file to every later round with `--rejected-findings` (see Multi-Round Reviews). A settled claim then needs new evidence before it comes back. (Origin: PR 380, 2026-09-14: rounds 6 and 7 each re-raised a false P1, and each cost a full verify cycle.)
- **UNVERIFIED findings:** the helper moves a finding to `unverified_findings` when its `quoted_code` is not within 10 lines of the cited line or in that file's diff. It prints the finding as `[UNVERIFIED P1]` and does not count it toward the exit code. Read the cited location once. If the claim holds on the real code, treat it as a normal finding; if not, add it to the rejected-findings file. Do not rerun the review only to clear it.
- After a fix, rerun affected tests and review the fix against the accepted finding. Use a focused helper recheck for materially changed behavior or unresolved risk; repeat the full review only when scope or risk expands. If publishing evidence for a new head, the recheck must cover the intervening changes and clearly state its scope; never relabel an old result as current.
- For security-audit suppression changes, verify accepted findings remain auditable: suppressed findings stay in structured output, active output keeps an unsuppressible suppression notice, and aggregate findings cannot hide unrelated active risk.
- Never switch or override the requested review engine/model. If the review hits model capacity, retry the same command a few times with the same engine/model.
- If the helper stops with `codex engine failed: codex is not logged in, or its login expired`, run `codex logout`, then `codex login`, then rerun the same command. A retry without a new login fails the same way.
- Be patient with large bundles. Structured review can take up to 30 minutes while the model call is active, especially with Codex tools or web search.
- Treat heartbeat lines like `review still running: ... elapsed=... pid=...` as healthy progress, not a hang. Let the helper continue while heartbeats are advancing. Pass `--stream-engine-output` when live engine text is useful; Codex and Claude filter tool/file chatter, other engines pass raw output through.
- Do not kill a review just because it has been quiet for 2-5 minutes, or because it is still running under the 30-minute window. Inspect the process only after missing multiple expected heartbeats, after 30 minutes, or after an obviously failed subprocess; prefer letting the same helper command finish.
- Tools are useful in review mode. The helper allows read-only inspection tools and web search by default so reviewers can check dependency contracts, upstream docs, and current behavior.
- Security perspective is always included, but it should not cripple legitimate functionality. Report security findings only when the change creates a concrete, actionable risk or removes an important safety check.
- For regression provenance, keep roles separate: blamed code author, blamed PR author, PR merger/committer, current PR author, and PR/date. If no blamed PR is traceable, use the blamed commit as the provenance: commit SHA, date, and author username. Do not guess a merger or frame missing PR metadata as a separate finding.
- If the blamed PR was merged by `clawsweeper[bot]` or another automation, identify the human trigger when practical. Check timeline/comments first; if rate-limited, use gitcrawl/cache or public PR HTML. Look for maintainer commands such as `@clawsweeper automerge`, `/landpr`, or labels/status comments that armed automerge. Report `automerge triggered by @login`; if not found, say trigger unknown.
- Do not invoke built-in `codex review`, nested reviewers, or reviewer panels from inside the review. The helper builds one bundle, calls one selected engine, validates one structured result, and stops.
- Stop as soon as the helper exits 0 with no accepted/actionable findings. Do not run an extra review just to get a nicer "clean" line, a second opinion, or clearer closeout wording.
- Treat the helper's successful exit plus absence of actionable findings as the clean review result, even if the underlying Codex CLI output is terse.
- Multi-reviewer panels are opt-in only. Use them when explicitly requested or when risk justifies the extra spend; the main agent still verifies every accepted finding before fixing.
- If rejecting a finding as intentional/not worth fixing, add a brief inline code comment only when it explains a real invariant or ownership decision that future reviewers should know.
- If `gh`/Gitcrawl reports `database disk image is malformed`, run `gitcrawl doctor --json` once to let the portable cache repair before retrying review; do not bypass the shim unless repair fails and freshness requires live GitHub.
- If Gitcrawl reports a portable manifest mismatch, source/runtime DB health error, or stale portable-store checkout, run `gitcrawl doctor --json` and inspect `source_db_health`, `runtime_db_health`, and `portable_store_status` before falling back to live GitHub.
- Do not push just to review. Push only when the user requested push/ship/PR update.

## Pick Target

Dirty local work:

```bash
~/.claude/skills/autoreview/scripts/autoreview --mode local
```

Use this only when the patch is actually unstaged/staged/untracked in the
current checkout. `--mode uncommitted` is accepted as an alias for `--mode local`.
For committed, pushed, or PR work, point the helper at the commit
or branch diff instead; do not force dirty modes just
because the helper docs mention dirty work first. A clean local review
only proves there is no local patch.

Branch/PR work:

```bash
~/.claude/skills/autoreview/scripts/autoreview --mode branch --base origin/main
```

Optional review context is first-class:

```bash
~/.claude/skills/autoreview/scripts/autoreview --mode branch --base origin/main --prompt-file /tmp/review-notes.md --dataset /tmp/evidence.json
```

If an open PR exists, use its actual base:

```bash
base=$(gh pr view --json baseRefName --jq .baseRefName)
~/.claude/skills/autoreview/scripts/autoreview --mode branch --base "origin/$base"
```

Committed single change:

```bash
~/.claude/skills/autoreview/scripts/autoreview --mode commit --commit HEAD
```

Use commit review for already-landed or already-pushed work on `main`. Reviewing
clean `main` against `origin/main` is usually an empty diff after push. For a
small stack, review each commit explicitly or review the branch before merging
with `--base`.

## Multi-Round Reviews

After each round, append every finding you rejected to one JSON file. Pass that file to every later round:

```json
[
  {
    "claim": "Prompt reads 'Run downloads-triage skill on'",
    "evidence": "grep -n shows line 42 reads 'Run the downloads-triage skill on'",
    "location": "skills/downloads-triage/SKILL.md:42"
  }
]
```

```bash
~/.claude/skills/autoreview/scripts/autoreview --mode branch --base origin/dev --rejected-findings /tmp/rejected.json
```

- Each entry needs exactly `claim`, `evidence`, and `location` (`file:line`). The helper refuses any other shape before it starts the engine.
- `evidence` must be primary: a command you ran and its output, a test result, or a verbatim doc citation. Do not write "the reviewer was wrong".
- The prompt tells the reviewer not to re-raise a listed claim without new evidence. This is an instruction to the model. The helper does not filter the output, so verify a re-raised claim like any other finding.

## Parallel Closeout

Format first if formatting can change line locations. Then it is OK to run tests and review in parallel:

```bash
~/.claude/skills/autoreview/scripts/autoreview --parallel-tests "<focused test command>"
```

From thunter009/skills, MIT.

More from thunter009/skills