check-suite

Files (1)

• SKILL.md

SKILL.md (first lines)

# Check-Suite Skill

## Purpose
Run `cherenkov check-suite` to audit an existing (or freshly generated) Playwright test suite against CHERENKOV's 6-gate quality bar. The command is suggest-only — it reports issues and proposes fixes but never rewrites tests (D7 invariant).

## When to Use
- You want to know how good a test suite is before deploying to CI
- You have an existing Playwright suite not generated by CHERENKOV and want to score it
- You need a pass/fail gate for PR checks on test quality

## Workflow

### Basic run

```bash
# Audit the default generated tests directory
cherenkov check-suite

# Audit a specific directory
cherenkov check-suite --baseline my_tests/

# JSON output for CI
cherenkov check-suite --output

# Only report failures
cherenkov check-suite --fail-on-finding
```

### Gates evaluated

| Gate | ID | What it checks |
|------|----|----------------|
| Syntax | G1 | TypeScript parses without errors |
| Structure | G2 | Correct imports (`@playwright/test`, `openapi-fetch` client) |
| AST | G3 | No raw `fetch`/`axios` calls — only typed client methods |
| Assertions | G4 | Each test has at least one `expect()` with a meaningful check |
| Compile | G5 | `tsc --noEmit` passes |
| Prism dry-run | G6 | Scenarios match expected status codes on Prism mock |

### Score interpretation

| Score | Verdict | Recommended action |
|-------|---------|-------------------|
| ≥ 0.9 | `auto_approve` | Safe for CI |
| 0.7–0.9 | `hitl` | Review flagged tests |
| < 0.7 | `regenerate` | Re-run `cherenkov generate` |

### Sample output

```
cherenkov check-suite v1.0
Scanning: stub/generated_tests/ (12 files)

  ✓ health.spec.ts         score=0.97  [G1-G6 pass]
  ✓ pets.spec.ts           score=0.94  [G1-G6 pass]
  ✗ orders.spec.ts         score=0.64  [G4 fail: no meaningful assertions on 201 response]
  ✗ users.spec.ts          score=0.71  [G3 fail: raw fetch() at line 44]

Overall: 10/12 pass  avg_score=0.87  verdict=HITL
Suggestions written to .cherenkov/check-suite-report.json
```

### CI integration

```yaml
# .github/workflows/pr.yml (fragment)
- name: Check test suite quality
  run: cherenkov check-suite --output --fail-only
  # exits 1 if any file scores < 0.7
```

## D7 Invariant
`check-suite` is read-only. It writes suggestions to `.cherenkov/check-suite-report.json` but never modifies `.spec.ts` files.

## References
- `cherenkov/cli/commands/check_suite.py` — command implementation
- `cherenkov/sdet/` — meaningful-assertion gate logic (E1.2)
- `cherenkov/stages/review.py` — 6-gate quality review engine
- `tests/unit/test_check_suite.py` — 13 unit tests

From moaidmoatasem/cherenkov-qa, Apache-2.0.

More from moaidmoatasem/cherenkov-qa