judge-study-aggregation
Aggregate many evaluation runs into a study-level verdict (estimand L2/L3, e.g.
Skill by serhiipriadko2-sys
Chalk Labs
Need help deploying judge-study-aggregation?
Chalk Labs sets up agents like this inside real teams: connected to your tools, tested on your data, with a human in the loop where it matters.
Install
Works with Claude Code, Claude apps, Codex, Cursor, Gemini CLI, GitHub Copilot, VS Code, OpenCode, Goose, Windsurf. Read the repository before you give any tool access to your data.
npx agentmarket install skill/serhiipriadko2-sys-judge-study-aggregation
Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.
Get the weekly diff
The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.
About
Use cases: productivity
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
Permissions
What this skill can do and what it asks for, from its SKILL.md. Checked Oct 25, 2026.
- Tools
- No tools found
- Secrets
- None asked for
- Install scripts
- None found
- Runs as
- Python package
Network hosts it talks to 1
codeload.github.com· found in install command
Files it touches
~/.claude/skills/judge-study-aggregation
What's inside
- Skill name
judge-study-aggregation- Description
- Aggregate many evaluation runs into a study-level verdict (estimand L2/L3, e.g. the unified-1000 bank) under the Independent Judge protocol — stratified per-family aggregates, missingness rates, hard-failure propagation (never averaged into scores), descriptive effect sizes with descriptive intervals, swap_consistency across pairs, and claim-ceiling enforcement (benchmark-local L3 only, never model-family L5). Trigger when the judge must summarize a task bank, produce a study claim, compare candidates across a corpus, or compute study statistics. Requires EXT31 in the runtime slot when working inside the ChatGPT Project.
- License
- NOASSERTION
- Instructions
- 25 lines in SKILL.md · view on GitHub
Before installing
- Ships executable scripts
Files (4)
- SKILL.md
- openai.yaml · agents/
- runs_schema.md · references/
- study_stats.py · scripts/
The license isn't open source, so the instructions aren't shown here. Read SKILL.md on GitHub
More from serhiipriadko2-sys/iskra
- 0.474score
judge-blind-workflow
serhiipriadko2-sys
Operate blind evaluation for the Independent Judge — neutral-label assignment, sealed identity manifest kept outside the judge surface, answer-key isolation…
SkillCFreshness: CClaude Code +9 - 0.474score
iskra-adr-governance
serhiipriadko2-sys
architecture decision record governance for iskra agents.
SkillCFreshness: CClaude Code +9 - 0.474score
judge-pairwise-swap
serhiipriadko2-sys
Run A/B(/C) comparisons under the Independent Judge protocol — pointwise-first scoring, comparability gate (CMP-001..004), mandatory order-swap (A→B and B→A)…
SkillCFreshness: CClaude Code +9 - 0.474score
judge-run-protocol
serhiipriadko2-sys
Execute one Independent Judge evaluation run (SINGLE_RESPONSE, ARTIFACT_REVIEW, CLAIM_AUDIT, AGENT_RUN, HIGH_STAKES_REVIEW) strictly by the v3.5-rc.2 protocol…
SkillCFreshness: CClaude Code +9 - 0.474score
judge-run-protocol
serhiipriadko2-sys
Execute one Independent Judge evaluation run (SINGLE_RESPONSE, ARTIFACT_REVIEW, CLAIM_AUDIT, AGENT_RUN, HIGH_STAKES_REVIEW) strictly by the v3.5-rc.3 protocol…
SkillCFreshness: CClaude Code +9 - 0.469score
judge-bias-calibration
serhiipriadko2-sys
Audit the judge itself — pack integrity QC (canonical criterion/gate/method ID consistency across the 30-file knowledge pack), acceptance-suite anchor runs…
SkillCFreshness: CClaude Code +9 - 0.465score
checkpoint-builder
serhiipriadko2-sys
build a clean project checkpoint with receipts and verification.
SkillCFreshness: CClaude Code +9 - 0.465score
graphrag-operator
serhiipriadko2-sys
search canon, repo, and memory as a graph instead of a flat list.
SkillCFreshness: CClaude Code +9 - 0.462score
iskra-toolchain-bridge
serhiipriadko2-sys
Use when expanding, auditing, installing, or verifying Iskra runtime connectors, Codex plugins, Agent Builder tool access, vault-safe git clone, browser…
SkillCFreshness: CClaude Code +9 - 0.457score
supabase
supabase
Use when doing ANY task involving Supabase.
SkillCFreshness: CClaude Code +9
Signals
- Stars
- 0
- Forks
- 0
- Open issues
- 8
- Last commit (this folder)
- Jul 19, 2026 (3 months ago)
From GitHub (fetched Oct 24, 2026).
Details
- License
- NOASSERTION
- Repository
- github.com/serhiipriadko2-sys/iskra
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
- Source
- GitHubgithub-graphql
Same skill elsewhere
Copies of this SKILL.md published by others; search shows only one of them.
More from serhiipriadko2-sys
- 0.524score
blind-spot-hunter
serhiipriadko2-sys
Identify omitted actors, hidden assumptions, stale evidence, authority mismatches, privacy boundaries, scale effects, partial failures, retries, rollback…
SkillBFreshness: BClaude Code +9 - 0.524score
evidence-compiler
serhiipriadko2-sys
Convert multi-source research, audits, fact checks, and consequential analyses into a typed dependency graph whose conclusions can be retracted when premises…
SkillBFreshness: BClaude Code +9 - 0.524score
action-governor
serhiipriadko2-sys
Evaluate proposed tool calls before execution by separating capability, host policy, user approval, execution, and verified effect.
SkillBFreshness: BClaude Code +9 - 0.524score
artifact-verifier
serhiipriadko2-sys
Audit files, repositories, releases, manifests, checksums, and ZIP archives for completeness, reproducibility, path safety, secret exposure, generated noise…
SkillBFreshness: BClaude Code +9 - 0.524score
counterfactual-fork
serhiipriadko2-sys
Generate bounded, decision-changing alternatives for plans, diagnoses, architectures, and strategic conclusions.
SkillBFreshness: BClaude Code +9 - 0.524score
parallax-synthesizer
serhiipriadko2-sys
Combine a frozen evidence map, counterfactual branches, authority decision, residual risks, and artifact receipts into one coherent user-facing result.
SkillBFreshness: BClaude Code +9
Done for you
Rather not set it up yourself?
Chalk Labs scopes, builds and runs agents for your team: the task audit, the eval harness, the integrations and the weekly ops report. Fixed scope, priced before we start.
See how we build AI agents →Read the case studies
Chalk Labs is a two-founder AI and growth studio. We build AI agents, CRMs and MVPs for Web3 and AI companies, and have shipped 50+ product and token launches.
- launches shipped
- 50+
- launches shipped
- media placements
- 500+
- media placements
- founders on every account
- 2
- founders on every account
Press we've placed: Forbes · CoinDesk · Decrypt · Cointelegraph · The Block
Rahil Jain, Co-founder, Chalk Labs. Ex-Goldman Sachs engineer. You talk to the person who builds it.
Get judge-study-aggregation running in your business
Tell us what you want to automate. A founder replies within one working day with a plan, not a pitch deck.
Cite this page
judge-study-aggregation skill. Agent Mart. https://agentmart.dev/skills/serhiipriadko2-sys-judge-study-aggregation (last updated 2026-10-24)