Nonobench
An open-source benchmark of how well LLMs solve nonogram puzzles, from 5x5 to 20x20.
Connector by mauricekleine · GitHub account verified by the MCP Registry
Chalk Labs
Need help deploying Nonobench?
Chalk Labs sets up agents like this inside real teams: connected to your tools, tested on your data, with a human in the loop where it matters.
Install
Works with Claude Code, VS Code, Cursor, Windsurf. Read the repository before you give any tool access to your data.
npx agentmarket install connector/mauricekleine-nonobench
Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.
Remote endpoint
https://www.nonobench.com/mcpRemote (HTTP)
Get the weekly diff
The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.
About
Use cases: llm-evaluation
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
Permissions
What this server can do and what it asks for, from its tool list, install config and code. Checked Oct 25, 2026.
- Tools
- 11 read · 0 write · 0 deleteRead-only
- Secrets
- None asked for
- Install scripts
- Nothing to install (hosted)
- Runs as
- Hosted (remote endpoint)
Tools by what they do 11
From the running server's own tool list (tools/list) on Oct 25, 2026.
Read 11 · Look things up without changing anything.
check_solutioncompare_modelsget_leaderboardget_model_puzzlesget_model_resultsget_puzzleget_puzzle_resultslist_familieslist_providerslist_puzzleslist_runs
Network hosts it talks to 20
api.elevenlabs.io· found in codeapi.maptiler.com· found in codeapi.nonobench.com· found in codeassets.local· found in codebeta.nonobench.com· found in codecesium.com· found in codecloud.maptiler.com· found in codedevelopers.google.com· found in codedoi.org· found in codegithub.com· found in codehuggingface.co· found in codemaptiler.com· found in codemauricekleine.com· found in codenonobench.com· found in remote endpoint, install command, codeopenrouter.ai· found in codeschemas.agentskills.io· found in codesimpleanalytics.com· found in codesitemaps.org· found in codetile.googleapis.com· found in codeui.shadcn.com· found in code
Files it touches
/private/tmp/claude-501/-Users-maurice-Projects-nonobench/1a8972a5-ea54-4051-8b17-8cf810766a7a/scratchpad/activity.csv/Users/maurice/Projects/nonogram-solver/build/nonogram_hybrid
Tools (11)
As the server listed them on Oct 25, 2026. Descriptions are the server’s own words; hidden characters are shown as [U+…].
check_solutionCheck solutionread-onlyCheck a grid against a puzzle's clues, using the same rule as the benchmark grader. Reports which rows and columns do not match.
compare_modelsCompare modelsread-onlySide-by-side core overall and per-size accuracy, cost, latency and token results for model or family names.
get_leaderboardGet leaderboardread-onlyModels ranked by accuracy; defaults to all effort levels for compatibility.
get_model_puzzlesGet model puzzlesread-onlyWhich puzzles one model solved, missed, timed out on, or has not run.
get_model_resultsGet model resultsread-onlyAccuracy, cost, latency and token use for one model, broken down by grid size.
get_puzzleGet puzzleread-onlyOne puzzle, including the clue text models were prompted with. The reference solution is only included on request; some puzzles have several valid solutions.
get_puzzle_resultsGet puzzle resultsread-onlyPer-model outcomes for one puzzle. Answers are omitted unless requested.
list_familiesList familiesread-onlyModel families, available efforts and best variants.
list_providersList providersread-onlyProvider ids, names, families and variant counts.
list_puzzlesList puzzlesread-onlyThe benchmark puzzles with their ids and row/column clues.
list_runsList runsread-onlyIndividual benchmark runs (one model on one puzzle), optionally with the raw prompt and model output.
Signals
- Stars
- 10
- Forks
- 1
- Open issues
- 0
- Last commit
- Oct 23, 2026 (7 days ago)
- Last release
- Sep 29, 2026 (1 month ago)
From GitHub (fetched Oct 24, 2026).
Details
- Registry name
io.github.mauricekleine/nonobench- Version
- 1.1.0
- Auth
- Auth not stated
- License
- MIT
- Repository
- github.com/mauricekleine/nonobench
- Website
- www.nonobench.com
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
Done for you
Rather not set it up yourself?
Chalk Labs scopes, builds and runs agents for your team: the task audit, the eval harness, the integrations and the weekly ops report. Fixed scope, priced before we start.
See how we build AI agents →Read the case studies
Chalk Labs is a two-founder AI and growth studio. We build AI agents, CRMs and MVPs for Web3 and AI companies, and have shipped 50+ product and token launches.
- launches shipped
- 50+
- launches shipped
- media placements
- 500+
- media placements
- founders on every account
- 2
- founders on every account
Press we've placed: Forbes · CoinDesk · Decrypt · Cointelegraph · The Block
Rahil Jain, Co-founder, Chalk Labs. Ex-Goldman Sachs engineer. You talk to the person who builds it.
Get Nonobench running in your business
Tell us what you want to automate. A founder replies within one working day with a plan, not a pitch deck.
Cite this page
Nonobench MCP server. Agent Mart. https://agentmart.dev/connectors/mauricekleine-nonobench (last updated 2026-10-24)