glyphsteer

Files (1)

• SKILL.md

SKILL.md (first lines)

# GlyphSteer — how to use it

Dual-regime retrieval steering via **compiled in-band glyph annotations**. An LLM writes a
controlled glyph vocabulary onto a corpus; the markers steer retrieval and then **vanish
from the output**. Package: `glyphsteer`, with its `grammar` extra for the syntax gate. It is HoneyC pointed at the
index — a glyph code is a HoneyC chain — and a cousin of Anthropic's Contextual Retrieval (the embedded form differs
from the displayed form), with a compiled, controlled vocabulary in place of free text.

**Run its Python** from a plugin install with `uv run --no-project --with "glyphsteer[grammar]==0.3.4" --with "agent-skilltree>=0.4.0" python3` — pipe a block in as `python3 - <<'PY'` … `PY` — and from a clone of the public repository where `./install.sh` ran, with the clone's `python3`.

**Run `glyphsteer`** from a plugin install as `uvx --from glyphsteer==0.3.4 glyphsteer …`, and from a clone where `./install.sh` ran, as `glyphsteer …` in the environment it installed into.

**Run `skilltree`** from a plugin install as `uvx --from "agent-skilltree>=0.4.0" skilltree …`, and from a clone where `./install.sh` ran, as `skilltree …` in the environment it installed into.

> A `packages/...` path below is in a clone of the public repository: run it from the clone's
> root (`.../chaincompiler`).

## The mental model (3 facts)

1. **Sidecar split** — a chunk has two representations that need not match:
   `indexed-form = clean ⊕ glyph_code` (what gets matched) vs `returned-form = clean`
   (what the reader sees). The marker steers findability, then disappears.
2. **One axis, two renderings** — FTS5's tokenizer **drops emoji**, so each axis carries
   BOTH: a **glyph** (emoji = the *dense* marker, a learned direction) AND a **tag** (ASCII
   sentinel like `gsxnegative` = the *lexical* marker, a rare max-IDF facet). You facet with
   the glyph; the library resolves it to its tag internally.
3. **The vocabulary is the gate** — annotation only counts if the glyph is in the vocab;
   unknown glyphs are dropped (syntax-only discipline, like the `*CC` compilers).

## Quickstart — annotate, index, facet, return clean (zero deps)

```python
from glyphsteer import SENTIMENT, Chunk, RuleAnnotator, annotate_chunk, build_index, search

corpus = [Chunk("a", "the deploy pipeline passed and works great"),
          Chunk("b", "the deploy pipeline crashed, this is broken and terrible")]

ann = RuleAnnotator.keyword(SENTIMENT, {
    SENTIMENT.by_name("positive").glyph: ["great", "works", "passed"],
    SENTIMENT.by_name("negative").glyph: ["broken", "terrible", "crashed"],
})
chunks = [annotate_chunk(c, ann, SENTIMENT) for c in corpus]   # attaches the glyph code
con = build_index(chunks, SENTIMENT)                            # 2nd arg = the vocab
neg = SENTIMENT.by_name("negative").glyph
hits = search(con, "deploy", facet=neg, vocab=SENTIMENT)       # facet by a glyph
print(hits[0]["text"])   # CLEAN text — the glyph is NOT in it (the hide)
```

Key calls: `annotate_chunk(chunk, annotator, vocab)` · `build_index(chunks, vocab)` ·
`search(con, query, facet=glyph, vocab=...)` · `facet_only(con, glyph, vocab=...)`.
The hide invariant: `from glyphsteer import check_hidden; assert check_hidden(chunk, vocab)`.
From the shell, over a JSONL corpus: `glyphsteer vocab` (the sentiment vocabulary) ·
`glyphsteer annotate corpus.jsonl -k 😡=failed,broke` · `glyphsteer query corpus.jsonl "deployment" -f 😡`.

## Grade a corpus, search shows grades (the flagship)

`GRADE` is a built-in quality vocabulary (🏆 excellent / ✅ good / ⚠️ fair / ❌ poor). An LLM
grades each chunk; search returns the grade badge and can facet to a grade.

```python
from glyphsteer import GRADE, GRADE_RANK, Chunk, build_index, search, grade_label
from glyphsteer.index import facet_only
EXC, POOR = GRADE.by_name("excellent").glyph, GRADE.by_name("poor").glyph
corpus = [Chunk("c1", "use parameterized queries to prevent SQL injection", code=EXC),
          Chunk("c2", "just concatenate user input into the SQL string", code=POOR)]
con = build_index(corpus, GRADE)
for h in search(con, "SQL", vocab=GRADE):
    print(grade_label(GRADE, h["code"]), h["text"])     # badge + clean text
audit = facet_only(con, POOR, vocab=GRADE)              # the "what's bad" view
```
Grading rides the **lexical** regime — a grade is a filter/display label, NOT a semantic
direction (quality ≠ sentiment: a correct chunk can be grim, an upbeat one can be junk).
Runnable demo, from a clone: `python3 packages/glyphsteer/experiments/grade_demo.py`.

## Author your own glyph language + keep a legend

An LLM can invent its own vocabulary, persist it, and accumulate a master legend.

```python
from glyphsteer import author, save_legend, load_legend, merge, render_legend
V = author([                                   # tags auto-derived; raises on collision
    {"name": "claim",    "glyph": "📌", "description": "a factual assertion"},
    {"name": "evidence", "glyph": "🔬", "description": "supporting data"},
])
save_legend(V, "legend.json")                  # persist the glyph↔meaning↔tag table
V2 = load_legend("legend.json")
master = merge(V, author([{"name": "risk", "glyph": "☢️"}]))   # last-author-wins
print(render_legend(master))
```

## Wire it into ChainCompiler's skill search

`skilltree.search` facets the skill corpus by glyph (SkillTree is `agent-skilltree`, which
the uv line above names). **The legend you pass MUST contain the
glyph you facet on** — here we use the built-in `GRADE` vocab (which defines 🏆) and save it
as the legend. Put `glyphs: 🏆` in a SKILL.md's frontmatter, then:

```python
from skilltree.search import build_index, search
from glyphsteer import GRADE, save_legend
save_legend(GRADE, "grade_legend.json")                   # the facet glyph (🏆) lives here
con = build_index("path/to/skill/tree", vocab=GRADE)      # reads `glyphs:` frontmatter → tags
trusted = search(con, "deploy", facet="🏆", vocab=GRADE)   # only 🏆-graded skills; badge returned
```
CLI: `skilltree search ROOT "query" --facet 🏆 --legend grade_legend.json`.
(Facet on any glyph defined in the legend — your own authored vocab works the same way.)

## Syntax consistency (rulecatcher under it)

A glyph code is a little DSL, so it goes through ChainCompiler's lint/gate engine
(**rulecatcher**, via `chaincompiler.bridge`) — the same gate the `*CC` compilers use; the
`grammar` extra brings ChainCompiler and rulecatcher (the uv line above names it). Three verdicts:
`ok` · `orthogonal` (known glyphs, wrong order → steerable) · `syntax_break` (foreign token → fatal).

```python
from glyphsteer import GlyphGrammar, SENTIMENT
with GlyphGrammar(SENTIMENT) as gg:
    gg.lint(SENTIMENT.code([SENTIMENT.by_name("positive").glyph,
                            SENTIMENT.by_name("urgent").glyph])).verdict   # "ok"

From sancovp/chaincompiler, MIT.

More from sancovp/chaincompiler