data-pokemon-tcg-foundations

Files (13)

• SKILL.md
• asset-class-properties.md · references/
• data-quality.md · references/
• domain-knowledge.md · references/
• findings.md · references/
• methodology.md · references/
• personas-and-decisions.md · references/
• reference-implementation.md · references/
• sparsity-and-eras.md · references/
• tape-anatomy.md · references/
• panel.sql · scripts/
• profile.sql · scripts/
• q.sh · scripts/

SKILL.md (first lines)

# Trading-Card Sales Tapes — Foundations

How a card sales tape is shaped, where it is reliably broken, and which analyses on it are wrong in ways that look right.

**This skill is written to be portable.** The structure, defects and traps below recur across card sales tapes because they come from how these markets and pipelines work, not from one vendor. Concrete numbers come from a **reference implementation** and are marked as such:

> **Ref impl** — measured against a Pokémon TCG lakehouse (~9.4M sales, Feb–Aug 2026, DuckDB over Parquet). Connection details, table names and environment quirks: [`references/reference-implementation.md`](references/reference-implementation.md).

Treat ref-impl magnitudes as **order-of-magnitude priors and worked examples**, not constants. Re-measure them on your tape — every section tells you how.

---

## What a card sales tape is

**It is not a price series.** It is a transaction log in which the same card trades simultaneously in several observable markets — a marketplace's raw listings, an auction site's raw sales, graded copies at each grade, off-condition copies — and **those markets do not move at the same time.**

Nearly all exploitable signal lives in the *relationships between* those markets, not in the price history of any one. Card-level price momentum is close to worthless at short horizons; the cross-venue basis is typically the strongest single factor.

The practical consequence: work at **card × venue × period** granularity, with periods of 14 days or a month. Collapsing to a single blended monthly median per card throws away almost all the information and leaves you with a few dozen effective observations.

## The five questions to answer before your first analysis

Any card tape you pick up. In this order:

1. **Which venues are actually usable?** Identity resolution rate per venue, and `trade_type` composition per venue. Most venues fail one or both.
2. **How is condition/grade encoded, and is there more than one encoding?** There usually is. Pick the *complete* one, not the well-named one.
3. **Where does each venue's coverage start and stop?** Per-venue distinct-days-per-period against the calendar.
4. **How sparse is the panel?** Fraction of card×period cells clearing your minimum sale count — broken out by era.
5. **What is the measurement noise ceiling?** Split-half reliability. This bounds every result you will ever get and usually matters more than model choice.

`scripts/profile.sql` runs all five. Do not skip to modeling.

---

## Seven things that will burn you

**1. There is probably no canonical grade column — find the complete one.** Card pipelines accrete condition encodings: a marketplace's own vocabulary, a parsed-from-title vocabulary, a normalized slug vocabulary. They coexist, disagree, and neither dominates. **Check both row count and price agreement before choosing.** Agreement on price plus disagreement on count means one column is simply more complete — take it. Never fold in an "unknown condition" bucket: it is not "probably NM", it is a different, usually pricier population of unparsed listings.

> **Ref impl:** two encodings; the slug one returned **2.36× more rows** at a **median price ratio of 1.000** across 1,826 cards. 678 distinct values across the two columns, 109,314 rows where they contradict, 893 rows on impossible grades (`PSA 172`, `CGC 2006`). Filed upstream as [lake-of-rage#1580](https://github.com/Evan-Kim2028/lake-of-rage/issues/1580).

**2. Assume a silent writer regression is in flight.** A new ingest path lands, emits its own vocabulary for one or two columns, and every downstream consumer's filter silently drops its rows. This is invisible by construction — the rows are *present and correct*, just spelled wrong — so it shows up as an unexplained volume decline, which reads as a demand collapse. **Profile distinct values of every filter column by month.** A vocabulary that gains a new value mid-series is a regression until proven otherwise.

> **Ref impl:** since a specific date, a new writer emitted a fourth vocabulary for *both* the grade and identity-confidence columns. **360,669 fully-identified rows** (card key populated on 100% of them) were invisible to the canonical filter, and the share was still growing. Same window, same venue: old encoding 100% resolved, new encoding **0.0%**.

**3. Drop the newest period, always.** The most recent bar is still filling. A signal built on it reads ingest lag as collapsing demand. This is universal and free to fix.

> **Ref impl:** final-bar volume fell 66% — part lag, part the §2 regression, which alone stripped 43% of that period's rows on one venue.

**4. Venue coverage windows differ, and a short one is usually expected.** Venues get added to a pipeline at different times and backfill to different depths. **Do not read a coverage gap as a market event, and do not file it as an outage** — it is the shape of the data. It only determines where your panel can start: begin at the first period where every venue you need is adequately covered.

> **Ref impl:** the second venue's history is shorter and has a 37-day interior gap. One period before the start bound had **48 sales on one venue against 48,836 on the other** — a "venue gap" computed from that is noise.

**5. Most "venues" are not secondary markets.** A tape that aggregates on-chain and marketplace sources will carry primary issuance, buybacks, burns, internal transfers and vault movements alongside genuine arms-length resale. Filter on trade type explicitly. Headline row counts per venue routinely overstate tradeable volume by 5–10×.

> **Ref impl:** one venue was 98.8% primary/buyback, another mostly burn/transfer, a third had **zero** secondary rows. Genuine secondary across all unresolved venues was **~148k**, not the ~900k the raw counts suggested.

**6. Lot sales stamp the lot price on every card.** Multi-card transactions frequently write the whole lot's price onto each row. Guard with `count(*) OVER (PARTITION BY <transaction key>) = 1`, or reconstruct per-card value. Check this even if it is currently confined to venues you exclude — it goes live the day one of them is resolved.

> **Ref impl:** in 4–10-card transactions, 86% shared one identical price — median $48/row against $5.73 for genuine single-card sales.

**7. Never compare venues unconditionally.** Venues differ in *price formation*, not just level: auction-style venues cluster on psychological ask points, bulk marketplaces are continuous and dominated by sub-dollar volume. A pooled cross-venue comparison measures product mix, not basis. **Only compare per card, with a minimum sale count on both sides.**

> **Ref impl:** same era, same condition, median price by venue: **$0.19 vs $10.00.** All of that 50× is mix.

## Two more rules for aggregation

- **Compute returns within a venue, then volume-weight across.** Venue mix moves violently as pipelines add and lose sources. Pooled medians fabricate large moves that are pure composition.
- **Report within-set IC, not just total IC.** A large share of forward variance is set-level. A factor with high total IC and near-zero within-set IC is a set-beta proxy — it cannot choose between two cards in the same set, which is the only decision that matters when buying one card.

> **Ref impl:** ~34% of forward variance was set-level; one factor scored total IC 0.491 / within-set 0.034.

---

## Sparsity is the binding constraint — reconstruct across conditions

**Most of a card catalogue never trades.** Expect the majority of catalogued cards to have zero sales in any given quarter. Every method question reduces to *"does this buy me more observations per card-period?"*

**And rawness declines with print age.** Older cards mostly did not survive in near-mint, so their tape is dominated by played-condition and graded copies. This is a property of physical card survival, so it should hold on any long-lived card line.

> **Ref impl** — raw-NM share of sales by era: newest 64.8% · recent 64.2% · middle-modern 46.5% · **mid-era 30.3% · vintage 29.4%.** Mid-era tracks vintage, not modern. The same grouping appears independently in the graded premium (vintage 7.5× / mid 7.7× vs modern 1.5–1.8×) and the played-condition discount (0.766 / 0.728 vs 0.895 / 0.924) — three unrelated measurements, same split.

The panel consequence — fraction of card×period cells clearing a minimum sale count:

| Era (ref impl) | NM-only | Condition-pooled |
|---|---|---|
| vintage | 56.0% | 91.3% |
| **mid** | **22.3%** | **83.8%** |
| middle-modern | 49.0% | 85.7% |
| recent | 90.3% | 97.8% |
| newest | 91.2% | 94.6% |

**A best-condition-only panel is not viable on older eras.** The fix is to convert every raw condition to a best-condition-equivalent price using a per-era factor, then pool.

**Condition legs invert cleanly; graded legs do not.** Condition ratios are tight enough for a multiplier; graded premiums vary far too much card-to-card. **Use grades as a directional signal, never as a price level.**

> **Ref impl:** interquartile spread 1.50–1.95 for played conditions vs 3.49–4.25 for graded. Pooling raised split-half reliability in *every* era (mid 0.897 → 0.967) while mid-era usable cells went **3,299 → 17,584 (5.3×)**.

Factor tables, the recipe, and its validation: [`references/sparsity-and-eras.md`](references/sparsity-and-eras.md).

---

## The anchor model

**In a multi-venue card market, one venue is the price anchor and the others error-correct toward it.** With `gap = ln(follower_price / anchor_price)` per card, regress each venue's next-period move on the gap. The anchor's coefficient is ~zero (it is exogenous); the follower's is negative (it closes the gap).

**This turns forecasting into convergence.** You are not predicting what a card will be worth. You are measuring a published anchor, measuring how far a slower venue sits from it, and collecting the gap.

The wider implication: **every unobserved venue — local shops, shows, social marketplaces — prices off the same public anchor and updates more slowly than the observed follower.** So the follower's measured half-life is a *lower bound* on how stale in-person pricing is. That is the mechanism behind in-person arbitrage.

**The anchor migrates with price band.** Expect the bulk marketplace to anchor cheap cards and the liquid auction venue to anchor chase cards, with the crossover somewhere in the middle. **Always fit this per band** — a pooled correction coefficient is a cross-sectional average that matches no individual band.

> **Ref impl** — follower correlation with gap −0.384 vs anchor −0.002; pooled gap decay ρ ≈ 0.78 per 14 days.
>
> | Price band | Follower vol share | β_follower | β_anchor | Anchor's share of correction | Gap half-life |
> |---|---|---|---|---|---|
> | <$10 | 75.2% | −0.282 | 0.013 | **4%** | 28 d |
> | $10–25 | 79.0% | −0.308 | 0.046 | 13% | 22 d |
> | $25–50 | 79.8% | −0.215 | 0.050 | 19% | 32 d |
> | $50–100 | 84.2% | −0.158 | 0.061 | 28% | **40 d** |

From Evan-Kim2028/agent-skills, MIT.

More from Evan-Kim2028/agent-skills