President · Local AI

Model field guide

A tiny AI that plays President.

The game uses Jev as a teacher, compresses its move choices into 849 numbers, and runs the student directly in your browser. Card rules stay in normal code. Only the strategic ranking is learned.

18 KBruntime + weights
240teacher-labeled states
92%unseen-game agreement
0runtime API calls
01 / ARCHITECTURE

Code owns the game. The model ranks moves.

This is not a general language model in the browser. It is a focused policy model. The game gives it a small list of legal moves, and it scores each one.

Deterministic code

Rules stay exact

TypeScript deals cards, checks rank and group size, creates legal moves, applies plays, tracks passes, ends rounds, exchanges cards, and scores the match. The model cannot invent an illegal action.

Learned policy

Strategy is compressed

The student learns which legal move Jev tends to prefer: finish now, react to a near-out opponent, preserve useful groups, shed cards, and avoid wasting control.

PresidentStatehand, target, opponents
Legal movescreated by game code
104 featuresone vector per move
Shared scorer849 learned values
Softmaxprobability per move
Inputs

What the model sees

Candidate rank and size, cards left, group breaks, control cards, the full hand shape, current target, public opponent counts, pass state, and threat indicators.

Hidden layer

What it combines

Eight nonlinear units combine signals such as “lowest card during a threat” or “shed an intact group when the hand is short.”

Output

What it returns

One score for each candidate. Softmax turns all scores into probabilities. The game uses the highest-probability move.

01B / WIN FORECAST

Four chances, one complete view.

A second browser model estimates each player's chance to win the round. Its four probabilities always sum to 100%.

Timing

One forecast per trick

The app calls the model only at an open-lead boundary, before the next player starts a trick. The probabilities stay fixed during that trick.

Complete information

All four hands are inputs

The 86 inputs contain all four rank-count hands, cards already played as derived from the hands, hand-shape summaries, round progress, and the next player through relative seating.

Network

86 → 16 → 4

Sixteen hidden units produce four player probabilities. The network has 1,460 learned weights. The browser API is evaluateWinProbability().

Seeded self-play data

Complete games use the existing local policy and sample its move choices.

11,050train examples
1,384validation examples
1,380held-out test examples
13,814total examples

Held-out results

The test games are not used to fit or select the model.

71.1%top-winner accuracy
0.698cross-entropy
0.379Brier score
0.028expected calibration error
01C / REAL-LOSS FORECAST

Who will finish fourth?

A second independent four-player distribution estimates who will finish fourth. This result is the real-loss probability. Its four probabilities always sum to 100%.

Timing

One forecast per trick

The model refreshes only at an open-lead boundary. It stays fixed during a trick. Open leads after the President has finished are also valid boundaries.

Complete information

The model sees the full state

The inputs include all four hands, played cards, round progress, and the player who plays next.

Network

86 → 16 → 4

The model uses the same complete-information architecture as the win model. It has 1,460 learned values and produces four real-loss probabilities.

Seeded self-play data

The real-loss model uses a separate data split for training and evaluation.

11,365train examples
1,430validation examples
1,425held-out test examples
14,220total examples

Held-out real-loss results

The test games are not used to fit or select the model.

69.4%top-loss accuracy
0.683cross-entropy
0.381Brier score
0.030expected calibration error
02 / TRAINING

Jev teaches. A tiny network imitates.

Distillation means that a smaller student learns from the outputs of a larger or remote teacher. Here, the teacher gives both its selected move and its full Choice probability distribution.

  1. Generate legal game states

    Seeded four-player games produce reproducible hands, targets, opponents, and candidate moves. Twelve fixed scenarios cover important edge cases.

  2. Ask Jev for teacher labels

    Each state is sent to the existing TypeSafe Choice question. Jev returns one selected move and a probability for every candidate. Collection is sequential and resumable.

  3. Encode state–move pairs

    Every candidate becomes 104 normalized values. The model can compare a changing number of legal moves because the same scorer is reused for each one.

  4. Minimize distillation loss

    The target is 80% of Jev’s full probability distribution and 20% extra weight on Jev’s selected move. Adam updates the 849 values. Weight decay limits overfitting.

  5. Stop on validation quality

    Training checks game 5 every five epochs and keeps the best checkpoint. Game 6 stays separate until final testing, so it measures a new game trajectory.

Teacher corpus split

240 total labeled states, grouped by game seed.

Top-choice agreement with Jev

The final test uses a game seed not used for fitting.

03 / DECISIONS

It learns preferences, not proofs.

Probabilities let the student express close calls. They do not explain why a move is correct, and they do not make the teacher perfectly consistent.

Typical priority

What the teacher prompt rewards

  1. Finish the hand now when possible.
  2. Block an opponent who is near out.
  3. Use a short finish sequence.
  4. Keep useful pairs and control cards.
  5. Use the lowest suitable move among similar options.
Important boundary

Rules never depend on probability

The model only sees legal candidates. Passing is removed when a card can be played. If only one move exists, code returns it directly. This keeps uncertain strategy separate from exact game behavior.

A real close decision from the final test

On a free lead with 11 cards, Jev slightly preferred a standalone 6. The local model slightly preferred an intact triple of 7s. This is one of its two test misses.

Jev teacher

Single 6
42%
Triple 7
33%
Pair 4
18%

Local student

Single 6
39%
Triple 7
45%
Pair 4
9%
The student is not random here. It learned a stronger “shed a whole group” preference than Jev used in this state. More varied teacher games would help calibrate this edge.
04 / TRADEOFFS

What we gain, and what we give up.

The local student is useful because it is focused. The same limits that make it tiny also keep it from matching Jev in every unusual state.

Offline speed vs. fidelity

  • No network latency, API key, or per-move bill.
  • 18 KB is easy to ship and cache.
  • Test agreement is 92%, not 100%.

Teacher imitation vs. optimal play

  • The model copies Jev, not a perfect game solver.
  • It can inherit teacher quirks and prompt conflicts.
  • Better game outcomes require a separate benchmark.

Frozen policy vs. adaptability

  • Results are stable after the weights ship.
  • Prompt or rule changes do not update it automatically.
  • Collect and train again when behavior changes.

Full probabilities vs. one label

  • Soft labels preserve Jev’s uncertainty and alternatives.
  • Extra top-choice weight helps match the played move.
  • This balance can trade calibration for agreement.

Known rules vs. learned judgment

  • Rules in code are easy to inspect and test.
  • Strategy in weights is compact but less explainable.
  • The boundary prevents illegal model actions.

Small corpus vs. coverage

  • 240 states are enough for a useful first policy.
  • The final test has only 25 states from one new game.
  • More seeds would give stronger evidence.
05 / RUN IT

Play locally or retrain the student.

Playing needs no API key. Teacher collection is the only step that contacts Vercel AI Gateway and creates billed usage.

Local game

Start the browser app

Run from the repository root:

cd experiments/president-card-game-sol
pnpm dev

# Open the game
http://localhost:3000/

# Open this guide
http://localhost:3000/model-guide.html
Distillation workflow

Collect, train, and verify

# Requires AI_GATEWAY_API_KEY and creates billed usage
pnpm distill:collect -- 240

# Rebuild the generated weight file
pnpm distill:train

# Check the stored weights without any API call
pnpm distill:evaluate

# Train and evaluate the win-probability model
pnpm win-model:train
pnpm win-model:evaluate

# Train and evaluate the real-loss probability model
pnpm loss-model:train
pnpm loss-model:evaluate

# Local behavior and build checks
pnpm test
pnpm test:types
pnpm build
Source map

Where each part lives

  • src/local-policy.ts Feature encoding, neural scorer, softmax, and browser inference.
  • src/local-policy-weights.ts The generated 849-value model. This ships to the browser.
  • src/win-probability.ts Win-model input encoding and browser inference.
  • src/win-probability-weights.ts The generated 1,460-weight win model.
  • src/loss-probability.ts Real-loss inference over the shared complete-information encoding.
  • src/loss-probability-weights.ts The generated 1,460-value real-loss model.
  • src/train-win-probability.ts Seeded self-play training and held-out evaluation for both forecasts.
  • src/distill.ts State generation, Jev collection, training, early stopping, and evaluation.
  • data/distillation.jsonl The 240 teacher examples. Development only; not in the browser bundle.
  • app/app.vue The game loop calls evaluateLocalPolicy() directly.
  • src/local-policy.test.ts All 12 expected moves plus a complete legal four-player game.