Mark II · The 10M training run
We trained Mark II from scratch in 54 minutes. On the same 200 reserved stories, prediction loss fell from 1.626 bits per byte for Mark I to 0.502 for Mark II. On 50 fresh openings, Mark II reached its end signal 44 times within the length limit, compared with 32 for Mark I. Some continuations still repeated phrases or lost track of the plot.
Training details
- Parameters
- 10,026,112
- Started
- 11 Nov 2026
- Finished
- 12 Nov 2026 · 54 minutes
- Hardware
- NVIDIA GeForce GTX 1070 · 8 GB VRAM
- Trained on · text
- TinyStories · 437,380 complete cleaned stories · 389,555,204 text characters
- Trained on · tokens
- 99,999,744 training predictions
- Vocabulary
- 4,096 byte-level BPE tokens, including 3 special markers
- Weights
- Open (on request)
- What it’s for
- Continue short stories and compare Mark I and Mark II on the same opening.
- To earn the next step
- Beat Mark I on shared tests for text prediction, plot continuity, repetition, and story endings.
