suum-exse / roadmap / our own models
1M to 1T. Ten times bigger, seven times over.
Each step is ten times the last, and each one has to earn the next. Pick a step to see where it stands.
Updated 13 Nov 2026
Research models. They don’t power suum-exse. suum-exse runs on proven open models until one of ours beats them on real tasks. Only then does it take over.
- 1M Done , 100% done
- 10M Done , 100% done
- 100M Training , 15% done
- 1B Next , 0% done
- 10B Planned , 0% done
- 100B Planned , 0% done
- 1T Planned , 0% done
Done Step 1 of 7
1M parameters
Progress100%
Done. Results are in the Build log.
- Parameters
- 1,000,000
- Started
- 22 Oct 2026
- Finished
- 11 Nov 2026
- Hardware
- Intel Core i5-6500 @ 3.20 GHz · CPU training
- Trained on · text
- TinyStories · 3,686 cleaned training stories · 3,122,441 text characters
- Trained on · tokens
- 3,129,813 corpus tokens
- Vocabulary
- 99 tokens (i.e., characters)
- Weights
- Open (on request)
What it’s for
Prove the pipeline end to end, from preparing TinyStories and training a 1M-parameter model to testing it and generating text in a local browser.
To earn the next step
Complete training, evaluate on reserved stories, verify browser generation, and publish the results in the Build log.
Latest note · 11 Nov 2026
MK1 trained from scratch with 1,005,536 parameters. Validation loss fell from 4.637 to 1.125. Across 200 reserved test stories, final loss was 1.128 and perplexity was 3.09. The saved checkpoint generates story continuations in the browser.
Done Step 2 of 7
10M parameters
Progress100%
Done. Results are in the Build log.
- Parameters
- 10,000,000
- Started
- 11 Nov 2026
- Finished
- 12 Nov 2026
- Hardware
- NVIDIA GeForce GTX 1070 [8 GB VRAM] · GPU training
- Trained on · text
- TinyStories · 437,380 complete cleaned training stories · 389,555,204 characters
- Trained on · tokens
- 99,999,744 training tokens
- Vocabulary
- 4,096 tokens (byte-level BPE, including
- Weights
- Open (on request)
What it’s for
Continue short stories from a prompt and compare MK2 with MK1 in the browser.
To earn the next step
Beat MK1 on our shared tests for text prediction, plot continuity, repetition, and story endings.
Latest note · 12 Nov 2026
MK2 trained from scratch with 10,026,112 parameters in 54 minutes. On the same 200 reserved stories, prediction loss fell from 1.626 bits per byte for MK1 to 0.502 for MK2, a 69% reduction. On 50 fresh openings, MK2 reached its end-of-story signal within the length limit 44 times, compared with 32 for MK1. Some continuations still repeated phrases or lost track of plot details.
Training Step 3 of 7
100M parameters
Progress15%
Of the training run.
- Parameters
- 100,000,000
- Started
- 12 Nov 2026
- Finished
- TBD
- Hardware
- NVIDIA GeForce GTX 1070 [8 GB VRAM] · GPU training
- Trained on · text
- FineWeb-Edu · 128,892 complete English web documents
- Trained on · tokens
- 124,857,600 pretraining tokens
- Vocabulary
- 16,384 tokens (byte-level BPE)
- Weights
- Open (on request)
What it’s for
Answer simple questions, follow short instructions, and use details from earlier turns in text chat.
To earn the next step
Answer basic questions, follow a requested format, and recall a new detail across two turns.
Latest note · 13 Nov 2026
Mark III has 101,480,256 parameters. Held-out loss fell from 5.32 nats at 10M pretraining tokens to 4.02 at 124,857,600 tokens on the same 16,384-token validation set. The model at it's 10M training token checkpoint was not passing our "basic questions and recall" test. The current pretraining run is in progress.
Next Step 4 of 7
1B parameters
Progress0%
Not started. Specs are filled in when it starts.
- Parameters
- 1,000,000,000
- Started
- —
- Finished
- —
- Hardware
- —
- Trained on · text
- —
- Trained on · tokens
- —
- Vocabulary
- —
- Weights
- Open (on request)
What it’s for
Small enough to run on one machine. First real assistant behaviour.
To earn the next step
Matches well-known open models of the same size.
Latest note
Starts when 100M is done.
Planned Step 5 of 7
10B parameters
Progress0%
Not started. Specs are filled in when it starts.
- Parameters
- 10,000,000,000
- Started
- —
- Finished
- —
- Hardware
- —
- Trained on · text
- —
- Trained on · tokens
- —
- Vocabulary
- —
- Weights
- Open (on request)
What it’s for
Competitive with small open models on everyday tasks.
To earn the next step
Good enough to try inside suum-exse, for members in the Lab.
Latest note
Planned.
Planned Step 6 of 7
100B parameters
Progress0%
Not started. Specs are filled in when it starts.
- Parameters
- 100,000,000,000
- Started
- —
- Finished
- —
- Hardware
- —
- Trained on · text
- —
- Trained on · tokens
- —
- Vocabulary
- —
- Weights
- Open (on request)
What it’s for
Competitive with today’s leading open models.
To earn the next step
Beats the open model suum-exse runs on, on at least one real job.
Latest note
Planned.
Planned Step 7 of 7
1T parameters
Progress0%
Not started. Specs are filled in when it starts.
- Parameters
- 1,000,000,000,000
- Started
- —
- Finished
- —
- Hardware
- —
- Trained on · text
- —
- Trained on · tokens
- —
- Vocabulary
- —
- Weights
- Open (on request)
What it’s for
Frontier scale. The model that could power suum-exse.
To earn the next step
Beats what suum-exse runs on, on your real tasks. Then, and only then, it takes over.
Latest note
Planned.
How to read this. Only the step we’re on has dates. Later steps depend on what the one before teaches us, so they’re planned, not promised. News lands in the Build log first. Members get training notes and early builds in the Lab: join the Community.
