See it run
Run a bundled tiny model first. You get a concrete target before writing implementation code.
Tiny LLM Builder
You've used LLM APIs. Now build one yourself — and debug it.
A self-paced project kit for developers who know Python and Git but have never implemented a Transformer.
What you'll build
You finish with a small decoder-only language model you implemented, tested, trained, generated from, and explained.
tiny-llm-builder/
├── tinylm/
│ ├── tokenizer.py
│ ├── attention.py
│ ├── transformer.py
│ ├── model.py
│ └── train.py
├── tests/
└── BUILD_REPORT.mdHow it works
Start from a working model, rebuild it piece by piece, then diagnose failures from their symptoms.
Rebuild every part against tests
Implement the tokenizer, causal self-attention, Transformer blocks, the model, training, and sampling.
Break, debug, explain
Diagnose planted failures from symptoms, fix the violated invariant, and record the root cause in your build report.
Why not just watch a “build GPT” video?
A correct walkthrough shows what working code looks like. This kit makes you prove that your own implementation works.
- Failing tests point to the invariant that is wrong instead of giving you a finished implementation.
- Planted bugs start with symptoms, so you practice diagnosis rather than copying a known fix.
- BUILD_REPORT.md makes you record the model, the evidence, the failure, and what you changed.
The commitment
This is a focused build for working developers, not a long course platform.
About 10–15 hours
Work through eight milestones at your own pace.
Python + Git
You should already be comfortable editing Python files, running commands, and using Git.
A CPU laptop is enough
No GPU, cloud IDE, API key, or hosted account is required for the project runtime.
Founding presale
Reserve the first release at the founding price while the complete kit is being prepared.
Presale opens soon
FAQ
What do I need to know first?
Basic Python and Git. You do not need to have implemented attention or a Transformer before.
How long does it take?
The complete path is designed for roughly 10–15 hours of focused work.
Do I need a GPU?
No. The kit is designed to run on a CPU laptop. A GPU is optional.
What if I get stuck?
Use the progressive hints first. For support, contact SUPPORT_CONTACT.
What is the refund policy?
You can request a full refund any time before release and within 14 days after release.
How is it delivered?
You receive a downloadable ZIP. There is no account or custom platform, and the project works offline after setup.
Will it change while I'm working?
No. You receive a fixed, complete release so the code, tests, hints, and instructions stay consistent while you work.
Related free lessons
Review the underlying ideas for free, then use the Builder kit to assemble and debug the complete system.