A free, hands-on course: build every part of a GPT-style language model yourself, in Python, on an ordinary laptop. No maths degree needed — every idea is shown with small code you can run.
What you will build
- A byte-pair-encoding tokenizer like the one GPT uses
- A counting model, then a neural one with embeddings
- Self-attention and a full transformer block
- A tiny GPT trained on your own text
- LoRA fine-tuning of a real open model, and running it locally
Lessons
- Lesson 1: How a language model actually works
- Lesson 2: Write a BPE tokenizer in Python
- Lesson 3: A language model made of counting
- Lesson 4: Embeddings and your first neural language model
- Lesson 5: Self-attention from scratch
- Lesson 6: Multi-head attention and the transformer block — coming soon
- Lesson 7: Train a tiny GPT on your own text — coming soon
- Lesson 8: Sampling: temperature, top-k and top-p — coming soon
- Lesson 9: Evaluating a model: loss, perplexity and test prompts — coming soon
- Lesson 10: Fine-tuning a pre-trained model with LoRA — coming soon
- Lesson 11: Instruction tuning and chat templates — coming soon
- Lesson 12: Quantize and run your model locally — coming soon
Before you start
- Python 3.10 or newer, and
pip install torch numpy - Comfort with loops, lists and dictionaries (see Python posts)
- A plain-text file of a few hundred KB to train on — any public-domain book works
Related reading: LLMs explained · Machine learning basics · RAG & agents