Build Your Own LLM — Free Course

🧠
← All courses

Build Your Own LLM — Free Course

A free, hands-on course: build every part of a GPT-style language model yourself, in Python, on an ordinary laptop. No maths degree needed — every idea is shown with small code you can run.

What you will build

  • A byte-pair-encoding tokenizer like the one GPT uses
  • A counting model, then a neural one with embeddings
  • Self-attention and a full transformer block
  • A tiny GPT trained on your own text
  • LoRA fine-tuning of a real open model, and running it locally

Lessons

  1. Lesson 1: How a language model actually works
  2. Lesson 2: Write a BPE tokenizer in Python
  3. Lesson 3: A language model made of counting
  4. Lesson 4: Embeddings and your first neural language model
  5. Lesson 5: Self-attention from scratch
  6. Lesson 6: Multi-head attention and the transformer block
  7. Lesson 7: Train a tiny GPT on your own text
  8. Lesson 8: Sampling: temperature, top-k and top-p
  9. Lesson 9: Evaluating a model: loss, perplexity and test prompts
  10. Lesson 10: Fine-tuning a pre-trained model with LoRA
  11. Lesson 11: Instruction tuning and chat templates
  12. Lesson 12: Quantize and run your model locally

Before you start

  • Python 3.10 or newer, and pip install torch numpy
  • Comfort with loops, lists and dictionaries (see Python posts)
  • A plain-text file of a few hundred KB to train on — any public-domain book works

Related reading: LLMs explained · Machine learning basics · RAG & agents