🧠 Lesson 1 of 12 · Build Your Own LLM course

Large language models (LLMs) feel like magic, but the core idea fits in one sentence: given some text, predict the next piece of text. Everything else — chatting, translating, writing SQL — is that single skill repeated very quickly. In this course we will build a small one ourselves, so let’s first get the mental model right.

Step 1: text becomes tokens

A model never sees letters or words. It sees tokens — chunks of text that each have an ID number. Common words are usually one token, rare words are split into pieces:

That is why models sometimes miscount letters in a word: the letters are hidden inside tokens. We build our own tokenizer in Lesson 2.

Step 2: the model outputs probabilities

For the input “The capital of France is”, the model does not “look up” an answer. It produces a score for every token in its vocabulary (often 50,000–200,000 of them) and turns the scores into probabilities:

One token is picked (usually a likely one), added to the text, and the whole thing runs again. A 500-word answer is roughly 700 of these loops.

Step 3: training = adjusting billions of numbers

Inside the model are parameters — plain numbers, billions of them. Training shows the model real text, asks it to predict each next token, measures how wrong it was (the loss), and nudges every parameter slightly in the direction that would have made it less wrong. Repeat over trillions of tokens.

Nobody programs grammar, facts or coding rules. They emerge because predicting the next token well requires them: to continue =VLOOKUP(A2, correctly, it helps to have learned how VLOOKUP works.

Step 4: attention lets tokens look at each other

The breakthrough behind modern LLMs is the transformer (the T in GPT), introduced in 2017. Its key part, self-attention, lets every token decide which earlier tokens matter to it. In “Priya sent the file to Rahul because he asked for it”, attention helps he connect to Rahul. We code attention by hand in Lesson 5.

Step 5: from text predictor to assistant

A freshly trained model (a base model) just continues text. Ask it a question and it may reply with more questions, because that is what often follows questions on the internet. To make an assistant, it is fine-tuned on example conversations and then refined with human (or AI) feedback about which answers are better.

What this course builds

You need Python 3.10+ and basic comfort with lists and loops. A GPU is not required — everything is sized to train on a laptop CPU in minutes.