Building a language model end to end — tokenizer, attention, training loop, and a working inference path. No frameworks doing the interesting parts for you.
Most explanations of language models stop at the diagram. This series goes the other way: every part gets built, run, and measured, until there's something you can type a prompt into.
We start with bytes and end with sampled tokens. Each part stands alone well enough to read out of order, but the code accumulates — by the last part it's one program.
Part 1: Building a Byte-Pair Tokenizer
Text goes in, integers come out. Building BPE from scratch and understanding why vocabulary size is a real tradeoff.
Part 2: Attention, Implemented Twice
The slow, obvious version first — then the batched one, and a look at what actually changed.