How LLMs work, hands-on
Three small demos I use when explaining GenAI to people. Move the sliders, type your own text, run the comparison. They also appear inside the blog posts and interview answers they belong to.
1. Picking the next word
A language model writes one token at a time. For each step it scores every possible next token, and two settings decide how adventurous the pick is. More in the interview answer on temperature and top-p.
2. Tokens, not words
Everything an LLM reads and writes is counted in tokens: the context limit, the price, the speed. This uses the real tokenizer of GPT-4o, downloaded when you scroll here (about 2 MB). Try Hindi: non-English text usually takes more tokens for the same meaning.
3. Finding the right page
Most RAG systems cut documents into chunks and search by similarity. For long, structured documents, letting the model read the table of contents often works better. The full story is in my post on vectorless RAG.