🧱 What Is an LLM: Tokens & Prediction

What Is an LLM: Tokens & Prediction: An LLM is your phone keyboard's next-word suggestion grown to planetary scale — and the mechanism matches one-to-one, not loosely: your keybo

An LLM is your phone keyboard's next-word suggestion grown to planetary scale — and the mechanism matches one-to-one, not loosely: your keyboard learned "what word usually follows" from your own typing history; an LLM learned "what token usually follows" from a huge slice of humanity's text, with billions of adjustable weights instead of a small frequency table. Here is the question worth sitting on: how does something whose ONLY skill is "predict the next piece of text" end up writing working Playwright code and structured bug reports? Because at sufficient scale, predicting the next token of code accurately REQUIRES having internalized syntax, APIs, patterns and even the shape of a good test — the prediction objective forces the competence in. Java comparison: an LLM is not a giant HashMap looking up stored answers — it is closer to a fitted function: like a regression that interpolates values it never saw, the model computes plausible continuations from weights, which is precisely why it can produce text that was never in its training data, both brilliantly and wrongly. The QA stake: everything risky and everything useful about LLMs in testing flows from this one mechanism — fluent-but-wrong hallucinations, run-to-run variation, prompt sensitivity — so a tester who has personally watched tokens being picked from a probability distribution (which you will do in the lab below) reasons about AI failures from the mechanism instead of from folklore.

Tokens: The Model's Alphabet

A model does not read words or letters — it reads tokens: sub-word pieces chosen by frequency. A common word is often a single token; a rare word splits into several pieces; punctuation and spaces count too. This is more than trivia for a tester: token count is the unit of context limits and of API cost, and the reason a model can struggle with character-level tasks (counting letters, reversing strings) while excelling at meaning-level tasks — it literally does not see individual characters the way you do.

Practice: Split the sentence into tokens through the model's eyes

Replace the TODO line with the token-split version of the sentence. Separate pieces with the | character.

Next-Token Prediction: The Whole Trick

Generation is a loop, not an act of composition: the model receives the context, computes a probability for every token in its vocabulary, one token is picked from that distribution, the pick is appended to the context, and the loop repeats. Temperature is the dial on the picking step: low temperature sharpens the distribution so the most likely token nearly always wins (predictable, repetitive); high temperature flattens it so unlikely tokens get real chances (creative, riskier). There is no plan, no draft, no lookahead editing — a complete test plan comes out of this loop one token at a time.

Step by Step: How an LLM Generates a Sentence

Text is split into tokens

The input is split not into words but into sub-pieces called tokens — a common word may be one token, a rare word may split into 3 pieces.

The model produces a probability distribution

The model computes a "this could be next" probability for THOUSANDS of tokens in its vocabulary — not one answer, an entire distribution.

One token is picked from the distribution

With low temperature the most likely token nearly always wins; with high temperature lower-probability tokens gain a chance — exactly why the same prompt can produce different runs.