Generative AI doesn't arrive fully formed. It's built in two stages: pretraining (a document completer) and fine-tuning (an assistant overlay). Each leaves a fingerprint on what the final system can and cannot do.
Trained on vast quantities of text to do one job: given everything so far, predict what comes next. Repeated billions of times. What emerges is not an assistant. It's a document completer. Ask it "Who is the president?" and it might continue with a civics lesson, a list, or a quiz. No concept of you, no concept of helping.
To turn the document completer into an assistant, you train it again. Curated examples of good assistant behaviour, then reward signals (RLHF) that nudge toward safe, helpful responses. This is where it learns to treat your input as a request, to answer rather than ramble, to decline harmful asks, to say "I am not sure."
The assistant behaviour is a trained overlay on top of the document completer. That is why fluent prose can sit next to confident nonsense in the same response. Both come out of the same machine.
Push hard on the assistant overlay and the document completer underneath sometimes pokes through: rambling, listing, predicting what a confident answer would look like rather than retrieving truth. The four properties (Steerability, Working Memory, Token Prediction, Knowledge) describe this dual nature in operating terms.
The overlay (it gave a confident answer it should have refused), or the document completer underneath (it generated plausible-sounding nonsense). Naming the failure mode is half of the fix.
Want the boardroom version of this?