The next-word task from n-gram counts through attention and pre-training to decoding, fine-tuning and RLHF
|
NLP - Predicting the next word (pdf)
|
703.25 K |
Next-word prediction is the task behind n-gram models, recurrent networks, BERT, GPT and the transformer. The information content of a word w after context c is -log2 P(w | c), entropy is the average information, and perplexity is the measure of prediction quality. Counting with maximum likelihood estimates and add-one smoothing is the first solution, followed by RNNs, sequence-to-sequence models and scaled dot-product attention, with a causal mask in the full transformer block. Students then follow pre-training with masked language modeling and GPT, scaling laws, byte pair tokenization, fine-tuning and prompting as two ways to adapt a model, decoding from probabilities, and the RLHF objective of maximizing reward under a KL penalty to a reference model. The 2026 edition is courselet 100099, The Next Word Problem.