The Next Word Problem - Full Course

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled

The Next Word Problem - Full Course

Next-word prediction from n-gram counts to the encoder-decoder transformer

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled
  • Free
  • Course Includes
  • 6 courselets
  • PDF slides, 213 pages in all
  • Room version of the slides for the 4 hours (zip)
  • Quizzes of 4 questions in all 6 courselets
Tags:
Next-Word Prediction Language Models Transformers N-grams NLP



What you will learn

  • Compute n-gram probabilities with smoothing and evaluate language models by perplexity.
  • Explain recurrent networks, LSTM gates, attention and the encoder-decoder transformer as steps of one development.
  • Compare pre-training, scaling, prompting, retrieval, fine-tuning and alignment as ways to build and adapt language model...

Course Content

6 courselets
The Next Word Problem (pdf)
1.13 M
The Next Word Problem - Unit 1 (pdf)
365.91 K
The Next Word Problem - Unit 2 (pdf)
467.65 K
The Next Word Problem - Unit 3 (pdf)
454.49 K
tHE Next Word Problem - Unit 4 (pdf)
467.11 K
The Transformer Architecture (pdf)
2.2 M
14.55 M

Requirements

  • Probability, linear algebra, Python

General Overview

Description

A language model is a conditional probability distribution over the next word given a context, and over 113 years the models have had more context, from none, to one word, to a window of words, to the whole sentence through a recurrent state, to all positions at once through attention.

  1. The Next Word Problem, a self-study deck on all four hours
  2. The Next Word Problem - Hour 1, with n-grams, smoothing and perplexity
  3. The Next Word Problem - Hour 2, with recurrent networks, LSTM gates, attention and transformers
  4. The Next Word Problem - Hour 3, with BERT, GPT, scaling laws, tokenisation and reasoning at answer time
  5. The Next Word Problem - Hour 4, with prompting, fine-tuning, retrieval, alignment and agent safety
  6. The Transformer Architecture, the encoder-decoder transformer of 2017 in seven elements

The 2025 edition is course 908, Natural Language Processing, with 20 courselets.

Recommended for you

blog
Last Updated 8th March 2025
  • 1
blog
Last Updated 19th July 2023
  • 0
  • 0
blog
Last Updated 16th June 2023
  • 7
blog
Last Updated 17th December 2022
  • 30
blog
Last Updated 16th January 2023
  • 49
  • Free
blog
Last Updated 7th January 2023
  • 8
  • Free
blog
Last Updated 14th March 2025
  • 52
  • Free
blog
Last Updated 3rd October 2025
  • 97
  • Free
blog
Last Updated 7th November 2022
  • 157
  • Free
blog
Last Updated 24th September 2026
  • 5
blog
Last Updated 15th January 2026
  • 102
blog
Last Updated 7th November 2022
  • 16
  • Free
blog
Last Updated 21st March 2025
  • 251
  • Free
blog
Last Updated 20th May 2025
  • 31
  • Free

Meet the instructors !

instructor
About the Instructor

https://www.joergosterrieder.com/