NLP - Foundations and Statistical Language Models

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled

NLP - Foundations and Statistical Language Models

Bigram and trigram models, add-one smoothing and perplexity for next-word prediction

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled
  • Wishlist
  • Free
  • Course Includes
  • PDF slides, 19 pages
  • LaTeX source and figures (zip)
  • Quiz, 4 questions
Tags:
Smoothing Perplexity NLP Language Models N-grams



What you will learn

  • Compute bigram and trigram probabilities from corpus counts
  • Repair zero probabilities of unseen n-grams with add-one smoothing
  • Judge a language model by its perplexity and name the cases outside the reach of n-gram models

Courselet Content

2 components
Foundations and Statistical Language Models (pdf)
376.83 K

Requirements

  • Conditional probability, Python with lists and dictionaries

General Overview

Description

The probability of a word sequence is the product of P(wi | w1, ..., wi-1). In an n-gram model the context is limited to the last n-1 words, so that P(cat | the) equals count(the cat) divided by count(the). Bigram probabilities are worked out by hand on a three-sentence corpus and then implemented in Python with START and END markers. A word never seen in training gives the whole sentence probability zero, and probability mass is reserved for unseen events by add-one smoothing and its refinements. Perplexity is the measure of how surprised a model is by text, and lower is better. N-gram models have no representation of long-distance dependencies, word meaning or context switching.

Courses that include this CL

blog
Last Updated 19th November 2026
  • 4
  • Free
blog
Last Updated 19th November 2026
  • 5
  • Free

Recommended for you

blog
Last Updated 30th July 2023
  • 2
blog
Last Updated 19th July 2023
  • 0
  • 0
blog
Last Updated 16th June 2023
  • 7
blog
Last Updated 17th December 2022
  • 30
blog
Last Updated 16th January 2023
  • 47
  • Free
blog
Last Updated 16th January 2023
  • 49
  • Free
blog
Last Updated 7th January 2023
  • 8
  • Free
blog
Last Updated 14th March 2025
  • 52
  • Free
blog
Last Updated 7th November 2022
  • 16
  • Free
blog
Last Updated 21st March 2025
  • 251
  • Free

Meet the instructors !

instructor
About the Instructor

https://www.joergosterrieder.com/