NLP - Introduction

  • 0 Rating
  • 0 Reviews
  • 4 Students Enrolled

NLP - Introduction

Next-word prediction from n-gram counts and word embeddings to transformers, decoding, fine-tuning and ethics

  • 0 Rating
  • 0 Reviews
  • 4 Students Enrolled
  • Wishlist
  • Free
  • Course Includes
  • PDF slides, 202 pages
  • Quiz, 4 questions
Tags:
N-grams Next-Word Prediction NLP Language Models Transformers



What you will learn

  • Compute bigram and trigram probabilities from counts and repair zero counts with smoothing
  • Trace the development from word embeddings and RNNs through sequence-to-sequence models to self-attention
  • Compare BERT and GPT pre-training, subword tokenization, decoding methods and fine-tuning with LoRA

Courselet Content

1 components
Natural Language Processing - Introduction (pdf)
2 M

Requirements

  • Probability and statistics, basic linear algebra, Python

General Overview

Description

Next-word prediction is the training task of language models from bigram counts to transformers. The probability of a word sequence is a product of conditional probabilities by the chain rule, and in an n-gram model the context is limited to the last few words. Unseen n-grams have probability zero without smoothing, and perplexity is the evaluation measure. Students progress from n-gram counts to word embeddings, recurrent networks with gated memory, sequence-to-sequence models with attention, self-attention, BERT and GPT pre-training, byte pair tokenization, decoding, fine-tuning with LoRA, quantization and distillation, and bias measurement.

Courses that include this CL

blog
Last Updated 19th November 2026
  • 5
  • Free

Recommended for you

blog
Last Updated 3rd May 2024
  • 17
blog
Last Updated 10th December 2023
  • 5
blog
Last Updated 30th July 2023
  • 2
blog
Last Updated 19th July 2023
  • 0
  • 0
blog
Last Updated 16th June 2023
  • 7
blog
Last Updated 5th June 2025
  • 3
  • Free
blog
Last Updated 16th January 2023
  • 49
  • Free
blog
Last Updated 7th January 2023
  • 8
  • Free
blog
Last Updated 19th November 2023
  • 7
blog
Last Updated 7th November 2022
  • 16
  • Free
blog
Last Updated 21st March 2025
  • 251
  • Free

Meet the instructors !

instructor
About the Instructor

https://www.joergosterrieder.com/