NLP - Notebooks

  • 0 Rating
  • 0 Reviews
  • 0 Students Enrolled

NLP - Notebooks

Six notebooks from n-gram models to BERT and GPT pre-training, each at an easy, an intermediate and an expert level

  • 0 Rating
  • 0 Reviews
  • 0 Students Enrolled
  • Wishlist
  • Free
  • Course Includes
  • Six Jupyter notebooks, weeks 1 to 6 (zip)
Tags:
Jupyter NLP Language Models Transformers PyTorch



What you will learn

  • Build n-gram, neural embedding, RNN, LSTM and sequence-to-sequence next-word predictors in Python
  • Evaluate language models with perplexity and translation models with BLEU
  • Contrast the masked language modeling objective of BERT with the causal objective of GPT

Courselet Content

1 components

Requirements

  • Python with NumPy, basic probability, basic PyTorch

General Overview

Description

The notebooks are six next-word predictors in Python, from an n-gram model with smoothing, backoff and interpolation, through a neural language model with word embeddings compared against skip-gram, an RNN and an LSTM with gate analysis, a sequence-to-sequence model with attention evaluated by BLEU and a transformer with self-attention, to the masked and causal pre-training objectives of BERT and GPT. They have an easy start, an intermediate level in which the model is built from scratch and an expert level with the mathematics, from the chain rule and perplexity to LSTM gates, scaled dot-product attention and sinusoidal positional encoding. Interactive widgets are used throughout.

Courses that include this CL

blog
Last Updated 19th November 2026
  • 5
  • Free

Recommended for you

blog
Last Updated 3rd December 2024
  • 5
  • Free
blog
Last Updated 10th December 2023
  • 5
blog
Last Updated 30th July 2023
  • 2
blog
Last Updated 16th June 2023
  • 7
blog
Last Updated 16th January 2023
  • 48
  • Free
blog
Last Updated 16th January 2023
  • 47
  • Free
blog
Last Updated 20th May 2025
  • 2
blog
Last Updated 7th November 2022
  • 11
  • Free
blog
Last Updated 19th November 2023
  • 7
blog
Last Updated 21st March 2025
  • 251
  • Free

Meet the instructors !

instructor
About the Instructor

https://www.joergosterrieder.com/