NLP - Sequence-to-Sequence Models

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled

NLP - Sequence-to-Sequence Models

Encoder-decoder models, the context-vector bottleneck, attention and BLEU for translation

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled
  • Wishlist
  • Free
  • Course Includes
  • PDF slides, 20 pages
  • LaTeX source and figures (zip)
  • Quiz, 4 questions
Tags:
Sequence-to-Sequence Encoder-Decoder Machine Translation NLP Attention



What you will learn

  • Build an encoder-decoder model with a context vector and teacher forcing
  • Explain the information bottleneck of a fixed-size context vector for long inputs
  • Implement attention over the encoder hidden states and evaluate translations with BLEU

Courselet Content

2 components
NLP - Sequence-to-Sequence Models (pdf)
431.84 K

Requirements

  • RNNs and LSTMs, softmax, basic PyTorch

General Overview

Description

Input and output have different lengths, as with the French Je t'aime and the German Ich liebe dich, two words against three. In an encoder-decoder model the whole input is encoded into a fixed-size context vector, and the output is generated one word at a time from that vector, between a START and an END token. With teacher forcing, true words are fed to the decoder during training. A long sentence is compressed into one vector with loss, and translation quality is lower beyond about 30 tokens. In attention, all encoder hidden states are kept and weighted at every decoding step, which gives interpretable alignments between source and target words. Translations are scored with BLEU, and beam search is used in decoding.

Courses that include this CL

blog
Last Updated 19th November 2026
  • 5
  • Free

Recommended for you

blog
Last Updated 10th December 2023
  • 5
blog
Last Updated 6th September 2023
  • 7
  • Free
blog
Last Updated 30th July 2023
  • 2
blog
Last Updated 16th June 2023
  • 7
blog
Last Updated 17th December 2022
  • 30
blog
Last Updated 16th January 2023
  • 47
  • Free
blog
Last Updated 16th January 2023
  • 49
  • Free
blog
Last Updated 7th January 2023
  • 8
  • Free
blog
Last Updated 20th May 2025
  • 2
blog
Last Updated 19th November 2023
  • 7
blog
Last Updated 7th November 2022
  • 16
  • Free
blog
Last Updated 21st March 2025
  • 251
  • Free

Meet the instructors !

instructor
About the Instructor

https://www.joergosterrieder.com/