NLP - Decoding Strategies

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled

NLP - Decoding Strategies

Greedy decoding, beam search, temperature, top-k and top-p sampling and repetition penalties

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled
  • Wishlist
  • Free
  • Course Includes
  • PDF slides, 19 pages
  • LaTeX source and figures (zip)
  • Quiz, 4 questions
Tags:
Decoding Beam Search Sampling Text Generation NLP



What you will learn

  • Implement greedy decoding and beam search with paths scored by the sum of log probabilities
  • Control randomness with temperature, top-k sampling and top-p sampling
  • Choose decoding settings for code, chat and story generation

Courselet Content

2 components
NLP - Decoding Strategies (pdf)
466.13 K

Requirements

  • Softmax probabilities, transformer language models

General Overview

Description

A language model gives a probability for each possible next word, and the text produced depends on the decoding method. In greedy decoding the most probable word is always taken, and the same input gives the same output. In beam search the k best paths are kept and scored by the sum of log probabilities, with a beam of 4 to 5 common in translation. Controlled randomness is added by sampling with temperature, top-k and top-p, where the next word is drawn from the smallest set of words with cumulative probability p. Repetition penalties of about 1.2 are applied to recent tokens. Suitable settings depend on the task, for example T = 0.2 with p = 0.95 for code. The same methods are treated in courselets 1019 (a lab) and 1021 (a deck).

Courses that include this CL

blog
Last Updated 19th November 2026
  • 5
  • Free

Recommended for you

blog
Last Updated 3rd May 2024
  • 17
blog
Last Updated 19th November 2026
  • 8
  • Free
blog
Last Updated 19th July 2023
  • 0
  • 0
blog
Last Updated 16th June 2023
  • 7
blog
Last Updated 16th January 2023
  • 49
  • Free
blog
Last Updated 7th January 2023
  • 8
  • Free
blog
Last Updated 16th January 2023
  • 47
  • Free
blog
Last Updated 14th March 2025
  • 52
  • Free
blog
Last Updated 19th November 2026
  • 7
  • Free
blog
Last Updated 7th November 2022
  • 16
  • Free
blog
Last Updated 21st March 2025
  • 251
  • Free

Meet the instructors !

instructor
About the Instructor

https://www.joergosterrieder.com/