The Next Word Problem - Hour 4

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled

The Next Word Problem - Hour 4

Fine-tuning, prompting, retrieval, alignment and agent safety, with a sonnet lab

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled
  • Free
  • Course Includes
  • PDF slides, 19 pages
  • Room version of the slides (zip)
  • Quiz, 4 questions
Tags:
prompting RAG AI safety Fine-tuning RLHF



What you will learn

  • Choose between fine-tuning and prompting for a given adaptation problem.
  • Apply chain-of-thought prompting and explain temperature, top-k and nucleus sampling.
  • Describe retrieval-augmented generation, the three steps of RLHF and the safety risks of agents.

Courselet Content

2 components
tHE Next Word Problem - Unit 4 (pdf)
467.11 K

Requirements

  • Hours 1 to 3, basic probability

General Overview

Description

A pre-trained model has a training cutoff, no access to private documents and no source for its answers. In fine-tuning, the weights are changed with hundreds to thousands of labelled examples. In prompting, zero to about ten examples are placed in the input. Sampling is controlled by temperature, a rescaling of the logits before the softmax, by top-k, a fixed number of candidates, and by nucleus sampling, the smallest set of words covering a probability p. In retrieval-augmented generation, documents are retrieved at answer time. RLHF has three steps, supervised fine-tuning on demonstrations, a reward model fitted to human preference comparisons and policy optimisation against that reward model. The risk surface grows with the autonomy of an agent, from read-only answering to multi-step execution without review. In a lab, sonnet lines are written by rejection sampling, and 800 attempts give four different lines with ten syllables and a rhyme.

Courses that include this CL

blog
Last Updated 19th November 2026
  • 2

Recommended for you

blog
Last Updated 3rd December 2024
  • 5
  • Free
blog
Last Updated 8th March 2025
  • 1
blog
Last Updated 3rd May 2024
  • 17
blog
Last Updated 19th July 2023
  • 0
  • 0
blog
Last Updated 16th June 2023
  • 7
blog
Last Updated 7th January 2023
  • 22
  • Free
blog
Last Updated 16th January 2023
  • 49
  • Free
blog
Last Updated 7th January 2023
  • 8
  • Free
blog
Last Updated 24th September 2026
  • 5
blog
Last Updated 13th December 2022
  • 141
  • Free
blog
Last Updated 7th November 2022
  • 16
  • Free
blog
Last Updated 21st March 2025
  • 251
  • Free

Meet the instructors !

instructor
About the Instructor

https://www.joergosterrieder.com/