Next Words - Lesson 15 - Retrieval, Agents, Multimodal Models

  • 0 Rating
  • 0 Reviews
  • 1 Students Enrolled

Next Words - Lesson 15 - Retrieval, Agents, Multimodal Models

Retrieval, agent loops, CLIP, long windows, four-bit weights and rank-r updates

  • 0 Rating
  • 0 Reviews
  • 1 Students Enrolled
  • Free
  • Course Includes
  • PDF lecture, 21 pages
  • Jupyter notebooks, 3 (zip)
  • Quiz, 5 questions
Tags:
RAG Quantization retrieval agents CLIP



What you will learn

  • Compute a cosine ranking of passages and the end-to-end accuracy of a pipeline of stages.
  • Compute the chance of at least one error over n independent steps of an agent.
  • Compute attention memory, weight memory under quantisation and the numbers trained by a rank-r update.

Courselet Content

2 components
Next Words - Lesson 15 - Retrieval, Agents, Multimodal Models (pdf)
306.14 K

Requirements

  • Cosine similarity, attention, Python

General Overview

Description

T5 with 11 billion parameters is outscored on Natural Questions by a retrieval-augmented generator with 626 million trainable parameters. The cat-and-mat passage has a cosine of 0.92 with the query. Stages at 95, 93 and 70 per cent give 0.62 end to end. In a ReAct cycle the chance of at least one error over ten steps at 5 per cent each is 40 per cent. In CLIP, images and captions are 768 numbers, and a temperature of 0.1 gives a loss of 0.021 nats. Full attention has 2 TB of scores per head at a million tokens. The accuracy of GPT-3.5-Turbo is 75.8 per cent with the answer passage first of twenty and 53.8 with it tenth. The weights of a 70B model are 140 GB at 16 bits and 35 GB at four bits. Students work through three notebooks.

Courses that include this CL

blog
Last Updated 19th November 2026
  • 1
  • Free

Recommended for you

blog
Last Updated 3rd December 2024
  • 5
  • Free
blog
Last Updated 8th March 2025
  • 1
blog
Last Updated 2nd August 2023
  • 2
  • Free
blog
Last Updated 16th June 2023
  • 7
blog
Last Updated 7th January 2023
  • 22
  • Free
blog
Last Updated 7th November 2022
  • 15
  • Free
blog
Last Updated 14th March 2025
  • 52
  • Free
blog
Last Updated 3rd October 2025
  • 97
  • Free
blog
Last Updated 24th September 2026
  • 5
blog
Last Updated 15th January 2026
  • 102
blog
Last Updated 13th December 2022
  • 141
  • Free
blog
Last Updated 21st March 2025
  • 251
  • Free

Meet the instructors !

instructor
About the Instructor

https://www.joergosterrieder.com/