NLP - Fine-tuning and Prompt Engineering

  • 0 Rating
  • 0 Reviews
  • 3 Students Enrolled

NLP - Fine-tuning and Prompt Engineering

Full fine-tuning, LoRA, prompt patterns, chain-of-thought, instruction tuning and RLHF

  • 0 Rating
  • 0 Reviews
  • 3 Students Enrolled
  • Wishlist
  • Free
  • Course Includes
  • PDF slides, 19 pages
  • LaTeX source and figures (zip)
  • Quiz, 4 questions
Tags:
LoRA Prompt Engineering RLHF NLP Fine-tuning



What you will learn

  • Implement a LoRA adapter whose rank r is far below the hidden size d
  • Write prompts with roles, explicit requirements and chain-of-thought steps
  • Choose between full fine-tuning, prompting, LoRA, instruction tuning and RLHF for a given task

Courselet Content

2 components
NLP - Fine-tuning and Prompt Engineering (pdf)
419.83 K

Requirements

  • Pre-trained models such as BERT and GPT, gradient-based training

General Overview

Description

A pre-trained model is adapted to a task by fine-tuning, by prompting or by a combination. In fine-tuning, a task-specific head is added to the model and the weights are updated. In LoRA, low-rank matrices of rank r, typically 8 against a hidden size of 4096, are trained while the original weights are frozen, and about 0.1 per cent of the parameters are updated. Prompt engineering patterns are roles and explicit requirements, and in chain-of-thought prompting the reasoning steps are requested. In instruction tuning, models are trained on triples of instruction, input and output, and in RLHF a model is aligned with human preferences. With fewer than 100 examples, prompt engineering is the first choice, and fine-tuning is the choice for accuracy of 95 per cent or more.

Courses that include this CL

blog
Last Updated 19th November 2026
  • 5
  • Free

Recommended for you

blog
Last Updated 10th December 2023
  • 5
blog
Last Updated 6th September 2023
  • 7
  • Free
blog
Last Updated 30th July 2023
  • 2
blog
Last Updated 16th June 2023
  • 7
blog
Last Updated 17th December 2022
  • 30
blog
Last Updated 16th January 2023
  • 47
  • Free
blog
Last Updated 7th January 2023
  • 8
  • Free
blog
Last Updated 20th May 2025
  • 2
blog
Last Updated 7th November 2022
  • 11
  • Free
blog
Last Updated 19th November 2023
  • 7
blog
Last Updated 7th November 2022
  • 16
  • Free
blog
Last Updated 21st March 2025
  • 251
  • Free

Meet the instructors !

instructor
About the Instructor

https://www.joergosterrieder.com/