Full fine-tuning, LoRA, prompt patterns, chain-of-thought, instruction tuning and RLHF
|
NLP - Fine-tuning and Prompt Engineering (pdf)
|
419.83 K |
A pre-trained model is adapted to a task by fine-tuning, by prompting or by a combination. In fine-tuning, a task-specific head is added to the model and the weights are updated. In LoRA, low-rank matrices of rank r, typically 8 against a hidden size of 4096, are trained while the original weights are frozen, and about 0.1 per cent of the parameters are updated. Prompt engineering patterns are roles and explicit requirements, and in chain-of-thought prompting the reasoning steps are requested. In instruction tuning, models are trained on triples of instruction, input and output, and in RLHF a model is aligned with human preferences. With fewer than 100 examples, prompt engineering is the first choice, and fine-tuning is the choice for accuracy of 95 per cent or more.