AI · Flashcard
What is PEFT (parameter-efficient fine-tuning)?
- AFine-tuning only a small subset of parameters instead of all of them
- BFine-tuning every weight in the model on a very small dataset
- CRunning a model at lower precision to save memory at inference
- DAdding retrieved documents to the prompt instead of training at all
Why this is the answer
PEFT adapts a model by training a tiny fraction of parameters, keeping cost and memory low while matching full fine-tuning on many tasks.
Official docs