13Parameter-Efficient Fine-Tuning

Full fine-tuning updates every weight in the model, which is the obvious way to adapt a pretrained model to a new task and, for large models, an expensive one. It demands the same memory as a training run from scratch, and it leaves you with a full-size copy of the model for every task you fine-tune. Parameter-efficient fine-tuning (PEFT) refuses that bargain: it freezes the pretrained weights and trains a tiny add-on, a few million parameters instead of billions. This chapter is mostly about one method — LoRA — that has become the default because it costs almost nothing to train, nothing at all to serve once merged, and barely dents quality.

13.1Why full fine-tuning hurts

The cost is not the weights; it is everything training drags along with them. Chapter 8's memory budget is the whole argument: mixed-precision AdamW carries about 16 bytes per parameter — the bf16 weights, their gradients, an fp32 master copy, and two optimizer moments — so the optimizer state alone is six times the size of the model. Fine-tuning a 7B model this way needs well over 100 GB of state before activations, which is why a model that runs on one GPU will not train on it. And the moment you fine-tune a second task, you own a second full checkpoint: ten specialized 70B models are ten times 140 GB to store and to load.

PEFT attacks both problems at once. Freeze the pretrained weights and they need no gradients, no master copy, and no optimizer moments — they sit in memory as a single read-only tensor, shared across every task. Only the small add-on gets an optimizer, so the 12-bytes-per-parameter tax falls on millions of parameters rather than billions. The base model becomes a fixed substrate; each task is a lightweight overlay.

13.2LoRA

LoRA (Low-Rank Adaptation) rests on a hypothesis about how a model changes when you fine-tune it: the update is intrinsically low-rank (Hu et al., 2021). Adapting a base model to a task does not rewire it — it nudges it along a handful of directions. Aghajanyan and colleagues measured this directly, tuning RoBERTa to most of its full performance through only a few hundred parameters projected into weight space, and found the "intrinsic dimension" of fine-tuning is small and shrinks as models grow (Aghajanyan et al., 2021).

So instead of learning a full weight change \(\Delta W\) (a \(d \times k\) matrix), LoRA forces it through a rank-\(r\) bottleneck. It writes the update as a product of two thin matrices, \(\Delta W = BA\) with \(B \in \mathbb{R}^{d \times r}\) and \(A \in \mathbb{R}^{r \times k}\) and \(r\) as small as 8, and the adapted layer computes

Only \(A\) and \(B\) train; \(W_0\) stays frozen. For a \(4096 \times 4096\) projection, a full \(\Delta W\) is 16 million numbers; at \(r = 8\) the two factors together are about 65 thousand — a 250-fold cut. \(B\) starts at zero, so the adapter begins as the identity and training only ever adds to a working model. The scalar \(\alpha / r\) decouples the update's magnitude from the rank you happened to choose.

The payoff at inference is that there is none to pay. Because \(W_0 + \frac{\alpha}{r}BA\) is just another \(d \times k\) matrix, you can fold the adapter into the base weights once and serve a model that is bit-for-bit the shape of the original — no extra layers, no added latency. This is the sharp contrast with older adapter methods that inserted extra modules into the forward pass and taxed every token forever.

13.3QLoRA and quantized adapters

LoRA shrinks the trainable state, but you still hold the frozen base in memory, and at 65B parameters even a read-only bf16 copy is 130 GB. QLoRA removes that wall by storing the frozen base in 4 bits while training the LoRA adapters in higher precision (Dettmers et al., 2023). Three ideas make it work without hurting quality: NF4 (NormalFloat4), a 4-bit type shaped for the roughly-Gaussian distribution of neural network weights; double quantization, which quantizes even the quantization constants to shave a little more; and paged optimizers, which spill optimizer memory to CPU RAM to survive the spikes that would otherwise crash the run.

The mechanics are worth stating plainly. The base weights live in 4 bits, but each is dequantized back to bf16 on the fly for its matmul; gradients then flow through those dequantized values into the adapters, never into the frozen quantized weights. The result is that a 65B model fine-tunes on a single 48 GB GPU, and QLoRA reports matching full 16-bit fine-tuning quality on its benchmarks. Chapter 16 covers 4-bit quantization for its own sake; here it is a means to fit the frozen substrate.

13.4Choosing rank, alpha, and targets

Three knobs decide how a LoRA run goes. Rank \(r\) sets the adapter's capacity: too low starves a hard task, but returns flatten quickly, and much of the benchmark literature finds little gain past \(r = 16\) for typical instruction tuning. Alpha scales the update; the common practice of holding \(\alpha / r\) roughly constant keeps the effective learning rate steady as you sweep rank. Targets name which matrices get adapters — the original work put LoRA only on the attention projections, but adapting the FFN matrices too (Chapter 4's two-thirds of the parameters) often helps, at proportional cost.

Where PEFT plateaus is the honest limit. Adapters excel at teaching behavior and format — the assistant persona, a domain's style, a tool-calling convention — which is exactly what SFT (Chapter 10) mostly does. They are weaker at cramming large amounts of new knowledge into a model, where the low-rank bottleneck bites and full fine-tuning still wins. Successor methods chip at the gap: DoRA splits each weight into a magnitude and a direction and applies LoRA only to the direction, closing much of the remaining distance to full fine-tuning at similar parameter cost (Liu et al., 2024).

13.5Serving many adapters

Keeping the adapter unmerged turns LoRA's one weakness into its best serving trick. If every task shares one frozen base and differs only by a few megabytes of \(A\) and \(B\), a server can hold the base once and swap adapters per request — hot-swapping a customer's fine-tune in milliseconds instead of loading a whole new model. The multi-tenant story is dramatic: thousands of specialized models backed by a single copy of the expensive weights.

The subtlety is batching. Merging fuses the adapter into the base and gives zero latency, but a merged model serves exactly one task, so a batch of requests for different adapters cannot share a forward pass. Systems like S-LoRA keep adapters separate and compute the base matmul once for the whole batch while applying each request's low-rank term with a custom kernel, serving thousands of concurrent adapters at throughput close to the unadapted model (Sheng et al., 2024). It is the same merge-versus-separate trade from the LoRA section, now decided by whether you optimize a single tenant's latency or many tenants' shared throughput — the serving concern of Chapter 17, and one more reason the 4-bit base of Chapter 16 matters.

PEFT changes the economics of the whole post-training story: adaptation stops being a per-task copy of an enormous model and becomes a small file you train cheaply, store by the thousand, and serve against one shared base. The next part turns from making models behave to making them fast — the serving and quantization machinery these adapters lean on.