Chapter 16

16Quantization

Draft pending — this page is an outline of planned content. Write it by creating content/quantization.md.

16.1Bits, ranges, and where the memory goes

Planned: Why weights dominate memory; the appeal of fewer bits per weight.

16.2Post-training quantization

Planned: GPTQ, AWQ, and the outlier-feature problem; calibration data.

16.3Formats you'll meet: GGUF, bitsandbytes, MLX

Planned: What each is for; K-quants and mixed-precision schemes.

16.4Quantization-aware training and the frontier

Planned: QAT, 4-bit and below, and the accuracy cliff.