10Small-Molecule Drug Design

A small molecule is not a protein, and that difference organizes this whole chapter. Where a protein is a chain of hundreds of amino acids whose statistics evolution has already written down, a drug like aspirin is a compact assembly of twenty-one atoms, with no sequence to read off and no homologs to align. Vast libraries of molecular structures exist — hundreds of millions of catalogued compounds — but molecules never reproduced under selection, so, unlike a protein family, a drug carries no evolutionary record of what works. Yet the job is just as hard: a small-molecule drug must grip one chosen protein tightly, ignore the roughly twenty thousand other human proteins, and survive the body's chemistry long enough to act. This is the small-molecule peer to protein and binder design (Chapter 9), and the book's molecular lobe is otherwise almost entirely protein-centric. The models split into two faces — one scores a molecule you hand it, the other generates a molecule to fit a pocket — and the intellectual spine running under both is a real, unsettled tension: whether to trust general models that learn from data and compute, or to bake in the physics that chemistry has understood for a century.

10.1The problem: potency, selectivity, and surviving the body

Fix a target protein with a druggable pocket (druggability and pocket-finding are Chapter 6) and ask for a small molecule that acts on it. Three demands pull at once. The first is potency: how little of the drug you need to get the effect, usually reported as an IC50 or EC50 — the concentration that produces half of the maximum inhibition or effect, so a lower number means a more potent molecule — and underneath it a binding affinity, the dissociation constant Kd from property prediction (Chapter 7), where smaller is tighter. The second is selectivity: binding only the intended target and not the thousands of related proteins whose accidental engagement becomes a side effect or a toxicity. The third is everything the body does to the molecule — the ADMET properties (absorption, distribution, metabolism, excretion, toxicity) that Chapter 7 treats as affinity-adjacent properties in their own right. A molecule can be exquisitely potent in a tube and still be useless because it is promiscuous, or because the liver destroys it in minutes.

To model any of this you first have to represent the molecule, and there are three standard views, each adding what the previous one throws away. The flattest is SMILES (Simplified Molecular-Input Line-Entry System), a text string encoding atoms and bonds — CC(=O)O is acetic acid — which lets you treat a molecule like a sentence and reuse language-model machinery. One step richer is the molecular graph, atoms as nodes and bonds as edges, the natural input for a graph neural network and the view that makes chemistry's connectivity explicit. Richest, and the one where potency is actually decided, is the 3D binding pose: a specific placement of the molecule's atoms inside the target's pocket, presenting the right pharmacophore — the arrangement of hydrogen-bond donors and acceptors, hydrophobic groups, and charges that the pocket is shaped to grip. Structure-based drug design lives in this last view; the flat views are cheaper but blind to the geometry that binding depends on.

10.2Models that attempt it

The models wear two faces, and it helps to keep them separate. The first face is scoring: given a molecule and a pocket, estimate how well it binds. Scoring a molecule against a target is an old problem — for decades the tool was QSAR (quantitative structure–activity relationship), regressing hand-crafted molecular descriptors against measured activity, the ancestor of every learned scorer here. The structure-based version is docking: search over candidate poses of the molecule in the pocket and rank them with a scoring function, a fast estimate of binding quality standing in for a real free energy. AutoDock Vina is the classical workhorse, pairing an efficient pose search with an empirical, physics-inspired scoring function (Trott & Olson, 2010). Gnina keeps Vina's search but replaces the scorer with an ensemble of convolutional neural networks that read the 3D complex directly — a learned scoring function grafted onto a classical docker (McNutt et al., 2021). The newest option folds scoring into structure prediction: the co-folding models from Chapter 8 (AlphaFold3, and the open Boltz and Chai lines (Abramson et al., 2024; Wohlwend et al., 2024; Chai Discovery, 2024)) predict the joint protein–ligand structure, and Boltz-2 adds a binding-affinity head on top, reporting correlation approaching that of open-source free-energy perturbation at over a thousandfold less compute, and outperforming every method submitted to the CASP16 affinity challenge in a retrospective evaluation (Passaro et al., 2025).

The second face is generating: rather than score a molecule you supply, invent one that fits the pocket. A molecular generative model samples new molecules from a learned distribution — over SMILES strings, graphs, or 3D atom clouds — instead of scoring ones you hand it. The pocket-conditioned 3D generators are the direct analogue of protein backbone design: TargetDiff diffuses atom coordinates and types inside a fixed pocket, growing a molecule shaped to fit (Guan et al., 2023), and Pocket2Mol samples atoms autoregressively from an equivariant network conditioned on the pocket, trading diffusion's iterative denoising for direct placement (Peng et al., 2022). DiffDock attacks the neighboring problem of pose-finding by framing docking itself as diffusion over the ligand's position, orientation, and torsion angles (Corso et al., 2022). Chapter 9 named these as a parallel small-molecule track in a single paragraph; here they are the main event, and they inherit two headaches the protein side largely escapes — whether a proposed molecule can actually be made, and how crudely structure alone predicts binding.

10.3The bitter lesson versus physical constraints

Here is the tension the rest of the chapter has been circling. Rich Sutton's bitter lesson is the observation, drawn from seventy years of AI, that general methods which leverage computation and data eventually beat methods built on hand-engineered human knowledge, and beat them by a wide margin (Sutton, 2019). Language modeling is the poster child: scale the data and the parameters, and structure you would have hand-coded emerges for free. Molecular ML feels the opposite pull. The field's instinct is to bake in physics — equivariance so a model's predictions rotate with the molecule, explicit energy terms, known bond geometry, and at the far end a force field (a parameterized model of the potential energy of a set of atoms) driving a physics simulation. The most rigorous form is free energy perturbation (FEP): simulate the alchemical transformation of one ligand into another and compute the relative binding free energy between them, calibrated in real energy units. Modern FEP+ protocols predict relative potencies of close analogs to roughly chemical accuracy, about 1 kcal/mol (Wang et al., 2015), which is exactly the hand-engineered, physics-heavy end of the spectrum the bitter lesson warns against.

What keeps this a genuine debate rather than a settled rout is data. Language had trillions of tokens; well-measured protein–ligand binding data numbers in the low tens of thousands of complexes, heavily biased toward a few protein families and the chemotypes industry already explored. In that regime, priors pay: a physical constraint is worth a great deal of missing data, which is why equivariant architectures and physics terms remain competitive and why FEP still wins for the narrow, high-value task of ranking close analogs in lead optimization. And yet the balance is visibly shifting — Boltz-2 approaching FEP-level correlation from learned data is precisely the kind of result the bitter lesson predicts, arriving earlier than many chemists expected. The honest position for 2026 is that the physics pole still holds where data is thin and calibration matters, the learned pole is winning where you need to triage millions of molecules fast, and neither has won outright.

10.4What they do well and what is still hard

Read the wins narrowly and they are real. These models triage enormous virtual libraries, turning millions of candidates into a testable few hundred with meaningful enrichment; they propose pocket-fitted scaffolds a chemist would not have drawn; they rank close analogs well enough (FEP at the physics end) to guide lead optimization; and co-folding now hands you a plausible complex and an affinity estimate without a crystal structure to start from. For prioritizing what to make and measure next, this is genuinely useful.

The frontier is everything causal and in vivo that a single scored pose cannot express. Start with the scoring functions themselves: they correlate only weakly with measured affinity (the "scoring power" they are weakest at), and even AI dockers that post excellent pose RMSD can be producing physically invalid geometry — clashing atoms, strained bonds — and failing to generalize to targets unlike their training set, while classical physics-based tools stay more physically valid (Buttenschoen et al., 2024). Selectivity is modeled only narrowly: within-family panels exist, but a scorer evaluates one target in isolation, the toxicity that kills a program comes from the thousands of proteins it was never scored against, and proteome-wide off-target prediction is not routine. Then the in-vitro→in-vivo gap swallows the rest — potency in a well says little about whether a molecule is absorbed, evades metabolism, avoids toxicity, and reaches its target in a living body, the same dish-to-patient translation gap that haunts target discovery (Chapter 6) and cell engineering (Chapter 11). Finally, generative models expose synthesizability: a pocket-conditioned generator will happily propose molecules no chemist can make. Synthetic-accessibility scores are a rough filter and retrosynthesis planning the real test, and the problem compounds with distribution shift — a model confident on known drug-like space extrapolates poorly to the genuinely novel chemotypes a program most wants, which is exactly where its own scoring is least reliable.

The mental model to carry forward: generative and scoring models have made it cheap to propose and rank pocket-fitting molecules, and the bitter-lesson-versus-physics balance is tilting slowly toward learning as data grows without ever fully abandoning the physical priors that scarce data still rewards. What stays hard is the causal, in-vivo chain — selectivity, ADMET, and efficacy in a body — that a scored pose cannot reach, which is why evaluation must be designed to measure it (Chapter 16), why the bench remains the judge (Chapter 17), and why closing this gap is one of the open frontiers the Outlook (Chapter 18) returns to.