Data, not compute or architecture, is now the binding constraint. The talk takes three angles on that. Measure data, so that the value of a dataset for a task can be predicted before training on it. Stretch data, so that a token's worth is known once fresh tokens run out. Amortize data, so that one training run pays for a whole family of models. It closes on open questions for a data-centric foundation for machine learning.

The base is the optimal transport dataset distance, OTDD (Alvarez-Melis and Fusi 2020). A labelled dataset is a distribution over (feature, label) pairs. The distance between two datasets is an OT problem whose ground cost compares features directly and compares labels by the Wasserstein distance between their class-conditional feature distributions. Labels become distributions, so two datasets with disjoint label sets still have a distance. It is model-agnostic, needs no training, and in the paper correlates with how hard transfer from one dataset to the other turns out to be.

Once datasets live in a metric space, they can be moved through it. The gradient-flow paper (Alvarez-Melis and Fusi 2021) treats a labelled dataset as a particle system and flows it in Wasserstein space to minimize an objective: impose constraints, adapt it toward a target for transfer, or reshape it so a fixed classifier can read it. The generalized-geodesics paper (Fan and Alvarez-Melis 2023) synthesizes new pretraining sets as OT interpolations of existing ones. Instead of choosing the nearest dataset in a collection, it builds the combination nearest the target, even across unrelated label sets.

The baseline is Muennighoff et al. (Muennighoff et al. 2023). With fixed data and a fixed compute budget, up to about four epochs of repetition costs almost nothing in loss. Past that, the value of extra compute decays toward zero. Their scaling law discounts repeated tokens and excess parameters.

Qin, Hamidieh and Alvarez-Melis (Qin, Hamidieh, and Alvarez-Melis 2026) generalize the discount. Compute-Data scaling laws add a token-effectiveness function η: the value of a derived token, from repetition or paraphrase, relative to a fresh one, anywhere from a full substitute to worthless. Fitted on Dolma-3 from 14M to 600M parameters, η is not a constant. It depends on model size, tokens per parameter and how much derived data there already is, and it saturates. Training falls into compute-bound, data-bound and model-bound regimes. By their fit, classical compute-optimal allocation is suboptimal across most practical settings.

The "synergistic domain" case matches Hamidieh, Mackey and Alvarez-Melis (Hamidieh, Mackey, and Alvarez-Melis 2026). Domain-aware scaling laws are fitted observationally across open-weight models with different pretraining mixtures. They estimate how one domain contributes to a benchmark in another, the code-helps-math effect, and second-order domain-domain terms. The estimates were then checked by training on predicted-optimal and predicted-anti-optimal mixtures. That this is the paper behind the abstract's phrase is inference from the title, not a statement in the listing.

Boomerang distillation (Kangaslahti et al. 2025) distills a large teacher into a small student, initialized by pruning the teacher, then grows the student back by swapping contiguous blocks of teacher layers in for student layers. No further training. The intermediate models interpolate smoothly in size and performance between student and teacher, and often match or beat separately pretrained or distilled models of the same size. It only works when the student stays aligned with the teacher, which is what the pruning initialization and the distillation loss provide. Code is at github.com/dcml-lab/boomerang-distillation.

Two follow-ups. Layer patching (Kangaslahti et al. 2026) casts the choice of which student layers to replace, at each size, as a shortest-path problem in a DAG; simple first-to-last or last-to-first orders do well, and a greedy KL-based method, KLPatch, often does better. ADAPT (Zhou et al. 2026) extends the trick across post-training variants (instruct, reasoning, chat), yielding L sizes times K variants from one distillation run.

The link to the data theme is in the accounting. A model family usually spends its pretraining tokens once per size. Boomerang spends them once.

David Alvarez-Melis is an Assistant Professor of Computer Science at Harvard SEAS. The listing gives nothing more. His arXiv record shows a line of optimal transport work back to 2017, Gromov-Wasserstein word embedding alignment and structured OT among it, and interpretability work with Tommi Jaakkola. The OTDD and gradient-flow papers are with Nicolò Fusi.

• OTDD needs a feature space in which a Euclidean ground cost means something. For pretraining-scale text, whose embedding defines the geometry, and how much do the transfer predictions move when the embedding changes?
• η is fitted on Dolma-3 up to 600M parameters. Does its functional form hold at frontier scale, or is the three-regime picture a small-model result? Is paraphrase effectiveness a property of the paraphraser as much as of the data?
• The synergy estimates come from open-weight models whose mixtures are self-reported and confounded with everything else about each lab's recipe. What identifies the causal effect beyond the confirmation runs?
• Boomerang inherits the teacher's data and its mistakes at every size. Does a patched intermediate model behave like a smaller model, or like a teacher with holes, for example in calibration or refusal behaviour?
• The three threads share a unit: value per token. Is there one quantity that OT distance, η and synergy are all estimates of?

Nothing here covers optimal transport or distillation. The stretching thread touches one small, concrete case:

• presci-6L256d: a language model that stops at 1543 — a 17.7M-parameter model on a corpus with a hard date cutoff, trained at 2.04 and then 19.94 tokens per parameter. A closed corpus is the data-bound regime by construction. Where the second run sits on an η curve is the question Qin et al. answer for open corpora.
• Generating a Math Corpus from First Principles — synthetic data added to that same closed corpus. Whether a generated math token is worth a period-English token, or helps it, is the synergy question at toy scale.

Earlier talks in the series: Tomer Ullman: The Bare Stage Before the Mind's Eye, Magnus Egerstedt: From Coordination to Collaboration in Multi-Robot Systems and Marcussen and Gong: Quality Control for Pattern Counting, Adaptivity in Quantum Learning.

Notes pending.