Foundation Models: Cell-State & Perturbation Prediction

This page describes the Foundation Models: Cell-State & Perturbation Prediction row of the Papers.md matrix: pretrained, transferable models whose target is predicting cell state and perturbation response across contexts. The row’s authoritative scope is its Taxonomy.md definition; this page synthesizes what currently sits in it.

Scope boundary

The discriminator the taxonomy names is pretraining-and-transfer, not subject matter. A single-task supervised predictor is not a foundation model even when perturbation response is exactly what it predicts, and it belongs in its architecture’s row instead. GEARS (#121) is the clarifying case: it predicts transcriptional outcomes for unperturbed genes, which sounds like this row, but it is a graph network trained end-to-end on one perturbation dataset, so it sits in GNN. A paper that merely invokes such a model, as an agent’s tool call or a queried service, keeps its own row rather than earning this one.

The row overlaps by design with the other foundation-model rows, because a model’s pretraining objective and its target are separate facts about it. Most of the references below also appear in at least one other row.

Cellular Engineering

• #57 State (Adduri et al. 2025, bioRxiv, Arc Institute): a transformer trained on gene expression from over 100 million perturbed cells, built around an explicit generative assumption rather than a direct regression. Because sequencing destroys the cell, the unperturbed state of a given cell is never observable, so State models the perturbation effect against the whole basal population distribution instead of against an individual cell, and splits the job in two: a State Transition module for the perturbation effect plus population heterogeneity, and a State Embedding module for experiment-specific technical noise. Its generalization test is worth reading before trusting any headline number in this area: a target context is held out and the model sees only 30% of that context’s perturbations, so what is measured is transfer to perturbations never seen in that cell type. Reported over 30% improvement in effect discrimination on large datasets, plus identification of strong perturbations in contexts where no perturbations were observed during training, using a cell embedding trained on 167 million observational cells. It also ships Cell-Eval, which is why it appears in Benchmarks & Evaluation Frameworks. Code at ArcInstitute/state; catalogued in Software.md.
• #120 C2S-Scale (Rizvi et al. 2026, bioRxiv, van Dijk lab and Google): scales the Cell2Sentence idea, representing a cell as a rank-ordered “sentence” of its most highly expressed genes, to a 27-billion-parameter LLM trained on over a billion tokens of transcriptomic data, biological text and metadata drawn from 825 scRNA-seq datasets in CELLxGENE and the Human Cell Atlas. Held-out evaluation reserves 10% of cells per study, and harder out-of-distribution checks use datasets released after corpus creation so leakage is structurally excluded. Perturbation-response prediction comes from targeted reinforcement-learning fine-tuning rather than from pretraining alone, which is why the paper also sits in Reinforcement Learning and Next-Token Prediction. The result the authors put weight on is a dual-context virtual screen nominating the kinase inhibitor silmitasertib for context-selective upregulation of antigen presentation, then supported experimentally in human cell models unseen during training. Code at vandijklab/cell2sentence.
• #124 Stack (Dong et al. 2026, bioRxiv, Arc Institute): trained on 149 million uniformly preprocessed human single cells, and structurally different from the models above in that it does not represent a cell independently. A dual attention mechanism runs intra-cell attention across gene-module tokens and inter-cell attention across the cells in a set, so a cell’s representation is informed by its neighbours in the same experiment. The consequence is in-context learning: unlabeled cells representing an arbitrary condition, a chemical perturbation or a different donor, can be supplied at inference time and the model predicts that condition’s effect on a target population without fine-tuning. Applied at scale it produced Perturb-Sapiens, a whole-organism atlas of perturbed human cells spanning 28 tissues, 40 cell classes and 201 perturbations, with subsets validated against in vitro stimulation profiles.

The three together trace a clear methodological arc, from modelling the perturbation effect against a population, to putting it in a language model, to making the context itself the input. For cellular agriculture the caveat is uniform across all three: the training corpora are human and mouse, so applying any of them to bovine or porcine cells is a transfer claim the papers do not make.

AI Tooling / Methodology

• #117 scGPT (Cui et al. 2024, Nature Methods, Wang lab): pretrained on over 33 million cells and fine-tuned across cell-type annotation, multi-batch integration, multi-omic integration, perturbation-response prediction and gene-network inference. Two design choices explain why it became a standard baseline. Genes are tokenized by name, so studies with different gene sets harmonize by taking the union of vocabularies rather than by intersecting to a shared panel. And expression is value-binned per cell, with bin edges recomputed for each cell so that the top bin always means “most expressed in this cell”, which makes the same input value carry the same meaning across sequencing batches of very different depth. It sits in the tooling column because it is a general single-cell model rather than a cell-ag application, and in Masked Language Modeling and Next-Token Prediction for its pretraining objectives. Code at bowang-lab/scGPT; catalogued in Software.md.

Adjacent methods

• Foundation Models: Next-Token Prediction and Masked Language Modeling: the pretraining objectives behind most of these models.
• Foundation Models: LM + Biological Priors: the cross-species tokenization route, which is the most cell-ag-relevant variant.
• GNN: where a single-task perturbation predictor such as GEARS belongs.
• Reinforcement Learning: the post-training step that gives C2S-Scale its perturbation performance.
• Benchmarks & Evaluation Frameworks: Cell-Eval and the wider argument over whether foundation-model gains survive comparison against simple baselines.

Further reading

• Research areas: Cellular Engineering for the surrounding single-cell work, and Metabolic Modeling for the open problem of bridging these embeddings to flux prediction.
• Software: Software.md for State and Stack, scGPT, and Cell2Sentence.
• Field overview: AIAgentsFoundationModels.md, which carries the Arc Institute and van Dijk lab project pages for these models.