Foundation Models (other modalities)

This page describes the Foundation Models (other modalities) row of the Papers.md matrix: pretrained foundation models for modalities beyond single-cell transcriptomics, covering multimodal omics, spectra, peptides, proteins, and small molecules. The row’s authoritative scope is its Taxonomy.md definition; this page synthesizes what currently sits in it.

Scope boundary

This row is defined by exclusion, which is unusual on the method axis and worth stating plainly: a model belongs here when it is a pretrained, transferable foundation model and none of the four transcriptomics-specific foundation-model rows fits. Those are Next-Token Prediction, Masked Language Modeling and Cell-State & Perturbation Prediction, plus LM + Biological Priors for models that inject structured biology into the backbone. A paper that merely invokes such a model, as an agent’s tool call or a queried service, keeps its own row rather than earning this one.

A consequence to hold on to: an exclusion-defined row will not read as a coherent research programme, and this one does not. A protein-phenotype model and a natural-product graph model share a training paradigm and nothing else. The grouping is a routing decision, not a claim that these papers speak to each other.

Sensory Prediction

• #267 UmamiMeta (He et al. 2025, Journal of Chemical Information and Modeling): predicts umami peptides using ESM2 protein-language-model embeddings, but the paper’s more interesting contribution is that it does not stop at sequence. It trains protein-peptide affinity models on 1,485 IC50 values from ChEMBL and 412 Kd values from GtoPdb and PpI[S/A]DS, converts both to pIC50 and pKd, runs molecular docking of peptides against taste receptors, and feeds the predicted and docked affinities back in as features alongside the language-model embeddings. The docked structures also become a heterogeneous graph the model learns over. It reports that umami peptides bind umami receptors more strongly than non-umami peptides do, while showing no significant difference in affinity to bitter receptors, which is a negative result worth knowing for anyone assuming the two taste axes separate cleanly. The assembled dataset, ummpep2024, merges TastepeptidesDB, TastePepMap, Bio-UWM, UmPred-FRL and iUmami-SCM into 1,580 peptides (972 umami, 608 non-umami), the largest of its kind at time of publication, reduced to 1,546 after peptides longer than 15 residues were dropped. Reported accuracy 82%, AUC 0.87. The web server is catalogued in Databases.md.
• #270 BERT4Bitter (Charoenkwan et al. 2021, Bioinformatics): predicts bitter peptides from sequence with no structural information, on the BTP640 benchmark of 320 bitter and 320 non-bitter peptides split 80/20. It sits in this row rather than Masked Language Modeling because the pretrained-representation work is done by NLP feature encodings applied to peptide sequence, Pep2Vec and FastText word vectors over the 20 amino acids plus TFIDF term weighting, rather than by masked pretraining on a peptide corpus. Reported accuracy 0.861 under cross-validation and 0.922 on the independent test, an improvement of 8.0 points of accuracy and 16.0 points of MCC over the prior method. It shares authors with #269 iUmami-SCM, and both sit alongside #322 Umami-BERT from a separate group, forming a body of sequence-only taste predictors trained on datasets small enough that the representation choice dominates the model choice.

AI Tooling / Methodology

• #42 OmicsLM (Sypetkowski et al. 2026, arXiv): connects quantitative omics profiles to natural-language biological tasks by treating a transcriptome the way a vision-language model treats an image. Each sample is aligned to a fixed panel of 19,260 GENCODE protein-coding and mitochondrial genes and compressed into a single 20,541-dimensional vector that concatenates an input-scale indicator (distinguishing count-derived single-cell from TPM-normalized bulk), the normalized expression vector, a 512-dimensional Funomics embedding and a 768-dimensional Geneformer-V2 embedding. A trainable affine projection maps that vector into the token-embedding space of a Qwen3 backbone, and it enters the prompt as a single <omics> placeholder token. Because a profile costs one token rather than a long gene list, several samples can be interleaved with instructions in one context, which is the capability the paper is built around. The tokenizer is also augmented with gene symbols so names like TP53 are not fragmented into subwords. Trained on more than 5.5 million instruction-following examples across over 70 task types, and released with GEO-OmicsQA, a multi-sample question-answering benchmark built from real GEO studies to measure language-guided reasoning that existing benchmarks miss.
• #91 NaFM (Ding et al. 2026, Nature Machine Intelligence): a foundation model for small-molecule natural products, pretrained on molecular graphs with a GIN message-passing backbone rather than on sequence. The pretraining is tailored to what makes natural products different from synthetic molecules: a masked graph-reconstruction objective over 180 atom categories and 12 bond categories, combined with scaffold-aware contrastive learning. That second part is the idea worth carrying: because a natural product’s properties correlate strongly with its Bemis-Murcko scaffold, the contrastive loss is reweighted by cosine similarity between scaffold MACCS fingerprints, so two molecules sharing a scaffold are penalized less for looking alike, treating them as false negatives rather than true ones. The paper argues, and tests, that models built for synthetic molecules are inadequate for natural-product taxonomy, and shows the representations capture evolutionary signal at gene and microbial level. Relevant to cell-ag anywhere a media component, flavour precursor or bioactive is a natural product.
• #224 ProCyon (Queen et al. 2025, bioRxiv, Zitnik lab): a multimodal model over protein sequence, structure and natural language, trained on ProCyon-Instruct, 33 million protein-phenotype instructions curated from 12 sources across five knowledge domains (function from GO, Reactome, EC and UniProt; domains from GO and Pfam; disease from DisGeNET and OMIM; therapeutics from DrugBank and IUPHAR; interactions from STRING and PDBBind). Open Targets was deliberately held out of training so it could serve as external validation, and protein-phenotype pairs are split into one training set and several test sets probing different levels of generalization. Beyond benchmark numbers, the paper generates candidate phenotype descriptions for poorly characterized proteins, including Parkinson’s-implicated ones identified after the model’s knowledge cutoff, and confirms multiple-sclerosis predictions against post-mortem brain RNA-seq. The cell-ag reading is about the 29% of human proteins with no experimentally validated function: livestock proteomes are annotated far more sparsely still, and this is the model class aimed at that gap.

Adjacent methods

• Foundation Models: Next-Token Prediction, Masked Language Modeling, Cell-State & Perturbation Prediction, LM + Biological Priors: the four transcriptomics-facing rows this one is defined against.
• GNN: the graph architectures NaFM and the peptide-taste models build on, when a paper is a task-specific network rather than a pretrained model.
• Chemometrics: the classical multivariate treatment of the same spectral data these models now consume end-to-end.
• Benchmarks & Evaluation Frameworks: including GEO-OmicsQA, released with OmicsLM.

Further reading

• Research areas: AI Tooling / Methodology, Sensory Prediction, and Metabolic Modeling for the open problem of connecting these embeddings to flux prediction.
• Databases: UmamiMeta for the umami-peptide web server.
• Field overview: AIAgentsFoundationModels.md.