Reinforcement Learning

This page describes the Reinforcement Learning row of the Papers.md matrix: learning policies from reward signals, spanning classical RL and policy-gradient control through to LLM post-training methods such as GRPO and RLHF. The row’s authoritative scope is its Taxonomy.md definition; this page synthesizes what currently sits in it.

Scope boundary

The row is defined by the reward signal, which is what lets it span two bodies of work that otherwise share nothing. On one side are bioprocess control papers where a policy learns to feed or cool a reactor by trial and error against a process model; on the other, LLM and generative-model post-training where a reward function encodes what a good answer or a plausible cell looks like. Both are policies optimized against rewards, so both are here.

The practical distinction to hold is against Bayesian Optimization and Active Learning, which also choose what to do next. Those two select the next experiment to run and are evaluated on sample efficiency against a real budget; RL learns a policy over a trajectory, usually against a simulator, and is evaluated on the policy’s return. A cell-ag team with a physical experiment budget usually wants the former; one with a process model it trusts can afford the latter.

Cellular Engineering

#120 C2S-Scale (Rizvi et al. 2026, bioRxiv, van Dijk lab and Google): described in full under Cell-State & Perturbation Prediction. It is in this row because its perturbation-prediction, natural-language-interpretation and biological-reasoning performance comes from targeted reinforcement-learning fine-tuning rather than from pretraining, which is what enabled the dual-context virtual screen that nominated silmitasertib.
#165 CellFluxRL (Wu et al. 2026, arXiv): post-trains an image-based generative virtual-cell model with RL, and its framing is the clearest statement in the matrix of what a reward function is for. A flow-matching objective trains at the pixel level and produces images that are visually plausible but can violate known cellular properties, wrong nuclear size, implausible morphology, a profile inconsistent with the compound’s mode of action. The paper designs seven rewards across three categories (biological function, structural validity, morphological correctness) and optimizes CellFlux against them, with a KL penalty back to the pretrained model that serves two purposes: preserving the diversity learned in pretraining, and mitigating reward hacking, where high scores are reached by degenerate output rather than genuine correctness. The authors argue the rewards are not in tension with the distributional objective but a targeted alignment of it, because real cells treated with a microtubule destabilizer really do have smaller, fragmented nuclei. Reported to improve over CellFlux on all seven rewards, with further gains from test-time scaling. The rewards double as new evaluation metrics.

Bioprocess & Scale-Up

#200 (Petsagkourakis et al. 2020, Computers & Chemical Engineering): the foundational demonstration for this column. Batch bioprocesses are hard to optimize because they run unsteady-state, behave stochastically, and almost always carry plant-model mismatch. The method parametrizes a control policy as a recurrent neural network, pre-trains it against a preliminary process model to get a starting policy, then updates it batch-to-batch from measurements of the true plant. Because the policy is stochastic and returns distribution parameters rather than an action, the exploration-exploitation tradeoff is represented explicitly as the variance, with a deterministic policy the limiting case as variance goes to zero. Tested on three case studies including a non-smooth one, against a more complex “true” model carrying process disturbance, and discussed candidly against nonlinear model-predictive control rather than declared superior to it.
#201 (Oh et al. 2022, AIChE Journal): starts from the observation that motivates most hybrid work here, that naive model-free RL will usually fail to improve an existing control policy because the available data is limited. Its answer is to learn an action-value function off-policy with a double deep Q-network and then use that function as the terminal cost of a model-predictive controller, so the model-based optimizer handles the horizon it can see and the learned value handles everything past it. Applied to an industrial-scale penicillin semi-batch process, it beat both model-based and model-free baselines and learned from less data than model-free RL.
#202 (Rajasekhar et al. 2024, Biotechnology and Bioengineering): temperature control of a nonlinear ethanol fermentation bioreactor with twin-delayed DDPG, tested unconstrained and constrained, with several reward formulations. With an integral-absolute-error reward TD3 reached MSE 0.22 against DDPG’s 0.29 unconstrained, and 0.38 against 0.48 constrained, and the trained agent rejected disturbances in inlet flow rate and inlet temperature alongside a setpoint change. The reward-function comparison is the useful part for a practitioner: the algorithm choice matters less than what the reward measures.
#203 (Wu & Cui 2025, HPCCE 2024): multi-objective control of lysine fed-batch fermentation, maximizing yield (product per substrate) and productivity (product per unit time) together rather than collapsing them into one number. The formulation is worth noting because the same two moves recur in cultivated-meat process optimization: the terminal time is itself an optimization variable, handled by a time-scaling transformation onto a fixed horizon, and the state and control constraints are folded into the objective by a penalty method rather than enforced directly. Reported to recover a better-spread Pareto set of yield-versus-productivity tradeoffs in far fewer iterations than the alternatives.

The four together trace the column’s arc from pure RL to hybrids that keep classical control in the loop, and every one of them is evaluated against a simulated reactor rather than a physical one. That is the standing caveat for this column: the papers demonstrate that a policy can be learned, not that it survives a real fermentation.

AI Tooling / Methodology

#161 ether0 (Narayanan et al. 2025, arXiv, FutureHouse): a 24-billion-parameter reasoning model for chemistry, post-trained with reinforcement learning (GRPO) on 640,730 experimentally-grounded problems across 375 tasks, from synthesizability to blood-brain-barrier permeability to receptor activity to scent. The claim it tests is whether reasoning-model post-training generalizes past mathematics, code and logic, and it does so without additional domain pretraining and on substantially less data than specialized models, exceeding general-purpose chemistry models, frontier models and human experts on molecular design. Two details in the appendix are more instructive than the headline. Reasoning traces used for the supervised fine-tuning stage were filtered by a separate model against an explicit rubric that rejects reasoning which presumes the answer it is justifying, and traces were summarized under an instruction to preserve verification, backtracking, subgoal-setting and back-chaining behaviours rather than compress them away. Whether a reward-trained model’s chain of thought is genuine reasoning or post-hoc justification is exactly the question a curator should ask of any such system. Relevant to cell-ag for media-component, scaffold-chemistry and flavour-molecule design. Code at Future-House/ether0.

Adjacent methods

Bayesian Optimization and Active Learning: the sample-efficient alternatives when experiments are physical and expensive rather than simulated.
Genetic Algorithms: the other search strategy commonly paired with a process model for bioreactor control trajectories.
Deep Learning: the network architectures these policies and value functions are built from.
Foundation Models: Cell-State & Perturbation Prediction and Next-Token Prediction: the pretrained models that RL post-training is applied to.
Chemistry / Synthesis Agents: the tool-augmented approach to the same chemistry tasks ether0 solves by post-training.