Large-scale manufacturing of induced pluripotent stem cells (iPSCs) is essential for cell therapies and regenerative medicines. Yet, iPSCs form large cell aggregates in suspension bioreactors, resulting in insufficient nutrient supply and extra metabolic waste build-up for the cells located at the core. Since subtle changes in micro-environment can lead to a heterogeneous cell population, a novel Biological System-of-Systems (Bio-SoS) framework is proposed to model cell-to-cell interactions, spatial and metabolic heterogeneity, and cell response to micro-environmental variation. Building on stochastic metabolic reaction network, aggregation kinetics, and reaction-diffusion mechanisms, the Bio-SoS model characterizes causal interdependencies at individual cell, aggregate, and cell population levels. It has a modular design that enables data integration and improves predictions for different monolayer and aggregate culture processes. In addition, a variance decomposition analysis is derived to quantify the impact of factors (i.e., aggregate size) on cell product health and quality heterogeneity.

Neural Architecture Search (NAS) has become a widely used tool for automating neural network design. While one-shot NAS methods have successfully reduced computational requirements, they often require extensive training. On the other hand, zero-shot NAS utilizes training-free proxies to evaluate a candidate architecture test performance but has two limitations (1) inability to use the information gained as a network improves with training and (2) unreliable performance, particularly in complex domains like RecSys, due to the multi-modal data inputs and complex architecture configurations. To synthesize the benefits of both methods, we introduce a sub-one-shot paradigm that serves as a bridge between zero-shot and one-shot NAS. In sub-one-shot NAS, the supernet is trained using only a small subset of the training data, a phase we refer to as warm-up. Within this framework, we present SiGeo, a proxy founded on a novel theoretical framework that connects the supernet warm-up with the efficacy of the proxy. Extensive experiments have shown that SiGeo, with the benefit of warm-up, consistently outperforms state-of-the-art NAS proxies on various established NAS benchmarks. When a supernet is warmed up, it can achieve comparable performance to weight-sharing one-shot NAS methods, but with a significant reduction (∼60%) in computational costs.

Biopharmaceutical manufacturing is a rapidly growing industry with impact in virtually all branches of medicine. Biomanufacturing processes require close monitoring and control, in the presence of complex bioprocess dynamics with many interdependent factors, as well as extremely limited data due to the high cost of experiments and the novelty of personalized bio-drugs. We develop a new model-based reinforcement learning framework that can achieve human-level control in low-data environments. A dynamic Bayesian network is used to capture causal interdependencies between factors and predict how the effects of different inputs propagate through the pathways of the bioprocess mechanisms. This model is interpretable and enables the design of process control policies that are robust against model risk. We present a computationally efficient, provably convergent stochastic gradient method for optimizing such policies. Validation is conducted on a realistic application with a multidimensional, continuous state variable.

Can a reinforcement learning (RL) agent remember and learn from the past, just like a human? Short answer: “yes but only after selecting relevant and valuable experiences from the memory.” I am glad to announce my new paper and its open-source project, both built on my research about “variance reduction based experience replay” (VRER). Long story short, VRER is a generic experience replay method with provable sample efficiency. VRER makes the reinforcementlearning agent remember by selectively replaying past experiences. This selective mechanism can adaptively filter out samples that are outdated, irrelevant and unstable. Our empirical study shows that VRER substantially improves the state-of-the-art policy optimization algorithms, such as trust region policy optimization and proximal policy optimization, in both convergence speed and robustness.

Background: Comorbid chronic conditions are common among people with type 2 diabetes. We developed an artificial intelligence algorithm, based on reinforcement learning (RL), for personalized diabetes and multimorbidity management, with strong potential to improve health outcomes relative to current clinical practice. Methods: We modeled glycemia, blood pressure, and cardiovascular disease (CVD) risk as health outcomes, using a retrospective cohort of 16,665 patients with type 2 diabetes from New York University Langone Health ambulatory care electronic health records in 2009-2017. We trained an RL prescription algorithm that recommends a treatment regimen optimizing patients’ cumulative health outcomes using their individual characteristics and medical history at each encounter. The RL recommendations were evaluated on an independent subset of patients. Results: The single-outcome optimization RL algorithms, RL-glycemia, RL-blood pressure, and RL-CVD, recommended consistent prescriptions as that observed by clinicians in 86.1%, 82.9%, and 98.4% of the encounters, respectively. For patient encounters in which the RL recommendations differed from the clinician prescriptions, significantly fewer encounters showed uncontrolled glycemia (A1c >8% in 35% of encounters), uncontrolled hypertension (blood pressure > 140mmHg in 16% of encounters), and high CVD risk (risk >20% in 25% of encounters) under RL algorithms compared with those observed under clinicians (43%, 27%, and 31% of encounters, respectively; all p-values <= 0.001). Conclusion: A personalized RL prescriptive framework for type 2 diabetes yielded high concordance with clinicians’ prescriptions, and substantial improvements in glycemia, blood pressure, and CVD risk outcomes.