Driven by the key challenges of cell therapy manufacturing, we create a probabilistic knowledge graph (KG) characterizing the risk- and science-based understanding of biomanufacturing process mechanisms. Then, we introduce KG-based Bayesian RL to guide optimal, robust, and interpretable dynamic decision making.