Toward intelligent acquisition policies: Reinforcement learning for causal multi-objective Bayesian optimization

Traditional Bayesian optimization relies on hand-crafted acquisition functions that treat variables uniformly, whereas multi-objective systems contain exploitable causal structure. Although recent advances established foundations for causal Bayesian optimization and multi-objective reinforcement learning independently, no existing approach combines these paradigms. Static acquisition policies cannot adapt to causal dependencies and competing objective trade-offs. Here, we present RL-CMBO (Reinforcement Learning for Causal Multi-Objective Bayesian Optimization), a reinforcement learning framework that learns intelligent acquisition policies through experience. Our approach shifts from hand-crafted functions to learned policies capable of discovering optimal intervention strategies. The framework integrates: (1) a meta-learning architecture adapting to task-specific causal structures, (2) state representation encoding Pareto front features and causal graph topology, and (3) constrained action space over Possibly-Optimal Minimal Intervention Sets (POMIS) ensuring causal identifiability. Our reward engineering balances hypervolume improvement, intervention costs, and Pareto front diversity–thus addressing competing objectives in causal systems. Experimental evaluation demonstrates that RL-CMBO outperforms existing methods across synthetic and real-world benchmarks, achieving improved sample efficiency while discovering intervention patterns that static acquisition functions fail to identify. This work establishes the first unified framework combining reinforcement learning, causal reasoning, and multi-objective optimization.