ESFM-MoE: Climate-semantic routing for Earth System Foundation Model (ESFM)

Weather foundation models are increasingly expected to operate under heterogeneous and imperfect observation settings while remaining computationally scalable. Building on the Earth System Foundation Model (ESFM) setting for heterogeneous data integration, we explore how Mixture-of-Experts (MoE) can support robust and efficient learning in multi-modal weather foundation models.

We introduce ESFM-MoE, an exploratory direction that combines conditional computation with climate-semantic routing, a routing principle that encourages expert specialization aligned with meaningful geophysical structure, rather than treating expert selection as a purely generic scaling mechanism. The motivation is that Earth-system data exhibit strong spatial organization, regime-like variability, and modality-dependent uncertainties; MoE offers a natural way to allocate capacity adaptively and promote structured specialization under such heterogeneity.

In this work, we discuss the design space and practical considerations of integrating MoE into Earth-system foundation models, focusing on how routing objectives and inductive biases can shape expert behavior and improve utilization. We highlight potential benefits for robustness to missing observations, scalable training and inference, and outline promising directions for climate-aware expert specialization in next-generation weather foundation models.