When Calibration Metrics Choose Your Model: Investigating Model Selection Choices for Different Modelling Purposes

Hydrological model structures are often selected based on legacy considerations—such as habit, practicality, or experience—rather than whether they are fit for a specific modelling purpose. This is problematic, as model structure alone can substantially influence modelling results and hence outcomes for e.g. design flow assessment. Automatic Model Structure Identification (AMSI) offers a way to address this issue by framing model choice as an optimization problem. AMSI combines the modular modelling framework Raven with mixed-integer calibration algorithms (DDS/PA-DDS), allowing the simultaneous optimization of model structural choices and parameter values with respect to user-defined objectives.

Here, we apply AMSI to explore a hypothesis space of more than 13,500 conceptual model structures with zero to 12 parameters per model. We test 14 calibration routines, including six single-metric, four multi-metric, and four multi-objective formulations, designed to reflect different modelling purposes that target flood, drought, and water-resources management assessment. Model evaluation uses metrics and hydrological signatures associated with different aspects of the flow regime to assess model suitability across these different purposes. All experiments are conducted on a test catchment located on the Eastern Coast of the US.

Each calibration routine is performed 50 times, yielding a set of preferred model structures. These are analyzed regarding their individual processes and equations, as well as model performance across purpose-specific metric and flow signature groups. Results show that model structural preferences vary with modelling purpose, favouring different process descriptions for different intended applications of the model. Within the tested hypothesis space, identifying suitable model structures is easiest for water-resources management (average flow behaviour), followed by flood (peak flow) modelling, and most challenging for drought (low flow) modelling. Multi-metric and multi-objective calibrations provide more balanced representations than single-metric approaches, with multi-objective calibration revealing explicit trade-offs between structural choices and multi-metric calibration reducing structural equifinality.