Evaluating LLM-assisted elicitation of conditional probabilities in causal networks for the assessment of nature-based solutions across the water-energy-ecosystem nexus

There is growing awareness among urban communities that nature-based solutions (NbS) can actively mitigate climate change impacts, while securing ecosystem services. However, assessing the full potential of NbS to provide these multifaceted benefits remains a challenge, as NbS function at the intersection of physical and social processes that occur at different spatial and temporal scales (what we call herein as the Water-Energy-Ecosystem, WEE, nexus).

This coupling of physical and social dynamics is naturally represented as a network of relationships, making causal probabilistic networks (CPNs) suitable for encoding causal structures and propagating uncertainty. In practice, however, nexus approaches often face scarce and heterogeneous data, necessitating expert knowledge to parameterise the conditional probabilities of CPNs, a process that is time-intensive and difficult to scale.

Large language models (LLMs) have been recently shown to complement expert elicitation of conditional probabilities, alleviating the resources required for the parameterisation of CPNs. Nonetheless, open questions remain as to whether (a) LLMs can support expert elicitation in complex, interdisciplinary domains in a transparent and reproducible manner, and (b) retrieval-augmented generation (RAG) improves elicitation quality by grounding probability judgments in problem-specific evidence.

To answer those questions, this work proposes a structured validation framework for LLM-assisted elicitation. Validation targeted model utility for impact assessment using: (i) probabilistic coherence (bounds, monotonicity expectations, leak dominance, and required interactions), (ii) scenario-based stress-testing to verify expected risk ordering, and (iii) repeatability analysis across repeated LLM elicitations to quantify stability of CPN parameterisations. Three elicitation modes were considered: (i) human experts, (ii) LLM-only (proprietary and open-source LLMs were used), and (iii) RAG-LLM using pre-trained, open-source LLMs and a curated evidence pack retrieved and cited during elicitation.

The framework was tested using a dynamic CPN, which delineates the effects of urban blue–green interventions that integrate stormwater source control and greening strategies on mitigating runoff, enhancing infiltration, and regulating the microclimate. To reduce dimensionality while retaining mechanistic detail, variables were discretized into binary states and parameterized via Noisy-OR gates, eliciting only single-cause activation probabilities and leak terms using a standardized questionnaire that also captures uncertainty intervals and confidence ratings.

The evaluation of LLM-only and RAG- enhanced elicitation suggests that LLMs can offer a viable initial parameterisation for CPNs, particularly in contexts where data are scarce. LLM‑generated parameter sets satisfied coherence criteria and exhibited low variance across repeated elicitation runs, while stress‑testing confirmed that the resulting networks produce plausible risk orderings. RAG‑enhanced open‑source models achieved comparable performance to proprietary counterparts while offering greater traceability. Nevertheless, disagreements with the expert-derived elicitation persist at the parameter level. Miscalculated parameters propagated downstream effects during part of the stress-testing with climatic and asset-degradation scenarios, underscoring the need for expert supervision.

Equally importantly, however, this work provides a validation framework that functions as a structured practical benchmark for integrating LLM-assisted probabilistic elicitation into complex nexus models for the assessment of NbS when observational data are limited or unavailable.