Making Scientific Data Accessible with LLMs while Preserving Authority and Reliability: Lessons Learned from Building Production Grade Agentic Systems

In this talk we will outline our learnings from building two production grade agentic systems for data discovery, retrieval and analysis. For both applications trustworthiness, reliability and reproducibility were key criteria that we took into account from the start.

Producing insights and surfacing data in a repeatable and transparent way is not simply a nice-to-have feature, it is indispensable for adoption. In practice, even if the answer from a chatbot is scientifically correct, users will only rely on the outputs for decision making if they trust the system. Users by now have enough experience with chatbots to know that LLMs tend to exaggerate and hallucinate. This has created a healthy skepticism with regard to output from agentic systems. In scientific domains, it is therefore not sufficient to guarantee a correct result, it also has to be presented in a way that is transparent and reproducible. Despite all advances in the LLM domain, this continues to be a challenge in building agentic systems.

We tackled this challenge by adopting a series of techniques that we will illustrate with concrete examples from building production ready agentic applications. One of the main principles is that we rely on the LLM mainly for orchestration of well known tools instead of relying on the generative capabilities of the models. For analysis we built the systems in a way that the transformations on the original data are reproducible. One technique is to use LLMs to write code for analyis that can be stored and used to reproduce the results. This allows end-to-end tracing of where the data is coming from and how it was transformed to produce insights through statistics and charts. We will also mention our approach to evaluation of the agent and share insights from early user research already performed for these systems.

We will illustrate these principles with concrete examples from the two agentic systems outlined below. 

The Destination Earth Digital Assistant, built in collaboration with ECMWF, provides general information about Destination Earth and helps users to discover and retrieve data from the DestinE Digital Twins.

Global Nature Watch, built in collaboration with WRI and the Land and Carbon Lab, provides governments, companies, and communities with trusted, open data and intelligence-driven insights on land conditions and land-use change to enable efficient and evidence-based decisions for nature protection and restoration.

GNW is publicly accessible today and the DestinE Assistant is also planned to be launched publicly before EGU26.