Exploring the Opportunities and Challenges of Using Large Language Models to Represent Institutional Agency in Land Use Modelling

Institutional agencies play a crucial role in land use change, but modelling their decision-making processes is challenging due to the complexity of the environment they operate within and the bounded rationality of human organizations. Large Language Models (LLMs) offer a novel approach to simulating human decisions. This paper aims to investigate the challenges and opportunities that LLMs bring to land use change modelling by integrating LLM-powered institutional agents with the CRAFTY land use model, in which land users produce a range of ecosystem services. The study develops a structured prompt development approach for coupling LLM-powered agents with existing large-scale simulations. Four types of LLM-powered agents are examined, which use taxes to steer meat production toward a prescribed policy goal. The agents provide reasoning and policy action output in each simulation iteration. The study also uses a technique called quasi-multi-agent to simulate multiple roles involved in the policy processes. Unlike authentic multi-agent simulation, the LLM-powered quasi-multi-agent leverages the LLM's ability to generate contextually coherent text and allows the agents to work as a scriptwriter who composes conversations between different roles. This approach conserves computational resources and has the potential to manage conversational dynamics in policy discussions. The efficacy of these agents is benchmarked against two baseline scenarios: one without any policy intervention and another implementing optimal policy actions determined through a genetic algorithm.

The findings show that while LLM-powered agents perform better than the non-intervention scenario, they fall short of the performance achieved by optimal policy actions. However, LLM-powered agents demonstrate human-like decision-making, marked by policy consistency and transparent reasoning. The agents also generate real-world policymaking strategies, including incrementalism, considering delayed policy influence, proactive policy adjustments, and balancing multiple stakeholder interests. Agents equipped with experiential learning capabilities excel in achieving policy objectives through progressive policy actions. The order of reasoning and proposed policy actions in the prompts has a notable effect on the agents' performance. The research points to both promising opportunities and significant challenges in integrating LLMs into large-scale land-use simulations. The opportunities include exploring naturalistic institutional decision-making and its impact on land use change, using LLM's information retrieval to handle massive institutional documents, modelling institutional networks, and human-AI cooperation. However, challenges mainly lie in the scalability and reliability of LLMs due to the dependence on LLM providers, the paradox of pursuing realistic institutional behaviours versus abstraction and simplification in existing models, and the effectiveness and efficiency in scrutinizing massive textual output, detecting illogical content in prompts, and inaccurate formatting.