Tabular Reinforcement learning for Robust, Explainable CropRotation Policies Matching Deep Reinforcement LearningPerformance

Abstract

Digital Twins are becoming an increasingly researched area in agriculture due to the pressure on food security caused by growing population numbers and climate change. They provide a necessary push towards more efficient and sustainable agricultural methods to secure and increase crop yields.
Digital Twins often use Machine Learning, and more recently, deep learning methods in their architecture to process data and predict future outcomes based on input data. However, concerns about the trustworthiness of the output from deep learning models persist due to the lack of clarity regarding the reasoning behind their outputs.

In our work, we have developed crop rotation policies using explainable tabular reinforcement learning techniques. We have compared these policies to those generated by a deep Q-learning approach, using both five-step and seven-step rotations. The aim of the rotations is to maximise crop yields while maintaining a healthy nitrogen level in the soil and adhering to established planting rules. Crop yields may vary due to external factors such as weather patterns, so perturbations were added to the reward signal to account for these influences. The deployed explainable tabular reinforcement learning methods perform similarly to the deep Q-learning approach in terms of collected reward when the rewards are not perturbed. However, in the perturbed reward setting, robust tabular reinforcement learning methods outperform the deep learning approach while maintaining interpretable policies. By consulting with farmers and crop rotation experts, we demonstrate that the derived policies are reasonable and that the use of interpretable reinforcement learning has increased confidence in the resulting policies, thereby increasing the likelihood that farmers will adopt the suggested policies.

Keywords: Digital Twin, Reinforcement Learning, Explainable AI, Agriculture, Crop Rotation Planning, Climate Change, Food Security