An Explainable AI-Driven Feature Reduction Framework for Enhanced Agricultural Yield Prediction

Agricultural yield prediction plays a crucial role in food security and economic planning, yet existing models often struggle with the complexity and high dimensionality of agricultural data. This study presents a framework that combines explainable artificial intelligence (XAI) with feature reduction methodology to enhance the accuracy and efficiency of rice yield prediction. Our approach addresses the dual challenges of model interpretability and computational efficiency while maintaining high prediction accuracy.

The framework begins with a systematic development of prediction models utilizing advanced machine learning (ML) and deep learning (DL) techniques. We implemented comprehensive pre-processing steps, including data normalization, feature engineering, and missing value handling, to ensure data quality. Our evaluation encompassed various models, including Random Forest, Gradient Boosting Machines, Convolutional Neural Networks (CNNs), and Long Short-Term Memory (LSTM) networks with attention mechanisms. To optimize model performance, we employed hyperparameter tuning through grid search, effectively mitigating issues of overfitting and underfitting.

A notable innovation of our framework is the incorporation of SHapley Additive exPlanations (SHAP), enabling transparent insights into the model's decision-making process. Leveraging this XAI approach, we introduced a novel feature reduction methodology that systematically identifies and removes negatively contributing features while maintaining model accuracy. Our analysis of a multivariate dataset which is a public dataset from rice fields in the an Giang province of the Mekong Delta, Vietnam, required the integration of diverse satellite datasets, including optical data from Landsat and radar data from Sentinel-1. This revealed distinct patterns of feature influence on yield prediction, facilitating the optimization of the feature set for maximum effectiveness. Key radar polarization bands, VV (Vertical-Vertical) and VH (Vertical-Horizontal), provided crucial surface backscatter data, capturing information on crop structure, growth stages, and post-harvest soil conditions. Notably, the feature min_vh consistently emerged as the most significant predictor.

The implementation of our feature reduction strategy resulted in significant improvements in both model performance and computational efficiency. By removing 15-20 number of identified negatively contributing features, we achieved approximately 3-5% improvement in prediction accuracy while substantially reducing the computational overhead and model training time. This enhancement in efficiency did not compromise the model's interpretability, demonstrating the robust nature of our framework.

Our methodology represents a significant advancement in agricultural modeling by successfully addressing the challenges of high-dimensional data while maintaining model interpretability. The framework's ability to identify and eliminate non-contributing features while improving prediction accuracy demonstrates its potential for wide-scale application in agricultural yield prediction. Furthermore, the reduced computational requirements make it a practical solution for real-world applications where computational resources may be limited.

These results validate the effectiveness of our integrated approach in handling complex agricultural data while providing actionable insights for yield prediction. The framework offers a scalable, interpretable, and computationally efficient solution that can be adapted for various agricultural prediction tasks, potentially transforming how we approach agricultural yield forecasting.