Knowledge-encoded deep fusion for yield estimation under extreme climate stress

Accurately modeling the impacts of climate stress on crop growth and yield is crucial for ensuring food security. Data-driven models are increasingly utilized for yield estimation because they can learn effective crop growth features from vast amounts of remote sensing and meteorological data. However, extreme climate stress conditions have few yield labels available for these models to modeling the interaction in crop responses. The response of crops to extreme climate stress often exhibits varied delays which are captured in remote sensing observations. In this study, we explicitly encode the time lag effect quantified by remote sensing and climate stress indicators into a two-stream fusion framework for estimating crop yield under extreme climate stress. Each stream employs a pyramid structure that progressively aggregates remote sensing and climate time series into feature embeddings. A time-lag-encoded cross attention mechanism fuses feature embeddings between the two streams, while phenology-sensitivity-guided linear attention is applied on top of the pyramid structures for processing ultimate time-lag encoded features. The proposed model is evaluated across nine Midwestern states within the US Corn Belt at the county level from 2006 to 2012, simulating climate stress situations with fewer samples. End-of-season results demonstrate that the knowledge-encoded two-stream model (RMSE=1.17 Mg ha^-1) outperforms both the feature-stacking-based two-stream model (RMSE=1.43 Mg ha^-1) and random forest (RMSE=1.68 Mg ha^-1) under extreme climate stress. The improved estimation performance indicates that knowledge-encoded data fusion is more effective than merely stacking multi-source input data. In-season results reveal that our model proficiently captures extreme events and effectively predicts yield 8 weeks in advance. The time-lag knowledge could be extended to other forms of climate stress. Also, cross attention enables integration with additional data sources to enhance the interaction modeling of complex biomass accumulation and yield formation.