Multimodal Deep Learning for Crop Yield Prediction Using Satellite Imagery and Weather Data

I. INTRODUCTION

District-level crop yield estimates traditionally rely on either weather-based agrometeorological models or vegetation-index trends in isolation, whereas the physiological relationship between crop stress and yield depends on both simultaneously, motivating a fused multimodal approach.

II. METHODOLOGY

Sentinel-2 NDVI time series at 10-day composite intervals were paired with district-level daily rainfall and temperature records across eight growing seasons and 42 wheat-growing districts, and processed through a dual-branch network combining a 1D CNN over the NDVI sequence with an LSTM over the weather sequence, fused via a concatenation layer feeding a final regression head predicting end-of-season yield.

III. RESULTS AND EVALUATION

The fused multimodal model achieved an RMSE of 187 kg per hectare on held-out districts, a 24 percent reduction from a weather-only LSTM baseline (246 kg/ha) and a 17 percent reduction from an NDVI-only CNN baseline (225 kg/ha), with the largest gains observed in districts experiencing mid-season rainfall anomalies where vegetation response lagged weather signals.

IV. CONCLUSION

Fusing satellite vegetation indices with weather time series meaningfully improves district-level yield forecasting over single-modality baselines, supporting earlier and more reliable procurement planning. Future work will incorporate soil-moisture satellite products as a third modality.

V. REFERENCES