Novel soil health assessment framework for legume-based rotation farmland by interpretable machine learning with causal inference

Accurate and robust soil health assessment is essential for sustaining legume-based rotation systems and informing their optimized management. To address the limitations of conventional methods in capturing management-induced variations, we developed an innovative framework grounded in the theoretical hypothesis that soil health reflects soil’s capacity to maximize production stability while minimizing input requirements. This framework synergistically integrates interpretable machine learning with causal inference and network analysis (CI-SHAP-NA), implementing a systematic workflow encompassing indicator selection, quantitative scoring, and multidimensional integration. Our framework was systematically implemented to assess soil health across diverse legume-based rotation systems in China. The results showed that CI-SHAP-NA identified a parsimonious yet highly informative set of indicators (soil organic carbon, available iron, and cellobiohydrolase) demonstrating superior explanatory power for critical soil ecological processes. The derived soil health index (SHI) by the CI-SHAP-NA framework demonstrated enhanced discriminative capacity (SHI range: 0.01−0.92) and strong concordance (R2 = 0.80) with conventional total dataset assessment while maintaining significant predictive validity for crop productivity (Pearson’s r = 0.21, p < 0.001). It consistently outperformed PCA and NA methods in both explanatory power and fairness comparisons. The selected indicators proved robust and non-redundant, as substituting any indicator significantly reduced the correlation and sensitivity of SHI. Furthermore, CI-SHAP-NA demonstrated strong transferability, showing a stronger correlation with yield (r = 0.25, p < 0.001) on internally established independent sites than PCA and NA. This framework successfully resolved previously obscured soil health gradients between contrasting management systems, with paddy-legume rotations consistently outperforming their dryland counterparts − a differentiation rigorously validated against traditional benchmarks. These findings collectively establish the CI-SHAP-NA framework as a transformative tool for soil health assessment, offering substantial advantages over conventional approaches in terms of analytical robustness, ecological relevance, and practical utility. Future research should aim to incorporate multi-functional indicators as well as evaluate the framework’s performance across varied agroecosystems.