Evaluation of Shannon Entropy-based Information transfer in nonlinear systems

We present a general theory for computing and estimating Shannon entropy-based information transfer in nonlinear stochastic systems driven by deterministic forcings and additive and/or multiplicative noises, by extending the Liang-Kleeman framework of causality inference to nonlinear cases. The method presents effective and computable formulas of the rates of information transfer between sets of causal and consequential system variables, relying on the evaluation of conditional expectations of the deterministic and stochastic forcings (Causal Sensitivity Method: CSM). The CSM can work with a) ensemble model runs, b) system time series in ergodic conditions and c) time series without a priori knowledge of model equations. The CSM also allows to express the information transfer parcels, which are attributable either to one-to-one interactions or to synergies across groups of variables and assess where the information is more relevant in the state space. The CSM is tested in two proof-of-concept low-order models: 1) a nonlinear model derived from a potential function and 2) the classical chaotic Lorenz model, both forced by additive and/or multiplicative noises. The CSM is also tested with a nonlinear regression model of the ice-cover time evolution, forced by radiation. The CSM estimation is much more robust and efficient than methods using the stochastic model’s full probability density function and its derivatives, whose estimation is rather unreliable in case of short data availability. The analysis also demonstrates that the CSM estimation is computationally cheap in the different experiments, providing evidence of the possibilities and generalizations offered by the method, thus opening new perspectives on real-world applications. This work was funded by the Portuguese Fundação para a Ciência e a Tecnologia (FCT) I.P./MCTES through national funds (PIDDAC) – UIDB/50019/2020(https://doi.org/10.54499/UIDB/50019/2020),UIDP/50019/2020(https://doi.org/10.54499/UIDP/50019/2020) and LA/P/0068/2020 (https://doi.org/10.54499/LA/P/0068/2020) and the project  JPIOCEANS/0001/2019 (ROADMAP).