Chapter 6 Transfer Entropy

6.1 Introduction

6.2 Nonlinear G-Causality

To compute the nonlinear G-Causality, we use the concept of Transfer Entropy. Since its introduction (Schreiber 2000), Transfer Entropy has been recognized as an important tool in the analysis of causal relationships in nonlinear systems (Hlavackovaschindler et al. 2007). It detects directional and dynamical information (Montalto 2014) while not assuming any particular functional form to describe interactions among systems.

The Transfer Entropy can be defined as the difference between the conditional entropies: \[\begin{equation} TE\left(X \rightarrow Y\right \vert Z) = H\left(Y^F\middle\vert Y^P,Z^P\right) - H\left(Y^F\middle\vert X^P, Y^P,Z^P\right), \tag{6.1} \end{equation}\] which can be rewritten as a sum of Shannon entropies: \[\begin{align} TE\left(X \rightarrow Y\right) = H\left(Y^P, X^P\right) - H\left(Y^F, Y^P, X^P\right) + H\left(Y^F, Y^P\right) - H\left(Y^P\right), \end{align}\]

where \(Y^F\) is a forward time-shifted version of \(Y\) at lag \(\Delta t\) relatively to the past time-series \(X^P\), \(Y^P\) and \(Z^P\). Within this framework we say that \(X\) does not G-cause \(Y\) relative to side information \(Z\) if and only if \(H\left(Y^F\middle\vert Y^P,Z^P \right) = H\left(Y^F\middle\vert X^P, Y^P,Z^P\right)\), i.e., when \(TE\left(X \rightarrow Y,Z^P\right) = 0\).

Empirically, we reject this null hypothesis of causality if the Transfer Entropy from \(X\) to \(Y\) is significantly higher than the shuffled version of the original data.

For this we estimate 400 replicates of \(TE(X_{Shuffled} \rightarrow Y)\), where \(X_{Shuffled}\) is a random permutation of \(X\) relatively to \(Y\). We compute the randomized Transfer Entropy at each permutation for each time-shift (\(\Delta t\)) from 1 to 10 days. We then calculated the frequency at which the observed Transfer Entropy was equal or more extreme than the randomized Transfer Entropy. The statistical significance was assessed using p-value \(< 0.05\) after Bonferroni correction.

6.4 Net Information Flow

Transfer-entropy is an asymmetric measure, i.e., \(T_{X \rightarrow Y} \neq T_{Y \rightarrow X}\), and it thus allows the quantification of the directional coupling between systems. The Net Information Flow is defined as \[\begin{equation} \widehat{TE}_{X \rightarrow Y} = TE_{X \rightarrow Y} - TE_{Y \rightarrow X}\;. \end{equation}\]

One can interpret this quantity as a measure of the dominant direction of the information flow. In other words, a positive result indicates a dominant information flow from \(X\) to \(Y\) compared to the other direction or, similarly, it indicates which system provides more predictive information about the other system (Michalowicz, Nichols, and Bucholtz 2013).

In the next sections we will provide empirical examples that show that Transfer Entropy can capture information flow in both linear and nonlinear systems.

6.5 Empirical Experiment: Information Flow on Simulated Systems

In this section, we construct simulated systems and test the nonlinear and linear formulations of the net information flow. We show that only the nonlinear formulation of net information flow is able to capture the nonlinear relationships in the simulated systems.

For the nonlinear case, we compute Transfer Entropy as defined in Eq. (6.1). Conversely, to estimate the linear version of the Net Information Flow, we computed the Transfer Entropy using Eq. (6.3), i.e., by estimating linear G-causality Eq. (6.2) under a linear-VAR framework.

6.6 Empirical Experiment: Information Flow on Global Markets


Schreiber, Thomas. 2000. “Measuring Information Transfer.” Phys. Rev. Lett. 85 (2). American Physical Society: 461–64. doi:10.1103/PhysRevLett.85.461.

Hlavackovaschindler, K., M. Palus, M. Vejmelka, and J. Bhattacharya. 2007. “Causality Detection Based on Information-Theoretic Approaches in Time Series Analysis.” Physics Reports 441 (1): 1–46. doi:10.1016/j.physrep.2006.12.004.

Montalto, Luca AND Marinazzo, Alessandro AND Faes. 2014. “MuTE: A Matlab Toolbox to Compare Established and Novel Estimators of the Multivariate Transfer Entropy.” PLoS ONE 9 (10). Public Library of Science: e109462. doi:10.1371/journal.pone.0109462.

Barnett, Lionel, Adam B. Barrett, and Anil K. Seth. 2009. “Granger Causality and Transfer Entropy Are Equivalent for Gaussian Variables.” Phys. Rev. Lett. 103 (23). American Physical Society: 238701. doi:10.1103/PhysRevLett.103.238701.

Michalowicz, Joseph Victor, Jonathan M. Nichols, and Frank Bucholtz. 2013. Handbook of Differential Entropy. Chapman & Hall/CRC.