# Chapter 6 Transfer Entropy

## 6.1 Introduction

## 6.2 Nonlinear G-Causality

To compute the nonlinear G-Causality, we use the concept of Transfer Entropy. Since its introduction (Schreiber 2000), Transfer Entropy has been recognized as an important tool in the analysis of causal relationships in nonlinear systems (Hlavackovaschindler et al. 2007). It detects directional and dynamical information (Montalto 2014) while not assuming any particular functional form to describe interactions among systems.

The Transfer Entropy can be defined as the difference between the conditional entropies: \[\begin{equation} TE\left(X \rightarrow Y\right \vert Z) = H\left(Y^F\middle\vert Y^P,Z^P\right) - H\left(Y^F\middle\vert X^P, Y^P,Z^P\right), \tag{6.1} \end{equation}\] which can be rewritten as a sum of Shannon entropies: \[\begin{align} TE\left(X \rightarrow Y\right) = H\left(Y^P, X^P\right) - H\left(Y^F, Y^P, X^P\right) + H\left(Y^F, Y^P\right) - H\left(Y^P\right), \end{align}\]where \(Y^F\) is a forward time-shifted version of \(Y\) at lag \(\Delta t\) relatively to the past time-series \(X^P\), \(Y^P\) and \(Z^P\). Within this framework we say that \(X\) does not G-cause \(Y\) relative to side information \(Z\) if and only if \(H\left(Y^F\middle\vert Y^P,Z^P \right) = H\left(Y^F\middle\vert X^P, Y^P,Z^P\right)\), i.e., when \(TE\left(X \rightarrow Y,Z^P\right) = 0\).

Empirically, we reject this null hypothesis of causality if the Transfer Entropy from \(X\) to \(Y\) is significantly higher than the shuffled version of the original data.

For this we estimate 400 replicates of \(TE(X_{Shuffled} \rightarrow Y)\), where \(X_{Shuffled}\) is a random permutation of \(X\) relatively to \(Y\). We compute the randomized Transfer Entropy at each permutation for each time-shift (\(\Delta t\)) from 1 to 10 days. We then calculated the frequency at which the observed Transfer Entropy was equal or more extreme than the randomized Transfer Entropy. The statistical significance was assessed using p-value \(< 0.05\) after Bonferroni correction.

## 6.3 The Link Between Linear Granger-causality and Transfer Entropy

It has been shown (L. Barnett, Barrett, and Seth 2009) that linear G-causality and Transfer Entropy are equivalent if all processes are jointly Gaussian. In particular, by assuming the standard measure (\(l2\)-norm loss function) of linear G-causality for the bivariate case as follows (see Section 3.3 for more details on linear-Granger causality):

\[\begin{equation} GC_{X \rightarrow Y} = \log\left( \frac{var(\epsilon_t)}{var( \widehat{\epsilon}_t)} \right), \tag{6.2} \end{equation}\]the following can be proved (L. Barnett, Barrett, and Seth 2009):

\[\begin{align} TE_{X \rightarrow Y} = GC_{X \rightarrow Y}/2. \tag{6.3} \end{align}\]This result provides a direct mapping between the Transfer Entropy and the linear G-causality implemented in the standard VAR framework. Hence, it is possible to estimate the TE both in its general form and with its equivalent form for linear G-causality.

## 6.4 Net Information Flow

Transfer-entropy is an asymmetric measure, i.e., \(T_{X \rightarrow Y} \neq T_{Y \rightarrow X}\), and it thus allows the quantification of the directional coupling between systems. The Net Information Flow is defined as \[\begin{equation} \widehat{TE}_{X \rightarrow Y} = TE_{X \rightarrow Y} - TE_{Y \rightarrow X}\;. \end{equation}\]One can interpret this quantity as a measure of the dominant direction of the information flow. In other words, a positive result indicates a dominant information flow from \(X\) to \(Y\) compared to the other direction or, similarly, it indicates which system provides more predictive information about the other system (Michalowicz, Nichols, and Bucholtz 2013).

In the next sections we will provide empirical examples that show that Transfer Entropy can capture information flow in both linear and nonlinear systems.

## 6.5 Empirical Experiment: Information Flow on Simulated Systems

In this section, we construct simulated systems and test the nonlinear and linear formulations of the net information flow. We show that only the nonlinear formulation of net information flow is able to capture the nonlinear relationships in the simulated systems.

For the nonlinear case, we compute Transfer Entropy as defined in Eq. (6.1). Conversely, to estimate the linear version of the Net Information Flow, we computed the Transfer Entropy using Eq. (6.3), i.e., by estimating linear G-causality Eq. (6.2) under a linear-VAR framework.

## 6.6 Empirical Experiment: Information Flow on Global Markets

### References

Schreiber, Thomas. 2000. “Measuring Information Transfer.” *Phys. Rev. Lett.* 85 (2). American Physical Society: 461–64. doi:10.1103/PhysRevLett.85.461.

Hlavackovaschindler, K., M. Palus, M. Vejmelka, and J. Bhattacharya. 2007. “Causality Detection Based on Information-Theoretic Approaches in Time Series Analysis.” *Physics Reports* 441 (1): 1–46. doi:10.1016/j.physrep.2006.12.004.

Montalto, Luca AND Marinazzo, Alessandro AND Faes. 2014. “MuTE: A Matlab Toolbox to Compare Established and Novel Estimators of the Multivariate Transfer Entropy.” *PLoS ONE* 9 (10). Public Library of Science: e109462. doi:10.1371/journal.pone.0109462.

Barnett, Lionel, Adam B. Barrett, and Anil K. Seth. 2009. “Granger Causality and Transfer Entropy Are Equivalent for Gaussian Variables.” *Phys. Rev. Lett.* 103 (23). American Physical Society: 238701. doi:10.1103/PhysRevLett.103.238701.

Michalowicz, Joseph Victor, Jonathan M. Nichols, and Frank Bucholtz. 2013. *Handbook of Differential Entropy*. Chapman & Hall/CRC.