Chapter 2 Stationarity, Spectral Theorem, Ergodic Theorem(Lecture on 01/07/2021)
Definition 2.1 (Uncorrelated increaments) We say that a stochastic process X has uncorrelated (orthogonal) increaments if for any ti<tj<tk<tl∈T, Cov(tj−ti,tl−tk)=0.
Definition 2.2 (Cross-covariance Function) A useful function for the study of coevolution of two stochastic processes, say X and Y, defined on the same probability space and with the same index set, is the cross-covariance function, defined as CX,Y(ti,tj)=Cov(Xti,Ytj)=E(XtiYtj)−E(Xti)E(Ytj) for ti,tj∈T. The cross-covariance function measures how correlated are the two processes.
If X and Y are completely uncorrelated, then CX,Y(ti,tj)=0,∀ti,tj∈T.Why is f.d.d.s. important?
One of the major usage of the stochastic process is to serve as the prior for the unknown function f(x) in a regression problem. In real application, we only have finite number of data. Therefore, what we really need is a prior on the finite dimensional random vector (f(x1),⋯,f(xn)), which is just the f.d.d.s. of the stochastic process.
Stationarity is a simplification on f.d.d.s.. The theory and method for a stochastic process are considerably simplified under the assumption of (either strong or weak) stationarity, which imposes certain structures on the finite dimensional distributions.
- Strong stationarity implies weak stationarity. The converse is not true in general. While for the Gaussian process, the opposite direction is also true. For Gaussian process the f.d.d.s. are all multivariate normal and we can characterize f.d.d.s. completely by the mean function and covariance function.
From the theory of Fourier analysis, any function f:R→R with certain properties (including periodicity and continuity) has a unique Fourier representation. f(x)=0.5a0+∞∑n=1(ancos(nx)+bnsin(nx)) which express f as a sum of varying proportions of regular oscillation.
Fourier representation transforms the uncountably infinite dimensional problem of estimating unknown function f(x) to a countably infinite dimensional problem of estimating an and bn.
We will express the covariance function of a stationary process using the Fourier transformation. Before doing that, let us fix some notations. By weak stationarity we have E(Xt)=μ,∀t and Cov(Xti,Xtj)=c(t), where t=|ti−tj|. Then Var(Xti)=c(0)=σ2 for all t∈R. With out loss of generality, we assume μ=0 and σ2=1. Under this assumption, the autocorrelation function and the autocovariance function coincide, i.e. Corr(Xti,Xtj)=c(t).
We are going to express (using the idea of Fourier transformation) the covariance function as a characteristic function of some random variable. That is c(t)=∫exp(itx)dF(x)=∫exp(itx)f(x)dx=E(exp(itx)) for some distribution function F. This distribution is called the spectral distribution of the stochastic process. If F has a density, then that is called the spectral density.
Theorem 2.1 (Bochner Theorem in Stochastic Process) A function ϕ is a characteristic function of a random variable if
ϕ(0)=1, |ϕ(t)|≤1 for all t.
ϕ is uniformly continuous on R.
ϕ is a non-negative function. i.e. for all t1,⋯,tn, and constant z1,⋯,zn, ∑ki=1∑kj=1zizjϕ(ti−tj)≥0.
Thus, only (2) needs to be assumed for c(t) to have a spectral density. From (2.3), using the result from Fourier analysis, we can express the spectral density f as f(x)=12π∫∞−∞exp(−itx)c(t)dt
The spectral density and the covariance function has a one-to-one correspondence.The above representation has been mainly defined when T=R. There are some special case when the index is not R.
- If the index point is multivariate, i.e. T=Rd, then c(t)=∫x1⋯∫xtexp(−itx)f(x)dx
When we are concerned about a stochastic process with a discrete index set, that is X={X(n,ω):n∈Z,ω∈Ω}. We are interested in Sn=∑nj=1Xj. When n is large and Xj are i.i.d. it is simple to study Sn, we then have CLT or LLN to give the asymptotic behavior of Sn. However, for a stochastic process, these assumptions do not hold. Still we have some results for stochastic process, these are known as the ergodicity theorem for stochastic process.
Definition 2.5 (Brownian Motion) A stochastic process B=\{B(t,\omega):t\geq 0,\omega\in\Omega\} is called a Brownian motion, if it satisfies
B_0=0;
B_t-B_s\sim N(0,t-s), \forall t>s\geq 0.
B_t-B_s is independent of B_s.
- The function t\to B_t is continuous.
Brownian motion is not a stationary stochastic process, because from (2) and (3) in Definition 2.5, Cov(B_t,B_s)=\min(t,s)