Chapter 19: covariance matrix

19.1 vector direct product

  • scalar = rank-0 tensor
  • vector = rank-1 tensor
  • matrix = rank-2 tensor
  • vector direct product = rank-1 tensor times rank-1 tensor equals rank-2 tensor: increasing rank
  • vector inner product = rank-1 tensor times rank-1 tensor equals rank-0 tensor: decreasing rank

scalar = rank-0 tensor

vector = rank-1 tensor

matrix = rank-2 tensor

19.1.1 vector direct product: increasing rank

vector direct product = rank-1 tensor times rank-1 tensor equals rank-2 tensor: increasing rank

UV=UV=UiVj=(U1U2U3)(V1V2V3)=(U1U2U3)(V1V2V3)=(U1V1U1V2U1V3U2V1U2V2U2V3U3V1U3V2U3V3)=(U1VU2VU3V)=(UV1UV2UV3)

19.1.2 vector inner product: decreasing rank

vector inner product = rank-1 tensor times rank-1 tensor equals rank-0 tensor: decreasing rank

UV=VU=ViUi=(U1U2U3)(V1V2V3)=(V1V2V3)(U1U2U3)=V1U1+V2U2+V3U3

19.1.3 tensor direct product: increasing rank

ST=SikTjl(i,j),(k,l){(1,1),(1,2),(2,1),(2,2)}(S11S12S21S22)(T11T12T21T22)=(S11T11S11T12S12T11S12T12S11T21S11T22S12T21S12T22S21T11S21T12S22T11S22T12S21T21S21T22S22T21S22T22)(S11S12S21S22)(T11T12T21T22)=(S11TS12TS21TS22T)=(S11(T11T12T21T22)S12(T11T12T21T22)S21(T11T12T21T22)S22(T11T12T21T22))

19.2 covariance matrix and its properties

8

C[X]=Cov[X]=V[X]=E[[XE(X)][XE(X)]]=E[[XE(X)][XE(X)]]=E[XXE(X)XXE(X)+E(X)E(X)]=E[XX]E[E(X)X]E[XE(X)]+E[E(X)E(X)]=E[XX]E(X)E[X]E[X]E(X)+E(X)E(X)=E[XX]E(X)E(X)E(X)E(X)+E(X)E(X)=E[XX]E(X)E(X)

X=[X]1×1=XC(X)=C[X]=E[XX]E(X)E(X)=E[XX]E(X)E(X)=E(X2)[E(X)]2=V(X)

19.2.1 V[X+b]=V[X]

V[X+b]=E[[(X+b)E(X+b)][(X+b)E(X+b)]]=E(X+b)=E(X)+bE[[X+bE(X)b][X+bE(X)b]]=E[[XE(X)][XE(X)]]=V[X]

19.2.2 V[AX]=AV[X]A

V[AX]=E[[(AX)E(AX)][(AX)E(AX)]]=E(AX)=AE(X)E[[AXAE(X)][AXAE(X)]]=E[A[XE(X)][A[XE(X)]]]=E[A[XE(X)][XE(X)]A]=AE[[XE(X)][XE(X)]]A=AV[X]A

19.2.3 V[AX+b]=AV[X]A

V[AX+b]=V[AX]=AV[X]A

references

8.