A Guide on Data Analysis
1
Introduction
2
Prerequisites
2.1
Matrix Theory
2.1.1
Rank
2.1.2
Inverse
2.1.3
Definiteness
2.1.4
Matrix Calculus
2.1.5
Optimization
2.2
Probability Theory
2.2.1
Axiom and Theorems of Probability
2.2.2
Central Limit Theorem
2.2.3
Random variable
2.2.4
Moment generating function
2.2.5
Moment
2.2.6
Distributions
2.3
General Math
2.3.1
Law of large numbers
2.3.2
Law of Iterated Expectation
2.3.3
Convergence
2.3.4
Sufficient Statistics
2.3.5
Parameter transformations
2.4
Methods
2.5
Data Import/Export
2.5.1
Medium size
2.5.2
Large size
2.6
Data Manipulation
I BASIC
3
Descriptive Statistics
3.1
Numerical Measures
3.2
Graphical Measures
3.2.1
Shape
3.3
Normality Assessment
3.3.1
Graphical Assessment
3.3.2
Summary Statistics
4
Basic Statistical Inference
4.1
One Sample Inference
4.1.1
The Mean
4.1.2
Single Variance
4.1.3
Single Proportion (p)
4.1.4
Power
4.1.5
Sample Size
4.1.6
Note
4.1.7
One-sample Non-parametric Methods
4.2
Two Sample Inference
4.2.1
Means
4.2.2
Variances
4.2.3
Power
4.2.4
Sample Size
4.2.5
Matched Pair Designs
4.2.6
Nonparametric Tests for Two Samples
4.3
Categorical Data Analysis
4.3.1
Inferences for Small Samples
4.3.2
Test of Association
4.3.3
Ordinal Association
II REGRESSION
5
Linear Regression
5.1
Ordinary Least Squares
5.1.1
Simple Regression (Basic Model)
5.1.2
Multiple Linear Regression
5.1.3
OLS Assumptions
5.1.4
Theorems
5.1.5
Variable Selection
5.1.6
Diagnostics
5.1.7
Model Validation
5.1.8
Finite Sample Properties
5.1.9
Large Sample Properties
5.1.10
Application
5.2
Feasible Generalized Least Squares
5.2.1
Heteroskedasticity
5.2.2
Serial Correlation
5.3
Weighted Least Squares
5.4
Generalized Least Squares
5.5
Feasiable Prais Winsten
5.6
Feasible group level Random Effects
5.7
Ridge Regression
5.8
Principal Component Regression
5.9
Robust Regression
5.9.1
Least Absolute Residuals (LAR) Regression
5.9.2
Least Median of Squares (LMS) Regression
5.9.3
Iteratively Reweighted Least Squares (IRLS) Robust Regression
5.10
Maximum Likelihood
5.10.1
Motivation for MLE
5.10.2
Assumption
5.10.3
Properties
5.10.4
Compare to OLS
5.10.5
Application
6
Non-linear Regression
6.1
Inference
6.1.1
Linear Function of the Parameters
6.1.2
Nonlinear
6.2
Non-linear Least Squares
6.2.1
Alternative of Gauss-Newton Algorithm
6.2.2
Practical Considerations
6.2.3
Model/Estiamtion Adequcy
6.2.4
Application
7
Generalized Linear Models
7.1
Logistic Regression
7.1.1
Application
7.2
Probit Regression
7.3
Binomial Regression
7.4
Poisson Regression
7.4.1
Application
7.5
Negative Binomial Regression
7.6
Multinomial
7.7
Generalization
7.7.1
Estimation
7.7.2
Inference
7.7.3
Deviance
7.7.4
Diagnostic Plots
7.7.5
Goodness of Fit
7.7.6
Over-Dispersion
8
Linear Mixed Models
8.1
Dependent Data
8.1.1
Random-Intercepts Model
8.1.2
Covariance Models
8.2
Estimation
8.2.1
Estimating
\(\mathbf{V}\)
8.3
Inference
8.3.1
Parameters
\(\beta\)
8.3.2
Variance Components
8.4
Information Criteria
8.4.1
Akaike’s Information Criteria (AIC)
8.4.2
Corrected AIC (AICC)
8.4.3
Bayesian Information Criteria (BIC)
8.5
Split-Plot Designs
8.5.1
Application
8.6
Repeated Measures in Mixed Models
8.7
Unbalanced or Unequally Spaced Data
8.8
Application
8.8.1
Example 1 (Pulps)
8.8.2
Example 2 (Rats)
8.8.3
Example 3 (Agridat)
9
Nonlinear and Generalized Linear Mixed Models
9.1
Estimation
9.1.1
Estimation by Numerical Integration
9.1.2
Estimation by Linearization
9.1.3
Estimation by Bayesian Hierarchical Models
9.2
Application
9.2.1
Binomial (CBPP Data)
9.2.2
Count (Owl Data)
9.2.3
Binomial
9.2.4
Example from
(
Schabenberger and Pierce 2001
)
section 8.4.1
9.3
Summary
10
Generalized Method of Moments
11
Minimum Distance
12
Spline Regression
12.1
Regression Splines
12.2
Natural splines
12.3
Smoothing splines
12.4
Application
13
Generalized Additive Models
14
Quantile Regression
14.1
Application
III RAMIFICATIONS
15
Model Specification
15.1
Nested Model
15.1.1
Chow test
15.2
Non-Nested Model
15.2.1
Davidson-Mackinnon test
15.3
Heteroskedasticity
15.3.1
Breusch-Pagan test
15.3.2
White test
16
Endogeneity
16.1
Endogenous Treatment
16.1.1
Instrumental Variable
16.1.2
Internal instrumental variable
16.1.3
Proxy Variables
16.2
Endogenous Sample Selection
16.2.1
Tobit-2
16.2.2
Tobit-5
17
Imputation (Missing Data)
17.1
Assumptions
17.1.1
Missing Completely at Random (MCAR)
17.1.2
Missing at Random (MAR)
17.1.3
Ignorable
17.1.4
Nonignorable
17.2
Solutions to Missing data
17.2.1
Listwise Deletion
17.2.2
Pairwise Deletion
17.2.3
Dummy Variable Adjustment
17.2.4
Imputation
17.2.5
Other methods
17.3
Criteria for Choosing an Effective Approach
17.4
Another Perspective
17.5
Diagnosing the Mechanism
17.5.1
MAR vs. MNAR
17.5.2
MCAR vs. MAR
17.6
Application
17.6.1
Imputation with mean / median / mode
17.6.2
KNN
17.6.3
rpart
17.6.4
MICE (Multivariate Imputation via Chained Equations)
17.6.5
Amelia
17.6.6
missForest
17.6.7
Hmisc
17.6.8
mi
18
Data
18.1
Cross-Sectional
18.2
Time Series
18.2.1
Deterministic Time trend
18.2.2
Feedback Effect
18.2.3
Dynamic Specification
18.2.4
Dynamically Complete
18.2.5
Highly Persistent Data
18.3
Repeated Cross Sections
18.3.1
Pooled Cross Section
18.4
Panel Data
18.4.1
Pooled OLS Estimator
18.4.2
Individual-specific effects model
18.4.3
Tests for Assumptions
18.4.4
Model Selection
18.4.5
Summary
18.4.6
Application
18.4.7
Other Estimators
19
Hypothesis Testing
19.1
Types of hypothesis testing
19.2
Wald test
19.2.1
Multiple Hypothesis
19.2.2
Linear Combination
19.2.3
Application
19.2.4
Nonlinear
19.3
The likelihood ratio test
19.4
Lagrange Multiplier (Score)
IV EXPERIMENTAL DESIGN
20
Analysis of Variance (ANOVA)
20.1
Completely Randomized Design (CRD)
20.1.1
Single Factor Fixed Effects Model
20.1.2
Single Factor Random Effects Model
20.1.3
Two Factor Fixed Effect ANOVA
20.1.4
Two-Way Random Effects ANOVA
20.1.5
Two-Way Mixed Effects ANOVA
20.2
Nonparametric ANOVA
20.2.1
Kruskal-Wallis
20.2.2
Friedman Test
20.3
Sample Size Planning for ANOVA
20.3.1
Balanced Designs
20.3.2
Randomized Block Experiments
20.4
Randomized Block Designs
20.4.1
Tukey Test of Additivity
20.5
Nested Designs
20.5.1
Two-Factor Nested Designs
20.6
Single Factor Covariance Model
21
Multivariate Methods
21.0.1
Properties of MVN
21.0.2
Mean Vector Inference
21.0.3
General Hypothesis Testing
21.1
MANOVA
21.1.1
Testing General Hypotheses
21.1.2
Profile Analysis
21.1.3
Summary
21.2
Principal Components
21.2.1
Population Principal Components
21.2.2
Sample Principal Components
21.2.3
Application
21.3
Factor Analysis
21.3.1
Methods of Estimation
21.3.2
Factor Rotation
21.3.3
Estimation of Factor Scores
21.3.4
Model Diagnostic
21.3.5
Application
21.4
Discriminant Analysis
21.4.1
Known Populations
21.4.2
Probabilities of Misclassification
21.4.3
Unknown Populations/ Nonparametric Discrimination
21.4.4
Application
21.5
Cluster Analysis
22
Causality
23
Report
23.1
One summary table
23.2
Model Comparison
23.3
Changes in an estimate
APPENDIX
A
Appendix
A.1
Git
A.2
Short-cut
A.3
Function short-cut
A.4
Citation
B
Bookdown cheat sheet
B.1
Operation
B.2
Math Expresssion/ Syntax
B.2.1
Statistics Notation
B.3
Table
References
Published with bookdown
A Guide on Data Analysis
9.3
Summary
