Principal Component Analysis

Covariance eigenvectors · variance explained · dimensionality reduction · reconstruction

Original data + PC axes
Projected data (PC1 vs PC2)
PCA finds directions of maximum variance (principal components) via eigendecomposition of the covariance matrix Σ. PC1 explains the most variance, PC2 the next-most (orthogonal to PC1). Variance explained = λᵢ / Σλⱼ. Reconstruction with k PCs: x̂ = μ + Σᵢ₌₁ᵏ (xᵀPCᵢ)PCᵢ. The "scree plot" shows where to truncate.