Question

For a classification problem PCA has been used to reduce the dimensionality of a feature space from 100 to 10. Which of the following option is true about the angle b/w first 2 and 10th principal components?

• A. 90$^\circ$< $\theta$< 180$^\circ$
• B. $\theta$ = 0
• C. $\theta$ = 90
• D. 0< $\theta$< 90$^\circ$

Hint:

A core mathematical concept behind PCA is finding the eigenvectors of the covariance matrix of the data. Eigenvectors of a symmetric matrix (like a covariance matrix) corresponding to distinct eigenvalues are always orthogonal. This is the reason why principal components are orthogonal to each other.

Answer 1

The Correct Option is C

Step 1: Understanding the Question:
The question asks about the geometric relationship (specifically, the angle) between the principal components derived from Principal Component Analysis (PCA).
Step 2: Key Concepts of PCA:
Principal Component Analysis is a dimensionality reduction technique that transforms the original variables of a dataset into a new set of variables, called principal components (PCs). These PCs have specific properties:
1. They are linear combinations of the original variables.
2. They are ordered by the amount of variance they explain in the data. The first principal component (PC1) accounts for the largest possible variance, PC2 accounts for the second-largest variance, and so on.
3. Critically, each principal component is orthogonal to all the preceding principal components.
Step 3: Detailed Explanation:
The property of orthogonality is key here. In geometric terms, two vectors being orthogonal means they are at a right angle (90 degrees) to each other.
- The first principal component (PC1) is found as the direction of maximum variance.
- The second principal component (PC2) is found as the direction of maximum variance in the data, with the constraint that it must be orthogonal to PC1.
- The third principal component (PC3) is found as the direction of maximum variance, constrained to be orthogonal to both PC1 and PC2.
- This process continues for all subsequent principal components. Each new component is orthogonal to all the ones that came before it.
Therefore, the angle between any two distinct principal components (like the 2nd and the 10th) is always 90 degrees.
Step 4: Final Answer:
The angle between the 2nd and 10th principal components is 90 degrees.