Question

For a classification problem, Principal Component Analysis (PCA) has been used to reduce the dimensionality of a feature space from 100 to 10.
Which of the following options is true about the angle \(\theta\) between the first and the tenth principal components?

• A. \(\theta = 0^{\circ}\)
• B. \(\theta = 90^{\circ}\)
• C. \(90^{\circ} < \theta \leq 180^{\circ}\)
• D. \(0 < \theta < 90^{\circ}\)

Hint:

Principal components are eigenvectors of a symmetric covariance matrix, and such eigenvectors are always mutually orthogonal.

Answer 1

The Correct Option is B

Step 1: Understanding the Question.
We need to work out the geometric relationship between two different principal components produced by PCA, specifically the 1st and the 10th, when a 100 dimensional feature space is compressed down to 10 dimensions.

Step 2: Key Formula or Approach.
PCA works by finding the eigenvectors of the data's covariance matrix. A covariance matrix is always real and symmetric. A core theorem of linear algebra states that eigenvectors of a real symmetric matrix, when they correspond to distinct eigenvalues, are always mutually orthogonal, meaning the angle between any two of them is exactly 90 degrees.

Step 3: Detailed Explanation.
The principal components are simply these eigenvectors, ranked from the one with the largest eigenvalue (capturing the most variance, this is the 1st principal component) down to smaller eigenvalues. The 10th principal component is just another eigenvector of the same covariance matrix, associated with a different eigenvalue than the 1st. It does not matter that we reduced 100 dimensions down to 10, keeping only 10 components does not change the fact that each kept component is still orthogonal to every other one, since orthogonality is a property of the eigenvectors themselves, not of how many we choose to retain. So the 1st and 10th principal components must be perpendicular to each other.

Step 4: Why the other options fail.
Option (A), \(\theta = 0^{\circ}\), would mean the two components point in the exact same direction, but PCA components are built to be linearly independent and orthogonal, never parallel. Options (C) and (D) suggest some in-between acute or obtuse angle, but that would only happen for arbitrary vectors, not for eigenvectors of a symmetric matrix, which are guaranteed to be exactly perpendicular, not just roughly so.

Step 5: Final Answer.
The angle between the first and tenth principal components is exactly 90 degrees, option (B). \[ \boxed{\theta = 90^{\circ}} \]