Step 1: Calculating \( P(|X|<1) \).
We are asked to find the probability that \( |X|<1 \), which is equivalent to finding the integral of \( f(x) \) over the interval \( -1 \leq x \leq 1 \):
\[
P(|X|<1) = \int_{-1}^{1} \frac{5x^4}{64} \, dx
\]
We compute this integral:
\[
P(|X|<1) = \frac{5}{64} \int_{-1}^{1} x^4 \, dx = \frac{5}{64} \left[ \frac{x^5}{5} \right]_{-1}^{1} = \frac{5}{64} \left( \frac{1^5}{5} - \frac{(-1)^5}{5} \right)
\]
\[
P(|X|<1) = \frac{5}{64} \cdot \frac{2}{5} = \frac{1}{32}
\]
Step 2: Calculating \( P(X^2<3) \).
Next, we need to calculate the probability that \( X^2<3 \), which is equivalent to finding the integral of \( f(x) \) over the interval \( -\sqrt{3} \leq x \leq \sqrt{3} \):
\[
P(X^2<3) = \int_{-\sqrt{3}}^{\sqrt{3}} \frac{5x^4}{64} \, dx
\]
We compute this integral:
\[
P(X^2<3) = \frac{5}{64} \int_{-\sqrt{3}}^{\sqrt{3}} x^4 \, dx = \frac{5}{64} \left[ \frac{x^5}{5} \right]_{-\sqrt{3}}^{\sqrt{3}} = \frac{5}{64} \left( \frac{(\sqrt{3})^5}{5} - \frac{(-\sqrt{3})^5}{5} \right)
\]
\[
P(X^2<3) = \frac{5}{64} \cdot \frac{18\sqrt{3}}{5} = \frac{9}{\sqrt{32}}
\]
Step 3: Conclusion.
Thus, the correct answer is (A) \( \frac{1}{32} \) and \( \frac{9}{\sqrt{32}} \).