Step 1: Find the coordinates of \(P\).
Equation of line \(AB\):
\[
m=\frac{9-5}{5-3}=2
\]
\[
y-5=2(x-3)
\]
\[
y=2x-1
\]
Since \(P\) lies on \(y=3x\),
\[
3x=2x-1
\]
\[
x=-1,\qquad y=-3
\]
Hence,
\[
P=(-1,-3)
\]
Step 2: Find the coordinates of \(Q\).
Equation of line \(BC\):
\[
m=\frac{7-9}{3-5}=1
\]
\[
y-9=x-5
\]
\[
y=x+4
\]
Since \(Q\) lies on \(y=3x\),
\[
3x=x+4
\]
\[
2x=4
\]
\[
x=2,\qquad y=6
\]
Hence,
\[
Q=(2,6)
\]
Step 3: Calculate \(PQ^2\).
\[
PQ^2=(2+1)^2+(6+3)^2
\]
\[
=3^2+9^2
\]
\[
=9+81=90
\]
Step 4: Final conclusion.
\[
\boxed{PQ^2=90}
\]
Hence, the correct option is \(\boxed{(B)}\).