Step 1: Express the terms of the arithmetic progression.
Let the first term of the arithmetic progression be \(A\) and the common difference be \(d\).
Then,
\[
a=A+4d
\]
\[
b=A+7d
\]
\[
c=A+12d
\]
since \(a,b,c\) are respectively the \(5^{th},8^{th},13^{th}\) terms of the A.P.
Step 2: Observe the relation among \(a,b,c\).
Compute:
\[
b-a=(A+7d)-(A+4d)=3d
\]
and
\[
c-b=(A+12d)-(A+7d)=5d
\]
Hence,
\[
\frac{b-a}{8-5}
=
\frac{c-b}{13-8}
=
d
\]
Therefore, the points
\[
(a,5),\;(b,8),\;(c,13)
\]
are collinear.
Step 3: Use determinant property.
The determinant
\[
\begin{vmatrix}
a & 5 & 1\\
b & 8 & 1\\
c & 13 & 1
\end{vmatrix}
\]
represents twice the area of the triangle formed by the points
\[
(a,5),\;(b,8),\;(c,13)
\]
Since the three points are collinear, the area of the triangle is zero.
Therefore,
\[
\begin{vmatrix}
a & 5 & 1\\
b & 8 & 1\\
c & 13 & 1
\end{vmatrix}
=0
\]
Step 4: Final conclusion.
Hence,
\[
\boxed{0}
\]
which corresponds to option (1).