Step 1: Understand the concept
The volume with coterminous edges \(\vec{p}, \vec{q}, \vec{r}\) is \(|[\vec{p}\ \vec{q}\ \vec{r}]|\), the modulus of the determinant of their components.
Step 2: Build the determinant
\[ \begin{vmatrix} 2a & 0 & 1 \\ 0 & a & -a \\ 3 & a & 0 \end{vmatrix} = 2a(0 + a^2) - 0 + 1(0 - 3a) = 2a^3 - 3a \]
Step 3: Maximise the modulus
Let \(f(a) = 2a^3 - 3a\). Then \(f'(a) = 6a^2 - 3 = 0\) at \(a = \frac{1}{\sqrt{2}}\) (inside \([0, 1]\)). \(f\left(\frac{1}{\sqrt{2}}\right) = \frac{2}{2\sqrt{2}} - \frac{3}{\sqrt{2}} = -\sqrt{2}\).
Step 4: Compare end points
\(f(0) = 0\) and \(f(1) = -1\). The largest modulus is \(\sqrt{2}\), so the maximum volume is \(\sqrt{2}\), option (B).
Final Answer:
The maximum volume is sqrt 2 cubic units. This is option (B).
\[ \boxed{\text{(B) }\sqrt{2}} \]