Concept:
Calculus - Indefinite Integration by Substitution.
Step 1: Choose an appropriate substitution.
To eliminate the square root in the denominator, let $1+x^2 = t^2$.
This implies that $x^2 = t^2 - 1$.
Step 2: Differentiate the substitution.
Differentiate $1+x^2 = t^2$ implicitly: $2x~dx = 2t~dt$, which simplifies to $x~dx = t~dt$.
Rewrite the numerator of the integral, $x^3 dx$, to utilize this: $x^3 dx = x^2(x~dx)$.
Step 3: Substitute and simplify the integral.
Substitute $t$ into the original integral:
$\int \frac{x^2 \cdot x~dx}{\sqrt{1+x^2}} = \int \frac{(t^2 - 1) \cdot t~dt}{\sqrt{t^2}} = \int \frac{(t^2 - 1) \cdot t~dt}{t}$.
The $t$ in the numerator and denominator cancel out perfectly, leaving a simple polynomial integral:
$\int (t^2 - 1) dt$.
Step 4: Integrate and substitute $x$ back.
Integrate with respect to $t$: $\frac{t^3}{3} - t + c$.
Now, substitute $t = \sqrt{1+x^2}$ back into the expression:
$\frac{(\sqrt{1+x^2})^3}{3} - \sqrt{1+x^2} + c$.
We can write $(\sqrt{1+x^2})^3$ as $(1+x^2)\sqrt{1+x^2}$. So the expression becomes:
$\frac{1}{3}(1+x^2)\sqrt{1+x^2} - 1\sqrt{1+x^2} + c$.
Step 5: Compare coefficients and calculate $3ab$.
Compare our integrated result with the given expression: $a(1+x^2)\sqrt{1+x^2} + b\sqrt{1+x^2} + c$.
By matching coefficients, we find that $a = \frac{1}{3}$ and $b = -1$.
The question asks for the value of $3ab$.
Substitute the values: $3 \cdot \left(\frac{1}{3}\right) \cdot (-1) = 1 \cdot (-1) = -1$.
$$
\therefore \text{The value of } 3ab \text{ is } -1.
$$