Concept:
The given differential equation is:
\[
(x^2-3y^2)dx+2xydy=0
\]
This is a homogeneous differential equation because every term has the same degree in \(x\) and \(y\).
So, we use the substitution:
\[
y=vx
\]
Then:
\[
\frac{dy}{dx}=v+x\frac{dv}{dx}
\]
Step 1: Convert the equation into derivative form.
\[
(x^2-3y^2)+2xy\frac{dy}{dx}=0
\]
\[
2xy\frac{dy}{dx}=3y^2-x^2
\]
Step 2: Put \(y=vx\).
\[
y=vx
\]
So:
\[
y^2=v^2x^2
\]
and:
\[
\frac{dy}{dx}=v+x\frac{dv}{dx}
\]
Substitute in:
\[
(x^2-3y^2)+2xy\frac{dy}{dx}=0
\]
\[
x^2-3v^2x^2+2x(vx)\left(v+x\frac{dv}{dx}\right)=0
\]
\[
x^2(1-3v^2)+2vx^2\left(v+x\frac{dv}{dx}\right)=0
\]
\[
1-3v^2+2v^2+2vx\frac{dv}{dx}=0
\]
\[
1-v^2+2vx\frac{dv}{dx}=0
\]
\[
2vx\frac{dv}{dx}=v^2-1
\]
Step 3: Separate the variables.
\[
\frac{2v}{v^2-1}dv=\frac{dx}{x}
\]
Step 4: Integrate both sides.
\[
\int \frac{2v}{v^2-1}dv=\int \frac{dx}{x}
\]
\[
\log|v^2-1|=\log|x|+\log k
\]
\[
v^2-1=kx
\]
Step 5: Substitute \(v=\frac{y}{x}\).
\[
\left(\frac{y}{x}\right)^2-1=kx
\]
\[
\frac{y^2}{x^2}-1=kx
\]
\[
\frac{y^2-x^2}{x^2}=kx
\]
\[
y^2-x^2=kx^3
\]
Hence, the correct answer is:
\[
\boxed{(B)\ y^2-x^2=kx^3}
\]