Step 1: Understanding the Concept
A product equal to zero means the curve is the union of the curves of the two factors.
Step 2: Second factor
\(x^2-5xy+6y^2=(x-2y)(x-3y)\), which gives two straight lines.
Step 3: First factor
\(ax^2+by^2+c=0\). If \(a=b\) and \(c\) has a sign opposite to \(a\), then \(x^2+y^2=-\dfrac{c}{a}\gt 0\), a real circle.
Step 4: Check the options
When \(c=0\) and \(a,b\) have the same sign, the first factor is only the origin, not two lines. When \(a,b\) have the same sign but differ, the first factor is an ellipse, not a hyperbola, and the second factor is a pair of lines, not an ellipse.
Final Answer:
The curve is two straight lines and a circle when a equals b and c has the opposite sign.
\[ \boxed{\text{(D) }\text{Two straight lines and a circle, when }a=b\text{ and }c\text{ is of sign opposite to that of }a} \]