Concept:
In any triangle,
\[
A + B + C = \pi
\]
We use trigonometric identities:
• \( \cos^2 x = \frac{1 + \cos 2x}{2} \)
• \( \cos A \cos B = \frac{1}{2}[\cos(A+B) + \cos(A-B)] \)
Step 1: Using angle sum property.
\[
A + B = \pi - C = \pi - \frac{2\pi}{3} = \frac{\pi}{3}
\]
So,
\[
\cos(A+B) = \cos\left(\frac{\pi}{3}\right) = \frac{1}{2}
\]
Step 2: Simplify \( \cos^2 A + \cos^2 B \).
\[
\cos^2 A + \cos^2 B
= \frac{1 + \cos 2A}{2} + \frac{1 + \cos 2B}{2}
\]
\[
= 1 + \frac{1}{2}(\cos 2A + \cos 2B)
\]
Using sum-to-product:
\[
\cos 2A + \cos 2B
= 2\cos(A+B)\cos(A-B)
\]
So,
\[
\cos^2 A + \cos^2 B
= 1 + \cos(A+B)\cos(A-B)
\]
Substitute \( \cos(A+B) = \frac{1}{2} \):
\[
\cos^2 A + \cos^2 B = 1 + \frac{1}{2}\cos(A-B)
\quad \cdots (1)
\]
Step 3: Simplify \( \cos A \cos B \).
\[
\cos A \cos B = \frac{1}{2}[\cos(A+B) + \cos(A-B)]
\]
\[
= \frac{1}{2}\left[\frac{1}{2} + \cos(A-B)\right]
\]
\[
= \frac{1}{4} + \frac{1}{2}\cos(A-B)
\quad \cdots (2)
\]
Step 4: Compute required expression.
Let
\[
E = \cos^2 A + \cos^2 B - \cos A \cos B
\]
Substitute (1) and (2):
\[
E = \left(1 + \frac{1}{2}\cos(A-B)\right)
- \left(\frac{1}{4} + \frac{1}{2}\cos(A-B)\right)
\]
\[
E = 1 - \frac{1}{4} = \frac{3}{4}
\]