Question

Consider that for a supervised learning task, the objective function being minimized is \(f_w(x) = wx\), where \(x \in \mathbb{R}\) is the input and \(w \in \mathbb{R}\) is the parameter. Stochastic Gradient Descent with a learning rate of 0.10 is used for parameter updates.

Suppose that at the end of iteration \(i\), the value of \(w\) becomes 10.00.

Let \(x = 10.00\) be the input for iteration \((i+1)\).

The value of \(w\) at the end of iteration \((i+1)\) is __________. (Rounded off to two decimal places)

Hint:

The gradient of wx with respect to w is just x, so the update rule reduces to w minus the learning rate times x.

Answer 1

Correct Answer: 9

Step 1: Write down the SGD update rule.
Stochastic Gradient Descent updates the parameter using \(w_{new} = w_{old} - \eta \cdot \dfrac{\partial f_w(x)}{\partial w}\), where \(\eta\) is the learning rate.

Step 2: Compute the gradient of the objective function.
The objective function is \(f_w(x) = wx\). Taking the partial derivative with respect to \(w\):
\[ \frac{\partial f_w(x)}{\partial w} = x \]

Step 3: Plug in the known values.
At the end of iteration \(i\), \(w = 10.00\). The input for iteration \((i+1)\) is \(x = 10.00\), so the gradient at this step is \(10.00\). The learning rate is \(\eta = 0.10\).

Step 4: Apply the update rule.
\[ w_{new} = w_{old} - \eta \cdot x = 10.00 - 0.10 \times 10.00 \]
\[ w_{new} = 10.00 - 1.00 = 9.00 \]

Final Answer:
The value of \(w\) at the end of iteration \((i+1)\) is 9.00. \[ \boxed{9.00} \]