Question

Consider that Linear Ridge Regression is being used to learn a prediction function \(y_{pred} = w^T x\), where \(w, x \in \mathbb{R}^2\) and Mean Absolute Error (MAE) is used to measure the prediction error. A weight of 0.20 is associated with the regularizer.

At an intermediate step of the training process, assume that the parameter \(w = [-3.00,\ 4.00]^T\). In the next step, for the input \(x = [1.00,\ 2.00]^T\), the predicted value of \(y\) is noted. Let the relation between \(x = [x_1,\ x_2]^T\) and the true value of \(y\) be \(y_{true} = x_1 + x_2\).

The value of the overall regularized loss function for this instance is _______ . (Rounded off to two decimal places)

Hint:

Compute the MAE term |w^T x - y_true| and the L2 penalty lambda times ||w||^2 separately, then add them together.

Answer 1

Correct Answer: 7

Step 1: Understand the loss function for Ridge Regression with MAE.
Ridge regression adds an L2 penalty on the weights to whatever base error metric is used. Here the base error is Mean Absolute Error, so for a single data point the overall regularized loss is:
\[ L = |y_{pred} - y_{true}| + \lambda \|w\|_2^2 \]
where \(\lambda = 0.20\) is the given regularization weight and \(\|w\|_2^2 = w_1^2 + w_2^2\) is the squared L2 norm of the weight vector.

Step 2: Compute the predicted value.
\(y_{pred} = w^T x = (-3.00)(1.00) + (4.00)(2.00) = -3.00 + 8.00 = 5.00\)

Step 3: Compute the true value.
\(y_{true} = x_1 + x_2 = 1.00 + 2.00 = 3.00\)

Step 4: Compute the Mean Absolute Error term.
\[ |y_{pred} - y_{true}| = |5.00 - 3.00| = 2.00 \]

Step 5: Compute the regularization term.
\[ \|w\|_2^2 = (-3.00)^2 + (4.00)^2 = 9.00 + 16.00 = 25.00 \]
\[ \lambda \|w\|_2^2 = 0.20 \times 25.00 = 5.00 \]

Step 6: Add both terms to get the overall regularized loss.
\[ L = 2.00 + 5.00 = 7.00 \]

Final Answer:
The overall regularized loss for this instance is 7.00. \[ \boxed{7.00} \]