"Throw the baby out with the bath water" is an idiom that warns against carelessly discarding something valuable while trying to get rid of something unwanted. It comes from the image of emptying a tub of dirty bath water without checking that the baby, who was being bathed in it, is not thrown out along with it.
Option 1, clean out everything, and options 3 and 4, being thorough, focus only on the act of removing or cleaning without capturing the idiom's real warning, which is about the mistake of losing something good in the process. The idiom is not praising thoroughness, it is cautioning against an overzealous or careless approach that ends up sacrificing something worth keeping.
Because the phrase specifically warns against losing the valuable along with the unwanted, option 2, throw out the good things with the unwanted, is the correct meaning.