We all like to think we don’t care what people think—but let’s be honest, some outfits attract side-eyes faster than others.