Despite my being a student, I still do receive responses similar to the response cited in the op. My response is usually, "Yes, but have you ever taken a real math class?" This usually results in some sort of explanation that mathematicians (most?) don't sit around doing calculus problems all day. Although, on at least one occasion, the person with whom I was speaking became very hostile at the suggestion that he did not know what real math was.