Skip to main content
Post Made Community Wiki by François G. Dorais
Source Link
R Hahn
  • 2.8k
  • 1
  • 20
  • 28

In mathematical statistics people often have experience about some method that works well in practice even though it "shouldn't" in all generality. The game is then to ask what conditions need to be satisfied to explain why the method works.

Here is an example which I was not personally involved in, so I can only speculate. This paper by Bickel and Li considers local polynomial regression methods and shows that they works as well as possible (in the sense of asymptotic optimality) when the data it is being used on has low dimensional structure. The idea is that people were finding that certain regression techniques were giving reasonable generalization performance in prediction problems even when the data was high dimensional so they figured that maybe the data wasn't actually high dimensional in some relevant aspect. But which relevant aspect, that's the challenging part.

To my mind, figuring out how to explicitly articulate the minimal conditions under which some ``obvious" fact is true is where the discover and understanding come in. It is a very different process than what a student does on problem set, where the statement and all the relevant conditions are laid out and the main job is deriving the stated implication.

Put another way: research has degrees of freedom on both ends -- you can find/create the answer and the question as pairs, rather than being handed the one and being asked to complete the set. This perspective of course doesn't cover all cases -- notably, that of people chasing down famous open problems. But it is a way in which one can develop a rigorous understanding from ``non"-rigorous reasoning. When one first starts thinking vaguely about a problem there is nothing there about which to be rigorous.