I think that those who claim to be philosophers of science or math should have had some real experience with doing research in those fields. It seems that a first-rate philososphy of math can only come from a practioner of the discipline. (I would suggest the same for philosophy of science.) Given this, a basic training would involve graduate work in those disciplines, not philosophy by itself.