How to assess research "impact" for tenure/promotion committees - MathOverflow

How to assess research "impact" for tenure/promotion committees

2012-04-06T22:13:35Z

Over the last several years, the college-level promotion & tenure committee at my university has increasingly been seeking to apply "objective" criteria for assessing the impact of candidates' research. Journal impact factors have been a favored metric, and we have tried to argue that these are not a reliable measure, for all the usual well-documented reasons. (The AMS statements on "The Culture of Research and Scholarship in Mathematics" have been helpful, if not entirely convincing.) Now they want to use more individual metrics, such as h-indices and g-indices. We would like to discourage this, but it's hard to argue convincingly to non-mathematicians about the flaws of such measures, and at the end of the day, they're demanding SOME numerical measure that they can use to compare candidates, both internally and to faculty at peer institutions.

So, I'd like to hear how other math departments have dealt with such pressures. In particular, how can you articulate standards in such a way as to maintain high expectations, but also to minimize the damage to candidates who might be doing well-respected research, but for whatever reasons this results in relatively few papers, or papers with relatively few citations? And if we do end up having to use something like an h-index, is there any way to collect data from comparable math departments so we can at least say something about what is a "good" value for a particular index for mathematicians?

Answer by Liviu Nicolaescu for How to assess research "impact" for tenure/promotion committees

2012-04-06T23:14:32Z

We have produced a list of top 10 journals for each area of mathematics represented in department plus a list of top 10 general subject journals so our candidates for tenure/promotion need to have publications in one of these journals. However I know for a fact that this has not stopped the administration from using impact factors, h-indices etc. Additionally tenure decisions seem to be more and more conditioned on having outside funding, NSF, NSA etc.

Answer by Deane Yang for How to assess research "impact" for tenure/promotion committees

2012-04-07T09:45:54Z

I think it is dangerous to be too qualitative or too simplistically quantitative about judging a candidate's accomplishments. I do believe, however, that looking at Mathscinet citations is a good start in evaluating both a person and the person's papers. Basically, you want to find evidence that a person's papers have had good impact in the sense that it has led to other respected work (as just by, say, the quality of journals and number of citations of those papers) that use or build on the person's contributions.

All of this is quite problematic when you are judging someone who has been publishing for less than 10 years, but in my experience, for someone out 10 years or more, looking at citations per paper or citations per year, as well as total number of citations, is a remarkably good guide for identifying the better mathematicians, in the sense that the ranking I get agrees rather well with my own subjective views. Then I simply focus on the exceptions and try to decide whether the citations are telling me that I've misjudged or whether the citations are simply misleading in that particular case. Even there, my conclusion is usually but not always the former.

But there are still difficulties. It is quite noticeable, even within pure mathematics, that people working in some fields (like PDE's) get a lot more citations than others. So, you have to be careful about comparing people across different fields.

Answer by Mahdi Majidi-Zolbanin for How to assess research "impact" for tenure/promotion committees

2012-04-07T17:24:25Z

This is not a direct answer to your question, but I think it is related. To get a perspective on why such things have been happening, I recommend the paper Neo-liberalism and Marketisation: the implications for higher education especially the section The Implications of Marketisation, starting on page 6.