24
$\begingroup$

Over the last several years, the college-level promotion & tenure committee at my university has increasingly been seeking to apply "objective" criteria for assessing the impact of candidates' research. Journal impact factors have been a favored metric, and we have tried to argue that these are not a reliable measure, for all the usual well-documented reasons. (The AMS statements on "The Culture of Research and Scholarship in Mathematics" have been helpful, if not entirely convincing.) Now they want to use more individual metrics, such as h-indices and g-indices. We would like to discourage this, but it's hard to argue convincingly to non-mathematicians about the flaws of such measures, and at the end of the day, they're demanding SOME numerical measure that they can use to compare candidates, both internally and to faculty at peer institutions.

So, I'd like to hear how other math departments have dealt with such pressures. In particular, how can you articulate standards in such a way as to maintain high expectations, but also to minimize the damage to candidates who might be doing well-respected research, but for whatever reasons this results in relatively few papers, or papers with relatively few citations? And if we do end up having to use something like an h-index, is there any way to collect data from comparable math departments so we can at least say something about what is a "good" value for a particular index for mathematicians?

Improve this question
$\endgroup$
9
  • 3
    $\begingroup$ publishing.mathforge.org $\endgroup$
    – darij grinberg
    Commented Apr 6, 2012 at 22:30
  • 16
    $\begingroup$ I disagree with you on this one, Will. The OP is looking for information ("how other math departments have dealt with such pressures"). $\endgroup$
    – Bill Johnson
    Commented Apr 6, 2012 at 22:31
  • 8
    $\begingroup$ I also think this is a fine question. $\endgroup$
    – Andy Putman
    Commented Apr 6, 2012 at 23:44
  • 4
    $\begingroup$ I think this question is more appropriate for academia.stackexchange.com $\endgroup$
    – JRN
    Commented Apr 7, 2012 at 0:50
  • 2
    $\begingroup$ I'm interested to hear answers to this question. There was no discussion of this on meta, that I could find. So I am voting to reopen. However, please do make this community wiki. $\endgroup$
    – Sam Nead
    Commented Apr 7, 2012 at 19:10

3 Answers 3

Reset to default
2
$\begingroup$

We have produced a list of top 10 journals for each area of mathematics represented in department plus a list of top 10 general subject journals so our candidates for tenure/promotion need to have publications in one of these journals. However I know for a fact that this has not stopped the administration from using impact factors, h-indices etc. Additionally tenure decisions seem to be more and more conditioned on having outside funding, NSF, NSA etc.

Improve this answer
$\endgroup$
4
  • 6
    $\begingroup$ I find this too restrictive. The value and impact of a math paper is all too often apparent only after it has been published, so it is unfair to discriminate against papers published in a lower ranked journal, if they have been cited by other good papers. $\endgroup$
    – Deane Yang
    Commented Apr 7, 2012 at 9:29
  • $\begingroup$ I agree with you, but that was the compromise we reached with the administration to mollify the over-reliance on the impact factor which for mathematics journals is much smaller than say biology journals. However our annual dean's report asks us to use web of science to generate a citation report and an h-index. This is a disaster since web of science citation statistics are very inaccurate for mathematicians. Some of my chemist friends have similar complaints about this over-priced data basis. $\endgroup$
    – Liviu Nicolaescu
    Commented Apr 7, 2012 at 10:35
  • 9
    $\begingroup$ It is very important to have a dean to understand that it is a very ba idea to compare numbers across different fields and that when you are hiring a mathematician, you're not trying to hire one as good or better than the biologist you already have but you're trying to hire one that is as good as or better than the ones at universities you're trying to match up against. $\endgroup$
    – Deane Yang
    Commented Apr 7, 2012 at 12:09
  • 2
    $\begingroup$ Wouldn't that be nice! $\endgroup$
    – Liviu Nicolaescu
    Commented Apr 7, 2012 at 13:47
7
$\begingroup$

I think it is dangerous to be too qualitative or too simplistically quantitative about judging a candidate's accomplishments. I do believe, however, that looking at Mathscinet citations is a good start in evaluating both a person and the person's papers. Basically, you want to find evidence that a person's papers have had good impact in the sense that it has led to other respected work (as just by, say, the quality of journals and number of citations of those papers) that use or build on the person's contributions.

All of this is quite problematic when you are judging someone who has been publishing for less than 10 years, but in my experience, for someone out 10 years or more, looking at citations per paper or citations per year, as well as total number of citations, is a remarkably good guide for identifying the better mathematicians, in the sense that the ranking I get agrees rather well with my own subjective views. Then I simply focus on the exceptions and try to decide whether the citations are telling me that I've misjudged or whether the citations are simply misleading in that particular case. Even there, my conclusion is usually but not always the former.

But there are still difficulties. It is quite noticeable, even within pure mathematics, that people working in some fields (like PDE's) get a lot more citations than others. So, you have to be careful about comparing people across different fields.

Improve this answer
$\endgroup$
6
  • 3
    $\begingroup$ Also, some work just doesn't get cited at all, even if it is used by thousands of mathematicians. For example people who work on efficient algorithms for computing software are at a huge disadvantage, whenever publications and citations are used to asses faculty: they do serious mathematical work, their results are often more widely used in the mathematical community than the most brilliant publications; yet, they are almost never personally acknowledged in the papers that use the software and they often do not publish in high profile journals. $\endgroup$
    – Alex B.
    Commented Apr 7, 2012 at 12:48
  • 3
    $\begingroup$ Another danger of focusing on Mathscinet citations is that they don't count citations appearing in the CS or physics literature, so this can substantially undercount citations for work near the interface between mathematics and other fields. $\endgroup$
    – Henry Cohn
    Commented Apr 7, 2012 at 13:10
  • $\begingroup$ And of course, the problem at hand is that issues such as how typical citation counts vary across fields of mathematics are too subtle for university-level committees; they just want one standard for "mathematics." Attempts to explain that candidate X works in a field where publication rates (and hence citations) are typically on the low side tend to be met with skepticism, even if that statement is backed up by the external letter-writers. $\endgroup$
    – Jeanne Clelland
    Commented Apr 7, 2012 at 13:49
  • 9
    $\begingroup$ Speaking from the point of view of a dean, I rely on my evaluation of the credibility of the departmental committee and the department head. Do I believe that they have real standards? Can I see evidence that the department knows what it's doing? When the department reviews candidates, are they realistic about the strengths AND weaknesses of cases? Do they get letters from high-profile people and read those letters carefully? I am much more comfortable taking the advice of people who I trust than I am in any citation ranking system. $\endgroup$
    – Jeremy Teitelbaum
    Commented Apr 7, 2012 at 15:36
  • 2
    $\begingroup$ If you must use citations, you should use Google Scholar in addition to Mathscinet because it captures citations missed by Mathscinet (vice versa also occasionally happens). $\endgroup$
    – Bill Johnson
    Commented Apr 7, 2012 at 15:58
1
$\begingroup$

This is not a direct answer to your question, but I think it is related. To get a perspective on why such things have been happening, I recommend the paper Neo-liberalism and Marketisation: the implications for higher education especially the section The Implications of Marketisation, starting on page 6.

Improve this answer
$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .