The success of university researchers is about to be quantified, and that fact has officials on college across the country sweating.
Considered by many as the pre-eminent system for assessing doctorate programs, the United States National Research Council report ranks 222 institutions nationwide over 61 different doctoral fields.
The ratings attract a lot of attention because they come out about once every 10 years. But this forthcoming report is already four years late, as the last edition came out in 1995. Though the release of this version is expected in the very near future, nobody knows exactly when that publication date will be.
The NRC, which recently released its ranking methodology, will be using survey data of participating universities from the 2005-2006 academic year. Applied to individual doctorate programs, the rankings rely on a wide range of criteria, which include the number of publications and grants produced by faculty, demographics like minority composition and current rankings.
Averaging all departmental scores, the University of Michigan scored third overall on the NRC’s previous list of rankings in 1995, behind the University of California, Berkley and first-ranked Stanford University.
In an interview this week, former University President James Duderstadt, who is also a professor of science and engineering, said the ratings carry great weight and the report’s findings have motivated re-evaluation of programs at many colleges in the past.
“If a program you thought was great turned out to be 15th or 20th, you often changed the leadership,” he said. “But, generally, Michigan has not aligned itself with these (ratings), because we’ve been high on the food chain.”
On the steering committee for the Division on Policy and Global Affairs of the NRC, Duderstadt said the forthcoming rankings will be more objective than they have been in the past. The NRC is foregoing a reputation-based assessment of departments in favor of hard data.
While many have applauded this more numerical approach, it does have the potential to yield surprising results, he said, hence the anxiety currently being felt by administrators everywhere.
“Institutions that were highly ranked in the old way may find themselves ranked much differently now,” Duderstadt said.
The effect of the rankings on a research institution’s hiring practices, funding and departmental evaluations has forced the NRC to become more transparent with its methodology, Duderstadt said. But allowing survey participants to become acclimated with the new system has come at the cost of a four-year delay in the release of the rankings, he said.
Duderstadt said the increasing anxiety surrounding the imminent release of the rankings is justified given the current economic climate. Institutions that lack significant financial backing will be forced to trim their departmental costs and may look to the rankings for guidance, he said. Moreover, key donors are likely to be swayed by the NRC’s departmental assessments.
“If it had come out two years ago, it would have been treated in the normal course of things,” Duderstadt said. “But now it will probably feed into some very difficult decisions that institutions are forced to make.”
Having seen three other releases of the NRC rankings during his time at the University, Duderstadt also said there were merits to the reputation-based approach that the survey data alone will not provide, such as considerations of esteemed professors and research projects.
“I don’t know that this is going to be successful,” he said. “It’s going to have an impact, but they may decide, after looking at the reaction, to go back to the earlier model.”
University Spokeswoman Kelly Cunningham said the ratings are an important tool for graduate students looking for a particular assessment of their department of interest. But she said that the NRC rankings are only one part of a bigger picture.
“(The rating) is one of the elements of choosing a graduate school, but, more important is choosing the right graduate school for the right person,” she said.
Cunningham said compatibility with departmental faculty, potential colleagues and research projects — all things that would also factor into a pre-doctoral student’s school choice — is outside the scope of the rankings. The same, she said, is true of the University’s overall approach to education and research.
“Interdisciplinary teaching is huge for us, and I don’t think that is covered in the ratings,” she said.
The large number of institutions that subscribe to the NRC ratings has caused some universities to re-evaluate their research activity and to set an explicit goal of improving their ranking.
As a result, changes have been made in the hiring practices, tenure and overall evaluation of faculty research, according to one paper written by Florida State University Professor Randall Holcombe.
But Cunningham said the rankings, although important, do not significantly influence the University’s research aims.
“We don’t change how we operate the academic enterprise based on the ratings,” Cunningham said. “They provide us with valuable information that we can use to assess the quality of our programs, but they are just one measure of success.”