Tuesday, October 10, 2006

Why Duke shouldn't be ranked 13th in the THES rankings

We've often aimed the spotlight at our public and private universities in Malaysia. For good reason, since our blog is called www.educationmalaysia.blogspot.com. But sometimes, a fact that is called to light must be clarified, even if it involves a non-Malaysia university. In this case, the university in question is Duke University, located in Durham, North Carolina, USA, and coincidentally where I'm currently doing my PhD in Political Science.

I have to thank Richard Holmes, a regular commentator on this blog and someone who has kept a very close eye on the THES rankings, for his expose on why Duke shouldn't have been ranked so highly on the THES rankings in 2005 as well as in 2006. You can read the details here and here.

I'll briefly summarize Richard's findings. He found that QS has once again made an error by wrongly inputting the number of undergraduate students at Duke into the figure for the number of faculty at Duke. The result of this was that Duke obtained the highest score in the faculty-student ratio category in the 2006 THES rankings and every other university's score was normalized against Duke's.

So the question now is this - where would Duke have ranked if the number of faculty was corrected inputted as 1,595 as opposed to 6,244?

I compiled the faculty-student ratio for all the universities in the top 200 based on the 2005 THES rankings from the following THES website and recalculated Duke's score with the right number of faculty inputted. Instead of the 66 points it scored in the 2005 rankings, it should have scored a 17, giving it an overall score of 44.2 (normalized against Harvard) instead of 58.7. This would have dropped Duke from being placed 11th in 2005 to 35th, just above University of Michigan and below Manchester University and UMist.

How about for 2006? I don't have the detailed faculty-student ratio for all the universities in 2006 (THES has not released this information yet) but according to the 2006 rankings, Yale was ranked second to Duke with a score of 93 out of a 100. Yale, according to the 2005 data, has a 0.33 faculty-student ratio (or about 3 students to a faculty member) while the correct data for Duke shows a 0.13 faculty-student ratio (or about 8 students to a faculty member). Hence, Duke has approximately 40% of Yale's faculty-student ratio.

Taking Yale instead of Duke as the top scorer in the faculty-student category and recalculating Duke's score to reflect that it has approximately 40% of Yale's faculty-student ratio would give Duke a score of 40 in this category as opposed to 100 previously. This would drop Duke's overall score (normalized against Harvard's) from 68.3 in 2006 to 50.9. This means that Duke would drop from being ranked 13th to being ranked 40th (a drop of 27 places) to just above Manchester University and just below Geneva University.

Given this gross error on the part of THES / QS, I think it is only appropriate that they own up to this and recalculate their overall scores and ranking.

Does this mean that I personally think that Duke is not such a great university or that it is only slightly better than Manchester University? Of course I don't!

Duke scored an inconceivably low score of only 39 in the peer review category which accounts for 40% of the overall score. When compared to other similarly ranked US universities (according to the US News and World Report), this score seems pretty implausible: Cornell (60), U of Chicago and Columbia (both 57), Princeton (68), Yale (72), Stanford (82) and Harvard (93). My subjective score for Duke would be somewhere in the range of 60 to 70. Even with the corrected faculty-student ratio score, with a peer review score of between 60 to 70, Duke would easily make the top 20 universities according to the THES rankings.

Having studied at two other top 20 universities (according to the THES rankings) - the London School of Economics and the University of Cambridge - I can say that my experience at Duke in terms of faculty, resources and colleagues have been as good as and in many ways better than my experience at LSE and at Cambridge.

Any ranking system will have its share of shortcomings and will usually be subject to intense scrutiny and criticism (just google "US News and World Report Rankings Shortcomings" for a sample). The relatively new THES worldwide university rankings is no different. Hopefully, THES will be open to some of these criticisms and be transparent about the way it collects its sample / responses and tabulates the final results. And hopefully, university administrators would stop for a moment to examine the details contained in these ranking systems before rushing to claim credit for obtaining a high ranking. A 'correction' within a single category can easily lead to egg on one's face.


Anonymous said...

anyone noticed that THES left out University of Durham in their rankings? Either they forgot it exists or that they ranked it so low that it doesn't matter. That's a major fault, imo. Durham is arguably top 5 in the UK (some even rate it as 3rd after oxbridge).

Anonymous said...

Actually, there are still many excellent universities from many countries not in the list. When they are eventually in the list, we will definitely say 'bye, bye' to UKM, UM, USM, UPM, etc.

Kian Ming said...

Actually, Durham University is ranked at 132 in the 2006 rankings down from 83 in 2005. Durham University has a great reputation in the humanities but I'm not sure about how strong it is in the sciences and social sciences.

Anonymous said...

in the US alone, there are so many more well-qualified universities that are not in the list especially the state schools. then there's european and east asian as well as indian unis that definitely deserve a place too.

bottom line - there will never be a complete and comprehensive list to rank universities globally. perhaps, maybe only the top 50 (e.g. harvard, ox, cam, mit, stan, etc.) can be safely ranked but all other schools are simply impossible to rank.

casper said...

Also, ultimately, it is the strength of the particular faculty that you study that really matters. For example, I am not sure if I want to go to Cambridge or Harvard to do engineering - Imperial or MIT would have been more appropriate.

Given Kian Ming's revealed above - looks like Uni who knows a bit of PR skill to play with the ranking data can indeed make their ranking better :-) Maybe what UM needs is the service of Ogilvy :-)

Anonymous said...

Dear Tony ,

Do you intend to "indirectly" telling us that you are currently doing PhD ?

Does North Carolina produce good PhD graduate ? Never heard before !!#*

Anonymous said...

Durham is great for undergraduate physics.

stocktube said...

i think PR play a major role ... try to get a country of which the govn anti these THES evaluator and i bet my last dollar that you won't make it ...

anyway i think it's time to drop this bombshell into UM since they've been complacent for so many years already ...


Anonymous said...

UM has long been dead!
Try visiting the campus. The scenario is more like a sleepy kampong than an ivory tower.

Anonymous said...

To Anonymous Wed Oct 11, 01:05:04 PM:

It might have escaped you that the post is actually written by Kian Ming, not Tony. And the fact that Kian Ming clearly stated that he is currently doing his PhD in Political Science, that's him being pretty direct about it, so I really don't get your "indirect" point.

All I can say is, Duke University is well known enough for the Fulbright Scholar Programme to offer a scholarship to Kian Ming to do his PhD.

Have you heard of the Fulbright Scholarship?

-roll eyes-