Friday, August 26, 2005

Universiti Malaya: 89th or Nowhere? (Part II)

My earlier post which highlighted the “credibility” issue with regards to the World Rankings Table compiled by The Times Higher Education Supplement (THES), whereby Universiti Malaya (UM) was placed on a respectable 89th position. I have argued that the methodology utilised is flawed and will not produce results that are anywhere near accurate, particularly for universities placed beyond the top 20 or so.

There is however, another university rankings report produced by the Institute of Higher Education, Shanghai Jiao Tung University (SJTU) which is often cited globally (for example, recently by the Guardian in the United Kingdom:
The respected Shanghai Jiao Tong lists 11 UK universities in the top 100, including Imperial College London, University College London, and Edinburgh University. Oxford has slipped two places in the past 12 months and is ranked 10th.

The data, recognised by academics worldwide, is dominated by US universities which get far more money in endowments than UK counterparts. Harvard is top of the list and though Cambridge pushed Stanford into third place, eight out of the top 10 places are filled by US universities.

So, in this rankings table – which ranks the worlds top 500 universities – where are the Malaysian universities placed? Believe it or not, no where. There’s absolutely no mention of any Malaysian universities.

Some statistics provided by the SJTU list:
  • There are 93 universities from the Asia Pacific listed on the top 500 rankings and Universiti Malaya is not in the list. On the other hand, the THES rankings table placed UM at a respectable 24th position for universities ranking excluding universities from the UK and US.

  • There are 34 institutions of higher learning from Japan alone in the above list.

  • Singapore has two universities in that list: the National University of Singapore (NUS) ranked 101st-150th and National Technological University (NTU) ranked 200th-300th. This is also a far cry from the THES rankings table, whereby NUS and NTU are placed commendably at 18th and 50th positions.
So based on this alternative list that you won’t find our Malaysian education authorities and vice-chancellors quoting from, are our Malaysian universities really that poor such that we are unable to crack even the top 500 universities of the world or for that matter, possibly the top 100 universities in Asia Pacific? If the compilation by Shanghai Jiao Tung University is accurate, then it is indeed a damning report on Malaysian institutions of higher learning.

Once again, before we jump to hasty conclusions, let’s review the SJTU rankings methodology (which you can find here).

Unlike the THES study which relied heavily on a survey of some 1,300 academics around 88 countries, the SJTU study relies purely on measured data and statistics such as the number of Nobel Prizes and Field Medals of alumni (10%) and staff (20%), highly cited researchers (20%), articles published in Nature and Science (20%), other citations (20%) and relative size of the academic institution (10%).

Note that the SJTU researchers readily admitted that the quality of universities cannot be precisely measured by mere numbers.
Therefore, any ranking is controversial and no ranking is absolutely objective. People should be cautious about any ranking including our Academic Ranking of World Universities.

It would be impossible to have a comprehensive ranking of universities worldwide, because of the huge differences of universities in the large variety of countries and the technical difficulties in obtaining internationally comparable data. Our ranking is using carefully selected indicators and internationally comparable data that everyone could check.

More significantly, the SJTU researchers have highlighted that the criteria used tends to favour science-based universities. As a result, a university such as London School of Economics which offers only social sciences and financial faculties are placed beyond the top 100. This is because publications for the social science subjects are more varied in nature and are hence more difficult to track.

In addition, “since English is the international language in the academic world, scholars in English-speaking institutions are more closely integrated into the global academic environment than scholars outside the English-speaking world. As a result, university ranking based on research performance may inevitably lead to bias against institutions outside the English-speaking world.”

However, we do not expect the above criteria to have significantly affected the ranking studies on Malaysian universities such as Universiti Malaya as it’s neither a social science-based university nor is it a non-English speaking institution.

More likely than not, the poor placing of UM in the SJTU rankings is due to the fact that there is a heavy bias towards “research” in the rankings as opposed to “education”. UM is clearly not known for its breakthrough research. Hence, UM may claim, as a saving grace, that it’s teaching quality, which is unmeasured, is actually better rated than its research prowess to justify a potentially better placement in the rankings.

So the only question left is if the researchers even included Universiti Malaya in their study? If UM is not included, then obviously it won’t be ranked. It was however noted in their study that they “have scanned more than two thousand universities” and “scanned major universities of every country with significant amount of articles indexed in major citation indices”. As such it is unlikely that Universiti Malaya has been excluded from the study.

Hence, why is there such a vast difference between the rankings provided by THES versus that of SJTU? Which study is more reliable? I’d like to think that Universiti Malaya is a better university than some of the universities that managed to qualify as the world’s top 500 universities in the study by SJTU. This is partly backed by the fact that I would actually prefer the top students from UM over the students from universities such as La Trobe (401st -500th), Murdoch (401st -500th), Newcastle (301st -400th) in Australia. However, as concluded in the previous post, I certainly have my doubts on the ranking of 89th provided by THES for UM which is placed better then renown UK universities such as Durham, Glasgow, Bristol, Bath and King’s College, London.

Even if UM is indeed placed in the more favourable category of 301st-400th ranked universities, it shows that there is plenty to be improved by the university to improve its standards. It is my believe that as THES improves their ranking methodology over time, and assuming that the standards at all universities remain unchanged, UM will find that it will drop down the rankings table significantly.

The fact that the UM Vice Chancellor, Prof Dr Hashim Yaakob who proudly used the results of the THES rankings table, may find that it will backfire on him in the subsequent years should UM slid down the same rankings table – not due to an actual fall in quality, but due to the fact that it was placed “wrongly” in the top 100 in the first place.

More on Malaysia’s ambition and vision for its higher education institutions in the subsequent post.


Ong Kian Ming said...

Hey Tony,

In my discussions with some Malaysian friends in the Malaysian Forum mailing list (a mailing list comprising maining US based Malaysian students), we discussed the THES ranking in depth. If you look at the details, there are two points that are noteworthy. Firstly, UM scored 68 in the international student category, the highest score that UM received in the 5 surveyed categories. By this measure, UM ranks 6th in the world i.e. it is the 6th most internationalized university in the world (even above Monash). USM's international student score is even higher, at 78, making it the 4th most internatalized university in the world. If two of our local top universities has so many international students, then maybe some of our local youth political organizations should direct their ire at the university administrators for letting in so many 'foreigners'. Monash, with a score of 64, has 30% of its student population listed as foreigners. There is no way that UM or USM has a 30% foreign student population at the undergraduate level, at least to me limited knowledge. My suspicion is that THES might have counted non-Malays as 'foreign' students thereby increasing the rating of UM and USM in this category. If this is corrected, as it should be, in the next THES ranking, it wouldn't surprise me if UM and USM drops way down the ranking.

The second noteworthy point is that UM's and USM's Citations / Faculty score which accounts for 20% of the total score is 0. There are 11 schools in the toop 200 with a 0 score in this category and two of them are Malaysian universities. Granted, this score is heavily biased towards the sciences as well as towards English publications (the School of Oriental and African Studies in London and Sorbonne in Paris both score 0 as well) but even then, a large research university with a faculty whom are presumably relatively competent in English should be able to produce a score that is higher than that of let's say LSE, a predominantly social science school, which scored a 6 in this category.

Finally, if you examine THES ranking of Top 100 schools by Science, Engineering&IT, BioScience, Social Science and Top 50 Arts and Humanities, you will find that UM is NOWHERE to be found in these rankings. So the question is, if UM is ranked top 100 in the world, then surely it must be ranked top 100 in at least one of the categories surveyed here, right? Apparently not.

It was so dissapointing for me to observe all those banners around UM when I came back during summer bragging of the fact that they are one of the top 100 universities in the world. When we examine the details, we have many reasons do doubt UM's position among the elite 100 schools. It would be really funny to see what happens if THES improves their methodology next year and UM falls off the top 200. Will the VC then say that the methodology of the survey is wrong? Let's wait and see.

Anonymous said...

Don't be too dissapointed, ranking is just like grades. I am sure there are some area that UM is number one in the world, and in other area number last.

-- Old Man

Boringest :| said...

Although this might seem short-sighted, I feel that there should first be an official ranking among the 16 (is there more?) public Malaysian universities/university colleges before comparing ourselves to international universities.

IMHO, this would be a more realistic form of competition among local universities and we could later graduate to a regional level and finally internationally.

Btw, Tony - do you happen to be a Rafflesian?

Anonymous said...

I Found a news from angkasa malaysia . please visit here