Monday, November 05, 2007

USM "Excellent", No "Less Satisfactory" or "Weak" Universities

Looks like the 'rankings' for the public universities are out before those for the private universities. Thanks to a friend from USM for letting me know about this NST article which came out yesterday.
17 out of the 20 public universities in Malaysia were evaluated and 'ranked'. Only 1 university made it to the "Excellent" category - USM. 6 universities made it the "Good" category and the remaining 10 were put in the "Satisfactory" category. None of the 17 universities evaluated were put in the "less satisfactory" or "weak" category.

According to the NST report,

And only one university - Universiti Sains Malaysia - made it into the second category of "Excellent"in the first ever university perception exercise in the country. It received a five-star ranking.

Universiti Malaya (UM), Universiti Kebangsaan Malaysia (UKM), Universiti Putra Malaysia (UPM), Universiti Teknologi Mara (UiTM), Universiti Teknologi Malaysia (UTM) and Universiti Islam Antarabangsa (UIA) were lumped in the "Good" category. They had four stars.

Those in the Satisfactory list are Universiti Malaysia Pahang, Universiti Malaysia Sabah, Universiti Malaysia Terengganu, Universiti Malaysia Perlis, Universiti Malaysia Sarawak, Universiti Pendidikan Sultan Idris, Universiti Sains Islam Malaysia, Universiti Teknikal Malaysia Melaka, Universiti Tun Hussein Onn Malaysia and Universiti Utara Malaysia.

None fell in the "Less Satisfactory" or "Weak" categories.
The Academic Reputation Survey (ARES), carried out by a team spearheaded by the National Accreditation Board (LAN), involved 17 of the 20 public universities.

First of all, I think it is a good first step for the MOHE to even make these rankings available to the public. I'm always in favor of having more rather than less information in the public realm. So I would commend the Minister for making this move.

Secondly, given that this is the first time that such a task has been undertaken in Malaysia, one would expect there to be a lot of flaws in such a ranking system. Even the Minister, Tok Pa, acknowledge this when he said that the results of the Setara rating system, which ranks public universities in three categories, namely, research universities, broad based universities and specialised universities, would not be released to the public as of now because of problems with the data and methodology. We would expect the methodology and the data collection process to improve over time.

These positives aside, I have to point out the flaws, of which there are many.

Firstly, the ranking system should not be based on survey data only. Some more objective data should be used in conjunction with survey data. Perhaps because the ARES specifically aims to capture public perception (or more accurately, the perception of universities among the academia and some members of the corporate world), we should not expect objective data to be included in this ranking. But, hopefully, such measures will be calculated and included in the Setara rating system.

What sort of objective measures? I can think of a few - the % of lecturers with PhDs and / or Masters, the average GPA score of students admitted, the ratio of lecturers to tutors, library book holdings, IT expenditure per student, just to name a few. The use of more objective measures should complement and balance out the measures calculated using surveys. Of course, one could still disagreement of the exact type of objective measure to use since some will obviously be more advantageous to the more established universities e.g. the average GPA score of students admitted. But I don't really have a problem with this given that these so called 'advantages' usually translate into a better academic environment in these universities. Would someone not rank let's say Harvard higher because it has a much higher average GPA entry requirement compared to let's say Diablo University?

The use of these more objective measures also forces the universities and the MOHE to collect these statistics, some of which are important to researchers as well as to interested parents, students, policy makers and so on. Wouldn't you be interested to know what the average GPA entry into UM is compared to UUM? Wouldn't you want to know how much more UKM spends on IT compared to USM?

Secondly, the high non-responses among the respondents who were sent the survey should be reduced. It was reported that the ARES questionnaire was sent to 954 respondents and but only 272 responses were received. That is a response rate of less than 30% which raises possible questions of whether the responses were biased towards any direction.

According to the report, the respondents were replied were broken down into the following categories:

242 public and higher education institutions, two Asean universities (National University of Singapore and Institut Teknologi Brunei), nine corporate bodies and 19 professional and certification bodies.

The large number of respondents from 'public and higher education institutions', which I'm assuming are institutions in Malaysia, probably explains why there were no institutions in the 'less satisfactory' and 'weak' categories.

Intuitively speaking, there should be more variance in the rankings of these universities beyond the top three categories, especially given that I didn't even recognize the names of some of the universities listed e.g. Universiti Malaysia Pahang (where is this university located?), Universiti Malaysia Terengganu (which I'm guessing used to be designated as a college), Universiti Malaysia Perlis (can Perlis support its own university?), Universiti Pendidikan Sultan Idris (I thought this was primarily a teacher training college?), Universiti Sains Islam Malaysia (never heard of), Universiti Teknikal Malaysia Melaka (no idea), Universiti Tun Hussein Onn Malaysia (is this new?). I haven't goolged these universities individually but if any of our readers are familiar with any one of these universities, please post a comment and enlighten us.

The fact that so many of the respondents were from our public universities probably meant that many of them didn't feel comfortable in giving some of these lesser known universities a 'less satisfactory' or 'weak' rating. I think even if the sample size included more corporate bodies or even overseas academics, it would run into the challenge of many respondents who cannot evaluate the universities which they have not heard of or are unfamiliar with.

The fact that many of our public universities are relatively new and unknown is probably why the following three universities were not included in the survey - University Darul Iman, Universiti Malaysia Kelantan and Universiti Pertahanan Nasional Malaysia (what?).

This being said, I think that the rankings are indicative of the relative 'quality' of our public universities. Most people would say that USM, UKM, UM are probably the top three public universities in Malaysia followed by UPM, UIA and perhaps UITM and UTM. The fact that USM was rated 'excellent' is perhaps a validation of some sorts for the more progressive policies of promotion and emphasis on research which the VC there is carrying (Dzulkifli Abdul Razak whose columns regularly appear in the NST).

What we require is a further refinement of the ranking system and more objective measures to be used in conjunction with survey responses.

I'll stop at this point and let our readers weigh in with their views.


Anonymous said...

Universiti Tun Hussein Onn started out as Institut Tun Hussein Onn under UTM, and moved to being a separate institute. Rather quickly (in the span of two to three years) it moved from Institut (ITHO) to Kolej Universiti (KUiTHO) to Universiti (UiTHO). My brother was the third batch (or second, I'm not too sure), back when it was called ITHO. He graduated, I believe, one year before it renamed into UiTHO.

It's a family thing (going to new universities) - I myself am in a new university (Singapore Management University)

As for Universiti Sains Islam Malaysia, whenever I travel back to KL, I notice huge signboards for the exit going to USIM. It's somewhere between Nilai and Sg Besi. Weird looking campus, to boot.

And the teaching university became a teaching university in 1997. Sadly, it probably had a better name (as in reputation) as a training college then now as a university.

The rest, I never heard of them

Anonymous said...

I was once on a train from London to Swansea, and sharing the same coach with a group of local University students who were discussing which university is better or the best.

Overhearing their comments made me laugh and reminded me of the issue of which of our universities are the best?

These students said almost each and every one of the universities got something to claim that they are the best or better :)

Each of the University may pride itself being the best using different yardsticks or baselines as the University.

It is all a marketing method. University A is the best in terms of THES index. University B is the best because it won the UKAS award. University is the best it won the Chinese newspaper award. USM is the besy because it is the top in the LAN test. UITM is the best because it is the best Bumiputra university. UM is the best because it is the apex university,

So every university can claim something. All win awards, so all are the best! hehehe!

There is not one single uniform benchmark to compare all the university

Until that is reached and agreed upon, we all can claim anything!

This syndrome of classifying universities is just a waste of time and money! What the universities should be doing is earnestly try to improve the quality of their graduates, make them acceptable in the market and by the employers. That is the acid test.

What is the point of claiming this or that university is the best when the graduates cannot English, think critically and are not pro active in solving problems?

Unknown said...

First, I like to state that, I am appalled by bloggers who comment out about a scenario that he seem not to know much about/ or do not have updated knowledge.

FYI much of the university that you never heard of, are the university that have been upgraded from university college status earlier this year.

Two, Whats more unbelievable is that the information about rankings etc. is not available on MOHE website.

Anonymous said...

Itu MOHE suka buat kerja ' hangat hangat tahi ayam'

Anonymous said...

Looks like some of our universities are looked upon as excellent or good in a NST survey. That's good enough for me. All is well in Malaysia.

Let's stop all this nonsense about improving the education system and get back to more serious topics like which politician is shagging which artiste or how large a margin of victory we should give BN in the next election.

Anonymous said...

Look at the end of the day, no one with the right mind will take these rankings seriously. As long as academic staff are hired and promoted based on race and religion, they can rank the Malaysia-Boleh universities anyhow they like.
If they are serious about rankings, they would have contracted the work out to an independent body like THES index. The ministry is known for manipulating data, eg, look at the way they refused to include the Malay students in UITM as part of the overall annual intake of university students.

Anonymous said...

Talking about THES, I don't think I would trust them either. Being a commercial outfit, they can be influenced. Even for their own ranking, THES database is not complete by omitting many of the top tier universities in the US. How do you trust their ranking when many of the universities don't even exist in their database? I know because I was asked to participate in their survey.
From the way a govt minister mentioned about THES, I suspect Malaysian unis will come out OK in the coming ranking. So, get ready to hear their trumpets.

Anonymous said...

the good:

- tok Pa becomes the hero by introducing the index
- all universitires are good
- improve universities' paper qualities
- telling everyone in the world we have 20 universities aside from a jumbo cabinet

the bad:
- in every ranking there should be good, average and bad in layman's term, so where's the bad
- it is a waste of resources, as there is no use of such ranking as long as quantity surpasses the importance of quality during intake
- only two foreign universities were involved, or a mere 2/272 tells the truth
- the methodology used in quantification, too many weaknesses

another projek barisan najis, with UMNO as the pillar - Najib

Anonymous said...

Cant we admit the fact our universities are lousy?

Anonymous said...

A friend of mine who read Science at UKM once confided in me that his supervisor had asked him to "modify" the data for his research thesis, which resulted in a statistically significant finding. And this is the same university in the Good category. Sigh.

Unknown said...

academic dishonesty is everywhere, remember the Korean scientist? Last I heard he is trying to resurrect his career in Thailand.

If the case is reported obviously the Prof. will be censured but as your friend is also eager to get the degree then he only confide in you.

Anonymous said...

This syndrome did not occur in 70's or 80's. Academic plagiarism or manipulation is most unforgiveable.
That is why before when we choose reknowned external examiners with illustrous publications who specialise in the field of research he will not bow down to such low ethics. His name and reputation is at stake.
That is also the reason why such manipulated papers do not get into reknowned journals as the referees would know!
In overseas such as UK the external examiner takes active part in the oral or viva to assess the candidates. Here, the board of examiners often do not invite the external examiner and just read the report submitted by the external examiner ( from which third world country?)
Its up to the board to decide irrespective what the report of external says

Anonymous said...

our universities are not APEX university. FULLSTOP. why do we have to keep trashing our own universities? like trashing your own brother. geesshh

Anonymous said...

Dude, it is not about thrashing. It is about revealing facts which should be brought under attention. If one can't take the constructive criticisms, then one will not improve. Such a simple reason.

Anonymous said...

Those 'new' universities that you have never heard of are mostly the university college or 'kolej universiti' which have been upgraded to university. For example the Univeriti Malaysia Terengganu was formerly Kolej Universiti Sains dan Teknologi Malaysia (KUSTEM), Universiti Teknikal Malaysia Melaka was KUTKM. Those new names can be a bit confusing.

Anonymous said...

I am surprised they ask NUS. I thought we hated Singapore so much, and always believe ourselves to be superior to Singapore, to the extent that we will n-e-v-e-r acknowledge any achievements by that nation.

This looks like a positive step forward. I am happy that at least, we are no longer in super proud and denial mode.