Thursday, December 11, 2008

Advice to UM VC: Clean up the promotion process

Below is an edited version of an email I sent to an academic in UM whom I've been corresponding with. I received news from this academic that the new UM VC was being pushed to bring UM back into the top 200 in the THES rankings within 2 years and back into the top 100 within 5 years. One of the things he's been trying to do is to 'force' UM academics to publish a minimum of 2 articles in journals which are considered 'impactful' journals (or ISI journals). My advice to the UM VC: Concentrate on cleaning up the promotion process instead of being distracted by this pipe dream of climbing the THES rankings.

Apologies for the late reply. Was overdosing on turkey yesterday.

This is what I wrote to the new VC in a previous post:

P.S. One of the first pieces of 'advice' I would give to Dr. Jasmon is to read the measurements in the THES rankings very carefully. That way, he can avoid making some of the same mistakes made by his predecessors. For example, Rafiah probably should not have tried to take credit for 'improving' UM's ranking from 246 to 230. After the top 100 schools, the differentiation in terms of scores between the rest of the schools is miniscule at best. A measurement error could easily push the ranking of a school from 250 to 210 or from 250 to 290.

Secondly, I would 'advise' him to stay as far away from these rankings as possible. Play down expectations by saying that UM is in no position to compete with the top universities in the world. Instead say that UM is trying to consolidate its academic resources in specific areas and trying to slowly but surely increase the % of PhD holders among its faculty as well as their publication records. This way, he can divorce himself somewhat from the vagaries of the THES ranking system.

Do you think he really understands how stupid the demands of his masters at the MOHE are? Top 100 within 5 years? Try getting all your faculty to have PhDs first.

Here are a few responses to what you've said below.

I totally agree with what you've said about the need for a customized approach towards trying to improve the different faculties and departments.

Here are a few of my thoughts on this and we may differ / disagree on some of the finer points perhaps because of my US training and also perhaps because of my field.

I think one of the most effective things which the new VC can do is to improve and make more transparent the promotion processes in the university. If he can do this, he'll be able to give the right incentives for the different academics to publish and make an impactful contribution to the academia, either in Malaysia or internationally. He should put in place certain processes which ensures that only the right people are promoted and that those who do not contribute academically should not be.

Here is what I think the criterion for evaluating promotions should be:

1) The number of papers in peer reviewed journals
- I really think that it's important for one's work to be evaluated by one's peer and that publications in these sorts of journals should count more than publications in journals which are not peer reviewed, even though some of these journals may be read more by the 'practitioners' in the field.
- We probably may disagree on this but I do think that there are tiers in peer reviewed journals. Some are obviously more 'presitigous' and harder to get published in than others. I think our scholars should be rewarded for being published in the more prestigous journals compared to the less prestigious ones. These would make our scholars more well known internationally as opposed to being published in less prestigious journals which may be read by more people in the region i.e. Asia or South East Asia.
- But I think the disagreement here may be more on an academic rather than a practical level. My impression is that the problem among our academics is that many of them don't even published regularly in peer reviewed journals, whatever the quality. My impression is based on my interactions with those in the political science faculty so I apologize in advance if this is not reflective of the faculty in your department. My guess is that there are probably easily over 200 political science faculty spread across the different public universities in Malaysia. Based on the stuff I've read in political science peer reviewed journals (and I define this quite loosely), there are probably 10 academics in our public universities that have actually published anything in these journals. I have no idea where the rest of the people publish but I'm guessing that they publish mostly in Malay journals which may not be peer reviewed and also Malay academic books which are probably not peer reviewed as well. I'm guessing that many of them don't even make the bar using these much 'looser' requirements. So for these academics, getting them to publish something, anything, is already a good start. Forget about the ISI journals.
- Let me give you an example from personal experience. I just finished completing a 1st draft of an article which I think has a pretty good chance of being published in one of the top political science journals. If I do make the grade, I'll probably be the first Malaysian political scientist to get published in a journal of this quality. To write the paper which I'm writing, I had to spend about 1 year collecting elections data from 152 countries. After that I had to do case study work for about 30 of those countries. And then I had to many regressions to test the modes which i wanted to test. I just sent the paper out to about 20 people for comments and feedback. After the feedback comes in, I'll have revise my paper again and probably run more regressions. And then I send it in to the peer reviewed journal. Wait to see if it is accepted. If it is, it will come back with comments from 3 reviewers and I have to amend my paper to take into account those comments. From the start of this project to when I hope the paper will be published will be 2 years, give or take. All this for a 35 page paper. I cannot imagine many Malaysian political scientists who may not have had the luxury of the training I've received of going through this process. If I were the head of the polisci department of a Msian public university, I would encourage the faculty, especially the younger ones, to publish in region journals which may be easier to get published in and go from there.

2) Editor of a edited volume

- Besides writing one or two articles in these volumes, an editor has the additional responsibility of collecting and editing these articles.
- Of course, there are also differences in the quality and impact of different edited volumes. Those which are published by good publishers obviously have more reach and impact than those which are published by less well known publishers.

3) Chapters in edited volumes.

- Although many of these volumes are not peer reviewed (except by the editor, who of course, would have different standards than anonymous reviewers in peer reviewed journals), they can be impact chapters which can genuinely add to the body of knowledge of a particular subject. It may by the preference of some who want to publish something quickly and do not want to suffer through the vagaries of a peer review process.

4) Conference papers

- These are not the same as the above since they have not been converted to papers or chapters. But they should be counted if it is in the process of being converted into something more substantive

5) Other organizational responsibilities

- These should be things which contribute to the intellectual life of the department including organizing important conferences, inviting speakers to come and share ideas, etc...

6) Writing articles for newspapers, magazines etc...

- I would probably benefit from this but I would put it very low on the totem pole. It reflects the responsibility of an academic to be a public intellectual but it is DEFINITELY NOT a replacement for being published in journals or books.

You can see where my priorities lie - they have to be in publishing material, the heartbeat of an academic's life.

I think if you can get the promotion review process, either internally or externally, to be able to take into account, holistically, the career of an academic thus far, the VC will have done a lot in terms of getting the message out there that you cannot be academically unproductive and still expect to be promoted. I still hear stories of how academics who are less accomplished are being promoted over those who are more accomplished because of racial quotas or connections with the VC or the right politicians. If a sound review process is in place, you don't actually have to put in place the 2 ISI journals in one year requirement. You can take a three year cycle of an academic's career and make promotion decisions on a medium term outlook.

Lastly, on the teaching load bit, I'd ask the VC to think creatively here. One possibility is to recruit more Masters and PhD students who can then act as TAs for the professors. This will decrease the marking load for many professors and hopefully allow them to be more productive academically.

Sorry for being so long winded but I've been thinking about this over the last day or so.


Kian Ming


Anonymous said...

Kian Ming, it is not just good publications, but more important, are the citations. A paper who has little good citation, or a lot of stupid citations, are useless too.

Abang said...

Kian Ming, I am lucky to have a friend who worked in NUS during their previous President. He is now in London with a new job.

You see, he is very obsessed with the "research university" idea I see developing here in another form. It costed NUS many many good staff from what I heard. Too much money spent on "research" and too little audit done on how these "researchers" are spending the money. It redirected a lot of valuable resources from the Professor and also Admin staff salaries to make a lot of people disappointed and leave.

And my friend was laughing when he gave me this quote:-
"Research is using money to create knowledge. Innovation is using knowledge to create money."
We must know how to balance both academic pursuits, and money generation so as not to end up with the same failures NUS faced, but did not dare say in public.

Idlan said...

I am speaking only about research in my area, but trying to find Malaysian-based researchers being published in the top 5 or top 10 journals in accounting and finance, and citations of papers written by Malaysian-based researchers, is only made slightly easier than looking for a needle in a haystack via the Google search engine.

I am struggling to understand how academics publish two papers a year in a ranked journal: I have been taught - and subsequently experienced - that to write a top notch paper in accounting / economics that involves experiments / hypothesis testing that is worthy of publication in a top journal takes the better part of a year to 18 months at least, from inception to publication, if not longer. Some top journals have a 3-5 month review turnover process.

So what do we do if we only have 6 months on average to write one paper? Do we end up cutting corners, and where?

Anonymous said...

It means that on average two [apers a year. The academic hv to prepare about 6 papers opr more in a production line capacity
I wonder how many of the two papers a year are vehicles for hitch hikers sych as the head, dept dean, dean etc Are they considered too even tho they do not contribute anything? (part of kawan tolong kawan technology)

Anonymous said...

We should focus on recruitment too.

Without proper academia and good administrators who understand the market, we will be stuck in backwaters even if we manage to get some publications up.

But honestly the new VC will have a very very difficult job.

The current academia and administrators will be too entrenched and have power to stop her initiatives, screw up her life. She needs a lot of support from Minister of MOHE, and even the Prime Minister himself. She needs to tell people, do this, or I sack you, and can sack that person with impunity for the next 5 years.

Then she has to really think about recruitment. There are only so few top and good academia in the world. And all top Universities are offering a lot of incentives to go to them. We cannot focus on recruiting only Malaysian academia, we must seek global talent. And for that, MOHE has to commit sufficient funds.

Next up, we have to recruit really good and experienced administrators who can move our UM forward. Academia are the content, Administrators are the backbone of any good university. And for this, we must also have the budget too.
Example, we try hire the student recruitment heads of NUS, we cannot pay them RM2,000 and expect them to come right?

Then we have also to look at validity of our course contents. I know at top Universities overseas including NUS down south has an content audit which require them to invite oversea Professors from top Universities (eg Harvard, MIT, Carnigie etc) to look at their course contents and give suggestions on any improvements needed. The audit is done like every 3 years.

Then your structure of promotion.

I think discussed before in other thread. Promotions to Assoc Professor must need certain amount of international citations (not lousy ones), and publications. Promotion to Professor must need at least 3 Professor from top oversea University to agree and approve then VC will approve.

And no political appointees to be given "Professor" title. This is such a disgrace to our real Professors.

All I have at the moment.

dingo said...

What's wrong with having a target of 2 publications a year?

What's wrong with having an aim to achieve Top100 in 5 years time?

Got target and action plan already still kena whack. Some people are never happy with anything.

Left Foot Frowning said...

All this ranking and reputation thing is really distracting MOHE from a very imporatant function of Universities.

Universities apart from improving population quality in terms of education, is a center of sorts for the country's technological improvements. Investment in improving our own Universities instead of turning them into a child's game of politics, gives us the tools and advantages of moving into the next cycle of growth and development.

Publications and rankings do help. But if research is confined to only certain areas non-critical of certain parties, few good academia will be attracted.

And University advancement, researches, attraction of top talents has a very important role in the very advanced science of today's world.

If we do not buck up, we will still be buying second rate weapons from Russian, America, and maybe soon, Singapore for our armed forces. And what this means, is that militarily, we can be easily overwhelmed by advanced weaponary resulting from good research of any invader.

The weaponary do not have to be weapons of mass destruction as America has shown. They can be simple electronic devices planted in correct places to disrupt communications. They can be sophisticated social disruption activities found and discovered from advanced social sciences research.

Whatever it is, our country will continue to be backward, and move even more backward, if we continue to play games with institutes of higher learning.

Left Foot Sighing said...

And editors, pardon my brashness.

I am very sad that fellow citizens contributing to your comments are mostly "inferior" to those I see on Singapore ones dicussing their own lives. I am not saying our people are no good. Just that, our thinkings seem to approach more on the cosmetic and surface issues, less on the underlying deep issues.

I cite just one example for readers to compare ( Time for us to really really start being a lot more introspective and see things on a deeper level. And also not to take comments voicing different opinions and criticism as upfront slights to start a mud-slinging competition.

Anonymous said...

Universities should not be blame if they cant attract good academician from oversea..i have been to one country, their lecturer is paid around USD5k = 15k which is equivalent to deputy vc or senior prof in ipta. if we really look at the malaysia government servant salary, then we never can compare with singapore, uk, us, aust, etc. we are far behind. jasmon idea to recruit first class student to pursue postgrade study in um, academician to publish 2 isi journals is a wise move. how on earth do u expect young lecturer/researcher to publish a highly cited papers??? who knows their paper/publication will be highly cited. doesnt mean that you publish in high impact factor journal your paper will be highly cited. to publish is better than not to publish and at least the bottomline is isi journal. my only suggestion to jasmon is that the researcher need to find their own fund to publish or publish in free journal. a lot of free journal out there.

Shawn Tan said...

Personally, I agree that publishing and getting the 'word out' is the most important part of academic life. However, a lot of my peers will agree with me when I say that we are disillusioned with peer-reviewed journals, including the likes of Nature or Science. There are certainly lots of flaws within the peer-review system itself but that's a whole other blog entry in itself.

So, this obsession with being X number of Malaysians published in Y journal seems a bit off to me. It's just the same as being obsessed with getting in the THES Top 100 within Z number of years. While I also agree that research needs to be critically reviewed, there are alternatives to journals.

In the realm of computer science, a lot of good work is being released through on-line mechanisms such as Open Source, which is similar to a peer-review process (with more eyeballs). A similar approach can be adopted for other fields. I'm sure that physical and natural sciences can benefit from a similar mechanism. I'm not familiar enough with social or political sciences to know if it could benefit from a similar process.

Anonymous said...

Top 200 within 2 years is IMPOSSIBLE.To publish good papers means you need good research, top notch professors. Malaysian research and professors are of very low quality. It takes years to build a quality research institution. Poor Jasmon is only given 3 years to do the impossible. Look at the top unis. Their chancellors are given much longer contracts.
Don't kid yourself Malaysia. Top 200 will never happen. Expect our unis to slide further down the rankings.


Kian Ping said...

Kian Ming,

In Australia, the government has spent a lot of money and years of effort to come out with a discipline-specific tiered outlet rankings.

This ARC endorsed list will be used to evaluate research excellence. Most universities will also use this for their promotion exercise. For instance, the criteria for promotion to professor include at least a certain amount of published papers in tier-1 (or A*) journals.

The first draft of the journal ranking list was out a few months ago. I guess they are now collecting feedback from the universities.

Kian Ping said...


To publish a paper in tier-1 journals take more than 18 months. For instance, in my field of finance, the top 3 are Journal of Finance, Review of Financial Studies and Journal of Financial Economics.

If you look at the papers published in these 3 journals, their working paper versions have been uploaded in SSRN for at least 2-3 years. Add up with the time that the researchers take to produce the paper, say 18 months, the whole process takes about 4-5 years before finally seeing 'your baby' in print. For those who demand citation counts, it will be another few more years of long wait.

Kian Ping said...

Kian Ming,

I wish you good luck in your paper. Hopefully, you can create history for being the first Malaysian scholar to publish in top tier journals in your field.

I am still waiting to see who is the first Malaysian to publish in top 3 finance journals.

to Left Foot said...

You mean something like this?

From Bronco to Warthog

ST Kinetics Awarded £150m for Delivery of over 1000 Bronco Armored All Terrain Tracked Carriers to Replace the British Forces' Viking All-Terrain Vehicles in Afghanistan

Earlier this month (December 08) the British MoD selected the Bronco for its 'Warthog' all terrain vehicle, replacing 108 Swedish made BvS10 Viking currently in service with the Royal Marines and British Army. The vehicles are fulfilling a capability gap addressed by Urgent Operational Requirement (UOR) stated by the units in Afghanistan. (more...)

The Viking, a 14.2 ton gross vehicle weight (GVW) troop carrier is configured to carry a payload of 6 tons or 12 soldiers. It was originally selected for the Royal Marines for its multi-role, all terrain, rapid deployment operational capability as a lightly armored vehicle capable of operating in jungle, desert and arctic conditions.

Despite their versatility, the Afghan arena proved too tough for the Vikings. While the vehicles effectively negotiated the terrain and provided limited protection against light threats, additional armor protection required to protect from the heavier threats typical to the Afghan conditions severely limited their operational capability and necessitated rapid replacement. Consequently, MoD was seeking a heavier, better protected vehicle to maintain the high level of mobility and operational flexibility without compromising protection.

Bronco with 18 ton GVW was selected for this role, not only for the added payload capability (which translates to capability to carry heavier armor) but also since its the curb already includes integral armor. Singapore Technologies Kinetics Ltd (ST Kinetics) was awarded a contract of about £150m (about S$330m) for the acquisition. The vehicle deliveries will commence in third quarter 2009, with the majority to be delivered in 2010. Four Warthog variants will be built under the contract - Troop Carrier, Ambulance, Command, and Repair & Recovery. ST Kinetics is Singapore Technologies Engineering Ltd (ST Engineering) land systems arm.

Some 600 Bronco ATTCs are already operational with the Singapore Army. The Bronco's articulated design delivers exceptional mobility across a wide range of terrain and climate. The basic vehicle is delivered with an armor protection which could be augmented to meet MoD requirements, primarily to increase protection against roadside bombs. Bronco will also deliver considerable increases in range, payload and internal capacity over incumbent vehicles currently being used in Afghanistan.

Anonymous said...

Many years ago, my mentor in UM circulated a book called "Apprentice to Genius" in the lab. According to this book, half of the Nobel laureates from America had work with another Nobel laureate as a student or junior staff. Excellence breeds excellence. I believe the way to bring excellence back to UM is to hire excellent staff. At the helm of a top university, one must understand what excellence is. Recently when returned to Malaysia for a visit, I was shock when top management of Malaysian universities would count a journal with ISI Impact factor of about 30 the same way as a local journal with ISI Impact factor of less than 1. With such attitude for counting number but not quality, how can Malaysian university move forward and upward in THES ranking. I believe a VC must must remember that he/she is also an educationist. A VC should develop the staffs and the students to bring up a university. Malaysian universities are having brain tumours. One just can't ease the pain by taking a Panadol. To be the Best, one must hire the Best. UM's hope lies in the strength of her staff. One good staff is better than 10 sleeping ones.