April 13, 2008

Excellence in science and technology depends upon the quality of education, especially at the higher levels. The Higher Education Commission (HEC) realised that without developing education in general and science and technology in particular, our country will continue to lag behind.

The HEC has thus brought about a paradigm change in the policy on tertiary education in order to bring about a change in the quality of our education in general and science and technology in particular.

A greater proportion of our gross income, GNP, is now being spent on science projects, scientific laboratories and awards and rewards. But social sciences are completely ignored. Many branches of science have simply been considered useless through propagating a utilitarian approach. Some branches of mathematics have been annihilated. A typical scientific cult has been created through appointments with huge salaries. Too much interference in the affairs of universities has created cronyism, which has destroyed the scholarship and merit by and large.

While we are spending a huge amount in these endeavours, we would like to make sure at the same time that the money spent is well utilised and the objective is achieved. We therefore have to evolve an adequate system of assessment.

One way to assess the quality of our highly educated work force is to monitor their research. The HEC uses an index called the Impact Factor (IF) to measure the quality of research in science. Since the HEC and the Pakistan Science Council often use this data for faculty assessments, it is but natural to expect that it should give us a fair assessment of one’s scholarship. However, it is evident now that this index gives a misleading impression of the relative standing of certain scientific journals.

Quantifying scholarship, in my opinion, is not an easy assignment. But we must also at the same time remember that scientifically developed nations must have evolved some kind of a system, which appreciates quality amongst scientists and technologists and discourages substandard scientific and technological research. Perhaps this is one of the reasons that the money and effort spent upon science and technology in these countries is paying back effectively. Of course it is reasonable to think that this is why they are doing so well in science and technology.

Below I would like to describe, taking the subject of mathematics as an example, why the Impact Factor, as it is calculated today, is not suitable for scientific journals and I would like to open a discussion on how to change the present method of IF calculation to reflect the realities of research.

First, let me explain how the impact index is computed. The IF of a journal for year Y (say, for Y = 2003 is the ratio C/N, where C is the number of citations during year Y (i.e., 2003 in our example) of papers published in a particular journal in years Y-1 and Y-2 (i.e., 2002 and 2001). N is the number of published papers during these two years (Y–1 and Y–2) in this journal.

The controversy over the use of the Impact Factor as a criterion for ranking scientists began with the publication of a list in 1999 by the Pakistan Council of Science and Technology (PCST) entitled The Leading Scientists of Pakistan. PCST published a similar list the following year entitled Scientific Research in Pakistan. A campaign against the use of the Impact Factor to rate scientists was launched as early as in 1999 and I wrote articles against it in one of the newspapers here.

On April 3, 2001, I was invited to speak on the pernicious effects of the Impact Factor on scientists and science in Pakistan at a seminar at the Ghulam Ishaq Khan Institute. I opined that the use of the Impact Factor would adversely affect the already poor state of science in Pakistan. The seminar generated an interesting discussion during the question and answer session.

In August 2001, the use of the Impact Factor to measure the worth of scientists in Pakistan was criticised at the forum of the second Pure Mathematics Conference 2001 in Islamabad. Mathematicians at the conference objected to the fact that a criterion, which was originally designed to measure the worth of scientific journals, was being used to measure the worth of scientists in Pakistan. This they said, in effect rendered mathematicians as the least productive of scientists in Pakistan. The Impact Factor was denounced as an inaccurate and poor attempt to quantify the subjective matter of praising the worth of a scientist.

Meanwhile in January 2001, I approached the Federal Ombudsman and filed an application against the government’s use of the Impact Factor to rate scientists. A hearing was thus held in which both sides argued their respective viewpoints. Finally in September 2001, the Federal Ombudsman issued its verdict in favour of the complainants.

In its findings dated September 26, 2001, the Federal Ombudsman wrote: “... it is acknowledged that the Impact factor criterion has its limitations, which are all the more significant in the case of disciplines of physics or mathematics. It would certainly be unfair that those in the aforementioned disciplines be judged wholly and solely on the basis of this criterion for the purpose of promotions, awards, research grants, etc. The same would amount to injustice and hence, mal-administration as defined in Article 2(2) of President’s Order No 1 of 1983. It is therefore recommended that the Impact Factor criterion may not be taken as the sole criterion for the assessment of individual scientists, especially those in the discipline of mathematics ...”

Proponents of the Impact Factor readily admit that the use of the criterion to assess the work of scientists has its defects. The then minister of science and technology, Prof Atta-ur-Rahman, had himself admitted “the drawbacks which the Impact Factor assessment process has” in a reply letter dated August 10, 2000 to me regarding the Impact Factor.

The then Dean of the Faculty of Natural Sciences, Quaid-i-Azam University, Professor Qadir, in an article advocating the use of the Impact Factor, also wrote that there were “many problems” associated with the Impact Factor.

The main counter argument of the proponents of the Impact Factor is that a rough yardstick for assessing the quality of scientific productivity in Pakistan is better than no yardstick at all. Both Professor Atta-ur-Rahman and Professor Qadir have stated this.

Critics however strongly believe that the negative impact and damage to science arising from the Impact Factor’s drawbacks and many problems will more than counter any advantage which the proponents say using the Impact Factor will have in Pakistan. I have been looking into the question of the validity of information gleaned from the journals’ Impact Index and have noted that, for the mathematical community at least, this index gives, in some cases, a misleading impression of the relative standing of journals. This data is now being used by our universities in faculty assessments for financial and academic awards, for awarding projects, etc.

Researchers who work in mathematical areas, which though very theoretical, have some resemblance with applied sciences, have wider range of journals to publish in them. Not only that it increases the probability of getting a paper published but also the process of getting it published is much faster than the journals which are exclusively meant for mathematics. These journals have much greater Impact Factors than the mathematical journals not because they are qualitatively better but because they have a wider readership and the time spent from acceptance of a paper to its publication is much shorter.

Also, the PCST uses the list of Impact Factors published by Thomson Company, which lists only 255 mathematical journals, whereas the list of journals with a non-zero impact factor is six to seven times the list produced by the MathSciNet of the AMS which has created its own Citation database.

This puts mathematicians in general and pure mathematicians in particular at a very disadvantageous position. The effects have been adverse. Young researchers are already feeling discouraged and out of favour. Due to economic and social pressures, they are already giving up research in pure mathematics. The number of students registering for MPhil and PhD in pure mathematics is on the decline.

Mathematics, especially fundamental research, has to be saved. It is with this aim that I am trying to correct the existing structure with the hope that our mathematical community may influence Professor Dr Atta-ur-Rahman to change the system for mathematics and create (and use) an impact factor suitable for mathematics.

At the 10th National Seminar organised by the Pakistan Mathematical Society, I explained what the impact factor is and described why the impact factor, as it is calculated today, is not suitable for mathematical journals and how to change the present method of impact factor calculation to reflect the realities of mathematics research. Ms Aneesa Tahirkheli, the State Minister of Education, who was the chief guest at the seminar, agreed that the use of Impact Factor needs some modifications to suit the very nature of mathematics. She agreed to form a group of representatives from the Ministry of Education, the HEC, and the Pakistan Mathematical Society, to sit together and help bring the necessary changes for improving the system of assessment of education and research in mathematics.

The deplorable state of mathematics in general and pure mathematics in particular is worsening rapidly. Wrong policies towards mathematics have already started revealing the damage caused to the proper growth of mathematics. The recently introduced condition for PhD students to produce at least one research paper with non-zero impact factor will deter students from doing PhD from Pakistan universities. A PhD from good foreign universities in developed countries have become more attractive for they require less rules and consume less time than Pakistan’s universities, none of which is considered of international standard.

Only a few mathematicians, who are hand picked members of an unofficial so-called “Task Group for Mathematics” and their friends are benefitting from these policies and earning millions of rupees for producing rehash of their research work with multiple authorship as marriages of convenience. It is better that we adopt a more pragmatic approach to this problem rather than a protectionist approach.

The HEC has spent a huge amount to purchase the list provided by the Thompson Company which is favourable to a fraction of individuals working in the area of so-called applied mathematics. The advices and protests of the Pakistan Mathematical Society, which is the only NGO representing mathematicians at the national level, have been ignored since its inception in 2001.

The writer is president, Pakistan Mathematical Society