The Analytical Angle: Taking an experimental approach on civil service reform

There is no one-size-fits-all solution and we shouldn’t hold out hoping to find it.
Updated Aug 01, 2019 04:00pm
Illustration by Mushba Said.
Illustration by Mushba Said.

Pakistan Tehreek-i-Insaf’s (PTI) 2018 manifesto calling for the transformation of governance through civil service reform unleashed a flurry of opinions on what exactly Pakistan should be doing to fix the public sector.

Commentators have offered their thoughts on the civil service’s outdated colonial structure, excessive reliance on generalists, lack of tenure options, bloated size, unequal opportunities for career advancement, low salaries and housing perks, corruption, need for e-governance, primacy of the federal service over the provinces and even meta-critiques on the process of reform itself.

While many of these points are valid, the debate as it is currently taking place risks not moving reform forward.

Related: Pakistan hasn’t succeeded in reforming its bureaucracy in the past. Can the PTI deliver?

In order for us to truly create change in the civil service, we need to actually test proposed policies using an approach that is analytically sound, designed for the context and rigorously evaluated and refined.

The good news is that this kind of analytical approach is feasible and can work in Pakistan. We know this because our team has spent the past decade collaborating with the government on experiments that test ways to enhance civil servants’ performance.

One of us, Asim Khwaja, is working with coauthors Adnan Khan (London School of Economics) and Benjamin Olken (Massachusetts Institute of Technology), and the Excise and Taxation Department of Punjab on a series of studies. This collaboration has yielded insights both on how to improve the civil service and how to approach reform more broadly.

Civil servants’ performance can be enhanced by either changing who they are or how they behave. However, replacing civil servants is difficult and disruptive, with no guarantee that their replacements would be any better.

Changing how we reward them, with their pay tied to how well they do their job, can be easier and effective. Indeed, two of our studies in Pakistan demonstrate that offering well-designed incentives to civil servants can significantly improve their performance.

Each study utilised a randomised controlled trial design. A “treatment” group was randomly selected to experience a reform and compared to a “control” group, which continued under “business as usual,” to accurately determine the impact of the policy.

In depth: A look at PML-N's selective anti-corruption drive

In the first study, the research team assigned a treatment group of tax collectors to receive a salary bonus based on their performance.

We found that over two years, tax revenues in circles where tax collectors were assigned to these performance pay schemes had a 46 per cent higher rate of growth in tax collection compared to those under regular pay.


Offering performance pay as an incentive was not only effective in improving tax collection, but it was also cost-effective: the government’s return on investment for instituting performance pay was 30pc. In fact, the returns could be substantially higher since our findings suggest these effects persist over time.

In a second complementary study, the research team examined whether non-monetary incentives can also work — specifically, whether the promise of merit-based postings to desired locations can motivate civil servants to improve their performance.

We found that a treatment group of tax collectors who were allowed to choose their subsequent postings based on how well they performed showed a 41pc higher growth rate in tax collection relative to the control group.

While this growth rate is slightly lower than what we discovered with monetary incentives, the non-monetary scheme had zero additional financial costs (aside from administrative and political costs), suggesting that it offered an even higher financial return on investment to the government. We expound on the results in an essay on the Evidence for Policy Design website.

What does this all mean for the immediate task of civil service reform?

Our findings provide powerful and actionable evidence that incentives matter. Both monetary and non-monetary methods can incentivise performance, and they work even without hiring or firing any civil servants.

A second, broader point is that our research shows that experimentation is the right way to work out the specifics of reform. 
 In order to attain that positive effect in the first study, we tried three different bonus schemes, all of which had the potential to incentivise performance.

We found that in our context, the simplest bonus scheme worked best, even though it rewarded only a single dimension of performance. This “revenue” scheme, where tax collectors were paid a bonus directly tied to the revenue they collected, had a 62pc higher growth rate in tax collection.

The most comprehensive scheme — the “flexible bonus,” where tax collectors received bonuses based on a range of subjective criteria set by senior staff — was the least effective.

Although the “flexible bonus” scheme was based on a more complete set of information, which we thought might make it a stronger incentive, in our context, the simplest scheme worked best.

Ultimately, in order to find the best policy option, one will need to test a range of schemes for each specific situation.

A third point is that experimenting with different options not only lets you identify benefits, it helps you watch out for side effects that can harm citizens. One concern with the simple “revenue” scheme is that only rewarding on tax collection could lead to over-taxation or harsher treatment of taxpayers.

In order to examine whether this would happen, our study collected information on tax assessment accuracy and taxpayer satisfaction. The good news was that we found no adverse impact. The troubling news was that that while some taxpayers indeed pay higher taxes, others instead report offering higher side payments in the new bargain made.

The point here is that collecting detailed information can help you contour the policy for the best possible outcome for all — including coupling rewards with penalties for undesirable behaviour.

Read next: Is the bureaucracy politically neutral during elections?

Finally, one should track the impact of a policy reform over time to help refine it. Findings from both our studies suggested that incentives need not be offered every year, but instead only at key moments for the department.

Our first study found that the increases in tax collection lasted beyond the period of performance pay (because newly recorded properties continued to generate revenue).

Our second study found that tax collectors who were offered merit-based transfers two years in a row were actually less effective than those who only experienced it once (possibly due to the disruption caused by frequent moves).

By understanding the nuanced effects of the policies over time, we can increase both their impact and their cost-effectiveness.

These findings can specifically help inform civil service reform in Pakistan, but they also speak on a broader level on how to affect policy change. For successful reform to take place, we need to create an environment where multiple — and often contrasting — ideas can be rigorously tested, compared and refined.

While hypothetical debates may offer the opportunity to put forward a range of policy options, we will never find resolution through highly partisan argumentation or the endless airing of expert opinions on talk shows.

There is no one-size-fits-all solution that will resolve all problems facing a nation, and we shouldn’t hold out hoping to find it. Any one study can be highly informative, but for real change, we ultimately need many such studies.

By systematically testing policies and making a case for them based on hard data, we can create a structure to discover the best reforms that stick and thrive.


The Analytical Angle is a monthly column where top researchers bring rigorous evidence to policy debates in Pakistan. The series is a collaboration between the Centre for Economic Research in Pakistan, Evidence for Policy Design at Harvard Kennedy School, and Dawn.com. The views expressed are the authors’ alone.

Email


Author Image

Asim Ijaz Khwaja is the Sumitomo-Foundation for Advanced Studies on International Development Professor of International Finance and Development at the Harvard Kennedy School, Co-Director of Evidence for Policy Design, and co-founder of the Centre for Economic Research in Pakistan. Follow him @aikhwaja

Author Image

Tiffany Simon has served as a Senior Research Manager at Evidence for Policy Design and is now a PhD student at Princeton University.


The views expressed by this writer and commenters below do not necessarily reflect the views and policies of the Dawn Media Group.

Comments (16) Closed

Tejaa
Mar 27, 2019 10:03am
Try to use technology in decision making as much as feasible, making human element as much less. Develop systems and processes, so to reduce both corruption and favoritism. Accountability and performance should be the hallmark of administration. Strict regulations need to be developed to curb corrupt elements in the administration. Subject matter experts need to be hired/placed at top positions in departments of their relative expertise.
Recommend 0
Mustafa
Mar 27, 2019 12:27pm
insightfull write up.
Recommend 0
Amer
Mar 27, 2019 01:42pm
Did the selected group know they were selected? It could be the 'Hawthorne effect': change of behaviour just because the subject is being observed.
Recommend 0
Fiaz Khan
Mar 27, 2019 02:12pm
That’s why we have officially sanctioned corruption
Recommend 0
SN
Mar 27, 2019 02:47pm
What of the manner in which these taxes are collected? The collection quotas have been incentivized, but the ground facts are that the self-same taxes are typically collected through coercion and under duress to fulfill the quotas.
Recommend 0
fahad
Mar 27, 2019 03:40pm
It is common sense that incentives increase motivation level of workers and give them reason to toil hard. It is just centuries old lesson.
Recommend 0
umer
Mar 27, 2019 04:24pm
Wow! What a revolutionary piece of research. More perks and incentives for the ruling elite. Allow them postings to much sought after locations i.e larger ponds where like sharks they could prey on an abundant supply of both large and small fish.
Recommend 0
Pervaiz Ahmed
Mar 27, 2019 04:43pm
There is a missing point which has been part of the old CSS system where it has now been replaced by the subject specialists. Like an MA Islamiat person is just the head of Health sector and a Doctor by profession is heading the POLICE department. whereas a language professional is heading the Audit and Accounts department. and the person who have CA ICMA and ACCA degree are not eligible for the job. Time has come to demolish this old rotten system and give birth to a meritocratic system
Recommend 0
AW
Mar 28, 2019 12:04am
It is not about financial incentives. Each grade 19 officer is costing about $10,000 per month to the public exchequer in terms of salary, government provided housing, cars, utilities and servants which no country can afford and Pakistan certainly not. Secondly the centralized CSS process to recruit generalists needs to be discarded and replaced with department wise specialty recruitment examinations and selection process to select specialists.Thirdly, the system of governance needs to be decentralized for improving public service with accountability
Recommend 0
Shehryar khan
Mar 28, 2019 01:37am
Reforms in civil services is a highly debated matter but no concrete steps has been taken so far to transform it and improve its efficiency. This obsolete system is not working properly due to shortage of expertise of the officials holding the chairs in their respective fields. Now it is right time to work on it and to improve its structure.
Recommend 0
Siva D
Mar 28, 2019 03:19am
Kudos DAWN for fostering rational and informed articles.
Recommend 0
Israr Ahmad
Mar 28, 2019 04:58am
@Pervaiz Ahmed, Exactly You are right!
Recommend 0
Azhar
Mar 28, 2019 11:00am
I don't get the super technicalities around incentives. There has been no civil service reform since 1973. Period. It is a rotten outdated system dominated by powerful vested interests. I am happy for them to be serving the masses. Incentives can only be commensurate to what they deliver. They are only one side of the coin. and if the bureaucrats want to be weighed in gold aka the private sector they must also be ready to be fired in case of non-delivery.
Recommend 0
Shahid Hassan
Mar 29, 2019 03:09pm
For a country of 220 million over-centralized structures cannot work even if manned by angels.Reformation of police, admin., tax, policy. economy, health, education, agriculture, ecology etc all need to be transferred to district levels for the benefit of the people. Trying to reform old, worn out, dilapidated and downright corrupt structures will not work. Unless far reaching, decentralized and people friendly reforms are instituted the suffering of the people will increase and the travesty of mis-governance will continue to ravage the land. Moreover foreign suggested (mis) reforms have never worked and never will.
Recommend 0
Ijaz shah
Mar 29, 2019 07:41pm
You cannot increase incentives unless a well defined system of Key Performance Indicators (KPI's) is developed first. There are many posts/job in the tax department where increase in tax collections cannot be taken as a performance measuring criterion. Apart from Tax department, there are many many other departments in the govt which do not directly deal with revenue generation. how can we bring about a change in those departments? The success of an employees depends on a lot of factors beyond their control including their team of juniors & seniors, policy, regulations, political will etc. It is time to re-build all departments on technical foundations and right from the top level, no irrelevant person having no background knowledge of his work area should be appointed. A doctor to head the health department, a professional accountant to head the tax department, an engineer to head the engineering organizations etc. In year 2019, is it still not clear to us??
Recommend 0
Aqdas Afzal
Mar 31, 2019 03:46pm
Professors Khwaja and Simon make an earnest case for revamping the incentive structure associated with public revenue officials. They base their recommendations on randomized controlled trials (RCTs), which have become the mainstay within the policy world for quite some time now. However, there are a number of problems with basing recommendations on results generated by RCTs and I will like to highlight only one. Where RCTs establish a tight causal connection between intervention and results (internal validity) the controlled design of trials make it all but impossible to replicate the results to other situations - RCTs lack external validity. Simply put, one can control various variables for the trial but it is impossible to control for the same variables when scaling up. Results generated by RCTs are not original insights - people generally know incentives work. The real problem is politics and power that prevents changing the status quo.
Recommend 0