The Analytical Angle: Why haven’t past education reforms had more effect?

When so many reforms are developed independently, they are prone to contradict each other.

Updated 01 Oct, 2019 04:42pm

The National Education Policy Framework launched under the government's 100-day plan calls for a number of changes to Pakistan’s educational system, such as a tech-based Smart Schools System, an Educational Volunteer Programme and an increase in the number of non-formal schools.

While the proposed reforms would be a major change to the current education landscape, they are but a new chapter in a long and active history of education reforms. If we can understand why those past programmes haven’t led to better outcomes for Pakistan’s children, we may be able to do better in the future.

Pakistan invests effort in fixing education

Over the past two decades, Pakistan has been remarkably prolific in passing education reforms. In every key area of policy, from teachers to accountability to basic inputs, the government has pushed through a major reform almost every five years.

The table below visually represents research we have done in collaboration with our colleague Jishnu Das at the World Bank on the reform experience of Punjab over a 15-year period; other provinces have shown similar levels of activity.



This pace is especially remarkable when compared to peer countries, where reforms are often subject to years of debate or blocked by powerful opposition groups like teachers’ unions or political opponents. Indeed, Pakistan has gained international attention as “perhaps the world’s largest laboratory for education reform.”

Pakistan invests money in fixing education

Vigorous activity in the policy arena has also been accompanied by a striking rise in education spending. National spending on education rose 160 per cent from 2011–2017, which fueled improvements in key educational resources like school infrastructure and teacher salaries.



Given the magnitude of changes to both policy and funding, improvements to educational outcomes seemed inevitable, but surprisingly, growth in enrollment and test scores — the two best measures of education success — has been disappointing.

There has been some improvement: today, most Pakistani children attend school at some point in their lives and more are persisting on to secondary school and university. Nonetheless, around 30pc of students do not reach sixth grade, and enrollment lags behind relatively poorer countries in Africa and Asia that have achieved near universal primary enrollment.



Learning growth is even more disheartening. Measures from the independent assessment organisation ASER show that test scores have fluctuated over the past five years and show only modest growth in the most recent round in 2018 — certainly not commensurate with the improvements in policy and influx of funding.



Why haven’t past reforms made more of a difference?

So many changes to education policy, so little to show for it: this represents a puzzle for both policymakers and academics. Analysing the effects of some of the reforms over the past two decades can reveal some of the forces that limit policy’s effectiveness.

First, unintended consequences.

Reforms can hurt just as much or more than they help. For example, the Punjab Examination Commission (PEC) exam, introduced in 2005, was intended to provide critical data on learning and equity.

However, because the exam was conducted in February, it appeared to compress the part of the school year when students are likely to learn the most: anecdotal evidence suggests that the months of concentrated learning were reduced to August through December.

Most teachers devoted January to test prep, and then had lower incentives to improve their students’ learning levels in the months following the exam, since any gains would be credited to the teacher of the next grade up.

Although we do not have the data to understand whether learning would have been greater in the absence of PEC, what is clear is that the shortening of the school year was an unintended consequence, whose ramifications are ill-understood.

In another example, several provinces have recently raised degree requirements for new teachers. While raising the bar has improved the qualifications of the average teacher, it also reduces the pool of potential teachers, which can worsen the problem of teacher shortages and high student-teacher ratios, especially in rural areas.

Opinion: Why attempts to reform Pakistani education fail

These examples are not intended to suggest that this reform should not have been implemented. On the contrary, collecting data on student learning and improving teacher quality are critical to improve learning. Rather, these examples emphasise that policymakers must carefully study the consequences of reforms and implement simultaneous policies to mitigate adverse effects.

Second, contradictory effects

When so many reforms are developed independently, they are prone to contradict each other. For example, provinces have made various efforts over the years to empower schools and parents to make decisions, such as by establishing School Councils that give parents and communities a role in deciding how some education funds should be spent.

At the same time, the government has vastly increased school monitoring and aimed to standardise school inputs, which reduces the decision-making authority of individual schools.

These efforts at decentralisation (through School Councils, school grants, contract teachers) and centralisation (through standardised curricula, monitoring, teacher hiring) stand in direct contrast and often prevent reforms on either end of the spectrum from generating a full impact.

Observing these examples, policymakers should pay greater attention to how each change is likely to interact with the broader education system. In particular, they should consider how standardisation might inhibit the ability of parents and schools to innovate and hold each other accountable, as was the intention of earlier decentralising reforms.

Piloting works

If you can imagine the unintended consequences of and contradictory interactions between policies in the profusion of reforms, then you can see why enrollment and learning gains might be slow even when certain reforms had positive effects.

The problem is, with so many hurried and overlapping changes, we don’t know which reforms have been effective and which have led to further problems. But there is a way to find out.

Before implementing reforms across the province or country, the government can try them out with a smaller group of schools. Pilot testing allows schools and teachers in the test group to provide critical feedback that can be used to improve the final policy design.

Piloting also creates the space for researchers to conduct rigorous evaluations to isolate the outcomes attributable to the specific policy in question, rather than a range of other reforms that are often introduced at the same time.

Perspective: Tackling the menace of a failed education system

Past education studies in Punjab point to the power of pilot testing. In 2004, our research group conducted an experiment to give parents and schools a “report card” showing average test scores for all public and private schools in a village.

With a careful research design, we were able to show that sharing information on learning quality led to higher test scores and enrollment and lower school fees across the communities we tested.

Drawing insights from this project, the Punjab government later decided to distribute semi-annual report cards with performance data on all public schools to help parents and School Councils hold schools more accountable for learning growth.

The current administration has a chance to take a rigorous approach to education reform, different from that taken by past governments.

After all, learning from the past is the bedrock of education.


The Analytical Angle is a monthly column where top researchers bring rigorous evidence to policy debates in Pakistan. The series is a collaboration between the Centre for Economic Research in Pakistan, Evidence for Policy Design at Harvard Kennedy School, and Dawn.com. The views expressed are the authors’ alone.

Email


Author Image

Tahir Andrabi is Stedman-Sumner Professor of Economics at Pomona College, co-founder of the Centre for Economic Research in Pakistan and the current Dean of School of Education at Lahore University of Management Sciences. Tahir has been a visiting scholar at MIT, a research associate at STICERD LSE and a consultant for the World Bank. He was a member of the tax and macroeconomic committees of the economic advisory board of the government of Pakistan in 1999-2000.

Author Image

Isabel Harbaugh Macdonald is a doctoral student in public policy at the Harvard Kennedy School, focusing on labor and development economics.


The views expressed by this writer and commenters below do not necessarily reflect the views and policies of the Dawn Media Group.

Comments (19) Closed

Fida
May 08, 2019 09:01am
We are focusing on inputs for reforming education →Raise education budget → Hire more teachers, Increase salaries →Improve teacher recruitment methods →Build more buildings →Revise curriculum →Build larger libraries →Build more universities (Isomorphism, doing the same thing) →Distribute more text books and laptops →Arrange more teacher training workshops →Involve more NGOs Who is going to measure the impact of teaching on students learning instantly after the class? Teacher goes into the class, spends 40 minutes and comes out thinking that he/she taught well and students learnt well! Quiz the students after the class is over and you will find instantly how much the teaching and learning disconnected are from each other. Most stake holders including teachers are playing what is called the school game. Expecting desirable results is insanity. Fida, GIKI
Recommend 0
Salman Amin
May 08, 2019 12:59pm
@Fida, Good analysis as unfortunately we are focusing on the quantity of student produce rather than the quality.
Recommend 0
harry
May 08, 2019 01:24pm
Better education needs money but in a country where the people are more interested in the after life than the present life, it hardly matters.
Recommend 0
Bharat Jha
May 08, 2019 01:40pm
If you look carefully, small kids of her age are in scarf. Is this stage of life to consider all these things or have freedom in her thoughts and action. We have actually made certain things or aspect of life very much unquestionable and rest are entangled to it .I believe this is root causes of everything.
Recommend 0
Mir
May 08, 2019 02:14pm
@Fida, Quizzing after each class is not a sustainable strategy to ensure the teachers do their jobs. What would you suggest the government does to discourage the 'school game'?
Recommend 0
ashar
May 08, 2019 02:23pm
I have few points to make here. The first is the statistics given in this piece. It is incorrect. These are collected from government sources and using government infrastructure. There is no way to verify this information. If it was the case, then IMF an World Bank would have done with the economic numbers given by different governments of Pakistan. Second is ideas such as more schools, more teachers, more salary will never work as it has not worked. The government teachers (at all levels) earn more than the private sector (few exceptions could be there in the private sector). The education system will never start improving unless there is a bigger middle class group, who are going to challenge the hegemony of the POWERFUL politicians (95% of them happen to be some kind of spiritual leaders). the bigger the middle class, more the people to question the hegemonic control of this small group and more appetite for change including changes in education sector.
Recommend 0
Tarik
May 08, 2019 04:18pm
What did Nawaz & Zardari do to educate poor Pakistanis?
Recommend 0
dr yahya
May 08, 2019 06:15pm
Who is really interested that the level of education improves?
Recommend 0
vin
May 08, 2019 11:59pm
@Tarik, When did a politician ever ran Pakistan?
Recommend 0
Junaid
May 09, 2019 02:24am
@Bharat Jha, So the root cause of the education problem in India is because little girls as young as 5 have bhindi on their foreheads right
Recommend 0
Neutral Reader
May 09, 2019 02:29am
A robust school inspectorate that regularly inspects schools against national standards is one way of monitoring teacher effectiveness and overall school performance but I think the problem in Pak is the curriculum which is not fit for the 21st century. There is too much rote learning of facts and not enough analytical skills mainly due to lack of proper equipment and resources to deliver quality lessons. It would be more cost effective for the government to buy large screen TV's for schools and deliver lessons by video link.
Recommend 0
Moth
May 09, 2019 03:07am
I am not an educationist and neither a neuroscientist. I am a logician and see things in an evolutionary process. Evolution is happening every day and every movement right around us. The brain is like a receiver of information and the children brain receive and grow. My theory is that you put your child in front of a computer/ tv and show him lessons about math, chemistry, physics, psychology, statistics, economics, etc. that child will learn all that knowledge without a teacher and a class room. That is the power of brain. But if the child nothing to learn then it is a different story. My suggestion is that educational institution like Beaconhouse should teach all Pakistani children with same level of education through Internet. Just poor families access to Internet and a computer/tv. That is it.
Recommend 0
Hassan
May 09, 2019 03:24am
In other countries school meals have been successful in improving attendance. Have they been considered in Pakistan?
Recommend 0
Fida
May 09, 2019 06:48am
@Mir, Govt. has failed to do anything. Then why should I suggest if they don,t have will, skill, mission and vision. I am the one to do my bit as a teacher. Do we teacher do it? Whey ask Govt. then? My brother: learn your subject and make a story of it which traps and engages your students in the class room. If you continue this practice, one day your students will be the ones running the Govt. and then you can suggest to them. I tell you: they will the capacity respect your opinion. Fida GIKI
Recommend 0
Jeetesh Dash
May 09, 2019 01:16pm
@harry, DOT man Dot on target . This is the basic issue . What exactly are you going to do with educated children if there is no industry to absorb them ? Boss I am an ardent supporter of children wanting to turn their lives around bit have you ever thought what is the gain of education ? Better develop the ECONOMY - everything else will follow - Government will have funds , Industry will require educated mass , employment will be good . But we have to decide which comes first - the cart OR the horse ?
Recommend 0
Bharat Jha
May 09, 2019 01:43pm
@Junaid, Yes Junaid, whether It is India or Pakistan if we force kid to these trivial things and suppress her thoughts or close their ability to question anything to everything, we can never get the true sense of education. I am not against scarf or Bindi (though Bindi is not religious ) I am against the age when parent or society forced them to do same
Recommend 0
rama
May 09, 2019 10:32pm
@Jeetesh Dash, We need basic education to lead a normal life in modern time. Even if you don't get job the knowledge stays with you.
Recommend 0
HMR
May 09, 2019 10:46pm
The culture of not questioning things (Because of socio-religious construct) and the fearful environment present in the institutes of education are one of the many reasons for the dire state of education in Pakistan. One of the possible solution lies in rebellion from a few brave academics who can fight for the change like Voltaire, Rousseau, Kant etc did in Europe.
Recommend 0
Texan
May 10, 2019 11:29am
Very misleading when you compare the spending in Rs with the devaluation from RS75/USD to RS140/USD and population growth. Its mere $70 per year per child if you assume 80,000 kids. Pakistan should be making high school education mandatory and spending 10 times more to get the proper quality of education.
Recommend 0