School Choice - II: Evidence and Way forward - Ensuring a Learning India S5 E.008

[33rd post in the 'Ensuring a  Learning India' series. 42 posts in total. One post per day. 9 more to go]

Our ultimate objective is to improve the learning outcomes of children. If that’s so, who does the job better – public schools or private schools? Some argue that private schools visibly do the job better. The litmus test is the increasing enrolments in private schools. Others argue that private schools do better because the characteristics of students who join private schools are different; may be students (and parents) in private schools are intrinsically motivated, and hence shifted out of public schools, may be they have better support structures at home, may be their parents are more educated and hence take care of their children, may be they have more income to support child in other ways. Thus, it is argued that the claim that private schools can serve all students better, especially the students of low-income communities, is questionable and thereby the voucher system too. It is useful to look at international experience to understand the nuances.

Globally, there are two forms of voucher programs – large scale and small scale. Large scale programs are often run by governments, full-fledged voucher programs covering all aspects of vouchers comprehensively. Small scale programs are either limited to certain set of people or experiments run by researchers to understand specific issues of vouchers. Each of these programs can help us understand different aspects of voucher system. Voucher systems as an idea needs certain dynamics – competition, schools and parents responding to prices, time to deliver results, and so on. We can thus learn the macro level aspects from large scale programs. Small scale programs, especially experiments help us understand and validate specific hypothesis and arguments regarding voucher system.

5.8.1 Evidence on effects of school vouchers 

The effect of voucher system on learning outcomes is typically measured using two approaches – (i) performance of voucher students vs. performance of non-voucher students; and (ii) performance of a country as a whole, as indicated by PISA rankings. The first approach explores if the specific set of students who received vouchers gained in relative to their counterparts who didn’t. The second approach tells us if vouchers helped the system as a whole to improve.

Columbia launched a large scale voucher program in 1991 called PACES. These were targeted at children of low-income communities for students entering grade 6 and they were renewed subject to academic performance. It was found that after 3 years of program, the winners of voucher lotteries scored 0.2 standard deviations higher on achievement tests.[1]

In case of Chile, initial evidence[2] suggests that there is no evidence to say that choice improved average educational outcomes, in the 150 municipalities that were analyzed. Even looking at the aggregate measures, PISA rankings, Chile stands around 50+, out of 70+ countries, though the voucher scheme was introduced way back in 1980s and measures were taken to ease the entry of private schools. Interestingly, there was a dramatic increase of Chile’s PISA test scores from 2003-2011. Evidence suggests that this is due to a 2008 reform in Chile, which increased the voucher amount to poorest 40% by about 40%.[3] It also finds that this targeted voucher increased the average school quality by 0.21 standard deviations and increased average voucher school quality by 0.16 standard deviations. It also suggests that 1/3rd of these gains are from changes in school choice and 2/3rds is due to improvement in school quality. But still, Chile’s PISA rankings aren’t encouraging.

 There are similar debates in Sweden on the effect of voucher programs, after its PISA rankings declined from 7th to 23rd. The PISA report says ‘No other PISA participating country saw a steeper decline in student performance over the past decade than Sweden’. The proponents argue that private schooling constitutes only 15% of enrolment in Sweden and hence voucher system isn’t at fault. However, if one uses the metric of, ability of vouchers to improve the performance of a system as a whole, Sweden’s voucher program isn’t an encouraging experiment.

There seem to be multiple experiences regarding the voucher program. So, what does the experience in general looks like? Survey of the Economics Literature regarding School Vouchers[4] [1] studies major voucher programs across countries and says, “The empirical research on small scale programs does not suggest that awarding students a voucher is a systematically reliable way to improve educational outcomes. Nevertheless, in some settings, or for some subgroups or outcomes, vouchers can have a substantial positive effect on those who use them….().. Evidence on both small scale and large scale programs suggests that competition induced by vouchers leads public schools to improve.”[2] 

India is also not an exception to debates on school vouchers. The proponents argue that our public school systems have failed and in the wake of transition to private schools, government should respect the rights of people and hence should rightfully give them the money that the government would have spent on them, had they been in a government school. The opponents argue that all government schools shouldn’t be painted with same brush and that there are some good government run schools like Kendriya Vidyalas etc., and also that the success of private schools is due to the nature of students entering them and not because of better teaching practices. The proponents of vouchers argue that, private schools are parents preferences and hence they should be respected, and also that the few good government run schools often cited also seem to be working better due to the characteristics of students and parents. For example, students of central government schools study in Kendriya Vidyalas, which are mostly located in towns, and are much different from public schools in villages, which constitute the majority of schools. The word ‘private schools’ can mean anything ranging from schools which charge Rs.50/- per month to the elite private schools. Often, when people say that private school perform better, they are comparing the elite private schools to an average government school, and when people say that private schools are no better than government schools, they are comparing low-cost private schools to the average government schools. One should note this difference in reference frame (average vs. average comparison or extremes vs. average comparison).

Karthik Muralidharan and Venkatesh Sundararaman conducted an experiment in Andhra Pradesh to explore the effects of voucher scheme. It was conducted in 180 villages in Andhra Pradesh, spanning across 5 districts, where parents were invited to apply for vouchers and were then given vouchers based on a lottery. The voucher was priced at the 90th percentile of the prices of private schools in these villages. The scores of vouchers beneficiaries and others were analyzed after 2 and 4 years. The three key findings from this study are -
  1. No difference of test scores between two groups of students in Math and Telugu (mother tongue), small positive difference in English and Social Studies, and large difference in Hindi.
  2. Private schools spend significantly less instructional time on Telugu (40% less) and Math (32% less) than public schools and spend more time on English, Science and Social Studies.
  3. Annual per student cost in the public school system is three times the mean per student cost in the private schools that participated in the study. 

Skeptics and opponents of vouchers point out that, the scores in Math and Telugu are a proof to say that private schools aren’t any better than government schools in handling typical students who attend government schools. Proponents of vouchers point out that, private schools achieved same results as that of government schools in Math and Telugu by spending less time (40% less in Telugu and 32% less in Math), teaching English (small gains) and Hindi (large gains) additionally, and at one third the cost of a government school, making private schools more productive. This difference of opinion is due to the different reference frames – metric for success.

One should note that the cost of voucher is at the 90th percentile of price distribution of private school fee in these villages. The effects might change if this price is increased as schools get more money to invest, which we don’t know yet. This experiment was conducted in a rural setting where the density of schools is less, with often only 0 or 1 schools in neighbourhood. The effects might change in contexts of higher density due to increased choice and competition. The paper also suggests that the impact of vouchers may have been higher in markets with greater choice and competition.

Overall, these results suggest limitations of the effectiveness of low-cost private schools.

5.8.2 Lessons from voucher experiments 

There is mixed evidence from voucher experiments from across the world.  Voucher program in Columbia resulted in gains, the one in Chile wasn’t considered effective for long and evidence of positive effects emerged after it changed its design. The survey literature says that it benefitted some subgroups in some settings and also improved public schools. Evidence from India suggests that there is no difference in Math and Telugu scores but private schools achieved that with lesser instructional time and only one third the costs. What do we make of this evidence? The contexts in which voucher systems will be successful, the pre-requisites, and the mechanisms through which vouchers can affect are yet to be fully understood and are subject to future research but there are some general broad lessons.

I. Design of voucher programs matter – School Vouchers is a broad theme; two programs can both be using vouchers but the rules of operation can result in widely different outcomes. The first such difference can be due to the value of the voucher. There is a difference between giving a voucher equivalent to the money being spent on students in public schools (which tends to be higher) and giving a voucher to cover minimal costs. Merely increasing the value of this voucher can affect the outcomes, as the evidence from Chile’s reforms in 2008 suggest. The second difference can be due to the conditions for renewal of voucher. Some argue that the program was successful in Columbia because the continuation in voucher program was subject to performance of children. The third difference can be due to the restrictions on private schools and students in using the voucher – some programs allow parents to top up the voucher with additional money, while some programs restrict the schools from charging money above the voucher value. Some argue that having the flexibility to top-up the voucher can again lead to inequalities due to ability to spend, because rich people can top-up more and send to better schools.

II. Restrictions on private schools – Choice and Competition are one of the backbones of the functioning of voucher systems, as argued by the proponents. Regulations on private schools can distort this competition. If the barriers to set up and operate private schools are high, then it restricts some people from entering the market stifling the competition – like the norms on infrastructure and allowing for-profit schools. The commitments and constraints on schools to use school vouchers also matter; ability for schools to accept or reject students, ability to charge above the voucher value and so on.

The credit constraints and labour market constraints also can determine the competition between schools. We discussed an experiment in Pakistan where, if only one school receives grant in the village, it spends that money improving the schools, thereby increasing enrolments. If spending this money would lead to increased enrolments, schools would have done so if there weren’t credit constraints. However, this didn’t result in increase in outcomes. When all schools in a village received money, they used competitively which might have resulted in the observed increase in outcomes. In countries like India, land can also become a constraint since building a good school requires significant amount of land.

III. System’s expectation from schools and students – We previously discussed that there are information asymmetries in education. This is not just about parents’ perception about the learning levels of their children but also about the absolute learning levels, towards which one has to aspire for. The systems in the country often adjust to the expectations of the system, usually reflected by the end of school examinations and other assessments. It is possible that all students shift to private schools, and all private schools do better than government schools but on absolute level, that might just be at much lower level than expected. PISA rankings etc. often serve as a guide towards such absolute standards but government should also ensure that these are the expectations from schools too, not necessary by making them to participate all of them in PISA but through other means. If these expectations are low, then all schools can stabilize themselves in a low equilibrium, even in the presence of competition, in addition to the informational asymmetries which may send noisy signal to parents about the quality.

IV. Quality and credibility of assessments – Centralized systems are supposed to be easy to monitor because of the similarity in procedures and hence expectations. When systems diversify or decentralize, monitoring the outcomes becomes essential. Information about quality of schools also becomes essential to bridge the information asymmetries. Thus, assessments become crucial. The quality of these assessments also determines the expectations from the system.

Some countries allow for internal assessments which are used to rate the schools. Some argue that such practices led to schools over grading students in order to get higher rankings. Thus, administering these assessments and their credibility also matters.

V. Support structures – One can set expectations but it is equally necessary to have support structures in place for these schools. Support structures can be anything ranging from consultancies on managing the school, training teachers, curriculum design, assessments and so on. There can be problems due to both lack of services and lack of information about these services to schools.

In the experiment in Pakistan that we discussed earlier, many schools spent on infrastructure but not on curriculum design and others and argued that other services are too expensive. When both schools and service providers were brought together inform of a fair, providers and schools learnt from that and providers started offering customized packages to schools and so on. Market may take care of these in long term but there is also a possibility for such deficiencies.

5.8.3 Should we implement voucher program? 

The discussion above suggests that the evidence on vouchers is mixed and that the design of vouchers and other pre-requisites are crucial for its success. In the light of all this evidence, there are some strong reasons for India to consider the voucher policy.

The first reason is the rights of children (and parents). There is a gradual shift from public schools to private schools in the recent years, and a significant section of these belong to the low-income communities. These students would have gotten government support in form of spending on schools, had they studied in government schools. If that is the right of children, then why is that right denied in the form of not giving any support, when they join private schools? We aren’t talking about high income elite school students; we are only talking about the students of low-income communities here. Right to Education shouldn’t just be limited to Right to Public School Education, it should be right to education anywhere, and government should support them regardless of the schools they choose to go, if it genuinely believes that education is these children’s right.

Second, supporting intrinsically motivated students or students of ability. We need to rephrase the framework of debate on impact of vouchers. As discussed earlier, the impact of vouchers can be debated on two fronts – ability of vouchers to improve the system as a whole (this would be equivalent to bringing up the PISA rankings), and the ability of vouchers to benefit certain sub groups of students. The general discourse on vouchers is about its ability to improve the system as a whole, as a substitute to government systems. If the global experience is any indicator, this isn’t as straight forward and easy as it seems. We noted above in the survey of literature on vouchers that, it helped certain subgroups in certain contexts. Evidence[5] from US suggests that educationally-disadvantaged groups gain the most from school choice programs.

Instead of looking at voucher program as substitute to public schools and with expectations to improve the system, we should look at it from the perspective of benefiting certain sub-groups. We should let those sub-groups who benefit from it avail those benefits. Denying benefits to these sub-groups (by not giving them vouchers) just because others aren’t benefited due to isn’t logical and fair.

Third, easing parents’ financial constraints. Many parents, especially of low-income communities are making significant investments (as proportion of their incomes) in educating their children. This is causing financial burden on these parents. The PROBE observes that some parents shift their children to private schools but then bring them back to government schools due to lack of money. Vouchers can ease burden on such parents, and helps them achieve their aspirations.

Fourth, good NGOs and schools benefit from it. It is just not that low-income private schools are needed and are facing constraints. Even some good schools of NGOs known for their innovative pedagogies and cater to students of low-income communities, are being forced to close down their schools due to lack of funds. Vouchers of students of these schools could have eased the burden of these schools, and helped them sustain, which is in the interest of the students.

5.8.4 Pre-requisites for voucher program’s success in India

We have earlier discussed lessons from voucher programs which highlighted some pre-requisites for the success of a voucher program. The default pre-requisite is that design of voucher scheme is crucial for its success and hence one has to be mindful of this fact. Apart from that there are certain India specific constraints, which in turn could be pre requisites for success of a voucher program.

First, legalize for-profit private schools. Choice and competition is one of the critical pre-requisite for voucher programs, and not allowing for-profit private schools is a significant entry barrier which stifles competition. 

Second, autonomy to public schools. The system as a whole improves if there is a healthy competition between public and private schools. While private schools are autonomous and have powers to respond to demands of parents, public schools often don’t have, which leads to an unequal ground. If one has to earn trust and let public schools genuinely improve through a healthy competition, they should have the autonomy to do so. 

Third, reform board examinations. We discussed the need to reform board examinations in earlier sections. We also discussed that even in competitive environments, schools can settle for lower equilibrium due to informational asymmetries. Thus, government needs to set the expectations of system from schools, and good quality board exams is one of the ways to do that. 

Fourth, credible and quality assessments - Credible and quality assessments are crucial for three reasons.

(i) As the system diversifies, or decentralizes or moves to a regulatory framework, monitoring outcomes becomes essential. Government needs to invest in both designing the assessments and also administering them. 

(ii) Systemic assessments signal the expectations of system from the school.

(iii) Credible and quality assessments keep us focused on outcomes. Are we just being a fox painting stripes on ourselves, to become a tiger, as discussed in governance theme? How do we ensure that we are not doing that? This is a true concern. Reforms like school voucher takes political capital to navigate through numerous hurdles. The implementation is also going to take up energy and resources of the government. All these efforts might give us a false sense of achievement that we are working hard on reforming education. But what if it turns out that we didn’t benefit anything from this reform? How to ensure that we can track this and not be complacent? This is another reason to have regular assessments tracking children’s outcomes and also benchmark them with international assessments, and participate in international assessments. This will ensure that we are not carried away by the extent of our efforts but remain focused on the outcomes.

Fifth, support structures to schools. Schools need support structures in terms of teacher training, curriculum design etc. which are essential for the improvements in quality. We have to invest in building those programs and if needed in some cases help schools get access to those. 

Sixth, ease unreasonable regulations on private schools. Apart from the legality of for-profit schools, there exist some unreasonable regulations on private schools to be able to operate. These have to be eased to lower entry barriers and enhance competition. 

5.8.5 Hurdles for school vouchers and way forward 

There are currently four significant hurdles to make voucher program a reality.

First, political economy associated with the voucher program. The teacher unions are strong and have strong say in policy matters pertaining to them. Some of them may feel that voucher system is a threat to public education system and hence their jobs will be in trouble.

Second, additional costs involved in implementing voucher programs. While one may argue that the money that would have been spent on students if they were in public school should be given to them as voucher, it may not be possible in reality. Some estimates suggest that teacher salaries constitute 80-90% of the costs involved in school education. These costs can’t be cut down and equivalent money can’t be given to students, owing to the permanent tenure of teachers. If one has to implement a voucher program in current context, it has to be through additional costs and that value may not be equal to the actual per-student government spending.

Three, fear of unknown and skepticism about implementation structures. Vouchers as an idea isn’t prevalent yet in India and hence there is a fear of unknown about it. There is also skepticism about the feasibility of implementation of vouchers, the process involved etc. especially in the light of the fact that many private schools are complaining about non receipt of reimbursements regarding the 25% EWS quota students in their schools.

Four, difficulty in reproducing the effects of vouchers – When a new idea is proposed, usually one asks for the rationale and evidence of its success. In some cases, there won’t be any previous evidence but they are just done on gut feeling and also sometimes because they have to be done. The common bias among decision makers is – I will implement the policy if it is my idea because there isn’t any evidence suggesting its negative effects but if it is your idea then I will only implement if the positive effects are demonstrated. In such a scenario, unless someone who is strongly convinced about vouchers come to power, it is difficult to take decision on this unless there is vast body of evidence on this. Also, the policy priorities are skewed towards preventing the worst from happening and hence it is difficult for people to trust something new with the fear of the worst. The bottom line being evidence is crucial.

Researchers can generate that evidence through experiments but the constraint with vouchers is that its success depends on too many factors and spillovers and also the time required. It is often not possible to replicate all of these in experiment settings.

What do we do in such scenario?

In the presence of all these hurdles, one has to go for second-best options for immediate benefits.

One, demand autonomy to public schools – There are strong stand alone reasons to give autonomy to public schools, which we discussed earlier but this could also possibly ease the path towards vouchers, because it can possibly lower the helplessness and feeling of lack of power to compete among public servants.

Two, vouchers for after school support systems – There are again strong stand alone reasons for after school support systems. The advantage of this context is that these aren’t possibly threat to public schools and hence the lower resistance due to political economy. These vouchers can help validate some of the exaggerations and skepticisms of vouchers. It can also help government to fine tune structures of implementation of vouchers, the lack of which is also sometimes mentioned as reasons for not to implement vouchers.

Three, train teachers of low-cost private schools – In the absence of vouchers, the second best way to help students in low-income private schools is to improve the quality of teaching processes through teacher training. Schools operating under financial constraints may find it unfeasible to train those teachers and also with the fear that these teachers might leave for other schools. Training to these teachers either in form of skill vouchers which can be used at teacher training centers or other forms should be advocated for.

[1] Angrist, J., Bettinger, E. Bloom, E., King, E. and Kremer, M., “Vouchers for Private Schooling in Colombia:Evidence from a Randomized Natural Experiment.” American Economic Review 92 (5): 1535-1558, 2002.
[2] Hsieh, C. and Urquiola, M., “When Schools Compete, How do they Compete? An Assessment of Chile’s  Nationwide School Voucher Program,” NBER Working Paper 10008, 2003.
[3] Neilson, Christopher, “Targeted Vouchers, Competition Among Schools, and the Academic Achievement of Poor Students,” mimeo, Yale University, 2013
[4] Dennis Epple, Richard E. Romano, Miguel Urquiola, “School Vouchers: A Survey of The Economics Literature”, NBER Working Paper 21523
[5] Howell, W. G., and P. E. Peterson. 2002. The Education Gap: Vouchers and Urban Schools. Washington DC: Brookings Institution.

No comments:

Post a Comment