Marking stripes on a fox's body doesn't make it a tiger - Pitfalls of imitations and arguments by examples : Ensuring a Learning India S3 E.003

There is a saying in Telugu which transilterates to 'puli ni chusi nakka vaatha pettukunnattu'. It means, a fox burnt parts of its skin, on seeing a tiger, in order to get stripes on its body. Obviously, the fox was trying to imitate the tiger here and hence thought that having those stripes will give it the qualities of a tiger. 

This simple saying captures pitfalls of a common phenomenon, argument by examples and analogies. It looks obvious and sounds like minimum common sense to not do mistakes like the fox above but we tend to do this more often. If you visit a country and observe that teachers there teach using a new method called, inquiry based learning or that they have uniform curriculum throughout the country, it is natural tendency for some to think that these should also be implemented in our country. However, the fundamental question is, why is it that inquiry based learning came to action in that country and not in ours? What are the underlying principles behind the functioning of such mechanism. Same is the case with the example of uniform textbooks and curriculum. It might turn out that either the underlying principles are something very different or that the observed phenomenon has nothing to do with the outcomes, for examples, stripes in the case of a tiger. Tiger is not ferocious because it has stripes!

There are three issues with such arguments by examples. One, by arguing merely with examples, you can prove either wise, by cherry picking examples, depending on your objective, You only need to have enough knowledge to be able to use them as per the situation. Two, such simplistic insights derived from examples can do more harm than good. It diverts our attention to non fruitful issues, occupying the precious scarce bandwidth of policy discourse. Three, actions based on such ideas can give an illusion that something happening but nothing changing in reality. The opportunity costs of doing this may be huge. This phenomenon is called by different names by different people. Lant Pritchett calls it isomorphic mimicry. Arvind Subramanian in his recent article argues something similar saying, it is not that manufacturing sector is important, but what are the underlying principles which makes that sector important?

Caveats of evaluations in education - Ensuring a Learning India S3 E.002

[16th post in 'Ensuring a Learning India' article series]

The previous post discussed the pitfalls of the seemingly obvious and underscored the need for evaluations which can disentangle the causal relationships. 

Randomized Controlled Trials (RCTs) or Controlled group studies, as they are popularly known is a specific method of evaluating interventions to explore the causal effects. The fundamental question in any debate is the comparison with the counterfactual. If someone says that giving free textbooks to students increases outcomes, they should be able to also answer the question, what if the textbooks weren't given during that time? Would the outcomes have increased regardless? May be something else has happened during that academic year which resulted in the increase outcomes and hence the effect cannot be associated with free textbooks. The intuitive way to address this challenge is to take two sets of randomly selected similar groups, give textbooks to one group and don't give to the other. The difference in outcomes between the group which receives free textbooks (treatment group) and the group which doesn't receive free textbooks (control group) gives the true impact of textbooks. If part of the increase in outcome is due to children getting competent with age, that's the same case with both the groups. Hence, the difference takes care of the other phenomenon which might have happened, other than the distribution of free textbooks.

RCT evaluations have become very famous in recent years and are being increasingly used to generate evidence on various hypothesis in developing sector and test the impact of policy interventions. This evidence is increasingly shaping the policy discourse. Just like any other phenomenon which gets reduced to simplistic observations, on reaching scale and reaching wide audience, RCTs are also being misinterpreted. This post discusses some of the caveats while interpreting evidence from RCTs. These aren't the errors or issues with the methodology of RCTs but are limitations which one must be cautious of, while interpreting the evidence. Researchers who conduct these evaluations well understand these nuances and caveats and are cautious about interpreting along those lines. It is important for others to do the same.

Pitfalls of the 'seemingly obvious' - Ensuring a Learning India S3 E.001

'Pitfalls in translating observations to policy' is the third theme in the 'Ensuring a Learning India' series. This theme discusses the commonly observed pitfalls while devising a policy option based on observations or data. It is important to understand and recognize these, moving ahead.

There are three major pitfalls in the process of translating observations into policy. One, focusing on the seemingly evident. Two, arguments based on examples and inferring from successful systems. Three, difference in moral frameworks. This theme will discuss each of these.

This post discusses the first pitfall, focusing on the seemingly evident. Seemingly evident observations are those which are striking and hence seem obvious. This can be in case of people both inside and outside the system. For people inside the system, if something troubles them on a constant basis, then out of this discomfort, they may exaggerate the extent of the bad effects of the issue and highlight that as *critical* constraint in achieving the outcomes. For example, outside school duties for teachers. It may be true that this issue is coming in the way of  teachers' functioning but some may overestimate its impact and perceive it to be the only critical constraint. Similarly, for people outside the system; if one visits a school and finds that students don't have uniforms, one of the immediate response could be - this is sad, students need tidy uniforms. Some of these can be touching emotionally and hence remain at the top of our mind even after the visit thus occupying the important space. In such cases, it is possible for the person to feel that, getting uniforms is the *critical* thing that should be done to improve schooling.

What's wrong with these policy proposals? Don't we need uniforms for students? Shouldn't we stop teachers from spending time outside classroom? It is true that all these are necessary but there are five issues with such approach of arriving at policy conclusions based on mere observations that seem obvious.

Key insights from 'Inside Primary Schools' report by Pratham - Ensuring Learning India S2 E.004

[14th post in 'Ensuring Learning India' series]

The second theme in the 'Ensuring Learning India' series is 'Status of Education in India'. The first three posts on this theme discuss status of learning outcomes, status of infrastructure and perceptions of different stakeholders respectively. All these are aggregates or snapshots at one point of time. But what would be like to track government schools over a period of time and see what's happening? Pratham has done that through its 'Inside Primary Schools' study. The study tracks about 30,000 children from around 900 government primary schools over a period of one year, covering Andhra Pradesh, Assam, Himachal Pradesh, Jharkhand and Rajasthan.

The text below is taken directly from the original report from the relevant sections. In the interest of the length of this post, only key points are mentioned. It is strongly recommended to read the complete report.

Teachers attendance and student learning

Schools with an average of 0 or 1 teacher present clearly have lower mean and median classroom scores than schools with an average of 2 or more teachers, on both the baseline and the endline tests. But beyond 2 teachers, there is no clear relationship between the average number of teachers present and classroom scores, on either baseline or endline.

Excerpts from PROBE 2006 - Ensuring Learning India S2 E.003

Public Report on Basic Education in India (PROBE) is a survey covering allschooling facilities, anda sample of 1376 households, in 234 randomly-selectedvillages of Bihar, MadhyaPradesh, Rajasthan,Uttar Pradesh andHimachal Pradesh. This was first conducted in 1996 and again in 2006. The 2006 survey visited roughly same sample villages but not the same households. This report tries to depict the picture of schooling system as perceived by parents, teachers students and community. This has rich qualitative observations from different stakeholders and hence give a decent picture of public schooling system.

This post reproduces some important excerpts from the PROBE 2006. If the reader has time, it is recommended to read the complete report because excerpts mightn't do fair justice to the narrative of the report. Observations on school infrastructure, enrollment and learning outcomes from the report are excluded in the excerpts below because they were already discussed in the previous two posts.

Before proceeding further it is useful to note two points. One, PROBE captures only different types of perceptions in most cases and not the proportion of people with each perception. Hence one should be cautious about drawing conclusions about it. However, even the information about different types of perceptions is extremely valuable. Two, this report is based on survey conducted in 2006, which is before the enactment of RTE Act. Hence, the scenario then might have been different regarding certain aspects.

Note: Italicizing or putting long text in quotes may make it unreadable and hence it isn't being done . But, please note that all the text below is from the PROBE 2006.

Some important statistics of Indian schools - Ensuring Learning India S2 E.002

In the previous post, this blog summarized the status of learning outcomes in India. This post summarizes status of infrastructure in schools. All the statistics are taken from District Information System for Education (DISE) 2013-14. Though some of these things are straight forward, it is important to know certain numbers just to get a sense of the broader picture.

There are 10, 93,969 government schools in total. These constitute 75.51% of the total schools. The private unaided schools (with no funding from govt) constitute 17.4 % of total schools and private aided schools constitute 4.69% of total schools.

Primary schools have 224 instructional days on an average and upper primary schools (grade 6-8) have 225 instructional days on an average, in an year. Even if they work for 5 hours a day, this comes up to around 1100 hours an year which puts India on the top of the list of countries ranked as per total instructional hours in an year.

91.08% schools constituted School Management Committees (SMCs). SMCs constitute of parents, teachers, headmaster and local community who are responsible for certain aspects of administration of the school.

There are 3.5 classrooms on an average in a government primary school. Private unaided schools have 7.8 classrooms on an average. Even if one grade has to sit in one class, it wouldn't get a room in an average government primary school. In this context, one should also note that there are 3.1 teachers per primary government school on an average, whereas it is 8.8 in case of private unaided schools. May be it's the reason for the multi grade teaching typically seen in government schools.

95.31% of government schools have drinking water facility and after the Swachh Bharat Abhiyan, all schools must be having separate girls and boys toilet, as per latest figures.

9.25% of primary government schools and 41.96% of government upper primary schools have functional computers. Kindly note this in the context of technology in education.

On teaches front, there are 46,12,429 government school teachers in India, who constitute 59.73% of total teachers in the country. Of these, 31.45% teachers are aged above 55, while 3.32% of teachers in private unaided schools are aged above 55. Contract teachers constitute 9.1% of the total teachers.

Pupil-Teacher Ratio is 26 in government schools and 25 in private unaided schools. Note that the average number of teachers in a primary school is only 3.1. It means that if there are 10 students per grade for all 5 grades in a government school and only 2 teachers, the pupil teacher ratio would be 25 but there are only 2 teachers to teach 5 grades.

There are 13,24,28,440 students in primary school and 6,64,71,219 students in grades 6-8. The average drop out rate is 4.67% in primary school, 4.68% in case of boys and 4.66% in case of  girls. The grade wise drop out rates are 4.83, 2.28, 3.58, 3.49, 9.47 for grades 1 to 5 respectively. The transition ratio from primary to upper primary school is 89.5.

Status of learning outcomes in India - Ensuring Learning India S2 E.001

'Pedagogy - Curriculum - Assessments' was the theme of first ten posts in the 'Ensuring Learning India' series. The second theme is 'Status of school education in India', where this blog would be summarizing important reports regarding this. The first post in this series is about learning outcomes.

What do we know about the learning outcomes in India? It depends on the definition of learning outcomes, the design of question papers and the methodology of implementation - sample population or census and if sample population, the methodology of sampling.

There are 8 different studies or surveys on learning outcomes in India. This post summarizes important findings from all these.

1) Annual Status of Education Report (ASER): ASER is the widely known report regarding learning outcomes. The questions in this survey test basic math and language skills. Questions available here. This is done in a sample set of villages and not in urban India. Children are surveyed at home, and not at school. It means that this consists of students going to private schools, government schools and even those who dropped out. It is not necessary that same children are surveyed every year.

On reading: "in Std III, only a fourth of all children can read a Std II text fluently. This number rises to just under half in Std V. Even in Std VIII, close to 75% children can read Std II level text (which implies that 25% still cannot)."

On Maths:
  • In 2012, 26.3% of Std III children could do a two digit subtraction. This number is at 25.3% in 2014. For Std V children, the ability to do division has increased slightly from 24.8% in 2012 to 26.1% in 2014.
  • There are other trends which are quite worrying. For example, the percentage of children in Std II who still cannot recognize numbers up to 9 has increased over time, from 11.3% in 2009 to 19.5% in 2014.
  • Similarly, the ability to do division among Std VIII students has been dropping since 2010. The proportion of Std VIII students who could correctly do a three digit by one digit division problem was 68.3% in 2010. This number has dropped to 44.1% in 2014.

Support systems for first generation learners - Ensuring Learning India S1 E.010

This blog  earlier discussed that majority of students in a classroom of a typical government school are way behind the class level. This blog also discussed that simplifying curriculum, slowing down and teaching to the level of the kid can possibly address this challenge but it is easier said than done. What if a student doesn't understand something in school? What if students, especially in primary school, need support at home? In case of rich students, they do have personal tutors and parents supporting them. What's the case with students of low income communities? 

Lack of support systems at home is one of the main differences between first generation learners and others. In the absence of such systems, it is difficult for students to catch up with concepts taught in classrooms. It is true that simplifying curriculum in classrooms, teaching to the level of kid and other measures may help address this but even after all this, students may need some individual attention. The advantage of after school support systems is that there is relatively lesser pressure to complete syllabus removing an operational constraint. Thus, they can give can relatively give better personal attention as compared to a regular classroom. 

Places like Hong Kong recognized this need and have started implementing after school support systems to help the first generation learners. These support systems are called by various names - remedial centres, tuition centres, study hours and so on. These are prevalent in India with around 24% of primary school students attending tuitions and with states like West Bengal having rates as high as 74% (ASER 2013, page 73). A recent phone survey conducted by Aam Aadmi Party in Delhi showed that 66% of the respondents send their children to private tuitions.

The effect of private tutoring systems is encouraging. This paper by Centre for Policy Research, Delhi says that private tutoring in India has ‘a large positive effect on test scores of math and language (separately or combined) for students in the age-group of 6-14 years. The effect is as large as an additional year of education or the effect of attending a private school instead of a government school.’ Further, the effect is stronger for the disadvantaged students – those who are less wealthy, and those whose parents are relatively less educated.

Need for research in pedagogy and aspects that need attention - Ensuring Learning India S1 E.009

In earlier posts in 'Ensuring Learning India' series, this blog discussed various aspects of assessments, need for teaching at the right level and technology in education. This post identifies the areas in these aspects that have pedagogy component and thus need attention.

1) Translating abstract themes into implementable processes: This blog discussed about teaching to the right level earlier. Teaching to the Right Level is an idea but how do we implement this in classroom for different concepts? There can be many possible ways to this. Which are better ways and what are the principles that one has to follow while building up pedagogy using this theme? How can one design processes that are not too cumbersome for teachers to follow but still be effective? All of these need significant time, efforts and iteration.

2) Understanding the reasoning behind children's conceptions and commonly made mistakes or misconceptions: This blog earlier discussed about  diagnostic assessments and customized treatment. To be able to diagnose children's problem, one has to first understand the nature of the problem and also need to understand the reasoning behind those conceptions to be able to do a customized treatment. 

An earlier post quotes an example on decimals and different possible reasons behind same mistake. Do we have such collection for all concepts?

For example, some students answer that the shaded portion is one third of the triangle. It is first important to understand that such conceptions are existent in some children and then one should also understand the reasoning behind such conceptions to be able to correct them.

Such knowledge database for all concepts in all subjects is required.

This understanding of students' thinking is also necessary to design questions for assessments.

Technology in school education - Ensuring Learning India S1 E.008

In the previous few posts, this blog discussed above the need for slowing down the pace of curriculum, teaching to the level of the kids at their pace and need for assessments. This post discusses the role of technology in all these aspects, issues with the current technology based learning tools, challenges in integrating technology in classroom, challenges from the governance perspective and few points on the evaluations on technology based learning tools.

Tools that help teachers also help students indirectly but for the sake of simplicity, they are still categorized.

Technology for teachers  Technology can help teachers in the following aspects.

1) Streamlining admin work: Teachers have to do a lot of administrative work as part of their job which often includes communicating with parents, sending messages, updating mark sheets, correcting homework and so on. Tech products can help streamline all of these. An app which sends message to all parents on one tap, a tool through which teacher can assign homework to students and track the status, tool which also corrects the homework and so on.

2) Aid in teaching process

a) Demonstrating experiments: Some experiments can't be demonstrated in classroom. Videos, simulations and images do a good job of communicating the essential aspects of complex phenomenon.

b) Granular diagnosis: In a previous post on assessments, this blog discussed that diagnosing students' difficulties is an important aspect of the learning process and the difficulties may vary across students. If teachers have to normally do it, they would have to design customized assessments for each child and spend lots of time analyzing the patterns which is humanely impossible. It is easier to do this through technology, a product which analyzes students responses on a set of questions and gives an automated report to teachers on the learning difficulties being faced by in their class.

c) Customized remediation: After diagnosing the problems, students have to be helped regarding the same. If all students aren't facing same types of difficulties, then each of them needs different type of remediation which again is an intensive and time consuming process for a single teacher to do. A smart tech product can do this.

3) Collaborative learning platforms: Today, there is little interaction between teachers to share their problems and there is no established platform to learn from others. Technology can serve as this platform connecting teachers from all across enabling sharing of information between teachers.

4) Remote teaching: Live video classes can be a good work around in cases where there is lack of teachers.

Criticisms of learning assessments

In the previous post, this blog discussed about the need for assessments and five aspects of assessments that need attention. This post discusses the various criticisms of assessments.

I. Assessments lead to perverse incentives and narrow the focus of education


1) Students' performance is often dependent on external conditions like the support systems at home, economic conditions and so on, which aren't totally in control of teachers. Hence, holding teachers alone accountable for the results isn't justified.

2) When the results of assessments are linked with teachers' pay and are used to promote or terminate teachers' employment, it leads to perverse incentives.
  • Teachers try to game the system to reach the targets, instead of actually teaching.
  • Teachers narrow the focus of learning, teaching only the concepts that are asked in exam.
3) Attaching high stakes to the results of assessments leads to mental stress among students. This narrows the focus of children, to study only aspects related to exam, which isn't good for overall development of children.

Counter arguments:

1) There are different types of assessments. The above problems are only in cases where there are stakes associated with the results of assessments. This doesn't mean the other forms of assessments, especially diagnostic assessments with low stakes that are meant to understand students' difficulties are bad.

Need for learning assessments and five aspects that need attention

In the last three posts (here, here and here), we discussed the negative effects of over paced curriculum, need to teach to the level of the kid. If teachers have to teach to the level of the kid, they should be first able to know the level of the kid and their difficulties. How do they know them? May be through an exam. The technical word used for this is called assessment.

This post discusses four aspects of assessments that need attention.

1) Granular Diagnosis and Customized Treatment: If you go to a doctor with fever, the doctor doesn't give same medicine for fever that (s)he has given someone else. The type of fever is first diagnosed and medicines are given accordingly.  But if a student doesn't understand a concept, do we do granular diagnosis like a doctor and give a customized treatment? Probably no.

The following example illustrates the nuances.

Suppose a student answers that 4.73>4.8, what could be the possible reasons?
  1. Student might have ignored the decimal and considered it as 473 and 48 and hence answered 473>48.
  2. Student might have first considered the number to the left of the decimal, 4 in this case. Since it is same in this case, the student might have then considered 73 and 8. Since 73 > 8, overall 4.73 > 4.8
There could be many other possible reasons, some that we know some that we don't know. Though the answer could be incorrect for many students, each of them did so due to different reasons and each of them require a different remediation, customized treatment.

It's not just the issue with students who have answered it incorrectly. Even those, who answered correctly might have done due to an incorrect reason. In this case, student might have considered 4.73 and 4.8 as 4 + 1/73 and 4 + 1/8. Since 1/8 > 1/73, 4.8 > 4.73. Thus, it requires multiple questions and probing to find out the actual difficulties. These aren't the theoretical scenarios. Researcher Kaye Stacey has documented around 15 such difficulties observed in students while learning decimals.

If teachers have to understand students' problems and remediate, such body of knowledge for all concepts is necessary which isn't currently not available or not upto the mark.. It is a public good and needs government attention.

2) Intra-school assessments: Schools usually conduct exams once in a month or  quarter, which are famously called unit tests or term exams or formative assessments. Often, questions from textbook are directly asked in these exams and hence students try to memorize them. These exams are to be designed better with questions that test the actual understanding of children.

3) Framing appropriate questions: Question making is also a skill. The following example illustrates its nuances (question taken from here).

The objective of these two questions is to test the concept of simple machines - load, effort and distance from fulcrum.

Which is a better framed questioned? The one on the right, because the question on the left can be answered even without knowing the concepts of simple machines. It can be answered by just knowing the concept of equality of numbers. Data from such questions will lead to misinterpretations about students' skills.

4) Designing question papers: There can be different purposes for an assessment or exam. One, to understand the difficulties being faced by students, similar to the questions on decimals discussed above. Two, to test the overall understanding of all concepts of a particular grade level. Three, a qualification test where the objective is to just test minimum competencies. Similar to exams where everyone above a particular mark are qualified. Four, to distinguish exceptional students from the rest.

Can we use same type of questions in all these exams? No. Question papers also have constraints of number of questions, length and time. If the objective of the exam is to test the overall understanding of concepts of a grade level and if we ask multiple questions on decimal like above, there won't be space for questions from other concepts. Similarly, if the objective of the exam is to distinguish exceptional students from the rest and if all the questions in the exam are easy then most people would be able to answer them. This defeats the purpose of the test.

What type of questions are to be asked? How many can be asked? What should be the composition? How should the marks be calculated? 

5) Aggregation of marks: If an exam has 50 questions, one mark each, the usual practice is to count the number of correct questions and share them as marks scored. If two students scored 35 marks, where first student answered 30 easy questions and 5 difficult questions but the second student achieved same 35 marks but by answering 35 easy questions, they aren't distinguished. It's the same 35 marks for both. Item Response Theory (IRT) is used to adjust for this where marks are aggregated giving weightage to each question based on its difficulty. It is also used for other aspects in assessments which are beyond the scope of this post.

There is an existing literature on design of question papers and aggregation of marks, which has to evolve. We need to put in efforts both to enhance this knowledge base and also incorporate them in the design of assessments to make them rigorous.

But are assessments always good? In the next post, we would be discussing the criticisms on assessments.

Stay tuned and do subscribe to the blog. :)

90% of students fall behind in a typical classroom today

In the previous two articles [here and here], this blog repeatedly stressed on one point. When a teacher is teaching to complete the syllabus and there is a difference in pace of curriculum and pace of learning, students fall back. If this happens over a period of time, the gaps increase. Hence, we need to seriously reconsider simplifying our textbooks, teach to the level of the child in initial stages.

But, is this just a theory or does this actually happen in a classroom? Do we have data to support the claim that students' level of learning at any point is lower than the level of content being taught? If some students are falling back, what is the percentage? Do such students constitute only a minority of the classroom?

Prof Karthik Muralidharan and Jedrzej Zieleniak of UC San Diego tracked same set of students in 100 representative government schools in Andhra Pradesh, India for 5 years, testing them every year from grade 1 to grade 5, and produced this amazing piece of evidence. Please find the paper here (Chapter 3) and summary here.

This one graph is powerful and speaks for itself. This is constructed with test scores of around 40,000 data points (one student can generate multiple data points since they are assessed every year). For more discussion on this, refer the paper.

Teaching at the Right Pace to the Right Level

In a class room there are two paces at race, the pace of teaching and the pace of learning. In the previous post, this blog discussed about pace of teaching and in this post it discusses about the pace of learning.

To re-iterate the arguments, when there is a difference in pace of teaching and pace of learning, the students below average lag behind and those above the average may feel bored. Thus, an ideal scenario is one where the pace of teaching is equal to pace of learning.

The gap between paces of teaching and learning can be bridged by adjusting either of these. In the previous post, this blog discussed about the over-paced curricula. Assuming that, the teacher decides to slow down the pace of teaching, to what pace should she reduce, in a class where there are students of multiple levels? The initial learning level of the child is another important aspect here. An ideal scenario is where teacher is able to diagnose difficulties of each student and teaches each of them at their pace. This would require one teacher per student which isn't feasible. Technology could be a possible solution where the advanced artificial intelligence programmes can identify the level of the child and teach him/her accordingly but the tools today haven't advanced enough to handle this. [More on technology in education later]

Negative effects of time bound and over ambitious curricula

In a typical class room today, where a one to many mode of teaching is prevalent, a teacher teaches to the assumed average child in the class. Teachers are under pressure to complete the syllabus. It's interesting that, when everyone is asking 'Are kids learning?', teachers are questioned whether they have completed the syllabus and not whether they have your ensured learning. Teachers typically proceed to next topic when they complete teaching the earlier concept not when they know that students have understood the earlier concept. Presence of a fixed syllabus and fixed time defines the curricular pacing, the pace at which teacher has to teach the concepts. This is irrespective of the nature of the class. This is similar to a gym instructor, having a same exercise regime for everyone and progressing everyone to higher levels at same pace.

Curricular pacing effects children in two ways. One, children who don't follow the concept are left behind permanently in the absence of remedial support. Two, children who are absent for few days can't follow the concepts later and hence fall back into the same trap. Courses are taught in a similar manner in universities too but the difference being, students in universities have capability to self learn based on minimal guidance while students especially in primary grades don't have. This is more pronounced in contexts where students don't have any support after the school to explain these concepts to them.

Lant Pritchett and Amanda Beattman in their paper titled The Negative Consequences of Overambitious Curricula in Developing Countries say
we show that all of the observed learning differences between poor performing and OECD countries could be accounted for only by an overly accelerated curriculum in poor countries – even if the countries have exactly the same potential learning.
That is, the observed learning profiles (rates of learning per year of schooling completed) can be flat just because the material being taught to too hard for students as the curriculum has moved ahead, leaving students behind.

Weekend interesting links to read

1. Pew Research Center has this interesting data on greatest threats as perceived by different nations.

2. How many licks does it take to reach the center of lollipop? The fluid dynamics of lollipop dissolution.

Apparently the answer is 1000. The fascinating result of this research is that, when lollipops are exposed to the flow of a fluid, reach the same intermediate shape during the process of dissolution, irrespective of their initial shape and speed of the flow.

Reforming board exams in India

Weak governance, lack of political will and many other similar reasons are often suggested as reasons for poor functioning of the public school system. If these are the real problems, then the question is - would addressing these problems lead to high quality education?

While one might be tempted to say "Yes", I would caution against it. Consider the scenario of high-end private schools. They don't have the usual problems that plague public systems like weak governance, lack of funds, personnel etc. Yet, many international assessments show that students of Indian high-end private schools are below international average.

Why is it that Indian high-end private schools are not matching up to global standards despite having every resource at their disposal? My argument is that poor quality of Indian board exams is the key reason for the below the potential performance of Indian high-end private schools.

This blog post explores this argument in detail and proposes an outline for board exam reform.

Nature of Indian board examinations

Board examinations in today's India serve three purposes. 
  1. Board exams set the expectations to the school system, because most schools use these results as a proxy of their efforts in their communication with parents, and hence teach keeping these exams in mind, at least in the 10th grade.
  2. The results of board exams signal the ability of a child. Performing well in board examinations needn't mean that the student is good but not performing well is considered has a negative signalling effect.
  3. Board exams also serve as certificate of minimum qualification, which is used as eligibility criteria for low end jobs.

Issues with Indian board examinations

1) Board exams aren't serving any of the above mentioned purposes effectively:

To begin with, board exams do a very poor job of setting expectations to the school systems. Board exams in India are the reflection of the widely prevalent rote based learning. Questions in these exams are often as rote as they can get. Because board marks are the only comparable and easily disseminable metric, most schools end up teaching only to the level required to excel in the board exam, perpetuating rote learning.

This artificially limits the potential of even those schools and students who have the capability to do much more. The fact that even many elite schools in India, despite not having administrative issues, perform below international average on PISA scores suggest that their true potential isn't being used. One of the major reasons is the lack of standards which set those expectations.

Miles to go - Independence Day 2015

In 2013, I had written in the context of elections that it's time to think about four  - focus on individuals and not on institutions, loss of hope among people, misplaced sense of rights vs. privileges and lack of middle class participation in politics. It's been just two years since then and we have made tremendous progress regarding these especially due to the changing nature of politics in our country. This is a reason for hope.

Politics is the greatest lever to bring about change. This choice of this path of reform was a double edged sword. Not being part of the politics meant that the situation is going to be worse but trying to be a part of politics meant significant compromise. Those who took up this path argued that given the constraints, some compromise is necessary and ends justify the means. Others argued that the very path of coming into government restricts the ability to do discharge the duties which is the original motive.  

Some well meaning people also tried to change this by starting a new political party but they all faced with five issues. Lack of leader with mass appeal, lack of ideology which can unite critical mass of people, difficulty building perception of winnability so that people don't feel that their votes are wasted, funding, the art of  contesting elections - campaigning, converting support into votes winning elections.

Thus politics as a path to reform was a difficult choice between moral compromise and pragmatism. So, what has changed? Two important phenomenon have occurred in the past two years.

Weak state capacity can reduce policy options

In 2012, Hanushek et al. published a paper titled 'Does school autonomy make sense everywhere?' The authors suggest 'autonomy may be conducive to student achievement in well-developed systems but detrimental in low-performing systems.' This result is important but the approach is more so. Usually, impact of a policy in a particular context is evaluated but the authors do something different. They "introduce the simple idea that the productivity of any input is directly related to the institutional structure of country that determines the basic environment and rules of schools, how decisions are made, the overall incentives in the system, and so forth." 

In summary, the analysis also considers the interaction between policy and the environment in which it is being implemented. This is a simple yet powerful idea. This is of high relevance in the debates on skill training, school choice and PPPs in education.

World Bank recently released  a report named 'Skills and Jobs - Lessons Learned and Options for Collaboration'. This has good summary of experience of skill training programmes from across the world and the experience. Not surprisingly, the skilling programmes differ widely in design across countries.
  1. Germany uses apprenticeship model where the firms (or individuals) are subsidized for training apprentices. The structure of the program, quality, content and other aspects are regulated by a central agency.
  2. In South Korea, SMEs weren't taking advantage of the funds earmarked for training due to lack of capacity. So, the government encouraged them to form consortium so that they could collaborate in providing training. Later, this expanded to include large firms and SMEs could learn from them.
  3. Singapore's Economic Development Board (EDB) invited Tata group to set up a training center. Government sponsored lands, buildings, 70% of the operating costs and paid the stipends for all trainees. Tata trained twice the number of staff that they need. At the end of training period, Tata recruited some people from this group and the rest were used as an asset to attract other engineering firms to Singapore.
It is important to note that the report says, "the strongest constraint in the transferability of such programs is that they require a strong institutional framework, in particular a clear legal framework."

There are two important lessons from this exercise.

One, weak state capacity reduces the policy options. The fact that capacity of a nation to enforce legal contracts and other aspects is weak automatically rules out the possibility of incorporating good ideas from other contexts. Choice to governments is as important as the choice to an individual. Lack of choices resulting from weak capacity is detrimental to already ailing systems.

Two, capacity to enforce regulations should also be considered as a factor while making policy choices apart from the usual metrics of context, cost effectiveness, political economy etc. Not all regulations are enforceable by everyone. The capacity to enforce depends on the strengths of the government, context, time and some times the amount of focus needed. Focusing on a particular policy to ensure its smooth execution also comes with its own opportunity costs, in cases of states with weak state capacity. All of these should be factored in while making policy choices.

For example, Rajasthan government is planning to draft a PPP policy for primary education. Some people have questioned 'If the state cannot oversee the functioning of its own schools now, what oversight will it provide for privately-run schools?'. Government failing to oversee public schools doesn't mean that it can't regulate private schools. The appropriate question to ask is, which policy can the government enforce better?

One must also note that the capacity to execute a policy should be used while prioritizing from among multiple policy options but not to filter the policy options. For example, if capacity to execute was to be used as a filter, then the Aadhar policy wouldn't have come into light. In such scenarios, if it is decided that it is necessary to implement the policy then putting extra efforts to ensure its proper execution is the only way out.

Industry boom can reduce returns to high school completion

It is widely believed that industry boom will increase returns to education and thus increasing investment in attaining education. This assertion assumes that education is required to get jobs generated due to the industry boom. A new paper by Prof. Cascio and Ayushi Narayan of Dartmouth College illustrate that this assumption needn't be true always.

Oil and natural gas extraction has increased in the last decade in US due to the advent of new technology called fracking. This produced lots of jobs, 6 lakh by an estimate, most of them being relatively good paying unskilled jobs, which don't need education. This increased the opportunity cost of continuing in school, it made sense for people to drop out and earn money instead of being in school. It impacted the educational attainment and dropout rates in these areas.
We explore the educational response to fracking, taking advantage of the timing of its widespread introduction and the spatial variation in shale oil and gas reserves. We show that local labor demand shocks from fracking have been biased toward low-skilled labor and males, reducing the return to high school completion among men. We also show that fracking has increased high school dropout rates of male teens, both overall and relative to females

This can possibly happen due the combination of four factors. One, when the schools aren't value adding, at least as perceived. Two, when there are economic pressures in the family of child to earn money. Three, there are easily available jobs which don't need education. Four, when the investment required on education in terms of time and thus the money, to be able to attain a job is high.

Compulsory voting in Gujarat - Will it decrease voting percentage?

The Indian Express has a good primer on The Gujarat Local Authorities Law (Amendment) Bill, 2009, notified recently. This law makes it compulsory for everyone to vote with ten acceptable cases of exemptions. A fine of Rs. 100/- will be levied on the non-voters. Notice will be sent to non-voters by the election commission and person will have to reply within 30 days. If the reason for not voting falls under any of the specified ten categories, one can approach election commissioner, submit the documents for the same. Interestingly, attending marriages and other functions is also one of the acceptable reasons for non-voting and in these cases the election commissioner is supposed to verify these by looking at the pictures of the function.

While there are debates about the desirability of such law in terms of personal liberty and on the feasibility of implementation, this is an interesting context to see how incentives play out because some studies suggest that small penalties sometimes increase the behaviour for which they are penalized.

An influential paper A Fine is a Price explores the exceptions to the deterrent hypothesis which says that levying fines will reduce the undesired behaviour. This is a famous study which is also discussed in Freakanomics. There was a pre-school where students were parents were coming late to pick up their children. School staff had to stay for long hours after the school hours to take care of the kids. So, the school announced a fine of small amounts for each hour of delay in picking up. It expected that this fine will act as deterrent and parents will come and pick up their children on time, as per the conventional wisdom of the deterrent hypothesis. However, it turned out that the delays have actually increased.

Comments on the discussion paper on high skill migration by Michael Clemens

I had earlier summarized discussion paper on high skill migration by Michael Clemens I had just quoted relevant excerpts from the paper in the earlier blogpost, just to ensure that I don't misinterpret him. I also refrained from making my comments on the paper. Having said that, I have three comments/doubts on the paper and two ideas specific to India, which can be an interesting aspect to explore.

To provide a context to the comments, I was initially of the belief that one should let individuals pursue their interests and not restrict them from migrating and that instead of spending efforts to block people, governments should create conditions making it attractive to stay in their country of birth. 

This was until I saw this report which said that there are only 120 medical doctors in the entirety of Sierra Leone country. If there are only 120 medical doctors in the country, and if some of them decide to leave, should the country allow them considering the scarcity of doctors in the country? These doctors might be more productive abroad than being in Sierra Leone but in cases of scarcity, something is better than nothing. Is this a sound case for blocking migration? [Even if it is the case that there are more than 120 medical doctors in Sierra Leone, for argument sake, let us consider this as a hypothetical example.]

I will be using this example in the comments below. 

Comment 1: In the paper, number (proportion) of people with tertiary degree is considered as metric for skill stock. But not all those with tertiary degree have same amount of skill. I understand that measuring this skill through value added is difficult to measure, since the value added is also dependent on the environment the person is present in, but theoretically it would be different from the mere number of people with tertiary degrees. If we interpret skill stock in this manner, our inferences might also change. For example, consider this graph

What if we had forced Sundar Pichai to stay back in India?

[For low skill migration, I suggest reading Lant Pritchett's 'Let their People Come'.]

Sundar Pichai, educated at IIT Kharagpur in India is now appointed as CEO of Google. This brings back the debate on effects of high skill migration. The common concern being that migration of high skilled workers, often educated with public money is harmful to India. In this context, this post summarizes recent discussion paper Losing Our Minds? New Research Directions onSkilled Migration and Development by Michael Clemens. Quoting relevant excerpts from the paper would be an apt description, I suppose.

The paper argues that traditionally the research on migration has been with the assumption that, migration harms development. With this assumption, the further research was about estimating the extent of the harmful effect and about policies to obstruct migration. "But that paradigm has accumulated an increasing load of anomalies that it cannot accommodate. It may be time for the literature to move forward resting on a new set of axioms", the paper says.

1) Let us consider this argument that the skilled migration has substantial negative effects on the skill stocks in the country. There are two effects at work here. One is the direct effect where migration of one person means one less worker in the country. Second is the indirect effects which include a variety of factors - growth, famine (famine or wars in country can make people migrate), reverse effects (as high skilled people migrate, this can increase expected returns of education and hence more people try to acquire these skills). 

Let us look at this graph to see which effect dominates.

Definition of high stakes examination

High stakes testing is a commonly used term in education space. Broadly, high stakes testing is of two types. One, high stakes for teachers where student scores decide their pay and terms of contract. Two, scores decide students' further opportunities.

It is also perceived that the gaokao exam in China is of very high stakes and has perverse incentives. But then what about 10th board exams? IIT JEE, CAT? Are these high stakes? The other argument here being, the perception of stakes is subjective and can widely vary. In this context, I tried searching for a concrete definition of the term 'high stakes testing for students' in the context of education but I couldn't find one. This post attempts to build one such definition.

I hypothesize that a high stakes test for students satisfies all the following criteria.

1) Exam completely denies opportunities ahead: An exam conducted in a school where teacher strictly orders child to perform well might seem high staked to a child but it doesn't come under the category of high stakes testing that we talk about.

We need to see if the exam determines whether the child can pursue career ahead in that field. For example, if we were to say, those who don't get x number of marks in 10th grade can't take up Maths in +2, we are completely denying further opportunities to students here. This would make the test high stakes. Note that this is different from students with low marks getting admissions only into not-so-good colleges. In this case, students are getting admission at least somewhere, they aren't being denied the opportunity completely.

Reading list for beginners - Public Policy, Development, Growth

My purpose of writing this post is to reiterate the importance of reading and provide a context and broad framework for beginners, who are interested in the field of development. I am neither an expert in this field nor do I have any formal training. In fact, the administration at my undergraduate college, refused to allow me to take up economics courses, because they thought I am unfit. I set out on this journey of exploration alone and hence I feel I can appreciate the constraints of a beginner without formal training. Hence, this blog post.

Three words of advice before we move onto the reading list. 

Want a reform? Start with proposing the extreme

I have a theory on policy advocacy - demand the extreme. In a negotiation, the one who first compromises (a lot?) automatically forces others to compromise at least a bit, to appear fair. So, always demand the extreme. What does this mean?
For example: Take the example of powers of RBI governor. If you directly say, we should have a committee with RBI majority instead of a single person, the debate would have been around single person vs. committee with RBI majority. There is no middle ground here for both parties to settle. In such situations, the debate becomes binary and in any negotiation, no party wants to appear to have lost. So, the consensus becomes difficult.
Instead, the other way to do it is, propose a committee with government majority. This time, instead of debate on single person vs. committee with RBI majority, there will be a debate on who should have majority in the committee, pushing the point of debate to the right. In the end, you agree to budge from the proposal of committee with govt majority to a committee with RBI majority. This would make opponents happy to have made you budge and you too have achieved your initial goal.
Nothing big. Usual principle of bargaining in roadside shops. Start with lowest possible and say - I have compromised this much, so you should at least compromise something.
The other example of this tactic is the JEE reform by Sibal. Initially Sibal made an extreme proposal, IITs didn't agree to it but Sibal was adamant about it. In the end, in order to resolve the debate, a middle path was proposed, which finally ended up pushing IITs from their original position.

Issues with skill training programmes

In this post, I mostly raise questions than to answer, regarding the skill development programmes. Hope to know the answers to some of these with time.

1) The need for skill training programmes assumes that there is a gap in demand and supply of skilled labour. The question to then ask is - if companies are unable to find people with skills, why don't the companies train people themselves? When Nokia set up its factory in Sri Perumbuduru, it trained its own people. Why isn't happening in other cases? Of course, it needn't be the case with all companies. Some small companies may not be able to train the employees due to the fixed costs involved. But, how big is this gap?

Also, as this outlook article also suggests, some companies seem to be saying that most skills required for the job are learnt on the job

2) Is this gap only purely technical? The following excerpt would suffice from this article 
So if some alleged skill gaps are really just reflections of firms operating on tight margins, why are employers talking about skill at all? Sometimes, it turns out, they aren’t. Several employers indicated that to them ‘attitude’ is a key component of skill. By attitude, they mean that workers should have a strong work ethic, be willing to make sacrifices for productivity (working night shifts, for example), and not be likely to add to the labour relations’ problems of firms. Two firms reporting highly cyclical demand actually indicated that a core ‘skill’ problem was how to have trained manpower when they needed it, and to not have to pay for it when they didn’t. In short, many concerns actually relate to labour relations and regulations, or simply the availability of manpower on demand, and fall outside the largely technocratic rubric of ‘skill development’.
Firms suffering economic skill gaps operate very differently from firms that are constrained by other factors, and they offer very different employment opportunities. For the benefit of workers and employers, it is imperative that we learn to tell them apart.
If this is an issue, how big is this and in which sectors?