Misinterpretation of farmer suicide data

Shamika Ravi has published an article on farm suicides using NCRB data. She makes four important claims.
1. The number of farmer suicides has actually been falling in recent years; fewer such deaths were recorded in 2016 than at any time in the previous 16 years. Nearly twice as many Indian housewives commit suicide as farmers do. 
2. Suicide mortality rates are higher in the relatively wealthy states of Maharashtra and Andhra Pradesh than in poorer Bihar and Uttar Pradesh. 
3. Nearly twice as many Indian housewives commit suicide as farmers do. 
4. India’s farmers are doing far better than many realize. One way to gauge the well-being of rural households is to look at how much they’re buying: they now account for 45% of the fast-moving consumer goods sector in India. This is remarkable given the vast disparity in disposable incomes between urban and rural households, and it implies that improvements in rural infrastructure, connectivity and digitization are translating into higher demand. Over the last three years, rural sales grew significantly faster than urban sales in both volume and value; consumption growth currently stands at a robust 9.7%.
It has been published by Indian Express, Business Standard, The Print, Bloomberg, and Mint. This has been commended by many others using terms like "hard data", "evidence-based", "logic based" and so on.

In reality, this article is exactly opposite to what it is claimed to be. It does a data abuse of serious proportions and logically incoherent for the following reasons.

One of the first sanity checks done in any data analysis before reaching such far-reaching conclusions, of the kind that the article does, is to first understand the data. The article shows no hints of this, even remotely.

NCRB data methodology changed post-2014. Hence, the fall of suicides witnessed in 2016 is not comparable with the data till 2014. This has been pointed out numerous times earlier by many people and in particular P Sainath (herehere, and here). In brief, the issues are 

  1. Lack of training of the who collect the data, who are unaware of the change in the categories in NCRB data.
  2. States started reporting zero suicides from around 2011. For instance, as P Sainath points out, Chattisgarh and West Bengal started reporting zero suicides from 2011. But the three year average of the number of suicides for these states before 2011 was 1,567 and 951. This is not to deny that farmer suicides can decrease. But, it is only absurd to think that a yearly average of 1500 and 1000 came down suddenly to 0 just within a single year. So, in these states with 0 suicides, the number of farmer suicides is less than that of other categories like young, housewives etc. Surely, the farmers must be living in a paradise in comparison to other professions!
  3. Police put "tenant farmer" suicides in "agriculture labourers" field because many tenant farmers do not have pattas. In 2014, farmer suicides fall by 50% accompanied by an increase in agriculture labour suicides.
  4. Increase in "Others" category of suicides. For five states accounting for 70% of suicides, "others" column went up by several hundred percent, along with the decrease in the farmer suicides. It's because states started categorizing farmer suicides in Others.
  5. Women farmers are put under "housewives" category because of patriarchal reasons and also that they don't have their names on pattas. It explains the high number of housewive suicide as compared to "farmers".

For all these reasons, the claims of the decrease in farm suicides in 2016, poor states having zero suicides, and suicides being higher in other categories of people are absurd.

One can debate about the solutions to farmer suicides - loan waiver or input subsidies - but it is criminal to deny even the existence of the problem, that too with intellectual dishonest acts of data dressing.

It is expected of a serious researcher to do basic data sanity checks before making such far-reaching claims. Unfortunately, neither did the researcher nor the numerous publications that published the article and those who sang praises for it bothered not to. More importantly, this is not the first time that Shamika Ravi has resorted to such data dressing. In my memory, this is at least the third such instance, the previous instances being her analysis of Odd-Even policy and demonetisation, which I had pointed out earlier.

No wonder that Dr Arvind Subramaniam once expressed concern over the sycophancy of Indian economists.

Prerequisites for lateral entry into bureaucracy

If you get an eye infection or if you want a cataract surgery, you go to an ophthalmologist, if you need a neurosurgery, you go to a neurosurgeon. But what if the General Medicine doctors end up dealing with all these cases requiring specialist requirement? It would be a disaster. It's the reason generalist doctors dread giving a medical prescription for issues that require special attention. 

Turn to policymaking, it's not the case, in fact, it's even worse. Generalists, who are not even generalists in policy analysis, but primarily implementers, design policies. We all are being operated by general medicine doctors for every ailment! It is definitely not a desirable scenario. For this simple reason, there have been demands for incorporating specialists in policymaking. 

There's been a wide debate and several commentaries on the pros and cons of lateral entry into bureaucracy. I am not going to discuss all that. In this post, I would like to touch upon an aspect which seems to be missing in the debates, the nature of responsibilities of a policymaking bureaucrat in India.

The lateral entry of specialists at higher levels of bureaucracy presumes that bureaucrats at that level are only into policymaking. It's not true in the case of India. Bureaucrats at such level deal with both the implementation and policymaking aspects. For instance, a sanitation secretary would also be monitoring the implementation of the toilet construct and be accountable for those numbers, while thinking about broader policy issues at the same time.

In such cases, if an outside specialist with no experience in Indian public administration is brought into this role, it would be a disaster. Getting things done in India's system needs a good knowledge of inner dynamics of administration, for which the bureaucrats are trained for years on the ground. The traditional bureaucrats have definitely an edge here.

So, if we are to bring external specialists to higher level policymaking, separating these executive and policymaking rules should be the pre-requisite. Lateral entry would be ineffective without doing such reform.

Again, this is not a unique insight. Even 2nd Administrative Commission recommends separation of executive and policymaking responsibilities of the senior bureaucrats, on the lines of UK.

In summary, lateral entry into policymaking levels of bureaucracy is highly needed but should be preceded by an administrative reform that separates the execution and design roles of senior-level bureaucrats.

Unfair criticism of RCTs

Commenting on the cynicism I noticed in policy circles about other's ideas, I wrote: "An established policy is judged by where it works, while a new policy is judged by where it does not work". I meant to say that often a new policy is dismissed for not addressing "all problems" but the status quo problems are continued and accepted despite addressing a lesser number of problems than the proposed one.

One of my friends read this and commented that it's not just the case with policies but it applies in general to organizations and humans too. Historically under-represented sections of people need to work "extra hard" to receive the same level of praise or recognition the dominant people in status quo get.

Unfortunately, it is true in the case of the criticisms on RCTs. Consider the following criticisms of RCTs

1. Show one RCT that resulted in a policy change. 
2. Only certain kinds of questions can be answered with RCTs.
3. RCTs displaced other studies

Each of these criticisms has the status-quo bias. To make it evident, ask the same questions on any other form of study.

1. "Show one qualitative study about the resource constraints in bureaucracy that resulted in direct policy change". Despite the availability of numerous such studies, why does this problem remain unaddressed? Does this mean that qualitative studies are useless?

2. Is there any one particular method of study that is universally applicable to all types of problems? Why is this question asked only in case of RCTs? Why don't people ask the same about other forms of research? 

3. The criticism of RCT displacing other studies sees research as a zero-sum game. Esther Duflo has, in fact, demonstrated that RCTs did not displace other studies. Instead, they only added to the existing number of studies.

The underlying reason for these criticisms is the lack of appreciation of the nature of research, misrepresentation of RCT researchers, and non-questioning of the existing state of policymaking. 

1. The nature of research in social science is that the attributable change doesn't often happen due to a single study. A series of studies shape an alternative narrative that results in change given the right conditions. Somehow, people miss this while criticizing RCTs.

2. No serious RCT researcher ever says that RCTs are the only necessary and sufficient evidence, and input for policymaking. For instance, consider this policy paper by Prof. Karthik Muralidharan where he clearly says "Policy formulation needs to consider technical, administrative, ethical, as well as political factors and even the best technical studies can only provide inputs into one dimension of policymaking." But people criticize as if RCT researchers profess that only RCT evidence is the necessary and sufficient one!

3. In their criticism of the RCTs, especially the bureaucrats don't question the existing state of policymaking. They need to question if the so-called irrelevance of RCTs is due to lack of relevance or inability to incorporate it in policymaking. I suspect that it's the latter because - forget about RCTs - policymakers often don't even use the other forms of existing research. Worse, a recent survey suggests that the top bureaucrats of the country can't do a simple and basic data analysis. Interpreting and incorporating evidence is a long shot in such case.

Further, on the criticism that "foreign researchers" don't research on relevant problems, the critical constraint, in fact, is the bureaucratic maze around getting necessary data. A prominent India-based researcher had once commented that the only way to do good macroeconomic research in India is to be in good books of RBI because it provides access to research data. This is, in fact, true of the nature of research in social sciences. Change doesn't often happen due to a single study. A series of studies shape an alternative narrative that results in change given the right conditions. Somehow, people miss this while criticizing RCTs.

In such situations, we would be better off throwing open the access to research. I am sure the gates will be flooded because research is where data is.

Overall, the criticism of RCTs being irrelevant, displacing other studies, and not resulting in direct policy change are unfair because every other form of research fails if subjected to these criteria. If other forms of research are not subjected to these criteria, why should only RCTs be?

The most important concern is the lack of access to quality data for the researchers. We would be better off if we focus on addressing this problem instead of debating on the merits of different methods on research.

Reducing corruption across sectors is the only way to fund education reform

Education reform needs a significant amount of money. Governments' lack of seriousness in reforming education is definitely an issue that prevents them from going the extra step to allocate necessary funds. But, there is another important issue as well - we simply don't have enough funds for education, even if a committed government comes to power and desires to allocate funds.

In some states, the money required to fill up vacant posts itself would amount to significant part of the current budgets. If one were to calculate the total money required for all the reform and presume that it should come only from the annual budget, it would hardly leave anything to spend on other sectors.

Politicians and bureaucrats commonly cite this as the reason for the State's inability to increase its expenditure on education. Such arguments seem reasonable but one should note that such arguments consider the status quo of inefficiencies and corruption as default, which needn't be the case.

A significant amount of money can be saved by reducing inefficiencies and corruption in expenditure across all departments, not just education, and also collect additional money by reducing inefficiencies and corruption in tax collection. It requires administrative reforms of massive scale accompanied by political reforms that correct the political economy incentives.

For instance, it requires reducing corruption and delays in infrastructure construction, which is interlinked to political funding because many contractors are funders of political parties. It means that political parties should reduce their dependence on such people for funding. It further requires a transparent electoral funding and a break to the practice of distributing money and liquor in elections, which puts a huge burden on the parties. Reducing inefficiencies is thus interlinked to political reforms.

Given that the scope to raise money through additional taxes is limited, saving money by reducing inefficiencies and corruption across government departments is the only way to make enough money available for education and related sectors.

The education reform is thus invariably linked to broader governance and political reforms. It would be great if education policy advocates realise this aspect and thus contribute their voice to the broader reforms as well, along with the education.


People have taken up #March4Education demanding more financial support to higher education institutions, of which more universities is also an implicit demand.
The popular counter-argument is that only the school education should be fully funded, while the undergraduate and postgraduate education shouldn’t be funded and be left to market forces.
Merely reiterating that public funding should be restricted only till school education, and fund higher education in this day and age is a classic example of lazy thinking, intellectual hollowness and the lack of thinking beyond 101 textbook ideas. It perfectly fits what Hirschman once said that he hated:
“I always had a certain dislike for general principles and abstract prescriptions. I think it’s necessary to have an “empirical lantern” or a “visit with the patient” before being able to understand what is wrong with him. It is crucial to understand the peculiarity, the specificity, and also the unusual aspects of the case” Albert O. Hirschman
I had earlier blogged saying that it’s time to retire the idea of restricting the definition of basic education to only school education, and expand it to higher education too.
In the context of the #March4Education and its demands, I summarize my arguments in support of the protest and response to the criticisms to the protest.

Response to the argument that extent of funding should be based on externalities

Flimsy defense of National Health Protection Scheme (NHPS)

Livemint has an op-ed on National Health Protection Scheme (NHPS), otherwise known as Aayushmaan Bharat. The article argues that NHPS is a good idea and suggests ways to ensure smooth implementation. It is a flimsy defense of NHPS.

The article concludes that NHPS is a good idea based on the following arguments.

1. "Out-of-pocket payments for healthcare services are very high in our country (about 70%, according to the National Sample Survey Office, 2014), which causes impoverishment to nearly 7% of our population."

2. Evidence from Karnataka's Vajpayee Aarogyashree programme lowered mortality for covered diseases and erased rich-poor disparities in concerned mortality rates. It also lowered out of pocket expenditure.

3. "Existing evidence shows that providing insurance to the poor not only saves lives but is also “cost-effective.

The above three points are either inappropriate or incomplete arguments in defence of NHPS, far from terming it as a step in the right direction.

1. There is no debate on the fact that the out-of-pocket expenditure on health care is huge in India. But, National Health Accounts data points out that 42% of the total out-of-pocket spending (OOP) is used towards buying medicines.

Given this scenario, a simple and significant step to reduce OOP would be to make all medicines free. Expenditure on NHPS with no coverage for medicines, while still requiring people to buy medicines for other conditions too, may not be of much use.

It is, in fact, pointed out in one of the author's own paper.

"people who had health insurance coverage did not see any significant difference in their total real OOP health expenditures, relative to people without any health insurance"

2. While the authors agree that not every insurance programme has been successful, the cited programme VAS is not a representative example. A host of other programmes have failed but one can give a benefit of the doubt regarding the implementation quality. It thus brings us to the next aspect - the range of conditions and cost-effectiveness.

3. Surprisingly, the article claims the existence of evidence which shows that insurance programmes are cost-effective. The evidence on the same is to the contrary. Insurance programmes are anything but cost-effective. Starting from the US where they spend 18% of GDP but still don't get timely and quality care to the Indian data, insurance programmes are known for NOT being cost-effective, as Gulzar Natarajan points out. 

4. The argument for primary and secondary healthcare as a prerequisite for good public health system is well taken but the budget allocation to the same is not proportional to its importance. The right question to ask is - assuming that there is an increased spending on health care, where should the increased funding go? Of course, if it ends up being the case that the overall expenditure on health care is NOT increased but money is rearranged towards NHPS, its a lost battle.

Overall, with no increased funding for primary and secondary care, with no coverage for medicines (42% of OOP), and with our existing weak state capacity, the defence for NHPS is flimsy.

The way out is to retain private insurance coverage for terminal illnesses with expensive treatments like cancer, but increase the spending significantly and use a bulk of it towards primary and secondary health care.

Comments on RS TV discussion on education reform

RS TV has conducted a TV discussion on education reform, in the context of government’s decision to reduce the syllabus by half.

The ideas and analysis presented in the discussion is symbolic of everything that’s wrong with the education reform discourse in India.
Consider two key points raised the discussion — “low salary of teachers” as the binding constraint of teacher quality, and a “library movement” as “the solution”.
The reasoning is that talented people aren’t becoming teachers because of the low pay, as compared to the corporates. It seems an obvious reason, so obvious that it clouds our vision preventing us to look beyond. But if we take a step back and probe this further, a different picture emerges.
As I have argued in my book, one needs to ask three questions
  1. If teacher salary is the issue, what’s the “comparative outcome” of teachers of different grades, receiving different salaries?
  2. In a counterfactual scenario, if teacher salary is the binding constraint, then increasing teacher salary should have significant effect on the outcomes.
  3. Finally, what’s the teacher salary in India, compared to our per-capita income?

A framework to think through a decision on a career in Public Policy

Some people have asked me for my thoughts on a career in public policy. I am synthesizing these in the post below.
The essence of my suggestion is that
the decision on a career in public policy should be based on 3 factors — nature of the person, nature of the problem that the person is interested in, and the available path.
In the following post, I first describe what I mean by each of those 3 factors mentioned above and I put the various combinations arising from these 3 factors in form of a matrix at the end of the post. If you directly want to jump to the matrix, please feel free to do so.

I. Nature of the person

The “nature” of a person has wide connotations but in our present context, we can categorize people into three broad categories. These categories are illustrative and there can be overlaps between them.
  1. Problem Solvers: These people are obsessed with the policy problem. They would go to any extent to solve the problem. If they need to do a massive protest, they would do it. If they need to enter politics, they would do it. If they need to sit and research for years, they would do it. In essence, they are path agnostic. They will figure out their path. They don’t need advice! Such people are analogous to entrepreneurs in business.
  2. Career-ists: Some people are interested in policy sector but they don’t want to take risks. They would like to pursue it more as a career, with stability. Such people are categorised in “career-ists” group. Such people are akin to MBAs, who want to be in the private sector but don’t want to take the risk of becoming an entrepreneur. They get an MBA, enter the corporate world and climb the ladder.
  3. Satisfaction-seekers: Such people are seeking satisfaction, something beyond the materialistic pleasures of life. Often, they want immediate tangible outcomes. For instance, the joy of donating money to a poor person.
Just to be clear, there’s no judgment involved in categorizing people. It’s just a description of people’s nature as it is.

On private health insurance

Government introduced private insured health-care for the poor up to Rs.5 lakh per year. I have only few things to say.
After spending 18% of its GDP on health-care, US’s health system is no better than some public funded systems like Thailand. This is in a country with higher implementation capacity than India’s.
By weakening public health system through inaction and making a complete shift to insurance model, we might be moving planning our own disaster. More on this later.

Teacher voucher policy as a compromise between government run schools and student vouchers

  1. Learning outcomes and cost effectiveness of budget private schools (BPS) don’t make a strong case to shift focus from government schools to BPS.
  2. However, if the current situation of neglect of government schools continue, two another parameters financial stress on families and plausible improvement in current state of BPS due to additional funds from vouchers — makes a case to shift focus from government schools to BPS.
  3. Since student voucher system may not be feasible in short-term due to financial constraints, a voucher policy for BPS teachers to help them receive better training is a good compromise policy.

There are two common perceptions regarding budget private schools (BPS)
  1. BPS perform better than government schools, even after accounting for differences in student characteristics.
  2. The learning per unit cost (cost effectiveness) is higher for government schools as compared to BPS.
These two arguments are cited as basis to move towards a voucher system where students get vouchers from government and they are free to choose the school they like, over the present system of government run schools where students’ choice is restricted. For instance, this Mint editorial makes the above two arguments to advocate for school voucher system.
Though these two arguments seem intuitively correct, careful research suggests that they aren’t necessarily true.

Learning the wrong lessons

Azim Premji University recently conducted a longitudinal survey of schools following two types of pedagogy — Activity Based Learning, and usual textbook based pedagogy. It says the following
The study assessed the ability of students to recognize characters in the local languages, and while students could recognize the root alphabet, they struggled to identify the compound character formed by joining “a” to the root character. While 32% students in Palghar could recognize the compound character, only 22% in Yadgir could recognize the same.
The lead researcher of the study says
Often, poor learning outcomes are attributed to lack of diversity in curricula, this research shows that even an activity-based curriculum couldn’t help the students as the teachers were ill-prepared and lacked the pedagogical tools required to teach Indian languages
How do we interpret the results of the study?
  1. Is it the issue with pedagogy — teachers lacking pedagogical tools?
  2. Is it the issue of systemic capacity/constraints (Gulzar calls them ecosystem-constraints) that constrain teachers’ efforts or overlook their lack of efforts?

Why do governments’ efforts to reform education not result in success?

    Okay. I have written too many posts on this topic. But, I learn something new each time I write, a new connection or a better phrasing of my arguments. Here is the nth version on this topic.
    My central arguments are:
    1. Policies are narratives writ large: The popular mental frameworks through which the problems are approached in policy narratives also end up being the lens through which policymakers also approach the problems. Therefore, errors in popular narratives also translate into policy.
    2. Problem with education policy is the mental frameworks through which the problem is approached. It has less to do with a particular policy. These mental frameworks are a part of the popular discourse that permeate into policy making. In particular, they are:
    • Misconceptions about the constraints
    • Silver Bullet Constraint approach where there is a quest to narrow down the problem to one or two constraints
    • An implementation framework with disregard to the nature of task of education and monitoring norms suitable for it

The Lost Einsteins

Now, we largely know that family characteristics and environment have a significant impact on a child’s career trajectory; in some cases, these factors even trumping the quality of schools they attend.
What is it about the poor families that makes children from these backgrounds less advantageous?
This is an important question because if the answer is good instruction at school, then fixing school will do the job. But, if the answer is that child’s disadvantages also arise due to out-of-school environment, then we need to expand the spectrum of our efforts.
My theory, as I blogged earlier is that the culture of expectations and exposure to opportunities in higher-income family environment, differentiate them from that of low-income family environment.
Culture of expectations is present in poor families too, who see education to get out of poverty but the exposure to opportunities is important. Such exposure does two things. One, it illustrates the wide range of possibilities and also removes mental barriers in process of deciding “what to do”. Two, the exposure demonstrates high likelihood of success building the confidence. Role models play an important role here. Prof. Anirudh Krishna has demonstrated its importance in Indian context.
Apart from income, the other social factors like gender, race, caste also are important.
Now, we have a new paper adding new piece of information to this analysis.
Raj Chetty et al. in his recent paper throws light on this question — what is it about the high-income families that makes their environment different from that of low-income families? [Read this for the summary of findings]
First things first, Chetty et al, demonstrate that there is a difference in the likelihood of success, based on income.
children from high-income (top 1%) families are ten times as likely to become inventors as those from below-median income families. There are similarly large gaps by race and gender.
To what extent is this difference due to the innate ability of students? The answer is — not much.
Differences in innate ability, as measured by test scores in early childhood, explain relatively little of these gaps.
Next, they explore the channels that drive the high likelihood of success in high-income families. They argue that the “exposure to innovation” is the key driving factor. They demonstrate it as follows
Growing up in a neighborhood or family with a high innovation rate in a specific technology class leads to a higher probability of patenting in exactly the same technology class.
The fact that exposure to specific technology leads to success in the same technology class demonstrates that exposure is the key driver.
There are gender-differences in this.
exposure effects are gender-specific: girls are more likely to become inventors in a particular technology class if they grow up in an area with more female inventors in that technology class
Overall, there are two lessons from this new piece of research.
  1. Exposure to innovation or broad possibilities in general is the differentiating factor between low-income and high-income families. So, policies trying to bridge the income gaps, should focus on creating exposure to the kids. Role models and technology are important tools in this process.
  2. The closer the role models to the socio-economic characteristics of the child, the better.
At a given point of time, there might not be enough role models for all local communities. One idea can be to pick up the students with promising potential from each local community, give them all possible support and demonstrate their success. It’s a long term project, but it will have impact in the long-term.
It may seem vague and abstract idea but it’s important to do this as part of education reform. If not, we are “losing many Einsteins”.
I like the analogy of “oil drilling” here. We take huge pain to explore all over the earth to mine the oil and create value out of it. Education is something similar. The human minds are mines of intelligence, waiting to be explored. The more we nurture them, the better for us. We might be losing out a lot by the virtue of not exploiting this valuable resource!

Technology based medical diagnosis

I have been recently exploring health sector. Unlike education, where everyone has seen schools from inside and have experienced, medical sector is a black box.
The excellent TV Series, House M.D, along with conversations with few doctors helped me realize two key aspects of medical diagnosis
  1. The key job of a doctor is to diagnose the disease with minimum symptoms.
  2. The key constraint in step 1 is the cost of medical tests.
Technology can help enhance doctors’ performance in both the steps.
Let me explain what the two aspects mean, before discussing the role of technology.
1) Job of a doctor: Suppose a patient goes with a headache and stomach ache. What does it mean for the doctor?
A headache can be due to numerous reasons and so can be a stomach ache. Further, headache and stomach ache may not just be stand-alone aspects, they can be symptoms of an underlying major disease.
The doctor’s job now is to use these signs to narrow down the possible diseases that explain these symptoms. This is essentially a mental exercise of matching these signs with diseases that the doctor is aware of! It’s called “differential diagnosis”
Currently, doctors do it using their memory. The problems with this are
a) doctors may not be aware of all possible diseases
b) even if they are aware, they can’t do the huge matching exercise
c) even if the doctors are aware, they might subconsciously discard some diseases because the probability of their occurrence is too low!
As one can see, this is an extremely inefficient process prone to high errors. It’s too much to expect doctors to remember many possible diseases, symptoms and match them instantly.
The key to improving it essentially involves two parameters — doctor’s memory and capacity of match-making process.
We may note that these exact things — memory and processing — are the two aspects that the computers are really good at! Hence, technology would be of a great help here.
A technology tool here would be one where the doctor inputs the symptoms and the tool does the matching and throws out the possible diseases!
This can improve doctors’ efficiency multiple times. It is primarily an IT challenge.

When NGO thinking percolates into policy narratives and policy making

    The danger of “attributable to me” approach is not in the non-government sphere. The real danger is in its percolation to the government sphere.
    Lant Pritchett in his usual brilliant style has mounted a new critique on the narrow RCT led development.
    Lant’s primary argument is that development is increasingly pursuing “attributable to me policies” rather than “optimal policies”. As an example for non-attributable research, he quotes the example of research that has built the narrative on necessity for reforms in India’s pre-1991 era. Research of this kind, Lant says, has contributed more to the increase in incomes of people than the narrow RCT led research.
    Lant argues that an important consequence of RCT led development is that
    individual actors care more about what they can take credit for than whether there is team success
    In these arguments, Lant largely focuses on the non-government sector and academia, where there is an
    • increasing tendency in NGO sector to fund only projects and research that can demonstrate causal effect; and
    • increasing tendency in academia of development economics to shift towards studying narrow questions (RCTs) rather than the big questions of development (what causes growth?).
    In this post, I argue that the dangers of perils of attribution is not in NGO or Academic sphere but it is in the percolation of such thinking into governments’ sphere.
    • NGOs: NGOs operate in a fundamentally different manner from the government. They are focused on a key problem, often a narrow and local problem. They can pursue only one aspect of the problem, keeping others aside. In such cases, the RCT led approach is useful because without it, we are just shooting in the dark, especially spending the valuable resources.
    • Academia: In academia too, the increase in RCT led research need not be an issue to worry. It might be a much needed correction. Historically, development research has focused on the macro question of how nations grow but it neglected the question of how do households come out of poverty. RCT research has just plugged in this missing gap.

Reasons for the death of small firms

Failures of firms led by women are linked to illness and family reasons

David Mckenzie has a good post on the reasons for the death of small firms in developing countries. He summarises the results into three categories.
  1. Most common reason for firm death is that less profitable and less productive firms end up making losses and closing.
  2. Firms, largely run by more educated owners, close because better opportunities arise for the owner.
  3. Small firms, particularly those run by women, close because of illness and family reasons.
The last one is an interesting fact. Firms led by women close because of family reasons. I suspect there’s some underlying sociological reason to this pattern. It’s an interesting area to be explored:
  1. Is this because women bear the additional burden of taking care of the household activities in traditional societies, that their businesses are more likely to get affected when family work load increases?
  2. Is it because women are more likely to divert their business resources to family, should a need arise?

Blinkers of education research and advocacy ecosystem

The problem in education research and advocacy is that no one ever identifies the lack of comprehensive reform as the issue. Everyone is trying to push through what they think are “new ideas”. It leads to fragmented and unproductive reform.

An IE article argues for recruiting more female teachers in secondary schools. The argument is that female teachers can serve as aspirational role models, increasing enrolment and investment in girls’ education.
This is reasoned based on an evaluation that found that reserving seats for women in panchayats had an “aspiration effect” in those villages, where the perceptions of women changed, investment on education increased and so on.
All’s well and good but there’s one small problem.
The percentage of female teachers in secondary schools is already 42%.
Given that the percentage of female teachers is already high, one can’t imagine significant marginal utility of recruiting female teachers over and above this. In any case, even if the government were to reserve 50% (maximum that’s possible), it’s only 8% above the current value. It is unlikely to yield any significant effects.
The kind of analysis in the article is reminiscent of a larger problem in the public policy eco-system.
  1. RCTs are interpreted too narrowly, often without context. The article in discussion is a good example. A particular evaluation may show effect because it addresses a critical constraint. But before translating it to other context and advocating a solution, one should check if that constraint actually exists and if the proposed solution has already been tried. Often, it isn’t done. As in the case of the above article, increasing female teachers is advocated without even considering the current percentage of female teachers. Such blind transposition is evidence waving policy, not evidence based policy.
2. A single issue is taken and a narrow solution is proposed as an idea of reform.I argued in my book that this is in fact unproductive and hurtful in the long-term for several reasons.
I argue in my book that there’s something called “discourse bandwidth” in policy. At a given point of time, only a few policy ideas can occupy the prime discourse. The ideas at the top end of discourse bandwidth have higher likelihood of turning into policy.
We should hence be careful of the ideas that we advocate. Pushing through one idea displaces others. So, if a narrow solution to a narrow problem is proposed, it crowds out other important problems and other important solutions.
3. Education is not a wheel with spokes, where adding each spoke (policy reform) incrementally adds to its strength. It’s like a pipeline, where the water reaches the end point only if all the parts of the pipeline are working.
It means that addressing one issue (policy idea) isn’t going to result in outcomes unless it’s accompanied by many other complementary policies.
Part of the problem in education research and advocacy is that no one ever identifies the lack of comprehensive reform as the issue. Everyone is trying to push through what they think are “new ideas”.
Thisas discussed earlier, crowds out both policymakers’ attention and also space in discourse bandwidthThis leads to a fragmented, narrow reforms that end up being unproductive.
The blinkered approach to education reform and advocacy was there since long. It only got exacerbated with the advent of RCTs, one of the negative effects of RCTs!