Understanding state capacity in the context of education and how to build it using PDIA

Gulzar Natarjan in a post on 'iterative adaptation' remarks that "approaches like PDIA, by their very nature, require strong states". (PDIA is Problem Driven Iterative Adaptation, pioneered by Pritchett et al. For more details, check their youtube channel, newly released book, free course).

This raises two important questions. 

1. Is PDIA for implementing a programme same as PDIA for enhancing capacity?
2. Is strong state a prerequisite for PDIA?

Gulzar argues that we need strong states to do PDIA of programmes, making capacity a prerequisite for PDIA of programmes. I am going to argue that 

i) PDIA for implementing a programme and PDIA for enhancing capacity can be different but if done appropriately, PDIA of a programme and PDIA for capacity can go together. 

ii) strong state needn't be a prerequisite for PDIA, instead it can be an outcome of PDIA. In other words, PDIA can be used to build a strong state.

The rest of the post is structured as follows:

I. Unpacking the elements of state capacity in the context of education
II. Understanding capacity challenges using the framework discussed in I
III. Why do scale-ups fail in education? Analysing using framework discussed in I
IV. How to build capacity using PDIA? 
V. Conclusion

I. Unpacking the elements of state capacity in the context of education

The term state capacity is a black box. Sometimes it hides more than what it reveals. However, it's difficult to quantify and compare. Researchers have used several proxies like adherence to international postal convention, accuracy of census data etc. Francis Fukuyama in his paper "What is governance?" discusses the challenges in disaggregating state capacity, and calls for a context specific approach.

It is thus crucial to unpack the black box of state capacity in the context of education. I had earlier posted an updated version of one such attempt made in my book.  For the purpose of simplicity, I am presenting the summary here.

State capacity in the context of education can be visualised as comprising the following elements. It presumes that evident first order challenges like lack of enough functionaries etc. are addressed.

1. Nature of personnel: It's about the intrinsic motivation and honesty of functionaries at lower level. Fukuyama calls it bureaucratic capacity.

2. Skill of personnel: Self explanatory

Nature of personnel and skill of personnel can be together be termed as "local capacity". It's essentially a proxy of - given a task, how well they can perform without monitoring, which is a function of nature of personnel and their skill.

3. Autonomy: What's functions are devolved to lower levels of bureaucracy or systems?

4. Nature of task: Does the task require discretion to be performed effectively? Lant Pritchett calls these as engagement intensive tasks (education and health care).

5. Monitoring norms of lower bureaucracy: Are they rule-bound or norm-bound?

6. Implementation design of public policies: While the lower bureaucracy has its own distinct norms and rules for functioning, implementation design also has its own design, with reporting structures etc. inbuilt into it.

7. Capacity of higher bureaucracy to design monitoring norms and implementation design of public policies: How well can policy designers formulate implementation design conducive to the particular context of lower bureaucracy?

8. Environment or communities in which the policy is to be implemented: While all the above 7 factors are internal to the bureaucracy, this factor is external to the bureaucracy. It's the environment in which policy is implemented. A bureaucracy that can implement a policy well in a context may not be able to do so in another context.  We can understand it using Joe Midgal's term "resistance to penetration of state".

II. Understanding capacity challenges using the framework discussed in I

The state capacity challenges in education can then be framed as follows, using the above framework.

1. Autonomy vs. Local capacity: Local capacity is a function of nature of personnel and their skill. This challenge is "what functions should be devolved for a given capacity?"

Giving less autonomy is sub-optimal while giving more autonomy can lead to negative effects (Hanushek et al, Leer, Bloom et al). Fukuyama calls it the inverted U relation between autonomy and capacity, suggesting that there is an optimal autonomy for a given local capacity.

2.  Interaction of "Local capacity - Nature of task - Monitoring norms": Generally the problem of low 'local capacity' is addressed by pursuing a strict rule based monitoring. It works for tasks that don't require discretion to perform and hence are amenable to rule based monitoring?

What should be done in case of tasks that aren't amenable to rule based monitoring but the low capacity mandates that (rule based monitoring)?

Education is an engagement intensive task that requires discretion, which by its nature isn't amenable to rule based monitoring. But, low capacity systems that can misuse it lead to rule based monitoring mechanisms.

This is a recurring phenomenon. Citing the constraints of low local capacity or with the presumption of such, engagement intensive tasks that are to be monitored using norms instead of rigid rules are somehow converted into mechanical tasks, that can be monitored using rigid rules. This mindset is particularly striking in higher levels of bureaucracy, for whom managing is equivalent to tightening the screws over people, forcing them to work. It works in case of rule-bound tasks but such approach isn't suitable to education by its very nature that works on presumptive trust.

A different version of the practice of turning engagement tasks to rule-bound tasks is reflected in our approach to teacher training, where teacher training is reduced to an exercise of 2-3 day lectures and adherence to record keeping instead of being a continuous supporting mechanism, what Atul Gawande calls, coaching

Three important studies that document the importance of bureaucratic practices are worth mentioning in this context.

i) Documenting the time-use of Cluster Resource Coordinators (CRCs) from Bihar, Yamini Aiyar points out that ours is a post-office state, where a strict rule based monitoring is followed and that the functionaries (CRCs in this case) feel powerless and believe that their job is only to do what's asked and not take initiative. Note that this is in context where we expect the personnel to use discretion and initiative.

ii) Kiran Bhatty and Radhika Saraf document the bureaucratic practices in education in detail. Confirming Yamini Aiyar's insight on data collection tasks (amenable to rule based monitoring) replacing mentoring tasks (not easily amenable to rule based monitoring) , Bhatty et al estimate that as many as 480 formats had reached the Block for just the 48 schools in their sample.

iii) Akshay Mangla documents bureaucratic practices in Himachal Pradesh and Uttarakhand. He argues that the higher outcomes in Himachal Pradesh as compared to Uttarakhand despite their similarity, has its roots in the bureaucratic practices in these two states.

3. Variance in nature of tasks performed by personnel: How to monitor a personnel who does tasks of two different nature - one task that is amenable to rule based monitoring and the other that isn't and requires presumptive trust.

While a rigid rule based monitoring leads to an environment of lack of trust, as Pratap Bhanu Mehta argues, norm based monitoring signifies 'trust'. Managing these seemingly contradictory aspects together is a challenge.

4. Variance in capacity of local systems: Nature of personnel, skill etc. vary highly across the system. Fairness metrics mandate same treatment to all local systems within the total system. For example, one can't give autonomy to some schools while restricting it to some. 

Such constraints lead to risk averse sub optimal approaches, where a risk averse approach, rule based monitoring is followed. In words of Lant Pritchett, it inhibits 'positive deviance'.

5. Alignment between rules mandated by implementation design of public policy and inherent norms of lower bureaucracy: If lower bureaucracy works on norm based monitoring, a public policy that mandates rule based monitoring in implementing that policy can disturb such structures.

For instance, UP government recently declared that they will hold teachers accountable for completing syllabus. In that context, if one implements a programme that focuses on outcomes, there's good probability that it won't succeed.

6. Alignment between expectations of monitoring authority and communities - vertical accountability vs. horizontal accountability: Parents may use child's discipline as metric to judge teachers, higher bureaucracy uses completion of syllabus as a metric, and activists and commentators use 'learning outcomes' as a metric! 

III. Why do scale-ups fail in education? Analysing using framework discussed in I

Governments' scale up of pedagogical interventions fail because they gloss over the 6 capacity challenges discussed above. They parachute pedagogy into classrooms, thus imposing it over a low capacity systems with non-conducive bureaucratic practices. For example, such interventions don't address the local capacity challenges (motivation and skill of teachers), don't try to change the culture non-conducive bureaucratic practices in lower bureaucracy and so on.

The famed Activity Based Learning (ABL) is a good example. This pedagogy, inspired from Rishi Valley School of Andhra Pradesh is scaled up over 7 states with UNICEF's assistance. Its evaluation study documents important insights on the reasons for its lack of success.

One of the important insights from the report is that academic support to the intervention was reduced to checking adherence to report making and not helping teachers with their classroom issues by providing them feedback.  It ends up happening in almost all scale ups because we are imposing pedagogy on a lower bureaucracy accustomed to being a post-office state. The usual challenges of funds and textbooks not received on time are additional. Addressing such challenges is not part of typical scale-ups, instead scale-ups gloss over them. 

In the end, interventions fail but the failure is attributed to pedagogy and not to the flawed approach. The parachuting of pedagogies keeps continuing, with a new pedagogy next time, by a new bureaucrat. Meanwhile, it reinforces the 'cogs in a wheel' attitude in teachers killing their initiative. 

IV. How to build capacity using PDIA?

Repeating the two points made in the beginning of the post.

i) PDIA for implementing a programme and PDIA for enhancing capacity can be different but if done appropriately, PDIA of a programme and PDIA for capacity can go together. 

ii) strong state needn't be a prerequisite for PDIA, instead it can be an outcome of PDIA. In other words, PDIA can be used to build a strong state.

On the difference between PDIA for programme and PDIA for capacity: while enhancing capacity requires addressing the 6 challenges above, a programme can be implemented without addressing those by resorting to temporary shortcuts for the challenges. For instance, the delay in fund release through usual channels is addressed by setting up a special vehicle, the challenges of monitoring norms are addressed by dismantling the intermediate structures and making everyone report to collector, as done in Mission Gunwatta and so on. These may help implement this particular programme but doesn't help in long term.

A metric to differentiate PDIA for programme and PDIA for capacity is - if one is to implement a similar programme next time, how easy is it compared to the first time? If one has addressed capacity challenges using PDIA, then implementing a new programme next time will be less difficult.

The questions now are i) how to enhance capacity using PDIA? ii) how to integrate PDIA to enhance capacity with PDIA to implement a programme?

The intuition behind an approach that addresses the above two questions can be understood from the following analogy. Consider a person with dysfunctional digestive system, equivalent to a system with weak state capacity. This person isn't able to digest food, which is equivalent to not being able to implement a programme. We can enhance the functioning of digestive system using food as a diagnostic tool.

Let me explain. You start with the easiest food to digest, let's say glucose or saline water. Once you administer the food, you check for enzymes that are to be secreted to digest glucose. If something is missing or not adequate, you fix that. This helps both correct the system, enhancing its capacity, and also digest the food (enhances capacity and helps implement programme). Next time, you take a slightly complex food and do the same. You gradually increase the complexity of food taking it to higher levels, using non vegetarian food that requires more capacity to digest. At each step, you are checking for the corresponding deficiencies and correcting them.

To repeat, we are using food (programme) as a diagnostic tool and at each stage we are correcting the deficiencies, thereby both enhancing the capacity and helping administer the programme.

The principles for a PDIA that enhances capacity and helps implement a programme are clear from the above analogy.

1.  Capacity isn't of one particular kind. There are different types of capacities and each has to be dealt accordingly.

2. Start with the least complex programme that doesn't overwhelm the initial condition of weak systems and gradually increase it with the increase in capacity. Complex programmes that require high capacity may backfire if we begin with them.

3. At each step, focus on easing implementation by going to the root cause of constraints and addressing them, instead of going for short-cuts. The measure of progress in this approach is - how easy is it if we are to implement a similar programme next time.

These principles can be applied in education in the following manner.

1. Think in terms of capabilities (capacities) to be developed and not just tasks to be completed: and Make a list of all constraints known and categorize according to the types of capacity required. For example, if the constraints are increasing parent engagement in schools, improving academic support systems to teachers, speeding up fund flows, the corresponding capacities are capacity of bureaucracy to engage with community, capacity to administer engagement intensive tasks, capacity to address the process challenges.

Many others may emerge in the process of PDIA but it's good to have a list to begin with. It can be appended as we learn new things.

2. Start with simple interventions that require less capacity and gradually increase: Start with the least complex pedagogy, that we know works. Fortunately, we have Pratham's tested CaMAL pedagogy, commonly called as Teaching at the Right Level (TaRL). It's simple to administer and addresses only the basics of learning outcomes.

3. At each step in its implementation, address the constraints as they surface

a) If one finds that issues like lack of transparency in teacher transfers is hampering teacher's attention and motivation, address that. 

b) If one finds that teachers aren't taking this task seriously because they are being judged on metric of completing syllabus and not enhancing outcomes through this programme, address it. Make it clear that teachers won't be judged by metric of completing syllabus and that they will be judged by improving outcomes using this method.

c) If one finds that there's delay in textbook disbursal, address that.

d) Similarly, by making the support personnel administer this particular programme, the bureaucratic practices of lower bureaucracy can be shifted from being rule bound to norm bound and personnel can be made accustomed to that.

e) Build the capacity of higher bureaucracy in the process

.... and so on.

Through this process we are both ensuring the implementation of the TaRL programme and are also enhancing the 'capacity' of the system by both easing the constraints and also shaping the norms and practices of lower bureaucracy. In next step, use a complementary pedagogy to teach higher level stuff and continue this process.

We note that capacity isn't prerequisite for PDIA here but a 'strong will' to continuously sustain the process and withstand various various political economy pressures that arise in the process. Such strong will is necessary both at political level and at levels of top bureaucracy. Also, in a resource constrained context, one might also have to commit to extra expenditure. We must also note that will in itself is not sufficient, an improper approach can squander the will.

Significance of the above approach

The significance of above approach is that it's 'comprehensive'. It sounds simple but it's not. Often, our imperfect mental models about the world make us believe that some specific things (textbooks, classrooms, changing norms etc.) are the binding constraints. But they end up not showing transformatory results. RCTs come and tell us that no such interventions work. The point here is not that these elements are unnecessary but it's that we have ignored all the other complementary elements.

The above approach forces us to think comprehensively about all elements because the process of implementing a programme throws up many other unknown constraints and the goal of implementing this programme forces us to address it.

V. Conclusion

Weak state capacity is the binding constraint in Indian public education. The weak state capacity is not just in terms of visible aspects like teacher absence etc. It also lies in the bureaucratic processes that convert engagement intensive tasks into rule-bound tasks thereby making them non conducive to activities like education.

Scale-ups in education don't work because they parachute pedagogy into classrooms glossing over the different types of capacity challenges. This can be addressed using PDIA approach that helps both implement a programme and enhance the capacity. Such approach addresses all elements comprehensively, as opposed to the current piece meal approaches arising out of our imperfect mental models about binding constraints. 

Strong will, both political and at top bureaucracy levels is a prerequisite for this process and if one doesn't follow appropriate reform approach, such will can be squandered.


Follow on Facebook: @iterativeadapt
Follow on Twitter: @iterate_adapt
Email subscription or RSS Feed: Enter id in the "Subscribe" text box, on the top right of the blog.

No comments:

Post a Comment