Criticisms of learning assessments

In the previous post, this blog discussed about the need for assessments and five aspects of assessments that need attention. This post discusses the various criticisms of assessments.

I. Assessments lead to perverse incentives and narrow the focus of education

Arguments: 

1) Students' performance is often dependent on external conditions like the support systems at home, economic conditions and so on, which aren't totally in control of teachers. Hence, holding teachers alone accountable for the results isn't justified.

2) When the results of assessments are linked with teachers' pay and are used to promote or terminate teachers' employment, it leads to perverse incentives.
  • Teachers try to game the system to reach the targets, instead of actually teaching.
  • Teachers narrow the focus of learning, teaching only the concepts that are asked in exam.
3) Attaching high stakes to the results of assessments leads to mental stress among students. This narrows the focus of children, to study only aspects related to exam, which isn't good for overall development of children.

Counter arguments:

1) There are different types of assessments. The above problems are only in cases where there are stakes associated with the results of assessments. This doesn't mean the other forms of assessments, especially diagnostic assessments with low stakes that are meant to understand students' difficulties are bad.

2) A system which can't hold anyone accountable for the flaws is ripe for downfall. It is well established that monitoring based on inputs isn't sensible. The absence of accountability and monitoring based on outcomes will only lend free hand to people and only amplifies the problems of poor attendance and efforts on part of stakeholders.

3) There are problems with every incentive, not just in education. Accountability and incentives aren't bad per-se but the negative effects depend on the structure and design of the incentives. So, instead of removing them in totality, the focus should be on designing them better. There are rigorous studies which showed that certain form of incentives to teachers do improve students' learning outcomes.

4) The effect of incentives are also dependent on the context of the system. If it is a poorly performing system where teachers aren't even attending classes, then having some form of incentives and penalties brings seriousness to the job and they will at least attend the school.

II. Inappropriate assessments

Arguments: 

1) Assessments don't test the real knowledge and understanding of students. They are rote based and only encourage rote based learning.

2) The context of questions in assessments aren't equally relevant to all contexts. For example, consider this question - "A pizza is cut into 8 equal parts and is distributed equally among 4 people. What part of the original pizza does each person get?". This might be appropriate for US context but students in an Indian village might have never heard of or seen a pizza. Hence they can't make sense of it and are at disadvantage in such assessments.

Counter arguments:

1) If questions aren't proper, then framing better questions is the solution but not stopping the assessments.

2) If the context isn't appropriate then having context appropriate questions is the solution but this shouldn't be an argument to stop assessments in totality.

III. Assessments aren't comprehensive

Argument: Results of an assessment where academic knowledge is assessed is not a true picture of children's capability.  A child's capability is much more than performance on the concepts taught in a classroom. The learning assessments don't capture the other aspects of children like critical thinking, emotional quotient, grit, perseverance, communication skills which are also crucial for an overall development of the child.

Counter arguments: 

1) The learning assessments are one among the various aspects that are necessary. Tools to measure other aspects aren't evolved yet. Hence, assessments on academic learning are the best possible approximation. The fact that other things aren't being measured shouldn't mean that academic outcomes shouldn't be measured. 

2) The learning assessments aren't mere test of knowledge of children. They also reflect the other abilities of child like critical thinking, perseverance (required for understanding difficult concepts and preparing for exam) and so on. A well framed question on geometry isn't merely a test of knowledge of geometrical concepts but also tests students' students' ability to analyze, logically deduce and make conclusions.

3) Themes like critical thinking, inference are abstract concepts. At the end of the day, one needs tools in order to teach them and assess them. The concepts taught in classrooms are tools through which one can help build these skills in students.

IV. Diagnostic assessments is only a temporary solution

Argument: Diagnostic assessments, similar to the questions on decimals discussed in the previous post which aim to identify students' difficulties are only a temporary temporary solution. The core problem is that these misconceptions arise because either teachers don't teach these properly or students don't have think critically enough before forming conceptions.

Today, one may identify misconceptions in each concept and patch them by giving ready made answers and telling them that it's wrong. Who will help them later or in fact in other aspects of life, where they form conceptions without thinking critically? How will you prevent them from making incorrect inferences and forming wrong conceptions?

Excessive focus on the diagnostics assessments and making them the prime objective is turning the focus away from addressing these core problems.

V. Students will learn with time, diagnostic assessments aren't important

Argument: Even if a student has incorrect conceptions about about addition or length today, they will learn them as they as they grow up. Hence excessive focus on these is unnecessary.

Counter arguments for IV and V: 

1) The argument that there won't be any misconceptions if students are taught using a perfect methodology (if there exists one such) assumes that students' mind is a blank slate and providing correct information in a perfect manner leads to correct conceptions. This isn't often true. Each student has different experiences and hence have different prejudices and conceptions about certain things. Even if same accurate information in a perfect manner is provided to children, each of them may interpret it differently based on their existing prejudices and past experiences. Hence, it is important to understand difficulties of each child, which are unique to him/her and treat accordingly.

Diagnosing and remediating is also a part of teaching, not something different. Appropriate teaching and remediation through diagnostic assessments aren't mutually exclusive. We need both.

2) The levels of motivation and support systems at home differ among students. Those with high motivation or good support systems at home can sustain even if they are wrong sometimes or don't understand the concepts. But, for students with low motivation or weak support systems at home, repeatedly getting marked incorrect demotivates them further and if this persists, it can lead to drop outs or permanent disinterest towards education. Hence, it is necessary to diagnose the difficulties of children and address them immediately so that they don't do mistakes, even if they are silly.

3) Some concepts may get auto corrected with age but some needn't be and this comes at a cost.
  • A student may understand the concept of 3rd grade in 6th grade but meanwhile, he/she may lose out all the concepts taught in 4th, 5th and 6th grade which are based on the concept taught in 3rd grade. This increases the learning gap. In case of  students with low motivation and weak support systems, this gap is harmful. Once a child falls back, it is difficult to cope up in environments with weak support systems and low motivations and it may lead to dropouts and disinterest towards education.
  • Not knowing some concepts may not be be harmful but misunderstanding some essential concepts, which are used in daily life can be harmful. For example, this famous survey in US about fractions show that even many adults don't have proper conceptions and hence respond differently to offers on discounts.

VI. No assessments at all

Argument: Assessments harm the child. Hence shouldn't be done.

Counter argument: The above argument puts all forms of assessments in one category. For example, diagnostic assessments are to understand child's difficulties. Without knowing what the child knows and doesn't know, one can't effectively teach children. The above argument also assumes assessment is different from teaching. Assessment is also a part of teaching process.

VII. Assessments alone don't lead to outcomes

Argument: Weighing a malnourished repeatedly won't increase her/his weight. Similarly, merely doing learning assessments won't help children learn.

Counter argument: Assessing children is only a necessary condition but not sufficient. There are many factors that are involved in helping a child learn and none of this alone is a necessary and sufficient condition. Learning assessments alone may not lead to outcomes but they are necessary in this process.

Overall, as usual the truth is somewhere in between. 
  • All assessments aren't of same category.
  • Assessments are necessary but not sufficient.
  • The design of assessments is important and in some aspects need improvement.
  • The results of assessments have to be used cautiously while attaching stakes to them.
The baby shouldn't be thrown out with bath water.

No comments:

Post a Comment