Value added

The problem with bad ideas is that very often they appear, on first viewing, to be very good ideas indeed. This is why the habit of critical thinking is such an important one, worthy of a much more prominent place in the school curriculum. If we don’t develop the ability, and indeed instinct, to look beneath the glossy wrapper, then we’ll end up being sold any old nonsense. And, speaking of school, it’s quite possible that there is no field quite as vulnerable to the seductive power of glib fashion than education. Here’s a favourite example of mine. Consider the following statement:

It’s the job of a school to add value. There’s no point spending billions of dollars on education if that very process doesn’t add something valuable. Therefore, in order for us to know whether or not our education dollar is well spent, it’s critical that we can measure and report on the value we are adding. What’s more, knowing what the student can already do must be the starting point for any successful teaching programme, informing as it does what the next step should be.

Now, that appears deeply reasonable, doesn’t it? Almost to the point of being inarguable. Plenty of people at the Ministry of Education would say so, for it forms the basis of a significant push in our schools at the moment. Which is interesting, because the statement is almost certainly wrong. I should explain.

The first thing that must be noted, in the spirit of Heisenberg, is that measurement is never a neutral activity. The thing we measure is changed by the act of measuring it. Furthermore, measurement is an expensive activity, in terms of time and resources. A regime focussed on measuring will therefore do two things; it will divert resources away from other activities (say teaching) and it will change the way we think about the activities we teach and learn. The first is unambiguously negative, the second potentially so. Hence, we must conclude that a focus on identifying added value is a good idea if, and only if, the benefits exceed the costs. Yet, oddly, almost none of the enthusiasts for the concept see it in these terms.

So, let’s consider in more detail what some of the potential costs might be. Einstein (to continue the physics theme) once said: Not everything valuable is measurable, and not everything measurable is valuable. (Or was it the other way around?) So, if the cost of measurement is the first objection, then here is objection number two: A great deal of what is valuable in a school is intangible. Take for example the experience of heading out into the NZ bush to camp with a bunch of your peers. The most valuable takeaways from this experience will surely elude measurement (a moment of tranquility beneath the stars, a sense of belonging to a group, laughter around a campfire, the memory of which resonates through the years, and so it goes on). Now, imagine somebody tells you you must justify the camp in terms of value added. You can’t turn to the truly valuable aspects, as you know any attempt to measure them will be bogus, and so you may turn instead to something that could be measured, but is comparatively trivial: perhaps a temporary boost in fitness level, or an ability to correctly pitch a tent. The problem is immediately apparent. Justify the camp in these terms and two things happen. First, somebody will demand that more of the camp is therefore devoted to the intended outcome (let’s make the walk more grueling, let’s have tent inspections) and then, inevitably, someone will notice that the same outcomes can be more efficiently achieved in a less authentic environment. So we have tent pitching competitions on the school grounds followed by a brisk run around the block. And all of our lives are diminished.

Now for objection number three. In many cases, measurement inhibits learning. Two examples will make the case adequately. There is good evidence (a number of the studies in this area have used ethnic groups to make their point) that assessment increases performance inequities. In particular, the stress reaction in students who over time have become accustomed to failure inhibits performance. The more a struggling student expects the effort they are making to be assessed, the less effectively they engage with the task. The looming failure is enough to panic them into a non-receptive state. The second example comes from the area with which I am most familiar, that of creativity. I remember once being in a small group listening to the English Poet Laureate, Andrew Motion, explaining why he was against grading papers in the elite creative writing paper he is responsible for. His argument will be familiar with anyone who has ever been involved in a creative endeavour. Motion suggested that as soon as you have grade levels, you have to have descriptors, and as soon as the student focuses on the descriptors (just what is the difference between a good and excellent piece of writing?) their approach becomes hopelessly mechanical, and the creative impulse withers. This is exactly my experience as a writer. The way to write is to find the story that needs to be told, and then, over time, to find the head space that allows you to discover the way to tell it. It’s a messy, frustrating and unreliable business. And nothing knocks you from your path more than the sense that there is some criteria to be met. Writing to please, according to some pre-conceived set of shoulds, is creatively disastrous. As any artist will tell you, the hardest piece of work to create is the one immediately following on from your greatest success. Expectations, be they imposed form within or without, are the enemy of creativity. So, careful attempts to identify value added will kill creativity and add to inequality. On to objection number four.

Implicit in the push for more careful assessment of educational achievement is the belief that learning occurs in stages, with each level of achievement being dependent on first mastering the previous step. At any point then, measurement allows us to identify the stage the student is is, and so provide them with the specific knowledge or guidance required to move on to the next stage. Now, this is undoubtedly true for a small subset of learning. Think of the very many foundation skills that must be mastered in mathematics before one can deal confidently with calculus – first one must have a sense of number, then of basic arithmetic, later of relations, and later still, algebraic expressions. We can properly identify an ever finer-grained sequence of skills which must be negotiated in an approximate order, and finding out where he gaps are can be crucial in remedying difficulties.

However, for a great deal of the learning we do, this is not the process. In these cases, knowing where the student is currently at adds surprisingly little to our ability to help them improve. A couple of examples will clarify. Occasionally, as part of a drama programme, I teach students to juggle. Now, juggling has a clear progession, from total confusion, through to making uncontrolled sense of the pattern, trough to sporadic success, then low level mastery, and eventually something like fluency. And yet, at every stage, the advice is the same. Practise some more. One of the clearest indicators of a person’s juggling prowess is their capacity to juggle five balls. Master that and you can properly call yourself a juggler. It’s very tricky. Here is what you need to know, in order to master this complex skill:
The five ball cascade uses exactly the same pattern as the three ball cascade, only the throws are higher, and need to land more consistently.
That’s it. It’s all the knowledge you need. And what advice can a tutor give you to help you achieve it?
Practise this every day. For most people, after some months of consistent practice, the brain will begin to see the pattern.
Again, that’s the extent of it. And a huge amount of what we learn and indeed need to learn, is of this nature. Our knowledge becomes embedded over thousands of hours of repetitive practice. Knowing where in the journey you are at, at any given time, is essentially inert information.

Teaching acting has a similar quality. The advice I am most likely to give an actor (beyond the foundational stuff about voice and audience awareness) is to try to inhabit the character. Find a way of stopping being you, and being then. Fully imagine your way into their world and situation. Understand why they do and say the things they do, and how that makes them feel. The task is essentially the same for a first time student as it is for a professional actor with decades of experience. One piece of advice, to be learned over a life time. Sometime the actor struggles to find a character, other times they get there quickly. Our job is to help them explore the role, and to provide the encouragement and criticism needed to motivate them in the search. Eventually, they become a little more proficient. Because it is not a case of moving sequentially through skills, where each level has a new instructional need, knowing where the student currently stands is again of no instructional use. This objection might be best summarised thus: knowing how much value you were adding (even if it were possible) would do nothing to help you add value.

In drama students, it is not so much that they move through a sequence of ability levels, as they are caught within one of a number of mutually exclusive orbits (like electron clusters, shall we say?) The workmanlike actor, over a lifetime, becomes slowly more proficient at being workmanlike, in the same way that the true talent slowly, over a lifetime, learns to more reliably let that talent shine. One doesn’t move from proficient to talented, nor could one even compare two actors caught in different orbits. They’re just completely different things. Occasionally, there is a quantum fluctuation, a random and inexplicable leap for one orbit to another, linked I suspect to moments of confidence or clarity, and when the leap is upwards it’s a fine thing to witness, but as for leading a student through the levels, the metaphor just doesn’t hold.

Objection number five. We can never be sure the progress we observe is value we ourselves have added. This is an interesting factor that is often overlooked in educational research. Put simply, an awful lot of learning happens despite us. Just because a student at the end of the year can do things they couldn’t do at the beginning, that’s no indicator that we helped them do it. A child at some stage learns to walk, despite us never teaching them (and anybody who noticed the change and concluded that they, the parent, had added value, would be a judged a moron). But so it is with the increasing intellectual and social sophistication that comes from undergoing brain development in adolescence. Now, these skills of social discernment and complex reasoning, although developing offstage, are key drivers in the capacity of students to reach new academic heights as they age. We see the results, but can not sensibly conclude we added value. Some studies of reading development note that in middle class homes, reading age increases over the holidays at much the same rate as it did in school time, whereas in socially stressed households, we often see students going backwards over the break (this is said to be one of the biggest drivers of educational inequality). So, what should we say of the school’s contribution to the reading development of the middle class child? How much is down to the teacher? How could we ever know? Why would we want to?

And finally, to round out the half dozen, my sixth objection is something of an echo of the second. For the whole value-added edifice pre-supposes that the purpose of education is to learn. In fact, that’s only one of the purposes. We do want the student to emerge from the machine more capable than when they went in. But, and how easy it is to forget this point, we also want the student to experience. School is not just a preparation for life, but it is also a significant part of the life itself, and so the goal of providing experiences that are of themselves worthy contributions to the totality of a life well lived, is every bit as important as the goal of teaching them something. And attempts to measure value-added tend to miss this entirely.

So, could it be that an idea that sounds very smart from the outside, (measuring student progress to inform teaching practice) is actually of practically no value (and significant cost) in most educational contexts? Could it be that as processes go, this one is expensive, promotes inequality, devalues what is most important, fails to recognise how most learning occurs, stifles creativity and yields end data that is largely meaningless? If that is true, then serious questions need to be asked about the competence of those setting educational policy, because they appear to believe the opposite is true. This is not just a New Zealand trend, so in this sense perhaps we should not judge our own officials too harshly. The international nature of this particular folly suggests that beneath the surface the forces in play are largely political.

Meantime, the highlight of my teaching year next year will be directing the full school musical. Students will come together to sing, to dance, to act and entertain. They will support each other, face down their fears, experience moments of pure magic and walk away with memories that will never leave them. Do pity the first fool to approach me and suggest I consider measuring the value I have added during that process.

Advertisements

3 thoughts on “Value added

  1. JP says:

    Hi Bernard,

    Very interesting and well argued piece.

    Grading and evaluations can also have terrible effects at the university level. The following is anecdotal but I’m sure there are many cases like this.

    I have a friend who, after working for a number of years, decided to go back to school. She got admitted to a one-year university program (10 courses). She is a bright young woman with an independent and original mind, interested an amazing number of subjects. So, she began her studies, the courses were interesting, as were most teachers. She was enthusiastic about the whole thing, understood very well (I could tell by talking to her), made deep connections between the different subjects, read a lot of additional material. I was very impressed.

    But then came evaluation season… She simply couldn’t cope well with having to prepare what were to her arbitrary assignments written according to arbitrary rules, of having to learn uninteresting details just to pass an exam. And so on. She finally got through all this but, as a result, she’s dropping out after one session and won’t continue her studies. It seems the university is not a place you can go just to learn – what they sell is not knowledge but diplomas.

    Perhaps you know this quote by Chomsky: The whole educational and professional training system is a very elaborate filter, which just weeds out people who are too independent, and who think for themselves, and who don’t know how to be submissive, and so on — because they’re dysfunctional to the institutions.

    It seems this was written precisely for my friend.

    Jean-Paul

    • Hi Jean-Paul

      Yeah, I’m sure the tension between fostering a love of learning, and measuring and categorising, applies across the sector.

      I’d not come across the Chomsky quote, so thanks for that. It would look good my classroom wall, I think.

      Bernard

  2. Dan says:

    Excellent post, thanks. Particularly the point about the experience of school itself often being the greater part of education.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: