Liberation from Levels

800px-charlesdickens

This man wrote ‘effectively’. But was his writing ‘sophisticated’?

Since national curriculum levels were abandoned, schools have had to build their own systems for describing and assessing progress. Unfortunately, as with many of their new found freedoms, few schools are making the most of this opportunity to transform curriculum and assessment, and make them more specific, reliable and meaningful. Instead, they are producing schemes of work and assessment filled with recycled versions of the old national curriculum level descriptors.

It’s understandable that schools do not want to leave level descriptors behind. They were statutory for many years, and they are still used by examination boards to describe grades. The only problem is, they are nonsense, and educationally harmful nonsense, too.

For writing, they typically involve statements like ‘wide vocabulary’ or writing ‘effectively’, or writing with ‘sophistication’. What do these words actually mean? How ‘wide’ is ‘wide’? What does ‘sophisticated’ look like? Such statements do not provide an objective standard of measurement, because they are open to multiple interpretations.

A far better method of assessing writing is by ranking it. Systematised comparative judgement looks like an excellent way of harnessing the reliability of ranking. Teachers cannot agree consistently on a mark for any given piece of writing, but they can agree very consistently, when shown two pieces of writing, on which one is better.

Even in the absence of the more systematised comparative judgement, teachers can mark by ranking. When they have a set of essays or stories to mark, they can put them in a rank order, and if they have to assign a grade, they can do so according to that rank order.

An experienced marker is doing some kind of comparative judgement in any case. He will say ‘I know what an A looks like’. What this means, is that he has marked hundreds or even thousands of pieces of work and compared them mentally, placing A grade work at the top of the rank order. An experienced marker will never be thinking to himself, as he gazes at a mark scheme, ‘Now is this sophisticated, or is this effective?’ Even if he uses those words because the mark scheme tells him to, what he will really be thinking is, ‘Is this top, middle or bottom? How does this compare to the best writing I’ve seen?’ and similar questions.

As well as being hopeless for accurate assessment, level descriptors undermine efforts to build a knowledge curriculum. They tend to be generic. In the study of literature, they will contain statements like ‘Can analyse the author’s use of language’. If we’re measuring progress against vague, general statements like this, we will not be thinking in terms of content that must be mastered, but in terms of general skills that can supposedly be demonstrated with any content. The content becomes indifferent.

Level descriptors take us away from specific content, because they never say clear, specific things like ‘Knows in what century Chaucer lived’, or ‘Knows Chaucer’s attitude towards late medieval society’. In fact, it’s rare to see the word ‘knows’ in a level descriptor. They tend to be about what you can do, not what you know, and thus they perpetuate the myth of general skills divorced from specific domains of knowledge.

If we want to assess attainment meaningfully, and if we want to build a knowledge-rich curriculum focused on specific content, we need to consign level descriptors to educational oblivion. Once we’ve left them behind, we will be able to focus our attention on specific content that should be mastered, and measure attainment against exemplar work.

Specific content and exemplar work give teachers and pupils something clear and definite to aim for. Pupils will be relieved and pleased to be shown a definite way forward, instead of wandering in the vague territory of the difference between ‘effective’ and ‘sophisticated’. And teachers will be able to say to pupils, ‘I want you to know this’ and ‘I want you to be able to write like this’, instead of muddling around with grade boundaries and tick boxes.

Learning Cannot Be Inspected

Magnifying-glass-green-brassRecently, David Didau highlighted an inspection report which made prescriptive judgements about marking, even though Ofsted’s own policies say that they don’t. The report was withdrawn, but in subsequent discussion on Twitter, it was apparent that this kind of thing is rather common. It might be worth starting a sustained campaign to pull Ofsted reports to pieces online. They’re in the public domain, and it looks like it would be a fairly easy job.

But even if we demolished dozens of Ofsted reports for being illogical, inconsistent with their own guidance, overly prescriptive or just plain dumb, would it have much impact on the ground? There would probably be an increased vigilance about what inspectors set down on paper, but that still wouldn’t stop them from demolishing careers and schools based on flaky judgements. They would just have to give different reasons for doing so.

Everyone knows that if you go into a lesson looking for faults, you’ll find plenty of them. Teaching and learning are a messy business, and they just aren’t worth making snap judgements on. What really matters is long term learning: true mastery of the central knowledge and skills for each academic discipline. This is simply not something you can judge based on a brief inspection combined with a pile of dodgy data.

Thus, the only way to stop Ofsted from making misguided judgements of teaching and learning is by stopping them from making judgements of this area completely. They should only inspect the management of a school. Is it well organised? Are teachers supported (yes, supported, not bullied and manipulated)? Are its finances audited?

If we want to find out what pupils are learning, we just need to give them regular objective tests of their subject knowledge. Testing should be a normal part of teaching, not a big scary bogeyman who pops up every five years or so. There should be annual national tests using multiple choice questions, not vague subjective written assessment. The national curriculum should be firmly focused on building up core knowledge incrementally throughout every child’s years at school.

Then teachers should just be left alone to teach that core knowledge. I reckon most of them would embrace traditional methods pretty quickly if they were actually required to teach to mastery a clearly defined body of knowledge. In essence, traditional methods are simple and effective. Teacher explains. Pupils practice. Teacher notices widespread misconception and explains further. Pupils practice more. It’s what teachers did for centuries before they were blitzed by the recent proliferation of potty pedagogies.

For further fine tuning, we need resources designed by experts, such as Engelmann’s ‘Expressive Writing’. There would be a boom in the production of such resources if most of the country were focused on teaching this way, because they actually wanted kids to know and remember stuff.

Just think: you’d never see another inspector in your room again. In exchange, all you’ve got to do is teach knowledge thoroughly. And that’s what you came into teaching to do, isn’t it? Isn’t it?

(Image from Wikimedia).

Marking: Full Books and Empty Heads

Book burningThere was, as Squealer was never tired of explaining, endless work in the supervision and organisation of the farm. Much of this work was of a kind that the other animals were too ignorant to understand. For example, Squealer told them that the pigs had to expend enormous labours every day upon mysterious things called ‘files’, ‘reports’, ‘minutes’, and ‘memoranda’. These were large sheets of paper which had to be closely covered with writing, and as soon as they were so covered, they were burnt in the furnace. (George Orwell, Animal Farm).

Marking exercise books has become the bane of many teachers’ lives. A colleague saw me in the staff room with a pile of books recently, and commiserated. I replied that I do not find the pile depressing, because of the way that I mark.

I check that everything has been done. If it has been done poorly, it must be redone. If it has been done decently, it gets a tick. If it has been done well, it earns one or more merits. I also highlight spelling and punctuation mistakes. So this is what I actually write, beyond the ticks:

  • ‘Redo’;
  • ‘1M’, ‘2M’ etc;
  • ‘P’ (for punctuation), and the mistake corrected;
  • ‘Ask’ for anything more complex (such as style / vocabulary);
  • Spelling corrections, for pupils to write out five times.

That’s it. When pupils get the books back, I allow some class time for writing out mistakes or redoing work, but for those who have a lot of this to do, because they have been careless, it becomes extra homework to complete the corrections. Pupils only use the left hand page in their exercise books, so the right hand page is always free for them to write their corrections.

The only time I write longer comments is on more extended pieces of work, which are done in class. They are done in class so that I can provide guidance beforehand and control the conditions under which they are done. In this way, I know that I am actually commenting on the pupil’s work, and I can be reasonably sure it will be on task and of sufficient length and quality to merit more detailed marking. Pupils do these extended pieces four times a term.

Just as with the exercise books, when the pieces are returned, class time is devoted to acting on advice. With a piece of extended writing, for example, I do a simple error count and write it next to the grade. If there is a high error count (twenty or more) then the whole piece must be written out again with all mistakes corrected. If the error count is lower, pupils rewrite a paragraph, focusing on improving a specific aspect of their writing, such as vocabulary or sentence variety.

I’m not offering this as a perfect system, but it is a sane and sustainable one. I teach a full timetable. I do take marking on the train with me, but I do not spend evenings and weekends doing it. I have family duties to occupy me then, with five young children at home, and I could not do a job which left me no time with my little ones. I’m not surprised when I hear of teachers leaving the profession because of the unreasonable marking demands placed upon them. If I were in their situation, I would have to find another school, or leave teaching completely.

Aside from issues of sustainability, there are many points to be made about the effectiveness of spilling red, purple or pink ink all over pupils’ work.

There is no proof that elaborate marking of exercise books leads to improved learning. It drains teachers and prevents them from doing things which would definitely improve their effectiveness, such as long term planning and resource creation.

Elaborate marking moves the focus away from what is in the pupils’ heads (the knowledge they have mastered) to what is written on pieces of paper. The two are not equivalent. What we have in books is a performance, not learning. And if it is homework, it is likely that it is not even the pupil’s own performance.

Why are we wasting so much time writing elaborate comments on these performances? Instead, we should be focusing on fine tuning teaching methods that build mastery. Likewise, school leaders should be building mastery in their staff, not picking over marking performances produced more for the sake of management than for the sake of education.

Like Squealer’s bits of paper, those exercise books will end up being disposed of. But what our pupils store in long term memory will stay with them.

(Image from Wikimedia).