The Problems with Grade Forecasting

Pig

You don’t fatten a pig by weighing it.

‘We know what we are, but not what we may be’- Shakespeare, Hamlet

It has become a truism in education that good professionals are able to predict the grades of their pupils accurately. Like most truisms, this needs further examination.

If a running coach spends their time making trainees run race after race, and times each one, he will be able to predict their performance in the final competition very accurately. But does that mean he is a good running coach?

No. Good running coaches do not spend their time making trainees run race after race. They break down the complex skill of successfully running races into its component parts, and assign their trainees exercises to build up those component parts. Instead of wasting time having them run races so that they can measure their performance, they have trainees build stamina and speed through a carefully designed set of exercises. If they want to know how their trainees are progressing, they look at how they are improving in those exercises, because they know that improvement in those exercises will ultimately lead to a better final performance.

Likewise, good teachers do not waste time making pupils do the final performance so that they can measure their attainment in it. Instead, they break down that final performance into its component parts, and focus on improving pupils’ abilities in those. If it’s an essay on Macbeth that pupils need to write, the component parts consist of knowledge of the play’s plot, characters and themes, knowledge of the literary and dramatic techniques used by Shakespeare to communicate these, and knowledge of the historical background which informs the play, such as James I’s Daemonologie and the struggles for power in medieval Scotland.

Instead of wasting time making pupils write essay after essay on Macbeth, the effective teacher will build gradually towards that final performance with lots of low stakes testing on core knowledge. These tests will look very little like the final performance. They could be multiple choice questions on Jacobean beliefs about witchcraft, or oral quick-fire questioning on the staging of the banquet scene in Act Three, or tightly focused analytical paragraphs on the imagery of damnation in one of Macbeth’s soliloquies, or a myriad of other methods of formative assessment that both measure and embed the knowledge and skills required for the final performance.

None of these formative assessment tasks will generate data that is useful for forecasting grades. But all of them will be integrated into a programme of teaching which leads to an excellent formal performance. Thanks to the benefits of retrieval practice, every little test will make the knowledge that much more secure.

The teacher who uses these methods will be focused on actually teaching, not generating data. They will be focused on pupils, not spreadsheets. And they may well forecast final grades less accurately than another teacher who puts their pupils through endless repetitions of the final performance, in the mistaken belief that this is the best way of preparing for it.

The obsession with accurate grade forecasts is anti-educational. If managers think it allows them to target their interventions more accurately, they are wrong. The kind of assessment that makes forecasts more accurate is not the kind of assessment that builds a precise picture of where knowledge gaps are, so that teaching can be more finely tuned. A whole essay is too complex to allow teachers to gauge exactly where they need to reteach key knowledge.

So let’s drop this obsession and focus instead on making the grades as good as they can possibly be. Otherwise we’re just generating self-fulfilling prophecies and preening ourselves on our professional skill, while in reality we are limiting the attainment of our pupils.

Advertisements

Music for All

One of the most memorable things about my first week working for the Inspiration Trust was when John Stephens, the recently appointed Director of Music, had the whole staff singing on Welcome Back Day. Not only did he have us singing, he had us singing different parts, each representing a section of the orchestra, and harmonising with each other.

Getting a large group of people singing in harmony, as John so impressively did, is a wonderful example of what can be achieved with simple, direct, traditional methods. He displayed the words and the notes on the projector, but I was behind a column, so I could barely see them. I had to rely on my ear and the example of those around me, but I could still join in. I could still learn. I listened and imitated, and I listened and imitated, and I got the hang of it, along with most of the other eight hundred or so people assembled in St Andrew’s Hall.

Listening and imitating is at the heart of education, traditionally understood. I would not have got very far in this if I had not accepted John’s authority, and been happy to join in with the crowd, which he had turned into something much more than a crowd by the exercise of that authority.

Music clarifies so many things about education. If we want to produce music, we must submit to authority – the authority of the notes themselves, mediated to us through the skill of the conductor. We must practise repeatedly until we get it right. We must build up our skills incrementally, not shirking the need to revisit the parts which we have not yet fully mastered.

And what we can produce if we accept authority and the need for hard work is something which brings people together under a civilising influence. It is something which sits above both pupil and teacher – it is the inheritance of human culture, which should be available to everyone. I’ll always remember a comment from a pupil in my very earliest days of working in schools, when I asked her what she thought of classical music. She said it was ‘music for posh people’. That is an attitude which we are going to demolish at the Inspiration Trust.

Striving towards something difficult but worthwhile, with a good will – we learn about these things when we take part in musical performance, and these lessons apply across education, not because of any complicated bit of neuroscience, but simply because we need to learn the necessity of hard work and practice in all academic and creative work.

This is what learning to learn is, if it’s anything: learning the habit of working hard, of practising, of persevering. And with the drive to get more and more pupils involved in music across the Trust, I am sure we will see the benefits of these habits well beyond choir and orchestra rehearsals and performances.

Creating a New Professional Paradigm

The professionalisation of teaching has gone hand in hand, historically, with the promotion of progressive ideas. One of the first campaigners was the American Horace Mann in the nineteenth century, and he succeeded in securing better pay and conditions for teachers, but at the expense of the simplicity of teaching. He introduced pedagogical notions which were beyond the ken of ordinary folk, thus developing a mystical aura around the teaching profession which would justify increased salaries and more job security for public educators, as well as the taxes which would pay for them.

This process went into overdrive in the twentieth century, as the professors of Columbia Teachers College carved out intellectual ground based on the exaltation of pedagogy in ever new and different forms, at the expense of subject knowledge. The politics of teacher professionalisation are thus deeply troubling from their inception. These early professors of education knew that they had to create their own territory, otherwise they would be seen as simply adjuncts to the subject-based university departments already in existence. This they did by inventing all kinds of new and complicated methods of teaching in which they could claim expertise, and derogating the transmission of ‘mere facts’.

In this view, teachers entering the profession are entering a sort of gnostic priesthood, and the sacred mysteries into which they are initiated are all of the mumbo-jumbo that comes under the title of pedagogy. While this intellectual trickery persists, we will not make much progress in placing knowledge at the centre of the curriculum and at the centre of our profession.

As a traditional teacher, I want to reclaim the essential simplicity of education as what G K Chesterton called ‘truth in the state of transmission’. But this does not mean the destruction of teaching as a profession. It means creating a new professional paradigm, which is primarily based around subject rather than pedagogical knowledge. It means teachers who spend as much time as they can developing their knowledge of the subjects they teach, rather than filling spreadsheets with pointless data or writing endless comments in five different colour pens.

It can be rather daunting to start thinking of yourself as a subject expert, rather than a pedagogical expert. But I implore my professional colleagues to see the beauty of this: wouldn’t you rather spend your time learning more and more about really interesting stuff, so that you could explain it better to your pupils? Has the model of pedagogical expertise really brought you, or your pupils, any joy?

Not All Reading Is Equal

We often hear about how important it is for children to develop a love of reading. But like so many statements about education, a vital ingredient is missing from this apparently laudable aspiration. It is comparable to bland proclamations that children should learn to be creative. What is missing is specific content.

Creating what? Reading what?

There are many things which I would not want my pupils to read. Reading is not an end in itself, but a means to an end. And the end is knowledge.

There are many kinds of knowledge which reading can bring, which are difficult to access in other ways. Great literature brings knowledge about the fundamental questions of human existence. Historical writing, scientific writing, philosophical writing and even journalism can carry new insights into the nature of the universe and humanity’s place within it. A fluent reader can gain access to thought that will open her mind and raise her aspirations. But she can also gain access to fake news and poisonous propaganda. Reading can poison the mind as well as nourishing it.

There is nothing intrinsically good about reading. It depends what you are reading, and it depends on how accurately you can interpret it.

Of course we want all our pupils to be able to decode fluently. But beyond this, we must think about how we nourish their thinking so that they can use this wonderful opportunity to expand their minds, not to poison or pollute them. A regular diet of knowledge will give them a taste for reality, so that they are increasingly able to discriminate between different authors, and reject what is false.

Liberated by Cognitive Science

Understanding the essentials of cognitive architecture is a wonderfully liberating thing. So many of our misconceptions about education, our own and our pupils’, can be laid to rest.

The first, and most important principle is the understanding of what education actually is: changes in long term memory. This simple definition can and should be refined, but it’s an essential starting point. If you haven’t remembered it, you haven’t learned it. You were wasting your time, as teacher, as a pupil, if the knowledge which you were supposed to acquire has not at least begun to make its way into your long term memory.

This affects the questions we ask about teaching. Instead of chasing endless proxies for learning, we begin to focus on the thing itself. Instead of asking ‘were the pupils engaged?’ or ‘did they enjoy the lesson?’ or ‘did they write lots in their books?’ we can simply ask ‘what did they remember?’ In the end, that’s all that matters.

But that question logically leads to others. If we are interested in long term memory, we have to be interested in the long term. We are therefore liberated from the obsession with individual lessons as units of learning. We must ask ‘what did they remember the next week, the next month, the next year?’ Now we are able to see education as a coherent long term programme for building mental schemas in the minds of pupils, not a series of isolated fragments which could be judged as ‘outstanding’ or ‘satisfactory’. Thus do the principles of cognitive architecture liberate us from the madness of graded lesson observations.

These are just a few of the ways in which understanding cognitive architecture liberates us professionally. But there is a wonderful personal liberation too. So often I have complained over the years about how poor my memory is. Recently, when I was explaining to a colleague how much explicit memorisation I require of pupils, he said, ‘My memory’s rubbish. I suppose you have trained yours to be able to do this?’ From his tone, it sounded like he thought such ‘training’ would require Herculean efforts, beyond the capacities of most ordinary mortals.

But when we understand that we all have extremely limited working memory, but practically limitless long term memory, and that repeated practice and testing can ensure that anything can be securely stored, then we know that we do not have a rubbish memory. We have a human memory, and we face the same limitations and have the same amazing potential as the rest of the human race. The obstacles we face to filling our memories with countless treasures of poetry, history and science are obstacles that we, and our pupils, can overcome with persistent, faithful, steady effort.

It is also a wonderful liberation to realise that we have not, in fact, forgotten most of what we learned in school. Every well educated adult has a vast store of tacit knowledge which enables her to read a wide range of texts, as well as listen to other articulate, educated adults and take part in intelligent conversation with them. It is amazing to consider just how much we do, in fact, remember, but this also places a solemn duty upon us. When we realise how much we know, we appreciate how important it is that we should pass this on, so that succeeding generations can also take part in this conversation. And we’ll never succeed in doing this unless we talk to our pupils!

So with the excitement and liberation of discovering the reality of how the human mind works, come solemn duties: to pass on knowledge, and to fight against the lies which prevent us, or our colleagues, from doing so, such as ‘education is what’s left when you’ve forgotten everything you learned at school’.

These lies are so often glibly repeated in staff rooms or displayed on classroom walls, and they strike at the very root of what education is, and even what human beings are. To fill the mind with knowledge, and to treasure this knowledge and pass it on to succeeding generations, is to perpetuate human civilisation. To refuse to do so is to abdicate from one of the most fundamental human responsibilities.

The Michaela Inspection Result Is Good News for Everyone

Ofsted have a lot of blood on their hands. In the very recent past, they were a progressive inquisition, striking fear into the hearts of teachers who actually wanted to teach. But they have been working to dispel this image, publishing a myth-busting document and engaging on social media to answer questions and promote their claim that no particular teaching style is preferred by inspectors.

Many of us remain skeptical, including the folks at Michaela. I attended the very first Michaela day of debates, in which Katie Ashford argued passionately for the abolition of Ofsted. At that time, of course, the school had never received an inspection.

So this question has been hanging over the school, as they have courageously pursued methods and policies that are entirely focused on helping everyone acquire knowledge and develop self discipline. Would an Ofsted inspector be able to stomach this wholehearted, unapologetically traditional approach? However much the likes of Sean Harford claimed that the inspection body had changed, the Michaela inspection was going to be a litmus test.

Now we can celebrate with them, because they have been awarded ‘Outstanding’ in every area. This in no way proves that they are outstanding. Any of us who have experienced the games and tricks employed by senior managers to obtain that coveted classification will be convinced that there are many mediocre to poor schools which have been labelled this way. The proof that Michaela actually is an outstanding school is there before the eyes of the many people who have visited, as I did two years ago, and spoken to the knowledgeable, polite, happy and confident pupils with which Michaela is filled.

Michaela actually is outstanding. Whatever Ofsted say, that is the truth. But the fact that they have been graded thus is good news for all of us, because it is a fantastic argument which ordinary teachers can use with their senior leadership every time they are asked to do something time consuming which doesn’t actually promote learning.

So next time you’re asked to mark with three different colours, you can point to the Michaela Ofsted report and politely indicate that they don’t mark books at all. Next time you’re told to introduce more group work to promote ‘active learning’, you can point to the Michaela Ofsted report and calmly point out that their pupils make fantastic progress with whole class direct instruction.

Never again should we accept the non-argument that has been used by so many senior leaders across the country to promote so many anti-educational, time-consuming, morale-destroying practices: ‘Ofsted are looking for this’.

No, they’re not. And they’ve proved it.

Further reading:

Can Ofsted Be Reformed?

My Experience of Ofsted Madness

Direct Instruction Transforms Behaviour

We must be very clear that the choices made by pupils are their own responsibility. If they decide to be rude or defiant, they have made that choice, and they must take the consequences. Few things make my blood boil more than hearing senior leaders blaming classroom teachers for pupil behaviour.

But at the same time, we must acknowledge that the methods used by teachers will influence the behaviour of pupils. When teachers spend their time trying to entice pupils to learn something through an endless variety of activities, the implicit message pupils receive is that they are consumers of an education product. And the customer is always right. They are at liberty to ignore the teacher if they don’t ‘buy’ what the teacher is ‘selling’.

So the endless and exhausting task of trying to persuade pupils that learning is fun will have a serious negative impact upon behaviour. On the other hand, when whole class instruction is used, with regular routines and the consistent expectation of full attention from all pupils all the time, classes that seemed to be impossible when they were faced with edutainment can become calm and ordered places. It doesn’t happen overnight, but with firm and persistent effort over a number of weeks, behaviour steadily improves.

Behaviour improves with direct instruction because all pupils know what is expected of them. A good course of direct instruction will include a large amount of repeated practice to ensure mastery. Not only does this make sense from a cognitive point of view, it creates calm and order, because the pupils are not only practising whatever element of the curriculum is being covered, they are also practising how to practise: how to focus the mind consistently on one clear area of study and repeat it until mastery is achieved. This kind of practice is methodical and reassuring, and satisfying in a quiet way. But no pupil could mistake it for entertainment, so they don’t respond as they would to entertainment: with boos, cheers or indifference.

Behaviour improves with direct instruction because when pupils are not practising, the lessons are directly led by the teacher, interacting with the whole class. The teacher stands at the front and expects every pupil to track her. She calls out key concepts and the whole class repeats them. She calls on individuals and they repeat the concepts, word for word; there is no ambiguity about what is expected of them. She goes through worked examples with the whole class, calling on individuals at key moments, without asking for hands up. Answering questions is compulsory, not voluntary. Everyone knows that if they are failing to pay attention, they will be spotted. No one is neglected. Everyone is included. Group work divides and excludes. Whole class interactive instruction is the most inclusive method possible: no one is left out, disaffected, labelled as useless, left behind, disenfranchised. No one has any of these common reasons to start misbehaving.

Behaviour improves with direct instruction because pupils are never asked to do things they cannot do. They are never asked questions to which they do not know the answer. The steady, incremental nature of a well designed programme of direct instruction means that pupils are never thrown in at the deep end. They gradually master each element of the curriculum, and the curriculum is coherently organised so that they are never required to run before they can walk. So often pupils begin to misbehave because they are baffled, so they give up and start mucking about instead.

If you want a calm, ordered classroom in which everyone can make progress, start using direct instruction. You’ll be amazed at how difficult pupils who ignored your every attempt to entertain them will quite contentedly work steadily on clear tasks with definite outcomes. They will gain the calm satisfaction of making progress, and happily leave behind the fraught and confusing role of consumer which had previously been forced upon them by misguided educational ideology.