General Reading Tests Are Always Unfair

roger-moore

Should pupils be taught who this man is? Maybe . . .

General reading tests are always unfair because there is no such thing as general reading skill. Every text will be much clearer to those who have relevant background knowledge. Whatever text you choose, you will favour those pupils who know something about its topic beforehand. The test will therefore not be fair, because it will privilege those who just happen to know something about that subject.

The idea that you can create a general reading test which will fairly assess reading ability across the school population is based upon a false notion of what reading entails. Once you have mastered decoding, the most important factor is knowledge. You could be brilliant at reading texts about football, but hopeless at reading texts about politics.

E D Hirsch describes how several recent studies have conclusively shown that background knowledge is the number one factor in reading ability, trumping academic ability and IQ in pupils, and complexity in texts. In 1998 the Recht-Leslie study showed that ‘poor’ readers could read well when they were familiar with the topic, which was baseball. They ‘illustrated the general principle: when a topic is familiar, “poor” readers become “good” readers; moreover, when a topic is unfamiliar, normally better readers lose their advantage’ (Why Knowledge Matters, p88). In 1999 Schneider et al proved the same point, but with IQ, while in 2011 Arya et al showed that text complexity was relatively unimportant compared to domain knowledge (see Hirsch, p88-89). These three studies show that if you know about something, you will be a good reader of texts concerning that topic.

Now consider what happens in the English language GCSE, as currently constituted. It is entirely ‘unseen’, so specific preparation of knowledge is impossible. The reading texts could be about anything. Let’s say the examination board selects a text about spies. Some pupils will know a lot about spies. Some may even be spy geeks. Others will have no interest in the topic, and will not have developed much domain knowledge. The English teacher may have spent one or two lessons doing some reading about espionage and discussing the topic, or they may not. We can hardly blame them if they haven’t, as they had no idea that this topic, among the thousands of possibilities, would come up in the examination. The spy geeks will ace that test, while others’ performance will be based mostly on the random criterion of how much they happen to know about spies.

The pupils who do well will be congratulated on their performance, but to a large extent, they just got lucky. The English teachers may even be congratulated if they happened to be teaching a large number of spy geeks. They may even get a pay rise or promotion based on this coincidence.

This is completely unjust, both to pupils and to teachers. General examinations given to the whole population should always be based on specified content. The more specific the content is, the fairer the examination will be, as it will enable all pupils to master that knowledge, and help all teachers to teach it effectively. As well as being fairer, only testing specific topics would promote education as the transmission of knowledge, rather than the development of mythical generic skills.

Because of the absence of specific content, preparation for general reading tests ends up being largely a process of tedious drills in how to serve up to the examiner what they are looking for. ‘Make sure you mention structure’, advise the English teachers, or ‘look out for similes and metaphors, and make sure you comment on them.’ How dull, compared to the richness and variety of content on which we could be focusing. But such a box-ticking approach seems inevitable when faced with general reading tests, because the knowledge required has not been specified in advance.

But we must not lose hope. We must make things better despite the unfair assessment system which currently exists, and we must work to persuade policy makers that fairness and the promotion of knowledge depend upon making tests as specific as possible.

How can schools best cope with the situation until such a time as assessment policy catches up with the findings of cognitive psychology?

The answer must lie in giving as much coherent knowledge as possible to pupils, especially in the earlier years, and in spending the minimum amount of time on tedious drills in examination skills. We can also make sure that any internal tests which we set are based on specific content which pupils are expected to master.

Instruction in how to give the examiner what they want should be left to the months immediately prior to the exam. The rest of schooling should be devoted to making pupils as knowledgeable as possible, because if we want our pupils to be creative and to think critically, if we want them to read well and write well, it is knowledge which counts.

Further reading:

Against Analysis, or Why William Doesn’t Engage the Reader

Comprehension Worksh**ts

Advertisements

5 thoughts on “General Reading Tests Are Always Unfair

  1. I don’t understand why this isn’t obvious to everyone and hasn’t changed.

    It was quite evident back in Y5 when my daughter achieved an exceptional score with the ‘Caves and Caving in Daveley Dale’ past SATS paper because she had significant experience of real dales and caves. Later in Y6 when queued up for the optional L6 she breezed through some homework based on an extract of The Woman in White because she had previously read every available Sherlock Holmes story (using me to explain the many new words and concepts) and then watched dozens of TV/film adaptions. By way of contrast I recall an evening of serious huffing and puffing over an extract of A Suitable Boy because she had very little knowledge and imagery of India.

    It’s Y9 now with GCSE options around the corner and she recently said she wished she could drop mandatory English. When asked why the response was a quite funny and well-reasoned polemic against too many lessons on (poisonously enunciated) ‘structure’ as opposed to content. I doubt there will be problems ticking the future exam passing boxes, but I’m resigned to ‘well-read’ depending on us nudging her towards what’s sitting on the bookshelves. Except that’s easier said than done now for a handful of classics where extracts have already been associated with tedium in lessons.

    Like

  2. Pingback: The Problems with Primary School Testing | The Traditional Teacher

  3. Pingback: English Language: The Vampire Subject | The Traditional Teacher

Thoughtful and reasonable discussion is always welcome.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s