General reading tests are always unfair because there is no such thing as general reading skill. Every text will be much clearer to those who have relevant background knowledge. Whatever text you choose, you will favour those pupils who know something about its topic beforehand. The test will therefore not be fair, because it will privilege those who just happen to know something about that subject.
The idea that you can create a general reading test which will fairly assess reading ability across the school population is based upon a false notion of what reading entails. Once you have mastered decoding, the most important factor is knowledge. You could be brilliant at reading texts about football, but hopeless at reading texts about politics.
E D Hirsch describes how several recent studies have conclusively shown that background knowledge is the number one factor in reading ability, trumping academic ability and IQ in pupils, and complexity in texts. In 1998 the Recht-Leslie study showed that ‘poor’ readers could read well when they were familiar with the topic, which was baseball. They ‘illustrated the general principle: when a topic is familiar, “poor” readers become “good” readers; moreover, when a topic is unfamiliar, normally better readers lose their advantage’ (Why Knowledge Matters, p88). In 1999 Schneider et al proved the same point, but with IQ, while in 2011 Arya et al showed that text complexity was relatively unimportant compared to domain knowledge (see Hirsch, p88-89). These three studies show that if you know about something, you will be a good reader of texts concerning that topic.
Now consider what happens in the English language GCSE, as currently constituted. It is entirely ‘unseen’, so specific preparation of knowledge is impossible. The reading texts could be about anything. Let’s say the examination board selects a text about spies. Some pupils will know a lot about spies. Some may even be spy geeks. Others will have no interest in the topic, and will not have developed much domain knowledge. The English teacher may have spent one or two lessons doing some reading about espionage and discussing the topic, or they may not. We can hardly blame them if they haven’t, as they had no idea that this topic, among the thousands of possibilities, would come up in the examination. The spy geeks will ace that test, while others’ performance will be based mostly on the random criterion of how much they happen to know about spies.
The pupils who do well will be congratulated on their performance, but to a large extent, they just got lucky. The English teachers may even be congratulated if they happened to be teaching a large number of spy geeks. They may even get a pay rise or promotion based on this coincidence.
This is completely unjust, both to pupils and to teachers. General examinations given to the whole population should always be based on specified content. The more specific the content is, the fairer the examination will be, as it will enable all pupils to master that knowledge, and help all teachers to teach it effectively. As well as being fairer, only testing specific topics would promote education as the transmission of knowledge, rather than the development of mythical generic skills.
Because of the absence of specific content, preparation for general reading tests ends up being largely a process of tedious drills in how to serve up to the examiner what they are looking for. ‘Make sure you mention structure’, advise the English teachers, or ‘look out for similes and metaphors, and make sure you comment on them.’ How dull, compared to the richness and variety of content on which we could be focusing. But such a box-ticking approach seems inevitable when faced with general reading tests, because the knowledge required has not been specified in advance.
But we must not lose hope. We must make things better despite the unfair assessment system which currently exists, and we must work to persuade policy makers that fairness and the promotion of knowledge depend upon making tests as specific as possible.
How can schools best cope with the situation until such a time as assessment policy catches up with the findings of cognitive psychology?
The answer must lie in giving as much coherent knowledge as possible to pupils, especially in the earlier years, and in spending the minimum amount of time on tedious drills in examination skills. We can also make sure that any internal tests which we set are based on specific content which pupils are expected to master.
Instruction in how to give the examiner what they want should be left to the months immediately prior to the exam. The rest of schooling should be devoted to making pupils as knowledgeable as possible, because if we want our pupils to be creative and to think critically, if we want them to read well and write well, it is knowledge which counts.