National Testing Should Be Broader, Simpler and More Frequent

Aren't you glad your pilot passed a few tests?

Aren’t you glad your pilot passed a few tests?

Test-bashing is an easy way of raising a cheer from teachers. Many American commentators blame standardised testing for all kinds of ills in their education system. But, as E D Hirsch points out, there is no alternative to regular testing if we wish to hold schools accountable:

Tests of academic progress are the only practical way to hold schools accountable for educating all children and are therefore essential to the twin aims of quality and fairness (The Knowledge Deficit, p91).

It may be true that we won’t fatten a pig by weighing it, but a good farmer will know how fat his pigs are, will be looking at why some are fatter than others, and seeking ways to make sure as many as possible are as fat as possible. He won’t be able to do any of those things unless he weighs them. Likewise, a good hospital will check outcomes in different wards and compare them. When there are major differences they will investigate why. Carl Hendrick describes an example of the importance of this kind of analysis in medicine here. Without analysis of results, medical reform is impossible. Likewise, without frequent objective testing, education reform is impossible. Charter schools such as KIPP have no fear of standardised testing, because it proves that their methods work. It is more likely to be the opponents of reform who attack testing, just as the opponents of Semmelweis’ medical reform questioned the validity of his data.

Moreover, national testing is needed, not just every few years, nor at the end of five years, as we now have in secondary school, but annually. Annual testing tracks whether pupils are ready to move to the next level. It holds schools accountable for every year of progress, so that every year must be taken seriously. Too often the early years of primary and secondary school are not taken seriously enough, because public examinations are such a distant prospect. Hirsch again:

Yearly testing is essential both to keep track of each student’s progress and to encourage teachers to cooperate in providing students with a coherent education in which each grade can build on the previous one (The Knowledge Deficit, p92).

The howls of protest at the proposal to repeat KS2 tests at the end of year seven show how popular annual tests would be with teachers. There is already much complaint about how teachers end up teaching to the test in year six, and in the years prior to GCSE exams. Teaching to the test is educationally damaging. But it happens not because of the frequency of tests or because they are particularly demanding, but because they are infrequent and undemanding, but very fiddly.

Frequency

The low frequency of national tests makes them seem like a big, scary thing, and so teachers focus on them excessively, and the teacher’s own anxiety often influences the pupils. It also places unfair burdens on the shoulders of the teachers who specialise in the years when testing is done. If national tests were an annual event, they would become much more of a normal part of school life, and would involve all teachers, at every level. No-one would be able to defer dealing with challenging material, leaving it to the following year when the pupil would not be their problem. This is particularly important at primary level, where the curse of developmental theory is keeping children at the intellectual and behavioural level of toddlers, and those who are keeping them there are not being held to account. Year six teachers are then left to pick up the pieces.

Fiddle

Current tests are undemanding in terms of breadth of knowledge, but fiddly in terms of technique. This is because of our British addiction to performative testing, where the assessment is supposed to mimic reality. Performative testing through methods such as essay writing (or, even worse, coursework) can only ever sample a very small part of a pupil’s knowledge. In contrast, well-designed multiple choice tests can reliably and quickly sample a much larger part of the knowledge domain. When the breadth of tests is sufficiently wide and the techniques for testing are not complex and burdensome, it becomes neither practical nor effective to teach to the test.

The marking of multiple choice tests is also reliable and objective, whereas essays are notoriously difficult to mark consistently, while coursework leads to outright cheating and inflated grades becoming the norm. Because of the difficulty of achieving consistent marking of something so complex and variable as an essay, exam boards end up depending much more on specific techniques and terminology. The techniques and terminology must then be drilled for the exam (or coursework essay) – the dreaded ‘box ticking’. Daisy Christodoulou notes how prevalent this is in an analysis of current GCSE history text books, which lack breadth of knowledge, but are packed with exam tips. Based on the content of these textbooks, ‘Clearly, to understand Germany from 1919-1945 it’s more important to meet the examiner than to meet Bismarck’ (Changing Schools, p49). Christodoulou goes on to point out that exams are too ‘tricky and technical’: ‘Many teachers feel forced into excessive teaching of exam technique because otherwise, candidates with a good grasp of the domain are penalised for not ticking the precise boxes on the mark scheme.’ (p51).

In his analysis of the pitfalls of essay marking, Hirsch examines the work of Princeton’s Educational Testing Service, and concludes that there is no guarantee of consistency beyond what has been agreed in one particular session by one particular group of examiners. Even after a laborious process of calibration (what is called ‘standardisation’ here in the UK), ‘after the lapse of a weekend’, further calibration was needed to keep markers in line. He comments:

In fact, one of the College Board’s reasons for instituting multiple-choice testing was the finding that the high-stakes grades given to a student’s performance-based test “might well depend more on which year he appeared for the examination, or on which person read his paper, than it would on what he had written.” (The Schools We Need, p184, quoting Godshalk, Swineford and Coffman, The Measurement of Writing Ability, 1966)

Achieving a Balance

For testing writing ability, Hirsch recommends a combination of an essay component with a generous multiple choice section. He suggests that an essay component should be included not because it increases the validity of the score significantly – researchers have found the improvement to be marginal – but simply because having some kind of essay component sends a message about the value of extended writing (see The Schools We Need p185-188).

In areas where there is no need for this message about extended writing, there is no reason why tests should not rely exclusively upon multiple choice questions. Whatever the specific decisions about including an essay element, the fact remains that there are methods already in existence for cheaply and reliably measuring academic progress in a way which precludes the possibility of teaching to the test. What is lacking is not the way, but the will.

Conclusion

At university level, demanding multiple choice tests are used to examine the knowledge of medical students. Here is a topic where it really matters whether you know your stuff, so they cannot risk the subjectivity and narrow focus of essay based exams. Nor can they worry about damaging the self-esteem of medical students by testing them too much. We have to ask ourselves whether it really matters to us whether school pupils know things. If it does, then we need proper annual testing to check that they do.

If the political will existed, we could soon determine in a clear and objective way which pupils had been dressing up in togas and talking about how it felt to be a Roman slave, and which ones had actually learned something about Roman history. Then parents could make a real, informed choice. If they believed schools should be teaching academic knowledge, they could select one with good results on annual national tests. If, on the other hand, they believed school was intended to be expensive babysitting and third rate entertainment, they could select one which just moaned about the tests and failed to prepare children properly for them by actually teaching them something.

The principle of objective annual national testing of a broad spectrum of knowledge fits with the principle of a specific, coherent national curriculum. Put together, these two strands of education reform could empower every pupil in our state schools with the knowledge that would give them access to literate culture, and provide a foundation for further learning throughout life.

As they await the coming of this millennial moment, though, it behooves those schools who do care about transferring empowering knowledge to every pupil to introduce their own system of a coherent curriculum matched with frequent, objective testing. There is no conflict between this approach and success in GCSE exams, because those who learn in this way will know far more than the curriculum requires, and formative assessment will of course continue to include extended writing where appropriate. Schools such as Michaela are shining beacons of the way forward, and the more they prove their effectiveness, the closer the millennium approaches.

(Image from Wikimedia).

Advertisements

33 thoughts on “National Testing Should Be Broader, Simpler and More Frequent

  1. I agree with the main thrust of the argument here. However, I would raise one issue: I don’t think it’s at all the “fiddliness” of national testing that makes it unpopular. It’s the high stakes nature of the tests for teachers, contrasted with the limited benefit for students that makes them particularly unpopular.
    True, some teachers just don’t like tests, and true the nature of the current system probably makes that worse, but the key to improving the popularity of reliable testing will always be in removing the unnecessary high risks attached to them.

    Liked by 1 person

    • Good point. Lemov and Hernandez in ‘Changing Schools’ point out that the unit of accountability must be the school, not the teacher. They are also very keen on objective testing for accountability, of course!

      Like

    • You can tell where the article is going as soon as the author mentions ‘merciless performance targets and obsessive testing.’ If parents want to pay thousands to have their kids trained to build boats and cook onions, that’s their choice I suppose. Not an option for disadvantaged youngsters who don’t have articulate, well heeled, Guardian reading parents.

      Liked by 1 person

      • Also, if “disadvantaged” parents did have the choice to send their child to somewhere like Drumduan – i.e. if such schools were state funded… If (heaven forbid) we gave the disadvantaged young people themselves a choice over how they should be educated, rather than paternalistically presuming to restrict their only hope of liberation from disadvantage to a relentless battery of mandatory testing (which requires and forces many to fail, by the way) – I wonder how many would choose the compulsory testing comprehensive over boat building and caramelised onions?

        Like

  2. Reblogged this on pedagog in the machine and commented:
    “there is no alternative to regular testing if we wish to hold schools accountable”

    Interesting to read this piece on the same day as reading about a school which doesn’t test students *at all* being held to account – and given high praise indeed – by HMI (see link below).

    The Traditional Teacher piece does a good job of summing up the ‘conventional wisdom’ perspective. But for me, important questions remain unexamined here: Do we *really* need testing in order to have accountability? Does accountability based on test scores encourage a ‘teach to the test’ performance-oriented culture which can actually serve to undermine a young person’s love of learning? These are rhetorical questions by the way, because the answers to me seem self-evident.

    Here’s a question that is far more interesting: if HMI are satisfied with what’s happening at Drumduan Upper School, why is every other school in the country bending over backwards to play the testing game?

    http://www.theguardian.com/film/2015/jun/13/education-school-tilda-swinton-scotland

    Like

    • The question of teaching to the test was one of the issues I *was* examining. I pointed out that our narrow, fiddly tests promote this, whereas broader, more straightforward tests would not.

      Like

    • It’s hilarious and bizarre that you call my proposals the ‘conventional wisdom’, given that what I propose has never existed in the British education system, which has been institutionally opposed to objective testing of broad academic knowledge for decades, and has subverted successive government attempts to introduce accountability by submerging testing and assessment methods in byzantine complexity. I’m calling for clarity, simplicity and accountability. I’m calling for radical reforms. You’re singing the same tune that has been conventional wisdom amongst ed school academics since the days of William Heard Kilpatrick a century ago.

      Like

      • I think the system is already characterised by rather a lot of testing. Ask my year 9s – they aren’t even doing GCSEs this year but they regularly ask me “Why do we have to do so many tests”? That is the sense in which your perspective is conventional – you are a proponent of testing in a testing-saturated system. Not only do we already have annual testing, we have 2 assessments per half term – 12 a year. It’s insane.

        I think it boils down to the mistake of falling for a one size fits all mentality. Some students like testing, and so do some teachers, and so do some parents. If you like testing you should go for it. But I think all tests should be optional. There are other ways in which students, teachers and schools can be held to account, if they choose not to play the testing game. So let’s set them free – what’s the worst that could happen?

        Like

      • But you still have not addressed the central point of my article. You are fed up with current testing, but instead of considering whether there could be a better way of testing, you want to throw it out altogether. Optional testing is the equivalent of abandoning testing. Tests are not optional for pilots, because it matters whether they know something. They should not be optional for schoolchildren either, because it matters whether they know something. Or perhaps you don’t think it actually matters?

        Like

      • To clarify, I agree that pilots should be tested. By opting to become a pilot, they undertake to meet the conditions of entry. The road safety example you give is an interesting one, because we don’t have national tests in road safety, and still the vast majority of people manage to get through life without being run over. Perhaps you think we should have annual road crossing exams as well?

        I don’t really understand why forcing someone to take an exam against their will is currently considered to be acceptable. I genuinely find it extremely baffling, and I am hopeful that rather than the Millenial vision you outline, we will soon look back on the age of mandatory testing with bemused regret

        Liked by 1 person

      • When something matters, it is not optional. When I tell my children to tidy their rooms, it is not optional. When I teach them to look before they cross the road, it is not optional.

        When I tell my pupils to do a test in order to find out whether they have mastered a body of knowledge, it cannot be optional, because it matters. Because I care. I’m testing them because I care whether they have learned anything or not. Not testing would be the easy, lazy option.

        Like

    • If you crash a car or lose a job, you get very obvious feedback. If you fail to master knowledge, you don’t, unless you are tested. Testing helps you know what you don’t know. It is a positive thing which helps people improve.

      Like

      • So, if I never take a multiplication test, you would never be able to figure out that I can’t do multiplication and I won’t know either? If I don’t understand how to write a paragraph, without a test you’ll never notice? If that was true, a high school teacher would have too test children on their alphabet, all computation, and more every year. Trust me, we know when kids don’t know the material, or they make it obvious over time. Performance, in the classroom and life clarifies our strengths and deficiencies. No test needed. Why don’t adults get tested daily, weekly, monthly, etc in the workplace? Performance.

        Like

      • Lol. Anthony, dear. Just as your objectivity differs from mine, so does any “authority”. To whom does (should) the learning matter? HE is the true assessor. Tests only measure what you can remember in that moment in time. They are false measures of knowledge and understanding.

        Like

      • Test results are not false or pointless because the information they provide is incomplete. Any scientist will tell you that proof only counts until we discover an exception. Nevertheless, we can discover what is probable and act upon that basis. Saying that knowledge is incomplete is not the same as saying it is useless.

        ‘Your objectivity differs from mine’. Such abstract relativism would soon crumble were someone to steal your car, or slap you in the face. Then you would suddenly transform into a positivist who believes in objectively verifiable external reality. Philosophical relativism can serve no practical purpose. It tends simply to promote intellectual laziness, because there is apparently no truth to discover, so why bother thinking?

        Liked by 1 person

  3. Pingback: Teacher Accountability | The Traditional Teacher

  4. Pingback: Why Aren’t More Schools Making Use of Their Freedom? | The Traditional Teacher

  5. Our educational system was created, and is built on “educational standards”. In order to determine if these standards are being maintained, regular assessments and tests must occur. The overwhelming majority of parents, who support the education system, want testing to continue. There will always be a few vocal minorities who pontificate on the evils of testing, but if the majority see the benefit of tests, let it continue on.

    The other consideration regarding ongoing assessment, is that it is tied into school success as well as student success…something that ensures if our system is actually working, and it points out where improvements needs to be made. Those that do not want to adhere to these standards, have the option of homeschooling their children, and/or provide alternative education if they do not agree with the system upon which it was founded over 150 years ago.

    The evidence here should speak for itself, however the howls of protest in how testing damages our children’s self esteem grow ever louder, in a world where mediocrity continues to be celebrated, where Honour Roll students are no longer acknowledged, and where 5 year olds have graduation ceremonies at their nursery schools. We are failing our kids by setting up these false environments of success. In the real world, the value of grades and tests very much do matter, if we want a strong society to exist. Thanks for the blog entry. Nice to hear one from the majority of us to speak up on this issue.

    Like

  6. Pingback: Non-Competitive Approaches Legitimise Envy | The Traditional Teacher

  7. Pingback: In Defence of Dame Sally Coates: Order and Clarity Unleash Creativity | The Traditional Teacher

  8. Pingback: Learning Cannot Be Inspected | The Traditional Teacher

  9. Pingback: The Emptiness of the English Curriculum | The Traditional Teacher

Thoughtful and reasonable discussion is always welcome.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s