The trouble with exams…

Those hard hitting undercover reporters at the Telegraph have been at it again today. After nearly scoring rather an own goal in the Murdoch Sky takeover by entrapping Vince Cable into saying inflammatory things with two pretty young ladies, it seems they’ve now been looking for another potential long running story by latching onto the perceived corruption of the British exam system. Their undercover reporters went to “13 meetings organised by (exam) boards used by English schools”, looking for evidence of what can be painted as corruption. And guess what? After looking pretty hard, it seems that they’ve found some.

The video on the Telegraph website, secretly filmed at one of Welsh exam board WJEC’s teacher training sessions, certainly looks damning. In it, senior examiners appear to tell the attending teachers what areas of the GCSE History syllabus will be covered in the upcoming exam paper. Because this fits the narrative the Telegraph are trying to shape, they comment that their investigations showed this to be a routine occurrence; that “teachers were routinely given information about future questions, areas of the syllabus that would be assessed and specific words or facts students must use to answer in questions to win marks”.

Fresh from his somewhat bizarre expenditure in sending schools across the country new copies of the King James Bible with a foreword he wrote (perhaps at God’s urging), chinless Education Secretary Michael Gove has been quick to leap onto a potentially non-controversial (for the government) issue about which he can be seen to be “doing something”. Eager for any opportunity to get his less than aesthetically pleasing mug into the papers, Gove has commented that such incidents "confirm that the current system is discredited", and has ordered new qualification standards watchdog Ofqual to conduct an investigation, the results of which may, politically, be a foregone conclusion.

The trouble is that, like so many recent media storms, this isn’t as simple an issue as the Telegraph are making out, and their pejorative language isn’t helping to clarify the issue. The sinister, X-Files-style air of conspiracy is generally enhanced when they claim that what they’ve been investigating was “series of secretive exam seminars, which are thought to have rapidly grown in popularity in recent years”, and for which shady examiners charge “up to £230 a day”.

Speaking as someone who used to work for one of the country’s leading exam boards, and administered one of these qualifications directly, I’m afraid I have to pour a little cold water over the Telegraph’s Oliver Stone-ish picture of a sinister cabal of exam boards and examiners conducting backroom deals in anonymous hotels. The “secretive exam seminars” they refer to are in fact very standard training days for teachers in how best to teach the courses prescribed by the board. Every exam board uses them, and the obvious advantage of having senior examiners present the courses is that they generally set the standards for the marking, and so are in the best position to advise on how students can get good marks.

Yes, these ‘sinister’ meetings can indeed cost £200 or so a day (though better financed exam boards sometimes offer them gratis). But the Telegraph story’s implication that this is some kind of bribe trousered by corrupt examiners is, frankly, bollocks. It’s charged by the boards, goes into the overall budget for training and marketing, and used to fund further training programmes. It is not, however much the Telegraph would like it to be, used by corrupt examiners to fly back and forth to shadowy meetings held by the Bilderberg Group, the Masons and the Illuminati.

But let’s look a little closer at the video evidence of the particular case that has sparked so much fury this morning. Are these examiners, as the Telegraph implies and so many people have clearly inferred, stepping over a line and revealing the questions in the upcoming exam to the privileged few teachers whose schools are prepared to fork out the training money?

Well, as with the Clarkson debacle last week, the selective clips used in some news outlets (including the BBC) certainly make it appear that way. But to give the Telegraph credit, they’ve been even-handed enough in posting a much longer clip that gives a better sense of context to the remarks. The oft-quoted soundbite (it’s in big letters on the Telegraph’s front page) is “We’re cheating. We’re telling you the cycle. Probably the regulator will tell us off.” That certainly sounds like someone who knows he’s doing wrong.

But if you look at the whole clip, it’s clear that the trainer is telling teachers that a particular topic is likely to come up, rather than a specific question on that topic. At this point, I should mention (as none of the press have) that every qualification is underpinned by a very carefully worded exam board document called a ‘specification’. Having been involved in developing and drafting these for new qualifications myself, I can tell you that they are obliged to cover the exact details of the course, and that no exam question can be set on a topic outside the areas of study contained therein.

Now I’m not familiar with the exact details of the WJEC History GCSE specification, but judging by the clip, this is a compulsory question which may be asked on one of three potential topics. As topics cannot be repeated from year to year, and there are three of them, anyone could check the past papers (freely available on boards’ websites, incidentally) and make a reasoned guess as to how these three topics will cycle round. In this case, these are newish qualifications, though, so the information couldn’t be surmised yet. But it could be within a year or so.

Undoubtedly this examiner is crossing a line in explicitly spelling it out to teachers, which illustrates how amazingly careful you have to be in wording what you say. But what he’s doing is simply telling the attendees what they could, in all likelihood, have worked out for themselves. It’s not the same as telling them the actual question, which is likely to be a bit more specific than “write something about Germany between 1933 and 1939”.

One teacher then asks whether, as educators, they should be teaching the children all the topics, rather than just the ones likely to crop up in the exams. And the trainer says that in an ideal world, they would indeed. But he acknowledges that teachers, driven by the tyranny of league tables and the political pressure to improve results year on year, may well find this an impossible demand; despite sailing way over the line, he’s trying to help – and these are the sort of questions teachers always ask in these training sessions, for precisely these reasons.

The second presenter is trying to spell out a section relating to modern US Presidents, and has incurred the wrath of the Telegraph by saying “off the record” that no question is likely to come up relating to “the Iraq war”. Again, this is clearly crossing a line, but not to the extent that it might appear. The examiner has just said that this section “now extends to 2000”, and points out the unlikelihood of questions coming up relating to “Clinton or Bush”. OK, it sounds as though the paper should cover Clinton. But which Bush, and which Iraq war? I’ll agree, the most likely interpretation is the ones from 1991, but there’s some doubt there.

Still, having said that, both trainers do seem to cross a line between helpful hints and actually breaching confidentiality. As Chief/Principal Examiners, they would be the ones responsible for setting the questions, and would have the necessary insider knowledge; this is clearly grounds for disciplinary action.

But speaking as someone who’s organised and on occasion presented such training, this debacle shows very clearly how careful one has to be, as a board representative, in wording what you say. Teachers, under pressure to deliver good results, often leap on potentially different interpretations of training speeches or even specifications themselves to justify their claims that markers have treated their students unfairly, and in representing an exam board you have to be conscious of the fact that anything you say is likely to be dissected with the sort of linguistic attention rarely seen outside a legal chamber. I’m not sure that what we’re seeing with these two trainers is outright corruption; more, it seems to be incompetence at knowing how far they can go in their statements as representatives of their exam board.

All of this, however, has been leapt on by the Telegraph and now our esteemed Education Secretary as evidence that the entire system of exam boards is riddled with corruption, and must be fundamentally reformed. The first contention is clearly nonsense – this is one qualification at one level from one (fairly small) exam board. WJEC don’t have the dominance of AQA, OCR or Edexcel, but even they presumably offer dozens, if not hundreds of qualifications at each of a variety of levels, each comprising multiple modules with their own exam papers. Discovering this kind of indiscretion in two senior examiners is like saying that all male film stars must be gay on the evidence that Rupert Everett and Neil Patrick Harris are.

But the idea that the system could do with fundamental reform? I actually think there might be something to that. The trouble with this kind of media shitstorm is the tendency for politicians (who usually have no personal experience of the issues surrounding education) to act in a knee jerk way and change the system for change’s sake, without considering whether the changes are the right ones. Every incoming government (and this one is no exception) wants to radically change the education system; in part, it’s to stamp their own authority in it, and in part it reinforces the standard party mantra that everything the other party did must be wrong. But that doesn’t mean that what they’re doing is right. As Sir Humphrey once spelled out the politician’s creed in Yes, Minister: “Something must be done. This is something. Therefore we must do it”. Or, more populist, Jeff Goldblum’s admonition to Richard Attenborough in Jurassic Park, “your scientists were so preoccupied with whether they could that they didn’t stop to think if they should”.

The most obvious bone of contention is that there are three (major) exam boards, and they’re in competition with each other. This should, logically, appeal to the Conservatives, with their mantra that free market competition guarantees quality. The trouble here is that, in education, standards should be consistent for everyone; how, then, should exam boards compete for schools’ business? Oh sure, there’s cheaper (or free) teacher support, or the advancement of new, innovative, technical systems. But ultimately, what schools want from an exam board is for a high proportion of their pupils to get good grades. With this being the sole criterion for choice, the logical progression is that boards can only compete by tacitly implying that schools will get better results with them – ie, that their courses are simpler and their exams easier. This is, of course, against the letter of the law; but it’s amazing the sophistry that can be employed to circumvent this.

In practice, if the mode of competition is to lower standards, the ultimate result is that well-worn phrase “a race to the bottom”. The ever-increasing numbers of students achieving high grades seem to bear this out; surely it should be impossible for these to increase every year without some lowering of standards?

Again, the reasons for this trend are more complicated, and have as much to do with politics as education standards. Under the current system, the marks required to get particular grades can vary, according to senior examiners’ assessment of an exam’s difficulty. These are set at a meeting presided over by a Chair, who is ultimately the person responsible for the qualification’s standards. The trouble is that, to meet government regulatory requirements, the Chair may be (and frequently is) obliged to overturn the examiners’ judgments to maintain the percentage of entrants at particular grades – this is seen as the only way to guarantee consistency of standards.

But a statistical standard really shouldn’t be how we judge the quality of education. A fundamental misunderstanding that it’s possible to quantify the unquantifiable is, in my view, at the root of much is what is wrong with the current system. That’s what gave us the overly simplistic school league tables under the Thatcher government, the continual tyranny of statistics under the Blair government, and now the addle-headed English Baccalaureate under the current one. None of these take into account the complexities of the issue; in their attempt to boil down so many things into a set of judgmental statistics, it’s along the lines of the oft-heard complaint against the Conservatives – that they know the price of everything and the value of nothing.

How to change it, then? To be fair to the Coalition, they have got all sorts of committees investigating that very question, although politicians rarely listen to reports that contradict their pre-existing prejudices (just ask the Advisory Council on the Misuse of Drugs). One common suggestion is to scrap the element of competition and simply have one, publicly owned, exam board. This is the case in many countries (Australia for one) with higher recognised educational standards than this one. On the face of it, that has a lot of merit – one standard, applied consistently to every school, with no ‘dumbing down’ and as much independence from ill-qualified politicians as possible.

In practice, unfortunately, it seems impossible. We have a long entrenched system of competing boards, originally sponsored by the Universities (OCR still is). The practicalities of buying their interests (though OCR and AQA at least are classed as charities) would be highly complex and expensive. And at a time when the government are busy introducing the most nakedly ideological Conservative policies since Mrs Thatcher, it hardly seems likely that a Conservative-led government would suddenly spend masses of public money to, effectively, nationalise a whole sector of employment. Not to mention the fact that they’re keen on introducing even more privatisation to education, with Free Schools and the ability of corporations such as McDonalds’ and JCB to award qualifications – initiatives started under the more-Tory-than-Tory New Labour.

Oh well, if we can’t do that, why not try removing politics from education? Altogether? Reduce the demand to boil results down to meaningless statistics for populist reasons; if no votes depended on the percentage of students getting an A, I suspect we’d soon see a fall in those levels to reflect real standards. But while political popularity is tied to such statistics, it would be a ‘courageous’ politician who would stand up and say “what we need is fewer people doing well in exams”.

A properly independent publicly funded education system, regulated independently with the minimum of government oversight and organised by actual educational professionals would be a real step forward. You’d think that would find favour with the laissez-faire, keep-the state-out-of-things Conservatives. But again, sadly where public money is concerned, the politicians are convinced that only they can make the best-informed decisions – a timeworn fallacy that’s applied to governments of every party over the years.

I appreciate that this has been a long ramble on the subject, but to return to the original point – two bad apples does not a rotten barrel make. The system we have is far from perfect, and I’m always glad when the debate about fundamental changes is spurred. But this knee jerk media-generated scandal is entirely the wrong way to go about that debate, and it’s appalling (but not surprising) that policy makers should so easily leap onto the mob bandwagon. There is scope for real change in this area, but I suspect we won’t see anything I’d consider improvement as a result of the Telegraph’s scandal mongering.