Guardian fails on Sats reporting

The Guardian has been getting itself into a muddle over its reporting of the Sats results.

At 11.49 this morning it published a version of its story to the RSS feed on its website which contained the following lines:

Of the 600,000 pupils who took the tests, 72% did not reach level 4, compared to 71% last year. This means they cannot add, subtract, multiply or divide in their heads, or write extended sentences using commas.

An hour or so later, when I checked the website itself, this had been amended (my emphasis):

Of the 600,000 pupils who took the tests, 72% reached level 4, compared to 73% last year. This means they cannot add, subtract, multiply or divide in their heads, or write extended sentences using commas.

First of all, this obviously makes no sense as a sentence now. The 72% are the ones who can add, substract, etc.

Second, given that the whole thrust of the piece is about declining standards in schools and possible marking inaccuracies, the lack of attention to detail here is striking. Guardian, check your stats and sort it out before you publish next time, not after.

This is not the only problem with this article, though. Take a look at the results and you’ll see that these changes are pretty minimal: no change in maths and science, and a 1% drop in children achieving Level 4 in English, reading and writing. This is not good news, obviously, but neither is it an educational catastrophe.

The Guardian, post-correction, states the figures accurately, but uses an inconsistent style of description to make them sound as bad as possible. For example:

More than a quarter of 11-year-olds in England failed to achieve the standard expected of them in their English and maths Sats.

Given that this standard (Level 4) is what is expected of ‘a typical 11 year-old’ (source), it’s not surprising that around a quarter are below it. More than a quarter are also above it: Level 5 achievement – in other words, 11 year-olds attaining the level of a typical 13 year-old – ranged from 19% in writing to 47% in reading.

Try this…

The statistics from the Department for Children, Schools and Families showed boys were falling far behind in writing. Four out of 10 boys did not reach the level expected of them, while 75% of girls did.

Using a negative natural frequency for boys (‘four out of 10 boys did not’) and a positive percentage for girls (’75% of girls did’) is unfair, as the eye and mind gravitate towards the ‘four’ and the ‘seven(ty)’. In fact the attainment difference is 15% – again, clearly a problem, though this is by far the largest differential.

For cherry-picking worst cases and wording them as direly as possible, not just for the shoddy and unmarked correction of published material, this article fails to achieve the standard expected for education journalism.