Making the Grade

It’s that time of year again when the exam results come out. Yesterday it was the Scottish results, and next week will see A-Levels, and beyond that, GCSEs. While many students will be toasting the merits of their hard work or otherwise, the inevitable question pops up every year as to whether the exams are getting easier. I’m always suspicious of the media reporting statistics (particularly at a time when young people who’ve worked hard for years might be feeling a bit sensitive), and I think they like to assume that people don’t have the time and inclination to check the information for themselves. But they haven’t reckoned on me.

For example, they often talk about the high “pass rate”. At GSCE, you can obtain the grades A*, A, B, C, D, E, F, G, and U. Of these, only U is technically a fail. A G is a pass, not a very good pass, but a pass nonetheless (generally you need at least a C in Maths and English to be considered for university and many jobs). Not every grade is available to every student. For example, back in the day, there were three Mathematics papers you could sit (Maths was the only one with three tiers, most other general subjects had two). If a student is weak, they are given a better chance of passing by sitting an easier paper, at the expense of the chance of a top grade. The higher paper was the hardest, and you could get grades A* to C (if you failed it, you still got a U). Intermediate allowed you B to E (I think) and Foundation D to G. I am led to believe it’s a similar story in Scotland. The fact that such a system exists is not reported.

The reason I bring this up is that it is deeply misleading to judge the merit of exams solely on a “pass rate”. To pass well, first you must pass; and schools achieve this by putting the weakest students forward for the easier exams. If the pass rates are going up, it could be a result of schools becoming more risk-averse by putting more borderline students forward for the foundation level exams.

But this is just one suggestion on my part (I’m sure those who work in education will tell me to get lost if I’m wrong!). It could simply be the case that young people are working harder or that education is getting better generally. My point is that you can’t come to a conclusion such as “exams are getting easier!” based on one number.

Are they even really getting easier anyway? Time for some numbers.

The below is a breakdown of results in Standard Grade Maths by year: class refers to the number of students sitting the exam in a given year, and the number on the top row is the result: the percentages below it are those that achieved the grade stated (1 is best).

Year  Class 1 2 3 4 5 6 7 No award
2002 59,047 17.75% 14.34% 22.16% 16.25% 19.25% 8.50% 1.66% 0.09%
2003 59,431 17.95% 14.07% 20.32% 17.55% 20.19% 7.66% 2.17% 0.09%
2004 56,767 17.99% 13.95% 21.37% 16.76% 19.72% 7.95% 2.16% 0.10%
2005 53,835 17.41% 14.21% 22.09% 16.41% 19.55% 7.92% 2.31% 0.10%
2006 53,776 18.72% 13.32% 22.09% 16.81% 20.06% 7.26% 1.63% 0.11%
2007 53,973 17.12% 12.40% 21.08% 16.28% 22.53% 8.04% 2.46% 0.09%
2008 50,974 16.63% 14.08% 24.59% 17.51% 20.04% 5.84% 1.25% 0.07%
2009 46,779 17.24% 15.02% 26.31% 16.56% 20.34% 3.83% 0.59% 0.10%
2010 43,985 17.13% 13.96% 22.76% 17.60% 20.50% 6.27% 1.71% 0.06%
2011 42,651 16.56% 15.87% 27.74% 14.54% 18.48% 5.42% 1.34% 0.06%
2012 40,879 18.20% 15.15% 26.77% 15.96% 19.25% 3.86% 0.78% 0.03%

Caveats are numerous: this is one subject (and not a particularly well-liked one) at one level in one country, but it should hopefully provide some level of idea to what’s going on.   [as a curious aside, it’s interesting to note the steady decrease in students sitting the exam. I wonder why this is?]

Looking at this, there does not seem to be that much of a change since 2002, save for a decrease in the number of students achieving a 6 towards the number of students achieving a 3. Indeed, under the standard measure we use at work for determining whether a population profile has changed (which I don’t think I’m allowed to disclose, so you’ll just have to trust me), the change is not significant. While more students are getting 1s, the figure actually went down the previous year. I would argue that if Standard Grade Maths were truly getting easier, more students would continue to get 1s year on year, but this is not the case; we’re 0.5% points away from the ten-year high of 2006.

We must therefore conclude that there is little evidence to suggets that Standard Grade Maths is getting any easier based on the grade distribution of the last ten years.

Below is a similar table, except the figures are the percentage of students who achieved the stated grade or higher in the given year.

Year  Class 1 2 3 4 5 6 7
2002 59,047 17.75% 32.09% 54.25% 70.50% 89.76% 98.25% 99.91%
2003 59,431 17.95% 32.01% 52.34% 69.88% 90.08% 97.74% 99.91%
2004 56,767 17.99% 31.94% 53.31% 70.07% 89.79% 97.74% 99.90%
2005 53,835 17.41% 31.61% 53.70% 70.11% 89.67% 97.59% 99.90%
2006 53,776 18.72% 32.04% 54.13% 70.95% 91.01% 98.27% 99.89%
2007 53,973 17.12% 29.51% 50.59% 66.87% 89.40% 97.45% 99.91%
2008 50,974 16.63% 30.70% 55.29% 72.81% 92.84% 98.68% 99.93%
2009 46,779 17.24% 32.26% 58.57% 75.13% 95.48% 99.30% 99.90%
2010 43,985 17.13% 31.09% 53.85% 71.45% 91.95% 98.23% 99.94%
2011 42,651 16.56% 32.42% 60.16% 74.70% 93.18% 98.60% 99.94%
2012 40,879 18.20% 33.35% 60.12% 76.09% 95.33% 99.19% 99.97%

The differences here are less easily explained: the proportion of students getting 3 or higher is just below a ten-year high (a small decrease since 2011), ten percentage points higher than in 2007 (which, the more I look at the data, looks like a blip: perhaps the exam format changed that year, or there was some other thing I’m not aware of). Most of the students have swung away from 5, 6, and 7 towards 2 and 3. Again, though, I would argue that if the exam was truly made easier over this time, there’d be a greater uplift of 1s over that period than the one percentage point we see.

As for the overall pass rate, a 6 or higher is adjudged as a pass, and while this is high at 99.19%, it isn’t even a ten-year high (2009 was better).

In conclusion, then, as I’ve said, it’s not enough to say that the increasing pass rate is evidence exams are getting easier. Other subjects may well prove me wrong, but in this case it’s encouraging for the kids that the media’s lack of sensitivity is even less justified.

Enjoy your achievements!

You’re welcome to use the data yourself, it’s available here.

This entry was posted in Uncategorized. Bookmark the permalink.

2 Responses to Making the Grade

  1. markdavo says:

    The reason for the decrease in the number of pupils taking Standard Grade Maths is because many pupils are now taking Intermediate 1/2 and Access 3 exams instead. These were originally brought in to give pupils who wanted to continue maths after Standard Grade, but did not have the Credit Pass (a 1 or 2) necessary to start Higher Maths.
    However, in maths at least, Intermediate 2 is perceived as ‘easier’ than Credit Standard Grade so many schools switched.

    However, trying to add in these results to yours is not possible, since SQA has not released Intermediate 1/2 and Access 3 results for S4 pupils only.

  2. annonymous says:

    The number of students sitting the standard grade exam may be going down as a result of the increased uptake of the equivalent intermediate 2/1 level exams which were supposed to replace sgs. Just saying.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s