Yesterday The Telegraph printed a story about S4C viewing figures for the football programme Sgorio:
Sgorio – Welsh for score – turned into a no score draw on the night despite regularly pulling in tens of thousands of viewers on other nights.
It is a regular show on Channel Four in Wales featuring top matches from the German, Spanish and Italian leagues.
Under the TV rating system, any programme with fewer than 2,500 viewers is regarded as “making no impact”.
Today the Western Mail ran a very similar story. Before we get into a discussion about what this might mean, let’s examine the figures. According to The Telegraph:
The figures were compiled by the Broadcasters Audience Research Board.
The body surveys viewing habits of 11,300 viewers in 5,100 British homes, and weights them according to the rest of the population.
How many homes own television sets in the UK? Let’s use BARB’s own figures:
In 2009, 25.9 million homes own at least one television set out of a total of 26.6 million homes in the UK.
Let’s concentrate on television-owning homes. According to this 97% of homes in the UK own a television.
The 5,100 British homes is 0.02% of the homes that own a television. BARB is collecting these figures by sampling one house in every 5000 television-owning houses in the UK, roughly speaking.
That may be adequate for popular programmes but is it enough to gauge the popularity of a minority language programme?
S4C publishes a top 20 chart. It would be good to have figures for other special interest programmes for comparison, but BARB doesn’t publish these figures free of charge. (Subscribe if you want.) Regardless, how meaningful are these figures in light of the sample size? I’m not aware that BARB has more fine-grained techniques for sampling S4C viewing, anyone know?
How many people live in Wales and in the UK? Let’s take the population figures from the Office for National Statistics:
Wales: 2.9 million
United Kingdon: 59.8 million
Let’s now look at sampled viewers. BARB’s sampled viewers make up a proportion of the total number of viewers in the UK. Let’s assume BARB has picked a fair distribution of sampled viewers in Wales. We could then expect the proportion of sampled viewers in Wales to match the proportion of the UK who live in Wales.
As a proportion, Wales makes up 4.8% of the UK population. Therefore we would expect BARB to have sampled approximately 548 viewers in Wales (which is 4.8% of 11,300).
I’m going to make an assumption here. I’m going to assume that out of the 2.9 million people in Wales, 97% have access to a television in their own home. This reflects the 97% of households who own a television in the UK. Therefore we calculate that around 2.8 million people in Wales have a television in their own home. My aim is to get an approximate impression of the scale here – the order of magnitude – to decide how trustworthy the statistics are. The figures may be slightly off, so please let me know if you have more accurate figures.
Therefore, each sampled viewer in Wales represents roughly 5000 of the television-watching population in Wales. This is similar to the figure for households above. I’m also assuming S4C’s heartland is Wales, although it is sometimes available in households outside Wales.
BARB’s threshold is 2,500 viewers for a programme “making an impact”. These are not real viewers, but figures extrapolated from the comparatively tiny sample size.
So according to this analysis, the conclusion that Sgorio made “no impact” rests on just ONE of the sampled viewers in Wales.
Again, is this enough to measure the reach of a programme, in particular one in a minority language?
UPDATE 21/10/2010: Comments are off but trackbacks and pingbacks are on.
I read this story elsewhere and was mortified. I grew up watching Sgorio and am very fond of it.
Of course it’s all guff. Sgorio is one of the most popular programmes on S4C and loads of people watch it. It’s a far superior programme to anything offered in English for football fans interested in World football. I even knew people with no Welsh at all who would watch it regardless.
Sadly, this all appears to be merely English press laughing at ‘pointless’ Welsh TV and making fun. Shame on them.
Do people still watch TV? 🙂
I’m no expect but I have some hazy recollections of this stuff from way back when…what you are talking about is the sampling error rate. Interestingly, the accuracy of a random sample has a lot more to do with the sample size than the size of the entire population. I seem to remember that the magic number is 400 – at a sample size of 400 then the error lies around +/-5% with 95% confidence.
The upshot of this is that if a survey of 500 odd viewers says that no one was watching a show, I guess that actually means there is somewhere between 0% and about 4% of the viewing public watching the programme.
more info on sample size vs survey accuracy
http://www.amstat.org/publications/jse/v12n2/smith.html
http://www.daa.com.au/analytical-ideas/sample-size/
I know that a significant number of English speakers used to watch S4C programmes like Sgorio, but I’m not sure how true that is by now. The channel is still aimed primarily at Welsh speakers so, optimistically, its potential audience is (according to Wikipedia figures) 611,000 (+ some of the estimated 150,000 Welsh speakers in England that are able to watch). So maybe 700k?
I do know that many Welsh learners and English speakers watch S4C using subtitles but we’d be fooling ourselves to believe this figure is at all significant. The core audience is still ‘fluent’ Welsh speakers (mother tongue or learners).
That potential audience is therefore 1.14% of the UK population, which is only 128 homes sampled by BARB.
BBC Three has a total budget similar to S4C, but has a far larger potential – although still minority – audience. Many of the ratings for its programmes have been (and sometimes still are) officially ‘zero’ according to BARB.
I think this just shows the fundamental problem with measuring the audience in the digital TV world as we move towards the kind of niche broadcasting which S4C has always been part of.
I’m sure S4C have more detailed figures available to them but to publish them would be embarrassing as most people are pretty idiotic (like English newspapers) and would compare the channel to BBC One, which is not at all a fair comparison.
Mark, I studied that too. I left the post quite open because my memories of the proper statistical method are pretty vague now! I’d welcome any follow-up post by someone with the skills.
Liam, it’s the kind of debate you get around public service broadcasting – especially special interest or minority language programming. I think demand can be legitimately questioned, as it can be in Dafydd’s reference to BBC Three. Demand probably is going to be lower that that of mainstream programmes, which is why I thought it would be useful to question the accuracy of the figures.
Dafydd, I chose not to look at Welsh speakers. The audience for Sgorio can be segmented along various lines – according to Welsh learners, the curious, football fans, and of course the fluent.
In a time when people seem more open to subtitled foreign films, etc, and many (if not most) S4C programmes have English subtitles available – as well as bilingual marketing – I didn’t want to favour Welsh speakers as a special group.
I’m not sure if “most people are pretty idiotic”. Even if true, I’d rather not resign myself to it. It’s about having other sources of influence available besides the press, which is why I would encourage blogging about things like this in a fair-minded way as I’ve tried to do.
🙂