Oct 17, 2007

A Flawed Survey

ST Oct 17, 2007
80% of readers say ST is important to their lives
By Oo Gin Lee

……. Addressing the Forum writers in the auditorium at Singapore Press Holdings' Toa Payoh premises, ST editor Han Fook Kwang noted that a readership survey in April found that nearly eight in 10 of the paper's readers polled in face-to-face interviews considered it an 'important' or 'must-read'.

Aha. Here we see something known as “survivorship bias” at work. The flaw in the survey is that its sample population comprises only people who currently still read the Straits Times. Thus the survey excludes all those people who had already stopped reading the Straits Times precisely because they considered the Straits Times to be “unimportant” or “unnecessary”.

Survivorship bias is a concept often mentioned in the financial world, in relation to the performance of unit trusts and mutual funds. For example, a fund manager may claim that more than 75% of its funds have outperformed the industry average. This sounds like an impressive statistic - until you find out how many funds the manager had already shut down, precisely because they performed below the industry average.

We say that there is a "survivorship bias", because the "more than 75%" statistic is based only on the funds that still survive. Those funds which had already died from their own poor performance are conveniently dropped from the survey.

26 comments:

Anonymous said...

But it may not be "survivorship bias" if the total readership is reasonably large relative to the size of the population? Off the cuff, I have the impression that most households do subscribe to the straits times?

Anonymous said...

The sample size is even more skewed than what you suggested.

The people who attended the event were:

(1) INVITED into the event
(2) they ACCEPTED the invitation
(3) they were chosen because they were new LETTERWRITERS for the Forum page.

If you write letters to the ST, obviously you think ST is important enough to warrant your time and effort. And if you accept an invitation into a high profile publicity event, wouldn't you have something nice to say about your host? Can't be rude, can you?

Why ST even bother is beyond me.

Anonymous said...

My stats teacher used to remind us of the importance of sampling and making sure the intended purposes of the survey.

For example, if you were trying find out the extend of poverty in Singapore and you only sampled people who owned apartments in Orchard road.

Or how about trying to prove that few people have not mobile phone access by mobile phone survey.

Hmm.. what do you think the result will show?

See said...

http://theonlinecitizen.com/2007/10/02/uniquely-singapore-f1-or-f9-%e2%80%9cresidents-willing-to-pay-more-for-service-and-conservancy%e2%80%9d/#more-517

In a report titled “Punggol 21 Plus masterplan is a long-term one: Grace Fu” on the 15th of September 2007, Channelnewsasia website stated:

“But according to feedback to HDB, 81 percent of residents said they were willing to pay more for service and conservancy to enjoy the new flat designs. About half of them said they were willing to pay above S$10 more than the usual rates.”

alex said...

IMHO, the ST is VERY IMPORTANT

i always use it for doing area cleaning. wiping the bunks' windows uses a lot of newspapers, especially so when u have to do it everyday.

no other papers offers more value than the Straits Times!!

Anders Brink said...

It is a bias, but calling it survivorship bias does not make sense. For a fund manager, closing down underperforming funds is a key skill. For the ST, they simply polled a biased population that is prone to self-reinforcement.

debbie said...

very true and enlightening. couldnt agree more.

Anonymous said...

I was at GV watching a movie once and they played a ST advertisement before the movie. The audience burst out laughing and jeering when the advertisement went on about something being a part of every singaporean's life.

Anonymous said...

Yeah it's very important to me. I have terminated my subscription. Hahahaha.

Anonymous said...

I've stop reading the ST for the pass 4 years,5 years.. can't even recall when I've stop buying/reading it.

Anonymous said...

Is the survey flawed? I think not.

It was probably crafted with an end in mind - to reinforce a desired conclusion, as opposed to discovering reality.

After all, it's all about reality and perception, isn't it? Wang San.

jc

Blogter said...

"The sample size is even more skewed than what you suggested.

The people who attended the event were:

(1) INVITED into the event
(2) they ACCEPTED the invitation
(3) they were chosen because they were new LETTERWRITERS for the Forum page.

If you write letters to the ST, obviously you think ST is important enough to warrant your time and effort. And if you accept an invitation into a high profile publicity event, wouldn't you have something nice to say about your host? Can't be rude, can you?"

Couldn't agree more. On top of that, IF it was a face-to-face interview, so if it was conducted by a ST interviewer, it would be even more difficult to say to his/her face that the ST sucks.

Perhaps not that many people subsribe to ST in the first place. Or not as many as would be expected of the only main paper in Singapore. Otherwise he could just have pointed to the high subscription rate.

lau min-tsek said...

"Off the cuff, I have the impression that most households do subscribe to the straits times?"

The article suggest that the question asked is whether the ST is "important" to their lives.

It does not ask if they think highly of the journalistic standards or any harder hitting questions, like independence of the editorial articles etc. Nor did it define what "important" means.

Eg, I may read the ST for the business news since I work in a bank. But I may think the world news is crap and skip it altogether. I read IHT for that. Is ST important to me. Yup - for work. But not for other things.

Actually, the results reflects BADLY of ST, IMO.

Why?

Think about it. For an INVITED event of (I assume) pro-ST letterwriters that probably has been pre-screened by ST, 20% do not think ST is "important to their lives".

That's a SHOCKING ONE out of FIVE who thinks that ST is, what..... unimportant?

Wow! One out of five of your INVITED guess has a low opinion of you! And they say it in your own publicity event!

What does this say?

Anonymous said...

Oh yes, that reminds me of a survey done when Mediacorp and now defunct SPH channels were competing with each other and each of their so called media watchdogs doing viewership surveys - well, they had very different conclusions about the channel viewership numbers depending on who hired them.

In any survey their is an element of bias. That is why some industries can survive eg the tobacco industry because they hired people to cover up the dirt rather than expose the dangers. When people get addicted, nothing can change. We are now just beginning to realise the effects of plastic use on our health, when all these dangers were brushed aside in the initial years so that the plastic industy could flourish. Same with telfon and margarine. Skewed surveys were done to con the public into accepting the products.

Anonymous said...

up next.... ....

ST surveys ST subscribers... ... says 100% of surveyees in singapores are subscribers to ST.

Mr Wang Says So said...

Actually:

(1) there was an event to which some readers were invited;

and

(2) there was a survey done among ST readers

but the article does not say that those who were surveyed were the ones who were invited ...

Anonymous said...

I was one of those people invited to the Forum event. Contrary to what some people have posted here, the people invited were NOT the ones polled. Kindly stop spreading rumours.

Kaffein said...

Totally true. I've stopped reading anyway. So how to do the sampling?

Totally skewed and screwed, if you ask me. I think ST is trying to justify their existence and make a statement in the recent highlight that ST ranked so low among the other developed countries newspapers.

Kaffein

Anonymous said...

Exactly!

So, invited guests were not polled. Then, who were polled? Who are the readers in the survey? When you pushed a number out, you got to have the data.

If there are no real data, then the number seems to be plucked out from thin air. Was the survey done by ST staff or an independent entity?

Even this independent entity need to be really independent, if you know what I mean. Not some company belonging to some BIG parent company.

crabbybrat said...

Thank you for putting this up, now I know why I felt so uncomfortable about that writeup.

Any publication that is not important to the reader will be dropped anyway. This survey isn't even valid.

Aristocrat said...

They obviously forgot to poll me.

Anonymous said...

Imagine you're in a communist country. ST is the only main paper, used by communist leaders to push their agendas. Would the ST be important? Yes, so you would know what the leaders expect of you.

playtime said...

Actually, I do find the straits times important for my news needs... but only for foreign news. I belive many who buy it feel the same.

If its singapore related news, i'd rather get it for other sources since mush of whats in the ST is obviously biased.

The right questions to ask readers will be.

1) Will you switch if another paper is available?
2)Without foreign news, will you still read it?

Anonymous said...

uh current readership of straits times hovers around 950,000, it has been decreasing for a few years already

Anonymous said...

I was in Singapore recently and was appalled at the standard of reporting of the ST. Still very much a pap government propaganda mouth-piece with a little more sophistications. No wonder the average Singaporeans are so ill inform about the fast changing world around them. My recommendations to Singaporeans is to avoid reading the local papers. Get your news free on the net and I am confident u will be better inform.

LuvMyCountryLicks said...

May I ask around - what is the best way to syndicate news..? RSS? Which site/s are the best for local, regional and global news.