Dec 23, 2008

Interpreting Statistics for our Education System

There is plenty of publicly available information about our education system. Interpreting that information, however, is not always easy. Furthermore our media frequently gets it wrong. Here's an example from TODAY:
Stumped by math, science at PSLE But Malays did better at‘A’ levels, statistics show
Tuesday • December 23, 2008

MALAY pupils continued to see a slight dip in performance at the Primary School Leaving Examination (PSLE) — especially in science and math — but showed a marked improvement at the GCE “A” levels as the revised curriculum took effect.

According to the Ministry of Education’s (MOE) annual report card, PSLE science has been a problem subject for Malay pupils, with pass rates going down from a high of 82 per cent in 2000 to 73.6 per cent last year.

It was the same for math, with just 56.8 per cent of Malay pupils passing the PSLE paper last year, compared with 64.6 per cent in 2000.

Chinese pupils had 89.8 per cent passes last year, down from 91.6 per cent in 1999, while Indians saw 74 per cent passes, a marginal fall from 74.7 per cent two years ago.

But Malay pupils made up for the slack at the lower level by registering their best performance in the “A” levels last year — 76.4 per cent getting at least 3A (advanced level) or H2 passes, as well as a pass in General Paper or the new subject, Knowledge and Inquiry — an 11.7 per cent improvement over 2000.
Here, the media is providing a snapshot of how the Malays are performing in the education system. The report says that Malay students continue to do worse and worse in maths and science at the PSLE level, but have shown very good improvement at the A-levels. So the reader's overall impression may be that the Malays aren't doing that well, but they aren't doing that badly either (because they managed to do a lot of "catch-up", by the A-level stage).

This conclusion is flawed. The simple reason is that the report says nothing about the percentage of Malay students who managed to qualify to take the A-levels at all.

If at the PSLE stage, the Malay students are already so far behind the other races, we know that the Malays must form a disproportionately large number of Singaporeans who drop out of school after the PSLE.

At the O-level stage, the gap widens even further. The latest 2007 statistics (available from the MOE) show that while 85.4% of the Chinese students achieved 5 or more O-level passes, only 59.4% of the Malays students managed to do the same.

In other words, once again a disproportionately large number of Malays will not even qualify for the A-levels (or for the polytechnics - see the cut-off points here as a rough guide). Instead many of these Malay students will have to drop out of school, or go to the Institutes of Technical Education.

As for the Malays who do enter junior college and take the A-levels, it is true that they do close the gap somewhat, relative to the other races. However, the majority of Malay students in each primary school cohort don't even get that far.

31 comments:

Anonymous said...

I suppose statistics is one thing that can be right or wrong, good or bad, if you put in the "right" context.

Million dollar minister salaries are not that high if you compared to the top 1% of earners.

Singapore's poorest are much better off than those in Africa.

When you prepare your job resume or do advertising of course must use some of these methods what, right?

So why should social or political aspects, which are more crucial for social stability or to "feel good" among those folks, be any different?

kate said...

Similar to "survivorship bias" we talk about when we look at the performance of unit trust.

From wikipedia:

In finance, Survivorship bias is the tendency for failed companies to be excluded from performance studies because they no longer exist. It often causes the results of studies to skew higher because only companies which were successful enough to survive until the end of the period are included.

Anonymous said...

What's the point of scoring high marks in school but spend 3000 when you earn 2000?

Anonymous said...

The problem with the data that you (or Today) provide is that it is NOT statistics. They are just numbers. Taking percentages is NOT statistics.

That, of course, does not mean that there can be no problems with data. As you rightly point out, MOE treats PSLE and A-levels as independent draws, which is hardly the case.

Oh well, its MOE. what can you expect...

keisimone said...

aka survivorship bias

http://en.wikipedia.org/wiki/Survivorship_bias

Anonymous said...

well.. data are just numbers manipulated to show what people want to show. i always get skeptical when i look at statistics after a year of sociology modules. hahahahha

Robert L said...

Come to think of it, the whole newspaper article is as good as shit and should be promptly flushed down the toilet.

They compare Malay results against 2000. Then they compare Chinese results against 1999. Whatzermatter? Can't get the results from the same year? Or are the data not convenient to suit the story?

Worse to come - then they compare Indian results with two years ago. Hello! That means 2005. What happened to 2000 or 1999? What's there to hide?

When people pluck bits and pieces from different pigeonholes to write their articles, there's just one thing to do. [FLUSH...]

kilroy said...

what can you expect...after all even when you pay for it ( the shitty times ) you are getting shit, so what else but more of the same esp when it is FOC!

korleur said...

Malay students dropping out of school after the PSLE? In Singapore education is compulsory till the secondary level, isn't it?

Mr Wang Says So said...

No, secondary education is not compulsory. Only primary education is compulsory. And even that rule has been around for a few years only.

Anonymous said...

I suppose statistics is the safest way for an administrator or politician to show that all the money being spent (and the salary our plitician is earning) are justified. Hpwever this onsession with statistic flies in the face of the official propaganda that those who buck the trend are what Singapore needs. In other wprds we are back once agin to cringing and hedging our bets. Talk about individualism but in practice let's flatten out that individualism towards a statistical mean.

Anonymous said...

Robert L - that's a good observation. I didn't notice the year of comparisons as there were so many figures around.

Seems like the writer already had an article in mind before selectively plucking the appropriate statistics to fit his story.

Shouldn't it be the other way round?

Anonymous said...

to anon 10.38am 24/12

This also the case when AOs are asked to put up policy papers for the PS, Cabinet etc...

Anonymous said...

You are right. Our clowns love to highlight and play with number. Am I surprised that misleading and misrepresentation information have been the cornerstone of our society because our "world-class" gov is the main perpetuator of such deceitful act ?

It is as good as saying that RI score much higher than other school but it did not mention that RI already screen off those weak students and select those best students perform who perform excellently in their study.

Anonymous said...

Utterly agree with Mr Wang's comments for once.

This is an utterly useless article.

What does it point to? Absolutely NOTHING.

Anonymous said...

Nice stats. However, no mention the percentage of students that need tuition in order to meet with our first world standards.

hugewhaleshark said...

Heh heh. You mean like when they said that the burden of GST falls more on the rich because rich people pay most of the GST collected?

Say it like it is so, and they will believe, so they think.

Anonymous said...

I'm wondering why are the media reports and govt policy in this "1st world" country still run on the racial divide? Thought we are supposed to be "One people, one nation, one Singapore"?

Anonymous said...

I am so ashamed to know that TODAY generates such stories.

Anonymous said...

and so the malays are doing less well academically? is there a systemic problem in our education? or a social problem in the malay community? or just a satistical flunctuation?

Robert L said...

[Preface: sorry, I have drafted my post before I came on site and see the post of 6:05 PM above.]

Dear Mr Wang, I have a new point to make about this matter, and I'll pose it as a question, because I don't have the facts to support it.

My question is this: is it possible that the Malay stream has students from Madrasah schools, which will make the Malay results not a fair comparison with English/Chinese/Indian streams which all follow the standard education syllabus?

I hope you or your other readers can enlighten us on this.

If exam results from Madrasah schools are included in the data for Malay stream, then I will leave it to you and other readers to assess whether the overall results will be pulled down, or up, or remain unaffected.

And further, if it is really true that the passing rate for Malay pupils has included those from Madrasah schools, then we have to admire how terribly clever the country's newspapers have been to avoid drawing this to our attention.

Amran Noordin said...

I think the MOE has always been playing with figures. They have always adopted the approach of telling the truth but not the whole truth (so help me God?). Not just the PSLE or GCE A, look at how the TIMSS thing has been used too. Read my blog for my views on how TIMSS has been used to tell the truth but not the whole truth.

Anonymous said...

Yup the title is stupid. Statistics shows what the owner of the statistics wants it to show. (eg. GDP growth)

If I remember correctly, I could count the number of malays with 1 hand when I was in JC, 10+ years ago.

Anonymous said...

It is same lah Wang.

You read the newspaper say SMU got 10 students with 4 A. But in actual fact, how many student got 4 A in the exams? I suspect more than 1,000. And their survey got 10 student with 10,000 a month salary, but the rest who never return survey are ignore. That is why sometime we at coffeeshop say SMU is Simply Made Up. Just like some reporters from you know where.

It is all about saying some, and forgetting some. Some reporters some times are just not the sharpest of the lot.

Anonymous said...

"MW: If at the PSLE stage, the Malay students are already so far behind the other races, we know that the Malays must form a disproportionately large number of Singaporeans who drop out of school after the PSLE."

I don't think there is a "disproportionately large number" of Malay students who drop out at PSLE level. How do you equate (Malay) students who don't score so well in Maths and Science with students who eventually drop out?? As far as I know, as a social service worker, there are a significant number of (Malay)students who did reasonably well in their PSLE to qualify for a secondary school in the Normal Academic stream. Most of them don't make it to secondary school for various factors such as financial difficulty, dysfunctional families etc. - Sophie

Mr Wang Says So said...

Sophie:

You can check the MOE website for the details.

The Malays beat the other races in Mother Tongue, by a very small margin. For every other subject (English, Maths & Science), the Malays underperform the other races. Especially for Maths and Science, the disparity in pass rates is quite large.

Now if Malays beat the other races only in Mother Tongue and only by a small margin; and underperform the other races in English by a larger margin; and underperform the other races in Maths and Science by an even larger margin, then ...

... the Malays must form a disproportionately large % of the PSLE failures.

And if you fail PSLE, you cannot go to secondary school.

Mr Wang Says So said...

Chart of overall PSLE pass rate:

http://www.moe.gov.sg/media/press/files/2008/12/performance-ethnic-groups-chart-a1.pdf

In 2007, 7 out of 100 Malays failed PSLE. 1 out of 100 Chinese failed PSLE.

So a disproportionately large number of Malays would not have been eligible for secondary school.

If you take the 10-year average (10 years being the entire period which the graph covers), you'll see that the annual average is as follows:

every year, 2 out of 100 Chinese fail PSLE and would not be eligible for secondary school;

whereas, 8 out 100 Malays fail PSLE and would not be eligible for secondayr school.

Anonymous said...

If this trend continues as Mr Wang had described, the Malays have been falling behind more and more for years, starting from the PSLE level, but they can always retake...eventually they should be able to pass?! This is really the bare minimum..

Anonymous said...

Even Business Times is careless and sometimes we suspect that our media uses statistics merely as a tool, but not necessary the truth.
HDB upgraders

Snippet of original article
"On the upside, the percentage of HDB upgraders continued to grow. In 2008, a higher proportion of purchasers with HDB addresses was registered with 37 per cent of all buyers expected to be HDB upgraders in 2008 compared to 22 per cent in 2007.

Based on available caveats in URA's REALIS, the number of buyers with HDB addresses in Q408 is 582. While this is a preliminary number, it represents 43 per cent of total caveats lodged so far in the fourth quarter. DTZ noted that this is higher than the 41 per cent in Q308, 36 per cent in Q208, and 28 per cent in Q108."

Anonymous said...

What's the point of compartmentalising academic achievement by race? Why are the MSM, and you, so hung up about it?

The main effect about banging on and on about how Malay students have been underperforming has been to ensure that they reinforce the stereotype, which is self-defeating.

There are more important and useful things for the Straits Times to discuss. Of course it seldom does so.

Mr Wang Says So said...

That is an interesting point for discussion.

It is the Education Ministry that has the numbers, and breaks it down by race. There is even a little piece of explanation given by the Education Ministry as to why it does this, year after year after year.

It is the MSM that systematically tries to portray some positive angle for the Malays, year after year after year. Example is the article that my main post discusses.