I've mentioned other times in this blog about the PISA test, a test conducted by the OECD every three years which tests 15-year-old students in various countries on reading, science, and math, with a special focus on one of the three categories each time the test is run. It has done this since 2000 and the results always create a lot of media frenzy in the West when countries figure out that their students scored worse than other countries. I expect a lot of education ministers to be fielding tough questions this week.
A lot of non-OECD countries also signed up for the test and Qatar signed up for the 2006 test, where they placed second last out of 57 countries in all three categories (beating out Kyrgyzstan in all three categories). The test continues to grow membership and for the 2009 test there were 65 countries or regions who signed up, including Qatar.
Well the results for the 2009 test were released today, this time focusing on reading but of course testing all three categories. How did Qatar do?
Reading: 61st (higher then Peru, Panama, Azerbaijan, and Kyrgyzstan)
Science: 61st (beating the same countries listed above)
Mathematics: 62nd (Azerbaijan pulled ahead in this category)
Now I was not expecting significant improvement from the 2006 test because educational reform takes a long time to see results. The kids who wrote this test would've been 12 years old back in 2006 so would already have been well through the educational system. But is Qatar making significant changes to its educational system? This is one of the wealthiest countries in the world and yet its students are clearly underperforming. Poorer Arab-speaking nations such as Jordan or Tunisia outperformed Qatar, and four South American countries also did better.
It does go to show that money does not necessarily buy a good educational system, many high-performing countries spent less per capita on education than high-spending countries such as Qatar or the United States. Thankfully the OECD also does a lot more analysis than just a simple score. Here are some highlights from the executive summary:
• students in urban schools perform better than students in rural schools, even if you factor in their social economic background
• children from single-parent households will score slightly lower on average than children from two-parent households (we are talking about 1% less)
• looks like your Mom was right, children who primarily read comic books do little better than children who don't do a lot of recreational reading at all. However, kids who do a lot of online research and reading do score better. Recreational reading from a wide variety of materials seems to be the best way to have a high reading score.
• Girls like to read more than boys and not surprisingly girls scored better than boys in all OECD countries -- and the gender gap is widening.
• Students tended to perform better on the test in schools that had greater autonomy over their curriculum and how students are tested
• higher teacher salaries, but not smaller class sizes, correlates with better performance on the test
• students tend to perform better in schools with a better disciplinary climate
Cool stuff. Keep an eye out for discussions on this test, and how well or poorly your country did, in your local papers.
(By the way, for some reason South Asian countries like India and Pakistan have not signed up for the test. I wish they would, if you're living in one of those countries, see if you can ask your education Minister why they are not signing up for the OECD test)
7 comments:
My main issue with the results are that bad schools are VERY bad, and the best schools in Qatar are simply "bad".
We really need better results.
Get a little more below the surface and check scores on each area in 2009 against those in 2006. The case is that although PISA focuses on one area in each cycle it tests all three on each application. Those results are comparable although not comprehensive. Gains between 2006 and 2009 for Qatar range from 30 to 50+ points. Reform only started in Qatar in 2003. Be patient!
In my defense, per my November 19 2009 post (last paragraph)
" The results of the 2009 PISA test are due out around March or April next year. I'm hopeful there will be a slight improvement but nothing significant, comprehensive educational reform will take many years. Any changes that were implemented by the Ministry since the 2006 test will take time to show up in the scoring. "
[wow was I ever wrong about when the results would be out]
I think one has to be careful about looking at the absolute score when comparing previous tests with the 2009 test because the questions are different. Can we conclude that Qatar has improved significantly if all of the countries near it also improved by around the same amount? That would mean the improvements from educational reform would be moot as anything the other countries did also made a difference.
I should also mention that if you do a search on my blog for PISA you will find a few entries. In one of them I summarize discussions I've had with Qataris about the educational system and the problems with it. Needless to say based on what the other Qataris have told me I do not think Osama Al-Assiry is being unfair in his criticism, there are significant issues that need to be addressed.
First, I never implied that Osama was wrong, I think that what he says makes a lot of sense. Second, my point was that comparison should not only be made with respecet to other countries within PISA 2009, but also looking at changes over time. Qatar participated both in 2006 and 2009. Third, I am not sure about what you mean of PISA 2009 results release being due in March, because the time elapsed between the 2006 and 2009 applications is almost exactly three years. In both cases results were due in December of the following year. You can have access to the 2006 and 2009 data at the OECD website and work out the comoparisons yourself. Finally, the issue of comparability between the PISA series goes beyond each test having the very same questions as the previous one because the way they are built allows for longitudinal comparisons. I suggest you wait for the release of the 2009 PISA Qatar National Report and there you will find many more precisions on this issue. I really value your discussions with locals about their educational system, the same I had during my six years living in Qatar, when I was in charge of the Qatar Educational Reform Assessment System. Anyhow, I just wanted to put things in a fair perspective. Enjoy Doha. It is worth.
[Third, I am not sure about what you mean of PISA 2009 results release being due in March, because the time elapsed between the 2006 and 2009 applications is almost exactly three years.]
That was a quote from my post of last year and yes I was way off when the results of the released --wish I could remember where I got that March/April date from.
[I was in charge of the Qatar Educational Reform Assessment System]
Wow, then you must have a lot more insight into this than I.
My December 10 post (today) was made before I had read your comment. What do you think? Am I totally off-base in my speculations about what might be some of the problems?
The greatest changes were in Turkey and Qatar where the average
science scores rose by 30 points - that is 1/2 a proficiency level. Not bad for only 3 years!
Post a Comment