Tuesday, January 03, 2012

Further Discussion on the India Pisa Results

Someone has posted a nice detailed comment in response to my recent post on PISA results for India. Because of the length of the comment rather than answer it in the comments section I figured I would reproduce it here:

The Indian results are unsurprising because it corroborates earlier studies all coming to a broadly similar conclusion. That Indian education is mediocre and overall level of human capital in India is very low. Please refer to the TIMMS test administered by Jishnu Das and Tristan Zajonc or the Wipro educational initiative results.

Your skepticism is unfortunately untempered by rationality. To expect high Indian results based from the selection of the highest scoring school in Qatar is the result of several cognitive biases. Chiefly there is the issue of selection bias, by comparing the most unrepresentative school in the nation with OECD averages. To apply your logic, I can pick the highest scoring school in the US or China in the PISA tests and it will make even the highest scoring Indian (Qatar) students, look like drooling idiots, relatively speaking.

For example Al Khur Interntational school only admits the children of the employees the state LNG monopoly. These aren't just the children of affluent Indians, these are the children of Indians that are, probably, greater than two standard deviations above the socio-economic mean of India. To be that selective in the US, you would have to administer such a test to only the children of Harvard, Yale, Princeton, MIT, etc alumni.

In fact, the Qatar results for Indian students, rather than showing the possibility of high Indian achievement actually comes to the opposite conclusion. That even a highly selected group of Indian students are just above the OECD mean (itself dragged down by relatively poor performing OECD nations). What does that say about the rest of India's human capital +/- one standard deviation from their mean? Nothing positive.

Thank you for the comment. I think you may be misinterpreting the background that led me to focus specifically on India and PISA. Throughout my life I've met many people from South Asia, either in Canada or here, and there always seemed to be a heavy focus by the families on education and there seemed to be more pressure on the children to study hard and do well in school than for most other kids. Documentaries on Indian education seemed to be about the competitive nature of the education system, typically focusing on teenagers spending long hours studying for various exams, or entrance tests to IIT, or whatever. Many South Asian schools here in Qatar did very well on the PISA tests compared to peers. The anecdotal evidence all pointed to India being able to post scores on PISA significantly higher than would be expected given its poorer economic situation than the OECD nations (on a per capita basis).

The problem was that the evidence was by and large anecdotal so could not be used to definitively make conclusions and I have never traveled in South Asia to see things for myself. That's why I was posting about how I wished South Asian countries would participate in PISA -- to see if a study on a large representative sample would support the conclusions that the anecdotal evidence indicated. It turns out it did not, but given all I had before was the anecdotal evidence that is why the PISA results for India were “below my expectations”.

I never stated that Al Khur Interntational achieving good results must mean that India has an amazing educational system, more akin to “Al Khur Interntational got good results, I’m curious as to how India would score generally on PISA”.
As it stands, what are the weaknesses in the Indian system: that society is not as education-focused as the anecdotal evidence led me to believe? Severe underfunding by the various levels of government? Too many children overwhelming the current educational resources? Poor pay for teachers compared to other fields? I was just curious as to what people from South Asia think the issues are.


Anonymous said...

Hey I appreciate my comment getting it's own post. To respond, the biggest problem as I mentioned is sampling bias from which you are gathering your anecdotal information. The only South Asians you are familiar with are those who have emigrated for professional reasons to Canada or Qatar. This is an extremely self-selected and unrepresentative sample.

Secondly, you specifically mention "families", which is even more selective of particular traits especially in Qatar. Living in one of the Gulf States, you cannot but be aware of the numerous single young men and women from the subcontinent that constitute the majority of the lower skilled workforce in the region that often have families and dependents back home. The only South Asians with school aged children working in Qatar are going to be the anglophone professionals with required skills, and not the construction workers and sales girls that actually constitute the majority immigrant workers.

Thirdly, a focus on education doesn't necessarily mean a successful education. Surveys in the U.S. also show that black and hispanic parents emphasize education as a tool of advancement, yet performance doesn't necessarily match promise. An anecdote of this was in an interview with Amy Chua who ounce noticed an immigrant mother trying to help her child with homework on a bus, the irony was that Chua didn't have the heart to tell them that answers the mother was trying to provide were wrong.

Fourthly documentaries are not evidence of anything but the interests and agendas of documentarists and journalists. Case in point, last year there was an effusive article about middle class education in India focusing on a teenage girl and her family. Unfortunately, most people simply lack the intelligence to read or listen critically and to synthesize information from sources beyond the obviously evident. The greatest flaw in the article was that it wasn't even about India's middle class. The NYT article included a video interview with the mother and scenes from school. What first jumped out at me was that the language of instruction used in the equivalent of a high school was English, not altogether unexpected but still uncommon even in India. Second, I noticed that one of the instructors was using a blue tooth headset. This raised another immediate flag. The final flag was the interview conducted with the girl's mother. The problem is that people are naturally prone to logical lapses and you have to train yourself to recognize them and counteract them. The great mistake that the reporter made was that because the teenage girl's life was so similar to her own experiences, that it was applicable to everyone. That middle class life style lived by the interviews that is instantly recognizable from someone in the West isn't middle class at all in India. It is a lifestyle of great privilege and rarity. The school was obviously not one of India's dysfunctional public government schools but a private one. No mere private one at that, but an extremely expensive one judging by the condition of the classroom and the teaching aids used by the instructors. The final identifier was the interview with the mother which was conducted in fluent English without a translator. But everyone in India speaks English you say! No, the mother was I would guess in her late 40's possibly early 50's meaning a very late pregnancy for a daughter of her age, rare in India. That the mother was fluent in English would have been a sign to an astute observer that this woman was exceedingly unrepresentative. The logic is simple. Women of her age cohort born in India, possibly sometime in the late 50's early 60's, would much more likely be illiterate than not (4 to 1 odds). That not only was she obviously literate and literate in the elite language of a society exhibiting diglossia meant that mother herself was a child of even more privilege for her time than the daughter!

To be continued...

Anonymous said...

In conclusion, a focus on education does not necessarily mean anything unless measurable deliverables are achieved, which they are not in India. Funding is not necessarily the problem in Indian schools as per capita education expenditures as a percentage of GDP is fairly solid, the problem is that much of it is being wasted by the government. Teacher incentives are low, resulting in high absenteeism. On any given day, anywhere between 15-40% depending on the state, do not even bother to show up to teach the class. Teacher credentialism is another problem, the existing quality of teachers is extremely low and the shortage of "skilled" labor means that anyone with the aptitude and capability to become a good teacher will seek employment elsewhere. Of course corruption too is a major problem, as elsewhere in Indian society. The worst example of this is the occassional school that exist on paper and payrolls that just aren't there in the real world.

There are also serious problems the way learning is conducted in India. It may seem redundant to say this, but there is an over-reliance on memorization and rote learning over analytical skills. Teachers and educational bureaucrats may bemoan this the world over and in many instances its simply a "grass is greener" phenomenon but it India's case it is probably the most apt. The PISA questions themselves cannot be answered by rote learning (What's the capital of Egypt? How many teeth does an adult human have?) but rather they focus fairly heavily on reasoning ability. You can find some sample PISA questions online and if you work through them yourselves and chart the steps of your reasoning you'll find that they don't require any esoteric facts imparted by schooling but rather some elementary logic and analysis. This is essential in order to make the test universal and applicable to students from 74 rather disparate countries.

Glen McKay said...

Thanks again for your detailed analysis and insight into the educational system in India.

True, a focus on education doesn't necessarily mean a successful education but it can certainly play a role, “one piece of the puzzle” is a phrase that comes to mind. A balance does need to be struck between parental monitoring of their child's education and pushing children too hard but I think positive parental involvement is still a factor. It also should not entail doing the child's worker or giving them the answers, which might have been what was happening in the Amy Chua example.

The US is a very tricky country to analyze in terms of educational performance, especially for minorities. One of the criticisms of the US educational system is how it is primarily municipally-funded so poor neighborhoods wind up with poor schools. Jonathan Kozol comes to mind as a writer who heavily criticizes the US system on these grounds. PISA does do some analysis of income levels versus results of the test and the differences can vary widely country-to-country (I only remember this because I believe in the 2006 test Canada had one of the lowest income vs score variances in the OECD, and this was highlighted in the PISA report).

What is most intriguing is issues you've highlighted in regards to the Indian educational system. It is a shame to hear that there are issues with regards to quality of teachers, incentivizing teachers, and corruption. Given India's size it seems like a Herculean task to be able to fix so I doubt there will be significant improvements in the PISA scores within the next decade. I also agree that a system with a focus on rote memorization of facts will not prepare students well for the PISA test.

Again, thanks for your input.

rec1man said...

India has 3 main racial groups

Aryan, which is Indo-European, and Y-Haplogroup R1A ( Eastern Europe ), 25% ( assume 100 IQ, same as white )

Low caste North Indian , Y-Haplogroup H1 ( gypsy ), 50% ( IQ measured in Netherlands as 88 IQ )

Low caste South Indian Dravidian, 25%
( possibly 92 IQ )

The low castes, H1 , L1 have gotten Affirmative action for 60 years and still lag behind and a clear sign of low genetic intelligence.

So, 75% of the Indian population is low IQ.

Next, to try to close the gap, many of the Indian states promote rote-learning, so that the quota candidates, with effort can score high and reduce the gap.

PISA requires COMPREHENSION, not rote memorising. So the PISA scores would be low due to bad teaching methods

Most of the Indians you have met are upper castes, with IQ in the white range.

The current chess champion is Vishy Anand, a brahmin upper caste, Y-Haplogroup R1A. So the higher ends of the upper castes are very competitive with white Europeans.

In the International Math Olympiads, India scores in the middle of the white countries. But all these medalists are upper castes, genetically related to Europeans

rec1man said...

Calculating Average Indian IQ from PISA

TN raw math PISA score = 351
TN implied IQ = 100 - 1.5 x 15 = 78

HP raw math score = 338
HP implied IQ = 100 - 1.62 x 15 = 76

Indian Avg IQ based on raw IQ = 77


Next step is to remove the bias caused by the PISA sample having
75% bilingual kids

TN mono-lingual = 378
Implied IQ = 500 - 1.22 x 15 = 82

HP mono-lingual = 401
Implied IQ = 500 - 15 = 85


Next there is a 40 point difference between scores for 'Village' and scores for 'Large city'
In HP and TN, The village category is over-represented by a factor of 4
Even worse, in HP, City and Large City are entirely removed from the survey sample

So adding an urban correction of 20 ( half the village-large city difference )

TN semi-urbanised mono-lingual = 378 + 20 = 398
Implied IQ = 85

HP semi-urbanised mono-lingual = 401 + 20 = 421
Implied IQ = 100 - 0.79 x 15 = 88

Current Indian IQ = 86


Next we look to the future as malnutrition is removed

The only Indian kids who go to govt school is for the mid-day meal,

If they are not starving they go to private school

Private schools score 45 more than govt schools and thats the future as poverty reduces


HP - future - semi-urbanised- mono-lingual = 401 + 20 + 45 = 466

Implied IQ = 95

TN - future - semi-urbanised - mono-lingual = 378 + 20 + 45 = 443

Implied IQ = 91

Future Indian IQ = 93

Given the huge bias in sampling towards over-representing the lower end IQ,
by the poverty pimp NGOs, I am certain that none of the CBSE or Cambridge schools
that serve the top 15% are included in the survey

And they have an entirely different IQ profile and cause an IQ bulge at the top end