Lies, damned lies, and statistics

Dear editor,

Peter Cowley asks for a discussion about the Foundation Skills Assessment 2009 Report Card for elementary schools in B.C. I submit that the report’s ranking system is misleading, that some of the differences attributed to small rural schools are questionable, and that standardized testing could be detrimental to good teaching.

From a person who has‚ “can teach high school statistics,” on his resume, standardizing test scores is a powerful statistical tool when samples are large, i.e. the writing scores for all fourth graders in B.C. We can converge data into a normal distribution around a reliable national mean. Schools that fall within the central cluster, called the standard deviation, are “less likely to show no differences” when comparing their results in say literacy. Not so for the schools that fall outside the cluster, though, and many rural schools tail at the end of the list. This has not stopped the Fraser Institute from publishing a rank. Parents might be puzzled by standard regression but they know a 6.7 is higher than a 3.6.

The Institute readily admits to the central limits in their report and counter that parents can still look for trends. Rural parents can compare similar regions of B.C. (Tom Fletcher, Feb. 3)

I looked at the fourth grade writing results among a few elementary schools in my region and found as much variety within schools as between schools. I performed a simple t-test on the five-year scores for Robert Scott and Sunset Elementary. An analogy is counting fish species each year in two different rivers. The results were not strong (p=.43171, df=8) and are more likely to show no differences between the two schools’ scores. A possible reason being a combination of small sample size, high variability, and bad luck.

Furthermore, the idea of a standard child is drawing us away from many present teaching practices which encourage individuality, discovery, inclusion, and skills. Why spend time having children do projects on topics they are passionate about, if the subject matter is not important on the exam? When standardized tests become “indicators of effective teaching” (Report, 2009) they also become incentives for teachers to teach to the test, or worse, to abstain certain students from taking the test.

Frederick Henderson

Port Hardy