10 Oct 2006

WASL-Wikiality: Sounds Like a Job for Mythbusters #

Wikipedia is an online encyclopedia where truth is determined democratically. If enough users agree that something is true, it becomes fact. Stephen Colbert of Comedy Central’s Colbert Report has coined the term Wiki-ality to describe the reality defined by such a process. This article will attempt to show that the WASL is based on Wiki-ality and does not stand up to investigation. To do this we will rely on the methods of another cable TV show, MYTHBUSTERS, wherein hosts Jamie and Adam use actual research to show what’s real and what’s fiction.

MYTH 1 #

International test scores predict the future economic position and quality of life of nations. The U.S. does not score at the top on such tests and is, therefore, losing its economic and social advantage.

What does the evidence tell us?

The First International Mathematics Study (FIMS 1964) was the first international study to show U.S. students in the middle of the pack in math scores. Forty years and several similar test results later, the Gross Domestic Product of nations above our students on FIMS was only 73% of that of the U.S. The economic growth rate of the U.S. was half-again as high as that of the high-scoring nations and the Quality of Life Index is lower for countries that scored higher on the FIMS. From 1950 to 2006 Americans have won 58% of the Nobel Prizes awarded in Medicine, Physics and Chemistry.**

International test scores DO NOT predict national success! MYTH BUSTED!

MYTH 2: The WASL is a valid and reliable instrument to measure student academic achievement. #

In a paper titled “Validity Evidence for WASL Reading and Mathematics Performance Standards,” Duncan MacQuarrie, Manager of Student Assessment and Joe Willhoft, Executive Director, Research & Evaluation for the Tacoma Public Schools report data that raise questions about the performance standards in reading and math at all school levels. “The reading standard at 7th grade deviates markedly from those at 4th and 10th grade. The mathematics performance standards are consistently difficult across the grade levels. However, the difficulty of the mathematics standards at grades 4 and 10 is much greater than is the difficulty of the standards for reading at those grades. These patterns raise additional questions about the reasonableness of the WASL performance standards.”

Similarly, Donald Orlich of WSU analyzed the 2004 Grade 5 Science WASL and the Grade 7 Mathematics WASL using criteria from developmental psychology and the scales of the National Assessment of Educational Progress (NAEP). His findings included the following:

The Grade 5 Science WASL exceeds the intellectual level of the vast majority of 5th grade children, meaning that their brain structures are not yet ready to handle the tasks required on the test.
The 7th grade Mathematics WASL is actually based on 9th grade work and has the same developmental difficulties as the 5th grade Science WASL.
His conclusions regarding the effect of testing children using these inappropriate tests include: “First, if the WASL tests are advanced beyond the mental cognition of grade 5 and 7 pupils, then for most children failure will be the ultimate end, regardless of instructional techniques used. Second, what psychological impact will failing an inappropriate science and math WASL have on students and their ultimate attitudes towards science and math, and schooling in general?”

Finally, according to an investigative report in the Snohomish County Tribune in September 2006, there is no record that the validity of the state standards have been examined.

Regarding reliability, scoring errors have been found in virtually all high-stakes testing around the country including here in Washington. Questions with no correct answer or more than one correct answer are among the more common slip-ups. Since many questions on the WASL require extended written answers and the papers are scored by people often thousands of miles away, the consistency of scoring by different individuals comes into question.

The evidence indicates that performance standards and/or WASL items at 4th, 5th, 7th, and 10th grades are not valid for the age group tested. Reported scoring errors and questionable inter-rater consistency cast a cloud on the reliability of the scores students receive. MYTH BUSTED