Philadelphia Metropolis


Lies, Damned Lies and...

| Comments

1862306-387873-student-filling-out-answers-to-a-test-with-a-pencil.jpgMark Twain once observed that there are three kinds of lies:  lies, damned lies and statistics.

Keep that saying in mind as the state investigation into tainted PSSA scores continues.  These are the standardized tests used to measure student performance in public and charter schools statewide.

As Kristen Graham and Dylan Purcell reported in the Inquirer Sunday, the state investigation is not complete - but it is complete enough to implicate a number of principals, assistant principals and some teachers in cheating.

A number of districts around the state remain under scrutiny, including Philadelphia.  Who did it and how widespread the practice was is still an open question, but it looks like the answers will be: a lot of people and very widespread.

Under pressure to improve test scores, these educators decided to finagle. An eraser usually was the instrument of crime, as they wiped out wrong answers and penciled in correct ones.  In some instances, according to the Inquirer report, they would interrupt the test as it was taken to "review" material that was the subject of the next cluster of questions.


We will know the true effect when the scores are released in the fall.  The state cracked down on the districts under suspicion, took over administration of the tests, and this presumably will generate more accurate results.

Graham and Purcell deserve great credit for breaking this story, based upon complaints from teachers at one middle school, and following it up the line. So does Dale Mezzacappa, at the Public Schools Notebook, pressured state officials successfully to release the results of their 2009 analysis of potential cheaters.

The state Department of Education deserves credit, too.  It conducted a sophisticated analysis of tests results that allowed investigators to look at outliers - schools or tests where the erasures exceeded the average.  It even engaged forensic analysts to look at whether those penciled-in circles were done by the same hand.

They have rounded up suspected principals and others and brought them in for questioning.  A number of them have admitted to tampering with the tests.  (Rigging PSSA results was not a crime - until this year when the state used the threat of criminal charges to out offenders.)

The bottom line: after years of rising, look for PSSA scores to fall in certain districts, including Philadelphia.

In my mind, that is an indicator of how widespread the cheating was.  It is one thing to make your school's numbers rise.  To make a whole district's numbers rise means the cheating has to involve a number of schools.

For me, this scandal may answer an enduring mystery on performance in the public schools in Philadelphia.  While PSSA scores have risen consistently over the last decade, by other measures the district has failed to show improvement.

An example is the National Assessment of Educational Progress (NAEP), a U.S. Department of Education test given to students in fourth and eighth grade across the country.

While the PSSA was more extensive - it involves all students in certain grades - the NAEP uses a sample of students.  In 2011, it tested 1,600 of 12,000 fourth graders in the district and 1,200 of 10,000 eighth graders.

While the PSSA's told one story - that of increasing proficiency among Philadelphia students, the NAEP (so many acronyms, so little time) told another one. 
In the NAEP test, Philadelphia students scored lower than the average in other comparable large cities.  And, in the NAEP, a smaller percentage of students were rated as proficient in reading and math than in the PSSAs.

How do you explain the difference: stellar performance in the PSSA, less than stellar in the NAEP?  From a reporter's standpoint, this is statistical swamp you could get lost in - comparing different tests, different norms, different sample sizes.

The investigation may cut through this Gordian knot and supply an answer:  Philadelphia students were not really improving their performance.  Their elders were cheating so it looked like they were.

If that is the answer, then it puts to lie the much-touted improvement in academic performance in the city's public schools.

It also is an indictment of the district's top-down Soviet-style method of management.  The commissar (say, Arlene Ackerman) demands an increase in agricultural production. The apparatchiks in the field deliver it, by producing data that shows an increase in crop yield. The objective is achieved. Victory is declared.

Meanwhile, the peasants starve.

-- Tom Ferrick









blog comments powered by Disqus
Site by