{"id":8235,"date":"2012-06-04T14:18:32","date_gmt":"2012-06-04T18:18:32","guid":{"rendered":"http:\/\/stateimpact.npr.org\/florida\/?p=8235"},"modified":"2012-06-04T17:21:50","modified_gmt":"2012-06-04T21:21:50","slug":"inside-a-scoring-center-in-the-standardized-testing-industry","status":"publish","type":"post","link":"https:\/\/stateimpact.npr.org\/florida\/2012\/06\/04\/inside-a-scoring-center-in-the-standardized-testing-industry\/","title":{"rendered":"Inside a ‘Scoring Center’ in the Standardized Testing Industry"},"content":{"rendered":"
\"\"<\/a>

<\/p>

We speak to the author of Making the Grades about how standardized tests scorers grade essays.<\/p><\/div>\n

Some educators have always been skeptical of standardized testing.<\/p>\n

But in Florida, educators got another reason<\/a> to question the accuracy of our test after the Florida Board of Education lowered the passing score of the Florida Comprehensive Assessment Test writing exam so more students would pass it<\/a>.<\/p>\n

A former employee with Pearson, the company that created and scores the FCAT, says the people who score the essays also lower their grading standards to produce the test results states predict their students will get,\u00a0though Pearson denies the practice.<\/p>\n

We spoke to Todd Farley<\/a>, author of the book Making the Grades: My Misadventures in the Standardized Testing Industry.<\/p>\n\n

<\/p>\n

Farley was a test scorer with Pearson from 1994-2000 and he trained scorers and managed project in\u00a0the standardized testing industry until 2008.<\/p>\n

And he tells us what it was like to score standardized tests.<\/p>\n

Q:\u00a0<\/span>Do you think standardized tests produce accurate results?<\/p>\n

A: <\/span>No, the conclusion of my book, what I wrote was, I think what the standardized testing industry produces are numbers. So I’m never surprised to hear numbers went up or went down, because that’s all we did when I was in the business. There were innumerable instances when we were scoring, half way through the project you would go, we have a tremendous number of lower half scores and not enough upper half. So I would stand in front of a group and I would go, ‘hey all that stuff I’ve been telling you for two weeks, lets just forget that and let’s give more upper half scores.’ And all the scorers would moan and complain and I would think it was a scam and they would think it was a scam and then we would do it. Because every one of us was in there to get paid.<\/p>\n

Q: <\/span>But Pearson did give evaluations. There were tests for the scorers that you guys had to take in order to be considered qualified to be a scorer, correct?<\/p>\n

A: <\/span>Absolutely, qualifying tests, you bet. You didn’t have to do it for everything, you didn’t have to do it for the reading test necessarily or science test, but for the writing test you almost always did and I always tell the story, the first writing project I ever worked on, I was just a scorer, I wanted to work on this project. There were two qualifying tests to take. I failed them both and was fired. I would say half of the hundred people in the room were fired and we walked out in shame. Literally in shame. And then the next morning, at 7:30 am they called me back and gave me the job again. So what they had done was just lower the qualifying score. So suddenly all of us who had failed had passed. Because they hadn’t had enough people to finish the project and they needed to get them done, so there, they solved that problem. Seemingly the same thing as your state has done this year with the students.<\/p>\n

Q:<\/span> And again, you were an employee with Pearson, you no longer are. I spoke to Adam Gaber who is with Pearson now and he said in an email that “the scores of the Florida writing exam are accurate, valid and reliable.” The Florida Department of Education agrees.\u00a0Let\u2019s talk a little about how tests are graded. The multiple choice test are scored electronically so there\u2019s probably not too much room for error there. But what about the open ended questions? The essays. Is there a standardized way to grade writing?<\/p>\n

A: <\/span>There are lots and lots of variables that decide scores other than just writing ability. Again as a human, you get there at 8:30 in the morning, maybe your coffee hasn\u2019t kicked in, maybe you\u2019re in a bad mood. You aren\u2019t quite as generous as you are 5 minutes before lunch. Or maybe you\u2019re more generous than you would be at 4:25 when your eyes are twirling around your head because you\u2019ve read 250 essays. Secondly, it matters when in the project it gets scored. Because like I said, every project begins with the best intentions. And we\u2019re all trying to do the best job. And then three weeks later someone is screaming at us: \u201cgive more higher level scores. So you might be giving threes to things you would be giving twos, two weeks before.<\/p>\n

Pearson spokesperson Adam Gaber said in an email that it is “completely untrue” the company lowered the qualifying test scores for their scorers.<\/p>\n

“All potential scorers undertake a very rigorous qualifying and interviewing process,” Gaber wrote.<\/p>\n

Pearson: How Florida test scorers are trained<\/strong><\/p>\n

“Once scorers are hired,\u00a0based on the hiring guidelines outlined by FDOE, scorers attend 3 to 4 days of training, practice scoring, and qualification,” Gaber wrote.<\/p>\n

He says scorers first grade practice tests, which are then reviewed by Pearson.<\/p>\n

“Before scorers can move to live scoring, they need to qualify on the item. The high qualification standards and criteria\u00a0set by FDOE\u00a0must be met before a scorer can score a live student response.”<\/p>\n","protected":false},"excerpt":{"rendered":"

Some educators have always been skeptical of standardized testing. But in Florida, educators got another reason to question the accuracy of our test after the Florida Board of Education lowered the passing score of the Florida Comprehensive Assessment Test writing exam so more students would pass it. A former employee with Pearson, the company that […]<\/p>\n","protected":false},"author":32,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[18],"tags":[1019,128,1006],"acf":[],"_links":{"self":[{"href":"https:\/\/stateimpact.npr.org\/florida\/wp-json\/wp\/v2\/posts\/8235"}],"collection":[{"href":"https:\/\/stateimpact.npr.org\/florida\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stateimpact.npr.org\/florida\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stateimpact.npr.org\/florida\/wp-json\/wp\/v2\/users\/32"}],"replies":[{"embeddable":true,"href":"https:\/\/stateimpact.npr.org\/florida\/wp-json\/wp\/v2\/comments?post=8235"}],"version-history":[{"count":1,"href":"https:\/\/stateimpact.npr.org\/florida\/wp-json\/wp\/v2\/posts\/8235\/revisions"}],"predecessor-version":[{"id":8287,"href":"https:\/\/stateimpact.npr.org\/florida\/wp-json\/wp\/v2\/posts\/8235\/revisions\/8287"}],"wp:attachment":[{"href":"https:\/\/stateimpact.npr.org\/florida\/wp-json\/wp\/v2\/media?parent=8235"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stateimpact.npr.org\/florida\/wp-json\/wp\/v2\/categories?post=8235"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stateimpact.npr.org\/florida\/wp-json\/wp\/v2\/tags?post=8235"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}