Maria Plecnik is the kind of teacher who gets chills in a 90-degree classroom when she connects with students during the first week of school. She’s the kind who brags about seeing their test scores go up or turning a kid who was always trouble into an “A” student.
In her seven years at Euclid’s Forest Park Middle School, her principal always told her she was doing a good job.
Teaching was her dream job. But this year, her dream faded.
Ohio is introducing a new way of grading teachers, one based on student test scores and whether students learn as much as expected in a given year. Under that system, Plecnik received the lowest rating, “Least Effective.” She cried when she found out.
This series about valued-added, a new way that Ohio is using to measure whether teachers provide a year’s worth of learning to their students, is the result of a partnership between The Cleveland Plain Dealer and StateImpact Ohio. StateImpact reporters Molly Bloom and Ida Lieszkovszky worked with Plain Dealer reporter Patrick O’Donnell and Plain Dealer data analysis editor Rich Exner to produce these stories.
- Overview: Using Data To Evaluate Teachers
- Pay vs. Value-Added Performance
- Secrets Of Two “Most Effective” Teachers
- Value-Added’s Poverty Factor
- How is Value-Added Calculated?
- Audio: Measuring Performance Through Growth
- Audio: Push for Performance Pay
- Video: Guide to Ohio’s New Way of Evaluating Teachers
Plecnik’s experience puts her in the middle of one of the biggest transformations to hit the classroom in the past 50 years.
Teacher evaluations, once based largely on a brief classroom visit from a principal every few years, will change dramatically next school year.
Ohio is one of 32 states experimenting with more rigorous, data-driven systems of grading teachers. In Ohio’s case, teachers earn one of five ratings that are a key part of their overall “final grades.” Those grades range from “Most Effective” at the top end of the scale to the “Least Effective” rating Plecnik received.
The rating is based on a statistical measure called “value-added.”
Most teachers — about 70 percent — fall into the middle, average categories. The state started calculating value-added for some teachers in 2010-11 as part of a pilot program and is phasing in the ratings.
What’s riding on these grades varies depending on how school districts choose to use them. But more than a teacher’s pride is at stake. The ratings could eventually become part of decisions about how much teachers are paid, what classes they teach and, if a district has to lay off teachers, their place on the list of who stays and who goes.
Many policymakers view this data-driven approach to sizing up teacher performance as crucial to weeding out bad teachers and rewarding good ones. But some teachers see the measure as a flawed attempt at quantifying something that isn’t easily quantifiable.
The scores offer a new view into what’s going on inside Ohio classrooms. StateImpact Ohio and The Plain Dealer obtained the value-added scores and ratings of about 16,000 teachers across Ohio in reporting this series.
Managing by the numbers
Policymakers love value-added because it picks up on two trends in education policy.
The first is the push to quantify every aspect of how schools operate and hold schools accountable for showing student progress through test scores.
The second revolves around research suggesting that having a great teacher is the most important factor in determining whether students succeed in school. That research suggests it’s crucial to find ways to tell great teachers apart from less-than-great teachers.
That’s not something that most current teacher evaluation systems do well. A 2009 study of a diverse group of districts in four states, including Ohio, found that less than 1 percent of teachers received unsatisfactory ratings, even in chronically low-performing schools.
With better measures of teacher quality in place, supporters say, the best teachers can be encouraged to continue teaching by offering them bigger raises, recognition and extra perks like more planning time. The worst teachers can be given intensive training. And if they don’t improve, they can be encouraged to leave the teaching profession through the stigma of receiving low marks, or be fired.
Plus, value-added is relatively cheap and easy to put in place compared with other school improvement efforts like those that involve hiring additional staff.
Value-added ratings can be one objective measure of how good a teacher really is, said Mary Peters of Battelle for Kids, which has helped implement Ohio’s teacher value-added ratings.
“Not just because they raise the most money and are the most popular in their building and parents love them and bake them cookies. But because quantitative data points to the fact that they are a great teacher,” she said.
Longtime teachers divided on ratings
Many teachers have a more skeptical view of value-added.
Olivia Carter has taught in Toledo for 28 years. When she got her rating from her principal last year in an envelope, she took it home and put it in her “important papers” pile. The sealed envelope is still there.
The information inside wouldn’t tell her anything new, she said.
“Being in the classroom, I know what I do. I can see it in the children every day,” she said.
To Carter, value-added, like standardized tests and state school ratings, is just another measure being imposed on Ohio teachers under the banner of “reform.”
But Emily Brown, a colleague of Carter’s at Toledo’s Grove Patterson Elementary Academy, sees value-added in a more positive way.
Brown said she taught for 13 years in the Toledo public schools before her bosses told her she was doing a really good job. Last summer, she got a text from her school’s principal. Brown had received the highest state rating, Most Effective.
“It’s nice to get that pat on the back,” Brown said.
–Teacher Olivia Carter
And for Katie Zielke, a math teacher at Columbus’ Johnson Park Middle School, receiving a low value-added rating was like a painful wakeup call.
Zielke had been teaching for 20 years and never heard any complaints from principals or colleagues. But two years ago, she learned that she had received the lowest possible value-added grade. The low rating was a surprise to her and to her principal.
“I kind of went, ‘Wow, what I was doing was not right,” Zielke said. “I was doing what I thought was best, but obviously I have to do something different.”
Scoring expectations versus outcomes
Value-added scores are about masses of cold, hard data. Specifically, the data gathered from state standardized tests.
Value-added scores are calculated by comparing how much students in each teacher’s class learned during the course of the year with how much they’re expected to learn.
How much they learn basically means how much their scores on state tests changed year to year. How much they’re expected to learn is determined by looking at how much other students at similar achievement levels have learned from year to year.
The better students do in comparison to their peers, the better their teacher’s score. That differential is assumed to be the teacher’s doing — the “value” he or she adds in the classroom. The Ohio Department of Education translates those scores into one of the five ratings.
Those ratings are still something of an experiment. Only reading and math teachers in grades four to eight get value-added ratings now. But the state is exploring how to expand value-added to other grades and subjects.
Among some teachers, there’s confusion about how these measures are calculated and what they mean.
“We just know they have to do better than they did last year,” Beachwood fourth-grade teacher Alesha Trudell said.
Some of the confusion may be due to a lack of transparency around the value-added model.
The details of how the scores are calculated aren’t public. The Ohio Education Department will pay a North Carolina-based company, SAS Institute Inc., $2.3 million this year to do value-added calculations for teachers and schools. The company has released some information on its value-added model but declined to release key details about how Ohio teachers’ value-added scores are calculated.
The Education Department doesn’t have a copy of the full model and data rules either.
The department’s top research official, Matt Cohen, acknowledged that he can’t explain the details of exactly how Ohio’s value-added model works. He said that’s not a problem.
“It’s not important for me to be able to be the expert,” he said. “I rely on the expertise of people who have been involved in the field.”
Fluctuations, other issues spark fairness concerns
But many teachers believe Ohio’s value-added model is essentially unfair. They say it doesn’t account for forces that are out of their control. They also echo a common complaint about standardized tests: that too much is riding on these exams.
“It’s hard for me to think that my evaluation and possibly some day my pay could be in a 13-year-old’s hands who might be falling asleep during the test or might have other things on their mind,” said Zielke, the Columbus middle school teacher.
A StateImpact/Plain Dealer analysis of initial state data suggests that teachers with high value-added ratings are more likely to work in schools with fewer poor students: A top-rated teacher is almost twice as likely to work at a school where most students are not from low-income families as in a school where most students are from low-income families.
-Teacher Katie Zielke
Ohio Department of Education officials say that’s probably because students in those schools with mostly poor students aren’t learning as much as students in other schools.
Teachers say they’ve seen their value-added scores drop when they’ve had larger classes. Or classes with more students who have special needs. Or more students who are struggling to read.
Teachers who switch from one grade to another are more likely to see their value-added ratings change than teachers who teach the same grade year after year, the StateImpact/Plain Dealer analysis shows. But their ratings went down at about the same rate as teachers who taught the same grade level from one year to the next and saw their ratings change.
In general, a teacher’s rating is unlikely to swing from one extreme to another year to year: About 3 percent of teachers who received the state’s top rating in 2010-11 received the state’s lowest rating in 2011-12.
Critics say that even less drastic year-to-year swings make the measure unreliable. Supporters say performance can change from year to year, some likening it to sports statistics like a batting average in baseball — measures for which year-to-year change is considered normal.
Some research has suggested that value-added ratings tend to agree with ratings from principals. But even that research showed that value-added ratings and principal ratings don’t line up exactly.
After all, said Kevin Kinne, principal of Zielke’s school in Columbus, value-added doesn’t take into account some of the most important things teachers are supposed to do — like care for students.
“It doesn’t tell about the kid that you drove home and saved from getting beat up by whatever situation because you cared enough to sit down and talk to them,” Kinne said. “And it doesn’t tell about the kid who came to school with some crazy parent situation and you were able to calm them and help them make it through the day. It doesn’t tell you any of that.”
In Ohio, value-added ratings and other growth measures will count for about half of a teacher’s final rating under the new evaluation system that will soon be in place in all school districts. Classroom observations and measures of professionalism make up the other half.
Officials at teachers unions say value-added ratings can be valuable tools but shouldn’t be used as the main factor in making decisions about teachers’ pay or hiring and firing.
Still, value-added ratings for teachers could be useful if many of the problems with value-added were fixed, said Arizona State University professor Audrey Amrein-Beardsley, who has studied value-added measures.
If value-added was shown to align with other measures of teacher quality, be independent of factors outside of teachers’ control and produce relatively consistent ratings from year to year, the model could be a good thing, she said.
“But in the current place we are, we are not ready for prime time,” she said.
Low score pushes one out the door
Stephon Fletcher started this year in Maria Plecnik’s classroom as a kid who would mouth off to teachers and do as much work as he felt like — which usually wasn’t much, he says.
He’s different now, he says, because of how and what Plecnik taught him. She showed that she respected him, and he in turn learned to respect her. His grades moved from mostly D’s and F’s to mostly A’s and B’s. And he’s at work on a book, tentatively titled, “Life and Why Things Happen the Way They Do.”
Stephon says the idea of Plecnik being an ineffective teacher is “outrageous.”
But Plecnik is through. She’s quitting her job at the end of this school year to go back to school and train to be a counselor — in the community, not in schools.
Plecnik was already frustrated by the focus on testing, mandatory meetings and piles of paperwork. She developed medical problems from the stress of her job, she said. But receiving the news that despite her hard work and the praise of her students and peers the state thought she was Least Effective pushed her out the door.
“That’s when I said I can’t do it anymore,” she said. “For my own sanity, I had to leave.”
StateImpact Ohio reporter Ida Lieszkovszky, Plain Dealer reporter Patrick O’Donnell and Plain Dealer data analysis editor Rich Exner contributed to this story.