Inside the Mathematical Equation for Teacher Merit Pay
Miami Herald reporter Laura Isensee contributed to this report. Read her story on Florida’s merit pay formula here.
School has always been about grading students. But now 24 states are starting to grade teachers.
Florida is using a mathematical formula to calculate how well teachers are doing their jobs. The grade it spits out will help determine how much a teacher gets paid and whether that teacher can keep his or her job.
But the formula is so complex even an advanced calculus teacher and former college math major can’t understand how it works.
CHARLES TRAINOR JR. / Miami Herald
Advanced calculus high school teacher, Orlando Sarduy, writes out the formula that will grade and help determine the pay of Florida teachers. Even for a college math major like him, the formula is too confusing to understand. He calls it a "mathematical experiment."
Coral Reef High School teacher Orlando Sarduy says just reading the formula is difficult for him.
StateImpact Florida and the Miami Herald partnered up to deconstruct the equation and try to figure out what’s going on here. We asked statisticians and policymakers how the formula works. The answer we got: No lay person, teacher or reporter can understand it. So just trust us.
“I would really challenge any sort of decision maker to look at [the formula] and explain it,” Sarduy said. “I understand just the basics, but this is really the technical nitty-gritty of what’s going on, and to me it looks the same as it would to a lay person, like ‘what’s going on here?”
How The Formula Works
The formula is designed to predict how students will score on the state’s standardized exam—the FCAT. And then it adjusts teachers’ pay depending on how well their students measure up against that predicted score.
The formula takes into account school and student characteristics that Florida officials say predict how well a student is going to do in school.
“I would really challenge any sort of decision maker to look at [the formula] and explain it.”
– Orlando Sarduy, advanced calculus teacher
Those officials decided there are only 10 factors that matter. They chose things like the number of students in a classroom, whether English is a student’s first language, attendance rates and disability status.
The statisticians created a formula that gives each of the 10 factors a certain weight.
That’s how the statisticians and policymakers who created the formula explained it to Miami Herald education writer Laura Isensee.
For example: If a student misses 5 days of school, the statisticians determine what the effect of missing 5 days of school will have on that student’s standardized test score.
The statisticians do this for each factor and every student. In the end, the formula predicts what a student’s test score should be given all these factors.
“And that prediction will be the grading stick for the teacher,” Isensee said. “If the student gets higher than the predicted score, the state thinks they must have a good teacher. If a student scores below the predicted score, then the teacher could be in trouble.”
The state has a list of all the different weights for all the factors.
But Isensee said, “The weights are all over the place, even for kids who seem to be in the same situation.”
Related
Posts
Here’s an example. Warning, difficult math ahead.
The weight for an English language learner in reading class is -7.3 if you’re in sixth grade, but it +12.9 if you’re in tenth grade.
Isensee says the formula requires a lot of trust.
“I tried to understand why the impact is so different for everyone and the statisticians basically told me, ‘Don’t worry about it, that’s the formula’s job. The formula knows how much weight to give everything.’
“The teachers have to trust that the policy makers chose the right factors, and the policymakers have to trust that the statisticians came up with an accurate formula.”
Kathy Hebda is with the Florida Department of Education. She says the formula Florida created is a state of the art model.
“We have contracted with leading national experts,” Hebda said. “We have a statewide committee that is steering the process and making recommendations to the commissioner about the model, that’s made up primarily of teacher. That kind of input and guidance is extremely important to make sure that the model works properly.
“We’re very confident in the process and the approach we’ve taken,” Hebda said.
Teachers like Sarduy are skeptical. He questions why Florida only chose 10 factors to begin with. He says there could be hundreds of factors that impact how well a student does in school.
“[The formula is] only as good as the variables that you’re actually looking out for as well as the test that you’re using to measure,” Sarduy said. “At the backbone of this is still an exam that’s made up. If the exam is invalid, the whole equation is invalid.”
The most influential factor in the equation is the score a student gets on the FCAT exam. It’s the factor with the most predictive value.
What Happens If You Don’t Teach an FCAT Subject?
In Florida about 60% of teachers do not teach a subject that is tested by the FCAT, like physical education teachers, health and history teachers, or chemistry and advanced calculus teachers.
“Health teachers, advanced calculus teachers – their pay will be based on how well kids read.”
– Laura Isensee, Miami Herald education writer
Insensee says until the state comes up with a test for every subject in every grade, teachers who don’t teach an FCAT subject are going to be graded on the whole school’s reading score.
“So heath teachers, advanced calculus teachers, their pay will be based on how well kids read,” said Isensee.
That has teachers like Sarduy pretty frustrated.
“It’s infuriating,” Sarduy said. “I have nothing to do with whatever that end result is.”
But the state does give teachers another chance to show off their teaching skills. Hebda says the grade the formula gives teachers is only half of the whole teacher evaluation process.
“The law is very clear that that’s 50% of an evaluation. The other piece of that is equally important, which is instructional practice,” Hebda said.
“What is the teacher actively doing in the class? What are the students actively doing in the class? And then what are the student outcomes? Those are the things that all go into making that final evaluation result.”
So principals and other educators will still do their own evaluation of teachers by sitting in their classroom and reviewing lesson plans.
But some teachers still feel like too much of the end result is out of their control.
The State Says Poverty Doesn’t Matter
Study after study we hear that poverty is the number one indicator of how well students do in school. But Florida policymakers made it against the law to include any socioeconomic status as a factor in the formula.
There’s no factor for poverty, homelessness, immigration status, race or ethnicity.
The state says the formula does not need to include a student’s socioeconomic status as its own factor in the formula because its already baked into the equation. Teachers would only be graded on how well they help poor students improve from the year before when they were also poor.
The rationale is that all kids, regardless of whether they’re homeless or poor, can improve at the same rate as kids from wealthy areas, if they have a good teacher.
“I like the fact that its attempting to isolate just the teacher’s [role],” Sarduy said. “But you have to realize, that’s me on the line. I’m now part of a statewide experiment.”
UPDATE: Here’s the formula in its entirety.
