Essays on Florida’s new writing test will be scored by a human and a computer, but the computer score will only matter if the score is significantly different from that of the human reviewer. If that happens, bid documents indicate the essay will be scored by another human reviewer.
University of Akron researcher Mark Shermis has studied the accuracy of automated essay scoring — computer programs which read essays and assign a score – in three trials. Shermis concluded the programs worked at least as well as human scorers in two of those trials.
An Australian trial of two automated essay scoring programs found machine-scored essays fell short of human grading on closed content driven writing prompts. But that trial used just one prompt and a small sample of essays.
A second trial, sponsored by the William and Flora Hewlett Foundation, tested eight commercial automated essay scoring programs and one developed by a university lab. the trial gathered more than 22,000 essays from eight writing prompts spread across six states.
The nine automated essay scoring programs performed on par with human scorers. The humans earned an accuracy score of .74, while the best of the automated essay scoring programs earned an accuracy score of .78. The machines scored particularly well on two data sets which included shorter, source-based essays.
“A few of them actually did better than human raters,” Shermis said.
Florida writing tests will be graded by a human and a computer program, according to bid documents for the new test. And just 2 percent of students will take a pencil and paper exam in 2015.
A computer program will grade student essays on the writing portion of the standardized test set to replace the FCAT, according to bid documents released by the Florida Department of Education.
The essays will be scored by a human and a computer, but the computer score will only matter if the score is significantly different from that of the human reviewer. If that happens, the documents indicate the essay will be scored by another human reviewer.
Florida writing tests are currently graded by two human scorers and the state has never used computerized grading on the exam.
The Florida Department of Education announced Monday it chose the non-profit American Institutes for Research to produce new tests tied to Florida’s Common Core-based math and language arts standards. Spokesmen for the agency and AIR said they had yet to sign a contract, were still working out the details and declined to comment about the specifics of the new test.
“It’s speculative at this point to think about what is on the assessments,” said Joe Follick, communications director for the Florida Department of Education.
But the bid documents show using computers to grade the state writing test will save $30.5 million over the course of the six-year, $220 million contract with AIR. The change was part of a list which trimmed more than $100 million from AIR’s initial proposal.
A Florida lawmaker has introduced a bill which would make college tuition free, but students would repay the cost over time.
A Florida lawmaker has proposed allowing students to attend college tuition-free, and then repay the cost with a percentage of their salary after graduating.
The proposal has been nicknamed “Pay It Forward” tuition because students making their payments keep tuition free for future generations of college students. Students might pay their Alma mater between 2 percent and 6 percent of their annual salary for as long as 25 years, depending on the terms of the program.
The idea was first proposed in Oregon, which is creating a pilot program for lawmakers to consider. In Florida, Sen. Eleanor Sobel, D-Hollywood, introduced SB 738, which would launch a pilot program to create a Pay It Forward program.
Tuesday, she sought support for the Ethan Rediske Act, or HB 895, which would exempt students from state standardized tests if parents, special educators and school superintendents could prove a medical need to skip the test.
“This incident caused anguish to my family,” Rediske told the board, “and shows a stunning lack of compassion and even common sense on the part of the Department of Education.
“You may ask yourselves: ‘If this is such a problem why isn’t there more public outcry from the parents of disabled children?’ I am here to tell you why. Parents of severely disabled children are exhausted. We spend our lives keeping these children alive.”
The bill would require state colleges to accept two years of computer programming if the courses applied to a student’s major. State universities would have the option of accepting those courses instead of a foreign language.
Senate education chairman John Legg, who is sponsoring the bill, said it would prepare students to fill high-tech jobs. Advocates argue Florida won’t produce enough computer programmers over the next decade to fill available jobs.
With state schools making the full transition to Common Core language arts, literacy and math standards, the group says Florida should modernize the nation’s oldest school grading formula.
“Now’s the perfect time to improve the system based on what we’ve learned from the last decade of school grading,” said president Trey Csar.
Among their recommendations:
Measure student test score growth over multiple years — Right now the state measure one year of growth. Multiple years would reduce the amount the year-to-year swings that sometimes effect student test results and grades.
Broaden the score ranges for each letter grade — This would prevent small increases or decreases in school scores from resulting in a change of several letter grades.
Maryland excludes the results of 62 percent of learning-disabled and English learners on the fourth grade National Assessment of Educational Progress reading exam. The state excludes 60 percent of those same students on the eighth grade reading exam. Both are the highest rates in the country.
NAEP recommends states not exclude more than 15 percent of student results. The national average is 12 percent. The NAEP reading and math exams are given every two years to a sample of students across the country.
Maryland officials say they allow a person or computer to read text aloud to those students on their annual exams. NAEP does not allow that accommodation, so the scores are not counted.
Florida excluded 12 percent of students with disabilities and those learning English from the 2013 NAEP results. Florida does not allow the “read aloud” accommodation on the Florida Comprehensive Assessment Test.
As the Post report, the change has a significant effect on Maryland’s score — especially compared to other states:
Florida school superintendents are asking lawmakers to extend the switch to Common Core standards and rewrite school grading and teacher evaluation requirements.
Florida school superintendents are asking state leaders to revamp the state’s A through F school grading system — including eliminating the letter grades — as the state completes the switch to new math, English and literacy standards.
Florida is one of 45 states to fully adopt Common Core, which outlines what students should know at the end of each grade. Supporters say the standards are more challenging and will prepare high school graduates for college or a job. Critics are concerned Common Core is a one-size-fits-all policy, won’t improve schools and may not be appropriate for young children.
Smith said school superintendents support the standards, but need more time to prepare.
Florida students improved their scores on all four sections of the National Assessment of Educational Progress, but eighth graders still trail the national average.
Florida student scores improved on a key national standardized test, including some of the largest eighth grade reading and math gains in the country.
But Florida eighth graders still trail the national average in math, according to results from the National Assessment of Educational Progress, also known as The Nation’s Report Card.
Florida eighth graders scored an average of 281, on a 500-point scale, in math and 266 in reading. The national average was 284 in math and 266 in reading.
In 2011, Florida eighth graders scored an average of 278 in math and 262 in reading.
Florida fourth graders scored an average of 242 in math and 227 on the reading exam. In 2011, Florida fourth graders scored 240 in math and 225 in reading.
Florida’s fourth grade gains were not considered statistically significant
The national fourth grade average was 241 in math and 221 in reading.
The math and reading exams are given to a sample of fourth and eighth grade students across the country and is considered the gold standard exam for comparing student performance across state lines. NAEP releases results every two years.