STAAR Rescoring Requests Spike After Schools Question Computer Grading

The Texas Tribune

AUSTIN, TX — The number of requests to rescore students’ STAAR tests nearly tripled from last year as more Texas schools sought reviews of written responses on the reading exams, thousands of which were initially graded by computers.

The Texas Education Agency began using an automated scoring engine in December 2023 to grade open-ended questions on the State of Texas Assessments of Academic Readiness, including on the reading and English tests, with humans manually reviewing a quarter of exam scores. 

In spring 2024, the state had 2,758 requests to regrade open-ended response scores on reading exams. There were 21,620 requests last year. This spring, that jumped to 64,489.

Meanwhile, the number of corrections doubled compared to the previous year. The state improved more than 13,300 reading scores after the reviews. Of those, humans initially assigned grades to 42% while computers graded 58%. Humans conducted all re-scores.

Allison Matney, director of the Texas Center for School Accountability, said historically, it was rare for district administrators to request STAAR rescoring. She attributed the increase to greater awareness about the automated scoring and review process.

“That traction of superintendents talking to each other — (and) media continuing to talk about it — made more districts aware that not only can they do this process, but actually they should, and they need to,” Matney said.

The state’s use of automated scoring for open-ended responses was initially criticized after a spike in students earning zeros on writing prompts in the first round of exams graded by computers. Agency officials, however, pointed to other causes for the low scores in 2023, including that many students were re-taking the exam.

State officials have stressed that automated computer scoring is different from generative artificial intelligence, such as ChatGPT, as the program does not “learn” from one response to the next.

Agency spokesperson Jake Kobersky said scoring essays by hand would require four to five times the number of human scorers and cost an additional $15 million to $20 million per year. It would also result in delayed results for districts, he said.

“TEA uses a hybrid model where human experts drive the scoring. … The result is a system that keeps human judgment at the center of scoring, applies it where it matters most, and delivers results to districts and families faster than hand-scoring every response would allow,” Kobersky said.

More than 3.2 million Texas students took the reading STAAR this spring, which includes third through eighth graders, as well as those who took the high school level English I and English II end-of-course assessments. In total, less than 0.5% of total open-ended responses on reading exams saw changes.

TEA officials began incorporating automated scoring after the state redesigned the STAAR to include fewer multiple-choice questions and more open-ended questions — which are also known as constructed responses.

To develop the computer scoring, the TEA gathered thousands of responses that were previously scored by people. From the sample, the computer learned the characteristics of responses, and it was programmed to assign the same scores as a human.

More than 250 public charter networks and school districts, including Cypress-Fairbanks, Dallas, Katy and Northside, submitted rescoring requests this year. Nearly 90% had at least one submitted reading exam see a score increase, according to agency data.

Public campuses and districts receive A-F accountability ratings based on the number of students who meet certain performance levels on STAAR. When campuses underperform on the test for multiple years, state law allows Texas education commissioner Mike Morath to take action, including by installing a conservator or replacing a district’s locally elected school board.

TEA automatically rescored extended constructed student responses at no cost if the exams were one point from qualifying the next performance level. Those four levels, which reflect how students tested against their grade level, are: did not meet, approaches, meets and masters.

Officials from some districts said the TEA’s rescoring process for students on the cusp of meeting the next performance level freed up their budgets and allowed them to submit more students for re-scoring. One said, since TEA automatically rescored students who were one point away from the next level, they submitted students who were two points away.

In a statement, Dallas ISD officials said the district submitted a rescoring request because they felt students benefit when human scorers review their answers. In total, the district submitted more than 5,100 reading tests for rescoring, and 34% returned with a higher score.

Dallas ISD officials estimated that the combined automatic and district-requested re-scores helped five campuses in the state’s second-largest school district improve their overall ratings before the state’s public release. One campus moved from a D to a C rating and four campuses moved from C to B, according to the DISD statement.

School districts must pay $50 per test if they request rescoring and the student’s score does not change. The rescore is free to districts if a score changes.

The Texas Education Agency’s rescoring process occurred before the release of A-F accountability ratings in August.