It’s a simple question: Will the new state assessment measure mastery of reading and math skills? Yet Kansas Education Commissioner Randy Watson and the consultants hired to prepare it refuse to say. Watson also declined to respond to other questions we sent recently.
The question was prompted by comments Dr. Karla Egan made at the May 13 State School Board meeting. Egan founded EdMetrics, the consulting firm working with the Kansas Department of Education on the development of the new state assessment. While explaining why cut scores that determine performance levels (basic, proficient, etc.) are set after the first test is taken, Egan said that it is standard practice on the psychometric assessment currently being developed for Kansas, rather than ahead of time, as is done on a mastery assessment.
When several board members asked for more explanation, Beth Fultz, Director of Standards & Assessments Services at KSDE, offered this comment about the difference between psychometric and mastery assessments:
“So this is not a mastery test, and I’ve spoken with the commissioner a lot about it. If you want to (do) mastery testing, that’s a totally different model. You set your expectations and you say, we expect, wherever you set it, students to know 80% of this material. We say they’ve mastered it and they can move on.
“That’s not what this test is, either, right? So you have to think of the purpose of the test and what we’re…setting it for, and this is, you know, based on this methodology that we’re using. And a group of, I don’t know what use anymore, because I haven’t been through standard setting a while, but you’re saying you’re the teacher with 25 students in your room. Think about… how many of these questions do you think that you would want your students to get right after you’ve been teaching them for a year.”
That garbled explanation sounds like grading on a curve. For example, if a teacher expects a certain percentage of the class to get an “A” with 90% or more of the answers correct, but no one reaches that threshold. The grading scale may need modification so that students with 80% or more correct receive an “A” and meet the expectation.
In that example, adjusting the score would overstate students’ mastery of the material, but setting cut scores based on a known, independent high standard, like proficiency levels on the National Assessment of Educational Progress (NAEP), could produce valid results. The 2024 NAEP results for 8th-grade reading showed 34% Below Basic, 41% Basic, 23% Proficient, and 2% Advanced, so setting cut scores to mirror those results would be reasonable.
Setting cut scores so that the 2025 results are very similar to the 2024 state assessment results would also be reasonable. However, if the 2025 state assessment results are significantly better than 2024, it would indicate that KSDE reduced standards.
Dr. Egan mentioned that adjusting scores to reflect NAEP or the 2024 state assessment results are options, so why won’t Watson say (or the State School Board order) that to be the goal? Alternatively, replicating the current state assessment results, which are very similar to NAEP, would ease fears that KSDE may try to reduce standards to make results look better. It may also make tracking progress toward the 2033 goals of the Blueprint for Literacy Act (50% proficient in reading and math and no more than 10% below grade level).
Watson insists that no students are below grade level
Another question Watson refused to answer is about the percentage of students who are below grade level in Kansas.
Watson insists that 33% of students in Level 1 are not below grade level, even though he admits they have limited ability to use reading and math skills. Level 1 was clearly below grade level in the adjacent graphic he used to explain the new standards in 2015. A few years later, ‘grade level’ was scrubbed from the definitions, but nothing else changed. Now, Watson says the state assessment only measures grade-level performance, so it is wrong to say any students are below grade level.
State board members heard a conflicting story during the cut score discussion at the May 13 meeting from Dr. Dan Gruman, a Shawnee Mission employee working with KSDE on the new assessment.
He said, “There’s a point at which we have to decide, from an accountability perspective, what percent of students are meeting grade-level standards. There is a point at which you have to draw a hard line on where that line is, and the word ‘proficient,’ we believe, provided better clarity, and to understand where that line is.”
Despite clearly saying that proficient (Level 3 on the new assessment) is the line at which grade-level standards are met, he denied that to be true when I wrote to him. Gruman tried passing off Watson’s definition, but he would not explain why he pretty clearly told state school board members that Level 3 Proficient meets grade level standards.
I asked Gruman and Watson if their explanation means all students are therefore at grade level, but both refused to answer.
Inconsistencies often expose attempts to mislead, and going silent, as Watson and Gruman did, only confirms deceptive intent. Repeated episodes like this (see here and here for examples) make it apparent that the Kansas Department of Education is about controlling the narrative rather than improving student outcomes.
We’ve said it before, and it remains true: Student outcomes won’t improve until adult behaviors change. It’s only a matter of how long state school board members and legislators stand by and watch educational malpractice before summoning the courage to compel change.
