Complicating “Statistical Soundness”: How States Legitimize Subgroup Composition and N-Size Decisions in ESSA Plans

This paper uses a critical quantitative lens to examine the discursive techniques used in states’ approved Every Student Succeeds Act plans to rationalize their subgroup accountability decisions. These subjective decisions about subgroup composition and n-size are used to conduct quantitative analyses to identify performance gaps and, therefore, shape how particular subgroups are constructed as (dis)advantaged. We identify the plans’ discursive legitimation strategies to draw attention to the range of subjective considerations that live behind those two seemingly “objective” decisions. For decisions about subgroup composition, state educational agencies (SEAs) relied heavily on appeals to past practice and student demographics and emphasized their efforts to achieve inclusive and stable accountability measures that effectively identified performance gaps. For n-size decisions, SEAs referenced past practice, statistical expertise, the practices of other SEAs, stakeholder consultation, inclusivity, and their intention to achieve an accountability measure that provides statistical soundness, student privacy, and responsiveness to school demographics.