Item ordering and computerized classification tests with cluster-based scoring: An investigation of the countdown method

The countdown method is a well-known approach to reducing the average length of screening instruments that are presented by computer. In the countdown method, testing is terminated once the result of the screener ("positive" or "negative") has been unambiguously determined from prior answers. Previous research has examined whether presenting dichotomously scored items in order from "least to most frequently endorsed" or "most to least frequently endorsed" is more efficient when the countdown method is used. The current study describes the Mean Score procedure, an extension of the above item ordering procedures to polytomously scored items, and evaluates its efficiency relative to the distribution of other possible item orderings in 2 real-data simulations. Both simulations involve item responses to the Posttraumatic Stress Disorder (PTSD) Checklist for DSM-5 (PCL-5). In the first simulation, items were scored polytomously, and a single cutoff point was used to determine the screening result. In the second simulation, items were converted to dichotomous scores, as well as categorized into 4 clusters; a positive result for the entire assessment was obtained if and only if a positive result was obtained for each cluster. The latter simulation also investigated the effect of reordering the clusters themselves on the efficiency of the countdown method. Results indicated that the Mean Score procedure does not necessarily produce the optimal ordering, but tends to assemble an efficient item ordering relative to the distribution of possible orderings. In the second simulation, reordering the clusters themselves affected efficiency. Future research directions are suggested.