TY - JOUR
T1 - Item ordering and computerized classification tests with cluster-based scoring
T2 - An investigation of the countdown method
AU - Finkelman, Matthew D.
AU - Lowe, Sarah
AU - Kim, Wonsuk
AU - Gruebner, Oliver
AU - Smits, Niels
AU - Galea, Sandro
PY - 2018/2/1
Y1 - 2018/2/1
N2 - The countdown method is a well-known approach to reducing the average length of screening instruments that are presented by computer. In the countdown method, testing is terminated once the result of the screener ("positive" or "negative") has been unambiguously determined from prior answers. Previous research has examined whether presenting dichotomously scored items in order from "least to most frequently endorsed" or "most to least frequently endorsed" is more efficient when the countdown method is used. The current study describes the Mean Score procedure, an extension of the above item ordering procedures to polytomously scored items, and evaluates its efficiency relative to the distribution of other possible item orderings in 2 real-data simulations. Both simulations involve item responses to the Posttraumatic Stress Disorder (PTSD) Checklist for DSM-5 (PCL-5). In the first simulation, items were scored polytomously, and a single cutoff point was used to determine the screening result. In the second simulation, items were converted to dichotomous scores, as well as categorized into 4 clusters; a positive result for the entire assessment was obtained if and only if a positive result was obtained for each cluster. The latter simulation also investigated the effect of reordering the clusters themselves on the efficiency of the countdown method. Results indicated that the Mean Score procedure does not necessarily produce the optimal ordering, but tends to assemble an efficient item ordering relative to the distribution of possible orderings. In the second simulation, reordering the clusters themselves affected efficiency. Future research directions are suggested.
AB - The countdown method is a well-known approach to reducing the average length of screening instruments that are presented by computer. In the countdown method, testing is terminated once the result of the screener ("positive" or "negative") has been unambiguously determined from prior answers. Previous research has examined whether presenting dichotomously scored items in order from "least to most frequently endorsed" or "most to least frequently endorsed" is more efficient when the countdown method is used. The current study describes the Mean Score procedure, an extension of the above item ordering procedures to polytomously scored items, and evaluates its efficiency relative to the distribution of other possible item orderings in 2 real-data simulations. Both simulations involve item responses to the Posttraumatic Stress Disorder (PTSD) Checklist for DSM-5 (PCL-5). In the first simulation, items were scored polytomously, and a single cutoff point was used to determine the screening result. In the second simulation, items were converted to dichotomous scores, as well as categorized into 4 clusters; a positive result for the entire assessment was obtained if and only if a positive result was obtained for each cluster. The latter simulation also investigated the effect of reordering the clusters themselves on the efficiency of the countdown method. Results indicated that the Mean Score procedure does not necessarily produce the optimal ordering, but tends to assemble an efficient item ordering relative to the distribution of possible orderings. In the second simulation, reordering the clusters themselves affected efficiency. Future research directions are suggested.
KW - Computerized adaptive testing
KW - PCL-5
KW - Respondent burden
KW - countdown method
UR - http://www.scopus.com/inward/record.url?scp=85015166612&partnerID=8YFLogxK
U2 - 10.1037/pas0000470
DO - 10.1037/pas0000470
M3 - Article
C2 - 28301192
AN - SCOPUS:85015166612
SN - 1040-3590
VL - 30
SP - 204
EP - 219
JO - Psychological Assessment
JF - Psychological Assessment
IS - 2
ER -