TY - GEN
T1 - A Semantic Text Processing System for Free-Write English Papers
AU - Depascale, Ryan
AU - Robila, Stefan A.
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - We present the development and evaluation of a semantic text processing system that evaluates student essays. The system processes n-many documents and suggests a letter grade, identifies papers that may need additional teacher action based on component/composite scores, and allows optional teacher input on features to generate the grade. The system was developed in Python using open-source libraries and is also available as open-source. Using a human-in-the-loop approach, expert teachers were interviewed as part of the design process. Assessing the documents on token, sentence, readability, dependency distance, and part of speech with user guided feature selection the system generated automated results where the true letter grade and machine letter grade corresponded exactly in 46% of papers and in a ±1 letter grade interval in 86% of papers. The program can be further extended to flag grades for potential human review based on user defined criteria with example code provided for papers marked as written above the high school level.
AB - We present the development and evaluation of a semantic text processing system that evaluates student essays. The system processes n-many documents and suggests a letter grade, identifies papers that may need additional teacher action based on component/composite scores, and allows optional teacher input on features to generate the grade. The system was developed in Python using open-source libraries and is also available as open-source. Using a human-in-the-loop approach, expert teachers were interviewed as part of the design process. Assessing the documents on token, sentence, readability, dependency distance, and part of speech with user guided feature selection the system generated automated results where the true letter grade and machine letter grade corresponded exactly in 46% of papers and in a ±1 letter grade interval in 86% of papers. The program can be further extended to flag grades for potential human review based on user defined criteria with example code provided for papers marked as written above the high school level.
KW - Automated Grading
KW - Natural Language Processing
KW - Open-Source.
UR - http://www.scopus.com/inward/record.url?scp=85148330058&partnerID=8YFLogxK
U2 - 10.1109/ISEC54952.2022.10025229
DO - 10.1109/ISEC54952.2022.10025229
M3 - Conference contribution
AN - SCOPUS:85148330058
T3 - 2022 IEEE Integrated STEM Education Conference, ISEC 2022
SP - 103
EP - 108
BT - 2022 IEEE Integrated STEM Education Conference, ISEC 2022
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 12th IEEE Integrated STEM Education Conference, ISEC 2022
Y2 - 26 March 2022
ER -