TY - JOUR
T1 - Applying the Systems Evaluation Protocol in the real world
T2 - Six case studies
AU - Urban, Jennifer Brown
AU - Linver, Miriam
AU - Chauveron, Lisa M.
AU - Archibald, Thomas
AU - Hargraves, Monica
AU - Buckley, Jane
N1 - Funding Information:
The initial development of the SEP was supported by a 3‐year grant (2005–2008) from the National Science Foundation (NSF). The SEP and the accompanying software program, the Netway, were refined in a Phase II Trial of the SEP that began in 2008 and concluded in 2015, also with support from the NSF. In this early research and testing, the SEP was implemented with cohorts of small‐ to medium‐sized educational outreach programs in two large STEM education systems: Cornell Cooperative Extension (CCE) in New York State, and Materials Research Science and Engineering Centers (MRSECs) across the U.S. Cohorts ranged in size from three to 18 programs; each program was represented by a working group typically consisting of two to four staff members.
Funding Information:
Finally, in this last case, the SEP was a tool used for Evaluation Capacity Building (ECB; Taylor‐Ritzler, Suarez‐Balcazar, Garcia‐Iriarte, Henry, & Balcazar, 2013 ) with staff and leadership from youth character organizations and evaluators. Increasingly, foundations, philanthropies, and other funding bodies seek to be not just learning organizations, but learning systems, whereby the ecology of evidence use and creation in their entire sector is enhanced. A national initiative funded by the John Templeton Foundation, the Partnerships for Advancing Character program Evaluation (PACE) Project, was designed to promote high‐quality evaluations in the field of character development through enhanced capacity building using true partnerships between evaluators and program practitioners. More information and empirical findings on this initiative are presented in Buckley, Hargraves, and Moorman (this volume); Chauveron et al. (this volume); and Hargraves et al. (this volume).
Funding Information:
This work supported in part by the John Templeton Foundation, grant # 60483, the S.D. Bechtel Jr. Foundation, grant # 8325, and the National Science Foundation, grant #s 0535492, 0814364, and 1811214.
Publisher Copyright:
© 2021 Wiley Periodicals LLC.
PY - 2021/3/1
Y1 - 2021/3/1
N2 - In this paper, we present six case studies that illustrate the application of the Systems Evaluation Protocol (SEP) in different real-world implementation conditions. The SEP is a step-by-step guide for how to implement Relational Systems Evaluation (RSE), accounting for the complex factors inherent in the larger systems within which a given program is embedded. We discuss the specific SEP steps used and products developed in each case study project, including an emphasis on how decisions were made to include particular SEP elements in different contexts. The six cases differ in terms of the scale of the programs involved, the nature of the SEP delivery or facilitation process, and the balance among multiple desired outcomes reflecting differential emphases on evaluation capacity building. As such, the cases reported here demonstrate that RSE, as implemented using the SEP, is a widely applicable approach to evaluation and ECB.
AB - In this paper, we present six case studies that illustrate the application of the Systems Evaluation Protocol (SEP) in different real-world implementation conditions. The SEP is a step-by-step guide for how to implement Relational Systems Evaluation (RSE), accounting for the complex factors inherent in the larger systems within which a given program is embedded. We discuss the specific SEP steps used and products developed in each case study project, including an emphasis on how decisions were made to include particular SEP elements in different contexts. The six cases differ in terms of the scale of the programs involved, the nature of the SEP delivery or facilitation process, and the balance among multiple desired outcomes reflecting differential emphases on evaluation capacity building. As such, the cases reported here demonstrate that RSE, as implemented using the SEP, is a widely applicable approach to evaluation and ECB.
UR - http://www.scopus.com/inward/record.url?scp=85104861711&partnerID=8YFLogxK
U2 - 10.1002/ev.20448
DO - 10.1002/ev.20448
M3 - Article
AN - SCOPUS:85104861711
SN - 1097-6736
VL - 2021
SP - 65
EP - 77
JO - New Directions for Evaluation
JF - New Directions for Evaluation
IS - 169
ER -