History of Program Reviews at CASEL
CASEL shared its first program review in 2003 with the publication of Safe and Sound: An Educational Leader’s Guide to Evidence-Based Social and Emotional Learning (SEL) Programs (CASEL, 2003). In addition to demonstrating how SEL programs contribute to the main mission of our nation’s schools, this publication summarized the current status of outcome research on SEL programs. It provided educators with practical information on the features of different programs that could help them select a program most relevant and suited for their particular needs. The guide presented information on 80 different programs and was, at the time, the most comprehensive research and practical survey of SEL programs available.
During the decade since the dissemination of Safe and Sound, several major advances have occurred in SEL research, practice, and policy that warranted updating the review. Many more research studies, including program evaluations, have become more rigorous, which increases the confidence one can place in their findings. New approaches to fostering academic, social, and emotional learning have been developed, some of which emphasize traditional explicit instruction, while others focus on pedagogy or deeper integration of SEL within academic curricula. There is also better information about the outcomes one might expect from programs offered at different grade levels. The 2013 Guide incorporates these new developments.
Because of the major accomplishments of SEL research over the past few years, the 2013 CASEL Guide is different from the 2003 guide in several ways. To begin, the current Guide is more selective and the criteria for inclusion are more rigorous than they were in Safe and Sound. Other important distinctions include:
- In Safe and Sound we identified 80 social and emotional learning programs that met our design criteria. These programs were included whether or not they had been evaluated. Of these 80 programs, we identified 22 “SELect” SEL programs that met a higher standard for evidence of effectiveness. In this review, our criteria for inclusion as “evidence-based” are more stringent. In the 2003 review SELect programs were not required to include a comparison group. However, the evidence base for SEL programs has grown dramatically and become increasingly rigorous in the last decade (e.g., Durlak et al., 2011). In the 2013 Guide, the evaluations of CASEL SELect programs had to include a control group in addition to pretest and posttest measurement of behavior.
- We have broadened the types of programs included in the current Guide. The primary focus of Safe and Sound was lesson-based programs that provided explicit skill instruction in SEL. In this Guide, CASEL SELect programs include those that address teacher instructional practices and pedagogy and programs that fully integrate social and emotional learning with academic content in specific core content areas.
- We expanded our coverage to include preschool approaches because, as noted in the introduction, the importance of preschool education to support child development and school readiness is increasingly recognized. There is also now a stronger evidence base for SEL programming at the preschool level (Denham & Burton, 2003; Camilli et al., 2011).
- In the previous review we included programs for kindergarten through twelfth grade without distinguishing among programs for students from different age groups. Our new reviews are more developmentally focused, which is why we are separating the reviews of programs for preschool, elementary, middle, and high school. The 2013 Guide focuses on preschool and elementary programs. The review of secondary school programming was released in 2015.
Table of Contents
- About the Guide
- Overview of SEL
- Review Process
- Inclusion Criteria
- Rating Framework
- Rating Tables (PreK Programs)
- Rating Tables (Elem. Programs)
- Program Descriptions
- Guidelines for Selecting Programs
- Summary of Findings
For this Guide, the review process began by establishing our inclusion and exclusion criteria. We then used several methods to identify potentially relevant classroom-based programs designed for use with a universal population of students.
2009: Initial call for nominations and identified potentially relevant programs
2011/2012: Additional outreach efforts to program developers and researchers.
2011/2012: Examined CASEL’s original program review, Safe and Sound, and other major literature reviews, national reports, key publications, and searched national database.
2012: All programs identified for possible inclusion were examined in several different ways by teams of trained coders. If the program was classroom-based and designed for use with a universal population of students, we requested copies of all available published and unpublished outcome evaluations that would meet our criteria from providers.
- We checked these reports against those we found through our own literature search procedures. Coders examined every outcome evaluation submitted by each program.
- We conducted an e-mail survey completed by program developers or their designated staff about the training they offered for program implementation. We supplemented these surveys through phone contact, if necessary, to clarify answers to certain questions. Our final sample consisted of 25 SELect programs.
- Graduate-level coders with extensive education and experience in social and emotional learning reviewed all program materials. The coders received more than 40 hours of training in the coding system from senior SEL researchers involved in the Guide development process
The review: For each review, coders scanned the complete set of program materials provided by the developers in order to familiarize themselves with the overall organization and content of the program.
The CASEL Program Guide review process is currently paused for continuous improvement purposes. During this time CASEL will be ensuring that our review process is efficient and straightforward, and that our inclusion criteria are both reasonable and rigorous. Our next formal review cycle is tentatively scheduled to begin on September 2019. Questions about the review should be directed to Brittney Williams.
- Coders completed an intensive content analysis of sample years of each program. In most cases this involved review of the preschool materials, first-grade materials, and fourth-grade materials, depending on whether those were found to be representative of the program as a whole. Additional grades were reviewed as necessary.
- Before the coders worked independently, they had to be at least 85% in agreement on all rating elements for a subset (20%) of the programs. Reliability was monitored throughout the process to maintain the same level of agreement (85%) on the remaining programs. Any disagreements in coding were eventually resolved through discussion among the raters and supervising staff.To avoid conflicts of interest, no one having any financial relationship to any program was involved in reviewing the programs or in discussions about programs.