CHICAGO - A pair of new reports being released today by the University of Chicago Consortium on School Research (UChicago Consortium) demonstrate the predictiveness of the 5Essentials Survey (also known as the My Voice, My School survey) results in Chicago Public Schools (CPS), in both high- and low-poverty schools, while also suggesting more can be done in order for the district and schools to use the data effectively for school improvement.
The survey has been administered in CPS since the 1990s and measures a school’s strength in five essential organizational conditions that influence student learning: Effective Leaders, Collaborative Teachers, Involved Families, Supportive Environment, and Ambitious Instruction. The current study, funded by the Bill & Melinda Gates Foundation, used both quantitative and qualitative methods to address questions about the survey’s predictiveness and how schools utilize the data generated by the survey. Specifically, the quantitative study used pre-pandemic data from 2011–18 to examine whether the 5Essentials Survey predicts school improvement in high- and low-poverty schools, and whether every school has a similar likelihood to develop or maintain strong 5Essentials results, regardless of poverty level. Using interviews conducted throughout 2019, the qualitative study examines how schools understand and utilize data from the 5Essentials Survey in the context of improvement efforts, and what factors facilitate or impede schools’ engagement with their 5Essentials Survey results.
The quantitative study found that 5Essentials Survey measures predicted a school’s effectiveness and ability to improve in both high- and low-poverty schools at both the elementary and high school level. At the same time, interviews with district officials and school staff showed that few schools use the 5Essentials Survey data in their improvement efforts, due to the absence of a coherent district-wide strategy and supports needed to use the data effectively.
“Student and teacher voice have always been very predictive of schools’ effectiveness and ability to improve. This research demonstrates that the 5Essentials Survey remains a valuable tool for district and school leaders, despite concerns about its use in accountability. The challenge going forward is how to use the data effectively, which ultimately has implications for equity and student success,” said Laura Davis, a Senior Research Analyst at the UChicago Consortium and the lead author of the qualitative study.
Key Findings:
Quantitative:
- 5Essentials Survey measures predicted school improvement in both high- and low-povertyschools at both the elementary and high school level. Meaning, the survey remains strongly predictive, despite concerns about its inclusion in district accountability.
-
On most 5Essentials Survey measures, chances of being strong on the 5Essentials were found to be similar for schools at all poverty levels. At the same time, there were some measures that were more likely to be strong in schools with students from neighborhoods with lower poverty levels. Strength in five of these measures was less prevalent in high- poverty schools at both the elementary and high school level. These included one student measure—Safety—and four teacher measures—Parent Involvement, Teacher-Parent Trust, School Commitment, and Student Discussion.
-
Among elementary schools with strong 5Essentials results, high-poverty schools improved more than low-poverty schools on more than one-half of measures. Meaning, improvements in school climate and organizational features in high-poverty schools had a greater benefit for student learning than in low-poverty schools
Qualitative:
-
Practitioners reported that the data sometimes seemed opaque and difficult to use. Theyalso noted its focus on school principals made it difficult for school leaders and their teams to engage impartially with the data.
-
CPS granted schools considerable autonomy in how they used their survey data. However, principals acknowledged that the lack of district-wide strategy or guidance on how to interpret and act on the results hampered use of the data for improvement.
-
Principals credited partnerships with leadership coaches as valuable in helping them interpret and act on their survey data. Similarly, systematic collaboration by instructional leadership teams increased schools’ capacity to use the data.
-
5Essentials Survey results remain predictive despite concerns about the survey’s inclusion in the School Quality Ratings Profile (SQRP) school accountability system. At the same time, some principals depicted their priorities for school improvement as heavily influenced by SQRP and accountability, and some teachers reported that accountability affected how they responded to the survey. Overall, these findings demonstrate that while the predictive power of the survey was not affected, the potential for survey results to inform school improvement was often overshadowed by the incentives and pressures created by the survey’s inclusion in accountability policies.
While these qualitative findings demonstrate the tension of using formative assessments for both improvement and accountability, school climate surveys remain powerful tools for school improvement. Research shows that positive school climates are associated with improvement in numerous educational outcomes—including higher attendance, graduation, and college enrollment rates, lower suspensions, and increased learner engagement, motivation, and self- efficacy. School climate surveys like the 5Essentials Survey are becoming widely used in school improvement efforts, as districts look for ways to measure and improve climate to support student outcomes. It will be critical for CPS and other districts to set and communicate priorities and provide support to schools around using this data to ensure success.
“Chicago’s long-term use of school climate surveys has led to two important conditions. First, CPS is rich in data about school effectiveness and improvement. Second, the district is committed to using student/teacher voice data to guide school improvement. Now, CPS has anopportunity to be a national model for other districts by supporting that desire with a clear vision and equally strong tools and capacity so that the surveys truly can drive improvements, “said Holly Hart, UChicago Consortium Survey Director and the lead author of the quantitative study.
About the University of Chicago Consortium on School Research
With the goal of supporting stronger and more equitable educational outcomes for students, the UChicago Consortium conducts research of high technical quality that informs and assesses policy and practice in the Chicago Public Schools (CPS). We seek to expand communication among researchers, policymakers, practitioners, families, and communities as we support the search for solutions to the challenges of school improvement. The UChicago Consortium encourages the use of research in policy action and practice but does not advocate for particular policies or programs. Rather, we help to build capacity for school improvement by identifying what matters most for student success, creating critical indicators to chart progress, and conducting theory-driven evaluation to identify how programs and policies are working
