HOT OFF THE PRESS! We have received our formal evaluation outcomes for the first year of our Investing in Innovations (i3) grant! The positive outcomes speak volumes of the high quality education that our school partners are providing our Bay Area youth, and all of YOUR support in ensuring that every student receives a truly Unconditional Education.
The study was conducted by SRI International. The lead evaluator, who has conducted over forty such evaluations during his career, said that these outcomes are some of the most compelling he has ever seen.
The formal evaluation questions for our i3 grant include:
After one, two, and three year(s) of implementation, compared to similar students in schools that did not participate in UE, did students who attended schools that participated in the UE program demonstrate:
The study was conducted by SRI International. The lead evaluator, who has conducted over forty such evaluations during his career, said that these outcomes are some of the most compelling he has ever seen.
The formal evaluation questions for our i3 grant include:
After one, two, and three year(s) of implementation, compared to similar students in schools that did not participate in UE, did students who attended schools that participated in the UE program demonstrate:
While we had a positive impact on academic outcomes (particularly math) overall, some of the greatest areas of impact can be seen when looking at specific subgroups of students:
This is a particularly important subgroup because UE has an important focus on improving the academic achievement, inclusion, behavior, and social-emotional well-being of students with disabilities. In addition, the project was funded under the “Students with Disabilities” i3 priority. While the comparison design study did not find any statistically significant results in year one, students with disabilities did realize statistically significant growth in reading and comprehension skills as outlined in the supplemental evaluation below.
The results for African American students were mixed, resulting in negligible effects when looking at Oakland and SFUSD students combined. In Oakland, African American students experienced small to moderate positive effects in mathematics (.27).
The ability of the UE model to effectively serve Latino students is crucial, as nearly 80% of students receiving UE in Oakland schools are Latino. Overall, Latino students experienced positive effects in reading (.19) and mathematics (.23). When looking at Oakland students, specifically, Latino students experienced positive effects in reading (.22) and math (.29). The impact on ELA mathematics achievement of UE was found to be statistically significant for Latino Students within Oakland, and for the overall, combined set of Latino students.
Similarly, the ability of the UE model to effectively serve English Language Learners (ELLs) is crucial, as many schools participating in the project serve high percentages of ELLs (nearly 50% of students in Oakland). Overall, ELLs experienced positive effects in reading (.34) and mathematics (.34). When looking at Oakland students, specifically, ELLs experienced positive effects in ELA (.32) and math (.36). The impact on ELA and math achievement of UE was found to be statistically significant for English Language Learners within Oakland, and for the overall, combined set of ELL students.
SUPPLEMENTAL EVALUATION FOR STUDENTS WITH DISABILITIES
Data for this supplemental evaluation came from the direct assessment of students with disabilities in the five participating Oakland schools. The assessment tool from the Special Education Elementary Longitudinal Study (SEELS) contains research versions of the Comprehensive Test of Phonological Processing (CTOPP), Woodcock-Johnson Tests of Achievement, the Student Self-Concept Scale, and the Attitudes Toward School Scale. The baseline (pre-) assessment was completed before the UE model was implemented, in the spring of 2014, and the post-assessment was implemented during the spring of 2015.
- Students with disabilities showed improvement on 9 of the 13 outcomes, with four of these showing statically significant improvement
- Within the Literacy measures, students showed statistically significant growth on word segmentation, passage comprehension, and two measures of oral reading fluency.
- No outcome measures for school attitudes or self-concept showed statistically significant improvement, although the positive effect sizes associated with social (.17) and overall self-concept (.24) were sizably different from zero.
These outcomes qualify us to apply for a $12 million dollar Investing in Innovations (i3) Validation grant this summer. Keep your fingers crossed!
Blog post written by:
Jenny Ventura, Director of Model Implementation and Assessment
Jenny Ventura, Director of Model Implementation and Assessment
RSS Feed