Introduction
This executive summary highlights the implementation and results of a pilot evaluation of LRF! conducted in Jordanian primary schools in 2021-2022. The purpose of the pilot evaluation was to test the intervention’s feasibility and evidence of promise as well as assess its readiness for an efficacy trial. The pilot was also intended to provide preliminary evidence on the impact of LRF!, the mechanisms of change, and lessons to inform future scale-up.
Study Overview:
Establishing a strong foundation in literacy skills for children is crucial to support their subsequent academic development (Zakaria et al., 2021). Arabic language learners face challenges in learning to read and write in Modern Standard Arabic, as it differs from the colloquial language spoken at home ("Abadzi," 2017). Data from the 2018 Early Grade Reading Assessment (EGRA) indicates that only about 19% of second and third-grade students meet or exceed the minimum levels of reading proficiency. Children may continue to face these early reading difficulties, limiting their ability to reach their full potential in the future.
The British National Centre for Social Research (NatCen), in partnership with Integrated International, conducted two Randomized Controlled Trials (RCTs) to evaluate the "Let’s Read Fluently" program. This was funded by the Queen Rania Foundation for Education and Development (QRF), supported by the Education Endowment Foundation, and in partnership with the BHP Foundation. Both studies applied different models for program implementation as follows:
First Pilot Study (2021-2022): This pilot study aimed to explore the potential impact of the intervention on students’ reading abilities in the early primary years (grades 1-3) and assess the feasibility of conducting a large-scale randomized trial to better evaluate its effects. Two models were implemented during the first pilot phase, each lasting for one semester:
Whole-Classroom Model: Targeting the entire first grade, where the classroom teacher applied the intervention material with all students three times a week for 30 minutes per session during the second semester.
Catch-Up Model: Targeting small groups of students from the first three grades identified as the bottom 20% in their class through an assessment tool. In this model, the resource room teacher worked with second and third graders in the first semester and with first graders in the second semester for 30 minutes, three times a week, as in the whole-classroom model.
Findings:
Evidence of Promise: The pilot evaluation showed evidence of promise for the whole-classroom model, with impact evaluations suggesting that this model could improve students’ literacy attainment. However, in the case of the catch-up model, there was no evidence of improvement. The pilot study had a small sample size and was not designed to measure impact robustly.
Feasibility of Implementation: The "Let’s Read Fluently" program was feasible to implement, and most schools delivered it as intended with minor adjustments. The pilot identified several potential improvements needed for the interventions, including adjustments to the training of resource room teachers in private/low-performing schools, coordination of program delivery, content of the training manual, and the process of selecting students from low-performing schools. Overall, the pilot results suggest that the whole-classroom model could potentially be scaled up to a full-scale efficacy trial.
Second Pilot Study: Catch-Up Model (2023-2024): Since the previous study found no evidence supporting the catch-up model ("Dimova et al., 2023"), this study re-evaluated the catch-up model with a modified approach that addresses the challenges of the previous model, to determine if this approach shows evidence of promise. The model targeted 'struggling readers' by supporting a small, selected group of students lagging behind their peers in Arabic literacy skills.
Findings from second study: The evaluation report results will be published in the winter of 2024.