Measuring Critical Education Processes and Outcomes: Illustration from a Cluster Randomized Trial in the Democratic Republic of the CongoReport as inadecuate

Measuring Critical Education Processes and Outcomes: Illustration from a Cluster Randomized Trial in the Democratic Republic of the Congo - Download this document for free, or read online. Document in PDF available to download.

Society for Research on Educational Effectiveness

Using reliable and valid measures of students' outcomes which are sensitive to change is critical for obtaining interpretable and therefore useful results from evaluations of school-based interventions. While measurement development for use in experimental evaluations receives a great deal of attention in the U.S., it lags behind in low-income countries. The Early Grade Math Assessment (EGMA, RTI, 2009) and Early Grade Reading Assessment (EGRA, RTI, 2009b) were developed by RTI international to provide snapshots of students' emerging math and reading abilities in low-income countries. The assessments use a series of discrete sub-tests (e.g., number identification, quantity discrimination, phonemic awareness, familiar/unfamiliar word decoding) and are increasingly employed by governments and Non-Governmental Organizations (NGOs) to diagnose the status of children's abilities and to fine tune interventions to meet the academic needs of children. As the popularity of evidence-based practices grows, EGMA and EGRA are also being used in experimental evaluations of school-based interventions. However, the use of multiple sub-test scores may inflate the risk of capitalizing on chance and undermine power to detect intervention impacts. The goals of the study are to describe and discuss the authors' conceptual and analytical approaches to developing valid and internally consistent measures of social-emotional processes and academic outcomes for use in a cluster randomized trial of Opportunities for Equitable Access to Quality Basic Education (OPEQ) in the Democratic Republic of the Congo (DRC). First, the authors present their approach to developing summary scores of children's early math and reading abilities. Second, they present psychometric analyses used to refine, reduce, and examine the underlying structure of scales intended to measure children's perceptions of school quality and socio-emotional well being. The analyses in this paper illustrate approaches to measuring children's academic outcomes, perceptions of school quality, and socio-emotional well being, as part of an impact evaluation in the Democratic Republic of the Congo. As such, they contribute to advances in devising more reliable and valid instruments to inform the development of increasingly effective educational interventions in diverse parts of the world. These findings provide a model for future use of evaluative educational instruments in low-income countries, and confirm the importance of critically evaluating such instruments for use in new populations.

Descriptors: Foreign Countries, Outcome Measures, Outcomes of Education, Cluster Grouping, Randomized Controlled Trials, Test Validity, Test Reliability, Well Being, Scores, Educational Quality, Elementary School Students, Grade 2, Grade 3, Grade 4, Mathematics Tests, Reading Tests, Nonverbal Ability, Reading Ability, Psychometrics, Student Attitudes

Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries[at]; Web site:

Author: Halpin, Peter F.; Torrente, Catalina


Related documents