Empirical Benchmarks of Hidden Bias in Educational Research: Implication for Assessing How well Propensity Score Methods Approximate Experiments and Conducting Sensitivity AnalysisReportar como inadecuado




Empirical Benchmarks of Hidden Bias in Educational Research: Implication for Assessing How well Propensity Score Methods Approximate Experiments and Conducting Sensitivity Analysis - Descarga este documento en PDF. Documentación en PDF para descargar gratis. Disponible también para leer online.



Society for Research on Educational Effectiveness

When randomized control trials (RCT) are not feasible, researchers seek other methods to make causal inference, e.g., propensity score methods. One of the underlined assumptions for the propensity score methods to obtain unbiased treatment effect estimates is the ignorability assumption, that is, conditional on the propensity score, treatment assignment is independent of the outcome. The purpose of this study is to use within-study comparisons to assess how well propensity score methods can approximate experiments under various conditions. The authors test three ways of constructing comparison groups: (1) using the sample from the states that are different from the original experiments, with pretest and demographic information; (2) using the sample from the same state and districts (local matching) with the experiments, with demographic information only; and (3) same as (2) but with pretest as well. Propensity score methods (optimal matching, propensity score weighting, and stratification) are used to estimate the treatment effects for three ways of constructing comparison groups, which are compared with the benchmark from the experiment to assess estimate bias. This study will contribute to the literature by providing empirical evidence about how well propensity score methods can approximate experiments under various conditions. This study uses data from four IES funded projects, among which three are large scaled experiments: (1) "Scaling up TRIAD: Teaching Early Mathematics for Understanding with Trajectories and Technologies"; (2) "Evaluating the Effectiveness of Tennessee's Voluntary Pre-Kindergarten Program"; and (3) "Experimental Evaluation of the Tools of the Mind Pre-K Curriculum", and one is a measurement study, "Learning-Related Cognitive Self-Regulation School Readiness Measures for Preschool Children Study". The analysis is undergoing. The authors report partial results in this report. Tables and figures are appended.

Descriptors: Educational Research, Benchmarking, Statistical Analysis, Computation, Comparative Analysis, Experiments, Statistical Bias, Program Evaluation, Curriculum Evaluation

Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries[at]sree.org; Web site: http://www.sree.org





Autor: Dong, Nianbo; Lipsey, Mark

Fuente: https://eric.ed.gov/?q=a&ft=on&ff1=dtySince_1992&pg=5070&id=ED562732







Documentos relacionados