Impact Studies

Studying Program Results

TEG designs rigorous evaluation studies using design parameters that consider units of analysis, sample size, anticipated outcome, propensity score matching variables, and statistical analyses to provide confidence in reaching MDES estimates.

Quasi-Experimental Design (QED)

QED is a rigorous research study that involves testing variables on specific groups without using a random pre-selection process. Each unit in the program group is matched with one or more comparison units based on known matching variables (i.e., presenting problems, demographics). Propensity Score Matching produces a unidimensional index that can be used to minimize differences across many variables considered concurrently. QED is commonly used in the field of education so that interventions are not withheld from specific groups.

Randomized Controlled Trial (RCT)

RCT is a rigorous research study that involves testing variables on specific groups that were randomly assigned to either the treatment or control group. Units (i.e., students, teachers, classrooms, schools) are randomly assigned to a program group or a control group using a “simple” random assignment (lottery) or stratified (blocked) random assignment in which units are put in discrete groups based on common characteristics prior to being randomly assigned. RCT is more difficult to implement, but is considered the “gold” standard.

Interrupted Time Series with Comparison Group (C-SITS)

C-SITS uses post-implementation data compared at regular intervals with data collected prior to the program. Differences in slope and intercept across pre-program and post-program years are compared with differences in slope and intercept in a comparison group. This is a “difference in differences” design and is most suitable to use at the school level.