Evaluating Teachers and Schools Using Student Growth Models
William D. Schafer, Robert W. Lissitz, Xiaoshu Zhu, Yuan Zhang, University of Maryland Xiaodong Hou, American Institutes for Research Ying Li, American Nurses Association
This is a highly technical paper, which compares the volatility and reliability of several different models but here is one of the conclusions:
” We cannot at this time encourage anyone to use VAM in a high stakes endeavor. If one has to use VAM, then we suggest a two-step process to initially use statistical models to identify outliers (e.g., low-performing teachers) and then to verify these results with additional data. Using independent information that can confirm or disconfirm is helpful in many contexts. The value of this use of evaluative change results could be explored in further research efforts….
Perhaps a better way to conceive educational program success is to characterize the challenges faced by educators and to compare programs based on success in meeting those specific needs (as suggested by Goe et al., 2008). For example, urban schools, rural schools, and suburban schools exist in distinctly different environments and expecting a variable such as a pretest score to represent a common construct among them seems unrealistic.
Besides geography, variables such as socioeconomic status, individual aptitude, home environment, and per-pupil expenditure while associated with each other, nevertheless may all be needed to represent institutional (school or individual teacher) challenge adequately. Constructing models to incorporate variables such as these and comparing outcomes with programs that have common environments may prove to have more value than VAM.”