Principals can be reliable sources of teacher evaluations, consortium study finds

As schools across the country face mounting state and federal pressure to overhaul teacher evaluation, a pilot initiative in Chicago provides evidence that principals are able to assess teachers accurately on practices that drive student learning, according to a new study from the University of Chicago Consortium on School Research [1].

“Rethinking Teacher Evaluation in Chicago [2]” found that teachers who received the highest ratings from principals on classroom observations were also the teachers whose students showed the greatest learning gains. This suggests that principals were able to distinguish between strong and weak teaching, and that the observation tool used in the Chicago pilot, the Charlotte Danielson Framework for Teaching [3], captured factors that matter for student learning.

These findings have important policy implications for states and districts across the country working to implement evaluation systems that include classroom observations. Evaluations that rely on classroom observations provide teachers with a common definition of effective teaching and feedback on how they stack up on those criteria. They also can serve as the primary source of information on teacher quality in grade levels and subjects that are not tested.

The study is particularly relevant in states like Illinois, which has selected the Charlotte Danielson Framework as the state model.

“This study shows that we’re moving in the right direction with our redesign of educator evaluations in Illinois. It shows the observation methods we’re moving toward are valid and reliable measures of solid teaching practice, and that they can be applied consistently,” said State Superintendent of Education Christopher A. Koch. “The state is going to use the lessons learned in the Consortium study as we design the state’s training for principals which will be critical for the successful implementation of our new educator evaluation systems.”

“Rethinking Teacher Evaluation in Chicago” summarizes findings from a two-year study of Chicago’s Excellence in Teaching Pilot, which was piloted in 44 schools in 2008 and then expanded to 100 in 2009. Overall, the report found that the pilot was an improvement on the old evaluation system and worked as it was designed and intended, introducing an evidence-based observation approach to evaluating teachers and creating a shared definition of effective teaching. At the same time, the new system faced a number of challenges, including weak instructional coaching skills among some principals. Key findings include:

• The classroom observation ratings were valid measures of teaching practices; that is, students showed the greatest growth in test scores in the classrooms in which teachers received the highest ratings on the Danielson Framework, and students showed the least growth in test scores in classrooms where teachers received the lowest ratings.

• Principals and trained observers who watched the same lesson consistently gave the teacher the same ratings; the highest agreement existed for unsatisfactory ratings. This finding relates to the fairness of the system and has important implications for districts using the classroom observations to make hiring, firing and promotion decisions.

• Principals and teachers said that conferences were more reflective and objective than in the past and were focused on instructional practices and improvement.

• Over half of principals were highly engaged in the new evaluation system. These principals were positive about the Framework, the conferences and the professional development they received. Principals who were not engaged in the new evaluation system tended to say that it was too labor intensive given the numerous district initiatives being simultaneously implemented in their schools.

• While principal ratings were generally consistent, 11 percent of principals consistently gave lower ratings than the observers and 17 percent of principals consistently gave higher ratings than the observers. Also, while principals and teachers were positive about the evaluation conferences and framework, many principals lacked the instructional coaching skills required to have deep discussions about teaching practices. This speaks to the need for ongoing professional development supports for principals and teachers.

The Joyce Foundation [4] funded the report, which is available online at www.ccsr.uchicago.edu [5].