Case Study: Lessons on Evaluation and Performance-based Compensation from Washington, D.C.’s IMPACT system

Context

2007 was a pivotal year for D.C. Public Schools (DCPS). Chronically struggling schools and low student performance — only 12 percent of eighth graders were proficient in reading — spurred D.C. into action. At the urging of newly elected Mayor Adrian Fenty, the D.C. Council passed the Public Education Reform Amendment Act, which placed the DCPS under mayoral control. To run the schools, Mayor Fenty appointed Michelle Rhee, who as school chancellor implemented several reform efforts, including closing under-enrolled schools and developing a rigorous teacher evaluation system known as IMPACT. IMPACT served as the basis for a performance-based pay system, IMPACTplus. IMPACT is distinctive for multiple reasons, including its:

• Use of multiple indicators, including impact on student achievement and multiple classroom observations, to assess teacher quality;
• Multi-tiered framework that allows the district to differentiate educators into five categories based on performance; and
• Link to high-stakes consequences and significant financial incentives in an effort to reward and retain the most effective educators.

About IMPACT and IMPACTplus

Launched in 2009, IMPACT was designed to set clear expectations for educators and provide meaningful data to guide practice. The previous evaluation system, the Professional Performance Evaluation Process (PPEP), did little to differentiate teachers by linking them to impact on student achievement. The same year that 12 percent of eighth graders were proficient in reading (2007), 95 percent of DCPS teachers were meeting or exceeding expectations.¹² Student performance did not appear to meaningfully inform teacher evaluation ratings.

IMPACT, however, makes teacher evaluation more comprehensive by using several factors. All teachers are now evaluated based on student academic growth, although specific measures and the weighting of each measure vary by teacher subject and grade. For example, those who teach students taking statewide math and reading exams will receive an individual value-added score. For other teachers, DCPS assesses impact on student achievement through other assessments. All teachers also undergo multiple classroom observations during the year by both school administrators and independent practitioners. Classroom observations are guided by district’s Teaching and Learning Framework, which outlines expectations for effective instructional practices.³ Finally, a teacher’s level of collaboration and professionalism, as measured by school administrators according to district rubrics, are factored into final evaluation ratings.⁴

Based on stakeholder feedback, DCPS has also made two major changes to the IMPACT evaluation system since its launch in the 2009–10 school year.⁵ First, the weighting of each evaluation component has changed over time. During the first three years of implementation, math and reading teachers eligible for a value-added score had 50 percent of their total score depend on value-added metrics. Beginning in the 2012–13 school year, the weighting for value-added scores decreased to 35 percent. Second, DCPS added another rating called “Developing” for teachers who are underperforming compared to “Effective” and “Highly Effective” teachers. Teachers with a “Developing” rating, however, are still more effective than “Minimally Effective” or “Ineffective” teachers — and thus have a longer grace period to show improvement.⁶

The IMPACT model’s design and incorporation of student learning outcomes are highly promising. Developed through a collaborative process and launched simultaneously with the IMPACT model, the Teaching and Learning Framework establishes a common set of standards and language for all schools, helps to guide professional development for teachers, and supports transparency in evaluation.⁷

In addition, DCPS’s model also offers an example of the use of teacher evaluation data in making performance management decisions. IMPACT evaluation results now lead to high-stakes consequences for low performers as well as performance bonuses and salary increases for the district’s most effective teachers. The district’s worst teachers — as well as teachers who underperform for multiple years in a row — are subject to dismissal. On the other end of the spectrum, the highest-performing teachers are eligible for annual bonuses and base salary increases based on IMPACTplus, DCPS’s performance-based pay system. The structure of outcomes based on the different ratings in the IMPACT model are:

• Ineffective: Teacher is subject to dismissal
• Minimally effective: Teacher in category after receiving two years of support is subject to dismissal
• Developing: Teacher in category after receiving three years of support is subject to dismissal
• Effective: Teacher progresses normally on pay scale
• Highly effective: Eligible for annual bonus of up to $25,000 and base salary increases up to $27,000.⁸

With IMPACTplus, DCPS offers highly competitive teaching salaries compared to other districts in the country. Under the old system, the highest maximum salary was just over $87,000; that figure has risen to $130,000 today.⁹

Outcomes

Now in the sixth year of implementation, D.C.’s evaluation and performance pay systems have yielded positive results in teacher quality — and potentially in student achievement as well. A 2012 TNTP report, The Irreplaceables, analyzed teacher retention in urban districts, finding that that DCPS retains a higher percentage of its best teachers compared to other school districts.¹⁰ In 2013, Dee and Wyckoff found that IMPACT helps teachers improve and that DCPS is retaining its best teachers. For example, teachers who received a “Minimally Effective” rating improved their performance the following year at higher levels than did teachers who were “Effective” or above. In addition, “Highly Effective” teachers eligible for a permanent salary increase also significantly improved their IMPACT scores. These results indicate that the potential of either a dismissal threat or financial incentive may affect teacher performance. Dee and Wyckoff also found that teacher retention varies by IMPACT category; those who are “Minimally Effective” are less likely to voluntarily stay in the system than their higher-performing peers.¹¹ A subsequent study released in 2016 found that turnover among teachers identified as low-performing by DCPS on average positively influenced student achievement, and the exit of teachers DCPS identified as high-performing negatively affected student achievement.¹² Together, these findings suggest that IMPACT improves the effectiveness of the DCPS teacher workforce. Additionally, students in the District of Columbia posted statistically significant gains in NAEP scores in both reading and math in grades 4 and 8 between 2009 and 2013, including among the largest increases in average scores in math of any state.¹³

Policymakers can look to the experience of DCPS as an example to inform development of evaluation systems or pay-for-performance systems. IMPACTplus, in particular, represents a data-driven system of rewarding high-quality teachers serving students most in need. RTT guidelines — and subsequent state policy proposals — have modeled substantially off of IMPACT. In addition to Washington, D.C., Tennessee, Louisiana, and Florida have also implemented strong teacher evaluation models that utilize objective measures of student achievement to meaningfully differentiate teachers.¹⁴