“You have to draw the line somewhere” - Anonymous

Setting targets and constructing trajectories using a data-driven approach is a critical, but often under-utilized, tool that, when done well, enables constant learning about your system’s progress toward its goals for students. At the core of targets and trajectories lies a simple concept: a properly defined, well-thought-out target should increase the likelihood of accomplishment, by clearly connecting the state’s aspiration to facts on the ground. Good targets are defined by five key characteristics that comprise the SMART framework. They should be:

• Specific. Target has clearly defined meaning and implications for expected change.
• Measurable. A concrete standard for measuring progress should be established.
• Ambitious. Target should feel like a stretch from the current level of performance.
• Realistic. Target should be grounded concretely in affecting factors, making it more than just a guess.
• Time limited. Target should have a defined deadline, both to create urgency and accountability.

After a target has been set, the next step is to estimate a trajectory for a given aspiration, from the current level of performance to the level suggested by the target. Trajectories should be thought of as hypotheses – your best educated guess – about the levels of performance your system will achieve en route to achieving its overall target.

Context and Challenge

As one of two first round Race to the Top (RTTT) winners, Tennessee sat squarely in the national spotlight as a leader in state education reform in March 2010. In the RTTT plan, the state set high expectations for student performance, and state leaders were determined to reach those targets, including a 90% high school graduation rate for all student subgroups by 2014-15.  If that goal were achieved, Tennessee would graduate approximately 6,000 additional students in the next five years.  With stakes higher than ever, the Tennessee Department of Education (TDOE) recognized that in order to achieve the statewide graduation rate goal, it would need support from all 136 local school districts. Therefore, the Education Delivery Unit (EDU) decided to suggest targets for each district, requiring that all districts improve, regardless of current performance.  That meant that each district’s target was set relative to its baseline, regardless of whether a district was far below or far above the state average. State leaders firmly believed that unless a district was graduating 100% of its students, there was room for growth.

State leaders recognized they would need to approach the task of setting district targets delicately so as not to create the sense that they were a top-down mandate.  The EDU faced a substantial challenge: How would they set sufficiently ambitious targets that ensured the statewide goal for high school graduation rate would be achieved without engendering push-back from districts, especially from those from whom the greatest amount of improvement needed to be seen?  In particular, how could the state ensure its large, urban districts (namely, Memphis, Metro Nashville, Knox and Hamilton) with a track record of underperformance were setting targets high enough for the statewide target to be met? It became abundantly evident that without substantial improvement from these and other large, low-performing districts, the state would fall short of its target.

With great variance in the level of district performance across the state, the EDU’s primary challenge was coming up with “SMART” targets for each district based on their historical performance, demographic composition and planned strategies.  Differentiation among districts was not a familiar approach in the TDOE, and the Department risked the potential of major push-back from districts about both the target itself being too ambitious, as well as the fact that the state was asking some districts to make greater gains than others.

Key Actions

Tennessee’s EDU played an influential role in setting high school graduation rate targets for their local districts. The EDU, with analytic support from EDI, examined system-wide and district performance utilizing historic trends, baseline data and targets to project the impact of certain scenarios on the statewide graduation rate. The EDU provided graduation rate data by district for school years 2007-2009, from which EDI calculated and sorted districts into quintiles according to their 2009, or baseline, rate. A number of variables upon which to base district targets were explored, including poverty rate and net change graduation rate over time. The EDU ultimately decided to use 2009 baseline rates as the mechanism by which to sort districts and make projections.

Once districts were sorted into performance tiers, the EDU asked the question, “Given districts’ past performance, and the performance of their peers, what can or should we expect future performance to be?” Next, with help from EDI, they designed a number of “what if” scenarios using data-driven hypotheses to project the relative contribution needed from each district in order to meet the statewide target.  Some of these scenarios are illustrated in the table below.

As the table suggests, it required a bit of back and forth analysis to identify a scenario that resulted in the state meeting its 90% target. One of the most powerful elements of the analysis was translating the graduation rate from a percentage to actual students. Not only did it help contextualize data that was often difficult to comprehend, but seeing the actual numbers of students needed to reach the statewide target added a sense of urgency and moral purpose to the work. EDI shared completed analyses with the EDU. EDU staff then determined which scenario made sense both pragmatically (could each of these districts realistically increase their graduation rates by the projected amount?) and politically (would there be push-back and if so, did the State have data-based evidence to bolster the rationale for districts’ targets?).

The analysis resulted in “SMART” graduation rate targets for each district that collectively, if met, would result in achievement of the statewide goal of a 90% graduation rate by 2014-15. While this gave the state confidence that it would be able to achieve its goal for students, there was one next step of paramount importance – getting buy-in and ownership over these targets from districts. This step required a bit of strategic thinking on the part of the EDU. With politics in mind, the EDU decided to make the state targets suggested, rather than mandatory. They used their regional staff that had previously established relationships with the districts to disseminate the suggested targets, and crafted a carefully worded memo to accompany each district’s suggested target. The note emphasized the moral purpose underlying the proposed targets, as well as the “all hands on deck” approach needed to raise the bar for students. Importantly, the note also recognized the support role the state would play in helping districts reach their target.

Results

Because targets were carefully crafted and communicated, over 90% of districts ultimately accepted the State’s suggested targets.  These districts then constructed their trajectories and district plans based on those numbers, and understand the gains they’ll need to make each year in order to be successful. Further, the State engendered the support of several key districts as a result of this process and opened the door for future collaboration.

If each district is successful in reaching its defined target by 2015, the state will graduate more than 5,545 additional graduates, ultimately changing the lives of thousands of Tennessee’s students.

For More Information

Deliverology 101, Section 3B