Choices Based On A Flawed System
The School Performance Framework Needs An Overhaul
In 2008, Denver Public Schools instituted the School Performance Framework that rates its schools on a number of metrics including growth, achievement, student and parent satisfaction. More recently, “academic gaps” was added.
Results from the SPF are released every year and categorize schools according to a color-coded system: Blue means “distinguished”; Green means “meets expectations”; Yellow means “accredited on watch”; Orange means “accredited on priority watch”; and finally Red, which means “accredited on probation.”
Schools labeled Red are in danger of being closed or restarted.
Supplying a multitude of data on each of the district’s schools, the SPF serves many purposes. It responds to federal and state legislation for both school and district accreditation. With teacher pay directly linked to student performance via a Colorado law passed in 2010, it also informs teacher pay.
Marriage made in heaven
The SPF has been promoted in parallel with the choice process to assist families in choosing the best schools for their children.
“In order for choice to work, it needs a market-based system,” said Jennifer Bacon, District 4 board director, at a recent meeting. “And it needs a marketing tool.”
Denver’s reform movement has capitalized on using both the SPF and school choice, a marriage made in heaven, to push its agenda forward. DPS explains its portfolio management style – part of the reform process – as follows:
“Through our family of high quality, diverse, autonomous public schools, including district-run traditional, district-run innovation, innovative management organizations (IMOs), innovation zones (iZones) and charter schools, we work to make these beliefs a reality – ensuring equity in school choice and accountability.”
Accountability links directly to the SPF. Working hand in hand, SPF and Choice have been used to guide parents and students toward some schools, and away from others since the implementations of reforms over a decade ago.
“It boils the entire school experience down to five colors, and, if we’re being honest, really down to pass/fail – either you are Green or Blue or you aren’t, and that is all anyone really cares about,” said Andrew Lefkowits, who heads the group Park Hill Neighbors for Equity in Education and whose children attend Stedman Elementary in north Park Hill.
“To think that all of the things that go into making a school experience meaningful, from the academic content, to the social skills, to the relationships with friends, with teachers, to the extra curricular activities … to think that could be boiled down to pass/fail – that everyone weighs all of those things the same way – does a disservice to very idea of a well-rounded education.”
Red, the color of shame
The formula used to calculate the SPF is opaque, difficult to understand and remains a mystery to most. More than 50 measures make up a typical school’s SPF score.
The color-coded system, which reduces those measures to one color per school, is overly simplistic and impacts schools that score lower with an aura of shame and ineffectiveness, much as the scarlet letter branded the heroine of Hawthorne’s novel.
The stigma associated with the color red works against schools by keeping students away. It lowers enrollment and thus, per-pupil funding also decreases. Decreased funding leads to fewer resources. This means teachers and students struggle even more to improve test scores.
Bacon says that though the recent SPF marked Stedman as a Red school, it did not highlight all the work that its newest principal, Greta Martinez, has done. She noted that in recent discussions about the school, emphasis was put on how it was currently doing, noting that Martinez was not responsible for the six months of data produced before she arrived at the school – but is still included in the rating.
“SPF is slow,” said Lefkowits. “It’s an average of last year and the year before’s data. People using it to choose a school for next year end up with a four-year gap between at least half of the data they are using and when they get to a school. We know that schools can change dramatically in four years.
“This benefits already privileged schools, as they tend to have less teacher turnover, less administration turnover, and more consistent student populations (both fewer kids leaving or coming mid-year, and more consistent demographics). For a school that has had any change – new leadership, changing student body, new program, etc. – we wouldn’t expect to see that show up on SPF for at least two years, and more likely three to four years.”
Growth versus status
Mostly, the SPF measures students performance on state mandated tests administered each year, with emphasis on growth (how much students improve) over status (proficiency or level at which students perform). In 2018, for example, schools could obtain 172 points for growth versus 55 points for status. That is a 3-to-1 ratio.
Critics say that an over-emphasis on growth over status can mask the fact that students who have improved over the course of a year are still reading or doing math below grade level. For example, in the latest SPF ratings, DSST Green Valley Ranch Middle School scored Green while Skinner Middle School scored Yellow. But scores on the CMAS state tests reveal that achievement at DSST GVR (48 percent are meeting or exceeding expectations in English Language Acquisition and 31 percent in Math) is far below that of Skinner (60 percent in ELA and 38 percent in Math).
Bacon says that growth is important but is careful to point out that the SPF mainly reveals how the students in a school scored on one particular day. The question is, “Do we want this tool to aid in Choice or do we want it to reflect one particular day?” she said.
13 groups push back
The SPF has seen many changes and has faced resistance since its implementation 10 years ago. More recently, it came under attack in 2014 when 13 organizations – including A+, the Donnell-Kay Foundation, the Gates Family Foundation, DSST Public Schools, Teach for America, Stand for Children, Latinos for Education Reform, and former DPS board member Jeannie Kaplan – asked DPS “to revise the SPF criteria and thresholds so district staff, parents, and the general public have a clearer understanding of the definition of a quality school.”
The need to define “quality school” is echoed by Bacon. Is a school defined by its SPF or is there more to it? Community values, she says, are also important. But how do we evaluate those?
In December, 2017, the SPF again came under attack. Noting that the district was significantly overstating literacy gains in elementary schools, six civil rights organizations – the NAACP, Padres & Jóvenes Unidos, the African Leadership Group, Together Colorado, Alpha Phi Alpha, Inc., and the Urban League of Metro Denver – called on DPS to correct its SPF ratings. The letter asked that distorted information be changed before the Choice season opened. DPS did not change the ratings.
The academic gap indicator
In 2017, a new metric was added to the SPF: the academic gaps indicator. This measure is meant to underline how historically underserved students – including students of color, students in poverty, English language learners and students with special needs are doing compared to better served students (white students).
Parents have complained to the school board that the academic gap indicator penalizes schools. Complaints stem from the fact that schools fall in their rankings and that this negatively impacts schools.
Bacon’s criticism of the gap indicator is that, “We are not assessing what the schools are doing; we are just highlighting the result.” She notes the need to provide concrete supports for schools that have large achievement gaps between groups of students.
Teachers call to end the SPF
In October 2018, DCTA teachers put out a forceful statement calling for an end to the SPF all together:
“The Denver Public Schools is the only school district in the state that has developed its own School Performance Framework rating, with the others all following the School Performance Framework system created and used by the Colorado Department of Education. Since the SPF ratings were released by DPS last week, there has been an outcry of anger by teachers and school leaders who feel that the system is deeply flawed or even purposefully manipulated each year by the District. Today, the Denver Classroom Teachers Association officially calls on DPS to end its SPF system and solely use the system created and maintained by the Colorado Department of Education.”
Reviewing invoices provided by DPS, taxpayers have paid close to $1.5 million in the last 18 months to RevGen, a business and technology consulting firm, for maintaining the SPF.
Much money has flowed to create and maintain the SPF as well as promote it through the Choice system. In early December DPS’ Communications Office sent out an email alerting parents to the Choice process, noting that it is “as easy as Find, Apply, Register.” The email links to enrollment guides that simplistically highlight the schools’ color ratings. There is no explanation of the complexities or failings of the SPF system.
In addition, both the enrollment guides and the DPS Choice website link to School Finder, a DPS tool to assist parents in finding their best options.
However, this website does not accurately reflect the latest SPF scores. Hallett Academy, for example, is still labeled as Red when it has made considerable progress and is now Orange. Stedman is labeled as Orange, when it has dropped to Red. Smith is labeled as Orange when it has moved up to Yellow. And Roots Elementary still appears as a possible choice, though the school announced in November it is closing at the end of the school year.
Given the amount of money poured into maintaining the SPF and promoting the Choice system, the fact that emails and brochures go out with links to out of date information is egregious negligence. At the very least, the website should be updated.
Is the money spent providing us with clear facts by which we can assess schools given how it is used?
Changes under Cordova?
“The SPF is a tool of the executive,” said Bacon, “And the executive is now changed.”
On Dec. 17, the DPS school board unanimously appointed Susana Cordova, formerly deputy superintendent for DPS, as the new superintendent.
On her public Facebook post, Carrie Olson, District 3 board director, explained her vote to support Cordova and called on her to “re-envision the SPF.”
How will Cordova use this executive tool? Will it be to hold the line and continue the reforms that were implemented by former superintendents Tom Boasberg, and before him Michael Bennet? Or will Cordova use it to reshape the district and take it in a new and different direction?
At a recent community meeting, before her appointment as superintendent, Cordova spoke of the SPF as a system that was too complicated and asked if there were ways to have multiple steps included when talking about “school quality.”
It is now in her power to seek a real answer to this question and take concrete steps for students, teachers, and schools to be supported by a system that has fewer flaws and reflects more accurately the state of DPS schools.
Lynn Kalinauskas is Education Chair for Greater Park Hill Community, Inc.