OBJECT LESSONS: The Elusive Connection Between Policy and School Improvement

By Pamela Burdman

It’s a familiar story. An education reform strategy is hailed as the beginning of a new day. Fast forward a few years. Policies and practices may have changed, but it’s not clear that students are learning more. Then the next policy comes along, promising to have the solutions.

Today, the optimism centers on federal dollars flowing into longitudinal data systems, charter school expansion, and new multi-state math and English assessments. The film Waiting for Superman features reformers firing ineffective teachers and offering incentives for those who succeed in raising students’ test scores. With advocates passionately pressing their own solutions, waiting for an opening to implement them, the failure of past policy is often seen more as an opportunity than a cautionary tale.

The research community is often implicated indirectly in this cycle. Education reform advocates typically cite some research evidence for their favored approach, even as others point to different, contradictory, research to oppose it. Less common is the research that probes more deeply into the failure of a policy: Were its assumptions wrong? Was the implementation faulty? Was the policy a poor fit for the existing political arrangements? Or did it simply need more time?

The Spencer Foundation, with its primary focus on funding research “for the purpose of making education better” has a particular interest in understanding how research can support effective policies. Given that reformers have yet to crack the code on school improvement, Spencer President Michael McPherson says, it may well be that researchers need to start asking some new questions.

“When people think about policy, federal or state, they tend very quickly to adopt as their point of view ‘what should the federal government accomplish?’ or ‘what should state governments accomplish?’ and then they fight ideologically about the ‘shoulds’,” McPherson noted. “There is much less tendency to ask the ‘can’ questions: ‘What is the government capable of doing, and what are the strengths and limitations of the tools that the federal government and other actors have in advancing educational improvement?’ Ultimately, education involves the work of teachers transforming human beings who are embedded in their local schools and communities. Finding levers governments can pull on to shape that work is a formidable task.”

For stimulating his thinking on these issues, McPherson credits The Ordeal of Equality, a 2009 book by David K. Cohen of the University of Michigan and Susan L. Moffitt of Brown University. The question posed by the book’s subtitle, Did Federal Regulation Fix the Schools? is a rhetorical but sobering one to anyone with a passing knowledge of U.S. school reform. The authors’ extensive account of why federal education spending under the Elementary and Secondary Education Act hasn’t achieved its aims describes a dynamic in which the federal government adopted increasingly ambitious goals in response to rounds of perceived policy failure, resulting in a growing gap between goals and results.

“The book is a history of the Elementary and Secondary Education Act,” said McPherson. “But it’s actually much more than that, in that it raises pretty basic questions about how it is that genuine improvement in education comes about. It’s a hugely important set of questions.”

The possibility that there are goals that policy makers cannot achieve with the tools they have at hand, or within the contexts in which they are situated, is too consequential for education reformers and researchers to ignore, in Spencer’s view. With that in mind, the Foundation devoted its annual Bixby Seminar to a discussion of the Cohen and Moffitt book. The fifty education scholars and experts who filled the University of Chicago conference room seemed to treat the topic with equal gravity, listening with rapt attention over the course of an hour as Cohen unpacked the book’s analysis.

Underlying Cohen’s argument was an insight about how policy in general works. “Policymakers are not the ultimate problem-solvers,” he explained to the audience. “Policy depends on those organizations and people with the problem to solve the problem.”

That, he claimed, is precisely where education policy can go wrong, at the very design phase, when its aims diverge from the capacity of the schools to change themselves. Absent this capacity, federal funding or mandates won’t suffice to improve schools. Cohen called this a “built-in capability problem” that is exacerbated by traditions such as federalism, separation of powers, and localism that combine to make U.S.-style government particularly weak. Given that education ultimately occurs behind millions of individual classroom doors, to expect any single policy to readily influence students’ educational performance is naïve.

In fact, most education experts believe that improving schools entails improving teaching. There is a strong argument, made by Cohen and others, that improving teaching in turn requires some form of professional learning in which teachers can build their skills. But the absence of common curricular frameworks in which to train teachers creates a systemic barrier to such training, which Cohen called “one of the most profound weaknesses” of U.S. schooling. Instead of grappling with these questions, however, federal policies simply assumed away the capability problem. The thinking seemed to be that teachers could be persuaded to teach better – rather than taught or otherwise supported to do so. “One of the underlying ideas which attaches especially to the push for accountability is that the problem in schools was more a problem of will than of skill,” he said.

As spelled out in Cohen’s matter-of-fact style, these observations sounded patently obvious. And yet, because his and Moffitt’s investigation penetrated a level where education researchers rarely venture, they provide a simultaneously fresh and compelling look at the Clinton-era Improving America’s Schools Act and George W. Bush’s No Child Left Behind Act. Consider this forceful passage, explaining why Cohen and Moffitt dubbed these reforms, which relied on Title 1 for leverage, a “faith-based initiative”:

They rested on neither strong evidence nor extensive experience but rather on an extraordinary belief in the power of particular policies. More precisely, Title 1 owed its new form to a potent combination of despair, hope, and ignorance: despair about schools and the modest gains that they had produced with Title I; hope for better education for all American students, but especially those from poor families, and for an entirely new sort of policy; and ignorance about the problems that policy was certain to encounter. Policymakers and educators now struggle with the consequences.

Making those consequences and their lessons part of the mainstream education research agenda is a goal for Spencer. In addition to the Bixby talk, the Foundation is partnering with the American Enterprise Institute on a conference in May that will further examine the federal role in education research, innovation, accountability, and equity. The event, Carrots, Sticks, and the Bully Pulpit: Sobering Lessons from a Half-Century of Federal Efforts to Improve America’s Schools, will feature a series of papers to be published in 2012.

“There’s a huge inclination to dismiss problems of program design and problems with the federal role as implementation goof-ups,” said Rick Hess of AEI, who is organizing the conference. “We’re seeing this play out right now with Race to the Top and i3 (the Investing in Innovation Fund). What academics could do best and should do more of is stop coming up with quick-fix remedies but really understand the challenges … so that folks who are in the policy-crafting business will have more coherent insights to draw upon in designing programs and executing them.

“Frankly there’s very little incentive for think tankers or academics or observers to figure out what government can and cannot do. There’s more incentive for people to come up with their preferred policy solutions. That’s how you get attention, that’s how you sell books or make a name for yourself.”

Indeed, the experience of The Ordeal of Equality may illustrate what a small appetite the press and policy world have for research focused on learning from the past. Though widely respected in education research circles, the book has so far received relatively little notice even in the education press – just a review in Teacher’s College Press and a short blog post by Checker Finn (himself a former federal education official) lauding it as “the definitive history of the federal Title I program.”

And even the reactions to the Bixby event are indicative of the underlying dilemma. The audience had no quarrel with Cohen’s critique of past policy wrongs. But when it came to prescriptions for the future, there was much more debate. In part, that may be because Cohen’s new-found conviction that some solutions are possible surprised many who know him as a skeptical realist. Indeed, in Ordeal, he and Moffitt seemed to strain to find recommendations they could support, and they mentioned them only in a broad-brush and half-hearted way. By the time the Bixby Seminar came around, a year and a half after he and Moffitt had finished writing the book, Cohen professed to be significantly more optimistic about the existence of measures that could realistically be expected to support school improvement.

He cited a couple of reasons for his optimism. First, the current effort of governors and state school chiefs to develop Common Core standards raises the prospect of shared curricular frameworks that could form the basis for teacher training and professional development. If that effort is successful, the new standards could alter the status quo in which the absence of common curriculum means teachers are taught to “to teach nothing in particular to no one in particular,” he said.

Second, Cohen and colleagues have studied a subset of comprehensive school reform designs (CSRDs), and he has followed the research into effective charter school networks by his students and others. While not all CSRDs and charter networks have produced improved student outcomes, a few have, including Success for All and America’s Choice, among CSRDs, and Aspire Public Schools, Achievement First, and Noble Street, among the charter networks. And those that have appear to share common design elements in that they:

  • provide detailed designs for instruction
  • focus on teaching practice
  • build infrastructure and manage quality, and
  • create new roles focused on providing intensive support for teacher learning

Calling these successful designs “remarkable inventions,” Cohen noted, “They understand if one wants to improve teaching and learning, one has to work on teaching and learning.”

The problem is that currently these successful approaches, though nurtured by (in the case of CSRDs) or permitted by (in the case of the charter networks) current policies, exist on the fringes of the education system. Cohen argues that it is possible at least in principle to import these approaches into the mainstream of U.S. education even within the current balkanized system of education governance. Such an effort would also depend upon the success of the Common Core, not to mention significant coordination among schools, school systems, state leaders, and federal policymakers. And yet, even the ability to point to an approach with some prospect of succeeding within current governance structures gives Cohen hope.

All this spurred questions from the audience during the seminar and afterwards. How could reformers tap the effective aspects of charter networks without strengthening the multi-pronged agenda of charter advocates, including those intent on toppling teacher unions? Who can imagine federal education officials pouring money into CSRDs now, when they are an invention of the 1990s? And even those who agree with the thrust of Cohen’s recommendations, such as Deborah Ball, his dean and collaborator at Michigan, noted after the talk that the window of opportunity may not be open for very long, and so considerable orchestration may be required.

And though the time horizon may be short, these questions and suggestions all point to a need for more research, building in part upon Cohen and Moffitt’s work. That research may need to draw upon a variety of disciplines. “You get economists in the room and they’re going to think about black box approaches,” noted McPherson, himself an economist. If economics on its own is a blunt instrument, perhaps historians, political scientists, social psychologists, even experts in the craft of teaching may all need to contribute to the conversation.

But more research on its own won’t suffice either. To the extent that Cohen’s conclusions were based on reviewing the public record and existing research, some shift in priorities may also be required. If researchers need to take the long view both in understanding educational problems and seeking to solve them, reformers may need to take the time to understand what the research is saying.

Pamela Burdman, a senior project director at WestEd, previously served as a program officer at the William and Flora Hewlett Foundation and an education reporter at the San Francisco Chronicle.

The Spencer Foundation was established in 1962 by Lyle M. Spencer. The Foundation received its major endowment upon Spencer’s death in 1968 and began formal grant making in 1971. The Foundation is intended, by Spencer’s direction, to investigate ways in which education, broadly conceived, can be improved around the world. The Spencer Foundation believes that cultivating knowledge and new ideas about education will ultimately improve students’ lives and enrich society. The Foundation pursues its mission by awarding research grants and fellowships and through other activities to strengthen the connections among education research, policy, and practice.