Urban educational challenges: Is reform the answer?

Susan Fuhrman, Ph.D.
Dean and George and Diane Weiss Professor of Education, Graduate School of Education, The University of Pennsylvania

My topic for this address is Urban Education Reform. Clearly, there are challenges in urban schools, but what's important and what I want to focus on is that there is no shortage of reforms intended to address them. The question I want to address is - why is reform so prevalent and so disappointing?

I think that there are three sets of reasons, some having to do with political factors, some having to do with an overemphasis on structural solutions, and some having to do with the research base that is supposed to guide us as we choose among the different ones. So, if "reform" is not the right metaphor for addressing urban challenges, as I believe it is not, what is? And, what should we do? I'll try to address that during this talk. In the course of it, I'll refer to some things we're doing right and maybe not so right here at Penn's Graduate School of Education. I'd like you all to help us improve as a graduate school of education that cares deeply about urban problems.

Urban educational challenges

To start with the background, you all know how concentrated students are in urban settings. Over 31% of all students attend school in 226 large school districts among the 15 or 16,000 school districts in America. That translates into 31% of all students in just 1.5% of school districts (Ladd, 32). So, urban education is where the challenge is, and for my money if you are not focusing on urban education, especially for a school of education located in Philadelphia, you are not focusing on the important issues of our time.

I don't need to dwell on the litany of challenges that accompanies the concentration of students in large districts. You know that there are higher than average proportions of students in poverty, of students with poorly educated parents, of immigrants and other students with limited English skills, of students from unstable family settings, and that there are greater rates of student mobility. There are greater shortages of teachers, more teachers with emergency credentials, poor facilities, and low achievement-generally significantly lower than suburban districts.

And, there are, of course, fiscal challenges. Great disparities related to wealth still exist. Districts with 25% or more of their school-age children in poverty had average total per pupil revenue that was only 89% of the average total per-pupil revenue elsewhere. So, poor districts spend less. And then, that's total revenue. If you adjust for the needs and the costs of educating children in urban areas with all the challenges I just outlined, and for the higher costs in urban areas where there are higher prices to be paid for labor and virtually everything else, then you find the real significant disparities.

Just to take some data that emerged from Standard and Poor's School Evaluation Service recent release of Pennsylvania school district data, when you adjust for costs and needs, Philadelphia spends as much as 23% below the state average expenditure per pupil. Now this is not to say that all of Philadelphia's problems, such as extraordinary low achievement, are related to money, but it is certainly to make a point about the disparities between richer and poorer areas in this country. On top of that, we know that Philadelphia citizens and other urban citizens are paying high taxes for every service that they must provide, not just schooling, and are overburdened with all of these taxes.

Urban Reform

Given these challenges, it's no surprise that most urban districts are extensively engaged in reform. I want to read to you a little section from a book called Building Civic Capacity by Clarence Stone and colleagues. This is a description of the recent reform efforts in the District of Columbia (p. 148).

As part of the decentralization effort, every school had been required to establish a local school restructuring team. And over forty "enterprise" and "renaissance" schools had been given special discretion to shape their own policies at the school level. Many schools offer specialized programs that had developed loyal constituencies. These included an elementary school with dual language, (English and Spanish) immersion, another elementary school with an Afrocentric curriculum, and special "academies" designed to provide high school students with career-relevant education (including a Health and Human Services Academy, a Public Service Academy and a Trans-Tech Academy). Public/private partnerships were in place in dozens of schools; many of these provided enriched career-related training, including separate programs for Culinary Arts, Interior Design, and Landscape Architecture, International Studies, Pre-Engineering, Business and Finance, and Travel and Tourism, and COMSTAT's computer and science partnership with Jefferson Junior High. In addition, during the early 1990s, an aggressive deputy superintendent spearheaded the expansion of the District's Early Learning Years program, which involves more child-friendly curriculum, and a heavy emphasis on making certain that teachers and principals receive the training and support they need to put the curriculum into place. The district is perhaps the only large urban school district to offer a full- day early childhood education program in every elementary school. And the program has been expanding to incorporate many three-year-olds. In 1995 Congress passed legislation initiating a major charter school program in D.C., but even before Congress acted, the public system had experimented with "School-within-School Charters," including a Montessori school, a Non-graded School, and a Media-Technology Social Research School.

That description could take place in any city in America except for the fact that Congress mandated part of it, because in the case of the District of Columbia, Congress is the big school board. So, this amount of reform is quite common. Frederick Hess, the author of Spinning Wheels: the Politics of Urban School Reform, studied 57 large school districts and reported that the mean district proposed 11.4 reforms over a 3 year period in the 1990s. (Hess, 1999). The problem is not the absence of reform. In fact, what we see is a picture of too much reform. Too many reforms, few of which are effective. Too many reforms that are undertaken; too few that are implemented. Further, the reforms are hardly coordinated; they seem to lurch in all different directions, reflecting opportunism more than any coherent improvement strategy.

Why this "policy churn," to use Hess's (1999) term? As I said before, I think that there are three sets of factors. Let me start with the political factors, and draw heavily from Hess (1999) who writes about how institutional incentives encourage a focus on proposing change, not on improving schools, on symbolic change, on inputs, and not on outputs.

Political factors

It is obvious is that it's much easier to propose a reform than it is to implement it. That explains part of the "policy churn". But there are a number of other factors that are particularly characteristic of urban schools that lead to this emphasis on symbolic reform, or this posturing around change. For one thing, it's hard to hold urban schools accountable for performance, even in the current climate of accountability.

There are so many complicating factors in urban schools-high mobility, teacher turnover-that it's really hard to determine the value added by schools. Leaders can escape accountability for performance, so it doesn't matter much if the reform actually works. Because if it doesn't work, there are many reasons we can say it didn't. "Well, the kids weren't here two thirds of the year when we tried to implement this new curriculum." And it's true, not a made up excuse; it's absolutely true.

Another factor is executive turnover. Urban superintendents hold office for about 3 years, which is much too short a time for reforms to really take effect. They are not around long enough to be accountable for results, so they tend to be held accountable for what they propose. And, in fact, superintendents' careers are built on advancing from district to district, to larger and larger districts with more and more prestige. Superintendents have to be active to build a reputation, and since they have a short term, they have to be active in that short term. And that means proposing and starting reforms, and calling attention to oneself for doing that, or in Hess' terms "overindulging in innovation."

This posture is reinforced by the foundation, corporate, government and, yes, academic communities surrounding education. We get benefits when they adopt our ideas so we push our ideas on them. And these are always "new" ideas because we aren't about to push somebody else's ideas on them. No one gets paid for working on old ideas, so we contribute to the problem.

The high visibility of education, particularly in urban areas, also contributes. There is much at stake. You can't work quietly. The public thinks that you're doing nothing if you're not attracting attention. You have to convince the community that you're acting if you're an urban school leader and proposing reforms is a way of rallying community support and resources. Also, reforms bring notoriety and prestige to a community. Unlike the quiet, much less glamorous work of improvement, school boards support reforming superintendents because it enables them to claim credit too. They can say, "We were the school board that initiated X Y and Z reforms."

Clarence Stone and his colleagues see additional reasons for urban school reform challenges such as the absence of civic capacity. (Stone et. al, 2001). Civic capacity is the ability of a community to collectively problem solve with a supportive array of relationships across elected and district leaders. Without it, long term support for reform is missing. Elected leaders have even more incentives than superintendents and boards to engage in eye-catching reforms; their electoral cycle drives them to short-term, catchy initiatives. Getting their cooperation for hard, long term work requires an investment in education that has to be deeply felt and that is not present in many communities.

One of Stone's coauthors, Jeffrey Henig, and his colleagues note in the Color of School Reform that in the four cities they studied, Atlanta, Baltimore, Detroit and Washington, DC, racial factors made long-term civic collaboration on education that much more difficult. (Henig et. al 1999). There was distrust and much history that was never overcome.

Structuralism

A second characteristic of urban reform that impedes its effectiveness is the emphasis on structural reforms, on centralization or decentralization, school based management, charters, choice, on altering patterns of authority within the district. The resort to structure characterizes American education reform efforts historically. We've played around with how to organize schools around grade levels with graded vs. ungraded classrooms, with large vs. small schools, with creating large, centralized bureaucracies, in Tyack's words, the "one best system," and then breaking them up.

Why the preoccupation with structure? Structures are visible, manipulable, and easy relative to the hard work of really improving teaching and learning. Structural reforms are tangible, you can see them; if you have a three year time frame, structural change may be something that you can do. You can see why, given the political incentives just discussed, people would gravitate to big, structural reforms.

But these reforms don't necessarily lead to meaningful improvement in teaching or learning. They focus on changing the incentives around which people work, "empowering," them, or monitoring them more closely, theoretically affecting people's will to work harder.
But sometimes, that theory of action is just wrong. For example, people don't necessarily work harder on instruction when they are "empowered" through school-based management. They may be tied up in meetings on keeping hallways clean or determining how council representatives are elected. Even if they do work harder, motivation is just a part of effectiveness. These structural reforms don't necessarily affect the other aspects that make educators effective, their knowledge, skills, and beliefs about whether children can learn. This is the "myth of omnipotent structure" to borrow a title from public administration literature (Anne Marie Houck Walsh as cited in course material, 1973). Policy wonks and educators alike seize on structural solutions without fully elaborating the connections between the structural change and the desired results. School based management may give teachers more authority, but what conditions would be necessary for them to use that authority to realize improvement?

Structure can, of course, be enabling, but it should not be seen as the entire solution.
I find the emphasis on structuralism particularly troubling in the case of charters and choice. They are clearly today's "silver bullets," the panaceas that are going to change everything. "Break it up," one hears often with respect to large, urban districts-but the next question, "and then what?" rarely gets asked.

The research base

This leads us naturally into the third area I will discuss: the research base underlying the reforms that cities are so busy adopting. Choice and charters are a good example of the kind of guidance or lack thereof that the research base gives us. With respect to those reforms, the evidence about their effects is increasingly clear; it's inclusive. There is no clear evidence of achievement gains, for example, there are gains in certain grades, certain subjects, and certain populations and not in others.

Why don't we have better research to guide us about reform, and if we did, would it matter? Would urban educators use research to choose reforms, instead of selecting the more visible, structural reforms we just discussed?

Let me address the more sensitive topic first and that is the quality of the research. Present company excepted, I want to say that the research base is much weaker than it could be. There are many important questions about which we could use more guidance. I think education research needs to be improved in several ways. In 2000, I gave a speech at AERA about three studies that had gained a lot of policymaker attention. (Fuhrman, 2000). Deborah Nelson, who worked with me on this, and I chose three studies that policymakers were interested in and referred to in a way that they don't often with other research. The three studies are the Perry Preschool study of early childhood education, the Tennessee STAR experiment on class size, and the NICHD (National Institute for Child Health and Human Development) studies of reading. I cited four qualities of these studies that enhanced their credibility, along with a host of contextual factors, such as the presence of research brokers to help popularize their results.

First, the studies did not try to answer a question with an inappropriate design. Or to state that positively, the studies tried to address a question with an appropriate design. Much education research tries to get at the "what works" question with studies that might show relationships between treatments and achievement but cannot answer the causal question as definitively as possible. Research rigor has everything to do with matching the design to the question, something by the way that I don't think we as a field think enough about or pay significant attention to in the training of new researchers. I know that we at Penn GSE have excellent methods courses-both quantitative and qualitative-but I often wonder how well we prepare students with the prerequisites they need to really benefit from the methods courses. How well do we prepare them with the ability to frame a good research question and to match it with a good design that employs one or more of these methods?

I certainly don't mean to imply by focusing on those three big studies or by the causal question that all research should be experimental or quasi-experimental. I don't mean that at all. The "what works" question is not the only question to be asked about reform options. We also want to know the manner in which policies exert an influence-not just whether they exert influence. We want to know how various design options play out in practice. We want to know more about the dimensions of problems, such as whether different population groups or types of schools experience issues differently. In other words, there are many things we want to know that don't require an experimental design. The important point is that research suited to the question is more likely to be considered rigorous by policymakers and by us than research that is stretched to answer questions it can't.

A second point about these studies is that they were longitudinal. Either the original study or follow-up studies looked at effects over a period of time, giving the results staying power and helping to sell them to policymakers. It's much easier to justify a program expense when the results of the program last, and there's no way to know that unless some longitudinal research is conducted.

Third, these studies were replicated by other studies, confirming their findings and lending them much greater power. Replication is a way to test findings-it's through repeated studies that we learn whether original findings can be confirmed, whether they hold up. If repeated studies get different results, there's good reason to question the original findings. Lack of replicability is what did in cold fusion, as I'm sure you all remember. But replication serves other purposes as well, purposes especially important to education reform. Repeat studies can confirm findings in different contexts, proving to policymakers that results are not just situational but have broader applicability. In other words, policymakers want to know that "this will work in my city." It's easier to make that case with studies done in a variety of settings than with evidence from just one.

And, replication creates a body of research that multiplies the importance of any one study, telling policymakers that a variety of researchers, perhaps even of different perspectives, agree on a conclusion. This last point is very important to policymakers. Few things irk them more about research than the fact that researchers often disagree with one another and can't provide clear guidance. It leads them to discredit and underfund research altogether.

Finally, these studies were incorporated into syntheses that helped make sense of the findings. This is what we need to do in order to create and understand the weight of the evidence. Policymakers want to know how the latest study affects what was known before, how new work fits into the total body of work, and how the research aggregates to form a conclusion.

We can do more to assure the credibility of research. We can assure that designs are suited to questions, that more studies are longitudinal, that replication takes place, and that work is synthesized to provide cumulative answers. It's true that much of this can't be done without additional funding, and lack of adequate support for education research creates a real challenge. Elaborate designs, longitudinal studies and replication are very expensive. And funders tend to put a premium on new work, just like reformers, rather than on repeating existing work or doing follow-up studies. Each funder wants to claim its own unique contribution, so it's hard to get both public and private funders to support confirmatory work that's not likely to be as splashy as the original. In the case of dissertations, we can also accept some blame. We push the new and the unique no matter how narrow and arcane it can make the topic. And we rarely think about the importance of replication and confirmatory studies when encouraging students to undertake research projects.

The fact that too few large-scale, longitudinal, and replication studies are done isn't all our fault, but we can't escape the blame. We value newness over replication ourselves-in our training of future researchers and our guidance about dissertations. We argue among ourselves over paradigm rather than spending the time necessary to see how the evidence accumulates across qualitative and quantitative work, across different research approaches. We are too rarely concerned that students become adept in combining methods. Certainly, we must convince funders that we need more money to do the kind of work that they value, the kind of work that has meaning for policy and practice. But we also need to prove that we are interested in doing the sort of work that can justify a much larger investment.

There are some encouraging signs. One example that I am very proud of is the Campbell Collaboration, which is taking shape here under the leadership of Bob Boruch. This collaboration, a new multi-national effort, will prepare, maintain, and promote access to systematic reviews of studies on the effects of social and educational policies and practices. The organization will provide regularly updated syntheses intended to help policymakers and other users by presenting the weight of the evidence. Not coincidentally, by deciding what research to include in syntheses, the Collaboration can have a great deal of influence over research standards.

However, it would be naïve of us to assume that better research would automatically have stronger sway in the marketplace of ideas that surrounds urban school reform. In fact, we have reason to worry about the climate for research and the value placed on evidence by practitioners. Tom Corcoran, Cathy Belcher and I studied the adoption and support for comprehensive school reforms in two large districts. We called them River City and Metropolis. We found that while district personnel wanted to use evidence about student learning in choosing reforms and talked about "best practice" in a way that implied research-based decisions, they often made choices based on ideology rather than results. At the school level, there was even less pretense about the importance of research.

Our major finding was that school personnel value the opinions of other educators much more than published research. The teachers we surveyed placed strong value on the endorsements of other teachers, with between 80 and 90% agreeing that these were the "best" source of evidence on quality. On the other hand, only around 60% gave such support to published research on evidence of effects. In fact, 35% of the teachers we surveyed in these two cities think that the findings published by education researchers should not be trusted. Almost the same proportion is likely to distrust anything but their own eyes and own measures as evidence of effectiveness in education.

Surely there is reason to distrust educational research as I have just discussed, but we haven't built a culture of attending to evidence in education either. Of course the relationship between a research culture that produces good evidence and a culture of use of evidence is circular. We need to work on both sides if we want to improve evidence-based decision making. It's important that we worry about the quality of the research we produce, but it's also important that we focus the education of practitioners on evaluating and benefiting from evidence. I know that the new mid-career leadership program we're designing at GSE takes evidence-based practice as one of its starting points. I'm encouraged by that development.

Improvement not reform

If "reform" is not the answer for urban schools because they already do too much of it, because they tend to rely on structural reforms, and because picking reforms that "work" is difficult to do based on existing evidence and educators might disregard the evidence anyway - what is the answer?

I'd like to shift the metaphor around urban school progress from "reform," to "improvement." This is not a new concept. Fifteen years ago, Richard Elmore and Milbrey McLaughlin wrote a very influential little book called Steady Work (1988). Improving teaching and learning, the heart of schooling, is slow, unending, not particularly glamorous, and hard work. It's not a matter of policies coming in from the outside, swooping down. It's a matter of continued attention to the basics and what matters in teaching and learning.

It involves deep investment in teacher quality and knowledge, through recruiting, compensating and developing teachers. It involves thoughtful, well-funded professional development. Professional development must be intensive, extensive (over a period of time), focused on the curriculum the teachers are teaching, followed up by coaching and other on site support. At the Consortium for Policy Research in Education, we surveyed elementary school mathematics teachers in California who had taken a variety of kinds of professional development in mathematics. Some of them took content focused units based on the curriculum that the students were learning on fractions. Some of them took equally worthy courses that were disconnected from the curriculum - collaborative learning, diversity training, things that we think of as important but that weren't directly connected to the 4th grade mathematics curriculum. What we found that was the teachers who took the curriculum related professional development, provided that it was intense enough and had enough follow up and support, changed their practice in ways that were envisioned by the reforms, and also had gains in student achievement that the other teachers engaged in other kinds of professional development did not have (Cohen & Hill, 2001). When I tell this story to a lay audience, the story that professional development focused on curriculum that students are learning has a bigger effect than professional development that is disconnected from what the students are learning, they look at me strangely, thinking, "you were paid to have that study done?" The answer is so common sensical. In fact, that kind of intensive curriculum related professional development is not what we do in education. We know that we do scatter-shot workshops. We teach about lyme disease and Right-to-Know with chemicals and all things that are important for the safety of our kids, but that don't influence student learning. If we want to influence student learning, if we want to improve students' knowledge of subjects and skills, then we have to think seriously about the professional development in which we engage. At Penn GSE, in programs like the Penn Literacy Network and the Philadelphia Writing Project, and the Penn-Merck Collaborative for the Enhancement of Science in Education, we have content focused professional development and we need to promote that force.

Improvement over the long run and steady work involves good curriculum design. We don't make enough time for teachers to collectively develop curriculum. We also don't provide adequate choices through the web or other means if they don't want to make their own curriculum. We have this enormously romantic notion that teachers want to teach all day and come home and write curriculum all night. The teachers that we studied in our research that are implementing the various reforms that we are studying do not want to do that. They'd like to have good curricula available to them so that they can make wise choices about what to use. Improvement involves developing leaders-administrators and teachers-who know good instruction, and can evaluate and support it. It means developing a collective vision for and responsibility for good instruction, overcoming the norms of isolation and building communities in which teachers are accountable to one another for good instruction. This is much like what we're starting at the new Penn-Assisted School where teachers are in each others' classrooms all the time and where they talk about their practice regularly. Granted, the school has only been open for two months, but so far it is a model; we hope that this kind of teamwork continues.

Steady improvement involves changing the culture of low expectations surrounding urban schooling. As we at CPRE have examined instruction in many settings across the nation, we see countless examples of teachers "protecting" their students by not presenting more challenging material. Believing that the students they teach from "disadvantaged" backgrounds need discipline, order and basic skills, even teachers who try to teach more complex material, even those who may be better prepared than others in terms of their own knowledge and skills, even those with supportive principals and other factors in their favor, doubt that poor and underprepared students can reach challenging and complex understandings. Encouragingly, experience-through professional development, observing experts teach their classes, seeing their own children engage in problem solving and more complex activities - can change these beliefs. In Kentucky, in 1994, only 35% of teachers agreed with the Kentucky reform principle that all children can learn and most at high levels. By 1999, 68% agreed. How did this change occur? In the context of a stable reform environment, which Kentucky had over all these years, teachers made incremental changes in their practice and student performance, even in the most disadvantaged settings, improved. Teachers could see that as they changed their practice, the students were learning.

Some of the efforts that I have described can be done by reallocating resources, some will require new money, but while certain structural reforms might make them easier, they don't necessarily require structural change. We can see schools and classrooms undertaking such efforts with more central direction, like District #2 in New York City, or San Diego, or with less central direction and more flexibility from district operating procedures such as we see in our own Penn Assisted School.

The tough work of improvement must be separated from the glamour of reform. It requires steady work. It requires realism rather than romanticism. It requires the efforts of all of us.

References

Cohen, D., & Hill, H. (2001). Learning policy: When state education reform works. New Haven, CT: Yale University Press.

Elmore, R., & McLaughlin, M. (1988). Steady work: Policy, practice, and the reform of American education. Washington, DC: Rand Corporation.

Fuhrman, S. (2000). Education policy: What role for research? Division L Vice Presidential address at the annual meeting of the American Educational Research Association, New Orleans, LA.

Hening, J., Hula, R., Orr, M., & Pedescleaux, D. (1999). The color of school reform: Race, politics, and the challenge of urban Education. Princeton, New Jersey: Princeton University Press.

Hess, F. (1999). Spinning wheels: The politics of urban school reform. Washington, D.C.: Brookings Institution Press.

Stone, C., Henig, J., Jones, B., & Pierannunzi, C. (2001). Building civic capacity: The politics of reforming urban schools. Lawrence, Kansas: University of Kansas Press.