June 15 - 16, 1998
Mapping The Terrain: Economics and Educational Research
Chicago, Illinois
Overview
On June 16, 1998, the Spencer Foundation, with the assistance of Michael McPherson of Macalester College and Richard Murnane of Harvard University, convened a group of economists to consider how research on education might benefit from the methods and approaches of economists. Accordingly, the conversation focused on the unique contributions that economists could make to the study of education, as well as the incentives and disincentives for doing such research.
This report is organized in two sections. The first discusses issues arising when economists conduct educationally relevant research. The second section presents a research agenda for educationally relevant research to which economists could make significant contributions. The discussion focused on four areas: (1) the roles education plays in contributing to personal income growth and national economic growth, (2) the organization and operation of educational systems, (3) educational finance and (4) educator labor markets.
In her opening remarks, President Patricia Albjerg Graham emphasized that the Spencer Foundation is interested in sponsoring educationally relevant research by economists. The Foundation’s definition of educational relevance is broad, including studies with findings that are applicable to real-world educational problems or that contribute to building educational theory. In selecting the projects it will support, the Foundation uses a peer review process – that is, in the case of proposals from economists, other economists who address problems related to educational practice and theory are selected as reviewers. Some of the difficulty for proposals from economists in surviving the peer review process may arise because, as one participant noted, economists have very high standards in judging whether particular research designs provide credible evidence of causal relationships.
Issues Arising When Economists Conduct Educationally-Relevant Research
The economics profession rewards empirical research that rigorously formulates and addresses problems in addressing causal relationships. Economists’ standards of what constitutes "rigor" are high. Rapid advances in the area of methodology in recent years have made the standards even higher. One consequence of the methodological advances may be that economists narrow the range of questions relevant to education that they study. Economists are not rewarded for empirical work on critical questions if their research designs do not provide rigorous analysis of causality. As a result, economists have incentives to tackle problems that do lend themselves to strong research designs, even if the problems are of limited practical or theoretical interest. In sum, there is a tension between the Foundation’s objective of funding research on important questions and the incentives facing economists to conduct research that provides rigorous analysis of causality. One way that this tension becomes evident is in the Spencer Foundation’s review process. The Foundation encourages proposals from economists as well as from other researchers on critically important topics in education. The economists who act as reviewers often find that the research designs for projects that address critical questions are inadequate. (Of course, economists would also question the adequacy of the designs that researchers in other disciplines propose for research. However, typically the Foundation only asks economists to review proposals submitted by economists.)
Economists do have unique contributions to make to educational research. One participant at the conference suggested that it may be useful to think about domains of questions. The first set of questions lies at the outer edge of educational practice and policy and focuses on the economic payoff to educational investments. A second set of questions, closer to the center, pertains to organizational issues such as vouchers, and public versus private schooling. A third set of questions, somewhat closer to practice, deals with financing and resources. In the innermost core is a set of questions related to teaching and learning.
Conference participants were in agreement that economists have pursued issues in the outer three circles and made contributions in those areas. There was less consensus on whether economists can contribute to understanding teaching and learning. Although some participants thought that economic tools might be applicable to research on teaching and learning, economists do not have the training to go into classrooms and understand what is going on. One proposed solution was that economists collaborate with researchers who have the expertise to evaluate the teaching and learning process. Such collaborations could be a step forward in tackling critical questions about teaching and learning.
Differences in conceptual perspectives, values, and methodological approaches challenge the best of intentions in inter-disciplinary collaborative work. The primary reasons why there is little collaboration between economists and other social scientists relate to issues of communication and issues of statistical sophistication. First, the technical language with which economists describe their analytical tools creates a language barrier of sorts that hinders communication with other social scientists. Second, although all social scientists use the same statistical tools, economists have achieved a level of sophistication that other social scientists and even some statisticians do not share. Economists have developed expertise in how to use statistical tools in difficult situations, such as when the subjects of an experiment change their behavior as a result of policy changes.
Case studies are useful in generating interesting research questions. It may be valuable to have an economist’s perspective involved in case study research in order to develop a broader set of research questions, including those based in economic theory. However, because of the high regard for experimental methods involving the random assignment of subjects to treatment and control groups, the case study method may not be the approach of choice for most economists. For example, a case study of a single classroom without a control group makes it difficult to know whether the same practices are occurring in other classrooms.
While economists may not be particularly strong at understanding what happens in a classroom, they are skilled at evaluating programs, comparing costs and benefits, and providing an analytical framework. The Tennessee STAR experiment on class size is a good example of educational research that has the basic qualities economists look for. From the economist’s perspective, the value of the STAR experiment stems from the use of random assignment to assign students to classes serving differing numbers of students. The interest that the STAR experiment generated among economists suggests that if this kind of experiment were conducted more frequently, there would be more research by economists that is relevant to education. One participant highlighted four strengths of economics compared to other social sciences. First, economists tend to focus on costs as well as benefits. They ask, for example, not only whether reducing class size results in improved student learning, but also whether the gains relative to the costs compare favorably with other ways to use scarce resources. In other words, said one participant, "we should move beyond asking ‘does reducing class size help?’ and ask instead ‘how much does it help?’" Second, economics provides a strong theoretical framework, one that is particularly valuable in analyzing the operation of markets. The theory provides guidance about the types of data that are needed to evaluate the consequences of educational policies that concern interventions in markets, such as pay increases for teachers, and educational vouchers for students. Third, economists tend to value long-term as well as short-term outcomes. This is critical to understanding the consequences of many programs, such as the Perry Pre-School Program. Finally, economists have solid statistical knowledge, particularly in non-experimental statistics. "We know . . .about how to handle situations where you don’t have all the data you want and there are omitted variables in your explanatory equation; that’s one of our strengths," said a participant.
Several weaknesses of economics relative to other social sciences were mentioned. Economists often ignore institutional details that are important in shaping policies and determining their outcomes. Economists may not have special capacities to understand teaching and learning processes, -- which lie at the center of the education process. These limitations of economics might be alleviated by interdisciplinary research.
Educationally-Relevant Research Topics Pursued by Economists
During the conference, participants raised many different topics and discussed how they are educationally relevant and why the economist’s approach to them is useful. Suggestions for educationally relevant research can be clustered into four categories: (1) the consequences of educational investments for individuals and societies, (2) the organization and operation of educational systems, (3) educational finance, and (4) educator labor markets.
Economists have made critical contributions to understanding the contributions of educational investments to personal income growth and national economic growth. They have explored the payoffs to vocational education in many countries. They have shown that education has the highest payoffs in societies undergoing economic changes. They have developed explanations for the increase in education-related earnings differentials during the 1980s in the U.S. and many other countries. In recent years economists have probed more deeply into the interpretation of statistically significant relationships between educational attainments and labor market earnings. This work provides new insights about the likely consequences for educational attainments and labor market earnings of particular policies such as increased scholarship aid and building schools in rural areas.
Economic research has contributed to understanding how the incentives facing different actors in educational systems affect their actions, and how these actions affect the performance of the systems. Fruitful work by economists relevant to elementary and secondary education includes analyses of the consequences of incentives for school principals and school boards, the consequences of performance contracting, and the impacts of merit pay for teachers. Work by economists has also improved understanding of factors that affect college admission strategies and financial aid plans.
Economists have proposed explanations for differences across countries in student test scores. At the core of the explanations is the view that incentives for students matter, and that these incentives vary across countries. Especially interesting are a number of recent theoretical papers by economists exploring the consequences for students of different types of standards-based educational reforms. These papers may provide a framework for fruitful empirical work once data become available on the outcomes of current efforts by states to introduced standards-based educational reforms.
A third area of suggested research concerns educational finance. Economists have learned a great deal about taxation that can be applied to education. The lessons concern the efficiency of alternative taxation and transfer strategies. The problem of how best to communicate this knowledge to non-economists still remains to be resolved. Research is needed on the effectiveness of education spending so that people can better understand the ways in which spending on education matters. Participants stressed that a strength of economics compared to other disciplines is the comparison of costs and benefits. This focus allows economists to research such questions as "What are the tradeoffs between teacher quality and class size? Should we be making tradeoffs? Should we have large class sizes and at the same time pay teachers more and recruit better trained teachers?" Models related to finance developed by economists are also useful in understanding the effects of such incentives as college financial aid and vouchers. Economists can help identify who will benefit and lose, as well as the size of the gains and losses.
A final area of research concerns issues pertaining to labor markets for educators. Some research explores the supply side of the market, examining the dynamics of teacher career paths, and the impact of incentives on the decisions of college graduates to enter teaching, and to remain in teaching, and decisions of teachers concerning when to retire. Other work has focused on the demand side of teacher labor markets, exploring the factors that affect school district hiring strategies. This research has improved understanding of the factors that influence who teaches. However, a better understanding of what goes on inside schools and the factors that influence how teachers and students interact would enhance this research.
Research is also needed on how labor market trends affect the availability and decision-making of principals and administrators. In comparison with teachers, little is known about the labor market for principals and administrators. As self-researching schools or school networks become more common, economists may be useful in modeling systematic ways that schools learn from one another and in estimating how teaching expertise as a resource is enhanced through this process.
A Note from The Spencer Foundation
The Spencer Foundation sponsors occasional conferences to stimulate research on important issues in education. We thank Michael McPherson and Richard Murnane who helped organize this conference, Emiliana Vegas for her role in writing this report, and the individuals who participated (see attached listing) for their good work and contributions. We hope that interested researchers will find the theoretical, methodological, and topical issues raised at the conference thought-provoking. The Spencer Foundation welcomes well-designed research proposals on the issues and ideas discussed in this report.