James G. March
I have Mike's letter and fear I am mostly at a loss as to how to respond to it. It certainly deserves a response and reflects an impressive level of thoughtful questioning and anguish; but I doubt that I can be helpful within the frame he sets.
The problem for me is that I think the question is framed incorrectly by (essentially) asking for a list of projects/studies/contributions and trying to assess their impacts. I realize that if you formulate the decision problem as a matter of choosing among a set of alternative research projects or programs in terms of their expected returns, you are driven to some such analysis; but I think that is (in this case) a poor way of formulating the problem.
There are two primary reasons why I think it is a poor way. The first reason is well known: The measurement metrics involved are ill-specified and in dispute, as is the contribution of any particular factor to them. Almost anyone can make a case for (oragainst) almost anything. The second reason is less commonly noted but is also well known. Imagining research as a collection of discrete projects with separable effects invites you to consider only the more minor effects and the more trivial research.
In the spirit of Mike's letter, I think you might want to begin by reversing the focus and asking yourself what affects education. Education changes all the time, and it varieswidely (both in its character and in its apparent effectiveness) across nations. Most of the studies of educational change or educational differences that I know attribute the changes and differences primarily to historical, demographic, economic, social, and political contexts. Occasionally, some role is assigned to changes or differences in ideas about education - for example, the recent enthusiasm for "market-like" approaches to educational management in the U.S. However, I don't recall any serious scholar trying to make a very direct association between the world views that affect education and specific research. It doesn't work that way. The contemporary dominance of an economic world view, for example, has very little to do with research results. Similarly, it would be extremely difficult to claim that cross-national differences in the amount or quality of educational research explain any appreciable amount of the cross-national differences in apparent educational effectiveness.
I think working back from education change or differences would be better, but it still would not be very good. Research impact formulations take you down the wrong path in a more fundamental way. As long as we think we have to justify specific research by its contribution to measurable social goals, researchers will create such justifications. The process may have some claim to usefulness. It focuses debate on contending claims. However, the justifications generated will be both fraudulent in the sense that they have little or no extra-subjective basis and pernicious in the sense that they substitute the pursuit of Relevance for the pursuit of Truth. As nearly as I can tell, the research that ennobles the human spirit is usually not driven by desires for affecting practice, and attempts to direct it in that way are likely to lead to corrupt, even dull, research and negligible impact. Good research seeks to satisfy an aesthetic of human intellect. It pursues Truth through elegance, purity, and grace. Usefulness in its usual definition is a possibly admirable occasional side effect but not a primary objective.
All of which is a longwinded confirmation of my original judgment that I don't really have anything useful to say and can only wish you well. I think the Spencer Foundation has an admirable record and has achieved that record without any appreciable influence by me. Being a proper academic, I do not consider the irrelevance of what I have to say an adequate reason for shutting up; but I am sure the Board of the foundation will find the formulation in Mike's letter substantially more congenial than they would the formulation in mine. It is a thought that allows me to face my continuing irrelevance with comfortably irresponsible equanimity and with unconditionally romantic optimism for the foundation's future.