Teaching, Learning, and Instructional Resources Major Grants


View by grant year

2008 | 2009 | 2010 | 2011| 2012

2008 Grant Summaries

Michelene Chi

Learning from Observing Learning with Dynamic Simulations
University of Pittsburgh

Previous research finds that one-on-one tutoring is the most effective form of instruction, but limited resources make that structure unrealistic in most schools. Chi and her colleagues have tested an innovative, more cost-effective use of tutoring, involving students observing videorecordings of one-on-one tutoring sessions. Based on their previous work, they have found that simply watching a tutor and tutee in an instructional situation is not sufficient for effective learning. Observers must be actively involved by engaging in certain tasks while observing: collaborating with another observer, re-viewing parts of the lesson on the video, and trying to solve the same problem as that shown in the video. Furthermore, students have better outcomes when observing sessions with “good tutees,” i.e., tutees who performed better on a pretest.

The current study further tests this approach. First, the research team is developing and improving materials and assessments based on findings from their earlier work. Whereas lessons in the preliminary research involved mechanical problems in physics primarily requiring procedural understanding, the present study uses lessons about natural selection, typically a difficult subject requiring conceptual understanding. Next, the team is using recorded tutoring sessions based on these new lessons in a lab study with pairs of 120 9th-grade student-observers, a larger sample than that used in the previous research, systematically varying the pre-performance levels of both the tutee and of the observing students. Finally, the team is conducting analyses to determine what makes videos more or less effective, developing scripted videos based on these findings, and testing the new videos in a second lab study.

David Alexander Gamson

Rereadig Our Past: The Cognitive Demands of Reading and Reading Comprehension, 1890-2005
Pennsylvania State University

Successful reading comprehension in school depends on students meeting the challenges posed by assigned reading tasks and by the texts themselves. The purpose of this study is to assess historical variation in the cognitive demands of reading tasks—i.e., the cognitive resources and skills required for reading tasks—through an analysis of reading texts and educational practices from the years 1890 to 2005 in the US.

There are four phases of the project: (1) an historical investigation to identify and locate the most popular and widely used reading texts, professional materials for reading teachers, research studies on reading education, and reading assessments; (2) linguistic analyses of these texts and other materials to assess and map variation in linguistic complexity over time; (3) assessments of the cognitive demands placed on students by reading comprehension tasks; and (4) examination of interactions between text complexity and task complexity. To complete these tasks, the project team will draw on a range of methods drawn from history, linguistics, cognitive science, and sociology.

Glynda Hull

Youth-Designed Social Networking: Investigating Emergent Literacies, Identities, and Relationships at the Intersection of Online and Offline Experience
University of California, Berkeley

This study examines the sociocultural, developmental, and educational implications of engaging children in social networking and other interactions as they develop their own on-line community, called “Kidnet.” Kidnet was created through an earlier stage of this project and is developed by youth participants from South Africa, India, and the United States. Participants can imagine, negotiate, plan, and build the network of multimedia artifacts and projects. The study gathers and analyzes evidence of participants’ decision making, interpersonal and intercultural negotiation, and knowledge construction. Particular attention is given to the content of exchanges and what these exchanges reveal about youths’ communication, literacy, and knowledge production across differences in language, ideology, culture, and geography.

At each of three locations in the participating countries, the study includes thirty middle school-aged youth, whose participation is being followed over three years. Youth are selected to represent the demographics of the larger social group in each local setting and the varied academic achievement levels within the school context. Multiple types of data are being gathered. Online data include records of each participant’s use of and contributions to the network, using a history-tracking system. Data will although be gathered through in-person administration of attitudinal surveys and skill inventories. Across sites, qualitative data collection includes daily observational field notes of students’ interactions; weekly audiorecordings of group interactions; and periodic interviews. An additional set of data includes youth’s creative work (e.g., stories, music, images, etc.). Analyses range from counts of particular events to correlation analyses; focused thematic coding of observational field notes, interviews, skill inventories; and discourse analyses around key events.

Marsha C. Lovett
Kenneth R. Koedinger

Improving the Feedback Cycle in Introductory Courses
Carnegie Mellon University

Lovett and Koedinger investigate the use of feedback to shape instructors’ practices and to enhance students’ learning in large, introductory college courses. This project is based on a process in which students engage in a learning activity, their performance is assessed, they receive feedback, and they have an opportunity to adapt their subsequent learning behaviors. While this process has been previously associated with improved learning in both online and face-to-face learning environments, there are significant impediments to its successful implementation. Among the obstacles are incomplete understanding of the theoretical basis for this model of learning, questions about how feedback should be coordinated with the rest of the teaching and learning process, and a paucity of tools to help both instructors and students manage complex information about students’ understanding.

The project addresses each of these problems in two distinct phases. All work is based on biology, chemistry, and statistics courses at different institutions. In phase one, the PIs build and test a model of learning that covers course content and that makes good predictions about student-learning. In phase two, they conduct a series of studies—natural experiments, observational studies, and experiments where feedback within the study courses is planned and manipulated—to investigate instructors’ and students’ use of feedback, and to inform design and development of the Digital Dashboard for Learning (DDL), a tool that will be designed to provide instructors and students with a meaningful display of information on how, and how well, students are learning.

The project takes place under the auspices of Carnegie Mellon’s Open Learning Initiative (OLI) in the context of three types of courses (biology, chemistry, and statistics) with extensive online components, situated in different types of institutions (liberal arts colleges, community colleges, research universities). In the OLI platform, each student interaction in OLI is automatically logged, producing a rich database that includes, for example, individual students’ answers to specific questions, precise times they complete particular activities, and patterns of feedback use over the duration of the entire course.

Robert J. Mislevy

Toward a Synthesis of Educational Measurement with Cognitive, Situative, and Sociocultural Perspectives on Learning
University of Maryland at College Park

Current approaches to educational assessment were developed with a view to trait and behavior psychology that is at odds with current models of educational performance. In this project, Mislevy aims to extend the conceptual architecture of the Evidence-Centered Design logic he has developed over the past decade to an integrated cognitive-sociocultural (ICS) perspective on the nature of knowing and learning. In so doing, he bridges a chasm between traditional assessment practices, informed by trait and behavior psychology, and integrated individual, situative, and social perspectives on cognition in order to develop a conception of test theory for assessment cast in an ICS perspective and amenable to probability-based modeling.

Over two years, Mislevy is developing three papers, each of which should contribute to the field of educational assessment by answering a particular question:

  • How can a narrative space be developed to connect assessment problems cast in an ICS perspective with models and methods that have proven useful to the design and analysis of assessments cast in trait and behavioral perspectives?
  • With regard to science inquiry in particular, how can a synthesized test-theory/ICS approach help science educators assess model-based reasoning?
  • With regard to language testing in particular, how can a synthesized test-theory/ICS approach provide a conceptual basis for language testing from an interactionalist perspective?

2009 Grant Summaries

Martin Carnoy

A Comparative Study of Teacher Quality and Student Performance in Southern Africa
Stanford University

Despite similarities in GNP per capita, family resources, and investments in schooling, South Africa and Botswana differ significantly in primary-school students’ performance on mathematics and reading tests. For example, on the 2003 Third International Mathematics and Science Survey (TIMSS), 8th-graders in Botswana scored 100 points higher in mathematics than their peers in South Africa. Regional data highlight similar outcomes. In this project, Martin Carnoy and a team of researchers in South Africa and Botswana attempt to explain these disparities by investigating the impact of school inputs and educational policies on student gains in mathematics. Using structural equation modeling, they seek to determine the relationship of teaching in classrooms and other school variables (e.g., language of instruction, reported violence in the school) to student learning gains in the two countries. To explain and analyze the impact of possible differences in quality of teaching and school context between Botswana and South Africa, they will examine, among other things, the relationship between teacher content knowledge/pedagogical content knowledge to student learning gains; the interaction between curriculum and teacher skills; the availability of textbooks, language of instruction, and class size; and the level of teacher training and its relationship to teacher mathematical knowledge and teaching quality.

Data will be collected in forty to fifty schools in each country. The PIs will randomly select these schools from a list of all primary schools with 6th grades in urban and rural areas of Botswana and South Africa within a specified distance from the Botswana/Northwest province (South Africa) border. Among the data to be collected are videotapes of 6th-grade mathematics lessons taught by 6th-grade teachers, test and survey data for all 6th-grade students, teacher questionnaire data, classroom observations, etc. In addition to these school-level data, information will be collected on teacher recruitment and assignment practices. These varied data resources are intended to capture variation in classroom practices in primary school classroom contexts in Botswana and South Africa and to provide insight on how these differences make a difference in student learning.

Carol Rutz, Carleton College
H. Scott Bierman, Carleton College
William Condon, Washington State University
Cathryn Manduca, Carleton College

Tracing the Effects of Faculty Development into Student Learning Outcomes

While there is ongoing debate about how best to measure student outcomes in higher education, there is less attention how teaching impacts these outcomes. In this study, Rutz and her colleagues approach this relationship by examining the relationships among faculty development, associated support structures, curricular change, and student learning. The ultimate goals of the study are to analyze faculty development and its impacts on instruction and student outcomes and to develop a model for a theory of change for planning, executing, and assessing faculty development that can be applied broadly in higher education in a cost-effective manner.

The study combines two three-year longitudinal projects—one at Carleton College, the other at Washington State University—to study the success of faculty development in improving student learning. Both projects focus on three cross-cutting literacies—writing across the curriculum, quantitative reasoning, and critical thinking. The study draws on diverse data from these projects, including pedagogical documents describing courses and instructional activities, questionnaires and surveys, focus groups, interviews, student performance and test data, and sample student work. The study’s analyses endeavor to control for biases introduced by self-selection into faculty development and to understand how faculty share learning among peers.

Eugenia F. Toma
J.S. Butler
Joshua Cowen
Megan E. Streams (Tennessee State University)

Teaching Careers in Rural Schools
University of Kentucky

In response to a state Supreme Court ruling that Kentucky schools were inequitable and inefficient, the Kentucky legislature in 1990 passed the Kentucky Education Reform Act (KERA). This sweeping piece of legislation in the areas of school finance, curriculum, and governance was intended to revamp the state education system in order to provide equal educational opportunities for all of Kentucky’s children, regardless of the property wealth of the district in which they lived. Dramatic changes to education finance in Kentucky resulting from KERA included the adoption of more centralized finance mechanisms and a statewide teacher salary schedule that linked teacher pay to both experience and credentials. In this study, Toma and her colleagues assess whether the reduction in expenditure variation mandated by KERA translates into reductions in de facto inequality between the state’s poor and richer school districts and to improvements in student educational opportunities in rural areas characterized by extreme poverty.

Building on previous research, Toma and her colleagues are particularly interested in the influence of reforms made under KERA on teacher quality within and across schools and school districts. Their analyses draw on extensive data, from 1980 to 2006, on teacher-level characteristics, such as salary, certification status, score on teacher entrance exam, age, gender, and ethnicity, as well as school- and district-level characteristics, such as district-level finances, district-level wage data for other occupations, and school-level demographics and standardized test scores. In analyses of these data, two questions are central:

  1. Do teachers who begin their careers in rural schools have weaker credentials than those who are initially hired in urban schools, and have these placement patterns been altered by the passage of school finance reform?
  2. Do teacher compensation, mobility, and retention differ in rural versus urban schools, and have these differences been altered by school finance reform?

The findings of this project should contribute to the literature on how school-finance reforms influence the flow of teachers to schools within a state, and to a better overall understanding of the special recruitment and retention issues facing rural districts.

2010 Grant Summaries

Andrea A. diSessa

Pathways to Equitable Science Instruction Based on Culturally Common Intuitive Knowledge
University of California, Berkeley

In this study, diSessa views students' prior, intuitive understandings of science that are derived from formal and informal experiences as important resources for student learning. Such prior understandings can be recrafted, combined with other ideas, refined, and used productively in learning science. In the proposed study, he will explore the nature and content of prior knowledge, and specifically how that knowledge can be stretched, supported, and built upon in educationally productive ways. He proposes to bring together ideas developed from research on conceptual change and ideas from culturally based studies of science instruction - what some researchers call "funds of knowledge," which are specific to the life experiences and cultural backgrounds of students. In drawing on these independent lines of research, diSessa proposes to develop a strategy for using culturally common knowledge about how the physical world works to help young people learn scientific concepts.

diSessa's research strategy is the design experiment: an iterative development of instruction based on current understandings of relevant scientific issues (e.g., students' intuitive knowledge); careful micro-study of early and pre-high school students' learning; and redesign and re-implementation of instruction. At the same time, a series of laboratory studies will be conducted, which may be formative studies of instructional units or studies of intuitive knowledge relevant to instruction. The instructional content of this work focuses on explanatory patterns of change and control that are powerful in science because of their general scope - ideas such as equilibrium, stability, and oscillation. Through this study diSessa intends to expand understanding about what it takes to help students see these science principles in familiar phenomena, advancing the study of intuitive knowledge and its role in teaching and learning.

Magdalene Lampert, University of Michigan
Megan L. Franke, University of California, Los Angeles
Elham Kazemi, University of Washington

Teacher Education for Learning In, From, and For Ambitious Practice

In recent years, a growing body of research has clarified what teachers need to do to enable children not only to do math competently, but also to understand it and use it to solve problems—or what Lampert, Franke, and Kazemi refer to as “ambitious” teaching. Still lacking are strong investigations of how to prepare teachers in university and school settings to do this kind of teaching. Furthermore, ambitious mathematics teaching is not an established practice in most schools. How, then, can preparation experiences be designed to address the problem of learning the practices of ambitious mathematics teaching, and what do teacher educators need to know about the institutional constraints and affordances for enacting this work?

In this study, Lampert and her co-PIs Franke and Kazemi assess the implementation and outcomes of a process they have designed and implemented at three sites—the University of Michigan, the University of Washington, and the University of California, Los Angeles—with six cohorts of 25-30 prospective teachers. The process prepares novices to teach in ways that foster both understanding and use of mathematics for elementary school children. The process rests on a set of instructional routines and instructional activities that enable novice teachers to learn in, from, and for ambitious teaching. For each mathematics instructional activity, a Cycle of Enactment and Investigation (CEI) guides the teacher-learner through performances in six stages, involving observation, collective analysis, instructional planning, public rehearsal, recorded enactment, and guided collective analysis of enactments of the instructional activity.

The study itself consist of analyses of a wide range of data on enactment of the design, how it is modified in use, and what prospective teachers learn from participating in it. Types of data include videotapes of class sessions, lesson plans and instructors’ handouts, video records of prospective teachers working in classrooms, their students’ completed work, and video records of teacher educators’ discussions within and across universities about the CEI process and instructional activities. Overall, the researchers aim both to assess the enactment and outcomes of their design for teacher education and to determine what is required to build a learning organization around the practice of teacher education entailed in the design.

Brian Rowan
Annemarie Palincsar
Joanne Carlisle
Karen Wixson

The Description of Reading Instruction Study: A Proposal for Continued Support. A Technical Proposal to the Spencer Foundation
University of Michigan

In carrying out this project, the principal investigators intend to describe the nature and contexts of reading instruction in U.S. schools and thus provide the education policy, research, and practice communities with accurate descriptive data on the nature and contexts of reading instruction in grades one and four. Although the need for these kinds of data and analyses are widely recognized, detailed descriptive data on reading instruction have not been collected routinely by government agencies or by other researchers in the U.S., and what data are available, according to the PIs, are of questionable accuracy. To address that void, analyses of already-collected data generated from a stratified random sampling of first- and fourth-grade teachers will enable the principal investigators to estimate the prevalence of various facets of reading instruction (e.g., decoding, fluency, comprehension, vocabulary, and writing) at different grade levels, and in different school and classroom settings, and to see how these facets of reading instruction are commonly taught. Such an examination will inform the extent to which instructional conditions vary across school and community settings and populations of children. Five research questions guide this project:

  1. How do various features of reading instruction (such as content-specific instructional practices, grouping and staffing patterns, and material use) cluster together to provide distinctive opportunities for students to learn to read at grades 1 and 4 in U.S. schools and classrooms?
  2. How prevalent are these patterned opportunities for learning as well as instructional supports for struggling readers, English-Language Learners, and students with Individualized Education Plans in grades 1 and 4 in U.S. schools and classrooms?
  3. Does the unfolding across grades of patterned opportunities to learn to read and supports for struggling readers vary as a function of school and classroom settings, that is, schools and classrooms serving students at different levels of economic advantage, linguistic and ethnic diversity, reading achievement, and behavioral regulation?
  4. Do the observed patterns of opportunity to learn at different grades and in different settings appear consistent with research on the effective teaching of reading?
  5. What are the implications of the observed findings for the formulation of education policy and/or the design of new instructional interventions?

Investigations of such questions should provide much-needed information about the nature and contexts of reading instruction in the U.S.


2011 Grant Summaries

Leigh L. Linden, Columbia University in the City of New York
Jean Grossman, Public/Private Ventures
Carla Herrera, Public/Private Ventures

Challenging Underserved Children to Achieve Academic Excellence

Although previous research points to the effects of high-quality educational experiences on children's test scores, graduation rates, and subsequent earnings, particularly for minority and economically disadvantaged youth, the evidence on the effectiveness of academically oriented out-of-school-time (OST) programs is mixed. Grossman, Herrera, and Linden seek to clarify this question by examining short-term and long-term effects of Higher Achievement, a high-quality OST program that provides students with academic support and assistance with placement in high schools that get them on track to college. Their study is based on a relatively large sample of students-fifth- through eighth-grade youth in the District of Columbia-within a rigorous randomized controlled trial design.

Research on the program began in 2006, and involves collection and analyses of multiple waves of data on three cohorts of students. The Spencer Foundation is providing support for collection of a fourth wave of data on the second cohort of students, which will assess the program's impact on students' decisions to apply to and matriculate at competitive area high schools. Overall, the study has two broad goals. First, it seeks to determine the program's impact on students' test scores, academic attitudes and behaviors, and matriculation at competitive high schools. Second, by analyzing the relationship of these outcomes to particular services provided by the program, it explores the mechanisms through which the program may affect student outcomes.

2012 Grant Summaries

Susan Carey
Deborah Zaitchik

Do Individual Differences in Executive Function Predict the Young Child's Ability to Learn Biology?
Harvard University

Teaching biology is hard. One reason for this is clear: Students do not come to the classroom as blank slates; instead, they bring with them systematic intuitive theories of many scientific domains. Science education therefore requires theory change, not merely the accumulation of new information. Given the central role of conceptual change in the acquisition of scientific theories, cognitive scientists and educators alike have put great effort into characterizing the social, pedagogical, and conceptual mechanisms that foster such change. This project explores the role of the Executive Functions– a suite of domain-general cognitive mechanisms that underlie higher order thinking – in the process of domain-specific conceptual change.

The co-PIs, Susan Carey and Deborah Zaitchik, focus on the developing theory of biology, hypothesizing that differences in the pace at which children develop this theory are due to individual differences in their Executive Functions. To test this hypothesis, the study measures each child's performance on three domain-general executive functions- inhibition, set-shifting, and working memory-as well as their performance on a conceptual battery of biology tests These conceptual tests focus on two developmental milestones in learning biology, the acquisition of 1) vitalist biology (understanding of the lifecycle and how body parts function as a system to support life); and 2) reproduction biology (understanding why offspring resemble their parents).

In one study, Carey and Zaitchik will replicate their preliminary findings that individuals' Executive Function scores predict their vitalist biology score and they will expand the investigation to see whether Executive Function scores predict reproduction biology as well. In a second study, the PIs will take advantage of a new curriculum (Tools of the Mind) currently being tested in Massachusetts schools that has been shown to enhance Executive Function skills in young children. Capitalizing on this larger effort, the PIs will explore whether children in the EF-enhancement curriculum show greater learning of their biology intervention (that is, larger differences between pre- and post-tests in biology) than those in the control curriculum. Given that neither the EF-enhancement curriculum nor the control curriculum mentions anything about biology, this study will provide a stringent test as to whether simply increasing children's domain-general executive function abilities will lead to better science learning.

Scaling Up a Promising Approach to Narrowing the SES Achievement Gap in Primary-Grade Social Studies and Content Literacy

Dr. Nell Duke, University of Michigan
Dr. Anne-Lise Halvorsen, Michigan State University

Scaling Up a Promising Approach to Narrowing the SES Achievement Gap in Primary-Grade Social Studies and Content Literacy

Duke and Halvorsen propose a study of the learning outcomes of children who have experienced project-based, integrated teaching of social studies and content literacy. They build on the research showing positive outcomes to both science and literacy learning from the integrated teaching of science and literacy and other studies suggesting that the integration of social studies and literacy might yield similar learning gains. In recent work funded by Spencer, Duke and Halvorsen developed two project-based units in content literacy plus three of the five strands of the Michigan second-grade social studies curriculum: economics; civics and government; and public discourse, decision making, and citizen involvement. Units were taught to second-grade children in low-SES school settings in Michigan. The outcomes on standards-based social studies and content literacy assessments rendered statistically insignificant the achievement gap between second graders in very low-SES and very high-SES school districts. Duke and Halverson now want to investigate whether these earlier achievement gains in social studies and literacy learning would be replicated in a larger and more representative group of teachers, across a larger number of second-grade classrooms, and across an entire year of curriculum (including geography and history).

Thirty second-grade teachers would be randomly assigned to learn and implement this approach to teaching social studies and content literacy or to a comparison group that would have the opportunity to implement the intervention after data collection is completed. Fifteen students per classroom will be selected at random for assessments at the beginning and end of the school year in social studies, literacy, and motivation. The classrooms would be drawn from schools with at least 80 percent of students receiving free and reduced price lunch and with school achievement scores below the state average on state exams in social studies, reading, and writing. Four additional classrooms would be drawn from very high-SES school districts and serve as a benchmark for the levels of achievement they hope to attain in the classes randomly assigned to the experimental group. The PIs propose to observe and document instruction of the project-based curriculum units as well as instruction in comparison classrooms. Other data would be generated by teacher interviews and measures of student progress. This project has promise for extending understandings about project-based pedagogy in social studies and content literacy and for advancing how schools approach achievement gaps in reading and social studies.

Elizabeth Graue, University of Wisconsin-Madison
Sharon Ryan, Rutgers, The State University of New Jersey

Life in Early Childhood Settings

Although the majority of preschool-aged children in the U.S. spend at least part of the time in out-of-home care, very little is known about the instructional quality, the emotional climate, and the learning opportunities that young children experience in preschool settings. With their examination of the interplay of state policy, program implementation, and children's experience in two state systems - New Jersey and Wisconsin - Graue and Ryan are developing a comparative case study to understand the opportunities for learning and teaching provided by each state's early childhood education policy context and how those policies are interpreted and experienced within each system at differing levels and in various administrative and instructional roles. In the field of research on pre-kindergarten programs, this study represents a unique effort to examine multiple levels of state pre-K systems at once, including the policy context, the instantiation of the state policy in local sites, and the experiences of children served in those sites.

By attending to the relationships among policy interpretation, quality of program, and children's outcomes, the PIs aim to avoid the limitations of prior studies that offer snapshots of early childhood education programs, but cannot explain why structurally similar programs (i.e., programs emanating from similar policies) vary significantly in quality of instruction. Reflecting the study's interest in the interrelationship of state policy, program implementation, and children's experience, data will be gathered from a wide variety of sources: policy documents, curricula, program descriptions, administrator and teacher interviews, classroom observations, child observations and interviews, and home visits. Ultimately, this project stands to improve our understanding of how state quality standards become actualized at the local community and classroom levels, the kinds of support provided to program administrators and instructional staff to understand and interpret quality standards, and how these factors combine to produce children's experiences and outcomes.