Leveraging Lotteries for Value-Added: Bias Reduction vs. Efficiency
Joshua D. Angrist, Parag A. Pathak, Christopher R. Walters, and Peter D. Hull
Massachusetts Institute of Technology
School districts increasingly use matching algorithms to assign students to public schools. Many of these matching schemes incorporate an element of random assignment. Randomization can be exploited to construct causal estimates of the effects of attendance at individual schools and various types of schools. This opens the door to improved estimation of value-added models (VAMs) and other measures of schooling outcomes. This study aims to answer the following research questions about VAM estimation using random assignment: (1) How can partially-randomized algorithmic matching be most effectively leveraged for VAM estimation? (2) How can relatively precise observational but potentially biased regression estimates of VAM be combined with unbiased but less precise lottery-based estimates to best measure school effectiveness? (3) How much bias is there in conventional regression estimates of VAM? (4) Estimates that seem to on target on average may nevertheless provide a poor guide to decisions about individual schools. How can randomized and non-randomized identification strategies be combined to minimize the risk of school misclassification?
The answers to these questions will be of practical value as well as of scientific and policy interest. The framework the research team proposes to develop can be used to make school report cards and accountability systems more accurate and to gauge the quality of currently used VAM estimates.
School Choice in Indiana: An Examination of Impacts and the Conditions Under Which Choice is Effective
University of Notre Dame
The proposed research examines choice options in the state of Indiana, specifically the voucher program and the expansion of charter schools. Berends and his team will address the following questions: (1) What is the impact of the Indiana Choice Scholarship Program on student achievement gains and the schools these students attend? (2) What is the impact of the charter schools on student achievement gains? (3) Are these voucher and charter school impacts greater for some groups of students compared with others, having effects on the racial/ethnic and socioeconomic achievement gaps? (4) How do schools of choice (charter or private schools) differ from traditional public schools in terms of organizational and instructional conditions, school leadership, professional capacity, school learning climate and funding conditions, and parent involvement and support that promote achievement? The study will examine the achievement gains and growth of Indiana students receiving vouchers and those choosing charter schools using student longitudinal data records from the Indiana Department of Education. With additional data collection from schools and teachers in a representative sample of K-8 traditional public schools, charter schools, and private schools, they will be able to examine the conditions under which the impacts of the voucher and charter schools occur.
The New 'One Best System?': Urban Governance and Educational Practice in the Portfolio Management Model
Katrina Bulkley, Julie Ann Marsh, Douglas N. Harris, and Katharine O. Strunk
Montclair State University
Dramatic recent changes in many US districts, consistent with the idea of a Portfolio Management Model (PMM), shift from direct district management of schools towards “portfolio managers” including central offices and charter authorizers. Portfolio managers oversee schools operating under varied governance structures, including charter and autonomous schools. Limited research examines this potentially fundamental change in public education governance and the effects of system change on educational practice, an issue of considerable importance. This study will examine ties between PMM infrastructure and practices of system-level actors, educational management organizations, schools, and intermediate outcomes linked with student learning. The research focuses on five central mechanisms in the PMM theory of change – autonomy, accountability, school choice, human capital, and capacity-building. The PMM theory of action relies heavily on ideas connected to principal-agent theory, altering governance structures to better align goals between “principals” (organizational leaders) and “agents” (those who answer to principals). Institutional theories supplement principal-agent theory by highlighting the cultural and political forces that shape educational change. Our mixed-methods study will utilize these theoretical frameworks to examine three educational systems – Los Angeles, New Orleans, and Denver – from the system to the school levels, using interviews, surveys, case studies, and administrative data.
Using Multiple Lenses to Investigate the Development of Content Knowledge and Teaching Practices in Relationship to Learning Opportunities
Elizabeth A. Davis and Annemarie Palincsar
University of Michigan
This project will advance understanding of education practice and its development and improvement. The research team will investigate, using complementary conceptual frameworks, how novice elementary teachers develop—from program entry through their first year of teaching—content knowledge and high-leverage teaching practices for ambitious science teaching. The team will describe trajectories of the development of this knowledge and practice over time and connect those trajectories to characteristics of the program, the novice teachers, and the schools in which they teach.
Elementary science is taking on increasing importance, yet novices face challenges in learning to teach science. At the elementary level, preparing teachers to teach science well must include attending to their preparation to teach the reading, writing, mathematics, and oral language competence that support science learning. Indeed, this team contends that elementary teacher preparation must always be conceived of at the level of a program, rather than at the level of specific courses or field work, because of the nature of elementary teaching.
This project will take place in a coherent, practice-oriented, innovative elementary teacher education program with an extensive research infrastructure in place. The research team will use a staggered-start, two-cohort, multi-level longitudinal study design. Data will include program assessments, video records of science teaching, interviews, and other program materials.
Broadening Participation in a Computational Future: Casting a Wide Net
Michael Horn, Kemi Jona, Kai Orton, and Uri Wilensky
STEM education faces two interrelated challenges. First, our education system is not producing enough computational professionals to fill demand. Second, women and minorities remain significantly underrepresented. The research team proposes a Design-Based Implementation Research project to research a model in which computational literacy (CL) curricula are embedded throughout required biology, chemistry, physics, and mathematics coursework. This would ensure that every high school student is exposed to CL activities in a context that demonstrates its relevance to real world career paths. The team will work intensively with three Chicago Public High Schools serving predominantly low-income minority student populations. Our project builds on existing work and seeks to address the following questions: (1) How does a progressive, graduated exposure to such curricular materials impact students’ performance on CL tasks and to the learning of STEM concepts themselves? (2) How does exposure to CL curricular materials affect students’ attitudes, identity, and educational and career expectations? Are these relationships similar or different for students of different genders, races/ethnicities, and socioeconomic backgrounds? (3) What individual and institutional factors affect the success of this and similar innovations in resource-constrained public schools?
The Education Governance & Accountability Project
Stephane Lavertu, Vladimir Kogan, and Zachary Peskowitz
The Ohio State University
The Education Governance and Accountability Project aims to improve understanding of the political institutions governing U.S. public education, so that they may be designed to promote democratic accountability and the efficient provision of K-12 education. The project seeks to achieve this goal by collecting panel data on school district politics across twenty states, making these data available to researchers, and applying rigorous social science research techniques to understand how the politics of public education affect school administration, instruction, and student learning. Specifically, the research team will collect data on school board composition and local school board and levy elections, as well as publicly available data on the characteristics of school and district staff, students, and residents, district finances and management, and student educational attainment and achievement. These data and the analyses that the project team produces will be made publicly available via the project’s website and will help to promote novel, interdisciplinary, and policy-relevant research on the political context affecting instruction and student learning.
Supports for Rich and Explicit Language and Vocabulary Instruction in Preschool Classrooms Serving Children from Disadvantaged Backgrounds
Beth M. Phillips and Carla Wood
Florida State University Research Foundation, Inc.
This project is an in-depth, multi-level longitudinal investigation into predictors, moderators, and mediators of the language environment created by teachers within preschool classrooms serving children from disadvantaged backgrounds, particularly those at high risk of poor school readiness and later academic difficulties because of home environments of poverty. The project also will target some classrooms serving English Language Learners. The focus is to identify malleable features of classroom contexts, and of teacher knowledge and skills, most relevant to the provision of high quality language support and instruction. Classroom observations, teacher and child language samples, and child language assessments will be conducted three times in 100 diverse classrooms. Both general and child-specific language environments will be assessed to allow investigation of bidirectionality between child language skill and the language support they receive; this will be explored for children who enter the classroom with high, moderate, and low language skills. As teacher pedagogical content knowledge is a key theoretical predictor of the language environment, a complementary measurement development study will support the refinement and utility of a preliminary item set. Findings will support effective professional development designed to improve the critical classroom language and vocabulary instructional environment for preschool children.
A Comparative Study of School Systems: Infrastructure, Practice, and instructional improvement
James P. Spillane and Donald J. Peurach
This is a comparative study of how domestic school systems design, manage and enact teaching, learning, and related practices. Central topics include: structural, organizational, and cultural differences among systems; how differences impact teaching, learning and related practices; non-cognitive “lessons” systems communicate to students, teachers, and leaders in informal socialization and intentional rules; how systems define and manage their environment; and how that environment affects them.
System structure includes formal organization, infrastructure, and social organization. System structure effect on instruction and related practices depends in part on the extent to which elements of structure are designed to influence and be influenced by instruction; the extent to which consistency is designed into elements of the system; the tightness of coupling among the elements; and the extent to which the environment supports system operation. Candidate systems include charter networks, urban LEAs that implement the Common Core, Catholic systems, Montessori, and sub-systems including New Visions for Public Schools in NYC and the International Baccalaureate.
Systems and schools within will be purposefully sampled. Evidence on instruction will include academic tasks and student work. Data analysis will combine deductive and inductive approaches using open and closed coding strategies.
Parenting Matters? Examining the Value and Optimal Approaches of Family Engagement in Educating Students of Color
Ming-Te Wang and James P. Huguley
University of Pittsburgh
Although family involvement remains a significant influence on academic achievement in middle school, the nature of this involvement may change across developmental stages in response to children’s psychological needs and evolving school structure. However, extant theories of family-school relations are largely derived from elementary school models and do not account for developmentally appropriate practices that may be unique to middle school contexts. Additionally, many family studies take a deficit approach that privileges White, middle-class family norms over those of families of color, and suggest that a lack of involvement among families of color, or those living in poverty, is chiefly responsible for diminished educational outcomes. However, families of color may be involved in their children’s education in ways that are not captured by White, middle-class standards. Thus, racially comparative, deficit-oriented models fail to affirm the cultural richness present in many communities, and ignore the more culturally responsive involvement approaches that low SES families and families of color contribute to their children’s academic success. In response, this project aims to broaden and refine current theories of family involvement in education by incorporating developmental and strength-based perspectives, and identifying parenting practices that are developmentally appropriate and culturally sensitive to the unique experiences of students of color in middle school.
The Development of Ambitious Instruction in Elementary Mathematics and English Language Arts
Peter Youngs, Corey J. Drake, Dorothea Anagnostopoulos, Julie J. Cohen, and Spyros Konstantopoulos
University of Virginia
Preparing large numbers of elementary teachers to enact ambitious instruction is a central policy challenge in the United States. Such instruction involves helping students develop strong conceptual knowledge and engage in disciplinary practices. Research indicates that ambitious instruction is associated with elementary student achievement in mathematics, reading, and writing. But no studies have followed elementary teacher candidates from multiple preparation programs to assess how their own characteristics (i.e., their teaching identity, beliefs, and knowledge) interact with their learning opportunities in university courses and student teaching to affect their enactment of ambitious instruction. Further, no studies have examined how novice teachers’ characteristics interact with resources and expectations in their schools to affect their instruction as full-time teachers. To meet the need for ambitious instruction, a stronger research base is needed about how to support beginning teachers in enacting such instruction. To that end, this research team proposes a four-year, longitudinal, mixed-methods study that will investigate a) how a purposively sampled set of three teacher preparation programs supports elementary candidates in developing ambitious instruction in both mathematics and English Language Arts (ELA) and b) factors that are associated with how graduates of these programs enact mathematics and ELA instruction as first- and second-year teachers.