Supporting What Works: Evidence Use Versus Evidence Transactions
TLDR: Causal research in education — the kind designed to tell us whether a particular program or practice actually works — has delivered real insights over the past two decades. But it has yet to deliver on its full promise. A big reason is that we treat evidence generation as a series of transactions and evidence use as someone else’s problem. Until we fund and design research with systems change in mind, the gap between what we know and what schools and students actually experience will remain.
Educational research encompasses a wide variety of designs aimed at answering different kinds of questions. A qualitative study might explore how students from different backgrounds experience a new school policy. A large-scale survey could document trends in teacher retention across states. Neither of those designs, however, can tell you whether a particular reading curriculum caused students to learn more than they would have otherwise, or whether an advising intervention actually improved college persistence rates. Answering questions about effectiveness — that is, whether we can causally link a specific policy, program, or practice to improved outcomes — requires a research design specifically built for that purpose.
The randomized controlled trial (RCT), also known as an experimental design, is the approach most often used to address “Does X work?” questions. In an RCT, the unit of study — a student, a classroom, a school — is assigned to one of two groups through a random process, much like a coin flip. One group receives the intervention being studied; the other (the control or comparison group) does not, or continues with business as usual. Random assignment matters because it is what allows researchers to attribute differences in outcomes to the intervention itself, rather than to pre-existing differences between the groups. If students in a tutoring study could choose which group to join, the students who signed up might already be more motivated – one example of what researchers call “selection bias.” We’d have no way to untangle the effect of the tutoring from all the other ways those students were different. Random assignment eliminates that problem. It’s also why participants can’t switch groups after the fact: doing so re-introduces exactly the kind of selection bias we’re trying to rule out.
Experiments in Education Are a Relatively New Development
RCTs have a long history in other fields. Clinical trials to test whether new cancer treatments improve patient survival have been a cornerstone of medical research for decades. Agricultural researchers have used experimental designs to determine whether new seed varieties produce more drought-resistant crops. The logic is the same: randomly assign units to conditions, and measure what happens.
In education, the story is more recent and more uneven. As researchers Larry Hedges and Jake Schauer have documented, the United States essentially abandoned experimental education research for roughly two decades, from the 1980s through the early 2000s — in large part because early large-scale experiments produced disappointing results. The shift came with the establishment of the Institute of Education Sciences (IES) in 2002, which invested heavily in RCTs and rigorous quasi-experimental designs across early childhood, K–12, and postsecondary education. Private philanthropy followed suit.
I have been part of this wave. Over the course of my career I have contributed to several large experimental studies in education. I believe this kind of research has a genuinely valuable role to play in improving how our systems of education — from early childhood through workforce training — serve learners. However, I have also witnessed firsthand how causal research in education has failed to live up to its promise. Not because the designs are wrong, but because of how we’ve built the systems around them.
The Transaction Problem
Here is a pattern that plays out with remarkable consistency in large-scale education RCTs: researchers identify a promising intervention and spend months — sometimes longer — building a multifaceted recruitment campaign to persuade school districts and other partners to participate. Partners must be willing to implement the intervention as designed and, crucially, equally willing not to implement it if they are randomly assigned to the control group. That is a significant ask. Case studies of RCTs in schools document the friction this creates: schools drop out, principals change, superintendents turn over, and what began as a cleanly designed study becomes something messier and more expensive than anyone planned.
At the other end, evidence dissemination looks a lot like a transaction, too. The study concludes. A report is released. Perhaps partners receive a presentation of findings. Too often, the research team then moves on. What happens to that evidence once the researchers leave the building is usually not a meaningful part of the project’s scope or budget.
The result is a persistent, well-documented gap between what causal research produces and what education systems actually do with it. Studies show that educators often feel disconnected from research, are unlikely to reach out directly to scholars, and may not see findings as relevant to their specific context. This is not primarily a failure of communication or dissemination — though those matter. It is a structural failure. We have built a research enterprise oriented toward generating findings and a practice world largely left on its own to act on them.
Complex Problems Require a Different Frame
The kinds of challenges in education that motivate expensive, time-intensive causal research do so precisely because they are not simple; otherwise, we would have solved them by now. They are embedded in complex systems: systems with shifting leadership, competing priorities, under-resourced implementation teams, and communities with their own histories and perspectives on what schools are for. A randomized trial can tell us that a particular math curriculum produced significantly higher test scores in the study sample. It cannot, on its own, tell a district how to embed that curriculum in its professional development calendar, how to support teachers who resist it, or how to sustain it through a principal transition.
This means we need to think about evidence use — not just evidence generation — through the lens of systems change. The question is not just “what works?” but “under what conditions, for whom, and how do we build the organizational capacity to actually use what we learn?” Research-practice partnerships, continuous improvement models, and design-based implementation research are all approaches that take this systems view seriously. They are also, not coincidentally, harder to fund and harder to fit into a traditional grant cycle.
What We Should Do Instead
Changing things will require different expectations from researchers, funders, and partners alike.
For researchers and evaluators: Understanding the systems you are working in is not background context — it is core to the work. That means engaging partners not just at the recruitment and dissemination stages but throughout: helping them understand what the evidence means, what it doesn’t mean, and how to navigate the organizational change required to act on it. This kind of sustained involvement is rarely budgeted for, which is why it rarely happens. But without it, the shelf life of a study report is short.
For funders: Fund the work above. This may mean funding by invitation, with a smaller number of deeper, longer-term partnerships rather than a broad competitive grant pool. If that is the choice, be transparent about it and revisit it regularly — it carries real equity implications for who gets to participate in the research enterprise. It also means allocating funding specifically for systems change work throughout the project or initiative, not just at the end, and requiring grantees to have a concrete plan for how evidence will be used to influence systems — not just disseminated.
For partners and participants: Ask early and ask directly: What support will you provide us, throughout and beyond this study, to understand and act on what we learn together? If the honest answer is a final report and a debrief session, that is useful information. It should shape how much of your staff’s time and goodwill you invest in the work.
Causal research in education has produced real knowledge. We know more than we did twenty years ago about what kinds of early childhood interventions improve school readiness, which tutoring models improve math achievement, and which college supports help students persist to graduation. The problem is not the research. The problem is that we keep treating evidence as a product to be delivered rather than a resource to be integrated. Until we build that integration into how we fund, design, and conduct research — and into what we expect of our partners — we will keep generating findings that sit on shelves while the systems we care about continue to struggle.