The Trap
Every official charged with spending opioid settlement money has been given the same instruction, and it sounds like wisdom: fund evidence-based practices. Spend the money on what works. Don’t repeat the tobacco settlement, where billions vanished into budget gaps and feel-good programs that no one could ever show had helped a single person.
The instruction is correct. The trouble is that almost no one who receives it has been told what “evidence-based practice” actually means — or, more dangerously, what it does not mean. And the gap between the phrase and its reality is exactly where the next debacle is taking shape.
This is the second in a series of papers on the evidentiary foundations of opioid abatement. The first, The $57 Billion Wager We Cannot Afford to Lose, argued that the settlement will be judged not by how the money was spent but by whether anyone can prove it mattered. This paper takes up the harder question underneath that one: what would proof actually require, and why is the field so badly positioned to produce it?
We write from a particular vantage. One of us began studying this specific problem — how to express evidence-based practice through real-world service delivery — in work first circulated in 2020, well before the settlement funds began to flow.1 What looked then like an academic puzzle has become a public one, with billions of dollars riding on the answer.
The First Misunderstanding: What “Evidence-Based” Means
Most people hear “evidence-based practice” and picture a settled thing: a treatment that has been proven to work, like a drug that cleared its trials. Fund the proven thing, the reasoning goes, and you have done your due diligence.
But evidence-based practice was never meant to describe a fixed catalog of approved treatments. As the field’s foundational definition has it, it is the integration of the best available research with clinical expertise, in the context of the individual’s characteristics, culture, and preferences.2 Read that again, because nearly every word cuts against the common understanding. It is an integration, not a lookup. It depends on expertise and context, not just on a study. And it explicitly centers the individual — their life, their circumstances, their preferences — as part of what makes a practice work.
It is worth giving this its proper name, because the distinction it draws is one that every clinician and service provider already knows in their hands. There is evidence-based practice, and there is evidence-based treatment, and they are not the same thing. Evidence-based practice is best practice: research and guidance, applied with skill and judgment, shaped to the person in front of you. Evidence-based treatment is the cookie-cutter version — a fixed protocol for a diagnosis, delivered the same way to everyone who carries the label. One is a profession. The other is a checklist.
Anyone who has delivered care inside the current payment system has felt the difference between them, whether or not they had the words for it. Evidence-based treatment is, in large part, what payers want — and for understandable reasons. A protocol is standardized; it can be specified, billed, audited, and held accountable. Judgment cannot be billed. Context cannot be audited. There is, as one of the field’s clearest observers put it, “a certain seductive appeal to the idea of making psychological interventions dummy-proof, where the users — the client and the therapist — are basically irrelevant.”3 So the system, reasonably pursuing accountability, rewards the version of “evidence-based” it can administer — and quietly strips out the expertise and the individual fit that the actual definition places at the center. The clinician feels their judgment squeezed to the margins not because anyone decided it didn’t matter, but because the part that matters most is the part that is hardest to put on a form.
This is the first misunderstanding, and it is the costly one. The narrow reading took hold not because it is correct but because it is administrable. A list can be checked. A registry can be cited. A box can be ticked. The richer reality is harder to administer, and so it quietly fell away — leaving recipients with a checklist version of an idea that was never a checklist. And a county that funds the checklist version, believing it has funded the practice, has already taken the first wrong turn toward the trap.
The Second Misunderstanding: What the Evidence Actually Says
Here is the finding that should reorganize how settlement recipients think about their money, and that most have never been shown directly.
When researchers account for everything that determines whether a person in treatment actually gets better, the specific evidence-based model being delivered — the named program, the thing on the registry — explains only a modest part of the outcome.4 The larger share is carried by other factors: the person’s own life circumstances and resources, the strength of the relationship between the person and those helping them, the skill of the individual provider, and whether anyone is engaged with the person and tracking whether they are actually improving — adjusting course when they are not.
Anyone who has done this work recognizes these factors immediately, because they are the texture of the work itself. The model on the page is never what carried the day; the connection with the person, the moment they decided to stay, the small adjustment that met them where they were — that is where the work lived. The research is simply naming, in its own terms, what practitioners have always known in theirs.
This is not an argument that evidence-based programs do not matter. They do. Using a validated approach beats using an unvalidated one, and a program with no evidence behind it has no business receiving public funds. The point is more precise, and more consequential: having an evidence-based program is a necessary condition, but it is not the thing that drives the outcome. It is the threshold you must clear, not the lever that moves the result.
A county that funds only the narrowly “evidence-based” line items — and pays no attention to the factors that actually carry the outcome — has cleared the threshold and missed the lever. It has done the administrable thing and called it diligence. And when, years from now, someone asks whether the money worked, that county will be in precisely the position the tobacco states were: able to show what it funded, unable to show what it changed.
A Third Misunderstanding: “Evidence-Based” Is Not One Thing
There is a third misunderstanding, and it is the one that most often leaves good programs stuck. Even once we are clear that evidence-based practice is a profession rather than a fixed protocol, we still tend to talk about it as though an intervention either is evidence-based or isn’t — as though there were a single list, and the job is to get on it. But when a real program holds itself up against the evidence, it is almost always in one of three different situations, and knowing which one you are in is the first honest step.
You may be expressing an established practice — delivering something that has already been validated, where the work is faithfulness to a known standard. You may be extending a practice — taking something validated in one setting and adapting it to a new population or context, where the work is disciplined adaptation and an honest account of what carries over and what doesn’t. Or you may be creating where little formal evidence yet exists — building a practice in territory the research hasn’t reached, where the work is constructing a defensible case from the ground up, one measured result at a time.
Every practitioner has done all three, often in the same week, without stopping to label them. All three are legitimate. None is a failure to be “real” evidence-based practice. But they demand different kinds of rigor — and a recipient who treats all three as a single checkbox will misjudge every one of them.
Where the Lever Actually Is
If the factors that carry the outcome are the person’s circumstances, their relationships, their engagement, and the attention paid to their progress, then the interventions that work on those factors are not peripheral to abatement. They are central to it.
These are the social and community-based interventions: peer support and recovery coaching, stable housing, connection to work, family engagement, the navigation that links a person to the care they need and keeps them there. For years these have been treated as the soft edge of the field — worthy, perhaps, but not “real” evidence-based practice, not the rigorous core. And notice why: they are precisely the interventions most often in the extending or creating situations rather than the expressing one. They don’t come with a registry entry and a fixed protocol, so the checklist mindset waves them off as unproven.
The evidence says the opposite. These interventions act directly on the factors that carry most of the outcome. They are not the soft edge. They are the lever. And they are exactly where a large and growing share of opioid settlement money is flowing — into recovery housing, peer programs, community organizations, and social services. Which brings us to the cruel irony at the heart of this paper.
The Bandwidth Problem
The organizations delivering these high-value interventions are, almost by definition, the ones least equipped to prove their value.
It is tempting to say they lack the infrastructure for rigorous evidence. That is not quite right, and the distinction matters. They do not lack intelligence, or commitment, or even research understanding. What they lack is bandwidth. A peer recovery organization with three staff serving a county of forty thousand does not have a research division. It does not have a data analyst, an evaluation budget, or an idle afternoon. Its people are extraordinary and exhausted, working in the most financially punishing corner of an already-punishing system. They were never offered a way to generate rigorous evidence within the constraints under which they actually live — so they never imagined they were missing one.
This is the quiet engine of the tobacco trap, and it has nothing to do with anyone’s competence. The interventions that matter most for outcomes are delivered by the organizations with the least capacity to document context, interventions, and outcomes. So the money flows to the right place but produces no evidence — not because the work isn’t working, but because no one in that work has a spare hour to prove it. Years later, the absence of evidence is read as the absence of impact. The programs are cut. The debacle completes itself, and the people who did the actual work are blamed for a failure that was structural from the start.
We are not speculating that this evidence is missing. The people doing the most careful national accounting of settlement spending have said so plainly. The Johns Hopkins–led tracking effort, working with KFF and Shatterproof, has built the only systematic picture of where the money is going — and it states, in its own methodology, that the quality of the funded programs and their alignment with evidence-based practice were not assessed.5 The most authoritative tracking project in the country can tell us where the money went, by category, but not whether any of it was evidence-based — because the information is not there to assess. The same effort has flagged spending activities not backed up by evidence,6 and noted that it could not distinguish genuine evidence-based prevention from one-time speakers and events dressed in the same language. The vacuum this paper describes is not a worry about the future. It is already documented in the present.
What This Means for Recipients
The conclusion is not “fund evidence-based practices,” which recipients have already been told and which, understood as a checklist, leads them astray. It is something more demanding, and more useful.
First, understand that an evidence-based program is your floor, not your ceiling. Requiring it is right. Believing it is sufficient is the trap.
Second, recognize that the interventions with the greatest effect on outcomes — the social and engagement-based ones — are the ones your accountability framework is most likely to undervalue and your funded organizations are least able to document. That is not a reason to fund them less. It is a reason to fund the capacity to measure them.
Third, understand that the binding constraint on evidence is bandwidth, not will or ability. The organizations doing the most important work cannot be asked to become research institutions on the side. If rigorous evidence is going to come from community-based abatement — and the settlement’s credibility depends on it — then the means to produce that evidence must be built into the infrastructure that supports the work — provided to organizations that have no spare capacity, not presumed to be something they can build themselves.
The tobacco settlement did not fail because the money went to the wrong places. Much of it went to reasonable places. It failed because no one built the means to know. The opioid settlement is being handed the same opportunity to fail, dressed up this time in the language of evidence-based practice — a phrase that, misunderstood, provides the perfect cover for repeating history while believing you are avoiding it.
The difference between the two outcomes will not be found in the list of what gets funded. It will be found in whether anyone can prove what changed. That is a question of method, and of detail, and of doing the unglamorous work of measurement well — which is precisely the work the field has always treated as someone else’s problem.
It is no longer someone else’s problem. It is the whole problem.
Disclosure
The authors are the founding team of Essentrify, Inc., which is developing infrastructure to help communities generate rigorous evidence from the kinds of interventions this paper describes. We have a commercial interest in the problem we are analyzing, and we disclose it plainly. We have tried to write an account that stands on its own merits regardless of that interest; readers should weigh it accordingly.
Notes
- Dormer, D. E., & Sheldon, J. (2020; updated March 2022). The Challenge of Expressing Evidence-Based Practice (EBP) Through a Digital Health Platform. ↩
- APA Presidential Task Force on Evidence-Based Practice (2006). Evidence-based practice in psychology. American Psychologist, 61(4), 271–285 (definition at p. 273). ↩
- Duncan, B. L., & Reese, R. J. (2013). Empirically supported treatments, evidence-based treatments, and evidence-based practice. In I. B. Weiner (Ed.), Handbook of Psychology (2nd ed.), Vol. 8, ch. 21. Quotation as cited therein. On the formal EBT standard, see also Chambless, D. L., et al. (1997), Update on empirically validated therapies, II. ↩
- On the relative contribution of model/technique versus client, relationship, provider, and feedback factors to outcome variance, see the common-factors literature summarized in Duncan & Reese (2013), drawing on Lambert, Wampold, and Asay & Lambert, The Heart and Soul of Change. We deliberately characterize the model’s contribution as modest rather than fixing it to a single estimate, which varies by analytic frame. ↩
- Johns Hopkins Bloomberg School of Public Health, KFF Health News, and Shatterproof (2024). Opioid Settlement Expenditures Methodology. kffhealthnews.org. The methodology explicitly notes that program quality and alignment with evidence-based practice were not assessed in the tracking framework. ↩
- Johns Hopkins Bloomberg School of Public Health, KFF Health News, and Shatterproof (2024). 2024 National Settlement Fund Tracking Project: What we learned. Opioid Principles. The report identifies that settlement fund expenditures included categories of spending not necessarily supported by evidence-based treatment or prevention models and highlights difficulty distinguishing evidence-based prevention activities from non-evidence-based approaches such as one-time educational talks or standalone events. ↩