Evidence Schmedivence: Grinding Axes
I have an axe to grind. Sour grapes if you prefer. If you do any reading about measurement-based care (MBC) or routine outcome monitoring (ROM), you will notice that MBC and ROM are constantly referred to as evidence based. This is total BS, a marketing and corporate misinformation campaign endorsed by all the corporate MBC providers as well as the American Psychological Association (APA). APA would rather support their corporate sponsors and academic good ol’ boys and plagiarizers than promote the actual research of MBC and who conducted it. Only two systems, the Partners for Change Outcome Management System (PCOMS) and the Outcome Questionnaire 45 (OQ) have extensive randomized clinical trial (RCT) support and were listed by the Substance Abuse and Mental Health Administration in the National Registry of Evidence-based Programs and Practices. Although there are RCTs of other MBC systems or measures, the OQ and PCOMS are the only systems or measures with consistent and substantial demonstrated results of improved outcomes and reduced dropouts (read the RCTs here).
In other words, MBC does have the oft-cited evidence base, but not with the measures, approaches, and software systems with which it is most associated. Flicking any form of any psychometric validity with any accompanying clinical process and calling it good is not evidence based.
The Worst Part
Moreover, and the part that rankles me most is that the MBC clinical process that is now promoted is described as though it was recently invented—when in reality, it is a rebranded and plagiarized version of a process with a long history. As just one example among many, a recent article by well-known authors in a top tier journal said:
Lewis et al. (2019), in their review of the MBC literature, established four core elements of MBC: (1) a routinely administered symptom, outcome, or process measure, ideally before each clinical encounter; (2) practitioner review of data; (3) patient review of data; and (4) collaborative reevaluation of the treatment plan informed by data.
However, to our knowledge, there have been no published descriptions of how to operationalize these elements into a well-defined and easily-implemented clinical model with steps drawn from the MBC and broader mental health literature. Also lacking in the literature is information on how providers should engage with clients in a way that embodies the transparent, collaborative style of MBC so they can make MBC relevant for each client…
Really, no published descriptions?!! Starting with the first PCOMS manual twenty-four years ago (Duncan & Sparks, 2002), now in its fourth edition (Duncan & Sparks, 2024), and many other publication (e.g., Duncan, 2010, 2012, 2014, Duncan & Reese, 2015; Murphy & Duncan, 2007), we articulated a “well-defined and easily-implemented” collaborative, transparent methodology as well as a full clinical protocol for practice, training, supervision, and, importantly, with an emphasis on social justice. We have been at this for a long time. While the field may only now be recognizing that MBC is far more than flicking forms, a client privileged, collaborative partnership has been an articulated part of the DNA of PCOMS since the beginning. See Duncan & Reese (2024) for a full history of MBC and PCOMS.
We operationalized these elements long before Lewis et al. reviewed the MBC literature (also without reference and particularly interesting because the authors were from Kaiser where I had trained over 3000 therapists in the clinical process of PCOMS to use with their measures) and empirically confirmed those elements, with the help of my friends, via RCTs. It is telling that these well-known authors did not include even one reference to our twenty-five years of doing MBC as a clinical process of client privilege, collaboration, and involvement of clients in all decisions that affect their care. Of course, PCOMS was a clinical process from the beginning because it arose from my clinical practice, not from research or academia. You would think that would be a positive, but apparently not with researchers and academics.
Making Sense of This
Why does this happen. I’ll leave that to your imagination but here is a clue. Scholarship can be a petty, ego-driven, and good ol’ boy publication and referencing process. The central question is, “Do you belong to the club?”
What to do? I have called for corrections and succeeded (which made powerful enemies in the club), emailed authors and sometimes received responses usually saying that excluding our body of work was unintentional and that actually they were big fans. Funny way of showing it. As co-developer of perhaps the most widely used measures on the planet (over 40,000 registered users on our website), a clinician, trainer, and implementer of MBC for 25 years, as well as the co-developer of MBC software containing 3 million administrations, co-leader of a research team with 8 MBC RCTs, original articulator of the clinical process often noted, it is hard to read anything about MBC without even a nod to this body of work in the references. It’s difficult way beyond my personal pride because it is a kick in the face to all the hard work, dedication, and sacrifices of my friends and colleagues. They tirelessly worked with me over the past 25 years without pay and grant support because they believed in PCOMS–that it operationalized the values of client privilege and collaboration, and improved the quality of care. These “oversights” dishonor the memory of my best friends.
With this blog, I am leaving this behind and taking the advice of friends. Let it go, Barry. Let it be.