ORS and SRS: Understanding the Psychometrics

In this historical series of blog posts discussing the journey to client-directed therapeutic services, the previous post shared how the clinical process of the Partners for Change Outcome Management System (PCOMS) was developed and first presented in Heroic Clients, Heroic Agencies: Partners for Change. This blog post addresses the first step toward legitimizing PCOMS in the larger scientific community, the psychometric validation of the PCOMS family of instruments.

Recall that the development of the PCOMS measures and the attending clinical protocol were organic processes. The Outcome Rating Scale (ORS) and Session Rating Scale (SRS) family of measures did not follow the typical path of rigorous instrument development and psychometric evaluation. Rather, the PCOMS measures started from everyday clinical practice and a desire to privilege the client in therapy. I had no idea that the ORS and SRS would become psychometrically validated instruments and a part of a designated evidence-based practice. But, of course, we never really know what is going to happen to us in this life.

Understanding Psychometrics and Validation

I was not a psychometrician and never really wanted to be. I was a therapist first and foremost. However, when it became apparent that the use of the ORS and SRS had clinical utility and could functionally guarantee the privileging of clients, I realized that they had to be psychometrically validated to achieve large scale dissemination.

The evil plan came to me: legitimize the measures and gain academic acceptance—and therefore, ensure client privilege and client-directed services.

So, I dug into the psychometric literature and learned how validation studies were done. My research yielded a very interesting fact. Namely, that all outcome measures, regardless of what they call themselves, are basically measuring the same thing, what researchers would describe as “loading on the same factor.” And that factor: global distress. I wondered, “Could it be true that the four-item ORS could generate reliable and valid scores?”

Validating the ORS and SRS: PCOMS Goes Legitimate

There was no funding for this research, so it took people who truly believed in the value of the measures and the clinical process. I reached out to my friends and colleagues who I had helped implement the measures. And David Claud, Chief Operating Officer of the first public behavioral agency (Center for Family Services [CFS]) to implement PCOMS, volunteered to conduct the study at his agency. My former student, Jacqueline Sparks (and now professor at the University of Rhode Island) was also at CFS and played an important part. CFS also contributed significantly to the validation studies of the Session Rating Scale (with the help of another former student, Lisa Rene Reynolds) and the Child Outcome Rating Scale (CORS). The CORS study could not have been completed without Bob Bohanske, Chief of Clinical Services/Training and the support of Southwest Behavioral and Health Services, another early adopter of PCOMS. (By the way, the Child Outcome Rating Scale was the first self-report measure for children 6-12 on the planet. Apparently, we didn’t care much what kids thought about the benefit of their therapy.)

Without my friends, the measures would not have been validated and PCOMS would have never gotten off the ground.

And, of course, the good news from these studies: all the measures performed quite admirably stacked up against much longer, laborious instruments (Read the studies: ORS, SRS, CORS). So, you can use longer measures, but you don’t have to. The brevity of the ORS and SRS really make a difference, because as is news to no clinician on the front lines, few have the time or inclination, to devote to the repeated administration, scoring, and interpretation of lengthy measures. Feasibility is critical. Clients quickly tire of measures that lack obvious face validity, require more than a few minutes to complete, or appear to take away from time spent with the counselor.

The stage was set. Now, with a detailed clinical protocol and validated measures, we could see whether PCOMS actually improved outcomes, whether privileging clients by design made a difference.

In this historical series of blog posts, the empirical support of PCOMS is the next topic and the road to the designation of an evidence-based practice.

But why wait? Start with the fruition of my now 23-year journey by requesting a free trial of Better Outcomes Now today and check in with your clients to improve both your effectiveness and efficiency: