Doing the Outcome Rating Scale the Right Way

The Outcome Rating Scale (ORS), the measure of the Partners for Change Outcome Management System (PCOMS), is quite different than other outcome instruments. It is an outcome measure, but it is also a valuable clinical tool that requires nuance in its use.

It is not a perfunctory piece of paper to be flicked at clients to tick off a box or satisfy a management edict. It is an evidence-based practice to improve client care. To make proper use of twenty years of psychometric validation and thirteen years of research demonstrating dramatically better outcomes and reduced dropouts, the ORS requires four steps.

Barry Duncan, the developer of the clinical process of PCOMS, identified these four essential components of doing the evidence-based practice, the Partners for Change Outcome Management System (PCOMS) to fidelity. This blog covers the first two steps.

1. Introduce, administer and score (if applicable) the Outcome Rating Scale (ORS) each visit or unit of service. Ensure that the client understands that the ORS is intended to: 1) privilege their voice and bring them into the decision-making process; and 2) will be collaboratively used to monitor progress in each encounter.

The ORS is given at the beginning of each encounter. In the first meeting, the ORS pinpoints where the client sees themselves, allowing for an ongoing comparison in later sessions.

Given that at its heart, PCOMS is a collaborative intervention, everything about the use of the measures and the results attained are shared with clients. Consequently, the client needs to understand what it’s all about, and especially these two points:

• the ORS will be used to collaboratively track outcomes in every session to ensure the client benefits.
• it is a way to make sure that the client’s voice is not only heard but also remains central to services.

Build a culture of feedback in the first meeting and administer the ORS as an invitation into a collaborative partnership. Avoid technical jargon, and instead explain the purpose of the measures and their rationale in a natural, commonsense way.

Make the administration, scoring, and discussion part of a relaxed and ordinary way of having conversations and working. The specific words are not important. Your interest in the client’s desired outcome speaks volumes about your commitment to the individual.

Given that the ORS is intended to make sure the client’s perspective is not lost in the shuffle, the whole point is missed if the client doesn’t get what the measures and PCOMS are about.

2. Discuss the clinical cutoff and contextualize the client’s score. Check with the client to see if the score matches their experience. Explain the expected treatment response, what is hoped will happen if therapy/service/treatment is successful.

Given that everything about PCOMS is 100% transparent, the task now is to discuss the number and make sense of it with the final authority—the client. The “clinical cutoff” provides a way to do this. “Clinical cutoff” is a statistical term that represents nothing ominous, nor does it say anything negative about the client.

It is only the dividing line between people who typically do not find themselves in therapy/service and those who do, differentiating between a so-called “clinical” population from a “non-clinical” one. The cutoff for the ORS is 25 (for adolescents, children, and caretakers, 28).

The ORS is really a measure of distress (or wellbeing), so the number 25 out of 40 generally means that those under 25 are reporting the level of distress typically associated with being a client, and those over 25 are reporting a level of distress generally associated with not being a client.

The client’s score in relation to the clinical cutoff provides a real-time picture of the client’s experience and the first opportunity for feedback. Use the cutoff to set the stage for the work, validate the client, and focus your efforts.

The average intake score of an outpatient setting is from 18 to 20, but anywhere between 20 to 30% of your clients will come in over the clinical cutoff. People who score under the cutoff are typically looking for a change, something different in their life, while those who score higher or over the cutoff tend to be folks more satisfied with the status quo and therefore may require a bit more context to understand what they are looking for from therapy/service.

Once we have the score, it’s time to say what the number means—to contextualize the client’s score using the cutoff as a jumping-off point to promote understanding and best use.

Discussing the cutoff:

• Helps check out whether the score makes sense to the client and fits what they were trying to convey in their marks or touches—to make sure you have a good rating.
• Allows you to validate the client and convey that he or she is in the right place.

What you will find in 95 out of 100 administrations in the first meeting is that the scale clients mark the lowest is the one they are there to talk to you about.

Give the score, say what it might mean using the cutoff as a reference point, and look for feedback to see if it fits. BON makes this easier because you reference the colors (mauve or green).

If it doesn’t fit for the client, then it’s good that you found out so you take another pass and ensure a good rating, one that represents the client’s experience of distress. Explain the ETR, what we hope will happen.

What about folks over the clinical cutoff?

There are two reasons that scores are above the cutoff:

• While most things are going well, there is a specific concern or issue for which help is desired.
• Most clients scoring above the cutoff are folks who someone else has either suggested or required their participation in therapy.

Clients who are mandated (or coerced) to therapy from the courts, their employers, partners, or child protective services, etc. (and nearly all kids are mandated) represent the lion’s share of clients scoring over the cutoff. In these instances, it is very helpful to have clients complete the ORS twice, once as themselves, and once as if they were the referral person.

If possible, it is preferable to get the referral person’s actual rating. BON allows the remote administration of the ORS if needed. This not only helps you track progress from the set of eyes that can make a difference for your client but also helps you identify what specifically the referral source is looking for as a sign of improvement.

Once the client has taken the ORS the second time from the perspective of the referral source:

• Note which score is lower reflecting more distress or problems (the referral score is almost always lower).
• Ask the client to help you make sense of how the referral source’s rating is so different than their view.
• Discuss what needs to happen to bring the referral source’s view more in line with the client’s view.

BON provides a way to have the client reconsider their over the cutoff rating via the redo feature (see the Nuances of the ORS webinar for a demonstration). Ask the client to redo the ORS from the perspective of the person who sent them to therapy/service. But first, go to that client’s ORS/SRS detail page and write down the scores of the four scales they just completed.

After the client fills out the ORS from the referral source’s perspective and you discuss the meaning of the differences and implications for the service, then ask:

“Given the referral person’s perspective, do you want to re-take the ORS or go back to your original scores?”

Whatever the client chooses is okay, but this offers an opportunity for the client to reconsider the ORS score. Mandated clients are no different from voluntary clients regarding the alliance.

Attaining the client’s rating as if they were the referral source is a great way to bring in the other view without challenging the client’s perspective. Almost always, clients will rate the referral source’s rating lower (more distressed) than they rated themselves. In a sense, the ORS allows the referral source’s view to be externalized, represented by the form itself, making it easier to talk about, and not risking the alliance. Few things are worth that risk.

One last thing to note about clients who enter therapy scoring over the cutoff: Even though the client may be reporting that things are going well, there will still be one scale that is lower than the rest, and that is often your invitation to collaborate.

Next Blog:  The most important nuance of the ORS.