Measuring Advocacy Outcomes; Capturing your service's heartbeat.
I don’t work directly in the field of Advocacy but my work in providing an online database platform for Advocacy case management and outcomes (called ADI) has now brought me into close contact with a number of services and their day to day operations....
My day job is actually as a customer experience researcher. What does that mean? Well amongst other things I spend a lot of time looking at data collected from people by businesses and drawing conclusions from it to improve their services. This is one of the reasons why the outcome reporting Advocacy services collect has been so interesting to me.
Motivations for measuring Advocacy outcomes.
Firstly are the motivations for measuring Advocacy. Talking to the services themselves their reasons for collecting the data are primarily to meet the contractual needs placed upon them by individual commissioners working for local authorities. These people obviously need to evaluate the impact and value of the services they are commissioning. However, I believe the services themselves should be the ones most vocal in promoting a data-led approach, due to the benefits proper data collection and analysis can bring to both the services themselves and the Advocacy partners they support.
A while back the social care institute for excellence posted on their website some of the main reasons why services should measure outcomes (this was in relation to IMHA services).
to measure the effectiveness of what they do
identify effective practices
identify practices that need improvement
demonstrate value
gain clarity of understanding and consensus around the purpose of a service
evaluate how well the organisation is performing
motivate people to deliver advocacy that has impact
These are all valid and worthy reasons to measure, but how do you go from discrete pieces of individual data to these wide and powerful findings?
Luckily these processes have been around a long time in the commercial world. Every large company in the world cares deeply about the experiences their customers have with their brand (to be cynical, this is mainly because it impacts their bottom line). However, this also means there are a number of highly refined tools and methods available that measure people's experiences at points in their customer journey and then use that information to extrapolated practical and insightful ways to improve. The process I’m about to outline is by no means the only way to collect data, however it’s fairly straightforward, useful and a tried and tested method from the commercial world.
Advocacy is a journey.
Becoming the partner of an Advocacy service sets people on a journey. Firstly there is the actual process to being referred and the initial contact with a service, (perhaps over the phone). This contact might be made by a professional, relative or the person directly contacting the service themselves. What is this experience like for the enquirer? Next, how long do people need to wait until they actually get an advocate assigned? When an advocate does make contact and starts the case what is the relationship with that advocate like for the partner? Does the advocate listen? Does the partner feel empowered? Is the advocate contactable? Are they timely in their actions and feedback? Finally how does the relationship end? Are the partner’s outcomes achieved? Does the partner have improved circumstances? All the steps noted offer the chance to possibly take soundings from partners, professionals and relatives, and basically anyone who has a touchpoint with your organisation. Any of those steps in the journey may represent a make or break point that people significantly define as influencing their experience of a service. In the customer experience business these are called “moments of truth".
How do you find your moments of truth?
You find your moments of truth by asking the users of your service. Breakdown your process into the different steps and then create a very short questionnaire. Ask people within your process to rate how important each step was to them, and which experiences with your service have contributed most to defining their opinions of you. Remember at this point you are not trying to find out if these steps offered good or bad experiences just that they were significant for partners in forming their opinions. Later on it will be at these points in the journey you will measure experience.
Taking a heartbeat.
Now you know the most significant steps in the process for your partners, you can consistently measure those experiences to take the pulse of your service. How do you know if your service is improving or deteriorating? If the metrics change for the better over time then your service is improving….if they are getting worse then it’s falling short. Continually taking a regular pulse of the experience your service offers provides a valuable view of change over time. Measuring in your moments of truth provides insights across the whole advocacy journey.
What should you actually measure?
Your commissioners might set specific outcomes they would like to see met so it makes sense to weave these into your regular soundings, however a very simple to administer and reliable metric to use is the Net Promoter Score. NPS is used throughout industry and is a single recommendation question answered on a scale. In our example a good question might be something like “based on your experiences today how likely are you to recommend this advocacy service to family or friends in similar circumstances?” Net promoter is collected on a 10 point scale and uses a simple subtraction of negative ratings (detractors) from positive ratings (promoters) to find a running metric. The format is very well known and a simple google search will show you how to calculate it.
Bringing it all together.
In simple terms the steps are:
Break your service into process steps as perceived by your partners.
Collect insights from your partners to understand which of those steps impact experiences most and as such are your “moments of truth”
Collect a simple measure such as NPS over time in all of your moments of truth.
Regularly review your heartbeat measures to look for changes.
So what would taking heartbeat measures actually achieve? Well, I’ve laid out some real world possibilities below. I think you will agree it seems pretty useful, so get measuring…Companies in the private sector sample and monitor experiences all the time, so why shouldn’t you?
Identify which areas of your journey need improvement by looking at the low points in NPS across all your moments of truth.
If you’ve changed something in your process you can monitor your NPS for three months to see if it’s improved things.
If you lose a member of staff, or add new resource you can monitor your NPS at that moment of truth to understand the impact.
Monitor NPS continually and use unexpected changes to investigate and fix problems in specific areas around key moments of truth you previously had no visibility of.