Engagement scores tell you what people think, not what changed

When the pandemic hit while I was in London, and we all went into lockdown for months on end, it was a dream come true for many people (or at least it was for those who finally got to work from home).

But when things started to settle down and leaders began to question whether we should return to the office, fear started to creep into the workplace.

Nearly every HR team sent a survey looking to answer the same question:

Are people more productive working from home?

It seemed like everyone had some unspoken agreement to answer this survey with a definitive ‘yes’.

No one wanted to jeopardise the one thing we’d all been asking for for years.

But in this instance, the perception of feeling more productive, and actually being more productive were two different things.

This is the trap I see a lot of People teams fall into:

Only using surveys about the perception of an experience as a metric of success.

PREVIOUS NEWSLETTERS

In case you’re a new reader (or just missed them), here are the past few newsletters:

I changed my mind about discovery. What if the best way to understand a people problem isn’t more discovery, but a small experiment?
Is your performance framework too complex? Why borrowing someone else’s performance process rarely works, and the principles worth deciding on before you design your own
What if your promotion was in your hands? How Juro flipped the script on how promotion decisions are made

Engagement scores tell you what people think, not what changed

The problem with only asking people how they feel about their experience is that it is entirely based on their perception which doesn’t necessarily always equate to commercial impact.

For a People Team to be seen as strategic, we also need to be able to measure the impact of our work in a way the business understands and cares about.

That means measuring commercial impact, not just qualitative impact.

If you’ve been running an initiative to increase productivity here’s how that might look:

Perception: 8 out of 10 people said they feel more productive at home
Commercial impact: Delivery times improved by 50% / Customer renewals increased by 20%

Engagement surveys have a place, they measure perception, and they’re useful when you want to know how people feel about a change you’ve made.

But to truly measure the impact of your work, it’s good to get in the habit of looking at how those perceptions correlate with some of the business metrics.

Vanity metrics versus actionable metrics

One of the reasons that our work doesn’t come across as “commercial” when we measure its impact based on the change it made in how people feel, is because perception isn’t an actionable metric.

An actionable metric tells you what to do next AND is tied to business outcomes.

The way to know whether you’re measuring impact using vanity metrics or actionable metrics is to ask yourself:

If this metric dropped by X% this month, would I know how to respond?

If the answer is “no” or “I’m not sure,” you’re measuring impact with a vanity metric.

A vanity metric tells you what happened in a way that makes you feel good.

Not what to change.

The way to make sure you’re measuring the impact of your work strategically, is to ensure you’re able to answer the impact it’s having on the business’ ability to deliver improved outcomes for customers.

So if a survey isn’t the way to measure impact, what is?

Look, I’m not saying you shouldn’t survey your people about their experience. I’m just saying it’s worth pausing to reflect on whether it will give you the data you need.

Start with the outcome the work was meant to change, then ask what change in behaviour you’re trying to drive to reach that outcome.

If it was a productivity initiative, define what productivity looks like with the teams the initiative was for and then measure that. It might be that they’re hitting their deadlines more consistently, reducing the amount of time spent resolving outages, or increasing customer renewals.

You can still capture this data via a survey if that’s going to be the easiest way to get that input, but make sure it’s not just a survey measuring perception.

And if your initiative was prioritised because of a low engagement survey question, then sure, re-run the survey to see whether it improved.

Three questions to ask instead

Next time you’re about to measure whether a piece of People work made a meaningful change, before you reach for a survey, ask:

1️⃣ What business outcome was this meant to improve? If you can’t name one, you’re measuring activity, not impact.
2️⃣ What would we expect to see change if it worked? Name the behaviour or the number, not the feeling.
3️⃣ If that metric moved the wrong way next month, would we know what to do? If not, it’s a vanity metric, and it’s worth finding a better one.

None of this means throwing out how people feel.

Perception is real, and when someone tells you they don’t feel supported, that’s data you act on.

It just isn’t the whole story.

The People teams whose work gets taken seriously are the ones who can show how their work is improving the performance of the business, not just the experience of working in it.

P.S PX Dojo’s next cohort starts on 5th Nov

If the ideas in this newsletter are landing, but Monday morning rolls around, and you’re still not sure how to actually apply them in your own organisation, you might like to check out what our friends are doing over at PX Dojo.

PX Dojo’s 6-week White-Belt PX Foundations Course gives you the methods, language, and a cohort of peers to practise the fundamental mindsets of People Experience.

The program involves 6 live 90-minute workshops, run weekly, to provide you with a safe space to experiment with the methodologies we discuss in this newsletter.

Plus a peer-coaching session to help you assess your org’s readiness for PX and how to translate the learnings to your context.

Secure a 20% discount on standard tickets with Crewmojo’s limited-use partner code.

Use code at checkout: CREWMOJO26