People who have known me for a while would have heard me repeat the above statement several times.
Some might say, “Obviously, what is new in that?”.
Yet, how many of you would find this scenario familiar? When executing a strategy or aligning teams to it, the carefully constructed list of obvious metrics would begin to show all sorts of cracks in the facade and fall apart, sometimes even within the first 3 months. Whoever takes action on the insights from these metrics would be left dealing with excuses on why a certain metric could not be tracked at all, presented with inaccurate data or even cooked up data.
While “what cannot be measured cannot be improved” remains true always, it is equally important to be careful when deciding what to measure. The insights from those measurements are even more important than the data itself.
Most teams (and their managers, however many levels up) will walk away at this point, either thinking “Thank god that is done. Now we can focus on the work, rather than wasting time on these things.” or “I really should have added another 2 metrics to be tracked”.
All this while, how many even give thought to what behaviors (or changes in behaviors) these metrics would induce in their teams as a whole or in individuals?
This phenomenon is not isolated to corporate offices.
Let me give an example that I have observed in my day to day life while using a popular online supermarket/home delivery app.
I started using this app during the Covid19/post-Covid19 period. They were relatively reliable, considering prevailing conditions. I continued using them for the sake of convenience but also because their prices were extremely competitive too. Over a period of time, there were few instances where the app gave me a cashback when the delivery time estimate was not met by the delivery agents. In those cases, and this is my speculation, it seemed like the agents were penalized in some form for delays in delivery. Then, the delivery agents started a new practice. They used to call me on my phone before or right after picking up my order. Some of them request (some of them simply inform) that they be allowed to mark the order as delivered but that they will deliver it within the next 10 to 15 minutes. I used to push-back when I could, arguing with them on why they needed to do that. But this behavior became so normalized that stopped bothering about it.
Over time, the service timelines improved. I moved to a new area and found that the same app can now deliver groceries to my home in 15 minutes since their warehouse was closer to my new house location. Most of the orders got delivered with a reasonable variance from estimate.
Interestingly, a few weeks back, I started seeing random delays with a variance from estimate crossing 300-400%.
When I reached out through their customer support channel, there was hardly any visibility on a reason or a fairly accurate estimate for delayed deliveries. Responses from the first line support (read as AI) were just the usual canned ones, usually stuck in a loop. To be very fair, the non-AI agents were no better, offering little in terms of explaining why my orders were inordinately delayed. It was then I noticed a message in the app that said that their agents are not penalized for delayed deliveries, nor are they incentivized for on time delivery. That gave me more insight in to what had happened.
When there was tangible, personal impact from tracking a particular metric, and that too applied rigidly we can see that people started gaming the system. The target became more important than the intent, which was, presumably, higher customer satisfaction. When the pressure got removed, the situation led to a different set of problems, like lack of accountability, disconnect between reality (like availability of delivery agents) and the algorithm’s estimated for delivery time, fall in ratings from customers on their delivery experience etc.
Therefore, it is imperative to select and use metrics more holistically, setting up a framework that balances several aspects rather than just one measure and combining it with a qualitative analysis, keeping context relevant.
Continuous feedback loops, including those involving other stakeholders, periodic reviews, flexible frameworks that adapt to changing ground realities are key in ensuring that the metrics we select and measure work as intended. That, along with a transformation of work culture, moving the team up on the accountability ladder, open and trust enhancing conversations would make it that much easier to measure progress towards goals.