For many people, these roles are essential to the success of Agile Transformation, for others, they are mysterious characters who don’t write code, don’t bring any value, and just have some 1on1 meetings where they talk to people and thus supposedly contribute to the overall team outcome.
More than once, I’ve been asked by managers or business owners how SM/AC work can be objectively measured to make sure their work is making a real contribution to the team. It’s true that this person’s work is not as easy to see at first glance as, say, the work of a developer or tester, but the good news is that the success of these roles can definitely be measured.
Let’s take a look at how to do it together! In this article, I’ll present 4+1 metrics for measuring their success.
Planning reliability
It is desirable to deliver at the end of the Sprint whatever we committed to as a team at the beginning of the Sprint. As a first metric, we can then track the percentage of how much work (e.g. story points) the team has planned and how much they have actually delivered. And delivered means really done, according to the Definition of Done, not work in progress or work that “just needs to be deployed”. I personally consider a >90% success rate to be a good number in this metric. However, the goal of the game is not to mindlessly chase that number, but to track the trend and whether you are gradually approaching that success rate. A good indicator of team improvement is also how stable the number is or whether it fluctuates up and down from Sprint to Sprint. And of course, the current status and how to improve it should be a regular topic at the team retrospective.
Mood in the team
It is proven that a satisfied and motivated team achieves better results in the long run. The team can be well motivated by the Product Owner (PO) with a sense of purpose. Satisfaction on the other hand can be influenced by SM/AC by regularly talking to people and removing obstacles (individual or team). I measure the mood of the team after each Sprint on a simple scale of 1 to 5, where 1 means “I’m putting on my shirt and going elsewhere for an interview” and 5 means perfect mood. For a better idea, you can put a smiley face next to each value to express the corresponding mood. Again, I’m not chasing maximum satisfaction every sprint. Just as storms occur in nature from time to time, these fluctuations in people’s satisfaction may not be detrimental and can make the team stronger. The important thing is to talk about the fluctuations and watch the trend. Once a deviation from the trend occurs, it’s always good to be aware of what’s behind it. That way, you catch the negative sentiment in the beginning and stop the spiral of negative energy so that it doesn’t start to spin. I also recommend noticing the trend of individual team member satisfaction and possibly including the discussion as a regular topic on your 1on1s with team members.
PO satisfaction
I’ve seen plenty of teams that had a stable trend of planning reliability, the team had a great vibe, just the PO was unhappy. You’re probably thinking, how is that possible? Sometimes the Scrum Master or Agile Coach defended the team too much from the PO’s pressure, other times the team planned too little work for the Sprint, and the PO wasn’t allowed to say a thing because “commitment is the team’s business”. Naturally, such a situation leads sooner or later to a frustrated PO. And that’s why I measure the PO’s satisfaction with the delivery of the Sprint goals on a regular basis. Once again, on a scale of 1-5, where 1 means absolutely dissatisfied and 5 indicates a very satisfied PO. This rating is made at the Sprint Review and serves as input to the Retrospective where we discuss this together as a team. This includes the PO, which should always be part of a Retrospective. After all, we are all one team. The PO’s satisfaction can also be rapidly impacted by SM/AC by having regular 1on1s and talking about the team, the business, and how they are working together and giving each other feedback (not just the negative but the positive as well).
Customer satisfaction
What we plan with the team, we deliver. We have a happy team and PO. It’s just that our customers complain that while we are a cool company, the new stuff takes ages and we work on unimportant things. Do you know what I mean? Try focusing on the feedback from your customers, called NPS or Net Promoter Score. A simple question like “How likely are you to recommend our product to a friend or acquaintance?” can serve you very well. Customers respond quarterly on a scale of 1-10, where 1 is “absolutely would not recommend” and 10 is “definitely recommend.” It’s always a good idea to also ask why they would (not) recommend us, which provides valuable input for improvement. Using calculation you get a value from -100 to 100. The best products or services achieve a value of 60+, which is considered a very good result. Again, we observe the trend that should be at least non-declining, ideally increasing. Don’t be discouraged if your first measurement is, for example, -40. I’ve seen teams/products get to the 40+ number within 1.5 years of systematic work. It may seem like SM/AC can’t control any of this and it’s all the PO’s job, however, it’s their collective work. SM/AC can for example establish a systematic NPS process, focusing on Lead time for new functionality, or help POs with backlog prioritization techniques and new features. 
Here’s an example of metrics over the last two quarters for one particular team and trend visualizations that can give us a clue as to what’s currently happening in the team and where to focus. They also clearly show the quality and success of the work of the Scrum Master or Agile Coach.
The 4 metrics above can be deployed in any environment and by tracking them, you will gain a comprehensive view of SM/AC performance. Intentionally, the combination includes not only team performance, but also people satisfaction, which is the foundation for long-term sustainability. To create things of value, the last pillar is the customer and their feedback. As in all other situations, there is a “what you measure is what you get” effect with Scrum Masters. That’s why we recommend tracking all 4 together. If we miss any of them, we risk bending the team’s work in a direction we don’t want. Even unknowingly, just out of a desire to meet expectations.
What is often forgotten in practice is the satisfaction of people in these roles. A dissatisfied SM/AC cannot bring the most value to the company and create a High Performing Team. Therefore, as a leader, remember to ask your people in this role about their satisfaction too, and what you could do more to help them work better and feel more supported. This will give you a +1 on your metrics puzzle and give you a complete view of your entire team.
But what to do when your team’s metrics aren’t great? Try these 3 proven tips as SM/AC:
- Check out how you do retrospectives – see our articles on retrospectives (CZ).
- Focus on breaking down large units of work into smaller ones using User Story Mapping technique to deliver results more frequently.
- Do you have 1on1s with team members (including the Product Owner)? If so, ask them what you can do better and what you should continue doing.
If you would like to try out the metrics in your company, you can use our template that we have created for you.
Next time we will take a look at how to measure performance at the organizational level.
