Author: Elizabeth Walsh

The management expert and author Peter Drucker famously said, “You cannot improve what you don’t measure.” Eliyahu Goldratt, author of “The Goal”, said, “Tell me how you will measure me, and I will tell you how I will behave.” Both quotes highlight the importance of metrics in everyday business operations.

Why do metrics matter?

If your team has a desire to improve, you have to first find out where the team stands currently. This simple rule applies to many other domains; from business strategy to personal goals, it is nearly impossible to change our current state unless we know what that is. 

If we wish to lose weight and live a healthier life, we need to know how we are doing against the standards; is our cholesterol or blood pressure too high or too low? The only way to know is to measure it. 

Another critical step towards improving your current state, whatever that might be, is to establish a desired future state. We must define what “good” looks like so that we can work towards it, and be able to assess if we are making progress as well as when we reach that goal.

As Goldratt noted, external forces and perceptions heavily influence us. Whether it’s our management, our partners, or our peers, we are ‌sensitive to how they perceive us; how they evaluate our performance has a tremendous impact on what we do. 

Impact of metrics for your team

So, how do these thoughts relate to Agile metrics for your team? 

We need to be careful about how we measure our teams and their performance, because using the incorrect method of measurement can lead to negative consequences. If we are interested in improving team performance, we must first define what the desired end-state looks like. 

What does “good” look like?

Is achieving a 25% increase in story points what we want? Or, is it a 10% decrease in rework? Alternatively, do we want the team to reduce technical debt by 25% before Release 2.0?

Defining specific measures is a critical step that many teams do poorly. If we want more output, perhaps measuring an increase in story points is the right approach. However, we must consider the possibility that more points completed does not actually translate directly to higher value delivered to the customer. Similarly, a decrease in technical debt, usually a positive, may or may not mean that the team has found a predictable, consistent approach to ensure that technical debt does not grow again in the future.

This may feel somewhat backwards, but to ensure you choose the proper metrics, I recommend you first consider the desired behavior, then select a metric associated with that behavior. 

What metrics make sense for you?

If you want the team to improve their ability to plan and forecast work more consistently, one metric you can track is percentage of work completed versus planned. 

This metric is easy for the team to calculate and monitor without doing much extra work; most Agile management tools will provide this by default. Percent complete provides direct insight into the team’s ability to plan and execute work, and also adapt to new learnings during a sprint or program increment. Define success as achieving a stable and repeatable completion rate.

Another example: If you wish to improve the quality of the work produced by your team, defect rate would provide great insight into how the team is performing. There are other related metrics that the team may use to achieve this, such as unit test coverage, which is also relatively easy to establish and monitor.

In summary, applying the right metrics for your team is a tricky challenge that requires some experimentation. Often, it makes sense to encourage the team to come up with their own measures to monitor. If you can help them understand the desired behavior and outcomes, and provide the space for them to define their own metrics, your team will take a big step in the right direction towards becoming a high-performing team.

If you are leading or working on an Agile team, you will most likely need to communicate progress and status to your leadership at some point. Many of the teams I have coached fall into common traps of either sharing erroneous metrics or blindly distributing data without providing context and explanation, which often lead to disastrous consequences. When metrics are misinterpreted and/or misused, it can cause a lot of damage to your team because it can generate a lot of questions, create a lot of anxiety for the leadership team, which ultimately translate into more stress for your team that may feel like they are being unfairly scrutinized or criticized. This outcome is bad for morale and will likely impair your team’s performance. Hence, it is my hope that this article will provide some tips and tricks that will enable you to communicate more effectively with your leaders and help you shape them into supporters rather than critics by deploying a sound approach regarding metrics management.

Why does your management care about metrics?

Before we can choose the appropriate metrics to share with your leaders, we must first understand the reasons behind this desire for data. There may be a number of reasons that motivate the management to ask for metrics, and it is important for you to be prepare to explain the data when you present them so that their needs are met. Here are a few questions that may be beneficial to consider:

1.     Are the managers accustomed to seeing traditional health status reports (i.e. Red/Yellow/Green status)? 
2.     Do the leaders need data to feel comfortable with your team’s performance?
3.     Is there a lack of trust by the management due to previous history?
4.     Will the data provide an accurate representation of how the team is performing? Is additional context necessary to tell the whole story?

If you are able to answer these questions to yourself, you should be in a good position to start preparing the metrics for broader consumption. While this may feel like a lot of extra work that you did not anticipate, having a sound communication strategy is a key technique that many Agile teams often forget to focus on, and are forced to learn the lessons the hard way.

Which metrics matter most?

Assuming that transparency is appreciated and open communication is the norm in your organization, routinely reporting metrics to your leadership team will usually facilitate a fact-based approach to project delivery, which can be a tremendous asset for your project team. Before we dive into the specific metrics I recommend, let’s first clarify the objective of the metrics.

1.     Communicate progress to maximize understanding
2.     Share successes to build confidence
3.     Clarify value being delivered

Ultimately, what truly matters to most organizations is that value is being generated, and that customer expectations are being met. The metrics you share should the directly connected to these themes so that your leaders will understand the overall impact your team is making. Here are a few basic metrics that you may want to consider as starting points.

1.     % stories completed vs. planned – Notice that this is a percentage metric, not pure story points (or velocity). This is a nuanced yet important distinction that may teams miss; reporting pure velocity (i.e. 100 story points completed) is not as valuable as percentage completed versus planned because discrete velocity offers no context. What does “100 points” mean? Is that good or bad? There is no basis for comparison, which can often lead to managers comparing Agile teams against each other based purely on story points completed; this is one of the worst uses for this metric that will usually lead to meaningless competition amongst teams and story point inflation. By looking at percent complete versus planned, we can see how well the team did against the original plan, and also evaluate the trends over time to see if a team is improving their ability to plan/forecast work.
2.     % stories accepted – Acceptance of work is important to track because it demonstrates value being delivered; this assumes of course that your Product Owner is providing an accurate representation for the end-users/customers. If your team can consistently achieve 90+% acceptance, that demonstrates that you have a high-performing team.
3.     Defect rate – Quality is an important measure that is often forgotten by organizations that are usually more obsessed with schedule delays and cost overruns. Even if your team is not producing software products, you can still measure quality and defects and evaluate the trend over time to communicate the team’s performance.
4.     User satisfaction – Although it is often challenging to obtain direct feedback from the customers on a regular basis, this is a critical metric that is worth tracking because customer sentiments will have significant impact over the longevity of your team and your organization. Metrics such as NPS (Net Promoter Score) can provide a simple yet effective way to determine how your customers perceive your product or service.
5.     Team happiness – Another measure that is often forgotten is team morale. How happy is your team? Are they engaged and motivated to do their best work? Do they have the support they need to create innovative products? If you aren’t paying attention to your team’s health, there’s a chance that they may not be as inspired as you may think, which may be a risk to your project.

How to handle misuse of metrics

Sooner or later, someone important or in a high-level position within your company will likely misinterpret the data that your team carefully communicated, despite your best efforts. How should you deal with this situation? The best approach in my opinion is to educate, educate, and educate some more. You may need to set up individual 1-on-1 meetings with this key sponsor or stakeholder to help him/her gain the proper understanding of the data. Try to resist the temptation of comparing your team against other teams. If you absolutely must do so, be sure to focus on business objectives and relative percentages instead of absolute values. Remember, your management is simply trying to understand how things are going, and they need time to learn and adapt as well.

Wrapping Up

I have a feeling that you probably didn’t think deploying metrics requires this much work. Developing an effective metrics plan is no small feat, but the risk of not doing so will likely be too great for you to ignore. When it comes to data, we are all easily influenced by what we see, which means we need to take care to focus on the right measures that will help us communicate the right story in the right moments; this can make a huge difference when it comes to project sponsorship and overall project success.

According to the latest release of SAFe®, “SAFe Scrum is an Agile method used by teams within an ART (Agile Release Train) to deliver customer value in a short time box. SAFe Scrum teams use iterations, Kanban systems, and Scrum events to plan, execute, demonstrate, and retrospect their work.”

SAFe 6.0 provides a few minor modifications to the construct of what was formerly “ScrumXP”, a merger of the Scrum framework and XP (Extreme Programming). I have highlighted a few key differences below to call out specific changes. Let’s explore these more closely and see if they truly matter to you and your teams.

Changes to SAFe Scrum

What are the key differences?

Terminology aside, the conceptual change from “ScrumXP” to “SAFe Scrum” is not materially significant. But, there are a few differences to be aware of.

Removing “XP”

SAFe indicates the XP practices (such as pair work, continuous integration, etc.) are now integrated into other areas of the framework, which is why they removed the “XP” moniker. I believe that this simplification makes sense and will probably reduce confusion, but not much value beyond that.

How big should a team be?

The next change is related to the recommended team size. SAFe 6.0 is now aligned with the most current version of the Scrum Guide (2020) in terms of ideal team size (no more than 10). While “5 to 11” from SAFe 5.X does not appear to be much different, this change increases the flexibility somewhat by eliminating the lower limit of five, implying you could have a team with fewer than five people if you choose.

Daily Stand-up is not daily!?

Next, we see a change regarding the frequency of the daily team meeting, formerly known as the Daily Stand-up. Instead of mandating a daily occurrence, 6.0 provides a more general recommendation of “about daily”, which means the teams may choose to NOT meet daily. While this may provide more freedom for teams to decide how often they wish to revisit their plan towards sprint/iteration goals, I am concerned that some teams will use this wording to justify a weekly meeting, which would follow a more traditional status meeting mentality. I believe teams should remain vigilant and have the discipline to meet daily until they have reached a state of maturity and understanding of how to collaborate effectively within the Agile/SAFe paradigm.

Master or Coach?

The next change is related to the title of the Scrum Master role. By adding the “Team Coach” moniker, they place more emphasis on the Scrum Master’s focus area, which is to provide coaching and guidance to teams rather than serving as an administrator of tools. I believe this is a good enhancement that highlights the key duties of the Scrum Master and helps to clarify the value of this role.

Clarifications of differences between SAFe and Scrum

The remaining items highlighted in the table are not specifically tied to the SAFe 6.0 update, but a clarification of the difference between SAFe and “classic” Scrum (e.g. the Scrum Guide). As you see, the time boxes for some events are different between SAFe and the Scrum Guide, which may not be significant, but they’re noteworthy. If you have a large train with several teams, these may make a difference in terms of the overall investment made in the various events.

Which changes should you care about?

Depending on where your organization is in your SAFe journey, some of these modifications may not be meaningful. If you have existing trains that have been operating for several months or years, making these adjustments may require more effort and time than you expect.

If you are just starting your journey and preparing to launch your first train, adopting SAFe 6.0 terminology and practices may be simpler for you because you will probably avoid the barriers of preconceived notions or mindsets that have been established already.

Closing thoughts

The important thing to keep in mind is that SAFe is a framework, and it should be customized to fit your context. The risk here is not knowing what to customize and unknowingly taking away practices or constructs that reduce your overall agility and/or quality.

To avoid this, it is usually a good idea to engage experienced SAFe practitioners and consultants to assist you with this process. SAFe can be a highly effective approach to building great products and solutions, but there will always be a learning curve with any framework, and seeking help can accelerate your adoption significantly.

SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile, Inc.