Agile Metrics – 5 Tips to Achieve Product Success

As an Agile Coach, one of the most common questions that new Agile teams ask me is: “What metrics do we need to track?” So, I’ll share what I tell them.

But, before we dive into the details about the metrics themselves, I think it’s worthwhile to consider the reasons that most teams are expected to track and report metrics. 

Why are we obsessed with metrics?

Here are a few of my observations from my experience working on various types of projects:

1. Senior management / leadership is accustomed to seeing metrics. In the traditional project management world, most organizations that follow the waterfall/plan-driven approach seem to gravitate towards the standard RAG (Red/Amber/Green) project health status. I’m not sure where this system came about, but it may be tied to tools such as Microsoft Project. 
2. Team leaders (or managers) generally are too busy to be directly engaged in the details of an Agile team, and need data to help them understand the current state of the project so that they can communicate the status upwards to their leaders and project sponsors.
3. The members of the Agile team need data to help them manage their work so that they can determine if they are on-track or need to ask for help.

While all of these are valid and understandable reasons to implement metrics, there are many dangerous pitfalls that teams can encounter when attempting to do so. 

Strategies for deploying metrics

Here are a few suggestions that may help new Agile teams deploy metrics effectively.

Start simple

Don’t try to implement 20 metrics immediately, even if they sound simple. From my experience, there is usually a higher-than-expected cost associated with each and every metric your team chooses to deploy, and they can slowly strip away valuable time from your team and their ability to focus on doing “real work”. 

My recommendation: start with no more than three metrics that can be tracked using the Agile management tool you are already using. Most industry-standard tools such as Azure DevOps or Atlassian Jira offer reports and some type of data analytics within the tool itself, so the start-up cost should be relatively low.

Don’t share it without explaining it 

Once your team starts tracking metrics, there will be a temptation to publish the data broadly to everyone and anyone within your organization. As a result, some reports may be shared unintentionally or without forethought. This may be a risky time for your team especially if your organization is not accustomed to the metrics your team is using (i.e. velocity, throughput, etc.), which means they could be misinterpreted by someone important but not close enough to the work to make sense of this data. 

My advice: prepare your team and help them to explain the data to whomever may have access to it. Learn to craft a message around the data to provide context, and help the target audience understand the “why” behind the data.

Connect to business objectives and value 

As a continuation of the previous point, explaining the “why” behind the data will be even more powerful if your team can connect the data to the business goals of your organization. 

For example, if your metric shows that your team is completing 90% of planned work on average for each sprint, your team is performing at a very high level and is helping the overall organization deliver consistent value to the customers and stakeholders. Be sure to make that connection to the business outcome, because the number in itself (90%) is meaningless without proper context.

Review and adjust regularly 

As your team matures, you will likely want to modify your metrics strategy, which may seem like a lot of work. Encourage the team to think about how they are doing and explore ways to adapt. The organizational objectives will likely change throughout the year, so it is vital for the team to stay in tune with those changes and evolve accordingly. For example, if quality is becoming an area of focus, it may make sense to start tracking defect rate.

Automate where possible 

As the team continues to apply more metrics, it will make sense to deploy automation where possible to reduce the ongoing overhead of managing the metrics and the data. 

Some possible solutions to consider include plug-ins for your Agile management tool or creating some simple scripts to automate the data collection process.

Final thoughts

To wrap up, if your team has not spent much time thinking about metrics yet, it may be a good time to start now before your leaders impose something that may not make sense for your product. If you take the initiative and begin tracking and publishing meaningful data associated with your team, you will help your team share successes and challenges and begin building trust with the sponsors and stakeholders. In time, this will help build a collaborative environment that leads to successful outcomes for your project.

Eugene Lai, Contributor