Category: Agile & DevOps

The management expert and author Peter Drucker famously said, “You cannot improve what you don’t measure.” Eliyahu Goldratt, author of “The Goal”, said, “Tell me how you will measure me, and I will tell you how I will behave.” Both quotes highlight the importance of metrics in everyday business operations.

Why do metrics matter?

If your team has a desire to improve, you have to first find out where the team stands currently. This simple rule applies to many other domains; from business strategy to personal goals, it is nearly impossible to change our current state unless we know what that is. 

If we wish to lose weight and live a healthier life, we need to know how we are doing against the standards; is our cholesterol or blood pressure too high or too low? The only way to know is to measure it. 

Another critical step towards improving your current state, whatever that might be, is to establish a desired future state. We must define what “good” looks like so that we can work towards it, and be able to assess if we are making progress as well as when we reach that goal.

As Goldratt noted, external forces and perceptions heavily influence us. Whether it’s our management, our partners, or our peers, we are ‌sensitive to how they perceive us; how they evaluate our performance has a tremendous impact on what we do. 

Impact of metrics for your team

So, how do these thoughts relate to Agile metrics for your team? 

We need to be careful about how we measure our teams and their performance, because using the incorrect method of measurement can lead to negative consequences. If we are interested in improving team performance, we must first define what the desired end-state looks like. 

What does “good” look like?

Is achieving a 25% increase in story points what we want? Or, is it a 10% decrease in rework? Alternatively, do we want the team to reduce technical debt by 25% before Release 2.0?

Defining specific measures is a critical step that many teams do poorly. If we want more output, perhaps measuring an increase in story points is the right approach. However, we must consider the possibility that more points completed does not actually translate directly to higher value delivered to the customer. Similarly, a decrease in technical debt, usually a positive, may or may not mean that the team has found a predictable, consistent approach to ensure that technical debt does not grow again in the future.

This may feel somewhat backwards, but to ensure you choose the proper metrics, I recommend you first consider the desired behavior, then select a metric associated with that behavior. 

What metrics make sense for you?

If you want the team to improve their ability to plan and forecast work more consistently, one metric you can track is percentage of work completed versus planned. 

This metric is easy for the team to calculate and monitor without doing much extra work; most Agile management tools will provide this by default. Percent complete provides direct insight into the team’s ability to plan and execute work, and also adapt to new learnings during a sprint or program increment. Define success as achieving a stable and repeatable completion rate.

Another example: If you wish to improve the quality of the work produced by your team, defect rate would provide great insight into how the team is performing. There are other related metrics that the team may use to achieve this, such as unit test coverage, which is also relatively easy to establish and monitor.

In summary, applying the right metrics for your team is a tricky challenge that requires some experimentation. Often, it makes sense to encourage the team to come up with their own measures to monitor. If you can help them understand the desired behavior and outcomes, and provide the space for them to define their own metrics, your team will take a big step in the right direction towards becoming a high-performing team.

If you are leading or working on an Agile team, you will most likely need to communicate progress and status to your leadership at some point. Many of the teams I have coached fall into common traps of either sharing erroneous metrics or blindly distributing data without providing context and explanation, which often lead to disastrous consequences. When metrics are misinterpreted and/or misused, it can cause a lot of damage to your team because it can generate a lot of questions, create a lot of anxiety for the leadership team, which ultimately translate into more stress for your team that may feel like they are being unfairly scrutinized or criticized. This outcome is bad for morale and will likely impair your team’s performance. Hence, it is my hope that this article will provide some tips and tricks that will enable you to communicate more effectively with your leaders and help you shape them into supporters rather than critics by deploying a sound approach regarding metrics management.

Why does your management care about metrics?

Before we can choose the appropriate metrics to share with your leaders, we must first understand the reasons behind this desire for data. There may be a number of reasons that motivate the management to ask for metrics, and it is important for you to be prepare to explain the data when you present them so that their needs are met. Here are a few questions that may be beneficial to consider:

1.     Are the managers accustomed to seeing traditional health status reports (i.e. Red/Yellow/Green status)? 
2.     Do the leaders need data to feel comfortable with your team’s performance?
3.     Is there a lack of trust by the management due to previous history?
4.     Will the data provide an accurate representation of how the team is performing? Is additional context necessary to tell the whole story?

If you are able to answer these questions to yourself, you should be in a good position to start preparing the metrics for broader consumption. While this may feel like a lot of extra work that you did not anticipate, having a sound communication strategy is a key technique that many Agile teams often forget to focus on, and are forced to learn the lessons the hard way.

Which metrics matter most?

Assuming that transparency is appreciated and open communication is the norm in your organization, routinely reporting metrics to your leadership team will usually facilitate a fact-based approach to project delivery, which can be a tremendous asset for your project team. Before we dive into the specific metrics I recommend, let’s first clarify the objective of the metrics.

1.     Communicate progress to maximize understanding
2.     Share successes to build confidence
3.     Clarify value being delivered

Ultimately, what truly matters to most organizations is that value is being generated, and that customer expectations are being met. The metrics you share should the directly connected to these themes so that your leaders will understand the overall impact your team is making. Here are a few basic metrics that you may want to consider as starting points.

1.     % stories completed vs. planned – Notice that this is a percentage metric, not pure story points (or velocity). This is a nuanced yet important distinction that may teams miss; reporting pure velocity (i.e. 100 story points completed) is not as valuable as percentage completed versus planned because discrete velocity offers no context. What does “100 points” mean? Is that good or bad? There is no basis for comparison, which can often lead to managers comparing Agile teams against each other based purely on story points completed; this is one of the worst uses for this metric that will usually lead to meaningless competition amongst teams and story point inflation. By looking at percent complete versus planned, we can see how well the team did against the original plan, and also evaluate the trends over time to see if a team is improving their ability to plan/forecast work.
2.     % stories accepted – Acceptance of work is important to track because it demonstrates value being delivered; this assumes of course that your Product Owner is providing an accurate representation for the end-users/customers. If your team can consistently achieve 90+% acceptance, that demonstrates that you have a high-performing team.
3.     Defect rate – Quality is an important measure that is often forgotten by organizations that are usually more obsessed with schedule delays and cost overruns. Even if your team is not producing software products, you can still measure quality and defects and evaluate the trend over time to communicate the team’s performance.
4.     User satisfaction – Although it is often challenging to obtain direct feedback from the customers on a regular basis, this is a critical metric that is worth tracking because customer sentiments will have significant impact over the longevity of your team and your organization. Metrics such as NPS (Net Promoter Score) can provide a simple yet effective way to determine how your customers perceive your product or service.
5.     Team happiness – Another measure that is often forgotten is team morale. How happy is your team? Are they engaged and motivated to do their best work? Do they have the support they need to create innovative products? If you aren’t paying attention to your team’s health, there’s a chance that they may not be as inspired as you may think, which may be a risk to your project.

How to handle misuse of metrics

Sooner or later, someone important or in a high-level position within your company will likely misinterpret the data that your team carefully communicated, despite your best efforts. How should you deal with this situation? The best approach in my opinion is to educate, educate, and educate some more. You may need to set up individual 1-on-1 meetings with this key sponsor or stakeholder to help him/her gain the proper understanding of the data. Try to resist the temptation of comparing your team against other teams. If you absolutely must do so, be sure to focus on business objectives and relative percentages instead of absolute values. Remember, your management is simply trying to understand how things are going, and they need time to learn and adapt as well.

Wrapping Up

I have a feeling that you probably didn’t think deploying metrics requires this much work. Developing an effective metrics plan is no small feat, but the risk of not doing so will likely be too great for you to ignore. When it comes to data, we are all easily influenced by what we see, which means we need to take care to focus on the right measures that will help us communicate the right story in the right moments; this can make a huge difference when it comes to project sponsorship and overall project success.

According to the latest release of SAFe®, “SAFe Scrum is an Agile method used by teams within an ART (Agile Release Train) to deliver customer value in a short time box. SAFe Scrum teams use iterations, Kanban systems, and Scrum events to plan, execute, demonstrate, and retrospect their work.”

SAFe 6.0 provides a few minor modifications to the construct of what was formerly “ScrumXP”, a merger of the Scrum framework and XP (Extreme Programming). I have highlighted a few key differences below to call out specific changes. Let’s explore these more closely and see if they truly matter to you and your teams.

Changes to SAFe Scrum

What are the key differences?

Terminology aside, the conceptual change from “ScrumXP” to “SAFe Scrum” is not materially significant. But, there are a few differences to be aware of.

Removing “XP”

SAFe indicates the XP practices (such as pair work, continuous integration, etc.) are now integrated into other areas of the framework, which is why they removed the “XP” moniker. I believe that this simplification makes sense and will probably reduce confusion, but not much value beyond that.

How big should a team be?

The next change is related to the recommended team size. SAFe 6.0 is now aligned with the most current version of the Scrum Guide (2020) in terms of ideal team size (no more than 10). While “5 to 11” from SAFe 5.X does not appear to be much different, this change increases the flexibility somewhat by eliminating the lower limit of five, implying you could have a team with fewer than five people if you choose.

Daily Stand-up is not daily!?

Next, we see a change regarding the frequency of the daily team meeting, formerly known as the Daily Stand-up. Instead of mandating a daily occurrence, 6.0 provides a more general recommendation of “about daily”, which means the teams may choose to NOT meet daily. While this may provide more freedom for teams to decide how often they wish to revisit their plan towards sprint/iteration goals, I am concerned that some teams will use this wording to justify a weekly meeting, which would follow a more traditional status meeting mentality. I believe teams should remain vigilant and have the discipline to meet daily until they have reached a state of maturity and understanding of how to collaborate effectively within the Agile/SAFe paradigm.

Master or Coach?

The next change is related to the title of the Scrum Master role. By adding the “Team Coach” moniker, they place more emphasis on the Scrum Master’s focus area, which is to provide coaching and guidance to teams rather than serving as an administrator of tools. I believe this is a good enhancement that highlights the key duties of the Scrum Master and helps to clarify the value of this role.

Clarifications of differences between SAFe and Scrum

The remaining items highlighted in the table are not specifically tied to the SAFe 6.0 update, but a clarification of the difference between SAFe and “classic” Scrum (e.g. the Scrum Guide). As you see, the time boxes for some events are different between SAFe and the Scrum Guide, which may not be significant, but they’re noteworthy. If you have a large train with several teams, these may make a difference in terms of the overall investment made in the various events.

Which changes should you care about?

Depending on where your organization is in your SAFe journey, some of these modifications may not be meaningful. If you have existing trains that have been operating for several months or years, making these adjustments may require more effort and time than you expect.

If you are just starting your journey and preparing to launch your first train, adopting SAFe 6.0 terminology and practices may be simpler for you because you will probably avoid the barriers of preconceived notions or mindsets that have been established already.

Closing thoughts

The important thing to keep in mind is that SAFe is a framework, and it should be customized to fit your context. The risk here is not knowing what to customize and unknowingly taking away practices or constructs that reduce your overall agility and/or quality.

To avoid this, it is usually a good idea to engage experienced SAFe practitioners and consultants to assist you with this process. SAFe can be a highly effective approach to building great products and solutions, but there will always be a learning curve with any framework, and seeking help can accelerate your adoption significantly.

SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile, Inc.

Upon its release early in 2023, SAFe® 6.0 brought about several changes to the framework, one of which may be the most important and impactful to you if you are a current SPC (SAFe Program Consultant).

The role of the SPC is key to any SAFe adoption because SPCs provide the foundation on which we build the change initiative. Because we need SPCs to provide the training, coaching, and mentoring that most organizations lack, having a keen understanding of the new SPC (SAFe Practice Consultant) is essential.

So what has changed other than the title? Let’s take a closer look.

What’s changed about the SPC role?

While the core responsibilities of the SPC remain largely the same, many subtle changes add up to a meaningful revision that is worth noting and studying, especially if you already have SPCs supporting your Agile Release Train (ART). While some changes may seem minor, others may alter how you perceive and leverage your SPC skill set.

In the following table, I have summarized the primary responsibilities of the SPC for both version 5.X and 6.0. You may notice that several items have either been merged or enhanced. Next, we’ll dive deeper into the importance of these changes.

Key Differences

Why were the changes made?

No, Scaled Agile Inc. isn’t just trying to keep you on your toes, or necessitate new business cards. These changes—some subtle, some not so much—were made for a few important reasons.

Emphasis on business agility, not just engineering or product development

Previous versions of SAFe emphasized Lean-Agile systems development, which focused primarily on defining, building, testing, and deploying of technology-centric products and services.

While this theme has been important, and will continue to be valuable for most business organizations, SAFe 6.0 recognizes the need to expand beyond IT functions because large organizations are very complex, encompassing much more than just IT. By placing focus on the overall business as a system, they have widened the realm of possibility for SAFe to other functions, such as Finance, Human Resources, Business Development/Marketing, and more.

This also redefines the Value Stream to expand beyond the boundaries of IT. For organizations that may have begun their SAFe journey within the technology world, this enhancement enables further application of SAFe to other parts of the company.

Focus on values and behavior; modeling over telling/teaching

Scaled Agile Inc. modeled the SAFe Program Consultant after an advisor, trainer, consultant role—someone who provides guidance and advice, and primarily tells an organization what they are doing right or wrong as they attempt to induce change where needed.

The new SPC places more emphasis on a different approach, which is akin to professional coaching. With new terminology such as “coaching” and “embodying”, SAFe 6.0 emphasizes a softer, more people-centric approach to the “consulting” responsibility. This is exemplified in the new prioritization of the Continuous Learning Culture within SAFe 6.0.

Which changes should you care about?

If you are still reading, chances are you’re an SPC yourself, or you already have SPCs within your organization who are doing great work. Also, there’s a high probability that your SPCs are trying to understand how this change will affect them.

There are several ways to explore the new SPC responsibilities and determine if/when you may wish to adopt them.

One method is to establish a community of practice for SPCs and encourage the members to exchange their ideas on what this change means to them and your organization. By allowing the SPCs to develop their own vision for the future state, they are much more likely to take ownership of this change and follow through on the execution.

To complement this approach, it may also be helpful to build a high-level change roadmap to help structure the application of the new responsibilities defined for the Practice Consultant.

Closing thoughts

The SPC role is the cornerstone to any SAFe-enabled initiative, so taking the time to make sense of these new and changed responsibilities is an important investment, especially if you have already made a commitment to apply SAFe within your company.

As with all changes, adopting the new role does not have to be a onetime project; if you can incrementally apply small changes that make sense for your SPC community, you may realize the benefits faster than expected.

Dive deeper into the new SPC role and how your organization can best adopt the changes to the role by exploring our learning course, Implementing SAFe – Certified SAFe Practice Consultant.

SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile, Inc.

When does a relatively obscure coding language become important to the C-suite of an international, multi-billion dollar enterprise?

When we’re talking about the only means you have to translate your existing custom scripted workflows from Jira on Server or Data Center to Jira Cloud.

For users in Jira Server or Data Center environments, the most commonly used applications to handle these kinds of custom scripts are:

• Scriptrunner
• Jira Miscellaneous Workflow Extensions
• PowerScripts

The trouble is, when users of these popular scripting platforms migrate to Atlassian Cloud, their scripted workflows break.

Why?

Because the scripting format in Atlassian Cloud is completely different. On Cloud, you must use Jira Expressions.

What is Jira Expressions?

Jira Expressions is a domain-specific language designed for Jira Cloud that allows developers and Jira admins to set up workflow transition customizations using conditions and validators.

In other words, every piece of custom automation a Jira Cloud user wants in place to streamline and optimize the flow of data, approvals, and workflow progress, gets created using Jira Expressions.

The basic function of Jira Expressions is to read Jira information very quickly so Admins can customize conditions and validators. For example, let’s say a particular type of Jira issue needs to be approved by a member of the HR team before it can move from “In Progress” to “Done” in its workflow. The code telling Jira Cloud to check this when the issue is transitioned must be written in Jira Expressions.

Why does it matter?

This seems straightforward, and really, it is.

The trouble we’ve run into, though, is that most organizations who are migrating to the Cloud don’t see the need to act on this early enough in the migration process. The depth and complexity this introduces into the migration took us by surprise early on as well. But now, after hundreds of successful migrations, we see the problem for what it is.

Here’s why you need to consider dealing with Jira Expressions long before your migration occurs:

• It’s not an easy language. Jira Expressions is JavaScript-based, but it’s not pure JavaScript, or any other commonly-used language. Translating your current scripts into Jira Expressions is not straightforward. Your IT team is going to have to take the time to learn how to use it. And it’s not easy.
• You probably have more custom workflows than you realize. Large organizations who have been using Jira Server or Data Center for a long time may have hundreds, even thousands, of custom workflow transition scripts running in their current instance. Few even have them documented. So, going through to prioritize and update them all will be a huge job.
• Atlassian’s documentation is limited. For such a relatively obscure and difficult scripting language, the available documentation is thin. And, thus far, there are no in-depth training programs available.

What happens if you don’t factor in Jira Expressions?

Unfortunately, we’ve seen this happen a number of times. While concentrating on a host of other migration-related concerns, the idea of making tiny tweaks to custom workflows feels unimportant.

But it’s not.

Here’s what could happen if you fail to factor Jira Expressions into your plans before the migration occurs:

• The cost of the migration could go up. A large-scale Cloud migration is a significant investment. Whether you’re doing it yourself or working with a migration partner, you want to plan and execute it as efficiently as possible. Dealing with Jira Expressions after the fact will add time and cost to the equation.
• It could be detrimental to your business. The saying goes, “you don’t know what you have until it’s gone.” Post-migration, your teams’ productivity depends on Jira working seamlessly. If some or all of your standard Jira functionality grinds to a halt because of broken scripts, it may take a long time to get back to pre-migration levels.

Cprime is an Atlassian Platinum Solutions Partner with a certified Cloud specialization. We’ve performed hundreds of Atlassian migrations, seen what does and doesn’t work, and developed a tried and true framework for moving you seamlessly from Server or Data Center to Atlassian Cloud.

That includes evaluating, streamlining, and executing your move from your current custom workflow scripting language to Jira Expressions. Contact a specialist today to make sure your migration goes smoothly.

Throughout the past few years, the Velocity metric seems to have become a polarizing element among Agile teams due to how this data point is being used. Several years ago, the Scrum Guide mentioned Velocity as a recommended metric for Scrum teams. Because of the way this information has been used and misused, it has since been removed as a recommended practice. Similar to Sprint burndown charts and the use of Story Points, metrics from Scrum teams seems to have become de-standardized, as teams are now free to choose how they track and measure their work.

Now that teams have more freedom, you may wonder whether this is good or bad for the teams. While it is positive that the team has more perceived autonomy, they also are at greater risk of being directed to track and report specific metrics to management because of the absence of an industry standard rule on Agile metrics.

Why Do Teams Measure Velocity in Scrum?

Within the Scrum/Agile world, there is no single equivalent of the PMBOK (Project Management Book Of Knowledge) which provides a formal, official standard organizations and teams can point to as “the guide”. Hence, Scrum teams are potentially under a higher level of influence by senior leadership that expects to see a specific measure of project progress, and Velocity is often the metric of choice.

So is the use of Velocity in Scrum necessarily a bad practice? Let’s look at both sides of the argument and see what makes sense to you.

Pros of the Velocity Metric

Velocity is typically a measurement of productivity by an Agile/Scrum team that usually estimates work using Story Points. The measure Velocity can follow units of Story Points or a simple count of total Product Backlog Items (PBIs). Teams usually track this data point on a per-sprint basis. For example, a team may have planned 12 PBIs (with an aggregate of 70 points) and completed 10 of them (totaling 50 points) within a single sprint. The Velocity is “50”.

The data is typically used to measure team performance trends over a period of time; an average Velocity can assist a team to forecast future output and allow the team to plan what business value they can deliver over the next several sprints.

All this may sound fairly benign so far, correct? When used properly, Velocity trends can be a great tool for a self-organizing, self-managing team. The problem arises when this data point is used in unintended ways.

Why Might Teams Want to Consider NOT Using the Velocity Metric?

In my experience coaching Agile teams, I have observed several practices that lead to undesirable behavior by the team, producing degraded morale and performance. Here are a few examples of such practices.

Cons of Measuring Velocity in Scrum

Comparison of Velocity between teams

Because the Story Point measure is a relative scale that is unique to each team, comparing two teams based purely on the total number of points they deliver is basically comparing apples to oranges; it makes no sense and provides no value. No two teams are identical in terms of their skills, experience, knowledge of the problem domain, interpersonal dynamics, or how they approach their work. So it makes no sense to measure their performance using a single, relative metric like Velocity. Doing so can damage the morale of the teams and motivate negative behavior such as Story Point inflation.

Requiring a Team to “Complete More Points Each Sprint”

One common aftermath of comparing team productivity using the Story Point measure alone is that someone in a leadership role will often mandate an increase in team productivity by directing or requiring a percent increase in output; for example, “Team A must complete 10% more points each sprint.” This will often cause unintended behavior such as teams artificially inflating their estimates just to achieve the directed goal; a PBI that used to be five points is now eight points, etc. This type of behavior puts the focus on meaningless increases in the Velocity metric, eliminating its value.

How to Mitigate the Use of Velocity in Scrum

So how should we try to inspire the proper team behavior towards delivering meaningful business value instead of making the metric look good? There are a few variations of the Velocity metric that can help focus the team on improving their predictability and performance.

Percent Complete Versus Planned

The metric I typically recommend over pure Velocity is percent complete versus planned. We can calculate this metric using points or just a total count of the number of work items. For example, if a team completes 10 out of 12 PBIs (or 50 points out of 70 planned), that is equivalent to 83 percent of stories completed, or 71 percent of points completed. The two numbers will probably vary because of the stories having different sizes.

If the team can track this over time—usually at least three sprints—the team will know the average trend, and that will help answer one important question: Is the team performing consistently? If the graph is showing extreme highs and lows, that shows that they need to stabilize the team. The root cause may be variations in team membership, abrupt changes in team capacity, or changes in priorities during the sprint. All of these potential causes provide opportunities for the team to improve. Generally, as a team matures and learns to work together, the Percent Complete Versus Planned should increase since they should know how to plan and execute work more effectively.

In conclusion, measuring Velocity in Scrum can be a powerful tool that can be used for good and for evil. The key is to pay attention to how you are using the metric, as well as the impact it has on your team. If a metric is helping your team do better work and achieve higher quality, chances are you are using the right one. If not, focus on improving by experimenting with alternative metrics that better achieve the goals you’re after.