Use the Net Promoter Score to Measure the Health of your ART
By Stephanie Garrido
Is Your Agile Release Train Healthy?
s a Release Train Engineer (RTE) in a scaled agile environment, it is crucial to validate whether your Agile Release Train (ART) is indeed “healthy.” A healthy ART radiates joy and excitement during PI Planning. The ART’s program board fills up before the draft plan review time, and team pride is apparent as they present their plans at the end of day one.
As they execute the work, the teams readily communicate any issues they experience, and trust they will receive help to achieve their commitments and goals. They work together, help each other, and they emit a sense of empowerment.
If this is not the experience of your current ART, it may be time to learn why. This article will present one method you can use to gather information on what is happening, interpret the results, and act on what you learn.
Gathering the data
When companies want to understand how their customers feel about them or their software products, many choose to measure their Net Promoter Score® (NPS®), a method created by Satmetrix.
According to Satmetrix’s website, this metric provides “the core measurement for customer experience management programs the world round.” The genius of the Net Promoter Score is its simplicity. The survey taker is asked one question: How likely is it that you would recommend [brand] to a friend or colleague? No multi-page surveys. No multiple-choice questions. Just one simple, thought-provoking question that yields powerful, actionable information.
T
o use this method on your ART, you just need to make a simple change to the question: How likely is it that you would recommend a friend or colleague to come work on [Program’s] Agile Release Train? Adding a simple question helps yield more complete and potentially actionable information. That question is, Why? To make this work, you would format a short survey like this:
Your ART team members will make their selections and provide a reason for their response. Once you have received the answers, you will notice some reactions run the gamut from sarcasm and jokes to serious systemic concerns. Not all the information will be actionable or even useful. But amid the noise, patterns will emerge, and you will use these patterns to create actionable data. But first, let’s interpret the results of your survey!
Interpreting the NPS Results
Following the Satmetrix pattern, here is how you should interpret the results of your 1-10 scores.
Promoters (score 9-10) are loyal enthusiasts who will keep working on your ART and even recommend their friends come work there, fueling a happy, healthy release train.
Passives (score 7-8) are satisfied but unenthusiastic team members who will probably want to work elsewhere.
Detractors (score 0-6) are unhappy team members who can damage the ART’s reputation and impede the health of the team through negative word-of-mouth.
Subtracting the percentage of Detractors from the percentage of Promoters yields the Net Promoter Score, which can range from -100 (every team member is a detractor) to 100 (every team member is a Promoter).
Reacting to the Results
When you first introduce the concept of the NPS score to your ART, the information generated could evoke strong emotional reactions. Managers, individual team members, or whole teams could become defensive over the NPS score or the comments. As a facilitator, you need to do a few things to help moderate any strong reactions that may occur.
Before starting the exercise, let the team know that it is ok if we are not happy with the outcomes.
Stay calm and neutral in your response.
Resist trying to calm their emotional responses.
Focus on what they are saying first, not how they are saying it.
Once you understand what they are saying, then also consider how they are saying it and what this means.
Remind the team of the intended outcome – to grow as a team over time.
Visualizing the NPS Results
As you notice where several team members have identified the same concern in the “why” field, begin plotting their responses on a Pareto chart like the one below, merging like responses into a single category. Note the number of times the same issue is identified and be sure your chart goes from highest to lowest number of responses so that it is easy to determine which response categories were the top two reported.
Once you have all your response patterns plotted in this format, you should be able to quickly identify which areas are causing your ART the most trouble. You can now conduct a brainstorming activity with your team so you can flesh out the actions required to resolve some of these issues.
Alternatively, you may also see patterns of the same positive responses, so your task may be to brainstorm how we do more of these things on our ART.
Taking Action on the NPS Results
By the time you finish gathering, interpreting, and plotting your survey results, you should have three things – a Net Promoter Score and two Pareto Charts – one with patterns of concerns and one with patterns of kudos. It is important to show both so that as the team moves forward, the enjoyable things about working on the ART do not change and even improve.
Before acting on the concerns reported, the team will need to go through an additional exercise, a problem-solving task. Start by doing the following:
Focus on attempting to solve just the top two issues revealed by your Pareto chart.
Split your ART into two groups and assign each group one of the issues.
Make sure to split managers/ART leaders up so that each group has a manager/leader represented.
Have each team brainstorm solutions to their assigned issue.
For some ideas on how to help your teams brainstorm solutions, ASQ provides a great Fish Bone Diagram template along with an explanation of how to use it. Once you have your Fish Bone Diagrams, the teams will need to conduct further analysis to understand the root cause of the problems they have identified.
At this point, it’s time to apply The 5 Whys, a simple but powerful technique for getting to the root cause of an issue. This HBR video provides a quick but thorough explanation.
Continuous Improvement with NPS Results
To see results over time, conduct the NPS survey at least once per quarter. Focus the team by setting a goal of 10% improvement in your NPS score each quarter. Maintain reasonable expectations about improvement metrics since extraordinarily complex problems may take more time to solve.
Above all, enjoy the process as you watch your ART learn more about itself and attempt to make your organization the best place to work! Happy scoring!