Ugh. What are you supposed to do when the executives ask for an “X”% improvement in the performance of your agile teams?
I’ve been researching and reading a bunch of things to try and come up with a solution to this problem and so far it seem like there is no solution. Any way you try and measure a team’s performance can be ‘gamed’.
- Velocity. Easy enough as the team tracks velocity each sprint. As soon as they become aware that they are being evaluated on velocity it is simple enough to make a 3 point story a 5 point story, or an 8 point into a 13 point. Boom! Instant productivity gain.
- Stories completed. This is a better metric, however it sets the team up for some bad practices. Product Owner needs a simple change? I need a Story for that. We are being measured on stories? Okay, let’s break the stories out into Dev and QA tasks, also let’s add a CRT task and start putting in tasks for our Agile Ceremonies.
In addition to the bad practices for the stories the good practices that you may be trying to instil, like breaking large stories down to smaller stories, will again lead to a false result in productivity improvement.
All the articles I have read about Agile Metrics have not been any help either.
- Five Agile Metrics you won’t hate list “Sprint Burndown”, “Epic & Release Burndown”, “Velocity”, “Control Chart” and “Cumulative Flow Diagram”
- Measuring Team Performance states that you should use the ability to hit the Sprint Goal, with the caveat that if a team continually hits their sprint goal then they are under performing. This might be a good way to evaluate how the team is doing, but not for measuring any kind of improvement.
The overall goal is definitely to make the teams better, more productive. The issue comes in when the metrics are to be used to evaluate the teams against each other. This can not and should not be done. A team that has a velocity of 120 points per sprint is NOT better than a team that does 40 points.
Executives that are not as experienced in Agile are looking for ways to either judge teams against each other or measure success of improvements through the use of metrics. In “Metrics you can bet on” Mike Cohn discusses how metrics can be misinterpreted and that as an organization you need to be very careful to measure at the correct level.
Great care must be taken when going down this path. Executives have investors and boards that they are accountable to and have to show value for the money they are spending on these activities.
I think better metrics are turnover rates, happiness factors and improvements made. Look at the teams’ retrospective action items, how many are they solving? How often are people leaving the organization?
No matter what metric is being gathered it is very difficult to use any of it to evaluate team against team or team performance over time.