July 12th, 2010

The One Where The Product Backlog Was Too Big


Andy, the ScrumMaster, was guiding the team through a session of planning poker to estimate the initial Product Backlog so the team could come up with a release plan for the new project. Murray, the Product Owner, had provided the team with the Product Backlog in time for the planning session although it did have to be rescheduled a couple of times as he couldn’t get hold of a couple of the account reps to get their requirements.

The team had timeboxed three hours for the estimation but they had only really just scratched the surface of the Product Backlog and time was nearly up. A quick look at the spreadsheet showed they were probably only about 10% of the way through the product Backlog and that was quite depressing; it was noticeable how the initial energy within the team had all but disappeared and there was still no end in sight.

Unfortunately, even though the team was co-located they had to use a tool to manage the Product Backlog because of the size; they would have preferred to use index cards but the number of index cards needed would have made that impractical. Andy could hardly complain because Murray had done a very thorough job in preparing the Product Backlog and the team now had a very good understanding of what was ahead of them on this project. In terms of fulfilling his responsibilities as a Product Owner, Murray was definitely on the right track.


He still felt something was not quite right though – it didn’t feel very agile – and wondered how to help his team keep create the release plan before they lost all motivation. Andy called a break, saying they would re-convene in an hour and asked Murray for a chat. “Are all of these items absolutely necessary?” he asked.


“Absolutely” came Murray’s reply, “I have even prioritised them as you asked. Priorities 1 through 5. I mean they wouldn’t be called requirements unless they were required, would they?”


Murray started to look worried. “Why? What’s the problem?” he asked.


“I don’t know.” said Andy “But something’s not right here. We have the team estimating a load of work and, at this rate, it’s going to take them the rest of the week just to estimate it all.”


“Could we try something quicker than planning poker?” Murray suggested


“Sure we could.” Andy said “We could try affinity estimation or some other quicker technique but I’m not sure that is really the point. Plus the team are getting a lot of benefit from the discussions that are coming out of the planning poker session”


“Could we cut the Product Backlog down to a more manageable size?” Andy asked


“Not really” Murray said “I don’t want to run the risk of forgetting something after the account reps have actually submitted their requirements”


When the team came back from the break Andy put his unease to the team.


“On the one hand, I think we have learned a huge amount about the product and the product backlog. The effort put in to estimating these items has been immense. Personally I am seeing a big drop in the energy levels and am wondering what the best thing to do next would be. What do you guys think?”


By playing back what he was seeing to the team he immediately did two things. First of all he acknowledged the elephant in the room that everyone else was concerned about but didn’t want to mention, and secondly reminded the team that if something isn’t working they have the power to change it.


“How about we see how much work we have actually estimated before carrying on with the rest?” came the suggestion from Greg, one of the developers. The rest of the team seemed keen on that and so Andy to guide them through commitment based planning to try and get a feeling for their velocity and then use that figure to see how many Sprints it would take them to complete the Product Backlog items they had already estimated.


It turned out to be eight four-week Sprints.


“So nearly eight months work.” Greg summarised “If we extrapolate that out, assuming an average representation of effort over the Product Backlog items, we are talking about nearly four years worth of work on the Product Backlog. This is with just what we know now, let alone what will probably come up over the next 3 or 4 years! Is it really worth the effort of estimating the rest of this stuff right now?”



Andy’s situation is far from rare. I regularly see teams with Product Backlogs that are unmanageably large and have even heard of teams with Product Backlogs in the 10,000 items range


The typical justification for this is to avoid losing information about what is required; users are also considered to be happier if they know that there is a plan to get to their requirement at some point rather than be told “it’s not going to get done”. Ultimately the Product Backlog is the responsibility of the Product Owner as it is their job to ensure a positive return on the investment of the project. If things do need to get done then they are perfectly within their rights to hold the item on the Product Backlog. Most Product Owners, however, are aware of the benefits of regular “tidying” or “grooming” of the Product Backlog, keeping it in a manageable state.


The larger the Product Backlog the more overhead there is for a Scrum team, the more items need to be re-prioritised and re-estimated, the more churn there is on requirements and the more stale they become. In Scrum we ask the ScrumMaster to help the Product Owner in their grooming responsibility and Scrum also allows the development team to spend up to 10% of their Sprint preparing for the next Sprint so, effectively, they are involved in grooming the Product Backlog too.


There are a number of options for Andy and the team here. As Murray suggested they could try a quicker estimation technique – such as affinity estimation, where team members don't vote but instead collaboratively move the items along a scale – to speed up the estimation process. 


This is often a very useful technique to cover more ground in an estimation session. My personal preference is that this is only used when a team is used to working together and, ideally, after a good investment of time into establishing benchmarks and assumptions through something like planning poker. Hopefully the development team were estimating the Product Backlog in priority order and so the items that had been estimated were the most important ones. If this is true then the decreased discussion and debate in the affinity estimation approach is likely to be less of a risk. However, in this particular situation, I agree with Andy that this probably wasn’t going to address the underlying problem.


Another option would be for Murray to group the Product Backlog items into themes of related requirements/features/user needs so that the teams would have less items to size. They would probably need a different set of cards for their planning poker session as these items would most likely be much bigger than the items they were estimating before but it would fit nicely with Scrum. We expect a Product Backlog to be very pyramid/iceberg shaped in terms of size relative to priority. See image below:


The Product Backlog Pyramid
We are comfortable with larger items if they are lower down the priority list as they will be broken down and estimated at a finer level of detail when they become more relevant i.e. closer to implementation.


Mike Cohn calls this a method of “providing multiple views into the Product Backlog” and another way of doing this is to provide views into the Product Backlog based on release schedules i.e. have only the Product Backlog items that are likely to be included in release one showing at the moment. This, combined with theming the Product Backlog, could make the teams’ job much easier without losing any of the detail, the prime fear of Murray.


I have also known teams that set a time limit on Product Backlog items such that, once an item gets added to the Product Backlog, the clock starts ticking and if it hasn’t been planned into a Sprint within say 4 months, it will automatically be removed from the Product Backlog. There is nothing stopping somebody from adding it again but nobody will be notified that it has been removed. If it is that important, people will notice it has gone and will be asking for updates on it regularly. If they don’t, or aren’t, then perhaps it wasn’t that important anyway.



Andy showed good instincts here. Something didn’t seem quite right – he sensed something was not working – and he aired his concerns. Equally he didn’t settle for the first answer as he felt it didn’t really tackle the problem, just dealt with the symptom. Also, he didn’t make the decision himself; he brought it to the attention of the team and asked them how they would like to proceed and what help he could offer them.


A product backlog should be a manageable size. This is ambiguous because what is manageable, or indeed appropriate, for one team will not be for another. We don’t want to be beholden to the overhead of managing unwieldy artefacts. Already the planning for this project was pushed back in order to capture more requirements than could ever realistically be delivered in a reasonable timeframe. This was wasted time (and effort).


“The longer design and requirements documents sit on the shelf, the more likely they will need to be changed […] the real problem is that the requirements were written too soon” – Mary & Tom Poppendieck, Implementing Lean Software Development


We will never know less about our project, and its requirements, than on day 1 so trying to get them all written down on day 1 is not only a folly but also a waste. Scrum is a great balance between very reactive approaches, such as kanban, and the overly predictive approach of waterfall. It allows us to look a little into the future without over-planning. Our product backlog should help us with that but can, all too quickly, take us the other way.



Scrum / scrum master / ScrumMaster Stories


Backlog / Product Owner / Requirements / The One With

0 comments | view comments | add comment


RSS Feed Subscribe to RSS