Software Estimation on a Product Team Part 4: Planning Poker Estimation

Let’s say that we’ve chosen an estimation approach—be it story points, T-shirt sizing, or any scale that makes sense for your team. How does the team arrive at an estimate for a given product backlog item? This is the topic of estimation techniques. I cover one popular technique here: planning poker estimation.

Before diving into the technique, however, it is important to understand why we need to be deliberate and rigorous about it. You may be wondering: why don’t we just get the team together, discuss freely, and decide?

The Anchoring Bias Problem

In the first part of this article series, we stated the principle of ownership: all team members own the estimates. For teams to abide by these principles, they will often need to arrive at estimates together. This typically holds true for at least some of the work; it may not be necessary to estimate all the work in the same manner. For example, some teams may delegate to a senior engineer, or they may agree that smaller tickets are to be estimated by any team member as needed.

Assuming the entire team must agree on the estimate, at least in some cases, what would the simplest possible technique look like? Perhaps the following:

1. Initial Discussion
   1. Team discusses a given backlog item
   2. Emphasis is placed on characteristics that would affect the estimate
2. “Anchoring” Estimate
   1. A participant speaks up and proposes an estimate for the item
   2. The session facilitator asks, “Does anyone disagree?”
3. Subsequent Discussions
   1. A vocal subset of participants voice disagreements (usually one or two)
   2. Discussion among them ensues while the other participants disengage
4. Biased Consensus
   1. Consensus is biased toward the perspectives of those who chose to speak up
5. No Consensus
   1. When a consensus cannot be reached, a decision is made on next steps
   2. Typically, the backlog owner will take ownership of the next steps for the backlog item

The above steps are then repeated for the next backlog item.

Before criticizing this technique, I should mention that the only way for a team to know whether a technique works for them is to try it. If the team is eager to experiment with this, then it should. However, there is one glaring bias that this technique is susceptible to: anchoring. 

Anchoring is a cognitive bias that is at risk of manifesting here. In particular, the first estimate that is mentioned in a conversation tends to “anchor” the conversation around that estimate. This means the other developers are less likely to suggest a number significantly different from the initial one proposed; consequently, the final estimate will likely be in the vicinity of the first number stated, and there is far less discussion about the estimate than there otherwise could be. 

For example, if one developer has a number in their mind that differs significantly from the originally proposed one, they will be less likely to speak up about it. They may even think that they’re the only one in the room who thought of the outlier estimate. Even if that were the case, there is often sound reasoning behind the outlier estimate, and this reasoning is crucial to share and discuss with the team. Anchoring bias can lessen a team’s ability to effectively discuss alternate viewpoints that yield significantly different estimates from the original one proposed.

The impact of anchoring bias is why I make the case for planning poker estimation. However, the technique yields additional benefits.

The Planning Poker Estimation Process

The roots of planning poker estimation have been rumoured to trace back to the 1950s. At that time, the RAND corporation developed the Delphi method to forecast the impact of technology on warfare. Barry Boehm and John A. Farquhar subsequently adapted it for use in software, naming the variant “Wideband Delphi” in Boehm’s 1981 book, Engineering Economics.

Although there are obvious similarities between Wideband Delphi and planning poker, the connection has been dismissed by James Grenning, who was the first to describe the use of planning poker in 2002. The technique was subsequently popularized by Mike Cohn in his 2008 book, Agile Estimating and Planning.

Mike Cohn describes planning poker estimation in this article. Here’s how I have seen the technique most commonly used:

1. Initial Discussion
   1. Cover just enough details for a first-pass estimate
   2. Discussion about actual estimates is off limits here
   3. Facilitator moves to the next step once everyone is ready to estimate
2. Independent Estimation
   1. Team takes a moment in silence to choose an estimate on their own
   2. All team members share their estimate simultaneously
   3. Estimate can be shared via a finger count, a chat window, or a special card deck
3. Outliers Debate
   1. Those who voted higher or lower from the group (i.e., the outliers) speak to “defend” their estimate
   2. A brief discussion may ensue
   3. Key questions, assumptions, or other implementation details may be discovered
4. Balanced Consensus
   1. Team repeats steps 2 to 3 for a few rounds or until consensus is reached
5. No Consensus
   1. If consensus is not reached in 3 rounds, the team attempts to determine the cause of the misalignment, and an action item is planned for instead
   2. For example, a technical assessment may be needed because the uncertainty is too high for a consensus to be reached

As with the unstructured discussion example, the above steps are repeated for the next backlog item.

1. Initial Discussion

If your team has already worked through several backlog items to completion and has adopted either an iterative or Kanban work pattern, your team likely has an understanding of “ready to implement” or “dev ready.” This means that the backlog item is ready for the team to begin implementation; in other words, the software engineers on the team can begin to work on code that the team intends to ship.

This ticket state is typically a good starting point for answering the question, “How do we know if this item is estimable by the team?” Note that this is only a possible starting point. Some teams may choose to estimate earlier; in fact, the team may decide that an agreed-upon estimate is a prerequisite for a product backlog item to be deemed “dev ready.”

That said, it is important that the facilitator of the estimation session encourage participants to keep the details sparse and the discussion brief. The point of the first step is to gather enough details so that the team can give a first-pass estimate, nothing more. Consequently, discussions around what the estimates could or should be are strictly off limits here. These discussions and ensuing details are reserved for steps 3 to 5. A good way of encouraging brevity and movement to the next step is to ask repeatedly, if necessary), “Do we have enough for a first-pass estimate?” If necessary, the facilitator can also timebox the discussion at this step to a few minutes.

2. Independent Estimation

Team members silently choose an estimate

Once a team gets into a groove, this step can go surprisingly quickly; elapsed times of under a second are not uncommon. The biggest danger here is moving to the next step before the whole team has their estimate “locked in”; there will often be a straggler that takes slightly longer than everyone else. If the team moves onto the next step prematurely, the undecided estimator’s estimate will now be influenced by their peers’ estimates, causing anchoring bias to creep in.

A common way to prevent this is to use a physical prop such as a deck of cards. Although specialized “planning poker cards” designed for this purpose can be purchased, a traditional deck of cards can be equally effective. In this case, the facilitator decides how the cards map to your team’s estimation scheme—be it T-shirt sizing, story points, or breeds of dog.

One approach that has proved effective over group video chat is the use of a hand gesture to indicate that you are ready with your estimate. In our case, we naturally settled on a closed fist, which we subsequently opened with the number of fingers that corresponded to our respective estimate. One danger of this is that you or your teammates may be tempted to change your estimate at the last minute. If you don’t trust yourself, holding up a card and flipping it solves that problem!

Team shares estimates simultaneously

A simple countdown usually suffices to make this happen. That said, if your team is using a method whereby participants can easily change their estimates at the last minute (e.g., by holding up fingers or submitting estimates in the chat), the facilitator should take care to ensure that delayed estimate submission is not happening repeatedly.

At this point, it’s important to recognize that some teammates may not be able to submit an estimate, for whatever reason. Maybe they can’t decide. Maybe they’re confused about the level of technical detail. Or maybe they just zoned out. We are all human after all. That’s why I believe that there should always be a “pass” option. Some planning poker decks cheekily use a “coffee cup” icon to denote this. Feel free to be creative. There’s no reason why estimation can’t be a fun activity.

If a large proportion of the participants “pass” on the estimate, this provides an important signal to the facilitator and the team. It could mean that more discussion is needed or that estimation needs to be deferred so that more detail can be gathered asynchronously.

3. Outliers Debate

The outliers defend their estimates

The facilitator forbids any discussion of the estimates themselves in the first step. Here, they open it up. However, it’s not meant to be a free-for-all. Because the discussion ultimately revolves around reaching a consensus on an estimate, it is the outlier estimators who will have the most important points of discussion.

This can also be the most challenging aspect of planning poker for the team to adopt, as it requires a high degree of team safety. Often, a team member who ends up being an outlier feels they just got the estimate wrong. It is critical that this notion is dispelled and that they are encouraged to share their reasoning. On several occasions, I have personally witnessed less experienced software engineers surface critical points at this step, which ended up saving the team weeks of wasted effort. An added benefit of such discussions is that it leads to a recognition of the value of a diverse team—be it a junior teammate’s beginner’s mindset or a senior teammate’s expertise—and a subsequent reinforcement of mutual respect.

Team repeats until consensus is reached, up to three rounds

Although an initial round of outliers is common, teams usually align by the third round. The follow-up rounds should also happen fairly quickly. It’s important that the facilitator focus the discussion around the details needed to establish consensus on an estimate. For example, they take detailed notes of assumptions or scope out certain edge cases entirely.

What happens if there is no consensus after three rounds? It ultimately depends on the team. Usually, the next step is gathering more detail on a ticket asynchronously. This could mean brainstorming how to address certain edge cases uncovered during the discussion or a technical spike solution to improve the team’s understanding of the scope. It could also involve splitting the backlog item into smaller user stories or tasks. 

In any case, by the third round, the next steps are usually apparent to the team. It is healthy and common for the team to fail to establish consensus on estimates, as long as the team can decide on next steps. Conversely, it is an unreasonable expectation to have every ticket estimated after a planning poker estimation session.

4/5. Consensus vs. Progress

The estimation session facilitator may be tempted to plan for a certain set of backlog items to be estimated by the end of the session and consequently allocate a timebox for each item during the session. I would advise against this strategy. The very nature of un-estimated work means that one cannot estimate the work required to determine the estimate. Some seemingly complex tickets are surprisingly quick to estimate, whereas simple items can end up being anything but. This is the hidden value of estimation: it exposes risk early in the product development process, potentially even before shippable code is written.

Instead, try this: prior to the estimation session, have the facilitator and the product owner establish a prioritized list of estimates and guidelines around when and how to check in during the session. For example, the product owner may instruct the facilitator, “If an item is taking longer than 10 minutes to estimate, check in with me, and we may decide to move on to the next one.”

Closing Thoughts

Software estimation is a practice that builds the skills of both individuals and teams. Individuals learn to ask the important questions, while teams learn to run estimation sessions smoothly through an unspoken understanding of focusing on what’s important (while efficiently deferring what is not). This skill-building opportunity of software estimation is precisely why it takes time for a team to adopt it effectively. I encourage your team to approach this practice with open-mindedness and patience. It’s easy to dismiss the practice as a waste of valuable time if the team only uses productivity-based metrics to assess progress initially (e.g., number of items estimated and precision of estimates). Instead, acknowledge that the team is adopting a new practice that will take time to learn. In that vein, brainstorm learning-based metrics to assess progress initially. Is the team learning how to split up and define the work appropriately? Is the team learning about risk earlier than it would have otherwise? Are team members learning from each other?

It takes time for teams to estimate reliably. Oftentimes, low estimate precision occurs because of factors that are impossible to address in an estimation session, such as unexpected technical debt or high team turnover. In these situations, estimates illuminate hidden problems that the team is facing. This is one of the many unexpected benefits of software estimation that enable teams to work better together and ultimately build better products.