#NoEstimates puzzle experiment in Melbourne

(first published on November 1st 2013, migrated from my deprecated Wordpress site)

Craig Brown and I ran a variant of Chris Chapman’s now famous jigsaw puzzle experiment at a meetup of the Melbourne Agile and Scrum User Group on Wednesday evening.

Everyone had fun and was intensely engaged throughout. There were loads of interesting dynamics emerging from the teams, perhaps surprising given the contrived nature of the experiment.

Set up

• We set up three same-sized (10–12 people) teams, each with:
• an identical jigsaw puzzle (way too big to be completed)
• a Product Owner (to provide the vision and direction) and
• a Scrum Master (to help the team achieve the PO’s vision)
• We opted for 3 * 15-minute iterations, with 3 minutes for a Retro in between
• Each team was told to use a different method — one was a Scrum team, one was a “mob team” and one was a “no rules” team. Here’s what that meant:

Scrum team

• Must have Planning (including estimation), Review and Retro in each iteration
• We provided Planning Poker cards for the estimation but the team was free to choose whatever estimation method they liked
• Must only work on “stories” agreed in Planning — new stories can’t be introduced mid-iteration
• Stories are only “done” when PO accepts them (in Review or before)

“Mob” team

• No formal ceremonies required
• Team all works on one story at a time until “done” (single-piece flow approach)
• No estimation
• Retro encouraged but not “enforced”

“No Rules” team

• Can work like the Scrum team, the Mob team, any combination of the two, or any other way they like

Outcome

• Scrum team delivered most stories (3; the other teams delivered 2 each)
• Whole group was asked to vote on which they thought was the best outcome
• “No rules” team won (emphatically)
• Scrum team lost

Interesting Observations

Here are some empirical observations of the evening’s events and outcomes, along with my interpretation of what they indicate in an Agile/#NoEstimates context (==> in bold italics underneath the observation).

Scrum team

• Delivered most in terms of stories but least in terms of value, both for their Product Owner and as voted for by the wider group
  ==> Output <> Value
  ==> Comparing teams in a useful way would require consistent measures of both effort and value velocity across teams
• Spent far too large a proportion of time (particularly the first iteration) in planning, and needed to be alerted to this fact
  ==> Consistent timeboxing is important to ensure there is time to do all that is required, and for less variability of outcomes
• A member of the team openly admitted that he inflated an estimate because he did not agree with the value of the story that the PO wanted to do next
  ==> Estimates are often gamed, and for various reasons

“No rules” team

• Implicitly chose not to estimate, but instead to maximise the time they had for building
• Eventually delighted their Product Owner (and wider group), but during the game the PO felt like:
• The approach to delivery was too ad-hoc, even chaotic, especially at the beginning
  ==> Teams must collaborate in order to be co-ordinated, improve and deliver the right outcomes
• Stories were too large (epic) so delivery all happened near the end rather than incrementally
  ==> Smaller stories have lower variability and can help with early and frequent delivery, creating better predictability for PO/customer and lessening the need for estimates
  ==> Larger, higher variability stories rely on estimates of time, or at least relative size, to provide the illusion of predictability
• Started with no process at all but this was deemed unproductive (with such a big team), so they split into smaller teams with focused goals
  ==> Smaller teams are more effective because it is easier to collaborate, change direction, gain consensus, etc.

General

• Scrum and Mob team both delivered purely incrementally (concentrating on edges) rather than iteratively (identifying a recognisable area of interest and building upon it), although stories were clearly too big
  ==> An iterative approach is crucial for risk management, predictability and delivering the right thing (value), i.e. without such an approach you have no choice but to estimate
• Product Owners all felt like they weren’t being listened to — this had particularly bad consequences for the Scrum and Mob teams, perhaps due to their purely incremental approach
  ==> Important for all team voices to be heard, especially given the PO is driving what should be built in order to deliver on the vision

Thanks for reading! If you are looking for help with your software or product delivery, I provide agile coaching, public training (both theory and practical) up to executive management level, and more. As well as public events, I can also run training internally in your organisation for a massively reduced cost, so please ✍ get in touch.