The 7 Dysfunctions of Scrum
Scrum seems to be the software development framework that all software companies are using these days. Or at least they all claim to be practicing scrum. To be honest, my best experience with scrum was on a labs team at Microsoft. That team was probably closest to executing scrum as defined by the official framework. On that team, we would often start from near-scratch each sprint, questioning what features we thought we might want to build. We had a backlog, but new things were added every time and things that had been on top previously might become lower priority or even removed depending on new information. All of our tasks were both end to end and small. We didn’t try to have any long term projects and didn’t have specific outside commitments. On a larger level, each quarter, we voted as a larger team on which projects we were going to work on. PMs would put together pitches — and this included re-pitching continuations of current projects (which were usually, but not always, selected). We were agile and we pivoted often.
Since then, I’ve worked on a large number of teams who have all claimed to practice scrum but have actually followed it to varying degrees with varying levels of waterfall hiding just below the surface. Many of these teams have had scrum dysfunctions which often took large amounts of time. I find that the longer I’m in software, the less patience I have for some of these dysfunctions. Most of the dysfunctions grow out of trying to smash together scrum and waterfall or from focusing too much on the scrum rituals and practices themselves while losing sight of why we were doing them in the first place. In many cases, they’re a result of both.
By examining these dysfunctions directly, I hope we can find ways to overcome them. The principals behind agile are strong and I’ve seen it be successful. However, I’ve also seen a lot of teams waste a lot of time. In the true spirit of agile, hopefully we can use these as a starting point for continuous process improvement.
1. The Myth of Perfect Predictability
There will always be unknowns. There will be unknown unknowns. Most teams I’ve been on want to get better at everything, including estimation, and that’s to be commended and encouraged. However, I’ve also seen that extend to trying to find a way to eliminate all unknowns. Let me give you a hint: after over 12 years working in software, I’ve never seen a team even come close. Of course, if you’re totally shooting in the dark or if you’re never including database work in your estimates, you should definitely change that. However, at some point you reach quickly diminishing returns.
Every team I’ve ever been on that has tried to account for everything has failed. We will think we got it all, but some new unexpected technical problem inevitably comes up. Then, we retrospect and agree to include this new thing the next time and something else comes up. We spend a lot of time talking about how we could have missed that and how to make sure we get it next time. And the cycle continues.
It’s still definitely worth taking the time to try to make sure you don’t continue to miss the same things. However, I’ve been on other teams where, after coming up with a high level plan, instead of worrying about minute details, we just added a large padding number to it. The surprising thing we found project after project with that team was that the padding actually fairly well accounted for the unknowns.
Over the longer time-horizon of a project (rather than just a sprint), many unknowns balance each other out. One sprint may have a lot of interviews while another has a site issue and a third may have nothing. Of course, it’s still useful to include any known big disruptions in your planning (e.g. half the team is taking the same two weeks of vacation), but don’t spend too much time sweating the small stuff. Look for general patterns that are throwing things off, but don’t spend hours trying to predict every minute detail.
2. Good Estimates Take Time
Along similar lines, really good estimates for work take time to put together. The more accurate you want the estimates, the more time it will take. Sometimes this is worth the trade-off, sometimes it isn’t, but good estimates never come for free. If you give me a one pager on a large project and I think about it for 5 minutes, I can give you a high level swag, but it has a lot of room for error. If you give me several weeks to plan out the architecture and details, it will be a lot better. If you let me actually prototype it, it will be even better. However, even if you let me do all of these things, surprises can still show up. Sometimes it’s worth having the detailed plan and strong estimates, but sometimes it isn’t. It’s also possible to spend more time trying to uncover all of the unknowns than it is to just do the work. It’s important to know the trade-offs. Don’t always focus on having better estimates — sometimes the high level is good enough. On the flip side, if you want the detailed estimates, know that getting them will take time and make sure to plan in that time accordingly.
Spending too much time on estimation plays out on a smaller scale too, with tasks and not just projects. I was on a team once that would spend a large amount of time debating each task when we did estimations and we would file separate tickets for writing the tests for each task and estimate those as well. We probably easily spent at least 90 minutes on estimation every two weeks (and sometimes 2–3 hours). Which, once it includes the whole team, is a huge chunk of time. What I noticed, is that while that team may have had better estimates for any given chunk of work — a particular task was more likely to be right, the overall estimates weren’t much better. For example, I was later on a team where we would just take the higher estimate number if the team was pretty evenly divided between two consecutive numbers or take whichever number the majority of the team had estimated if there was a clear winner. If three consecutive numbers showed up, we would often take the middle one. We would only discuss if someone felt strongly or if there was a wider spread than three numbers (which was rare). We probably spent 15–30 minutes on estimation every two weeks. We may have been off more often on individual tasks, but once you picked up a few of those tasks into a sprint, it largely came out the same. We would underestimate some but we would also overestimate some. Most of the time on that original team, even when we debated, we came to the middle number anyway, so the end result of what was chosen also didn’t actually change too much.
All of this is to say that you can take time and get better estimates, both on the project level and the task level. You can take the time to fully consider and include everything that anyone on the team happens to think might come into play. However, it takes significantly more time and often doesn’t provide significantly better results. I’m not going to say that it never makes sense to take the time, but I will say that it would take quite a bit of convincing to get me to go back to those 2 hour estimation sessions.
3. Emphasis on Zero
In strict scrum, you commit to a set of sprint goals at the beginning of a sprint and should have it completed by the end (although on most teams I’ve been on, we’ve skipped the goals and just picked a set of tasks). In theory, this is a cohesive end to end, fully functioning body of work. You should be able to demo what you accomplished during the sprint at the end. The point of doing this is to make sure there’s a focus on making incremental progress that’s always fully functional. This allows user testing earlier and also speeds up how quickly you can deliver value to customers by releasing tiny, fully useful things at a time. This also leads to integrations happening earlier and problems being uncovered earlier. Rather than trying to integrate four fully formed systems after 6 months of work, you’ve been integrating tiny pieces constantly. With this in mind, you want to fully complete some chunk at the end of the sprint and therefore, your burn down should be at zero at the end.
This sounds great in theory. The problem comes in when you actually try to do this. I’ve found that teams often lose sight of the idea of end to end, cohesive chunks of work, but don’t always lose sight of ‘getting to zero.’ We suddenly end up in a situation where we’re not getting the benefit of fully completed chunks but we’re still focusing on getting to zero on an arbitrary chunk of work. If it’s arbitrary, getting to zero is no longer important — why does it matter if you’re at zero? What really matters is that you’re completing a consistent, predictable number of points sprint over sprint. If there’s a high emphasis on zero, several things tend to happen. The team tends to be less ambitious, knowing that if they go for a stretch goal and can’t finish it all, they’ll be penalized for not reaching it. Likewise, if members of the team run out of things to work on, they’re less likely to pull something additional into the sprint, knowing that they’re unlikely to finish the new thing, and therefore, will again, be penalized. I think we can all agree that we want our teams to be ambitious and to work hard through the end of the sprint, even if they got more done than anticipated.
Instead of emphasizing zero, focus on emphasizing a predictable cadence and a priority order. Did we complete a similar number of points as last sprint? Was there a big jump or significant drop? It’s useful for the team to be predictable, and by consistently completing similar numbers of points, this is predictable. Typically though, outside people who care about predictability care about it because they want some idea of when something might finish and if we pull in 75 points, knowing we can finish 50 and don’t prioritize it, it’s pretty unclear which points will be completed. However, if we pull in that 75 points but have a clear priority order, anyone using it to plan externally can know that the first 25 will almost definitely get finished, the next 25 have a high probability and the last 25 might get started but probably won’t finish this sprint. The information is there without the misaligned incentives.
4. Predictable != A Pretty Burndown Chart
On the topic of predictability, most of the teams I’ve been on constantly talk about getting more predictable. It makes sense that upper management wants to have a better idea of how much can be completed in a given time frame, so predictability is important. However, most of us then get sidetracked by focusing on how we’re trying to measure that predictability rather than actually improving our predictability.
One of the common tools used to measure if we’re accomplishing what we expect is the burndown chart. This chart shows how many points are still open in the sprint over time. Ideally, this will be a line that goes steadily down to zero. As it turns out, however, it’s possible to make our burndown charts look really good without actually becoming any more predictable. It’s even possible for a team to have an amazing burn down chart while still being completely unpredictable.
Focusing on the burndown chart naturally leads to trying to improve the burndown chart. However, I’ve often seen that by trying to improve our burndown charts, we end up burying the information that would actually help us improve predictability. Let me explain. For a team to be predictable, we need to know how much of the work we planned at the beginning of some time period actually got done. That tells us how quickly we can complete the remainder of the work we need to do on a project. Even if we consistently only complete 1 “week” worth work in a two week sprint, this is predictable. We can compute from this when we will complete the remaining work with that work’s current estimates.
However, in order to improve a burndown chart, I’ve seen teams do three things that all have negative affects — removing things from the sprint, re-estimating up and re-estimating down. First, in an effort to better acknowledge all of the work actually done by a team during a sprint, teams want to add stories with points for everything unexpected that comes up. However, they don’t want to completely mess up their burndown chart due to that unexpected customer issue, so they do what we called “Brick in, brick out.” This practice basically means, if you add points to a sprint, due to a bug, or whatever, you should remove an equal number of points from the sprint. Doing this does actually have several advantages — it improves your burndown chart, it allows the individual or the team to make a conscious decision of what’s getting eliminated from the sprint instead and if done early in the sprint, can reset expectations about what will actually be completed in the sprint. However, the big problem with doing this, is that it totally loses the information of what was originally planned. It’s impossible to track how much of what was originally in the sprint is still in the sprint. Doing this too often also leads to the planning at the beginning of the sprint being at least partially meaningless. What does it matter if I’ve committed to doing something this sprint if there’s a 50/50 chance that I’m going to replace it with something else? It also completely loses the information of how many points I should plan at the beginning that will likely actually get accomplished in the sprint. It may be the case I consistently do 30 points planned from the beginning and another 20 unplanned points, but because I keep pulling things out, it’s hard to see that. Now in a future sprint, I’ll see that I completed 50 points and continue to plan for 50 points even though I will almost definitely only finish 30 of those.
Another practice I’ve often seen in conjunction with this is re-estimating work. Basically, if I get to the middle of a sprint and realize that a task I’ve picked up is far larger than I anticipated, I increase the points on that task and then brick something out (remove something else from the sprint). It will improve the burndown chart, but suddenly, I’m hiding the fact that the task was misestimated in the first place. I’ve also completely lost the information about how many points I completed as compared to my original estimates and therefore, how many points I’ll likely be able to complete each sprint as compared to my remaining estimates. The speed at which I’ll complete the rest of the work is now completely buried because I’m updating the estimate on the fly to match the actual completion rate. For example, it may be the case that I underestimated everything on the project by 20%, but because I’m updating the estimates as I go, there’s no way to see that and we will continue to operate under the assumption that the remaining estimates are correct. Meanwhile, if we left the estimates the same, our velocity would slow and we’d realize that we couldn’t take as many points into each sprint, which naturally adjusts the completion time required for the remaining work.
The final practice I sometimes see is changing an estimate down at the start of a new sprint if it gets carried over from the last sprint. This often feels like the right thing — that way we’re reflecting the actual amount of work left. However, it turns out that if we never do this, it works out about the same. Think about it — if we often complete tasks but they don’t get released until early the next week, we might knock out a bunch of points at the beginning of the sprint, but we will probably again have a bunch hanging over the end that are almost complete. It typically averages out so that assuming you’re releasing regularly, the number of points completed in the sprint is the same whether we somehow find a way to divide those points into the actual sprint they were completed in or if we just leave them. Meanwhile, if points are changed, it loses the information of the original estimates and we no longer have a mapping of original estimate to actual completion. If you really want to have the points in the actual sprint they were completed in, the better practice would be to split the ticket into sub-tasks with the total points being the same as the original.
At the end of the day, a burndown chart can be a useful thing to look at and a really bad looking one can definitely be indicative of problems. However, a good looking one does not necessarily mean there are no problems — in fact it can actually mean that the problems are still there but are now completely hidden and impossible to fix. I’ve also seen enough teams falling into one or more of the above traps that I no longer like using burndown charts at all. In my experience, burn up charts or velocity charts are often a better measure of what’s actually happening and if there are problems.
5. Points Aren’t Everything
I was once told that in strict scrum you shouldn’t add points to bugs. However, I’ve only been on one team that actually stuck to that. Likewise, we shouldn’t be adding points for things like meetings, interviewing, etc. However, as engineers and as people, we like to have all of our work recognized. We also operate under this perception that if we don’t allocate points (representing time in this case), we won’t be given the space to complete tasks. So we make tasks for everything and put points on all of them. Adding points to all of our tasks can still allow for predictable sprint completion — a team can complete a consistent number of points each sprint (and in fact this is probably the best way to get a consistent number). However, it makes it less clear how many points of each sprint are done toward the main project. It is possible to separate out types of points, but I’ve never seen a team actually do it. Instead, teams include all of these points when adding things to a sprint during planning, but they never actually separate them again when they look at their run rate to estimate how long it will take to complete a project.
For example, when I’m planning a sprint, I may consistently put in a 2 point task to cover all of the tech lead work I do. However, when the project manager is looking at when a project will complete, they will almost always just take the number of points left on that project and divide it by the average number of points completed in a sprint. This projection assumes that all points in the sprint are for work directly for the project when in fact this is far from the truth.
If we don’t add points to any of these outside tasks (including bugs), we’re only assigning points to our truly planned work and the team’s average completion rate will more directly reflect the actual rate towards project completion. This naturally builds in a buffer. The average completion rate will more accurately reflect the continued completion rate of only the planned items.
The other solution here is to separate project tasks from all others. If you track how many points toward the project you accomplish each sprint, over time you can accurately estimate how many you will likely continue to be able to complete. This option has the added benefit that once you’re separating points, you can also start to get an idea of what else might be using large numbers of points. The hard part of this option is that I don’t know of any tools that allow you to easily separate like this, so it’s challenging and time consuming to actually track.
On top of adding points to everything, nearly every team I’ve ever been on has been constantly tinkering with how we estimate, how we assign points, etc. This isn’t necessarily bad as it does drive toward continuous improvement. However, this means that it’s never possible to truly reach a steady state. Scrum aims to find your team’s velocity so that you can use that to plan future sprints. However, almost all of the time I’ve spent on teams, we’ve talked about how our velocity isn’t predictable yet because we have a new team member of because of a holiday or because the team structure changed or most commonly because we haven’t gotten good at estimating yet.
While erratically bad estimations make us unpredictable, changing how we estimate is at least as big a problem. In fact, bad estimations, as long as they’re consistent, aren’t actually a problem. For example, if we always estimate low, a buffer will naturally get built in as we use our past velocity (based on actual completion) to plan future sprints and the and the same goes for if we always estimate too high. This is actually the reason why strict scrum emphasizes using points rather than time. Using time causes us to focus more on getting that time estimate to actually match the time taken to get it done. Meanwhile, if points are used (even if they start with some semblance of a time reference point — 1 day = 1 point), we can more easily ignore if 2 points is actually 4 days on average.
6. Expecting Velocity to Match Calendar Time
On some level velocity does indicate the time it will take to complete a given number of points. The mistake often comes when we assume that velocity is the full story or that if the points don’t match the time, we need to re-estimate. There are two major problems with:
remaining time to completion = points / (avg. person velocity * people)
The first is dependencies. In an ideal world, if we put 3 people on a project, we’d be able to perfectly parallelize everything and keep all three people busy the entire time. They’d consistently whittle down on the remaining tasks until everything is done. In practice if you get good at dividing tasks into the smallest tasks possible and allow people to pick up tasks on the fly, it’s possible to move towards this, but it’s still never fully possible. The dependencies must be taken into account on some level. After a certain point, adding more people to a project will slow it down, not speed it up. Certain things will always need to be completed before other things.
The second problem is assuming that points are a good representation of calendar time. When we estimate, we assume that points are roughly a representation of developer work hours. If I estimate this to be 1 point, it will take roughly one uninterrupted day for one engineer to complete (for example). However, some tasks, such as writing a design doc, may only take 2 uninterrupted days of work to complete, but they likely require time waiting for others to review, scheduling a meeting, responding to feedback, etc. In essence, it could easily take several weeks to complete, but very little of the developer’s active time will be spent on that task and in fact, they will likely be doing other things at the same time. With this in mind, the estimation based on developer work hours is probably the right one in terms of planning how much work can be accomplished in a sprint. However, when planning a timeline, these points are no longer an accurate representation.
All of this is to say that while scrum velocity can certainly help in planning out your waterfall timelines and deadlines, they’re not the entire picture. If you find that your team keeps missing deadlines, it’s not necessarily because of any problem with estimation or velocity, but rather a problem with project planning. Trying to fix this with changing estimation likely won’t fix the problem either because it still won’t take into account dependencies and because all points don’t translate into the same number of days on the calendar even if they represent exactly the same number of days of work.
7. Ineffective Standup
I’ve basically never seen standup be super effective. I’m not saying it isn’t possible, I’ve just never seen it. In theory, I love the idea of standup — its a chance to get the team together to understand what everyone is working on and to surface blockers and accidental cross-work. However, in practice, I’ve seen that when a team is large, many people (including me) have a hard time actually paying attention to every person through the whole standup (and if everyone is zoning out, what’s the point?). Meanwhile, when a team is small, team members are mostly already talking about these things anyway and the standup is largely redundant.
The other big problem that sometimes happens with standup is that people wait to talk to their teammates until standup and then forget what it was they were going to say. In these cases, it actually leads to less communication, not more. While we do want to try to prevent too many interruptions to people, it’s also important to communicate at a point when things are still pertinent.
My current team has been playing with exactly what we do while at standup and we keep trending back to the typical status updates because it’s easy. That said, I think that we would all agree that it’s not the most effective thing we could be doing. We just aren’t totally sure what is. The key here is to figure out what we’re hoping to get out of standup. Are we trying to surface things that are blocked? Are we trying to surface unknown dependencies? Is the team not talking enough otherwise and should be forced? For all of these, we should also be asking ourselves what the most effective way to solve these problems is — maybe that isn’t standup at all.
Conclusion
While I’ve focused here on the dysfunctions, Scrum actually has a lot of advantages as well (my favorite being retrospectives which promote team growth and learning). I’m not going to attempt to go into them here, but I will say that I’m actually generally pro-Scrum. That said though, I’ve seen a lot of time wasted on Scrum processes in ways that are extremely counter-productive. While a lot of the things that I’ve outlined here are very different, I think we can improve in all of our processes and become more efficient if we focus on why we’re doing something and make sure we’re accomplishing the why and not just the what.
