The problem I’ve noticed is that people don’t feel comfortable being asked to think about how to improve their way of working at a certain fixed point in time. It’s not natural. So what about being more flexible? Doing retrospectives whenever and wherever the team bumps into problems and improvement opportunities?

The result of forced retrospectives is low interest in the activity, lack of participation, maybe even absenteeism; this results in low quality feedback coming out of the session. Some people don’t talk, some people talk about stuff that has nothing to do with the way of working but with the product itself (”the problem we have is that our code base sucks, we need to refactor”), and in general you as ScrumMaster get the feeling that -even though it was valuable- you could’ve done much more with your team’s time. Like going for a beer.

My initial approach to the problem was to think that my retrospectives sucked. I took it up as my responsibility: I put up a Post-it saying “Organize better retrospectives” and I undusted Esther Derby and Diana Larsen’s book and put it in my to-read pile. Alas, I never got around to doing anything (maybe because we were doing “good enough” or maybe because I had other priorities). The project went on to succeed without any improvements in the retrospectives, so nobody complained too much.

Then I moved to a new project and a new team. Now, when you start coaching an implementation of Agile in a large team, during the first month or two it doesn’t really make a lot of sense to organize a retrospective. Why? Because you are in the middle of changing so many things, that the question “how can we improve” doesn’t make much sense. You (as Agile Coach) have a pretty good idea of what has to be done next, the team is busy doing it, and basically everything is improving so fast already and there is so little experience with the new way of working that most of the questions you would get at a forced retrospective are things that you still haven’t gotten around to implementing yet. Retrospectives make most sense in two situations: when the methodology is relatively stable and teams are seeking ways to improve, and when things are going wrong and people want to be heard out. When a team is at the “Shu” level and you are coaching them, and you are doing a good job, most questions that arise tend to be “coming up” items in your checklist.

But then something happened. During meetings that had other purposes, I noticed that issues would come up and people would start complaining about things. If you think about it, this tends to happen a lot: someone tells a joke about how things work at the company and everyone laughs, or someone complains about something, or gives an excuse and you hear mumbled agreements in the crowd. People were talking about real problems that are not being solved.

My antennas went up. This is the type of feedback you want at your retrospectives! Why wait? My proposal: interrupt the meeting and start discussing the issue. This is the perfect time to do it: you have a person that has openly put it forward, you have a crowd that supports him, and nobody is forcing anybody to talk about this. Leave the original purpose of the meeting aside for a moment, and switch to retrospective mode! The feedback and openness you get from the team at this moment is invaluable. As long as you are there to coach the impromptu retrospective, it should be possible to come up with concrete actions in a short time and still keep the original purpose of the meeting within sight.

And the best thing is that as long as these situations happen often enough, you don’t have to organize a formal retrospective at the end of the sprint… and you can go for a beer with your team instead!

We will discuss three strategies to non-waterfall software development:

• Pure incremental
• Pure iterative
• Iterative and incremental

Missing the big picture: the “pure incremental” approach

In this approach to agile software development, it seems that you literally only look at the first story in your backlog, implement it fully, and then scratch your head and ask yourself what comes next. This would imply no analysis and design up front, not even the slightest hint of it (sounds like no Product Backlog and no Product Owner). This is clearly risky in the sense that nobody - nor the development team nor the customer- has any idea on what they are doing or where they are going. They might be moving fast, but in a (potentially) random direction.

Analysis & Design      Delivery 1             Delivery 2             Delivery 3

Pure Incremental

Honestly I have never heard of any team doing this except for pseudo-Agile teams that think that Agile means no documentation and cowboy coding. In large projects, you normally tend to have  the opposite problem (too much analysis and design up front). In any case, I believe that if this is a serious risk the root cause of the problem lies in the lack of a strong Product Owner. Remember the Product Owner (not the team) is responsible for achieving ROI, so if the team ends up making the sunflowers instead of the self-portrait, it’s the PO’s call. I talk more about this at the end of the article.

Never finished: the “pure iterative” approach

Which brings us to the opposite tactic: to go pure iterative. Paton and Adzik seem to defend this approach as the best way to minimize the risk of not delivering anything, and this is where I strongly disagree. I actually see the risk of not delivering “anything” as even larger than in the first case.

In pure iterative (or “spiral development”), you start with an outline of your application and you iteratively refine it until the customer is satisfied. This would seem to bring the advantage that if the customer is happy with a lower level of detail (a.k.a. “quality”), you would finish your project faster and with full coverage of all the scope. Sounds nice in theory.

Analysis & Design         Delivery 1           Delivery 2         Delivery 3

Pure Iterative

What’s wrong with this? In my experience, two very dangerous things.

First, you are still in a “big bang” mentality. Until you reach the minimum level of quality that the customer needs, you basically have nothing usable. Taking the example above, if the customer would run out of budget after Delivery 2 but this would prove to be not good enough to go live, he is left with nothing.

Second, by taking a breadth-first approach, you are investing precious development effort on medium and low priority features that might otherwise never be implemented. You are basically distributing your budget among high and low ROI work equally. You are potentially wasting the customer’s money, since if he later decides to scope cut the lower half of the picture, he will already have paid for a large part of it.

In large projects, these are two very realistic risks. If you were to underestimate and run out of budget, it is much safer strategy to deliver 50% of the scope 100% done than to deliver 100% of the scope 50% done. The first scenario will probably get your 50% approved and this might buy you more budget. The second situation will probably make you fail your Phase 1 UAT and get your project cancelled.

So, what’s the ideal approach to reduce risk?

Iterative and incremental

I believe the ideal approach much closer to scenario one, but with a catch: some analysis and design up front. This is what is normally referred to as iterative and incremental, a mix of both tactics, but I would heavily bias it towards the incremental.

Analysis & Design     Delivery 1              Delivery 2             Delivery 3

Agile: Iterative & Incremental

By first taking a high level approach to defining both the scope and the architecture of your application you eliminate the risk of not knowing where you are going or how to get there.

With this mixed strategy, you get the full benefits of an incremental approach: allowing the customer to leave stories open until the moment they are about to be implemented, maximum ROI, possibility to scope cut or even stop the project at any moment and still get quality working software. And all without any risks of missing the big picture or not being able to evolve the architecture accordingly as the application grows.

Not converging

The risk for the team of not converging to a working application is zero, because it is not their job. This is the responsibility of the Product Owner. A well maintained backlog and burndown will clearly show the project planning and thus where it converges. Any change in requirements will be immediately reflected in the backlog, and any new stories will push the delivery date further ahead. As long as all this is decided by the customer, this is fine. The original XP Days keynote speaks of clueless customers who continuously change direction for lack of an understanding of what they want or need. If this were to happen, the most you can do is detect it early on and try to help take corrective actions. In any case, this is a business risk and not so much a software development risk. It is the cost the customer pays for not having a good Product Owner.