re: The Waste of Defects - Bugs are Stop-the-Line Issues

Martin Jul — Sun, 28 Sep 2008 22:47:55 GMT

Hi Victor and Andrew

Thanks for taking part in the conversation.

If you look into Theory of Constraints you will find a good discussion about why stop-the-line makes sense.

The idea is that you look at the throughput of the whole organisation. In a sequential process there will be at least one step that has the lowest capacity, so it is the one limiting the total output.

Many management systems for production are focused on resource utilization - in TOC we care only about maximizing the utilization of the limiting resource and in fact it is recommended to have slack (less than full utilization) in the other steps.

The main principle then, is to look at how we utilize the limiting resource. We want to make sure that we are not wasting it by letting it be idle, by producing the wrong product or by having to do rework (see the lean wastes: community.ative.dk/.../Lean-Principle-Number-1-_2D00_-Eliminate-Waste.aspx). We want to do everything possible to get the most from it to improve the throughput of the whole system.

Let's assume that the limiting step is the development team. Imagine a stream of work flowing from idea to production software. Now, imagine that the development team emits a piece of defective software. What happens now is that the piece comes back and they have to do it again, and since the overall throughput of the whole process is governed by the development team's capacity it means that the defective piece has limited the output of the entire development process:

If it takes one unit of time to do one feature, and one more to fix the defect when it comes back it means we get only the value for one feature for the price of the two units. Since it governs the output of the entire system this is a very sizable reduction if the defect production rate is high.

It is often a good idea to add a quality control step before the limiting step to make sure that it does not waste its capacity working on something defective that will require it to work on the same feature again later.

This is one of the reasons why Scrum planning works since we have the Conversation about the requirements with the Team and Product Owner and try to clarify the acceptance criteria BEFORE we start working. This is essentially a quality control gate to make sure that we do not waste the limited resource on producing the wrong thing.

As for Victor's example, Deming has a good discussion about quality control and a statistical model for calculating the cost/benefit of quality control (eg. testing) in "Out of the Crisis". It would be interesting to relate this to Victor's example and the specific process and cost pattern.

The idea from TOC is that you always have to look at the whole system and optimize for the current bottleneck. This is not necessarily the same at every phase of the project (eg. at one point the developers could be the constraint, later in the project it could be the systems testing team's time that is most precious and should not be wasted).

Goldratt's The Goal is an excellent business novel that explains the principles of Theory of Constraints and Clarke Ching has just written a new book on the same theme for software development called Rolling Rocks Downhill (I have only read a preproduction manuscript of it so far, but it is very good).

Goldratt: en.wikipedia.org/.../Eliyahu_M._Goldratt

Ching: www.rollingrocksdownhill.com

re: The Waste of Defects - Bugs are Stop-the-Line Issues

Andrew Goddard — Fri, 26 Sep 2008 06:11:30 GMT

Victor,

The whole point of the article it to eliminate technical debt.

If you had a car and sometimes in some corner cases, when you turn a corner at a certain speed and your car breaks down and you have to start your journey again, would you take it to your mechanic to get it fixed?

Think about it - if I'm a user, times how many users, using your product and I have to spend 10 minutes (times x number of users) to do this workaround, how much productivity are users losing over time.

So is a 45 hour investment really a waste? It could be a waste compared to doing nothing - sure - but I think you'd probably make the investment based on productivity loss and potential customer loss - not to mention having healthy software that can be continually built on at a continual pace.

The mindset is to move from a "work-around" culture to a stop-the-line culture.

re: The Waste of Defects - Bugs are Stop-the-Line Issues

Victor Volle — Wed, 24 Sep 2008 12:06:57 GMT

What if:

a "feature" has been implemented and in some corner cases (does not occur often) there is a bug. And there is a workaround for the users that takes them 10 Minutes. Investigating the cause of the bug might take 2-40 hours and fixing it might take 5 hours. Would you fix it, if there are features that are not yet implemented at all?

So I think fixing everything immediately is a waste of time. You have to do some "triage". And therefore you probably will create technical debt.

re: The Waste of Defects - Bugs are Stop-the-Line Issues

Kang Leay — Wed, 03 Sep 2008 02:27:47 GMT

it's quite good of saying "Keep it clean", the habit our team pursues.