in

Ative at Work

Agile software development

Ative at Work

  • Good Project, Bad Project

    The secret to writing good automated tests is to be a good software designer.

    It sounds trivial - but the sad fact is that more often than not the attempt to implement test automation falls due to poor software design where people are struggling with strong dependencies where you cannot test a single class or module without also configuring a whole environment of databases, webservices, populating whatever tables etc.

    When the novice experiences this the natural, but incorrect reaction, is to write fewer and fewer tests at a higher and higher level of integration, the limit case being manual end-to-end integration testing through the GUI.

    It is a symptom of bad software design, not automated testing being worthless.

    In a good project the test suite does not need to be changed when external data changes. In a bad project, the test suite breaks when time passes or every time someone runs the end-of-month job to add interest to the accounts, or when the ticker price changes for a stock.

    In a good project, only a few localized changes to tests and production code are needed as for a requirement change. In a bad project, there is impact analysis and lots of similar changes all over the code-base to accomplish this.

    In a good project most of the code does not know from where it gets its data - in a bad project, even a simple interest calculation business rule knows about how to look up the database and the interest rate web service.

    In a good project there is typically 1-2x as much test code as production code. In a bad project there is often 5x-25x as much production code as test code.

    In a good project most tests exercise only a single class or a few functions. In a bad project every tests activates a code-path through many parts and tiers of the application.

    In a good project most of the test code is asserting that things went as expected. In a bad project, most of the test code is setting up the environment to run the test.

    In a good project there are tests at all levels of the application. In a bad project they are mostly through the UI and often even manual.

    In a good project you can change the application without risk. In a bad project change is painful, infrequent, risky and requires executive sign-off.

    In a good project you can deploy new code to production safely many times a day. In a bad project you only do it reluctantly every monthly or a few times a year.
  • Tweetable Code - Introducing Docjure for Succintly Reading and Writing Spreadsheets

    How would you like to be able to SMS or tweet your Excel data-crunching code?

    With Docjure, it is possible:

    (->> (load-workbook "spreadsheet.xlsx")
         (select-sheet "Price List")
         (select-columns {:A :name, :B :price}))

    This reads the A and B columns from the Price List sheet in the spreadsheet.xlsx file into a list of maps where the a column has key :name and the B column has key :price.

    I believe that is pretty good for code small enough to fit in an SMS.

    Exporting your business data to spreadsheets for further analysis is just as easy:

    ;; Create a spreadsheet and save it
    (let [wb (create-workbook "Price List"
                              [["Name" "Price"]
                               ["Foo Widget" 100]
                               ["Bar Widget" 200]])
          sheet (select-sheet "Price List" wb)
          header-row (first (row-seq sheet))]
      (do
        (set-row-style! header-row (create-cell-style! wb {:background :yellow,
                                                           :font {:bold true}}))
        (save-workbook! "spreadsheet.xlsx" wb)))

    In just a few lines of code and you have exported the price list to a worksheet, complete with a yellow-background bold header line for extra style points.

    In our business applications, bridging to Excel has provided huge benefits:

    • Users love it - having their data in Excel enables them to do much more than a static report that can only be changed by a software developer.
    • Developers love it - It eliminates a lot of tedious code for generating bespoke reports as we can easily export data into an Excel report template.
    • Project managers love it - using spreadsheets provides an easy to understand data exchange format and the flexibility to change reporting features quickly late in the project.
    • Sponsors love it - the project saves a lot of time and reduces training cost by leveraging an application the users already know.

    As an inspiration, here are some of the things we have used it for:

    • Use Excel to calculate currency trading strategies - the traders calculate their currency trading strategies in Excel then import it into the trading system. They benefit from the flexibility of a powerful tool they already know and the trading system takes care of the technical details of the actual trades.
    • Exporting to Excel for bespoke analysis - would you like to check your currency trading record? Export all the information to Excel and the traders can slice-and-dice it easily with little or no technical assitance. This is much easier than setting up a reporting database for them to link to.
    • Using Excel sheets for content management to facilitate translation - translating an application to all the languages of an international enterprise was easy. Rather than using complex technical XML-files we made Excel sheets the data format for the application strings and had the subsidiaries add their translations to that. Then we put an adapter on the application to convert back and forth to XML-config files and RESX files used internally in the web and Silverlight parts of the application. The translators had a great experience, they had a familiar tool with spell checking and it reduced waste in the translation process by presenting a well-known easily accessible format.
    • Exporting to Excel for reporting - set up a spreadsheet template with a data sheet and some reports based on that, then populate it with data from your application. This allows the users to easily change or add reports with no change to the software application itself, thus dramatically reducing the turn-around time for new reports. 

    We believe this is so great that we just have to share it:

    Therefore, Docjure is free and open-source.

    It is written in Clojure for the JVM. Get it at our GitHub page here: http://github.com/ative/docjure and be sure to follow our updates on Twitter: @ativedk (http://twitter.com/ativedk)

  • How Scrum Reduces Rework

    I was talking to a group of project managers today in one of our inspiration meetings. One of the inspiring questions that came up was related to why we should use Scrum or agile methods.

    One short answer is this: if we routinely plan to spend 1/3 of our project on testing and fixing defects there is a big benefit even if Scrum only helps reduce the amount of rework.

    The next obvious question, then, is what are the most common causes of rework and how Scrum deals with these. Luckily we have some data on that, thanks to Otto Vinter (http://www.ottovinter.dk) and Søren Lauesen who wrote an article on the causes for rework (cited below).

    In the project they analyzed they found four main causes of rework:

        ⁃    38% was caused by misunderstandings.
        ⁃    29% by missing requirements.
        ⁃    24% by changed requirements
        ⁃    9% by other causes.

    Let's have a look at how Scrum addresses these.

    Scrum Reduces Misunderstandings
    The main problem is misunderstandings. This is why the "let's discuss the acceptance criteria" discussion between the Product Owner and Team before the development on a given feature starts is valuable. If we can avoid misunderstandings altogether the potential upside is a reduction in defects of 38%.

    Scrum Reduces Superfluous Requirements
    Then, the missing requirements. The just-in-time planning of Scrum and the demonstration of running tested software at the end of every iteration makes the identification of missing requirements trivial, so we will learn about these quickly. As such this does not reduce the amount of work due to missing requirements.

    However, as we only commit to one sprint at a time it helps us to focus on building only the relevant, valuable features, which means that we also get a lot of feedback on which requirements are superfluous.

    This means that we can more easily kill the half-baked ideas that we came up with in the beginning of the project when we had very little experience in the domain. Rather than building everything we thought of and then adding the missing stuff we look at the actual project and add the highest valued missing features week by week.

    Thus, it gives the potential benefit of making us able to make the project smaller than if we had to commit early. That means less work, less rework and less complexity to maintain afterwards which is a great win over the lifetime of the project.

    Scrum Makes Changing Requirements Inexpensive
    One of the cornerstones of agile development is to make change cheap and easy. The reason we have Change Control Boards and other bureaucratic measures to prevent change from happening is that it is often very expensive and therefore we want to control it.

    Agile development is all about making it inexpensive to adapt. If change was simple and did not introduce any huge risks of defects, downtime, deployment costs etc. we could happily embrace it. If we learned something new we could change the system accordingly, thus improving it and  increasing its value. That is why we cherish changes.

    Scrum tackles this head-on. Going through a full development cycle from idea to production ready code every iteration it highlights all the expensive steps in our development process. It spotlights error-prone development practices, expensive manual testing, the bureaucratic inter-departmental coordination and the costly deployment procedures. By forcing our way through the entire pipeline every cycle we force ourselves to optimize the delivery pipeline. Not doing so is simply too expensive.

    The key is to embrace the pain and use it as an impulse to fix the root causes of the problem rather than yielding to the lure of the palliative solution of resigning to doing it less frequently or managing the complexity rather than removing it.

    It may take years to improve the organization to be able to do this efficiently, but the way there is rewarding. It gets better step by step, iteration by iteration. If you want to get to somewhere else you need only clear thinking and perseverance. It is as simple as that. Just get started - and keep going.



    Read the article:

    Analyzing Requirements Bugs (2000), Vinter O., S. Lauesen - Bug Report Department in Software Testing & Quality Engineering Magazine, Vol. 2-6, Nov/Dec 2000

    For extra points, analyze your own projects and fix the causes that are most relevant to your organisation.

  • Reboot Your Management

    How would you rebuild a country after a world war? How would you reboot your management?

    This was the topic for my presentation at the Reboot conference.

    It concerned Deming's 14 principles for management, the humanistic, long-term thinking, keep learning, use the scientific method, and build from quality philosophy, that helped shape companies such as Toyota and Honda..

    Researching the background of his work, it was interesting to see how the World-War 2 Training Within Industry principles played a big role in shaping his ideas and how its focus on operating from a basis of scarcity - such as saving material, time and labour to win the war faster, plays so well in an entrepreneurial setting and in cutting through the big-company trap of just throwing more money at problems.

    The slides from the presentation are in English and I have posted them for download and comments here:

    http://www.ative.dk/om-ative/downloads.aspx



    Read more Reboot stuff:

    Web: http://www.reboot.dk
    Twitter: #reboot11

    Posted jun 26 2009, 03:05 by Martin Jul with no comments
    Filed under: ,
  • What is a Truly Lean Company? - Deming Distilled

    Deming's work is the foundation of the Toyota Production System and modern-day lean. I recently conducted a workshop his 14 points for management. One exercise was to group them to get to their essence. Here is the the result:

    • “Practise engaged leadership” - hands-on and collaborative (points 7 and 2 – institute leadership and lead the transformation)
    • “Improve constantly” (points 1, 5 and 3 – provide jobs by building quality in and constantly improving the system)
    • “Look at the organisation as a whole” (points 9, 4 – end silos and look at total cost)
    • “Develop you competencies” (6, 13 – institute learning while working)
    • “Measure the system, not the individuals” (11, 10 – remove slogans, quotas, substitute leadership)
    • “Engage the people” (14, 8 – include everyone in improving the organisation and remove fear)
    • “Develop intrinsic motivation” (12 – remove the barriers to doing great work)

    As you see, the lean principle of Respect for People is right at the core: leadership is participatory and collaborative, and the aim is to create jobs that people are happy to do.

    A more elaborate description of a company based on these principles would be something like this:


    A “Deming 14” company aims to sustainably provide great jobs by being highly competitive.

    It is based on respecting the people and driven by their intrinsic motivation and the passion for doing a good job.

    It leverages this passion to constantly improve the quality of the work by removing obstacles in the production system as they are identified. The focus is always on the end-to-end process rather than the individual steps.

    It constantly takes the time to make experiments to learn how to do things better and uses the best new knowledge to improve.

    To enable free, objective experimentation the company removes fear and focuses on the system rather than the individuals.

    As a basis for the improvements, the company actively takes the time to learn on all levels, all the time in the context of the actual work.

    The success of the company is built on engaging everyone in the work, and leadership is hands-on with an understanding of how everyone’s knowledge and contribution is needed to succeed.

     

    Think about that for a second. How does your company compare?

     

    PS: If you are interested in learning more about the 14 points and their application in IT there is a new edition of my handbook "Ud af krisen" available on our website as a free PDF-book in Danish. Get it here: http://www.ative.dk/om-ative/downloads.aspx

    Posted maj 07 2009, 03:12 by Martin Jul with no comments
    Filed under: ,
  • New Lean Reading List: Shingo Prize Winners Announced

    The recipient's of this year's Shingo Prize have been announced.

    The Research and Professional Publication Prize category is a great reading list for new lean books:

    Off to the bookstore!
  • Improve your IT-organisation with Deming's 14 points

    I have written an introduction to Deming's 14 points and how they can help you improve your IT-organisation. It is a 15-page PDF in Danish. You can download it here:

    Ud af krisen - http://www.ative.dk/media/989/ud%20af%20krisen%20-%20ative%20-%20a4.pdf

    Enjoy - and please send me feedback on how to improve it. :-)

    All the best,

    Martin

    Posted dec 12 2008, 11:20 by Martin Jul with 4 comment(s)
    Filed under: ,
  • Out of the Crisis - Deming's 14 Points

    In a time where so many are excusing their failure with externalities and saying "it's the crisis" we need leadership and entrepreneurship more than ever. We are in a time of creative destruction and it is precisely these moments that offer great opportunities to those who seek out the new ways of success.

    In the early 1980es, the West faced a similar situation. We were blaming our failure on the Japanese manufacturers "dumping" products - selling below cost, because "no-one could create products so cheaply". Inefficient companies were threatened. Unproductive jobs were lost. The dinosaurs who would not or could not adapt lost their positions. And those who found news ways to improve productivity prospered.

    At that time Dr. Deming came to the fore. His book, "Out of the Crisis" provided a comprehensive overview of the profound system of production that he had taught the Japanese to help them recover from the Second World War. The time had come to apply it in the West.

    It has much wisdom and it is as relevant today as it was then.

    Here are his 14 points, the essence of his teachings:

    1. Create constancy of purpose toward improvement of product and service, with the aim to become competitive and stay in business, and to provide jobs.

    2. Adopt the new philosophy. We are in a new economic age. Western management must awaken to the challenge, must learn their responsibilities, and take on leadership for change.

    3. Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.

    4. End the practice of awarding business on the basis of price tag. Instead, minimize total cost. Move towards a single supplier for any one item, on a long-term relationship of loyalty and trust.

    5. Improve constantly and forever the system of production and service, to improve quality and productivity, and thus constantly decrease cost.

    6. Institute training on the job.

    7. Institute leadership. The aim of supervision should be to help people and machines and gadgets to do a better job. Supervision of management is in need of overhaul, as well as supervision of production workers.

    8. Drive out fear, so that everyone may work effectively for the company.

    9. Break down barriers between departments. People in research, design, sales, and production must work as a team, to foresee problems of production and in use that may be encountered with the product or service.

    10. Eliminate slogans, exhortations, and targets for the work force asking for zero defects and new levels of productivity. Such exhortations only create adversarial relationships, as the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force.

    11. (A). Eliminate work standards (quotas) on the factory floor. Substitute leadership. (B). Eliminate management by objective. Eliminate management by numbers, numerical goals. Substitute workmanship.

    12. (A). Remove barriers that rob the hourly worker of his right to pride of workmanship. The responsibility of supervisors must be changed from sheer numbers to quality. (B). Remove barriers that rob people in management and in engineering of their right to pride of workmanship. This means, inter alia, abolishment of the annual or merit rating and of management by objective.

    13. Institute a vigorous program of education and self-improvement.

    14. Put everyone in the company to work to accomplish the transformation. The transformation is everyone's work.

     

    Reading assignment:

    Read (re-read!) Deming's "Out of the Crisis" to get the basics.

    Read Peter Scholtes' "The Leader's Handbook" for a comprehensive guide on to how to use it.

    Then get back to work.

    Stop blaming the crisis. Now is the time to lead.

    Posted nov 05 2008, 02:57 by Martin Jul with 2 comment(s)
    Filed under: ,
  • The 14 Most Common Project Management Mistakes of and How To Avoid Them

    The Danish Computerworld magazine had a few pages on the 14 most common mistakes of project management recently. It is very illuminating to see how agile practices target these mistakes and help avoid them.


    1. The project does not have the right people with the right competencies.

    While ComputerWorld offers more analysis of skills and workload  as a solution, agile practices provide a simple solution: a cross-functional, dedicated team is the basic building block for success.

    When this is not possible use the retrospective mechanism to highlight our impediments (e.g.. overburdened key people) and continuously work on improving the situation. For example, I am working with one team that had this exact problem and they made a working agreement that the key people should always pair with the one of the others while doing the work only they can do. This provides a fast-track learning for the others and results in a situation where more team members can help on the resource-constrained tasks, thus reducing the workload on the overburdened key people.

    2. The Project Lacks an Experienced Project Manager


    The article states that a project can often run off track if there is no experienced project manager at the helm.

    You can almost feel the underlying assumption that more management is the solution for all problems. 

    Scrum has a very radical approach to this, based on a completely different assumption. Eliminate the project manager and give the team the mandate to do what it needs to make the project succeed, then get out of the way.

    It is a radical approach - but it works. The key assumption here is that the team disciplined enough to actually follow any practice, but if they are not, no amount of experienced project managers will be able to save the project from trouble. It also relates to the next common mistake:


    3. The IT-department Does Not Follow a Standard Project Management Process

    You will be be wasting a lot of effort if you don't have an explicit process that people are actually following. And "oops" - it's not enough to have a corporate procedure handbook - only practice counts. Even if people try do do heavy processes and spend a lot of money on training I frequently see that they get very little benefit, since people are still doing what they like if the process is broken. This creates a cognitive dissonance where the training material is focused on one thing - "learn the little red book by heart and be a good party member" and the real world is completely different. It is a situation that is well described in Solzhenitsyn works on the Soviet era where everybody was lying to themselves.

    The only way to sidestep the need for process is to have a small group of highly experienced professionals who will follow good practices intuitively without noticing.

    For the normal situation, lean offers a better solution:

    The take here is to have a simple process, follow it rigorously, and when you master it, continuously work to improve it. This is why we like Scrum so much - it is simple: 3 roles, 3 activities and 3 artifacts, and yet very powerful.

    In our training we frame it this way: "Everyone starts with the Scrum, but end up with exactly what they need". That is the whole point - not writing a nice handbook and doing something else, but rather doing a working agreement to work in a certain way, and continuously reflect on these agreements and revise them as needed to fit the situation.


    4. The IT-department is Paralyzed by Too Much Progress

    The key issue here is keeping to the specification to keep the project on plan (as if!) rather than adding a few new features that the customer might like.

    The agile approach here is simple. We never overplan, and we talk intensively with the customer, showing them the real application and getting real-world feedback on where the project needs to go. We even do "Customer Co-Location" and have the customer and team work in the same office to consciously boost this communication and learning cycle.

    It is value-driven software development, rather than plan-driven development, thus acknowledging the fact that the plans were done in the beginning of the project when we have very limited knowledge of the business and implementation domain.

    As an aside, this is also why I like to speak of "the plan hypothesis", rather than just the Plan since it does not lead to the confusing doublethink that the plan is more real than reality.


    5. Changes to the Project Scope are Not Tracked.

    The issue is that the budget explodes - and the timeline.

    This also reflects an underlying assumption, namely that the project must deliver the initial specification to succeed.

    Agile takes another approach. Again, we are back to value-driven development and incremental delivery. By delivering working software frequently - for example, every two weeks, there is no need to track changes to the scope too much since we don't plan more scope than a few weeks ahead. Problems solved by not getting into the bad situation in the first place, rather than putting in a Change Prevention Committee and a big bureaucracy to keep the project in stasis.


    6. Missing Up-to-Date Data on the Project Status.

    Computerworld gives a one-word advise to remedy this: "Solution: Software."

    If you chose to use software at least I would advice to be very careful about what you measure. Earned Value and other methods might look fine, but unless you have a proven track-record in your organisation about being able to use them to RELIABLY PREDICT the delivery of software I would advise to use a simpler metric:

    In lean as in agile, we focus on value as defined by the customer: Running, Tested Features working flawlessly in the production environment.

    This is a metric that is very simple to track, and hard go game. No more 80% done for many months.

    They key agile practices here are Frequent Delivery, Information Radiators like kanban boards with backlogs, burndowns to communicate the current status of the process.

    The Japanese call this Mieruka - making information visible so we can act.


    7. They Ignore the Problems

    In my book this is one of the most frequent problems I meet in large organisations. The "I know it sucks but I am not allowed to fix it", "It's the other department's fault" and all this silo-think.

    The lean approach is to map out the entire end to end value creation process and continuously optimize the value delivery flow, not just in one department, but the whole thing.

    Key agile practices here are current state analysis to uncover the flow, Retrospectives and other activities as a source for inspiration to Kaizen, the continuous rigorous experimentation with new practices and measuring how they work, one small change at a time so as not to introduce to much variation into the system that might slow down our evaluation of the new practice.

    Scrum does this very explicitly by tracking these "Impediments" as on a day-to-day basis combined with a working agreement that the ScrumMaster gives to priority to ensuring that they are removed promptly.


    8. They Don't Spend Enough Time to Define The Project Scope

    The issue here is that if the project scope is not well defined it will explode.

    Agile is very disciplined about this. We take our production capacity as a given and actively manage the scope every sprint. There is even a role for this. The hardest job in Scrum is that of the Product Owner who has to manage all the new ideas and trade-offs every day. It is difficult and is based on the principles of maximizing the value created by constrained set of resources similar to capitalistic entrepreneurship rather than the plan-economy approach of fixed scope as suggested by Computerworld. Come to think of it, next time someone suggests fixed scope on a large timeframe look puzzled and say, "oh, you mean production quotas as they tried in the Soviet Union?"


    9. The Dependencies Between Projects are not Visible.

    This is a frequent killer in large organisations and the bane of many a SOA implementation. Layers and layers of code from various projects, "The Enterprise Service Bus project", "The Mainframe Project", "The Workflow Project" that all need to coordinate to deliver the end-to-end features.

    Computerworld cites an expert who advise to map out the dependencies to better manage them.

    Agile advises to eliminate them altogether.

    The keys to success here are cross-functional teams that have all necessary competencies to deliver the whole functionality, and, if that is not possible (and remember - "thinking you can't" is one of the biggest wastes), to have all teams integration continuously and track the impediments on a day-to-day basis.

    Rather than managing dependencies we aim for just-in-time synchronized delivery by all teams. The key here is solid organisationwide product management, as autonomous, independent teams as possible, frequent delivery, integration and continuous automated testing to make it all spin. It is hard but the alternative is not very attractive, so start going in that direction.

    The secret to success is that of Bounded Context - don't try to standardise everything on a corporate-wide basis but let teams do what they need to succeed inside clearly bounded contexts and only spend your effort on managing a limited number of integration points where these Bounded Context need to communicate.


    10. They Don't Take Murphy's Law Into Account

    The issue here is to mess with the plan meet a deadline set by an external party, like the CEO. Computerworld offers to talk to the CEO about what it takes in terms of time, resources and money to meet the deadline.

    Agile calls this Customer Collaboration or engaging the stakeholders in the work to profit maximise the portfolio. The key here is to be honest and transparent and measuring only delivered features, not milestones. Otherwise the Portfolio management will be just like the planning meetings in the Führer bunker where the generals were talking about all their imaginary shadow armies and their victories to please the Führer rather than confronting reality.

    I think every project manager should make a decision to never lie, speak against better knowledge or make wildly speculative statements about the future - in fact, the project status reports that I have encountered over the years and their "green-shifting" of bad news to good news as it filters up the chain of command have often indicated that the companies would be better off abolishing these report altogether and measuring status only in terms of what has been delivered without defects into production.

    I think the former Avis car rental CEO, Robert Townsend, put it well when he refused to do earnings forecasts saying that, "we don't do in public what we can't do consistently well."


    11. They Omit Transformation Management

    This mistake is related to build "the next big thing" and forgetting to manage the transformation to make it work in the daily life: simply building nice new software that is not used.

    There are various aspects of this, and it is at the heart of the agile practices. Value is measured in customer terms so it is a waste to produce things that are not needed, or to produce software that is needed and not provide the necessary support and training to make it useful in the daily work. This change management process and the resistance to change should not be omitted.

    Some of the agile practices in that area are Customer Collaboration (on-site customers to make sure that we build what is needed), Frequent Feedback by delivering software to be used in the daily work and getting feedback about what needs to be improved and how we can focus the effort to make it even more valuable - in short - the big Lightbulb Idea is to simple have an ongoing conversation about the project with the people who will benefit from it.


    12. Incomplete Project Plans

    The symptom of this is that the project members don't know what should be finished when and thus have a hard time meeting these secret expectations.

    This touches one of the areas where agile is really a hardline process. Planning-wise it is much more like situational planning in the war room than the big Politbureau meetings and their production quotas and five-year plans.

    We work a lot with management to make this happen and it is very, very difficult to do well.

    First, we advise a simple visual roadmap for the portfolio to put on the wall - just the big themes or strategic priorities on a timeline (don't add the details until the last responsible moment). In Scrum terms this is the "long-term" Product Backlog.

    Then, for the next 2-3 sprints we have a Product Backlog consisting of a prioritized list of features to be done. They are explicitly managed according to trade-offs such as between the value they represent, the cost of doing it.

    Then, for each sprint the team picks the top priority items and commits to delivering them at the end of the sprint. Along with this we have not only a short description of the feature to be done, but also two other things: Conversation and Confirmation.

    Conversation means talking to the Product Owner to really understand how the feature should work. Confirmation is talking about the acceptance criteria, putting up a target and making it crystal clear for everybody how we will verify that we have completed the job.

    The last thing is that the Team commits to what they can finish. We don't try to whip them into doing more. We say, "here is the most valuable things we can work on, let's estimate it and make good trade-offs to maximize the value created by your work, and then you, the people who will do the work, pick as much of the most valuable work that you can do without cutting corners."

    It is simple, direct and takes a lot of discipline. It could also be seen as the Stephen Covey "Begin with the end in mind"-habit for software development.



    13. The IT-department Does Not Reject Unrealistic Deadlines

    The symptom is that the IT department has a reputation for not delivering on time.

    The agile approach as outlined above is to have the team commit to what they can do, based on evidence of what they did last time. It is empirical and if we are honest and keep a consistent definition of done, delivering good, tested software at a constant high quality every time, our production capacity will soon be quite stable and predictable and something that can be measured rather than gamed. It is the end of overdue projects since we basically only commit to very small projects at a time.

    On a bigger scale - say a 6-12 month project timeline, agile shifts the focus back to entrepreneurial thinking and saying, "OK Product Owner, we can deliver at this rate, how do you want to prioritize the effort." The Product Owner then, continuously evaluates the project business case and as long as the marginal utility of new features is better than stopping the project and starting work on something else, the project continues. It agile business, not just agile software development. And it is very very hard to do well since so many organisations are built on smoke and mirrors.


    14. Poor Communication with Sponsors And Stakeholders.

    Back to the basics again - "Customer collaboration", "Visible Status", "Frequent Deliveries", "Inspect and Adapt"... if you miss out on this point it is one of the best places to get started. Even if your organisation is not ready for "this agile stuff" you can try some ideas as suggested by Lee Henson:

    "Instead of that weekly hour-long status meeting, how about meeting for 10 minutes every day to talk about the project"?
    "How about keeping a visible list of the highest priority features to be built next?"
    "Would you mind looking at the system every couple of weeks as we build and and giving us some feedback on how we need to adjust the priorities for what to do next?"
    "Since we have already build the most valuable features first, shouldn't we put it into production and get a faster ROI? Maybe it is already "good enough" and we can finish early, under budget?"


    I wish everyone great success in overcoming these mistakes and showing our profession better ways to work. The need is clear - and it is up to us to make it happen.

     

  • How to Improve Your Development Process Using Factory Physics

    There are only three types of buffers: extra capacity, extra inventory and extra time. This is one of the great take-aways from Factory Physics that I have frequently quoted since Thomas Blomseth introduced me to it on our lean study-tour to Japan earlier this year.

    It is an essential piece in approaching process optimization.

    Most processes have (and this is not a bad thing) critical steps that govern the throughput of the whole process.

    For example, consider a development organisation that wants to improve its output.  Assuming that the development process consists of the definition of work (the work of the Product Owner), the implementation of this (the work of the development team), the acceptance of the work (the PO again) and the user testing (the test team).

    In our case, let's assume that we have an overburdened product owner who does not have enough time to specify the work properly and is hard pressed to check the work done by the development team, finding many misunderstandings after the work has been done, leading to much rework. We also assume that the PO is actually involved in doing a lot of testing - something that is often the case, but many times also a sign of poor acceptance criteria in the requirements.

    In our case, the work is governed by the PO - in the work definition role and in the acceptance role. Until this bottleneck is removed, we can quickly conclude that there is no point in hiring extra developers since they would just be sitting idle. The PO capacity is the bottleneck.

    Let's try the exploring the issue from the perspective of the  three buffers.

    First, we consider adding extra capacity for the PO role.

    There are several approaches to think of - the first being "eliminate waste". We look at the PO's work and see if he is spending time on non-critical or non-value-adding work. This can either be eliminated or delegated - in fact, it is often a good idea to delegate to someone who are not so good at it if they have spare capacity. This includes training others to help extend the capacity of the PO-role.

    The next thing is to look for extra inventory. To use this approach we would build up inventory of PO work in front of the development team, for example by giving the PO a head start on the project to build up a huge backlog so the team cannot catch up. Call this an analysis phase and we get to waterfall development. The downside here is that by doing this we practically eliminate the learning about the project that occurs in the feedback loop between work definition and implementation so it is probably not a good idea either.

    This also applies to having the PO focus on the definition only and building up inventory before the "to be acceptance tested by the PO" stage, effectively building a huge inventory of work-in-progress of an unknown quality - something that is generally regarded as a very risky endeavour.

    An alternative approach could be to eliminate the queue in front of PO acceptance could be to keep the work flowing and the cycle-time in check could explored by testing the hypothesis that we could off-load the queue by timeboxing on the PO testing and automatically promoting features to system testing after, say, one week in the "Waiting for PO acceptance" state, or by having the PO do prioritised testing of the key features inside this timebox or even uniting the PO and user testing into a single team. The hypothesis we should test here is that if there is extra capacity in the user testing stage it can be used to substitute for PO capacity even if it mean more rework or more work for the testers, using some of their spare capacity. By applying an end-to-end process this could turn out to be an improvement of the total system.

    The last approach is to look for the extra time angle. The approach here is to lower the capacity of subsequent steps to near that of the PO throughput (usually with enough extra capacity to absorb the variation in the PO step so the system is kept stable). In our case this could be moving people from the development team do useful work on other projects or let them use their spare capacity to improve their development process - cleaning up legacy code, improving the build process, learning new skills etc. By trading for the buffer of extra time we slow down the delivery of the project to the gating factor (PO capacity), but waste less developer time in the meanwhile. In some cases this is a very good approach as a smaller team has less overhead for communication etc.

    Turning back to the upstream steps we must also examine the PO role with respect to constraining the definition of work. We can address this in many ways, but the best way is to ask the team and the people involved to see what is really going on in the "Gemba" where the value-creating work is being done.

    In some cases it could be that the PO is so stressed by the amount of work that badly defined work is being accepted by the team which in turn increases the burden on the PO to check and suggest changes to the work produced in the "PO acceptance" stage - something that impairs the POs total capacity by a vicious cycle on the constrained resource.

    In this case we can explore approaches such as involving the team or the downstream testers more in the work definition stages to improve the requirements and make PO acceptance a lesser burden by delivering the right product.

    By starting with Factory Physics, systematically exploring the problem from the angles of extra capacity, extra inventory and extra time gives us a simple, yet powerful tool to ask good questions and analyze the process. As such it is a great starting point for the team to generate ideas to try out to improve the process - and that's the way to creating better software faster.
  • Learning TPS from Toyota in Nagoya

    One of the high points of my recent Japan excursion was visiting the Toyota factory in Nagoya. We did a factory tour and meet with Keiichi Fukunaga, the manager of Toyota's department for teaching the Toyota Production System to other companies.

    One of the things we discussed was the tricks speed up the transformation to TPS.

    We were not allowed to take pictures in the factory, so here is Martin in the Toyota F1 in their showroom. 


    Keiichi-san's experience is that top management and front-line workers are usually quite enthusiastic about the transformation. For the front line, he noted, all you need is to go into their context and make some changes that are helpful to them. It is not just about efficiency but also about caring for the human side of the equation. He stressed the point that TPS should never be used as an excuse to fire people (the latter is what is called LAME – Lean As Misguidedly Executed on the www.LeanBlog.org).

    For top management it should be easy to understand the TPS agenda – after all it’s translates to a better competitive position.

    For middle management TPS can sometimes be a bit of a challenge. I think it is related to the fact that a lot of middle management consists of managing “waste” efficiently. Eliminating waste also reduces the importance of many of the bureaucrats. At this point he mentioned that it has taken Toyota 71 years to get to where TPS is today, so I think he was hintingat the need for patience when tranforming organisations. That also reflects our own experience in Ative.

    In any case, he said, the key principle when doing transformation is to point people’s attention to the things that allow them to learn and improve.

    For example, he mentioned an example from working with a company that made ashtrays for cars. They used a huge oven, bigger than the size of the conference room we met in, to do this – and it took a long time. This looked like waste to him so they had a discussion about why a room-sized oven would be necessary to make such a small thing as an ashtray. Following this line of questioning and measuring the temperature and other variables of the ashtrays in process they were eventually able to arrive at an elegant solution with a small table-sized oven and a much faster process, too.

    From perspective of the ashtray company it could be seen as a big step forward – something called kaikaku – radical improvement – in some lean literature – but he really did not like this word. He said that even getting rid of a huge oven follows the principles of small step by step improvement, so even when it seems radical for the uninitiated we should call it just what it is: kaizen.

    We also talked about the role of Dr. Deming and statistical process control. He said that it is very important to know this but it should not be the only tool in the toolbox. In the example just mentioned there was no need for Deming to improve the situation – it was all a matter of seeing the waste and removing it. This common sense approach is reflected in the spirit of the Toyota credo of “Good Thinking, Good Products”.

    Martin Jul in Toyota City with the corporate logo. 

    PS: For more information about the factory tour my travel companions from BestBrains have written an article here: Lean Study Tour day #3.

  • What's Your Favourite Lean Books?

    People often ask me about recommendations for the must-read books for lean and agile software development.

    Here is a list of some of my favourite lean books:

     

    • W. Edwards Deming - the grand old man of the field, building on a strict statistical discipline. His book Out of the Crisis is a wonderful treatise on his thinking including his famous "14 Points for Management". This is definitely a must read that will change the way you think about management and quality!
    • Taiichi Ohno - one of the greatest industrial innovators of the 20th century, the father of the Toyota Production System. After spending his career relentlessly optimizing manufacturing at Toyota he wrote the book Toyota Production System: Beyond Large-scale Production that describes his work.
    • Womack & Jones - Their books are great and it is well worth to read them all to see a lot of the principles and case studies for lean thinking. Also, it is quite interesting to see that software development is now rediscovering some of the things that manufacturing learned much earlier - in the case of Toyota as early as in the 1950s and 1960s. Begin your studies with The Machine That Changed the World, a five-year study of the global auto industry from MIT and go on with the Lean Thinking and Lean Solutions. They give a fascinating perspective on manufacturing and plenty of examples of the lean principles and they applications. These are the books that brought lean to the mainstream.
    • Mary and Tom Poppendieck - with a background in manufacturing and software they have translated the concepts of lean to software development. They have written two great books and Implementing Lean Software Development and Lean Software Development - an Agile Toolkit. Both books are well worth reading a present a both the principles and lot of cases in a friendly, colloquial manner. Highly recommended!
    • Matthew May - I really like his approach to elegance and simplicity. May has worked with Toyota and their corporate university and his book The Elegant Solution offers insight into their innovation process - the principles it is built on and the practices that make it work.
    • Jeffrey Liker - his The Toyota Way is a very good introduction to the application of lean methods at Toyota. This is one of the best lean books I have read. Definitely a favourite!

     

    What is missing? What are your favourites? Don't hesitate - post a comment now :-)

    Posted feb 11 2008, 01:39 by Martin Jul with 2 comment(s)
    Filed under: ,
  • Beyond The Valley Of Tears

    Sabine Canditt of Siemens showed an interesting curve for Scrum implementation in her excellent ScrumGathering presentation in London recently. It shows how introducing the Scrum initially creates chaos and then stabilizes with a quiz success and good motivation. This is when the team gets into the rhythm and starts eliminating the waste that's directly under its control. After that we often see the velocity stabilizing and the the team's motivation plummeting as it becomes more and more clear how much of the waste cannot be removed by the team alone. This it becomes clear that the organization itself is the main impediment. When the team realizes this and has no mandate to remedy the situation we end up in The Valley Of Tears.

    At this point, some organizations will decide that Scrum does not work and give up.

    Better organization can look to Deming for inspiration on what to do.

    Leadership is the key word and it is the key success factor for going beyond this point. The reason is simple. Large organizations are built to be robust so that they can accommodate a high turn-over of average employees and still generate predictable results. They are designed like supertankers to stay the course and to prevent change. Management, then is about keeping the machine running. Leadership is providing the vision and support to rebuild it in a better way as we go along, upsetting the status quo and going against the very nature of the robust organization. This is not only a middle management issue. Leadership and the constancy of purpose to drive improvements and innovation starts in the boardroom and it builds on the concepts of continuous improvement, learning and quality work.

    It is a great challenge and our best wishes for the New Year go out to all the leaders who will make this happen in 2008.

    Happy New Year!

  • Scrum Gathering, Fall 2007 – Open space

    The Open space on the last day (Friday) of the Scrum Gathering 2007 offered many interesting sessions and discussions. Two of these sessions were: “Product Owner practices – prioritizing backlog items isn’t enough!” and “Integration Teams (from Scrum in the Enterprise) – I just don’t get it!”  

    The product owner role
    Scrum has been used for some years now and is starting to face some challenges. One of these is the Product Owner role. Many projects and companies have introduced Scrum with great success (improved performance for the dev team, progress is visible for the team and the management etc.). But after a period of time the progress is stagnating (after aprox. 3 iterations). The improvements made in the first period are related to issues internal in the team, things they can change them self. After proximally 3 iterations the team starts to face issues that are related to the rest of the organisation and the PO has a central role in this interaction.

    The overall conclusion was that Scrum and the current PO role don’t solve these problems. There is a big gab between the PO and the senior management, sales, marketing etc. This area needs more focus and we must find some solutions in order to insure that Scrum continues as a successful framework.

    To share our challenges and experiences we decided to create a public malinglist. Information will be published when the list is up and running.

    Integration Teams – “I just don’t get it!”
    Henrik Kniberg opened the session with the question: “does anyone have a example that shows how the Integration team works?”.

    There were many god suggestions: 

    • The integration team is responsible for infrastructure etc. to make it possible for the underlying teams to test the solution. This is close to Kens defition in “The Enterprise and Scrum”. No one had any expirence with this approach and we didn’t find any explanation on why you need a hierarchy of integration teams to support your dev teams. For me it starts to smell like silos!
    • The integration team is a virtual team of team members from the underlying dev teams that is responsible for the integration. No one had tried this and I don’t think that it will work.
    • The integration team is only working with the Backlog, breaking down the epics into smaller chunks and defining the user stories. In other words: a team that helps the PO doing his job. This is only suitable in large organisations. The question is wetter this will enforce silos and information handover?
    • If the project contains hardware it is not possible to do Continues Integration. Therefore you might need an integration team (the silo way).

    Many god ideas but no real life example that shows how this is supposed to work.
    It would have been nice if Ken had staid for the open space and joined this discussion!

    I need to look deeper into Kens book, because I still doesn’t get it.

  • Variation And Waste - The Case for Frequent Delivery

    Imagine you are in an organisation with a one-year release cycle and you miss your annual deadline by one minute. The cost? A one-year delay.

    Now, try to invent a control system to ensure that we avoid this kind of delay.

    You would probably add some middle management to manage the dependencies between the various parts that need to be integrated for the release. You would need more management capacity for expediting work when interdependencies juggle the plans and you would need to spend time on replanning.

    You would do detailed analysis to identify risks and try to mitigate the risk, no matter how little the probability that the event would occur. You need more people to sit and worry about possible future events.

    Then, you would probably also make sure that leading up to the deadline nobody was on vacation, and you might order a code-freeze for the last months to avoid adding risk late in the project.

    You see where we are headed. Step by step, we are building the standard, enterprise IT-organisation.

    Since the cost of missing the deadline is so high, we have created a big bureaucracy and complicated our everyday work to deliver on time. Productivity is sacrificed to reduce the variation.

    The elegant solution is different. What if we tried to make missing the deadline a no-stress event, where a little variation in the development process would not cause a full-year delay? What if we designed our system to be robust with limited variation so that small things have small impact?

    This is the lean approach. We try to drive down the variation and create a simple management system rather than accepting it and building a complex system on top to tame it.

    Imagine we reduced the release-cycle to one week, shipping roughly two percent of our annual work at a time. The variation is smaller. Miss a deadline by a minute and you release the following week. The cost of slipping is much smaller. There are no more critical weeks or month-long code freezes. You need less bureacracy. There is less replanning. People can go on vacation when they want.

    Releasing becomes a no-stress event, something we do every week. It becomes just another day at the office.

    On top we get some business benefits. Since we ship every week we can deliver incremental value faster. The cash-flow position is better since we get an earlier return. We become more responsive. If the market suddenly fancies red buttons rather than grey buttons we can ship that next week. Our annual-release competitors will have to wait until their next release - or even more if we make the change during their code-freeze.

    The net result is an organisation that is sound from a business perspective, flexible, simple to control, less stressful and more fun to work for.

    That's why we call reducing variation an elegant solution.

    Posted okt 24 2007, 09:23 by Martin Jul with no comments
    Filed under: , ,
More Posts Next page »
© Ative Consulting ApS