Edge of Chaos | Agile Development Blog

The concept of deadline has always been one of the sacred cows for any human activity.  Normally, with very few exceptions, what do people do with any project? They set a deadline and think that they will care about meeting this deadline somehow. You can see lots of examples for that in construction industry. Remember, China and the 2008 Olympics. They had a deadline for building Bird’s Nest arena in quite a short period of time. In this case, it was really necessary to meet the deadline since the Olympics was to start on August 8, 2008.

How about software development projects? Product development? With agile methodology?

I think there are 2 options here: agile team does have external pressure for deadline, and the team is free to choose when to release. The main criteria for release is quality as observed in this tweet.

When reading blog posts and articles on lean and kanban (this one for example), I noticed that people are very wary of calling things their names and avoid saying it out loud - DEADLINES ARE CRAP. What matters is the work flow, the quality, the limits of WIP and the Definition of Done. So, if a team has no external pressure, the recommendation is to avoid extra stress by imposing unnecessary deadlines.

There’s no rush for perfection, as they say. But how about external pressure? How about clients who bluntly say — I want this job done yesterday? Frankly, as I worked in outsourcing, I always felt an urge to say to such clients — if you want to have this done yesterday, you should have moved your a** yesterday and select a vendor who you trust yesterday. With custom on-demand projects, clients often fail to understand that meeting a deadline and completing a project on time is the responsibility of both customer and vendor, with customer even more responsible.

So, if you’re with an agile software shop and you have such clients, there are 3 options:

1. educate your clients on deadline concept as above and say bye to them
2. bend over backwards, have stress, and deliver something-not-sure-what on a client’s “yesterday” deadline
3. patiently explain the client that only a part of what they want can be done by their deadline and prioritize work.

The sad thing is that the majority of clients will go elsewhere hoping to “get the job done yesterday”. But if you really are competent, do good work, your reputation will start building up and at the end of it all you’ll get a bunch of devout customers - especially if you work in a local community.

Lean and Kanban software development adoption is growing. More and more companies setup Kanban Boards, limit WIP and eliminate Muda.

This collection of links will help you understand all that buzz around Lean/Kanban and decide whether it is worth trying. I’ve read all the articles and posts below, so this list is a truly selected thing ;).

Articles and Blog Posts

• Lean Software Development. Wikipedia summary about lean software development. It is a good start to digg into the topic (as usual).
• Kanban Development Oversimplified. Most likely the best article to start with Kanban. Very clear, very detailed. Good work!
• Kanban, Flow and Cadence. This blog post with many nice pictures describes three important properties of Lean: Kanban – Controlled Work, Flow – Effective Work, Cadence – Reliable Work.
• Scrum-ban. Interesting attempt to mix Scrum and Kanban, taking the best from both worlds. Kanban with iterations is possible.
• Beyond Scrum: Lean and Kanban for Game Developers. Article describes real Lean/Kanban implementation for game development industry. The section on how to improve The Flow (3 strategies: Time-boxing, Levelling workflow, Reduce waste) is especially good.
• Adventures In Lean. Series of posts about Lean approach with focus on real problems solving (handling bugs and emergency fixes in Kanban, setup pipeline, bottlenecks, etc.).
• Lean and Kanban. Several posts on the topics in this blog.

Presentations

• Kanban for Software Engineering . Detailed presentation about Lean and Kanban. Focuses on many problems and solutions in a software development process.
• Kanban: A Process Tool . Another good presentation that emphasizes the simplest rule in kanban: Limit the number of things in work to a fixed number. Maybe a good start indeed.
• Practical Experiences and Tools Applied to a Kanban Sustaining Engineering System . Fantastic presentation about Lean/Kanban implementation with real examples by Alisson Vale.
• Kanban vs. Scrum. Rich and lengthy presentation that compares Scrum and Kanban. A very good visualization (which is a key in Kanban :). Shows why Kanban fits better for multiple development teams working on the same project. I tend to share this opinion as well.
• A Kanban System for Sustaining Engineering on Software Systems . Very interesting presentation of Lean implementation at Corbis. As I know, it is one of the first successful attempts to run Kanban in a software shop. Must read.

Lean/Kanban Blogs

• Agile Management Blog. Lots of interesting posts from David J. Anderson (well known engine of Lean software development
• Richard Durnall Blog. Pull and Push systems, interviews, lean roots and principles. Nice reading with hand-drawn diagrams.
• Lean Software Engineering. Corey Ladas and Bernie Thompson are blogging about Lean, Scrumban and Kanban, Theory of Constraints, software development and other topics you did not even hear about.
• AvailAgility. Karl Scotland’s posts are very interesting (and helpful) to read. Isn’t Kanban just a Task-board? Check the blog to get an answer.
• The Agile Executive. Many insights into Kanban and summaries from the first lean conference.
• Software Just in Time. Lean concepts and real lean applications posts by Alisson Vale.

Lean/Kanban People in Twitter

• David J. Anderson. Lean/Kanban software development pioneer.
• Corey Ladas. Product development methodologist. Author of Scrum-ban book.
• Henrik Kniberg. Optimize, debug & refactor IT companies. Author of Scrum vs. Kanban presentation (which is very good!)
• Karl Scotland. Agile Coach. He runs AvailAgility blog with great insights into Lean and Kanban.
• Rob Lally. Renaissance Technologist.
• Alisson Vale. Alisson implemented outstanding Kanban process in his company.

Tools

There are just several Kanban tools on the market. To be honest, I don’t like TRICHORD UI. LeanKit: Kanban looks much better, but it can work for small teams only on my opinion. Anyway, it seems Kanban tools vendors’ race just began.

If you know other tools that support Kanban, drop a comment and I’ll happily include them into the list.

• LeanKit: Kanban. In beta so far, but looks quite neat. Maybe useful for small teams.
• TRICHORD. Desktop project management application with Kanban boards.
• Radtrack. Registration does not work, but I found the screenshot via Google. Looks like LeanKit so far.
• Zen. Looks like the most promising tool among other three. Nice features description.

Did I miss something interesting? Drop a comment!

Fact: Complexity causes 50% of product returns

Fast and easy-to-use applications are quite rare. Simplicity and Performance are two major properties of any killer software product. We, as software developers, should pay attention to these properties as non-functional requirements, but in real life we often tend to implement more features, more functions, more settings, more, more, more… The race is hard to win with this strategy along the whole way.

I must confess we’ve done almost the same to TargetProcess. We went on with providing more and more features and options. Definitely we tried to keep the application simple and fast, but these goals were secondary. We’ve stopped. And changed.

Now we are focusing on better performance and usability. We think that TargetProcess is a quite feature-rich application that fulfills most needs in agile project management. It is time to stop and find answers to the questions: “Where do people get stuck with our software?”, “What is complex and how it can be simplified?”, “How to make TargetProcess enjoyable to use?”, “How to make TargetProcess the most performant software in our niche?”. The questions are hard to answer and address quickly, but we are looking for the answers.

Steven C. Seow wrote an excellent book about principles that should be taken into consideration for any “performant” application Designing and Engineering Time: The Psychology of Time Perception in Software. I read it and want to share some interesting observations.

Hick–Hyman Law describes the time it takes for a person to make a decision as a result of the possible choices he or she has. Simply speaking, less functions — simpler and faster choice. It leads to several conclusions what we should do as software developers:

• Minimize options. Obviously, if you have 50 elements on the screen, it takes time to choose which action is required. If you have 20, the UI is faster to work with.
• Keep it simple. Well, it is a general principle for all the facets of agile software development.
• Short system messages (10 words max). People don’t have time to read long messages. Delays kill usability.

Some concrete numbers from the book:

• Response time of typical action in the application should be about 2 seconds.
• If response time takes more than 5 seconds, it is required to show a progress indicator. User should know that system is working on the task.
• If system response time is more than 7 seconds, people tend to leave web site or switch to another task. It breaks interaction flow.

Noticeable Performance Improvement

If you are going to improve performance, it should be faster by more than 20%. Otherwise most people will not see the difference (it was shown in some researches that performance difference is noticeable in a range between 10% and 18%). For example, if you are improving search function with 10 seconds response time, it is required to make response time at least 8 seconds (or less).

Flow

Flow is very important for good user experience.

Flow is the mental state of operation in which the person is fully immersed in what he or she is doing by a feeling of energized focus, full involvement, and success in the process of the activity.

Our goal is to make the Flow possible. I think it is the hardest thing in software development.

When user opens an application and sees complex UI, it is frustrating for him. Application should match user experience and skills. Simple or Advantage modes, clean and simple UI (Hick–Hyman Law!), balanced options and functionality — this sounds familiar, but hard to develop.

• Goals for using Kanban David Anderson put a nice list of Kanban usage goals. “Goal 1. Improved performance through process improvements introduced with minimal resistance”
• How we do MVC and Our “Opinions” on the ASP.NET MVC . Very good posts about ASP.NET MVC challenges, tricks and solutions. Must read if you are starting serious project based on ASP.NET MVC.
• Tim Sporcic blog. I like this blog a lot. Tim has a talent to express his thoughts and most posts are very interesting and helpful. If you are using ExtJs, ajax and mvc pattern — just add the feed to your feed reader.
• jQuery vs. MooTools. Extensive comparison of two popular JavaScript frameworks. If you evaluating choices, don’t miss this reading.

• In Defense of Eye Candy. Nice article. It discusses why aesthetics is important in a design: “when we talk about how emotions influence interactions, it’s closer to the truth to say things that are enjoyable will be easy to use and efficient.” And another fantastic quote “how we ‘think’ cannot be separated from how we ‘feel’“.
• Discovery-driven Growth: The Only Plan Is to Learn as You Go. Quite lengthy, but interesting interview about business development in a current economy. Discovery-driven growth is a way to go.
• Kanban and Time-boxes. Does Kanban compatible with iterations and time-boxes?
• Wanted/Needed: UX Design for Collaboration 2.0. Designing collaboration software has some specifics. Can we create a framework that help with it?

It’s good to get questions from people who work with TargetProcess. Indeed, questions and feature requests are like a plancton for this product ocean. We thrive on them, and we take the power to move forward from our clients’ feedbacks. Some of the questions are worth to be blogged on, to initiate a discussion and get the right direction.

So, one of these questions is about Gantt charts, whether we’re going to implement them in TargetProcess or not. Notably, for the most part this question is asked just this way — are you going to implement Gantt charts? People forget to explain what are they looking for in Gantt charts and why do they need them whereas in reality it makes a huge difference if they want Gantt charts for dependencies on tasks level or if they want Gantt charts for Program-level planning, to get a “bird’s eye view” on what’s going on with all the projects, as they put it.

For those who want Gantt charts on tasks level, to manage dependencies and to nurture a gigantic critical path for the sake of nurturing it - the answer is always “no”. There can be no Gantt charts for tasks and dependencies since agile is not about tasks dependency and critical path management — it’s about flexibility and “temporary dependency”, if you want to call it this way. The most you can get about dependency on User Stories and Tasks Level in TargetProcess is indication of related User Stories with a custom field. We’re going to implement this in the next release.

I will not be going into much detail with the reasons for this “no”, re-writing multiple articles on this topic, Google is always right at your fingertips But I’ll just give you a link to a very comprehensive post.

The only way for Gantt charts to stay alive in TargetProcess is in high-level planning of Program-Projects, that’s what you might call a “what’s- going-on-in-the-forest” view. We already have these Gantt charts in plans. That’s how they will look:

Originally, TargetProcess has been more focused on managing just one project. But as we get feedback from people, we see the need to implement more elaborate features for managing multiple projects on Program-Project level — those Gantt charts provide a quick scan look to all the projects.

Follow our roadmap for the Gantt charts. Your feedback is really important to us, so we encourage you to register at our Help Desk Portal and support forum and vote for Program level Gantt charts here.

• Why Small Companies Will Win in This Economy. “Small is the new big. Sustainable is the new growth. Trust is the new competitive advantage.”
• So, Now You’re an Agilist… What’s Next? Interesting presentation from Jurgen Appelo with attempt to bring complexity science into agile software development. And interesting debates thread between Jurgen, me and Glen Alleman.
• Feature Crews: kanban systems for software engineering in the large. We are using branch-by-story at TargetProcess and something similar to feature crew. I think the idea is very good.
• Getting Things Done Mind Map Diagram. GTD is quite interesting technique, I am trying it right now (and liking it so far). This mind map is really good quick intro to GTD.
• Things — task management software. Great GTD software for Mac users. Very clear concept with perfect implementation.

I wrote previously about agile certification. I don’t like the idea. However, some new arguments arose recently.

For example. Peter Stevens wrote: “In a time when every office worker gets told ‘A Microsoft Office certification is good for your career,’ it is clear that certification is part of the game”.

It is not the case for companies applied agile software development. For example, at TargetProcess we do not pay almost any attention to official certificates. They all sucks. I personally interviewed many people having several Microsoft Certificates, but many of them were bad developers. They were coders, and that is something we are fighting against. It appeared that certification has nothing in common with developers skills. We hired several people with no certificates at all, and we hired several with certificates. There is no any relations between good developer and certificates they have.

Agile implementation changes company culture. Without this shift agile adoption will fail in the long term.

I do believe that there are many Certified Scrum Masters that did not get the Scrum process right. People are different. Someone may attend the course and work hard to improve his knowledge. Someone may attend the course and take SCM title with honor, thinking that he knows everything to implement Scrum in his company.

On my opinion that is one of the reasons why Scrum adoption fails in companies.

How To Certify Meta Process?

Software development process is a complex thing, that should be adopted to each company, environment, etc. Some best practices may not work in some conditions. It means we can’t include practices into certification test at all. What we can include is things that shape the process, things that focus on development process improvements, like retrospective meetings, communication empowerment, self-organization, emergency. But can you imagine how the questions sound like?

• Have you something in place that enables process evaluation and improvements?
• Do you have practices that power communication?
• Do you have practices that enable self-organization?

Guess what answers can we get on such general questions: “Yes, our top managers meets every year and discuss development process improvements”, “Yes, we have Outlook, it is great for communication!”, “Yes, team located in one room, so it is easy to shout out and assign particular task”. The result is obvious though.

What Type of Agile Certification May Work?

Agile certification may be developed in the future, but for the very strict set of conditions. For example, there may be quite good set of questions for the team under the following conditions:

• Team size: 4-6
• Project type: Typical web site
• Project complexity: Average
• Distributed Team: No
• Region: Europe
• Technology: Ruby

If someone creates questions based on the criteria above, that may work. All other attempts to certify teams using General tests will miserably fail in the long term.

I am really glad that finally we released v.2.14. Check the press release: TargetProcess Announced v.2.14 Release With Full Waterfall Support.

Many complex systems are based on simple rules. A set of several simple rules leads to complex, intelligent behavior. While a set of complex rules often leads to a dumb and primitive behavior. There are many examples.

Ants Colony

How ants search for food? They do not have cell phones, cars and mini-markets near the nest. They should have something simpler to communicate.

Here is how ants work:

1. Travel randomly  in search for food.
2. Take a piece of food and head straight back to the nest. On the way back to the nest lay down an odor trail.
3. Notify nestmates of the discovered food encouraging them to leave the nest. These newly recruited ants will follow the odor trail directly to the food source. In their turn, each ant will reinforce the odor trail until the food is gone.

Sounds simple? Take a look at this very nice ants colony model. Drop some food and enjoy the action.

Birds Flocks

Birds flocks are beautiful. You may think that the movement gets orchestrated by one savvy bird. But this is not the case. A bird flock is guided by three simple principles (every decent bird knows them):

1. Separation: steer to avoid stumbling upon local flockmates.
2. Alignment: steer towards the average heading of local flockmates.
3. Cohesion: steer to move towards the average position of local flockmates.

Simple? Yes, it is. Look at the picture on the right. It’s just amazing!

Game of Life

Game of Life was invented in 1970 by John Conway. It is a cellular automaton and simulates the birth, death, etc., of organisms based on certain rules:

1. Each cell with one or no neighbors dies, as if of loneliness.
2. Each cell with four or more neighbors dies, as if of overpopulation.
3. Each cell with two or three neighbors survives.
4. Each empty cell with three neighbors becomes populated.

Simple rules. But these rules lead to fantastic diversity of the forms. Different types of the forms have been discovered e.g.  still objects, oscillators, gliders, spaceships, etc. It is impossible to predict the state of a system after several thousands steps. Cellular automaton has properties of a complex system such as emergency and butterfly effect. Small changes in the stable structure (for example, adding one more living cell) may cause death of the whole cells population.

Moreover, it is possible to build a computer based on Life! It is possible to implement logic based on stable structures and execute simple calculations. The Game of Life is a Turing machine! How can someone suggest that such a simple system based on four rules has so much power?

Take a break and have fun with Life game simulation.

Simple Rules and Software Development

Is there any relation between simple rules and software development? Sure, there is. Software development process should be simple. Process complexity leads to mechanistic and dumb behavior.

1. Information exchange and collaboration vs. standard status reporting meetings.
2. Learning vs. stagnation.
3. Emergent behavior and creativity vs. Following rules, standards, instructions.

Agile methodologies are simple. Scrum is a very simple thing. It has just several rules, roles and artifacts. Well, it is a lot harder to really implement Scrum. It is hard because you need to break stereotypes and habits. Many people are resistant and don’t want to try new things. However, Scrum works. It works because of its simplicity, it lives in accordance with complex systems.

Scrum stimulates learning, feedback, communication and cooperation. Emergency is possible in Scrum.

Here is one sample of unnecessary complexity — too many hierarchy levels:

A potential problem, unlikely in small and medium organisations, is deep organisational structures. According to Peters and Waterman (1982)18, both Toyota and Roman Catholic Church have only five layers of management in comparison to Ford’s seventeen.

Do you by any chance happen to have a software development process description in a huge 100+ pages document? Are you still in business?

• Part 1. Agile Software Development and Complex Adaptive Systems: Intro
• Part 2. Software Development is Complex Adaptive System. No Doubt
• Part 3. Edge of Chaos and Hyper Productive Software Development Teams
• Part 4. Simple Rules, Complex Systems and Software Development