There is often very strong resistance to the introduction of Agile methodologies from many quarters. This resistance may be direct, or it may be indirect. Ultimately however it is related to the organization’s or individual’s perceived threat to their position, authority or compensation (i.e. power and money).
This article discusses some of the impediments that may be faced by a transition to Agile methods.
What Enterprises will successfully adopt Agile?
This question is asked a lot, but in a different format. The question that is asked is usually “What types of projects are most suitable for applying Agile practices?” Although these appear to be quite different questions they not because ultimately it’s not the type of project that determines Agile success, but rather the organizational and cultural constraints of the enterprise.
The following quotation from Tom Poppendieck nicely sums this up. I’ve added the emphasis in order to drive the point home.
“The constraints on when agile approaches can be used are primarily organizational and cultural, not project types. Some organizations and some contexts are incompatible with agile/lean thinking. When these organizations eventually come up against a strong competitive threat, they will fail to meet it unless they change their values and mindsets. Lean/Agile is at it’s foundation, the fourth industrial paradigm, the first being Craft Production, Factory Production with machine tooling, Automation and Taylorism. These come along every hundred years or so and take a few decades to work through. Each paradigm includes the preceding one and makes it dramatically more productive.
There is no need to sell agile except to organizations that want to survive long term. If they don’t see the threat/opportunity they cannot succeed with agile or lean nor can they sustain economic viability in the long run.” -Tom Poppendieck
The main traits that I look for when I work with an organization is a willingness to listen, and to change accordingly. Many organizations will insist that they listen to their employees and that they have an agile culture. Although this may be true to a certain extent, the real test always come when the Agile values conflict with the organizational values.
If the organization does a lot of up-front analysis and design, what is the response when you ask them to write User Stories? Do they insist that Use Cases are better than User Stories, or do they ask you to show them how? If the organization traditionally provides individual offices for their developers, what is the response when you suggest common team rooms? Does the organization try to justify their current status quo, or do they experiment with alternatives?
Provided that an organization is willing to listen and change then all these issues will be eventually resolved; it will only be a question of time and effort.
People, Process, Technology, Teams, Management and Culture
When initially introducing Agile practices to a team, the difficulties experienced by the team are all centered around the immediate adoption of the practices, and the consequences of that adoption. After some experimentation with Agile methods the focus becomes centered on larger problems that confront an entire team. Once experienced with Agile practices the problems are large still and will confront may teams. The scope and nature of the problems faced by an Agile team grow over time as knowledge of Agile practices spread throughout the organization in ever increasing circles of influence.
In an effort to discuss the issues faced by Agile teams, I’ve broken down these issues into the following categories: People, Process, Technology, Teams, Management and Culture. For the remainder of this article I’ll discuss the first three categories (namely People, Process and Technology). I’ll leave the remaining categories for my next (and final) article of this series.
Organizations will have people issues with or without introducing an Agile methodology. But I think it’s fair to say that the introduction of Agile methods increases the rate of change, and expose any problems that might have been previously hidden beneath the surface.
I feel that any solutions to people problems should be based around fostering good clear communication between team members, and should emphasise team work over individual heroism. The following are typical situations that are frequently seen in Agile projects:
- Overly specialized skill-sets requiring work to be handed-off several times before it can be described as complete. For example, an Analyst will describe the problem before handing-off to a DBA, who will work with the database before handing-off to a developer, who write the functionality before handing-off to a Tester. The act of handing-off partially completed work is very expensive, and it is desirable to keep this to a minimum. An approach to overcoming this is to ensure that all hand-offs [ideal there will be no hand-offs but reality often dictates otherwise] are informal and to encourage pairing at all stages. Encouraging the team to work in small chunks of work also helps in this regards as it forces the team to meet and communicate more frequently.
- Lack of ownership by the team. This may be a problem at the start of an Agile project. Team members who are use to being handed instructions often have difficulty engaging with the project. Their learnt reaction is to wait until being told. In order to work like a team and to take ownership of the problem, teams need to be able to communicate in an open fashion. This can be encouraged by letting the team work directly with the customer, by ensuring that technical conversations are not shut down and by soliciting input from different members of the team.
- Architects, Developers, Testers, etc refuse to interact with the team. It is not uncommon for some team members to refuse to participate in the Agile process. They may have many justifications for this … they have to many prior commitments, their work cannot be broken down in a fashion that’s amenable to Agile SW development, their time is too important, they work better as an individual etc. These are nothing but excuses. Regardless of the reason, the end result is that they are deliberately putting themselves outside the team. The team has to make it’s best effort to resolve these issue and to work with others who are attempting to seperate themselves. However, every member of the team needs to contribute and if one or two members of the team are putting themselves above the team then this needs to be quickly address by management, usually by the quick remove of the individual(s).
- Senior Management giving mixed signals regarding support for Agile. Nothing is more efficient at crippling an Agile project than a few carelessly chosen words from Senior Management. Senior management need to decide prior to introducing Agile methods if this is something that they are commited to. Regardless of their own personal perspective, once an organization has decided to adopt an Agile methodology senior management have an obligation to publicly support the effort. Agile teams and customer must also realise that they play an important part because unless they communicate their experiences to the wider organization, their successes will remain unacknowledge. It’s important for Agile teams to communicate frequently with senior management, to clearly articulate their successes, and any obstacles to their success.
One of the common attributes for all Agile methods is that they are very process-light. Scrum specifically is often described as more of a framework than a process. What Agile methods explicitly acknowledge is that we don’t need complex processes to build software. In fact, the opposite is true.
But for something that is conceptually simple, a lot of projects have difficulty trying to keep it simple. Here are some commons process related problems:
- No single customer (Product Owner) can be identified. This is common a problem with there is a large seperation between the business units and the software development departments, or in situations where several different groups have an equal interest in the success of a project. In my experience, most business customers really want their projects to be successful but they lack the understanding of how they can constructively interact with software projects. Methodologies such as Waterfall or RUP have reacted to negative customer involvement by trying to distance the customer from the development teams. Agile on the other hand seeks to integrate the customer, but with a better understanding of the interaction. In the situation where a single customer cannot be identified, it’s really necessary to identify the project sponsor … the single person who ultimately approves the funding of the project. By working with the sponsor, and clearly articulating the need for a single business represetative on the project team this situation can usually be quickly resolved. In the situation where several different groups have an equal interest in the success of a project, the team still needs a single representative who is willing to work with each of the different groups and prioritize accordingly. Again, working with the project sponsor will provide the path to a solution.
- Management wants to combine elements from both RUP and Agile. The end result of doing this will have the disadvantages of both methodologies without the advantages of either. There are several different approaches to doing this; RUP management process combined with XP engineering practices being the most common. The motivation for doing this is often a desire to continue to provide familiar reports without having to re-educate senior management or business partners. I personally feel that this approach under estimates senior management, and does the organization a disservice. Better results can be had by educating management and business partners while adopting a single approach whether it be RUP or Agile.
- Scrum Master refusing to protect the team. The primary function of the Scrum Master (or Iteration Manager) is simply to protect the team and to remove impediments. It takes a lot of courage to report problems to senior management and to make changes that others in the organization may view as a direct threat to their position. At the very best most SM will [at some point] have their credibility questioned. Without an effective SM, however, who will protect the team? Who will standup to those who would distract the team with unrelated activities, or to the Product Owner to say that their expectations are unreasonable? The SM role is essential to maintaining health communications between the team members and to protect against distracting external influences. If the team isn’t getting the help they need, then perhaps the firstly problem that needs to be resolved is a more effective SM.
Process practices are common to all Agile methods, but engineering practices are not. Extreme Programming (XP) is different in that it explicitly prescribes engineering practices in addition to process practices. But why would specific engineering practices by of interest and how is this different from other (Waterfall, RUP) methodologies?
The heart of the answer lies with reducing the cost of producing a usable product on a frequent and repeatable basis. Iterations are common to all Agile methods, and the outcome of those iterations is typically a usable product. It takes some considerable overhead to build, test and package a product. In order to be able to do this repeatedly (every iteration) in a cost effective manner requires a significant amount of automation, hence Agile engineering practices.
Processes which either increase the cost [of producting a usable product] per iteration, or do not actively reduce the cost per iteration should be immediately eliminated. Examples of wasteful practices are listed below:
- Not having a reliable build system and processes. The source control, build and testing of software is an intimate part of any software development team. It simply makes sense to have this process as efficient as possible. If it takes 2 hours for a developer to check in code and do a build then automating the process is going to allow more construcive use of that time. Clearly, if the team is building and releasing code many times per iteration then any effort expended to create a reliable build and testing framework is going to be well worthwhile.
- Not addressing QA issues. With more traditional development methodologies testing is usually addressed at the end of the cycle. So, it’s not surprising to see teams take up Agile and then not pay approapriate attention to defects and code quality. Code quality will quickly become a problem with repeated iterations, especially if the team is trying to build new functionality on top of deflect laden code. In addition, delaying testing until the later stages of a project will result in late discovery of unanticipated side-effects. Producing high quality code from the very onset should be the ultimate goal of every project. It is only by addressing quality issues early and proactively that the fully nature of the software will be known.
- Ineffective tools mandates by a committee or external parties. Large organizations often feel the need to control the number and types of tools that are used by project teams. This can be for a number of very good reasons such as lisencing, IP and IP rights management, and security. But when the authorization process or the tools themselves act as an impediment to the team making progress, something needs to be done. If a tool doesn’t meet the developers needs, then why should they use it? If the authorization process for introducing CruiseControl takes more than a few weeks, then what does this say about the organizations attitude towards software quality? At this point the SM and project team need to make their point of view known to senior management. They need to explain why tools that are relevant to the work that they’re doing should be supported, and that having the right tools can make the difference between a product and a high quality product.
Why all the discussion on Agile impediments?
Dealing with issues, problems and difficulties is part of everyday work life and there is nothing new in Agile methodologies which tells us how to deal with these problems. So why all the discussion on Agile impediments?
For me than answer is that Agile methods make visible those problems that were previously ignore, hidden or otherwise invisible to the organization. It is only by actively acknowledging the dysfunction that teams and organizations have that we are able to address them. And it’s only by discussing these problems that we’re able to share information, to help us understand that many of the problems we face today are shared by individuals in software organizations all over the world.
In my next article I’ll wrap up this series with a discussion on impediments that arise from teams, management and organizations.