Agile Project Management
Agility
- Agility is the capability to efficiently and effectively adapt to an ever changing environment.
- Agility can be defined at different levels: personal, departmental or organizational.
- Note that it is not possible to implement agility directly. Agility is a property of a system that is present to some varying extent, depending on how the system operates.
- One can identify three drivers which will cause a system to become more agile: flow, learning and collaboration.
- Flow refers to the way work is processed by the system. If the processing of work occurs at a steady and sustainable rate, the system has a high level of flow. Such state of high flow is often accompanied by a feeling that work becomes easy and routine takes over. Some people refer to this state as “being in the zone”.
- Learning refers to mechanisms in the system which allow the system to learn from past experiences, mistakes or other people, but also to uncover unknown unknowns (knowledge discovery).
- Collaboration refers to various ways how people in the system can work together to achieve a single goal. Different forms of collaboration can be distinguished:
- Helping. One person decides to help another person finish his or her job. This way of collaboration is useful when you have nothing else to do. In this approach, the person who helps takes the initiative to collaborate. Most likely this person will not be as productive as the person he or she is helping due to lack of experience in the task at hand.
- Apprenticing. A specialist involves another person in his or her job, so this person can help but also becomes more experienced. In this approach, the expert takes the initiative for collaboration. This approach is useful to prevent specialists to become bottlenecks in the process.
- Synchronizing. This is a form of pre-emptive collaboration. The idea is to never start a task that one cannot complete without the help of another person unless we are certain that this other person will be available. Synchronizing implies that one synchronizes with the key resources to make sure they will be available when necessary.
- Swarming. When faced with a very critical or highly uncertain task, swarming could be a useful collaboration approach. The idea is for the team to get together and discuss the task in order to decide on a plan of action and then spread out for a limited amount of time (e.g. 30 minutes) to each work on a part of the task individually. Next, they come back together to discuss the progress and synchronize with each other, before they scatter again to continue working. They repeat this process until the work is done.
- Integrating. Often a task will require different people to do parts of it. If the task is moving a lot from one person to another person, the integrating approach would imply to work together on it. The idea behind this collaboration technique is that moving work from one person to another implies a transaction cost (waiting time because both people need to be available, time required to bring the other person up to speed, errors due to miscommunication, …). By integrating the work, these transaction costs can be eliminated.
- Pairing & Mobbing. Pairing and mobbing are two techniques where either two people (pairing) or an entire team (mobbing) work together on a single task. While one person takes the lead (e.g. controls the keyboard), the rest observes, asks questions, makes suggestions, … . Often the role of leading the collaboration is passed around from team member to team member at a fixed time interval.
Impact of agility drivers on work efficiency
How work works
- While agility refers to the capabilities of a system to adapt to a changing environment, it is important to realize that the drivers behind agility also influence the work efficiency of a system.
- Efficiency of work can be expressed in two different ways:
- Resource Utilization. This is the perspective often used by management and refers to the percentage of available time that a resource is is actually working. A resource utilization rate of 0.8 implies that a resource is only working 80 percent of the time that he/she is available. Sometimes this is also called chargeability, which refers to the percentage of working time (or the corresponding cost) of a resource that can be charged to the customer.
- Flow Efficiency. This is the reciprocal of the average lead time. Thus as the lead time (time between arrival of a job and completion of the job) goes up, the flow efficiency goes down. The more waiting time occurs during the execution of a job, the higher the lead time and thus the lower the flow efficiency. This perspective is often taken by the customer and is strongly connected to the concept of ‘flow’.
- To understand how work works, we must be aware of four reinforcing loops.
- Loop 1: Whenever we start work, we create work in progress. A certain percentage of this work in progress will get blocked due to unforeseen circumstances, missing knowledge or waiting for external input. When work gets blocked, it will make workers idle, which will lower resource utilization. This is typically compensated by starting new work. However, as more work gets started, the WIP will further increase, creating a reinforcing loop.
- Loop 2: As mentioned before, a certain amount of work in progress will result in blocked work. To unblock this work, we need to perform unblocking work which is non value adding. The fact that resources must spend their time on non-value adding work, decreases the time they have to finish work. This results again in a reinforcing loop. The less work is finished, the more work remains in progress, which increases the likelihood of blocked work, which increases the amount of non value added work, which furthermore decreases the amount of finished work and thus the loop continues.
- Loop 3: As the average work in progress increases, the average lead time will also increase assuming a steady-state process where the average processing rate remains constant. This follows directly from Little’s Law which states that the average lead time equals the average work in progress divided by the average processing time. As the lead time increases, the flow efficiency will go down. But at the same time, the amount of urgent work will go up. Urgent work is defined as work that must be started immediately and should get priority in order to meet the deadline. However, more urgent work thus implies more work that should get started and the work in progress will again increase, resulting in a third reinforcing loop.
- Loop 4: Increasing lead times also imply that the time between the moment a client requests a job to be done and the moment we can deliver the output to the client increases. The more time passes between these two moments, the longer we must wait for feedback from our customer. In a non-stable environment, such feedback delays increases the likelihood that the deliverable is not completely as the customer desires, resulting in necessary non value adding work. As known from loop 2, non value adding work decreases the rate at which we can finish work, which increases the work in progress, which will have a detrimental effect on the lead time, further increasing the feedback delay, resulting in a fourth reinforcing loop.
- Note that each of the four loops have the effect to increase the work in progress, which according to Little’s Law has a negative effect on flow efficiency. However, unless an organization considers flow efficiency important, this is not necessarily a problem. After all, we can always keep resource utilization high by starting new work. The perverse effect however is that this approach will decrease flow efficiency even further.
Agility drivers and work efficiency
- Flow as a driver implies that one keeps the rate at which work is processed steady at a sustainable rate. One common way to implement this to set a limit on the work in progress. Thus, whenever a worker becomes idle and the WIP has reached its limit, the worker can no longer start new work, thus breaking the first reinforcing loop.
- Collaboration as a driver can be used as an alternative approach to keep the resource utilization rate high whenever workers become idle due to blocked work. Instead of starting new work, they can decide to help other colleagues complete their work or to unblock blocked work items. Helping colleagues to complete their work will increase the rate at which work can be finished, which lowers the work in progress. This approach lowers the negative effect of the second loop on work in progress.
- Learning as a driver should counteract potential feedback delay. If the system incorporates sufficient points where we can learn (e.g. from the customer or environment), the delays in feedback will be smaller again, which will result in less non value adding work, which again has a positive effect on the work in progress by breaking the fourth reinforcing loop.
- Note that all three drivers ultimately have a positive effect on the work in progress, trying to break the reinforcing loops. As a consequence, this approach allows one to both increase the flow efficiency as well as the resource utilization.
Agile Project Management Methods
Origin
- Agile Project Management finds its origins mainly within the field of software development.
- Traditionally, software development methodologies followed a waterfall approach which resembles traditional project management methods in nature, with a strong focus on processes, predictability and risk control, assuming a stable environment.
- In response to such heavyweight software development methods, various lightweight methods were introduced in the 1990s, such as Scrum, Extreme Programming and Rapid Application Development.
- While such lightweight methods were introduced before the Agile Manifesto, the latter is often considered the start of the Agile movement.
- The Agile Manifesto was written in 2001 by 17 software developers who came together to discuss the lightweight development methods. Together they published the Manifesto for Agile Software Development.
- Fundamentally, agile approaches (of both project management and software development) differ from traditional approaches in the sense that they are adaptive rather than predictive and integrative rather than waterfall.
- Adaptive vs predictive. In a predictive approach the focus is on analysing and planning the future in detail, preparing for potential risks. The idea is that if one can predict the future, one can plan for it. In an adaptive approach it is assumed that the environment changes fast, often and in unpredictable ways. The focus of adaptive approaches is not to plan everything ahead of time but to leave flexibility to respond to a changing environment. Flexibility to address changes and the ability to detect these changes are key elements in an adaptive approach.
- In a waterfall approach, planning, execution, testing and delivery are clearly separated in sequential steps. In an integrative (also called agile) approach, these steps are integrated in one phase, which is typically iterated multiple times. These iterations are also incremental by nature, which means that each iteration builds upon the output of the previous iteration enhancing it further towards the end deliverable.
The Agile Manifesto
- The agile manifesto identifies four core values:
- Individuals and interactions over processes and tools. While tools and processes are important, it is even more important to have competent people working together effectively.
- Working software over comprehensive documentation. While good documentation is useful, the main point of development is to create software not documentation.
- Customer collaboration over contract negotiation. While a contract is important, it is no substitute for working closely with customers to discover what they need.
- Responding to change over following plan. While a project plan is important, it should not hinder to accommodate to changes.
- The agile manifesto also identified twelve principles:
- Customer satisfaction by early and continuous delivery of valuable software.
- Welcome changing requirements, even in late development.
- Deliver working software frequently (weeks rather than months).
- Close, daily cooperation between business people and developers.
- Projects are built around motivated individuals, who should be trusted.
- Face-to-face conversation is the best form of communication (co-location).
- Working software is the primary measure of progress.
- Sustainable development, able to maintain a constant pace.
- Continuous attention to technical excellence and good design.
- Simplicity - the art of maximizing the amount of work not done - is essential.
- Best architectures, requirements, and designs emerge from self-organizing teams.
- Regularly, the team reflects on how to become more effective, and adjusts accordingly.
- While the agile manifesto was written with software development in mind, many of its ideas can be transferred to projects in general, giving rise to idea of agile project management. Some core principles that return in agile project management are:
- Iterative, incremental and evolutionary development.
- Efficient and face-to-face communication.
- Very short feedback loops and adaptation cycles.
- Focus on quality.
Scrum
- The origin of Scrum goes back to the work of Hirotaka Takeuchi and Ikujiro Nonaka, who developed a new approach for product development based on cross-functional teams.
- The Scrum framework as agile software development approach was developed by Ken Schwaber and Jeff Sutherland.
- For a good overview of Scrum as Agile Project Methodology, we refer to The Scrum Guide
- Some key takeaways are:
- Scrum defines three key roles: a product owner, a team member and a scrum master.
- Scrum assumes that the work is done by a single self-organizing cross-functional team which contains a diverse set of skills required to perform the work.
- The role of a product owner emphasizes the need to continuously evaluate the needs of the customers.
- Scrum defines five key events: a sprint, the sprint planning, the daily scrum, a sprint review and a sprint retrospective
- The sprint represents a single iteration which is timeboxed (fixed amount of time). If one considers the work assigned to a single sprint as 1 work-item, one could state that Scrum uses a WIP-limit of 1.
- Scrum defines three main artefacts: a product backlog, a sprint backlog and an increment.
- The increment is the potentially releasable output of the sprint. Scrum assumes an incremental approach. Therefore, the increment refers to the work done in a specific sprint integrated with the work of all previous sprints. At the end of each sprint, the end deliverable has received an update which moves it closer to the end goal.
- Scrum mainly focusses on the collaboration and learning drivers of agility.
Kanban
- The Kanban method goes back to a lean manufacturing method which was inspired by the Toyata Production System. Its key idea are to visualize work items to give participants a view of progress and to allow work to be pulled rather than be pushed.
- Over time, Kanban and Kanban boards in particular found their way to software development projects.
- Today, Kanban is used in all different kind of contexts outside software development.
- For a good overview of Kanban, we refer to Essential Kanban Condensed
- Some key takeaways are:
- Kanban focusses on the visualization of work (by means of a Kanban board).
- Kanban limits work in progress (WIP). It does so by defining a commitment point and a delivery point in the process. The commitment point is where the team and the customer commit to the work being done, whereas the delivery point is where the customer commits to accept the output.
- Kanban advises to make work (and kanban) policies explicit.
- Kanban focusses on managing the flow at a steady and sustainable rate.
- Kanban implements feedback loops.
- Kanban assumes collaborative improvements.
- Kanban focusses mainly on the flow driver of agility.
- Typically, Kanban is not only applicable to projects, but also to operations. As a project method it is often combined with other method to give sufficient guidance for project management.