There have been several articles written related to risk management. Three such articles have come from some of the leaders in Agile methods, and support to some degree, our views on applying risk management processes to the Agile world. Mike Cohn, from Mountain Goat Software wrote a blog article referencing a risk burn-down chart as a tool to manage risk in an Agile project. Michael Lant an Agile practitioner and blogger recently wrote an article entitled “5 simple steps to Agile Risk Management”. And thirdly, Preston Smith and Roman Pichler, two leading Agilists discuss an approach to Agile Risk Management through “Lean proactive techniques for the identification, analysis, prioritization planning and monitoring.”
In Mike’s Cohn’s blog article he indicates that explicit risk management may be unnecessary in short or low-risk projects due to Agile’s use of “short iterations, single-minded focus on working software, heavy emphasis on automated tests and frequent customer delivery.” He states “for projects of this nature that the savings from explicitly managing risks is outweighed by the cost of explicitly managing risks. “ But he does imply to some degree that as projects get larger, more complex or of higher risk, that explicitly managing risk might be important.
In Mike’s article he references John Brothers work related to the risk burn-down chart. Essentially the project team takes a risk census, which defines the probability of risk occurring at the feature/ function level. Then the risk is further quantified in terms of the size of the loss should the risk come to bear. This is expressed in terms of the number of days that would be lost. The probability and the size of loss are then multiplied to get the risk exposure in days. So if the probability of risk occurring is 25% with an estimated loss of days at 20 days then the risk exposure is 25% times 20, or 5 days of risk exposure. All feature risks exposure values are summed and plotted to give us the total risk exposure for the sprint. This would be done for each and every sprint or iteration and tracked in a burn-down chart similar to our normal story point burn-downs. We could then show a downward trend line that represented the expected reduction in risk across iterations superimposed on our risk exposure line. If the risks are not going down as expected over the iterations then decisions could be made to focus the next sprint on risk reduction activities.
This provides a sensible approach to Risk Management that can assist the development team in knowing if the risks are increasing overtime thereby allowing them to “press the reset button” so to speak and revisit those emerging risks iteration by iteration. However, it may be focusing on risks as perceived by only the development team. A broader view of risk overall is critical for larger efforts.
Michael Lant defined another approach in his blog article “5 Simple Steps to Agile Risk Management.” He starts out with the basics of Risk Management stating that:
“Risks are influencing factors that might adversely affect the outcome of a project. Risk is the direct result of uncertainty. If there is no uncertainty, it is not a risk – it is a certainty. Risk analysis is used to help a team understand uncertainty that could affect the outcome of the project. Risk management (sometimes called risk mitigation) is the plan that the team puts into place to pre-empt, contain or mitigate the effects of risk to a project.
Within simple projects things can and will go wrong and risk management must be applied in order to plan for those risks in order to minimize the impact. Lant discusses 5 steps (6 if you include repeating the first five steps). The first of these steps is to identify the risks on 2 dimensions. The first dimension is those risks that are helpful and harmful. The helpful ones are the factors that advance a project’s objectives and the harmful ones are those that hinder the results of the project.
The second dimension has to do with whether the risk is caused by internal influences or external influences. Lant recommends constructing a SWOT analysis (Strengths, Weaknesses, Opportunities and Threats) in 4 quadrants. Risks that are helpful and internal are ones that we consider strengths, whereby those that are harmful and of external origin are considered threats. He then classifies these risks as those that could affect the outcome of the solution, the budget, privacy, security, resources or scope.
Then, in a similar way that we assessed or quantified risk in the burn down chart discussed in Mike Cohn’s article he provides a 1 to 5 rating on the impact of the risk (5 being extreme and 1 being minimal) and also on a scale of 1 to 5 the probability that the risk will occur (5 being very likely to occur and 1 being unlikely to occur).
From there he produces a matrix, which identifies target values of probability to impact values. Those that are of low probability (rated as 1) and low impact (rated as 1) are given a score of 1. Those that have a high impact (a 5 rating) and high probability of occurring (also 5) receive a score of 25 (5 rating on impact times 5 rating on probability). He then takes each risk and their ratings and creates a “Risk Register.” Those receiving the highest values are things that require urgent action. Conversely those with the lowest values should be reviewed but with no immediate action required. Then each of the risks that require attention based on their ratings has a defined set of risk mitigation activities defined by the team. These would then be added to the product backlog in included in a subsequent sprint.
Preston Smith and Roman Pichler have also defined effective risk management as involving risk identification, assessment of the severity of the risk, prioritizing the risks based on severity, creating mitigation plans and continuously monitoring these risks across each iteration. In all of these approaches there is the process of identifying, assessing and building mitigation processes that are integrated into each sprints backlog and continually monitoring these risks throughout the life of the project.
We have found that the approaches discussed thus far all have great merit in the management of risks throughout Agile projects. In much the same way that was described by the experts above, we have found that continuously measuring uncertainty, value and level of effort is essential to a strong risk management program. Assessing risks at the feature level provides us with a good picture of where we need to focus our risk mitigation efforts and allows us to more adequately prioritize our product and sprint backlogs. It also allows us to determine which features and functions have the highest value with perhaps the highest risk. This leads us to making sure we take care of the high-risk, high-value elements first. Normally these may be architectural in nature or proof of concepts that must be ironed out first to ensure that our software products have a strong foundation. It also leads us to identify low-value, high-risk components that could be removed (or certainly placed at the bottom of our development activities) that would otherwise require extensive resources for very little value to the target software product. In a more traditional paradigm the low-value, high-risk features would take up valuable resources and provide no real value, adding to the risk and cost of the solution.
The Sources of Risk: A Broader View
In the discussion above we were defining risk, uncertainty, value and level of effort in somewhat quantifiable terms. We were trying to boil our perceptions of risk, value and effort down to “hard data” if you will; a number that we would track and manage based on the development teams assessment of just perhaps a handful of variables. But risks can surface from all dimensions of a project. Some of these risks can be reduced to “hard data” and yet others are harder to spot. We have compiled a short list of sources of risk that may be very difficult to track and manage on an ongoing basis, but could have significant impact on the outcome of a project. Below is a list of potential risks that also must also be tracked and managed.
Project Management related Risks:
- Lack of product vision and evolution
- Little to no standard estimation and prioritization process
- Institutionalized “learning” (learning from the past)
- Little or no planning implemented for ALL project activities
- Failure to institutionalize tracking and reporting to allow visibility and transparency of all project aspects
- No well-defined escalation pathways
- No established acceptance criteria for deliverables
Request Management Process related risks such:
- Lack of a defined process to accept requests/changes
- No process for reviewing requests and preliminary requirements.
- No established process for points of clarification, feedback on the request and recommended path forward.
Scope management related risks:
- Lack of entrance/exit criteria
- Limited or no walkthrough of the design considerations
- No process for collaboration regarding detailed design specification
- No focus on clearly articulating assumptions and responsibilities
Quality management related risks:
- Lack of building testing early in the cycle (test cases in requirements)
- No ongoing Project Quality Audits (PQAs)
- Lack of frequent inspections of applications as they are being developed (user demonstrations)
- No defined quality measures (defects, rework, etc.)
- Lack of rigorous performance and load testing against requirements
- No traceability matrices
- Lack of a proper quality acceptance process.
Customer satisfaction related risks:
- No “formal” customer satisfaction survey program
- No “formal” feedback loop for business sponsors, stakeholders, end-users, IT group.
Yes, we have key measure that we all understand, like velocity, burndown and deployable software and quality measures, but as this short-list indicates, there are softer measures such as communications, transparency, team cohesiveness/moral and stakeholder perception that could threaten our projects. We needed a way to maintain a much broader view of risk within our projects; a way to consider areas of risk that were harder to quantify, capture and/or measure throughout the life of our projects.
So we have been embarking on an effort to try and understand not just the “hard data” as it relates to emerging risks, but also the softer data. We began to ask ourselves “What if we could continuously ask the right questions throughout the Agile process and combine the answers with the hard data in a comprehensive dashboard to assess where risks are emerging without being paralyzing our development efforts?”
In a follow-up article I will describe some of the ways we are trying to merge both the “hard data” and “softer data” to give a more complete picture of risk throughout the life of our Agile projects by using CAI’s APO solution.
Pete
