Chapter 4 Knowledge Discovery

4.1 Managing the unknowns

  • A large part of project management is managing the unknowns. As the definition of a project implies unique and temporary work, it also implies that many aspects related to the outcome of the project and the work required to produce the outcome is unknown at the start.
  • When talking about unknowns, one can distinguish between “known unknowns” and “unknown unknowns”. Dealing with “known unknowns” is the field or risk management.
  • Once one is aware that each project contains “unknown unknowns”, it becomes clear that a process is needed to reveal these “unknown unknowns” and turn them either into things we know or things of which we now know that we don’t know. We call this process “knowledge discovery”.
  • Knowledge discovery is an active process that aims to acquire new and additional knowledge with respect to the project, either through experience or education.

4.2 Challenged versus unchallenged demand

  • The difficulty of discovering “unknown unknowns” and acquiring new knowledge lies in the obvious fact that we are unaware of these unknowns. Therefore, we need some structured approach for knowledge discovery.
  • It is important to be aware that the risk of “unknown unkonwns” already exists at the onset of a project. Typically, when a project is initiatied, the sponsor (internal or external) comes with a specific demand. However, there is a risk that this demand is not what is really needed in order to create value.
    • Therefore, one could challenge the original demand from the start, to make sure the demand is not based on false assumptions (as long as we don’t know whether the underlying assumptions are true or false, there is the potential for unknown unknowns).
    • At the same time, many project managers will tell you from experience that what a customer/sponsor asks is often not what they need.
    • However, they will also be able to tell your that at the start of a project, you as a project manager will lack the necessary context and background knowledge to make such judgement calls (whether the requested project outcome is also the required project outcome).
  • When you assume that the requested project outcome is also the required project outcome, all that remains is project risk - i.e. delivering according to requirements, on time and within budget - which requires proper risk management.
  • When you want to challenge whether the requested project outcome is also the required project outcome, you are actually challenging whether the project delivers value - i.e. desirability, feasibility and viability. Challenging the assumption that the project will produce value is an important focus during knowledge discovery.
  • Note that in practice you will not always challenge the demand. There are many occasions where you simply accept the requested outcome as such and focus on delivering according to requirements. External factors and experience typically guide a project manager to make such call.

4.3 Knowledge Discovery and project outcome value

  • Product risk relates to the risk that while the outcome of a project meets the predefined requirements and is delivered on time and within budget, it still fails to deliver the expected value.
  • The value of a project deliverable is defined by three dimensions: desirability, viability and feasibility. For each of these three dimensions a different approach exists to uncover hidden assumptions and discover “unknown unknowns”.
    • Desirability is related to the question whether the project deliverable meets an actual need of the client. Each project addresses a ‘friction’ that exists between the current AS-IS situation and a more desirable TO-BE situation. The more the project outcome helps the transition towards the TO-BE situation, the more desirable it is. To challenge the desirability of a project, one tries to discover the tension between the AS-IS and TO-BE situation.
    • Viability relates to whether the project outcome can deliver value in a sustainable and user-friendly way, such that it will be actually used by the end-user. A typical approach to challenge viability is by means of designing prototypes of the project outcome and testing them.
    • Feasibility relates to whether the project outcome can actually be built. Therefore, the typical approach to discover hidden assumptions in relation to feasability is by means of building a Minimum Viable Product.
  • While the discussion above might suggest a predefined order between the three dimensions of value, this is often not the case in reality. Typically, these three dimensions are tested simultaneously by means of iterative and overlapping approaches.
  • While knowledge discovery has a very important role during project initiation, it is certainly not limited to the initial project lifecycle stage. Particularly in agile project methods (as we will discuss later), the discovery of unknown unknowns is a continuous effort.

4.4 Knowledge Discovery is a process of incremental effort

  • During project initiation, the request of a project sponsor is tyically a fragmented story (with important information often missing) and at the same time contains (elements of) a solution that is based on hidden and untested assumptions.
  • At the same time, as the project manager, you often lack the necessary background knowledge and context to identify these issues, which makes it difficult to efficiently discover the “unknown unkowns”.
  • As identifying “unknown unknowns” becomes increasingly easier as the knowledge about a project increases, the knowledge discovery proces should follow an approach of incremental effort:
    • First, one should focus on collecting as much as information as possible. This information can still be fragmented, ambiguous and conflicting at this stage, which is fine. This first “Capture” step requires the least effort.
    • Next, one can move towards the “Synthesis” step, which implies that one creates a coherent and shared understanding of the problem at hand. By “shared” we refer to the fact that the understanding of the client and the project team should be aligned. Therefore, this step should be a story of co-creation with the client and already requires more effort.
    • Thirdly, once we have reached a shared understanding of the situation, it is time to analyze the context in search for assumptions and to test these assumptions against reality to see if they hold true. If not, one has spotted an “unknown unknown” which either will be solved by changing the project outcome or becomes a “known unknown” that can be dealt with by traditional risk management.

4.5 Common pitfals to be aware of during the discovery process

  • During the discovery process, one has to be aware of three common pitfalls: ambiguity, invisible gorillas and ugly babies.
  • Ambiguity refers to the fact that two people can observe the same facts, but see a different story. This is particularly problematic when your interpretation of the project context is not aligned with that of the client. This does not imply that you should assimilate the interpretation of the client, but rather that discovery should be a social activity where you interact as much as possible with your client to develop a shared understanding. At the same time, you should always welcome such conflicts between interpretation. Often we need conflicting ideas and interpretations of reality to actually learn and make progress.
  • Invisible gorillas are a metafor for assumptions that we make about reality in order to efficiently reason about it. These assumptions are so ingrained in our thinking that we are often not aware of them although they are present in full sight. The key take away is to develop a scientific attitude where you question everything and make your assumptions explicit.
  • Ugly babies is a a metafor for the fact that during the project initiation stage and during the discovery process, the initial ideas don’t seem to be of high quality and often contain many hidden assumptions. However, the key take away is to not quit an idea too early and to keep incrementally improving. Like many ugly babies, over time these projects could deliver something of high value.

4.6 Two types of learning

  • A central theme in Knowledge Discovery is ‘learning’. However, we should distinguish two types of learning that can and will occur during knowledge discovery and of which one should be aware: intuitive learning and rational learning.
  • Intuitive learning typically occurs unconsciously and is a fast way of learning based on experiencing reality. Four phases (OODA) can be distinguished in the intuitive learning cycle.
    • It all starts by observing certain events which unfold themselves. These observations are the raw information on which decisions and actions will be based. Be aware that because of how our brain works, we filter our observations. Many events will unfold themselves without being noticed or observed by an individual. At the same time they can be absolutely obvious to other individuals (cfr. the invisible gorillas).
    • Next, we move towards the Orient phase. This implies that we will put the observation in a context that is mentally constructed. Because of genetic heritage, cultural traditions and past experience, the same observation will be interpreted differently by different individuals (they orient themselves differently).
    • The third phase is where one decides what to do based on the observed event and the interpretation given to it in the previous phase.
    • Finally, the fourth phase (Act) refers to the actual action one takes in response to the observed event, based on the decision made in the previous phase. Obviously, this action will trigger new events which will result in a new iteration of intuitive learning.
  • Rational learning is a very concious process which is much more deliberate and slower. This is the type of learning that corresponds to the well-known PDCA cycle in management.
    • During the Plan phase, the objectives of the learning are established as well as the processes are designed to deliver the desired results.
    • Next, the Do phase allows the plan from the previous step to be executed. Small changes are usually tested against a baseline and data is gathered to evaluate how effective these changes were.
    • During the Check phase, the data and results from the previous phase are evaluated against the expected outcomes. Based on this evaluation against the original assumptions, one discovers new knowledge and can improve.
    • These improvements are implemented during the Adjust phase, which is typically followed by a new iteration of the plan phase, resulting in continuous improvement.
  • These two types of learning are not independent from each other and can be connected to each other. Typically one starts with the OODA cycle of intuitive learning which is action oriented, fast and conservative. However, when during the orient phase, frictions arise between the observed events and the observer’s expectation, one can decide to move over to the more thoughtful, slow and explorative way of learning by developing new hypothesis and experiments in the Plan phase of the PDCA cycle. After the necessary iterations of the PDCA cycle, when knowledge has been discovered and the friction between event and expectations has been removed, one can switch back to the OODA cycle of intuitive learning and act in line with the new knowledge.
  • Being aware of these two types of learning is an essential step. Becoming mindful when to switch between these types of learning is essential to master knowledge discovery.

4.7 Methods and Techniques

  • Understanding the process of ‘knowledge discovery’ is a first step in order to unveil the ‘unknown unknowns’, but is often insufficient to actually do so succesfully. To this end, there are various methods and techniques which facilitate the action of knowledge discovery. By applying these methods, one increases the probability of discovering new knowledge.

4.7.1 Effectuation

  • To reach a goal, two broad strategies exist: causation and effectuation.
  • Causation starts from the goal in mind and tries to figure out how to reach this goal effectively and efficiently.
  • With effectuation, the goal does exist but is rather a direction than an exact destination. With effectuation as strategy, one starts from the available resources and takes a step in the direction of the end goal. After each step, one evaluates again and takes a new step forward. However, because one starts from the available resources and one re-evaluates after each step, it is not unlikely that one discovers new information that will lead to another goal than originally set forth.
  • 5 strategies to implement effectuation as a strategy are:
    • “Bird in the hand” principle: Don’t wait for the ultimate idea, but just start with what you got and what you know and try to take steps forward.
    • “Affordable loss” principle: You should only invest what you ar ewilling to lose. Minimize the risk of a project by only investing what you are willing to lose - rather than focusing on what can be achieved if the project succeeds. By taking small steps in one direction instead of working towards long-term goals with unpredictable outcomes (effectuation logic), it is possible to avoid investing time or money that you are not actually willing to lose.
    • “Crazy Quilt” principle: Entering into new partnerships can bring the project new funds and new directions. It is often more valuable to collaborate with various types of partners who are willing to commit, than to search for potential partners who might not be available or motivated. However, it is important that in this case one is open to let the project change direction as the new partners offer different and surprising perspectives.
    • “Lemonade” principle: Mistakes and surprises are inevitable and can be used to look for new opportunities.
    • “Pilot in the plane” principle: Be aware and make sure that you are always in control of your project. Create your future with the elements that you control and with the partners that you choose.

4.7.2 Impact mapping

Following comes directly from https://www.impactmapping.org/)

  • Impact mapping is a strategic planning technique that prevents organisations from getting lost while building products and delivering projects, by clearly communicating assumptions, helping teams align their activities with overall business objectives and make better roadmap decisions.
  • An impact map is a visualisation of scope and underlying assumptions, created collaboratively by senior technical and business people. It is a mind-map grown during a discussion facilitated by considering the following four aspects: * Goal: The centre of an impact map answers the most important question. Why are we doing this? This is the goal we are trying to achieve. * Actors: The first branch of an impact map provides answers to the following questions. Who can produce the desired effect? Who can obstruct it? Who are the consumers or users of our product? Who will be impacted by it? These are the actors who can influence the outcome. * Impacts: The second branch level of an impact map sets the actors in the perspective of our business goal. It answers the following questions: How should our actors’ behaviour change? How can they help us to achieve the goal? How can they obstruct or prevent us from succeeding? These are the impacts that we’re trying to create. * Deliverables: Once we have the first three questions answered, we can talk about scope. The third branch level of an impact map answers the following question: What can we do, as an organisation or a delivery team, to support the required impacts? These are the deliverables, software features and organisational activities.

4.8 Resources