It is impossible to solve the decision/selection problem without a comprehensive and correct Fact set. Furthermore, if the Fact set is incomplete or incorrect, wrong solutions may be determined.
Gathering the Fact set represents 75% of the decision/selection problem solution work and time, yet it is the most poorly performed. The problems with gathering Facts are:
Inadequate process – The typical methods used to gather Facts are inadequate:
Ad hoc – A subject matter expert uses their experience and begins without a formal plan. This is frequently the approach taken but quickly bogs down where even as few as 100 Facts need to be answered. Fact sources lose confidence in the process when they are asked to answer the same Fact or for the same supporting documentation multiple times. An ad hoc approach is high risk and high cost
Flat checklist – A flat checklist is a document designed for repetitive use in gathering a defined set of Facts. It is the most widely used tool, but has several major disadvantages when used to gather and organize large Fact sets. These disadvantages are:
Limited capability – A flat checklist is practically useful up to 250 Facts. When the Fact count exceeds this level, it becomes unwieldy
Extraneous Facts – As the number of Facts increases, the inter-relationship amongst Facts means that there are certain Facts that are not needed based on the combination of answers to previous Facts. These extraneous Facts cannot be readily and consistently identified and excluded in a flat checklist. Asking for all answers by default wastes time and unnecessarily increases variability and risk in the resulting Fact set
Static organization – The flat checklist is fixed and cannot be adjusted to the needs of a Fact source or project. Practically, this leads to multiple versions that soon become uncoordinated and dated, which in turn lead to missed or wrong Facts
No supporting documentation control – The answers to Facts often include supporting documentation. Since the flat checklist and supporting documentation are separate, it is difficult and inconvenient to maintain checklist to documentation synchronization, especially where (i) multiple users with different physical locations assemble or access the supporting documentation and (ii) a single supporting document applies to multiple Facts
Updates – Answers are often updated multiple times over the process of completing the Fact set. A flat checklist has no systematic way to monitor updated answers and supporting documentation.
Decision tree – A decision tree is a software tool that guides a user through a static linear path of Facts. It differs from a flat checklist in that with each answer, the user is directed to a branch dependent on that and previous answers, so there is some refinement of Facts to be determined based on previous answers. However, a decision tree is not viable for a large Fact set because (i) the order of Facts affects the result and (ii) all possible branches must be pre-defined to their end points. With even 100 Facts, the number of end points is too large to define. In addition, since the decision tree is static, a path down one branch cannot change to another branch without going backward to their common point in the tree. This makes changing any answer to a Fact more difficult and risky the farther it occurs in the process
Configurator – A configurator is a software tool designed for repetitive use to steer a user through a Fact gathering process with the objective of configuring an end result (hence the name). Their most common use is to configure a product (e.g. an automobile) and there are no practical limitations on the number of Facts that a configurator can address. Configurators eliminate extraneous Facts (they adjust future Facts presented with each current answer). The problem with a configurator is that it is forward uni-directional – when a valid endpoint is reached, it is not possible to look backward and determine (i) why a particular Fact did or did not have to be answered and (ii) the relationship a particular Fact has to other Facts. In real life where judgment based solutions are employed, such as in the selection/decision problem, both of these capabilities are critical to a correct and comprehensive Fact set and to being able to understand and explain why any particular Fact set is or is not valid.
Finally – none of the above methods of assembling Facts have any intelligence – they have no way to anticipate and suggest answers based on learned knowledge and relationships. Hence the amount of work and risk in obtaining the 1,000th Fact is the same (perhaps even greater due to the possibility of inconsistency) as that required to obtain the first Fact, and the knowledge gained from one project is not dynamically connected and available to future projects. This is counterintuitive and counterproductive – experience gained should make it easier to answer Facts for both the current and future projects.
Lack of knowledge at Fact source– The Fact source often has little or limited knowledge of the Facts required. It can be equally difficult and risky in obtaining answers from a Fact source who believes they know exactly what is required
Lack of Fact source time – There are normally many individuals who contribute to gathering large Fact sets, and these individuals through position or workload often have limited time to provide answers. The difficulty of obtaining required help increases each time an individual is re-approached to clarify or expand on a Fact because the Fact was not fully or properly answered the first time
Lack of required data– The Fact source has little concept of the number of Facts required to solve their problem. Accordingly, they become frustrated as they perceive they are being approached continuously and apparently randomly over a period of time seeking to answer additional (or previously answered) Facts. Often, the likely answers have not yet been fully developed, are not readily available or must be derived or reformatted, increasing the Fact source’s frustration.