Studies have clearly shown that when decision makers are evaluating information to make a choice, they introduce biases into their rationale for approving or rejecting the idea or project. Many industries have instituted methods and tools to remove decision bias and focus on producing data that measures actual capabilities rather than perceived ones.
Confirmation Bias: Occurs when decision makers seek out evidence that confirms their previously held beliefs, while discounting or diminishing the impact of evidence in support of differing conclusions.
Anchoring Bias: The over-reliance on a single piece of information or experience to make subsequent judgments. This limits one’s ability to accurately interpret new, potentially relevant information.
Overconfidence Bias: Occurs when a person overestimates the reliability of their judgments. This can include the certainty one feels in their own ability, performance, level of control, or chance of success.
Halo Affect Bias: This is an observer’s overall impression of a person, company, brand, or product, and it influences the observer’s feelings and thoughts about that entity’s overall character or properties.
The Credit Score
A credit score is a statistic forecast of individual credit risk, and substitutes human judgement in deciding who to lend money to, because it is more accurate and faster.
The SAT test is an un-biased measurement of college readiness and forecasts future academic success.
Personality/Cognitive Profiles (Myers-Briggs, DISC, etc.)
Measure behavioral styles and provides potential employers data to forecast an individual’s suitability for a specific role.
Private independent rating services provide un-biased scoring of bonds that indicate the issuer’s strength to pay back the bond and indicate the risk of investing.
These execution capability measurements are filters that remove bias and measure an individual’s or institution’s ability to execute. Current processes for approving IT projects do not apply any bias-removing filters for execution capability in the decision-making process. As a result, many projects are approved too early to move forward, which are then impacted by vulnerabilities which could have been corrected before the project life-cycle was started.
The report is part of the overall decision making process to identify where vulnerabilities exist and how they are being corrected before project initiation. This provides the ability to address critical issues on their own, without expending project dollars.
The model is intended to work as indicated below:
Should You analysis addresses the value of the initiative to the organization, whether it adds value to the business or program, and what the financial impacts are forecasted to be. The assessment of this material continues to be the same in the decision process.
Can You focuses on execution capability. Here, the ability of the organization to execute on the idea or project is measured, and vulnerabilities are corrected. The purpose is to remove risks and obstacles before they affect project execution, and before the project life-cycle is engaged and costs are incurred.
The correction of vulnerabilities before the engagement of a project is much less than if they are corrected after a project has been initiated. Studies have indicated that many of the root-cause factors for bad project outcomes are not IT-related, and that the project management practices are only reactive once the project has been engaged.
When a government agency or corporate business unit take on a new initiative/project, the outcome will in some way transform current people, processes, and technologies (PPT). Some elements of PPT that meet business needs today will no longer work in the next phase of the business model, which is being introduced through the project being implemented.
How far or aggressive the change/transformation is determines how many of the current practices will remain, while new PPT are necessary to work in the future model after the project is complete.
This gap in capability factor is what decision makers most often overlook through their decision biases. The assumption is that the elements in current capabilities have the skills, processes, and technical know-how to meet the future need. This incorrect assessment leads to overlooking vulnerabilities that will undermine the ability of the transformation project to succeed.
An independent global analysis of projects with low outcomes indicated the following ranked factors of impact:
Of the 13 factors identified, 10 of them can be corrected before any project is approved. Factors such as wrong project management, poor deliverables, and poor communication are most commonly only identified while a project is in execution, and therefore cannot be addressed as a vulnerability before initiation.
The remaining factors are the basis for execution capability analysis prior to approving and engaging a project. Through the development of domains of measurement that are directly correlated to the most common factors (of which there are many), the gap in capability and associated vulnerabilities can be calculated.
Execution capability is measured and represented as an un-biased score (rather than an opinion based assessment), as with similar execution capability metrics identified earlier in this position paper. Initiatives are scored independently. A scoring of overall execution capability of an institution is insufficient, as conditions vary from initiative to initiative. An overall assessment of an agency’s execution capability will not identify accurately the specific vulnerabilities that exist for a specific initiative.
The following factors are recommended for measuring execution capability for technology resource projects:
Compiled through structured methods of data collection and scoring (i.e. SAT, personality profiles, bond ratings and credit scores), execution capability for each specific initiative reflects a moment in time for the specific project and the specific conditions that will impact its execution.
The Origins of Execution Capability
Execution capability is a sub-category of decision bias, which is a principal factor measured in behavioral economic theory. Behavioral economics, along with the related sub-field behavioral finance, analyzes the effects of psychological, social, cognitive, and emotional factors on the economic decisions of individuals and institutions, and the consequences for market prices, returns, and resource allocation.
Behavioral economics is primarily concerned with the bounds of rationality of economic agents and functions as a method of economic analysis that applies psychological insights into human behavior to explain economic decision-making and the impacts of decision bias.
Decision bias – also known as cognitive bias – are tendencies to think in certain ways that can lead to systematic deviations from a standard of rationality or good judgment, and are often studied in behavioral economics.
Although the reality of these biases is confirmed by replicable research, there are often controversies about how to classify these biases. Some are effects of information-processing rules (i.e., mental shortcuts), called heuristics, that the brain uses to produce decisions or judgments.