Why decision analysis? How to properly handle situations that are of uncertain nature? What are the tools used by business people to analyze the uncertainty, so that one can better make decision? I am trying to brought up several practical elements in regards to these questions.
Why decision analysis?
Making decision can be a very simple intuition process. However, it becomes much more complicated when the consequence of decision is really big, or, when the decision need to withstand critical eyes of many people.
Let’s assume that you, a company owner, just learned about a promising oil reservoir lying under the seafloor of Gulf of Mexico. However, the process of petroleum development involves hundreds of millions to billions of upfront investment before you get the first drop of oil (that is your return of investment). You are not so sure whether the oil recovered in the future can give you enough return of investment and make the company keep running. What should you do with this opportunity?
That is the real challenge that all oil companies need to face to everyday. Nevertheless, there are many other similar occasions for other policy makers and decision makers. In general, big risks, big influence and exposure to people’s critics lead to the emergence of a practice so-called decision analysis.
Elements of decision making — with an example on oil field development
Decision making on personal level usually is simply some mental moments that happen so fast without realizing the detailed process. However, we still can sort out three aspects underlying the process of decision making: knowledge — what do you know about the situation, goal — your objectives that have been properly prioritized, and options — among which you need to make a choice (explore more options is also an option). If the decision involves participation of other people, we also need an explicit justification of the decision — you need to communicate your decision making process to other people to have a better choice, or defend an existing choice. This can be as simple as a response to questions such as “why do you go to this restaurant?”, simply because “I prefer this restaurant because I like spicy food.”
Making business decision in big organizations involves similar factors. For example, suppose an oil company is concerning about purchasing an oil field. First is to have the knowledge, there will be a team working on acquiring the data of many different aspects: Geology — what is the environment of deposition, where are faults are, etc. Geoscience — what is the quality of seismic data, who did the interpretation, etc. Engineering — how much oil is lying underground, how many wells are drilled, to what extend has this reservoir being exploited, what kind of ground facility we have, etc. Financial — what is the break-even point, what is the investment return, etc. Second, the company need to be more specific about one’s goal: this is usually implicitly assumed, however, it is also beneficial to think how can one gain from this opportunity, what is the long term vision, how much risk one can take, what is one’s special expertise that can be utilized, etc. Third, potential options much be proposed: sign the contract or not, build platform with large handling capacity or just lease a small size one, drill appraisal wells or directly target the sweet spot, develop the asset or farm out the asset, etc. Finally, decision making should be open to relevant stakeholders, one should justify the choice and acknowledge the uncertain issues.
Uncertainty about objective situations
Uncertainty is the subjective state of not knowing the future. In many practical areas, the future can be steadily shaped by the human effort. For example, the owner of a restaurant can make strategical moves to make the business better.
However, in certain areas, the uncertainties are simply there, not depending on how we practice. For example, for a given oil reservoir, the original-oil-in-place (OOIP) doesn’t depend on how good your drilling techniques are. Moreover, the likelihood of having a major earthquake doesn’t depend on the effort of insurance company. So, this type of uncertainty is quite objective. It is in this part the decision analysis is really needed. More example situations for this type of objective uncertainty:
- The insurance company asks whether or not there will be a terrorist attack in next five years.
- The agriculture department asks whether the drought will continue so that we are endangered by insufficient crop yield.
- Local government asks whether the ground water level will keep going down so that thousands of high-rise buildings will have structure integrity issues.
- Policy makers wonder how much the carbon tax should be to effectively help the industry becoming more ecological.
Example of subsurface uncertainty in oil field development
As I am engaging in the topic of “decision making under uncertainty” recently, I finally realized why people bother to run hundreds or thousands of simulations, with a crazy number of parameter combinations (I think the bad impression came from some unreasonable parameter choices mentioned in some academic papers).
As shown in the book “Modeling Uncertainty in the Earth Sciences”, the uncertainty analysis process has the following basic picture: many (usually hundreds) reservoir simulations are run with different reservoir models, one tries to update the reservoir models by comparing the simulation prediction data to the actual production data measured from the field. Then, one make prediction of the future using the updated collection of models.
Here, the big issue behind all of this practice is: “Decision making under uncertainty“.
Use probabilistic model to quantify causal links with uncertainty – with example of Bayesian network
Probabilistic model is very different from deterministic mathematical models. Deterministic models such as reservoir simulation are used to simulate the physical process with deterministic rules. As I mentioned in an earlier post that, people use petroleum reservoir model to simulate the production behavior given the geometric property, reservoir property, fluid property, and development parameters. The reasoning is based on mass balance and Darcy’s law.
However, in probabilistic models, we are modeling causal process with uncertainties instead of definite values. Each possible states are associated with a probability value. Here, I want to mention a particularly interesting probabilistic model: Bayesian network. I learned this from a great online course “Probabilistic Graphical Models“. Daphne Koller, the course instructor from Stanford University, illustrated the idea of Bayesian network using an example of determining the cost of car insurance.
In this network model, the cost depends on the year of the car, and the situation of accident (severe/ mild/ none). The accident level in turn depends on the vehicle year, size and driver quality. The vehicle year influences whether there is an ABS system, and whether there is an airbag. These factor in turn influence the accident level. In addition, the driver quality is connected to several other attributes, through the relationship of “influence” or “influenced by”. For every causal link in the network, we specify a conditional joint probability distribution, e.g. a severe accident on an old car has 50% chance of high cost outcome, a mild accident on an new car has 20% chance of high cost outcome. For each root node of the causal network, we specify a prior probability, such as: there is a 60% chance that a car is made after 2000. In this way, the joint probability distribution of all attributes can be fully determined by the network. For example, we can calculate conditional probability of the statement “this car is made before 2000” given that “it is a low cost situation”. Or, we can calculate any conditional probability given any level of information.
It is amazing to see that the internal structure of the network is “alive”: once we have changed our state of knowledge, this information will be transmitted along different causal paths! For example, if we know a person is a good student, then there is an increased odds that he/she “focused”; at the same time, good student also increases the odds that he/she is of “young age”. These two attributes competing with each other, and the net result is a slight increase in accident.
The beauty of probabilistic graphical models lies in the parallelism between the mathematical model and the logical reasoning used by everyone. Moreover, we can mathematically show that the information actually “flows” from one attribute to another inside the network. In this way, the whole reasoning process becomes a quantitative model that everyone can grasp. I think this types of methods will be a very popular approach in future decision making practice.
Robust decision-making: scenario-based decision analysis
Alternatively, one can use many concrete samples (or scenarios) instead of probability values to represent the uncertainty state. It is more naturally for human mind to imagine some concrete scenario rather than some probability values. That leads to scenario-based decision analysis. In the practice of robust decision making under “deep uncertainty” (deep uncertainty simply means that even experts cannot agree with the situation), many scenarios would be constructed and being treated as a “possible future”. This step is called “exploratory modeling” or “scenario generation“.
Sometimes, it involves generating a full combination of uncertain parameters. For example, to study subsurface uncertainty, one may assign each parameter high/ mid/ low values, e.g. reservoir porosity at 8%, 10%, and 12%, or water-oil contact at 3400 ft, 3425 ft, and 3450 ft. This approach is similar to the concept of “design of experiment”. The other approach involves trying hard to think of different possible futures that can not be captured by simple parametric variation. For example, a reservoir modeler may noticed that a dimmed area on the seismic plot may represent a poor reservoir situation but previous assessment ignored that. Therefore, decisions should be made with the consideration of this newly found scenario.
Scenarios are specific world states. So there are many different ways to describe the scenarios. For example, some people tend to represent scenarios in a tree structure, as shown in the figure above (from scenariothinking.org). Decision makers thus make specific decisions considering these scenarios (e.g. preventive steps, proactive means).
Having many scenarios (may be thousands or more) in hand, the scenario discovery technique can quickly process these scenarios to help people navigating through many scenarios, interpreting the uncertainty, and find patterns and trends between the scenarios and the decision makers’ specific interest. RAND Corporation, a big nonprofit institute focusing on improving policy and decision making, considered the idea of scenario discovery as one of the “future methodologies”.
Essentially, the idea of scenario discovery is: the decision maker first think about the goals and clearly specify the unsuccessful situations. Data mining methods, either classification and regression tree (CART) or patient rule induction method (PRIM) are used to identify the problematic region in the feature space where unsuccessful situation would like to occur. As a result, one identifies the problematical scenarios in terms of certain range of some input parameters.
For example, the paper “Discovering plausible energy and economic futures under global change using multidimensional scenario discovery” used scenario discovery method in the context of R&D policy making of the carbon free technologies. Features being used in generating different scenarios include several R&D efficiency parameters and some labor efficiency parameters.
The result of the scenario discovery would be some grouped cases with certain interpretations. The figure above from the paper shows the different high-level scenarios being identified. For example, Scenario 1 has a high annual consumption and low carbon emission and is therefore labelled as “Green High Growth (HG)” scenario, and Scenario 2 is labelled as “Green Low Growth (LG)” scenario. Similarly, Scenarios 2, 5, and 4, perhaps are thought of as “Late Transition”, “Technological Slowdown”, and “Growth Focused”, respectively. Meanwhile, some trends and relationship of the data have been explored. In this way, decision makers will have a clear understanding of what are the major possible scenarios, and handle the policy with due flexibility.
Many available mathematical models can be used in different situations. However, to know which to use in a particular situation can be quite intelligence-demanding. Some business decision framework can be model by “decision tree” method or “Baysian network”. The latter is more complex but more powerful because it incorporates different probabilities. It can model the reasoning process in terms of probability. Alternatively, scenario-based method is always a good choice for decision making under uncertainty. Scenario discovery is one of the tools to do this analysis. Although, it will be questioned by many decision makers (see Critique of Shell’s use of scenario planning), it will still be used in some organizations for some high-impact decisions.
Gerst, M. D., P. Wang, and M. E. Borsuk. “Discovering plausible energy and economic futures under global change using multidimensional scenario discovery.” Environmental Modelling & Software 44 (2013): 76-86.
Koller, S. “Probability Graphical Models” course, Stanford University
RAND Corp. Future methodologies – scenario discovery page.
Scenario thinking wiki: scenariothinking.org
My previous post: Introduction to reservoir simulation