Decision Analysis

 Why do we need Decision Analysis?

 

The above are classis cases that render themselves to decision analysis.

Decision Analysis is a structured process that helps in making decisions when faced with complex, competing alternatives. It depicts the consequences of different decisions which can be analyzed to arrive at an optimal decision. It is most effective in uncertain situations with limited choice.

Effective Decision Analysis requires that business understands:

1.       Values, objectives and goals that are relevant to the decision problem

2.       Nature of the decision to be made

3.       Areas of uncertainty affecting the decision

4.       Consequences of each possible decision

Decision Analysis is important as there may be unknown factors relevant to the decision problem, interrelated factors for consideration, conflicting perspectives on a situation  or trade-offs between the available options. Decision Analysis takes all these factors into account and assists in arriving at a well-informed decision

Decision Analysis has the following main elements

v  Outcome: Analyst use mathematical models to predict possible business outcomes. Can be done using:

Ø  Financial Analysis – Estimates the market value of an asset.

Example: Discounted Cash Flow, Net Present Value, Internal/Average Rate of Return, Pay-Back period, Cost-Benefit Analysis are sample evaluation techniques.

Ø  Non-Financial Analysis – Outcomes are not expressible in financial terms. Can be done using metrics/relative scoring of outcomes.

Example: Customer satisfaction rating, defects per thousand etc.

v  Uncertainty: Uncertainty becomes relevant to a decision problem when it is impossible to know which outcome will occur. This may be due to missing information or because the outcome depends on how others respond.

A common method of dealing with uncertainty in decision problem is to calculate the expected value of outcomes. This involves estimated the percentage chance of each outcome occurring and then multiplying the numeric value associated with that outcome by that percentage.

v  Trade-offs: Trade-offs are important when the decision involves multiple, possibly conflicting objectives. Effective methods include:

Ø  Elimination of Dominated Alternatives – A dominated alternative is any option that is clearly inferior to some other option. If an option is equal to or worse than some other option when rated against objectives, the other option can be said to dominate it. In some cases, an option may also be dominated if it only offers very small advantages but has significant disadvantages.

Ø  Ranking objectives on a similar scale – One method of converting rankings to similar scale is proportional scoring. Using this method, the best outcome is assigned a rating of 100, the worst a rating of 0, and all other outcomes are given a rating based on where they fall between those two scores. If the outcomes are then assigned weights based on their relative importance, a score can be assigned to each outcome and the best alternative assigned using a decision tree.

Tools for Decision Analysis

§  Decision Trees: Provide a highly effective structure to lay out options and investigate possible outcomes of choosing the options. It provides a framework to quantify the values of outcomes and the probability of achieving them. This helps in analyzing the consequences of a decision thoroughly.

§  Influence Diagram: Compact graphical and mathematical representation of a decision situation. Typically applicable in Team Decision Analysis since it allows incomplete sharing of information among team members to be modelled and solved explicitly.

§  Probability Distribution: Provides the possible outcomes of a problem along with the probabilities of getting the various outcomes. It assigns a probability to each measurable subset of the possible outcomes.

§  Monte Carlo Distribution: Values are sampled at random from the input probability distributions. Resulting outcome from that sample is recorded. This is done hundreds or thousands of times and the result is a probability distribution of possible outcomes. It provides a comprehensive view of what may happen and how likely it is to happen.

Advantages of Decision Analysis

§  Provides an effective technique to determine the expected value of an alternative scenario.

§  Consistent usage of financial analysis helps to with quantitative measures upon which to make project investments.

§  Forces stakeholders to honestly assess the importance they place on alternatives.

§   

Disadvantages of Decision Analysis

§  Requires specialized knowledge and skills, mathematical skills, understanding of probability and so on.

§  Decision-makers may be reluctant to re-visit decisions even when the information is available on areas of uncertainty that might change the optimal decision.

§  Results of Decision Analysis may be treated as more certain than they actually are, if the assumptions/limitations are not properly understood.

Case Study - Decision Analysis - Practitioner

NextGen IT Services is an IT Service Provider organization, based out of India. Fast Wheels Automobiles (FWA) is a leading car manufacturing organization, with headquarters in Atlanta, USA, and spread across many locations in the world. FWA is considering a total overhaul of their existing IT systems and applications. NGIT has just won a 5-year umbrella contract. Under the terms of the contract, development work is expected to be allocated in phases. Anjali is the BA who has been assigned to manage requirements for a project MyPortal (related to development of an employee portal). She has conducted a brainstorming session and has a long list of requirements generated. Anjali has organized a session to focus on one of the high level requirements that relates to usability. The participants are Brian Hills (HR SPOC), Miriam Cohen (Finance SPOC), Walter Moor (Admin SPOC) and Tim Dalton (Delivery SPOC) from FWA. This team has been together frequently to finalize the requirements for MyPortal project. Additionally, Anupam Patel from NGIT development team has also joined the session

Anjali: Hi, all. Today, we are going to take some decisions on the requirements using Decision Analysis. To do this effectively, we will be using certain techniques.

Brian: That sounds interesting, but can you explain exactly what you mean by Decision Analysis?

Anjali: Decision Analysis is simply a process to make key decisions in a structured, objective and wise manner.

 Walter: Come on, Anjali. We make decisions every day. Don’t tell me we need to have a process for this.

 Anjali: Many managers just rely on their experience and instincts to make decision. Do you think that is the right approach?

 Walter: Well, no, no exactly…

 Miriam: I think what Walter means is that it would simply be too much time and effort to put a structure and process around decisions.

 Anjali: Agreed. That’s why I said key decisions.

 Walter: And how would we define key decisions? That’s subjective.

 Anjali: Well, we could make it objective by defining thresholds or triggers for executing Decision Analysis process

 Tim: Such as?

 Anjali: Schedule impact > 10%, purchase over $10,000, high risk impact, management or design decisions, things like that.

 Walter: Why would MyPortal project need such structured decision making?

 Anjali: Actually, I had planned that today. Our agenda would center around one particular area, usability. MyPortal is going to be used by all employees in FWA, so it is a high impact project in terms of affecting all employees in the organization. And there are multiple approaches for solutions to be considered while designing the screens. Rather than approach, this is an ADHOC manner, putting a structure around the process will help us to take more effective decisions.

 Brian: Let me take a shot at this. I would guess when the decisions do not have a major impact.

 Anjali: Also when we cannot use objective criteria. Can any of you tell me what factors listed under usability in the previous session?

 Tim: I remember. We listed, 1. User Task Analysis, 2. Readability, 3. Site Navigability, 4. Accessibility and 5. User Experience

 Anjali: Great. Now, take the last one, User Experience. Here, some questions we need to consider would be related to: does the user feel satisfied after interacting with the site, does he feel the site is useful, does he enjoy the experience, does he feel good when he enters the site – it is obvious that we cannot quantify such feelings and emotions, so we will not have a definite objective criteria here.

Miriam: Right, we get it. Let’s move on, Anjali.

Question: Walter wonders whether Decision Analysis is usually done in a group, with all members participating, the way Anjali is planning the session. Which of the following is the decision making approach being adopted by Anjali?o    1. Directive

o    2. Consultative

o    3. Democratic

ü  4. Consensus

Note: Consensus correct. Anjali’s approach is to get all members to participate in a decision that all will support.

Anjali: Okay, the design and development teams have come up with 3 prototypes, based on the latest user trends. I have here with me, Anupam Patel, from the development team. He will give us a demo on the prototypes and answer any queries we have. All of the above factors can be evaluated through these prototypes. For each of the factors, we will as a group, rank each of the prototypes on a scale of 1 to 10, where 1 is the lowest and 10 is the highest. We can also assign weightage to each of the factors, based on their relative importance. We can then thrive at a final score for each prototype and decide on which one addresses our requirements best. Does that sound ok to you all?

Miriam: Sounds okay. But I am not sure if we can assign weightage to all these factors, because they are all equally important. What do the others feel?

 Walter: I agree with you, Miriam. I don’t think we can say navigability is more important that readability. And that applies for all the factors.

Anjali: Well, if Brian and Tim also feel the same way, we can just assign the same weightage to all the factors for now. Later on, if we feel one of the is more important that the other, we can change the weightage and the score will reflect the change. Shall we proceed?

 Tim: Yes, Anjali, I think that’s a good idea. Let’s move on.

 Anjali: Okay, the first one is on User Task Analysis. To rate this, we need to consider questions such as, how easy it is for the users to lean to perform tasks? For complicated tasks, is the help sufficient? Can any of the tasks be streamlined further? How prone to errors is the task?

 Walter: That gives us a good idea how we need to evaluate this. Can Anupam give us a quick demo so we can observe for ourselves and rate this?

Anupam gives a demo on all 3 prototypes and explains the features in each. The team discussed and agrees that PT1 can be assigned a score 5, PT2 can be rated 9 and PT3 can be rated 7. Anjali captures this on the whiteboard. They repeat the exercise for all the factors

Question: Miriam asks Anjali whether some of the techniques for Decision Analysis require specialized knowledge and skills. What would tour response be? ü  1. Yes o    2. No Note: Yes is correct. Many of the techniques require some basic mathematic knowledge, an understanding of probability and similar concepts. KEY LEARNINGS  Decision Analysis is a process to make key decisions more objectively and wisely. Making informed, high quality decisions reduces project risk and increases probability of project success.         Not every decision requires the use of Decision Analysis process.   Guidelines must be established to decide which issues should be subjected to a formal Decision Analysis process. The generation and consideration of multiple alternatives early in the Decision Analysis process increases the likelihood of effective decisions being taken. Tools and techniques exist that can be applied to help make the DAR process objective and effective.  Case Study-Decision Analysis - Specialist  PM Skill Pro is an organization that specializes in products to support project management activities. They have been in operation for more than 10 years and some of their products need to be upgraded to meet the requirements of a new generation of users. Harini is the product manager for the XpertMetrics tool. She has been asked to prepare a strategy to improve marketability of the tool. As it stands, the tool is in need of major enhancements. Harini does some market research, analyses the data she has gathered and explores some alternatives on what needs to be done to make the tool more attractive to prospective customers. She then presents her ideas to the management team to get management buy-in her decision.  Harini uses the decision tree technique to decide the right approach. She has set up a meeting to present her business case to her leader, Mitun Sharma (Delivery Head) and Arvind Gupta (Head of Product Development). Harini: Good Morning, Mitun! Good Morning, Arvind! Thank you both for taking time to attend this session. As you know, this session is to discuss the strategy for increasing the customer base for XpertMetrics tool.  Arvind: Hello, Harini. I am eager to see what you recommend to achieve this goal for our organization.  Mitun: Harini, why don’t you take us through the background, whatever analysis you have done and your recommended solution?  Harini: Sure, Mitun. Let me give you a quick background on XpertMetrics  As you are aware, this tool to aid PMs with metrics collection, storage, analysis and reporting was developed about 10 years ago. This has been a popular product and we have a strong customer base. However, over the last couple of years, the application software landscape has changed drastically. In the last year alone, we have received about 100 change requests from our existing customers requesting us to improve existing functionality, introduce new features and to improve non-functional requirement such as usability of the screens, performance, scalability etc... Arvind: I noticed this feedback in some of the customer surveys we did. Go on, Harini! Harini: Arvind, I have worked with the marketing team to collect data on what our competitors have to offer. We also have conducted a survey to understand better what the clients are seeking. Based on this, we have come to the conclusion that we need to immediately take effort to upgrade the tool in a big way. Mitun: So what options did you explore here?  Harini: We have two possible options  The first is to phase out the existing tool completely and develop a new tool from scratch. The advantages are that we would be able to move to the latest technology and come up with something trendy and innovative that would appeal to the new generation of users. The disadvantage would be that it would be costly in terms of time and effort. The second option is to keep the existing tool as a base and see how we can exchange it to meet the changing user needs. This would mean less effort but the end product may still not be as light and attractive as users would like it to be, though it would serve the basic functionality. Arvind: So what is your recommendation and how do you justify this?  Harini: I used the decision tree technique to arrive at my recommendation. Let me project this onto the screen for you. Harini: As you can see, I have two branches, one for each of the possible solutions. The result of both the decision is uncertain, so I started by assigning a cash value to outcome based on the market data gathered. Next, I estimated the probability of each outcome and calculated the values. Here are the calculations I used. Mitun: I see that you have also estimated the cost of each solution. Harini: Yes, then I subtracted the cost from the outcome value. This gives me the value that represents the benefit of the decision. Arvind: Following your reasoning, it appears that the better of the two options is to go for fresh development. Harini: Yes, Arvind. Based on my analysis, I feel that this will yield better return in the long run, even if it will be more expensive to arrive at the solution. Mitun: Looks like you have put in a lot of thought and effort into this, Harini. Good work! Arvind: Yes, thank you, Harini. I would like to have a more detailed look at the data and analysis that led to this recommendation. I will get back you in a couple of days with the next steps. Harini: Sure, Arvind. Do let me know if you need any further details. Question: Why did Harini choose to use the Decision Tree technique? o    1. Structured method to lay out options and investigate possible outcomes o    2. Suitable for multiple alternatives and where many different factors to be considered o    3. Helps to form a balance picture of the risks and rewards ü  4. Useful when the outcome depends on how others will respond. Note:  Options 1, 2 and 3 are incorrect because they may apply to other decision making techniques as well. Option 4 is correct. Decision tree technique is usually chosen to deal with uncertain outcomes. In the above scenario, Harini cannot be 100% certain on the client response to the new tool. KEY LEARNINGS ·         Decision tree is a method of assessing the preferred outcome where multiple sources of uncertainty may exist. ·         Decision trees clearly layout the problem so that all options can be challenged. ·         This technique allows us to analyze the possible consequences of a decision. ·         It provides a framework to quantify the values of the outcomes and the probabilities of achieving. ·         It helps us to make good decisions on the basis of existing information and common sense. Quiz 1.       Decision Analysis helps in making wise investment choices using tools as Financial Analysis and Cost-Benefit Analysis? State whether True or False? ü  A) True o    B) False 2.       A business analyst is confronted with 3 different options to address a customer need – outsourcing, in-house development and buying COTS product. He discusses with the project manager and together they decide to move ahead with the option to buy COTS as that is the easiest route. Is this the right Decision Analysis approach? ü  A) Yes, the problem domain and alternatives are identified. This is typically Decision Analysis process. o    B) No, the analyst should have done decision analysis independently. o    C) No, the analyst should have outlined the consequences of each decision/outcome and then recommend the optimal decision. 3.       You have been asked to perform a decision analysis to determine the solution scope? Which of the below would be your approach? ü  A) Be aware of the objectives, goals and nature of the decision being made. o    B) Perform a cost-benefit analysis to compare the cost of the system against the benefits. o    C) Perform a financial analysis to estimate the market value of the asset. o    D) All of the above 4.       Which among the below is not an element of Decision Analysis as suggested by BABOK? o    A) Uncertainty ü  B) Root Cause o    C) Outcome o    D) Trade-offs 5.       Non-Financial Analysis can be used to determine the values of outcomes using metrics and relative scoring techniques. State whether True or False? ü  A) True o    B) False 6.       Due to uncertainty involved with respect to the outcomes, the team decides to perform a Probability Distribution of the outcomes and calculate the expected value of the distribution. Which of the below depicts the exact approach to do this? ü  A) Identify the various outcomes with their numerical values. Determine the probability of occurrence of each outcome. Plot the probability distribution by multiplying the probability of the outcome with the numerical value. Expected value is the weighted average of the outcomes, with each probability serving as the respective weight. o    B) Identify the various outcomes with their numerical values. Determine the probability of occurrence of each outcome. Plot the probability distribution by multiplying the probability of the outcome with the numerical value. This gives the expected value. o    C) None of the above 7.       Which of the below are effective methods to deal with trade-offs involved in a decision-making approach? o    A) Payback-Period. o    B) Internal Rate of Return ü  C) Dominated Alternatives elimination ü  D) Ranking of objectives 8.       Which of the below is NOT an aspect of Monte Carlo Simulation? o    A) Provides a comprehensive review of what may happen and how likely it is to happen. o    B) Is depicted as hierarchical structure. o    C) Provides a probability distribution of the possible outcomes. o    D) Done multiple times by sampling out values from the input probability distributions. 9.       Out of the 5 different choices, it was found that 2 choices were inferior to others. They also had more disadvantages as compared to the other choices. The team discarded the 2 choices and then performed a detailed analysis on the other choices. Was the team right in discarding the choices? ü  A) Yes – Elimination of dominated alternatives is a method to perform trade-offs during DA o    B) No  - All choices should have been analyzed further as part of DA. 10.   Decision Trees and Influence Diagrams present the alternative choices, uncertainties and the evaluation measures determining how well the objectives are achieved in the final outcome. State whether True or False? ü  A) True o    B) False