Formative Scenario Analysis

The Scenario approach we discussed in the lecture and the seminar is used in part as system assessment tool (impact matrix) and as assessment tool for consistency of visions. Originally, it was presented in full length in the Book "Embedded Case Study Methods" by Roland Scholz and Olaf Tietje (2002).

We will run you through the basic steps of the FSA by doing the related task in Assignment 8. For each step a brief explanation of its purpose, general steps and the application on the assignment is given.

What is Formative Scenario Analysis?

FSA is an explorative scenario anaylsis / consruction tool. It requires a good uderstanding of the respective system or case by knowing relevant impact factors and their direct interactions (see systems thinking for causal loop and Scholz&Tietje 2002 for details). It yields a complete list future states of the system by combinations of all potential future levels of each impact factor. Scenarios are typically selected by their consistency (i.e. how logical or coherent the combinations of possible future levels of impact factors are).

Why are we conducting FSA?

Important qustion! Actually, every FSA assigns a significant amount of time for formulating a clear statement, concerning the goal of it. But in terms of solution oriented research, we need some ideas of possible futures in order create an understanding how the system we ivenstigate can theoretically develop. Potentially, not probably. As Scholz and Tietje point out, "possibility is a prerequisite of probability". So even if not very probable, we want to know, if there is possible future state which is within the realm of our idea of desirability or sustainability. For this, we need the complete set of possible futures. Needless to say, that a good understanding of the system, based on sound evidence or broad expert knowledge, is critical for scenario quality.

The "cookbook" - 9 steps of FSA

The FSA typically consists of nine steps. You will find slightly different process depictions in the original book by Schloz and Tietje, but the resulting worl flow is essentially the same.

In the following, we run thhrough the single steps and relate to our case description of Lake Ordeal.

Procedural steps of FSA: taken from Lecture on FSA 08.12.2016 by Daniel Lang

1-1 System and Goal Definition

In General: The guiding question for this step is, What is the problem or case? Why is the FSA conducted? For what benefit or pupose?

Don't take these questions lightly. Preparing a concise explanation and description will not only result in a document or an introduction of a paper. The process of doing so, briefs the study team and facilitates getting on the same page.A good example how such a goal definition can look like is presented by Spoerri et al. 2009 in the introduction. In the case study methods, the case is usualy deliminted by spatial boundarires and general perspectives on opbjects of analysis.

In the Lake Ordeal example: At the lake, we want to focus on the problem of social cohesion and economic effects of the overharvested resource. Our case or study area is the lake shore of one country at the lake. There is extraction of fish for subsistence and for exportation. People are mainly imployed in fishey and fish processing industries. The source of income attracted more people from further away than there are jobs, creating unemployment, exploitation, and a strong informal sector. The fish for exportation is largely fished by industrial fisheries (large ships) creating more relative pressure on fish stock than artisanal or small scale subsistence fishing. However, the state gains signifiant revenues from expertation depending on international market prices for fish, which can be used for subsidizing social infrastructure and new alternative fields of economic activity. Now the FSA relevance, why do we do the FSA? There are different policy ideas for solving the problems of depleting fish stocks and social cohesion and unemployment. One is rooted in the idea of a social welfare state which needs revenues from export, the other is to support the capacity for subsistence, which needs replenished fish stocks and less pressure on them by indutrial fishing for alleviating competition. However, world market prices are uncertain and fluctuating and depend on trade agreements between neighboring countries and their fishing activities. We need the scanrios for spanning the field of possibilities in order to configure effective coping strategies.

2-1 Impact factors identification and selection

In General: The task here is to determine the most important factor in the system for which we want to understand possible future states. As you can imagine, this is a critical step because scenario quality will depend a sound understanding of the system. IMagine a flawed system description assuming factors and impacts which do not exist or are false. The possible futures will necesssarily flawed as well. For pracitce reasons we focus primarily on the methodology here. In reality, study teams work together with experts and involved stakeholders. For understanding the impact of pice changes on export and resulting state revenues we would need the knowledge of economists and ministerial staff. Plausible changes of fish stocks according to different quantities of extracted fish is the domain of marine ecologists. Necessary conditions for subsistence from fishing can best be assessed by artisanal fishermen. With this extended study team there are more methods to derive a meaningful set of impat factors. One also presented in the original book is the plus-minus method.

In the Lake Ordeal example: given the description above, we derive a quick and dirty set of possible impact factors.

Table 1 - Impact Factors and future states

This is first try to find some relevant impact factors. Note that the column on plausible future state corresponds to Step 3-1. Literature has it, that in a real case study you end up with around 20 impact factors from which a smaller set of around 12 are selected for scenario construction. Having many impact factors makes the actual scenario construction and consistency analysis computationally expensive and time consuming.

Selection of relevant impact factors is facilitated by the impact assessment, where usually the most active factors are chosen.

2-2 Impact assessment and 2-3 impact analysis

in General: for a meaningful construction of scenarios we need a thorough understanding of system strcture and dynamic. This knowledge is usually revisited when the final set of impact factors are selected for scenarios construction, at the consistency analysis and in the description and interpretation of scenarios.

We analyse impacts between impact factors by starting at row one and asking in our case, “does the state of the fish stock as described directly influence export rate?” If there is influence we insert a “1” and a “2” for a strong impact. In practice, you need to operationalize well, so detailed expert knowledge translates in a meaningful matrix. It is important, that we ask for direct impacts only. Going this direction, we have the sum of impacts in each row as the activity that the factor of the row exerts on other factors. Vice versa, the column sum is the passivity score of the respective factor.

In the Lake Ordeal example: In our case, the export rate is the most active factor, with the biggest influence on other system elements. Societal well-being is the least active and the most passive factor. As it is of major interest as a kind of target variable in our problem, we would still want to keep for scenario construction.

Table 2 - Impact matrix

We can sort the impact factors in a system grid for a better overview.

Figure 2: System Grid

We see, that according to our assessment, societal well-being is mostly a function of other factors while e.g. the world market price is fairly active but not influenced by any other factor. Export rate, government spending, Fish-stock and job oportunities do have an influence on others while also receiving impact from others. They are referred to as factors with a high “involvement”.

We can now go on and think about indirect impacts. While going through the direct impacts, you might have thought that e.g. the export rate does have an impact on societal-well being. Either negative by depleting fish stock, or positive by creating government income.

One way to assess these direct impact is (you know it) drawing a system graph. In this perspective it is essentially the same as a network of impact factors. you can also run a so-called mic-mac analysis (check in the book). Here, the matrix needs to be coarsened, by only filling in a “1” for a strong impact. When the matrix is multiplied with itself repeatedly, the rankings become stable and the most indirectly active or sensitive factors become visible. In many cases, the indirect assessment does not change the general picture and often the scenario analysis proceeds with the simple direct impact assessment.

So what do we do with this now? For going into the scenario construction and consistency analysis, we should pick the most important factors in order to reduce the computing expense. In our example the set seems quite alright, but we want to end up with 4 factors in the example, so which ones can go and which ones do by themselves best explain the system?

Usually the most active remain in the sample. But as societal well-being is one of the key target-variables in our problem, we will leave it in the set. Same goes for fish stock, for being highly active. As a highly external factor determining the other very active factors (export rate and government spending) we will also keep world market prices. government spending is explained by world market prices and export rate. job opportunities are one idea to remedy fish stock depletion by government spending. So for our Scenarios we think of job opportunities as a function of world market price. To make the example not too laborous, we exclude the world market price and end up with:

societal well being
fish stock
job opportunities

Note that this example is only supposed to illustrate the reasoning that can lead to a limitation of factors. In reality this is hopefully backed by expertise on the respective subject matter.

3-2 Consistency assessment (see 3-1 future levels in 2-1)

In General: Now, for each possible future level of an impact factor, there will be a column and a row. This matrix is similar to the impact matrix, but now, we are assessing the plausibility of co-occurrance of each possible pair of future level. This is on "co-occurrance only". And that'st the other difference: there is no direction in the assessment, which means, we'll only have to fill in one half. Going on with the example will clarify this.

In the Lake Ordeal example: As we have already done the future levels in table 1, we can proceed with the consistency matrix:

Table 3 - consistency matrix

The consistency matrix gives an overview of the logical coherence of two co-occurring future levels of an impact factor. For instance, in our little example we assume that job opportunities take the pressure off the fish stocks, because less people are fishing. But, these opportunities are “paid” for by export revenues which is determined by world market price. And we assume reasonable business and government, exporting less when prices are high (because we need less to reach the goal).

3-2 Scenario construction

In General: For making the actual construction, we create a scenario for every possible combination of future levels of each impact factor. This is a combinatoric construction and is usually done by a machine. Dealing with four impact factors and eight related future levels is still quite manageable. But when you think of 12 factors with some having three future levels, you just can't do that by hand. In our case there are 3 factors with 2 levels which results in 2³ = 8 Scenarios. But considering our 6, still superficial, impact factors from above and assume good systems knowledge and thus each having three possible future levels, we already end up with 3⁶ = 729 Scenarios. Now, with that number it is clear that we need a systematic way to select scenarios. Which we start by getting rid of the inconsistent ones.

In the Lake Ordeal example:

Table 4 - Scenarios

We need to take the consistency value of each pair of impact factors in a scenario. For example: consistency of S3 = (high social.. & depl. fish.. = -1 ) + (high social.. & high jobs.. = 1 ) + (depl. fish & high jobs.. = -1 ) = -1. So this scenario is not logical. We would not select it. Later, we will actually filter out any scenario which contains an inconsistency at any place because one inconsistent combination is enough to render a scenario itself inconsistent.

Table 5 - Consistency calculation

So here we have our two consistent scenarios. Due to the many inconsistency ratings in the consistency matrix, there are only two scenarios left, where there is no inconsistency at all: S1 and S8.

4-1 Scenarion selection

In General: normally there would be a some hundreds or thousands of scenarions. Basing the selection only on consistency values, we might still end up with very similar scenarios. This wouldn't tell us much about the actual width of the option space. We are interested in how different the future can look like, not in incremental differences between the most plausible futures. For this purpose there is another routine to calculate the reative difference between scenarios (which is coming sool)

4-2 Scenario interpretation

In General: after all this number crunching, we need to get back to some sense making. Out of the hundreds of possibilities, what do the selected ones tell us? Can we tell a story, using all our systems knowledge, that is compelling and reveals som more insight on the possible path to that future? Can we articulate a common preference for one of the scenarios presented?

In the Lake Ordeal example: The two Scenarios above basically refer to two narratives: in S1, things are going quite well. Fish stock recovers, people are well and they have new jobs outside the fishing industry. In S8, there are only jobs in the fishing industry, which leads to further depletion of the fish stock and leads to persistent problems caused by overfishing.

Of course, this is simplistic and of little help as it only confirms our prior belief that jobs outside the fishing game is the solution. You can go about the analysis with more impact factors and a more sophisticated assessment of impacts and consistencies. But for now, the general workflow should be clear.