Sensitivity Analysis

Description

The sensitivity analysis step shows in a visual way how much impact each input of a model has on the different outputs of this model.

Application

Sensitivity analysis helps you to increase your understanding of the relationships between inputs and outputs of your models. This can help you finding errors in a model if unexpected relationships are encountered; you can use these findings to improve your model in an iterative process. It might also help you to find patterns you weren’t aware of so far, improve your understanding of the data behind the model, and provide guidances on where to concentrate your design/engineering efforts to improve product performance.

Another way to analyse the input-output-relationships is to use the Explain predictions step.

How to use

When you first click on Sensitivity Analysis you have to configure the manipulator:

• Select a Model on which the analysis should be performed.
• If you want to do the analysis for all inputs and outputs leave the checkbox All inputs and outputs ticked.
• If you untick the box two new fields appear. Assign all inputs for which the analysis should be done to the field Varying inputs. The outputs for which you want to see the results to the field Outputs.
• Select one of the available Methods for the sensitivity analysis. Currently Sobol, Morris and FAST are available (see below for more info on the methods).
• Show only most sensitive inputs: the manipulator uses a heat map to show the results (like Intelligent correlation). If you have a high number of inputs, the plot becomes confusing. You can limit the number of parameters that are shown with this parameter.
• Number of samples (N) configures the number of data points which are used to calculate the sensitivity. The higher the number the better the statistics of the analysis but the longer the step takes to finish the calculations. 1000 is the pre-filled default value and will work in most cases.
• The last parameter depends on the selected method. See below for explanations on these parameter.

More on this step

Sensitivity analysis can't handle categorical data. If your model uses categorical inputs, these inputs need to be fixed to be able to use sensitivity analysis. Create a Fixed Parameter step and use it for all categorical inputs. 

Sobol method

Sobol is a variance-based method. The design space is randomly sampled with N points as specified. All of these samples cause a variance in the output results. This variance is decomposed and attributed to the input variables (First order) and combination of variables (Second order). The sensitivity of the output to an input variable (or a combination of inputs) is therefore measured by the amount of variance in the output caused by that input. Therefore, the attribution of all inputs and all input-combinations add up to 1.0. Variance-based methods allow full exploration of the input space, accounting for interactions, and nonlinear responses.

If you select to run Sobol with First order you get a heat map in which each row corresponds to one output and each column to an input. The colour of a box indicates how strong the sensitivity of the output regarding the input is. The deeper the red the higher the sensitivity. The numbers in the legend are the fraction of the total variance in the output attributed to an input (combination).

If you select to run Sobol with Second order the shown heat map looks different. You get a heat map for each output separately. On the x- and y-axis are all inputs which are used for the analysis. The plot is a matrix of all possible input combinations. As the combination a-b and b-a are the same, only half of the matrix is filled. Also, the diagonal isn’t filled which would be the combination of an input with itself. In the example below the dark red field is the variance in the output caused by the input combination gas_turbine_exhaust_pressure and compressor_discharge_pressure (the contribution would be roughly 0.5 = 50%).

Morris method

The Morris method is a one-step-at-a-time method (OAT). The method starts from N random points in your design space. OAT means that starting from one of these points one input variable is changed and the resulting delta in the output is calculated. Starting from that point the next input variable is changed. This way starting from each of the N points the method moves through the design space on random paths, varying only one input at a time. For each input the average change of the output is given. That is, for Morris the legend gives absolute values in the dimension of the output.

The delta values in the output can be positive or negative values depending of the direction of the output change. If positive and negative values occur for a certain input (that is, the model is behaving non-monotonic) the average value as calculated by Morris can be very small as negative and positive contributions cancel out each other. That is one of the main drawbacks of the Morris method. Also, as the OAT approach doesn’t change multiple variables simultaneous it can’t detect interactions between input variables (second or even higher order effects).

The parameter Number of grid levels controls the step size the algorithm uses when varying the inputs. The number should be even. Typical values are 4, 6, and 8.

FAST method

The FAST (Fourier Amplitude Sensitivity Test) method is a variance-base method like Sobol. Based on N random samples the contribution of each input to the output variance is calculated. FAST uses Fourier series to solve the problem. This is computationally more efficient than for example Sobol but limited to first order effects. The generated heat map is looking like the Sobol-generated and the numbers are to be interpreted the same way.

The Interference parameter (M) controls the number of harmonics to sum in the Fourier series decomposition. In practice M is frequently set to 4 which is also the pre-filled default value. If M is increased the method becomes more accurate but the computational effort is also going up.