Series Model

Modified on Wed, 15 Nov 2023 at 10:29 AM

Description

This step trains a model using series-based data, combining both curves (series, such as time series) and values (single scalars) as inputs and outputs.

Application

This model unlocks in particular the use of sensor data and time-series data, but could also be applied to other series, such as frequency plots for spectral analysis, or stress-strain curves for material design.

How to use

As with all AI models, you will require at least one dataset to train the model.

Model type: Select which type of model you want to train. The options are:
- Basic models will lose overall picture of input series, but can be used with simple and smaller data sets. This model is similar to a conventional tabular model, but will handle the series aspect of the data more easily.
- Intermediate models and Complex models should be preferred when more data is available and when the relationships in the data set are more complex. In this case, it is advised to experiment with both model types and select the one that provides the best accuracy on unseen data. These two model are both based on a Convolutional Neural Network architecture and therefore have the same hyper-parameters (see below).
Series Data: Select the data you want to use to train the model. This data must be in a specific format for the model to train successfully (see requirement below).
Series Identifier: Select the column that will identify different series. If multiple columns are needed to uniquely identify series, these columns could be concatenated in a previous step to create one single identifier column.
Series Field: Select the column that contains the field or unit of the series/curves used to train the model. This column should be common to all series (input and output). Typically, this would be time, but could also be frequency, deformation, or simply an ID column (1, 2, 3, …).
Curve inputs: Select columns that will be used as curve/series inputs.
Value inputs: Select columns that will be used as value/scalar inputs. For each given series, these columns should be constant.
Curve outputs: Select columns that will be used as curve/series outputs.
Value outputs: Select columns that will be used as value/scalar inputs. For each given series, these columns should be constant.
Series Model Name: Write the name you want the series model to have

Advanced Options

Clicking Show/Hide Advanced Options will display the additional options. The following options are available for all model types:

Disable check for constant value and use first value instead

This can be useful when constant values have insignificant differences between them but should still be considered as constant. If this option is not ticked, an error will occur, saying that a value selected as constant is actually not constant.

Hyper-parameters which are specific for the Model Type Basic:

Model

Ridge will be a simpler, regularised model, while RF will help capture more complex relationships.

Hyper-parameters which are specific for the Model Types Intermediate and Complex:

Number of training steps	Choose the number of steps you want the model to train for. More steps are likely to produce a better model, but will require more time to train.
Early stopping	This parameter defines the “patience” of the model when doing early stopping. During the training, if the model doesn't improve (e.g. error doesn't decrease) for the number of steps selected, then the model will be considered to have converged and will stop training. If the option “disabled” is selected, then the model will always train the number of training steps selected.
Learning rate	The learning rate is a crucial hyperparameter that determines the step size at which the model adjusts its internal parameters during training. A higher learning rate can lead to faster convergence, but it may also cause the model to overshoot optimal parameter values. On the other hand, a lower learning rate might lead to more precise parameter updates but could increase the training time. Finding an appropriate learning rate is often a trial-and-error process, and it plays a significant role in achieving optimal model performance.
Loss function	The loss function quantifies the difference between the predicted values of the model and the actual target values in the training dataset. The goal of training is to minimize this loss, which essentially measures how well the model is performing. This step offers two options: Mean Squared Error (MSE) MSE will penalize larger errors more heavily. It calculates the average of the squared differences between predicted and actual values. This means that larger errors have a greater impact on the loss, which can be suitable when you want the model to prioritize reducing significant outliers and focus on minimizing the variance in predictions. MSE is commonly used in regression tasks when the target variable's magnitude matters and the goal is to find a model that closely fits the data. Mean Absolute Error (MAE) MAE will treat all errors equally and won't heavily penalize outliers. It calculates the average of the absolute differences between predicted and actual values. Unlike MSE, it doesn't square the errors, making it more robust to extreme values. MAE is a good choice when you want a more resistant loss function that provides a more balanced representation of errors across the entire dataset.
Dropout	Dropout is a regularization technique used to prevent overfitting in neural networks-based models. During training, dropout randomly deactivates (sets to zero) a proportion of the neurons in a layer. This helps prevent the network from relying too heavily on specific neurons and forces it to learn more robust features. Dropout effectively introduces a level of uncertainty in the network's predictions, making it less likely to overfit the training data. However, during inference (testing), all neurons are active, and their outputs are scaled to account for the dropout effect.
Curve importance vs value	This parameter enables the user to specify if they want to attach more importance to the curve (series) or to the values (scalars) outputs. A value close to 0 will make value outputs more important when training the model, while a value close to 1 will make the curve outputs more important. As the data is not normalised in this model, this parameter can also be used to balance the values when the scale of the curves is different from that of the values. For example, if a curve output is in average 5 times larger than the value outputs, setting this parameter to 0.2 (which is 1/5) would produce similar contributions for the value and curve outputs.

Examples

Examples of data sets in the correct format (top left table) and incorrect formats (other three tables). Regions producing the incorrect format were highlighted in red.

Example of the step after completing training: