Main Content

Update Metrics

Update performance metrics in incremental learning model given new data

Since R2023b

expand all in page
  • Update Metrics Block Icon

Libraries:
Statistics and Machine Learning Toolbox / Incremental Learning

Description

The Update Metrics block outputs the performance metrics of a configured incremental model for binary classification (incrementalClassificationLinear), multiclass classification (incrementalClassificationECOC), or linear regression (incrementalRegressionLinear), given new data.

Import a trained incremental learning model object into the block by specifying the name of a workspace variable that contains the object. The input port mdl receives a bus signal that represents an incremental learning model fit to streaming data. The input port x receives a chunk of predictor data (observations), and the input port y receives a chunk of responses or labels for measuring the model performance. The output port IsWarm returns a value indicating if the model is warm, which means that it tracks performance metrics. The output port metrics returns the computed performance metrics. The optional input port w receives a chunk of observation weights.

Examples

Perform Incremental Learning and Track Performance Metrics Using Update Metrics Block

Perform Incremental Learning and Track Performance Metrics Using Update Metrics Block

Perform incremental learning and track performance metrics with the Update Metrics block.

Configure Simulink Template for Rate-Based Incremental Linear Regression

Configure Simulink Template for Rate-Based Incremental Linear Regression

Configure the Simulink Rate-Based Incremental Learning template to perform incremental linear regression.

Configure Simulink Template for Rate-Based Incremental Linear Classification

Configure Simulink Template for Rate-Based Incremental Linear Classification

Configure the Simulink Rate-Based Incremental Learning template to perform incremental linear classification.

Configure Simulink Template for Conditionally Enabled Incremental Linear Classification

Configure Simulink Template for Conditionally Enabled Incremental Linear Classification

Configure the Simulink Enabled Execution Incremental Learning template to perform incremental linear classification.

Configure Simulink Template for Conditionally Enabled Incremental Linear Regression

Configure Simulink Template for Conditionally Enabled Incremental Linear Regression

Configure the Simulink Enabled Execution Incremental Learning template to perform incremental linear regression.

Ports

Input

expand all

Incremental learning model (incrementalClassificationLinear, incrementalClassificationECOC, or incrementalRegressionLinear) fit to streaming data, specified as a bus signal (see Composite Signals (Simulink)).

Chunk of predictor data, specified as a numeric matrix. The orientation of the variables and observations is specified by Predictor data observation dimension. The default orientation is rows, which indicates that observations in the predictor data are oriented along the rows of x.

The length of the observation responses y and the number of observations in x must be equal; y(j) is the response of observation j (row or column) in x.

Note

The block supports only numeric input predictor data. If your input data includes categorical data, you must prepare an encoded version of the categorical data. Use dummyvar to convert each categorical variable to a numeric matrix of dummy variables. Then, concatenate all dummy variable matrices and any other numeric predictors. For more details, see Dummy Variables.

Data Types: single | double | half | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | Boolean | fixed point

Chunk of responses (labels) for measuring the model performance, specified as a numeric, logical, or enumerated vector.

The length of the observation responses y and the number of observations in x must be equal; y(j) is the response of observation j (row or column) in x.

For classification problems:

  • Each label must correspond to one row of the array.

  • If y contains a label that is in mdl.ClassNames, the block issues an error.

Data Types: single | double | half | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | Boolean | fixed point | enumerated

Chunk of observation weights, specified as a vector of positive values. The Update Metrics block weights the observations in x with the corresponding values in w. The size of w must be equal to the number of observations in x.

Dependencies

To enable this port, select the check box for Add input port for observation weights on the Main tab of the Block Parameters dialog box.

Data Types: single | double

Output

expand all

Flag indicating whether the incremental model tracks performance metrics, returned as logical 0 (false) or 1 (true).

ValueDescription
1 (true)The incremental model mdl is warm. Consequently, the block tracks performance metrics in the bus signal output.
0 (false)The block does not track performance metrics.

Model performance metrics updated during incremental learning, returned as a matrix with two columns and m rows, where m is the number of observations in x. Specify the performance metrics using Model performance metrics to track. The block ignores the metrics specified by the Metrics property of mdl.

The columns of metrics are:

  • Cumulative — Model performance from the time the model becomes warm (IsWarm is 1).

  • Window — Model performance evaluated over all observations within the window specified by the MetricsWindowSize property of mdl. The software updates Window after it processes MetricsWindowSize observations.

Data Types: matrix

Parameters

expand all

Main

Specify the name of a workspace variable that contains the configured incrementalRegressionLinear, incrementalClassificationLinear, or incrementalClassificationECOC model object.

  • The predictor data cannot include categorical predictors (logical, categorical, char, string, or cell). If you supply training data in a table, the predictors must be numeric (double or single). To include categorical predictors in a model, preprocess them by using dummyvar before fitting the model.

  • The NumPredictors property of the initial model must be a positive integer scalar, and must be equal to the number of predictors in x.

  • The ScoreTransform property of the initial model (classification only) cannot be "invlogit" or an anonymous function.

Programmatic Use

Block Parameter: InitialLearner
Type: workspace variable
Values: incrementalRegressionLinear model object, incrementalClassificationLinear model object, incrementalClassificationECOC model object
Default: "Mdl"

Loss function for cumulative and window performance metrics to track during incremental learning. For more details on loss functions, see loss.

If mdl is an incrementalClassificationLinear or incrementalClassificationECOC model object, you can specify one of the following:

NameDescription
binodevianceBinomial deviance
classiferrorClassification error
exponentialExponential
hingeHinge
logitLogistic
quadraticQuadratic

If mdl is an incrementalRegressionLinear model object, you can specify one of the following:

NameDescription
mseWeighted mean squared error
epsiloninsensitiveEpsilon insensitive loss

The Update Metrics block ignores the metrics specified by the Metric property of mdl.

Programmatic Use

Block Parameter: Metric
Type: character vector
Values: "classiferror" | "binodeviance" | "exponential" | "hinge" | "logit" | "quadratic" | "mse" | "epsiloninsensitive"
Default: "classiferror" (for incrementalClassificationLinear and incrementalClassificationECOC) | "epsiloninsensitive" (for incrementalRegressionLinear)

Select the check box to include the input port w for observation weights in the Update Metrics block.

Programmatic Use

Block Parameter: ShowInputWeights
Type: character vector
Values: "off" | "on"
Default: "off"

Specify the observation dimension of the predictor data. The default value is rows, which indicates that observations in the predictor data are oriented along the rows of x.

Programmatic Use

Block Parameter: ObservationsIn
Type: character vector
Values: "rows" | "columns"
Default: "rows"

Specify the discrete interval between sample time hits or specify another type of sample time, such as continuous (0) or inherited (–1). For more options, see Types of Sample Time (Simulink).

By default, the Update Metrics block inherits sample time based on the context of the block within the model.

Programmatic Use

Block Parameter: SystemSampleTime
Type: string scalar or character vector
Values: scalar
Default: "–1"

Data Types

Fixed-Point Operational Parameters

Specify the rounding mode for fixed-point operations. For more information, see Rounding Modes (Fixed-Point Designer).

Block parameters always round to the nearest representable value. To control the rounding of a block parameter, enter an expression into the mask field using a MATLAB® rounding function.

Programmatic Use

Block Parameter: RndMeth
Type: character vector
Values: "Ceiling" | "Convergent" | "Floor" | "Nearest" | "Round" | "Simplest" | "Zero"
Default: "Floor"

Specify whether overflows saturate or wrap.

ActionRationaleImpact on OverflowsExample

Select this check box (on).

Your model has possible overflow, and you want explicit saturation protection in the generated code.

Overflows saturate to either the minimum or maximum value that the data type can represent.

The maximum value that the int8 (signed 8-bit integer) data type can represent is 127. Any block operation result greater than this maximum value causes overflow of the 8-bit integer. With the check box selected, the block output saturates at 127. Similarly, the block output saturates at a minimum output value of –128.

Clear this check box (off).

You want to optimize the efficiency of your generated code.

You want to avoid overspecifying how a block handles out-of-range signals. For more information, see Troubleshoot Signal Range Errors (Simulink).

Overflows wrap to the appropriate value that the data type can represent.

The maximum value that the int8 (signed 8-bit integer) data type can represent is 127. Any block operation result greater than this maximum value causes overflow of the 8-bit integer. With the check box cleared, the software interprets the value causing the overflow as int8, which can produce an unintended result. For example, a block result of 130 (binary 1000 0010) expressed as int8 is –126.

Programmatic Use

Block Parameter: SaturateOnIntegerOverflow
Type: character vector
Values: "off" | "on"
Default: "off"

Select this parameter to prevent the fixed-point tools from overriding the data type you specify for the block. For more information, see Use Lock Output Data Type Setting (Fixed-Point Designer).

Programmatic Use

Block Parameter: LockScale
Type: character vector
Values: "off" | "on"
Default: "off"
Data Type

Specify the data type for the metrics output. The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType.

For more information about data types, see Control Data Types of Signals (Simulink).

Click the Show data type assistant button to display the Data Type Assistant, which helps you set the data type attributes. For more information, see Specify Data Types Using Data Type Assistant (Simulink).

Programmatic Use

Block Parameter: MetricsDataTypeStr
Type: character vector
Values: "double" | "single" | "half" | "int8" | "uint8" | "int16" | "uint16" | "int32" | "uint32" | "int64" | "uint64" | "boolean" | "fixdt(1,16,0)" | "fixdt(1,16,2^0,0)" | "<data type expression>"
Default: "double"

Specify the lower value of the metrics output range that Simulink® checks.

Simulink uses the minimum value to perform:

Note

The Metrics data type Minimum parameter does not saturate or clip the actual metrics output. To do so, use the Saturation (Simulink) block instead.

Programmatic Use

Block Parameter: MetricsOutMin
Type: character vector
Values: "[]" | scalar
Default: "[]"

Specify the upper value of the metrics output range that Simulink checks.

Simulink uses the maximum value to perform:

Note

The Metrics data type Maximum parameter does not saturate or clip the actual metrics output. To do so, use the Saturation (Simulink) block instead.

Programmatic Use

Block Parameter: MetricsOutMax
Type: character vector
Values: "[]" | scalar
Default: "[]"

Specify the data type for the response output from the predict block (IncrementalRegressionLinear Predict, IncrementalClassificationLinear Predict, or IncrementalClassificationECOC Predict) that is internal to the Update Metrics block. The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType.

When you select Inherit: auto, the block uses a rule that inherits a data type.

For more information about data types, see Control Data Types of Signals (Simulink).

Click the Show data type assistant button to display the Data Type Assistant, which helps you set the data type attributes. For more information, see Specify Data Types Using Data Type Assistant (Simulink).

Programmatic Use

Block Parameter: PredictDataTypeStr
Type: character vector
Values: "Inherit: auto" | "double" | "single" | "half" | "int8" | "uint8" | "int16" | "uint16" | "int32" | "uint32" | "int64" | "uint64" | "boolean" | "fixdt(1,16,0)" | "fixdt(1,16,2^0,0)" | "<data type expression>"
Default: "Inherit: auto"

Specify the lower value of the internal predict response output range that Simulink checks.

Simulink uses the minimum value to perform:

Programmatic Use

Block Parameter: PredictResponseOutMin
Type: character vector
Values: "[]" | scalar
Default: "[]"

Specify the upper value of the internal predict response output range that Simulink checks.

Simulink uses the maximum value to perform:

Programmatic Use

Block Parameter: PredictResponseOutMax
Type: character vector
Values: "[]" | scalar
Default: "[]"

Specify the data type for the loss value output used internally by the Update Metrics block to evaluate the cumulative loss and the loss over the window specified by the MetricsWindowSize property of mdl. The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType.

When you select Inherit: auto, the block uses a rule that inherits a data type.

For more information about data types, see Control Data Types of Signals (Simulink).

Click the Show data type assistant button to display the Data Type Assistant, which helps you set the data type attributes. For more information, see Specify Data Types Using Data Type Assistant (Simulink).

Programmatic Use

Block Parameter: LossDataTypeStr
Type: character vector
Values: "Inherit: auto" | "double" | "single" | "half" | "int8" | "uint8" | "int16" | "uint16" | "int32" | "uint32" | "int64" | "uint64" | "boolean" | "fixdt(1,16,0)" | "fixdt(1,16,2^0,0)" | "<data type expression>"
Default: "Inherit: auto"

Specify the lower value of the internal loss output range that Simulink checks.

Simulink uses the minimum value to perform:

Programmatic Use

Block Parameter: LossOutMin
Type: character vector
Values: "[]" | scalar
Default: "[]"

Specify the upper value of the internal loss output range that Simulink checks.

Simulink uses the maximum value to perform:

Programmatic Use

Block Parameter: LossOutMax
Type: character vector
Values: "[]" | scalar
Default: "[]"

Specify the data type for the additional response output from the predict block (IncrementalRegressionLinear Predict, IncrementalClassificationLinear Predict, or IncrementalClassificationECOC Predict) that is internal to the Update Metrics block. The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType.

When you select Inherit: Inherit via internal rule, the block uses an internal rule to determine the additional predict data type. The internal rule chooses a data type that optimizes numerical accuracy, performance, and generated code size, while taking into account the properties of the embedded target hardware. The software cannot always optimize efficiency and numerical accuracy at the same time.

For more information about data types, see Control Data Types of Signals (Simulink).

Click the Show data type assistant button to display the Data Type Assistant, which helps you set the data type attributes. For more information, see Specify Data Types Using Data Type Assistant (Simulink).

Programmatic Use

Block Parameter: AdditionalPredictDataTypeStr
Type: character vector
Values: "Inherit: Inherit via internal rule" | "double" | "single" | "half" | "int8" | "uint8" | "int16" | "uint16" | "int32" | "uint32" | "int64" | "uint64" | "boolean" | "fixdt(1,16,0)" | "fixdt(1,16,2^0,0)" | "<data type expression>"
Default: "Inherit: Inherit via internal rule"

Specify the lower value of the additional predict output range that Simulink checks.

Simulink uses the minimum value to perform:

Programmatic Use

Block Parameter: AdditionalPredictOutMin
Type: character vector
Values: "[]" | scalar
Default: "[]"

Specify the upper value of the additional predict output range that Simulink checks.

Simulink uses the maximum value to perform:

Programmatic Use

Block Parameter: AdditionalPredictOutMax
Type: character vector
Values: "[]" | scalar
Default: "[]"

Specify the data type for additional operations on the loss value used internally by the Update Metrics block to evaluate the cumulative loss and the loss over the window specified by the MetricsWindowSize property of mdl. The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType.

When you select Inherit: Inherit via internal rule, the block uses an internal rule to determine the additional loss data type. The internal rule chooses a data type that optimizes numerical accuracy, performance, and generated code size, while taking into account the properties of the embedded target hardware. The software cannot always optimize efficiency and numerical accuracy at the same time.

For more information about data types, see Control Data Types of Signals (Simulink).

Click the Show data type assistant button to display the Data Type Assistant, which helps you set the data type attributes. For more information, see Specify Data Types Using Data Type Assistant (Simulink).

Programmatic Use

Block Parameter: AdditionalLossDataTypeStr
Type: character vector
Values: "Inherit: Inherit via internal rule" | "double" | "single" | "half" | "int8" | "uint8" | "int16" | "uint16" | "int32" | "uint32" | "int64" | "uint64" | "boolean" | "fixdt(1,16,0)" | "fixdt(1,16,2^0,0)" | "<data type expression>"
Default: "Inherit: Inherit via internal rule"

Specify the lower value of the additional loss output range that Simulink checks.

Simulink uses the minimum value to perform:

Programmatic Use

Block Parameter: AdditionalLossOutMin
Type: character vector
Values: "[]" | scalar
Default: "[]"

Specify the upper value of the additional loss output range that Simulink checks.

Simulink uses the maximum value to perform:

Programmatic Use

Block Parameter: AdditionalLossOutMax
Type: character vector
Values: "[]" | scalar
Default: "[]"

Block Characteristics

Data Types

Boolean | double | enumerated | fixed point | half | integer | single

Direct Feedthrough

yes

Multidimensional Signals

no

Variable-Size Signals

no

Zero-Crossing Detection

no

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using Simulink® Coder™.

Fixed-Point Conversion
Design and simulate fixed-point systems using Fixed-Point Designer™.

Version History

Introduced in R2023b

See Also

Blocks

Objects

Functions

Topics