anomalyDetectionMetrics

Anomaly detection metrics

Since R2022b

expand all in page

    Description

    An anomalyDetectionMetrics object encapsulates anomaly detection quality metrics for a set of images. Images of anomalies belong to the positive (true) class. Normal images belong to the negative (false) class.

    Note

    This functionality requires the Automated Visual Inspection Library for Computer Vision Toolbox™. You can install the Automated Visual Inspection Library for Computer Vision Toolbox from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

    Creation

    Create an anomalyDetectionMetrics object using the evaluateAnomalyDetection function.

    Properties

    expand all

    This property is read-only.

    Confusion matrix, specified as a 2-by-2 table. Each table element (i, j) is the count of images known to belong to class i but predicted to belong to class j.

    This property is read-only.

    Normalized confusion matrix, specified as a 2-by-2 table. NormalizedConfusionMatrix represents a confusion matrix normalized by the number of images known to belong to each class. Each table element (i, j) is the count of images known to belong to class i but predicted to belong to class j, divided by the total number of images predicted in class i. Elements are in the range [0, 1].

    This property is read-only.

    Metrics aggregated over the data set, specified as a 1-by-8 table with columns corresponding to these metrics:

    • GlobalAccuracy — Ratio of correctly classified images to total images, regardless of class

    • MeanAccuracy — Ratio of correctly classified images of each class to total images, averaged over all classes

    • Precision — Positive predictive value

    • Recall — True positive (TP) rate

    • Specificity — True negative (TN) rate

    • F1Score — Harmonic mean of precision and recall

    • FalsePositiveRate — False positive (FP) rate, or fallout

    • FalseNegativeRate — False negative (FN) rate, or miss

    This property is read-only.

    Anomaly detection metrics for each class, specified as a 2-by-2 table. The table has rows "Normal" and "Anomaly" and columns corresponding to these metrics:

    • Accuracy — Fraction of correctly classified images for each class (normal or anomaly).

    • AccuracyPerSubClass — Fraction of correctly classified images for each subclass within the normal and anomaly classes. The value of AccuracyPerSubClass is a table with C rows and one column, where C is the number of subclasses.

    Examples

    collapse all

    This example uses:

    Open Live Script

    Load test images and corresponding labels, then create a datastore that reads the test data. The data set consists of grayscale images of handwritten digits 0–9.

    [Xtest,gtLabels] = digitTest4DArrayData;
dsTest = arrayDatastore(Xtest,IterationDimension=4);

    Load a pretrained FCDD anomaly detector. This detector has been trained to classify the digit 8 as normal and all other digits as anomalies. Therefore, specify the set of anomaly labels as the set of digits between 0 and 9, excluding 8.

    load("digit8AnomalyDetector.mat");
anomalyLabels = setdiff(string(0:9),"8");

    Classify each test image as normal or anomalous.

    predLabels = classify(detector,dsTest);

    Calculate metrics for the anomaly detection results.

    metrics = evaluateAnomalyDetection(predLabels,gtLabels,anomalyLabels)
    Evaluating anomaly detection results
------------------------------------
* Finalizing... Done.
* Data set metrics:

    GlobalAccuracy    MeanAccuracy    Precision    Recall    Specificity    F1Score    FalsePositiveRate    FalseNegativeRate
    ______________    ____________    _________    ______    ___________    _______    _________________    _________________

        0.7662           0.839         0.98971     0.748        0.93        0.85204          0.07                 0.252      

    metrics = 
  anomalyDetectionMetrics with properties:

              ConfusionMatrix: [2×2 table]
    NormalizedConfusionMatrix: [2×2 table]
               DataSetMetrics: [1×8 table]
                 ClassMetrics: [2×2 table]

    Display the confusion matrix.

    cm = metrics.ConfusionMatrix
    cm=2×2 table
               Normal    Anomaly
               ______    _______

    Normal       465        35  
    Anomaly     1134      3366

    Version History

    Introduced in R2022b

    See Also