Esta página aún no se ha traducido para esta versión. Puede ver la versión más reciente de esta página en inglés.

ClassificationECOC clase

Superclases: CompactClassificationECOC

Multiclass model for support vector machines or other classifiers

Description

ClassificationECOC is an error-correcting output codes (ECOC) classifier for multiclass learning by reduction to multiple binary classifiers such as support vector machines (SVMs). Train a ClassificationECOC classifier using fitcecoc and the training data.

Trained ClassificationECOC classifiers store the training data, parameter values, prior probabilities, and coding matrices. You can use these classifiers to:

  • Estimate resubstitution predictions. For details, see resubPredict.

  • Predict labels or posterior probabilities for new data. For details, see predict.

Construction

Create a ClassificationECOC object by using fitcecoc.

Propiedades

expandir todo

Trained binary learners, specified as a cell vector of model objects. BinaryLearners has as many elements as classes in Y.

The software trained BinaryLearner{j} according to the binary problem specified by CodingMatrix(:,j). For example, for multiclass learning using SVM learners, each element of BinaryLearners is a CompactClassificationSVM classifier.

Tipos de datos: cell

Binary learner loss function, specified as a character vector representing the loss function name.

If you train using binary learners that use different loss functions, then the software sets BinaryLoss to 'hamming'. To potentially increase accuracy, set a different binary loss function than this default during prediction or loss computation using the BinaryLoss name-value pair argument of predict or loss.

Tipos de datos: char

Binary learner class labels, specified as a numeric matrix. BinaryY is a NumObservations-by-L matrix, where L is the number of binary learners (size(CodingMatrix,2)).

Elements of BinaryY are -1, 0, or 1, and the value corresponds to a dichotomous class assignment. This table describes how learner j assigns observation k to a dichotomous class corresponding to the value of BinaryY(k,j).

ValueDichotomous Class Assignment
–1Learner j assigns observation k to a negative class.
0Before training, learner j removes observation k from the data set.
1Learner j assigns observation k to a positive class.

Tipos de datos: double

Categorical predictor indices, specified as a vector of positive integers. CategoricalPredictors contains index values corresponding to the columns of the predictor data that contain categorical predictors. If none of the predictors are categorical, then this property is empty ([]).

Tipos de datos: single | double

Unique class labels used in training, specified as a categorical or character array, logical or numeric vector, or cell array of character vectors. ClassNames has the same data type as the class labels Y. (The software treats string arrays as cell arrays of character vectors.) ClassNames also determines the class order.

Tipos de datos: categorical | char | logical | single | double | cell

Class-assignment codes for the binary learners, specified as a numeric matrix. CodingMatrix is a K-by-L matrix, where K is the number of classes and L is the number of binary learners.

Elements of CodingMatrix are -1, 0, or 1, and the value corresponds to a dichotomous class assignment. This table describes the meaning of CodingMatrix(i,j), that is, the class that learner j assigns to observations in class i.

ValueDichotomous Class Assignment
–1Learner j assigns observations in class i to a negative class.
0Before training, learner j removes observations in class i from the data set.
1Learner j assigns observations in class i to a positive class.

Tipos de datos: double | single | int8 | int16 | int32 | int64

Coding design name, specified as a character vector. For more details, see Coding Design.

Tipos de datos: char

Esta propiedad es de solo lectura.

Misclassification costs, specified as a square numeric matrix. Cost has K rows and columns, where K is the number of classes.

Cost(i,j) is the cost of classifying a point into class j if its true class is i. The order of the rows and columns of Cost corresponds to the order of the classes in ClassNames.

fitcecoc incorporates misclassification costs differently among different types of binary learners.

Tipos de datos: double

Expanded predictor names, stored as a cell array of character vectors.

If the model uses encoding for categorical variables, then ExpandedPredictorNames includes the names that describe the expanded variables. Otherwise, ExpandedPredictorNames is the same as PredictorNames.

Tipos de datos: cell

Description of the cross-validation optimization of hyperparameters, stored as a BayesianOptimization object or a table of hyperparameters and associated values. Nonempty when the OptimizeHyperparameters name-value pair is nonempty at creation. Value depends on the setting of the HyperparameterOptimizationOptions name-value pair at creation:

  • 'bayesopt' (default) — Object of class BayesianOptimization

  • 'gridsearch' or 'randomsearch' — Table of hyperparameters used, observed objective function values (cross-validation loss), and rank of observations from lowest (best) to highest (worst)

Tipos de datos: table

Binary learner weights, specified as a numeric row vector. LeanerWeights has length equal to the number of binary learners (size(CodingMatrix,2)).

LearnerWeights(j) is the sum of the observation weights that binary learner j used to train its classifier.

The software uses LearnerWeights to fit posterior probabilities by minimizing the Kullback-Leibler divergence.

Tipos de datos: double | single

Parameter values, e.g., the name-value pair argument values, used to train the ECOC classifier, specified as an object. ModelParameters does not contain estimated parameters.

Access properties of ModelParameters using dot notation. For example, list the templates containing parameters the binary learners using Mdl.ModelParameters.BinaryLearner.

Number of observations in the training data, specified as a positive numeric scalar.

Tipos de datos: double

Predictor names in order of their appearance in the predictor data X, specified as a cell array of character vectors. The length of PredictorNames is equal to the number of columns in X.

Tipos de datos: cell

Esta propiedad es de solo lectura.

Prior class probabilities, specified as a numeric vector. Prior has as many elements as there are classes in ClassNames, and the order of the elements corresponds to the elements of ClassNames.

fitcecoc incorporates misclassification costs differently among different types of binary learners.

Tipos de datos: double

Response variable name, specified as a character vector.

Tipos de datos: char

Score transformation function to apply to predicted scores, specified as a function name or function handle.

To change the score transformation function to function, for example, use dot notation.

  • For a built-in function, enter this code and replace function with a value in the table.

    Mdl.ScoreTransform = 'function';

    ValueDescription
    'doublelogit'1/(1 + e–2x)
    'invlogit'log(x / (1–x))
    'ismax'Sets the score for the class with the largest score to 1, and sets the scores for all other classes to 0
    'logit'1/(1 + ex)
    'none' or 'identity'x (no transformation)
    'sign'–1 for x < 0
    0 for x = 0
    1 for x > 0
    'symmetric'2x – 1
    'symmetricismax'Sets the score for the class with the largest score to 1, and sets the scores for all other classes to –1
    'symmetriclogit'2/(1 + ex) – 1

  • For a MATLAB® function or a function that you define, enter its function handle.

    Mdl.ScoreTransform = @function;

    function must accept a matrix (the original scores) and return a matrix of the same size (the transformed scores).

Tipos de datos: char | function_handle

Observation weights used to train the classifier, specified as a numeric vector. W has NumObservations elements.

The software normalizes the weights used for training so that nansum(W) is 1.

Tipos de datos: single | double

Unstandardized predictor data used to train the classifier, specified as a numeric matrix. X is a NumObservations-by-p matrix, where p is the number of predictors.

Each row of X corresponds to one observation, and each column corresponds to one variable.

Tipos de datos: single | double

Observed class labels used to train the classifier, specified as a categorical or character array, logical or numeric vector, or cell array of character vectors. Y has NumObservations elements, and is the same data type as the input argument Y of fitcecoc. (The software treats string arrays as cell arrays of character vectors.)

Each row of Y represents the observed classification of the corresponding row of Tbl or X.

Methods

compactCompact multiclass, error-correcting output codes model
crossvalCross-validate multiclass, error-correcting output codes model
resubEdgeClassification edge by resubstitution for multiclass, error-correcting output codes model
resubLossClassification loss by resubstitution for multiclass, error-correcting output codes model
resubMarginClassification margins by resubstitution for multiclass, error-correcting output codes model
resubPredictPredict resubstitution responses for multiclass, error-correcting output codes model

Inherited Methods

discardSupportVectorsDiscard support vectors of linear support vector machine binary learners
edgeClassification edge for multiclass, error-correcting output codes model
lossClassification loss for multiclass, error-correcting output codes model
marginClassification margins for multiclass, error-correcting output codes model
predictPredict labels using multiclass, error-correcting output codes model
selectModelsChoose subset of regularized linear classification models

Copy Semantics

Value. To learn how value classes affect copy operations, see Copying Objects (MATLAB).

Ejemplos

contraer todo

Train an error-correcting output codes (ECOC) multiclass model using support vector machine (SVM) binary learners.

Load Fisher's iris data set.

load fisheriris
X = meas;
Y = species;

Train an ECOC multiclass model using the default options.

Mdl = fitcecoc(X,Y)
Mdl = 
  ClassificationECOC
             ResponseName: 'Y'
    CategoricalPredictors: []
               ClassNames: {'setosa'  'versicolor'  'virginica'}
           ScoreTransform: 'none'
           BinaryLearners: {3x1 cell}
               CodingName: 'onevsone'


  Properties, Methods

Mdl is a ClassificationECOC model. By default, fitcecoc uses SVM binary learners, and uses a one-versus-one coding design. You can access Mdl properties using dot notation.

Display the coding design matrix.

Mdl.ClassNames
ans = 3x1 cell array
    {'setosa'    }
    {'versicolor'}
    {'virginica' }

CodingMat = Mdl.CodingMatrix
CodingMat = 3×3

     1     1     0
    -1     0     1
     0    -1    -1

A one-versus-one coding design on three classes yields three binary learners. Columns of CodingMat correspond to learners, and rows correspond to classes. The class order corresponds to the order in Mdl.ClassNames. For example, CodingMat(:,1) is [1; -1; 0] and indicates that the software trains the first SVM binary learner using all observations classified as 'setosa' and 'versicolor'. Since 'setosa' corresponds to 1, it is the positive class, and since 'versicolor' corresponds to -1, it is the negative class.

You can access each binary learner using cell indexing and dot notation.

Mdl.BinaryLearners{1}                % The first binary learner
ans = 
  classreg.learning.classif.CompactClassificationSVM
             ResponseName: 'Y'
    CategoricalPredictors: []
               ClassNames: [-1 1]
           ScoreTransform: 'none'
                     Beta: [4x1 double]
                     Bias: 1.4505
         KernelParameters: [1x1 struct]


  Properties, Methods

Compute the in-sample classification error.

isLoss = resubLoss(Mdl)
isLoss = 0.0067

The classification error is small, but the classifier might have been overfit. You can cross-validate the classifier using crossval.

Access properties of binary learners, such as estimated parameters, using dot notation.

Load Fisher's iris data set. Use the petal dimensions as predictors.

load fisheriris
X = meas(:,3:4);
Y = species;

Train an ECOC classifier using the SVM binary learners and the default coding design (one-versus-one). Standardize the predictors and save the support vectors.

t = templateSVM('Standardize',1,'SaveSupportVectors',true);
predictorNames = {'petalLength','petalWidth'};
responseName = 'irisSpecies';
classNames = {'setosa','versicolor','virginica'}; % Specify class order
Mdl = fitcecoc(X,Y,'Learners',t,'ResponseName',responseName,...
    'PredictorNames',predictorNames,'ClassNames',classNames)
Mdl = 
  ClassificationECOC
           PredictorNames: {'petalLength'  'petalWidth'}
             ResponseName: 'irisSpecies'
    CategoricalPredictors: []
               ClassNames: {'setosa'  'versicolor'  'virginica'}
           ScoreTransform: 'none'
           BinaryLearners: {3x1 cell}
               CodingName: 'onevsone'


  Properties, Methods

t is a template object that contains options for SVM classification. All properties are empty ([]), except for Method, SaveSupportVectors, StandardizeData, and Type. The function fitcecoc uses the default values of the empty properties. Mdl is a ClassificationECOC classifier. You can access properties of Mdl using dot notation.

Display the class names and the coding design matrix.

Mdl.ClassNames
ans = 3x1 cell array
    {'setosa'    }
    {'versicolor'}
    {'virginica' }

Mdl.CodingMatrix
ans = 3×3

     1     1     0
    -1     0     1
     0    -1    -1

L = size(Mdl.CodingMatrix,2); % Number of SVMs

The columns correspond to SVM binary learners, and the rows correspond to the distinct classes. The row order corresponds to the order of the ClassNames property of Mdl. For each column:

  • A 1 indicates that fitcecoc trained the SVM using observations in the corresponding class as members of the positive group.

  • A -1 indicates that fitcecoc trained the SVM using observations in the corresponding class as members of the negative group.

  • A 0 indicates that the SVM did not use observations in the corresponding class.

For example, fitcecoc assigns all observations to 'setosa' or 'versicolor', but not 'virginica', in the first SVM.

You can access properties of the SVMs using cell subscripting and dot notation. Store the standardized support vectors of each SVM. Unstandardize the support vectors.

sv = cell(L,1); % Preallocate for support vector indices
for j = 1:L
    SVM = Mdl.BinaryLearners{j};
    sv{j} = SVM.SupportVectors;
    sv{j} = bsxfun(@plus,bsxfun(@times,sv{j},SVM.Sigma),SVM.Mu);
end

sv is a cell array of matrices containing the unstandardized support vectors for the SVMs.

Plot the data, and identify the support vectors.

figure
h = zeros(3 + L,1); % Preallocate for handles
h(1:3) = gscatter(X(:,1),X(:,2),Y);
hold on
markers = {'ko','ro','bo'};
for j = 1:L
    svs = sv{j};
    h(j + 3) = plot(svs(:,1),svs(:,2),markers{j},...
        'MarkerSize',10 + (j - 1)*3);
end
title('Fisher''s Iris -- ECOC Support Vectors')
xlabel(predictorNames{1})
ylabel(predictorNames{2})
legend(h,[classNames,{'Support vectors - SVM 1',...
    'Support vectors - SVM 2','Support vectors - SVM 3'}],...
    'Location','Best')
hold off

You can pass Mdl to:

  • predict to classify new observations.

  • resubLoss to estimate the in-sample classification error.

  • crossval to perform 10-fold cross-validation.

Load Fisher's iris data set. Specify the predictor data X and the response data Y.

load fisheriris
X = meas;
Y = species;
rng(1); % For reproducibility

Create an SVM template, and standardize the predictors.

t = templateSVM('Standardize',1)
t = 
Fit template for classification SVM.

                     Alpha: [0x1 double]
             BoxConstraint: []
                 CacheSize: []
             CachingMethod: ''
                ClipAlphas: []
    DeltaGradientTolerance: []
                   Epsilon: []
              GapTolerance: []
              KKTTolerance: []
            IterationLimit: []
            KernelFunction: ''
               KernelScale: []
              KernelOffset: []
     KernelPolynomialOrder: []
                  NumPrint: []
                        Nu: []
           OutlierFraction: []
          RemoveDuplicates: []
           ShrinkagePeriod: []
                    Solver: ''
           StandardizeData: 1
        SaveSupportVectors: []
            VerbosityLevel: []
                   Version: 2
                    Method: 'SVM'
                      Type: 'classification'

t is an SVM template. Most of the template object's properties are empty. When training the ECOC classifier, the software sets the applicable properties to their default values.

Train the ECOC classifier, and specify the class order.

Mdl = fitcecoc(X,Y,'Learners',t,...
    'ClassNames',{'setosa','versicolor','virginica'});

Mdl is a ClassificationECOC classifier. You can access its properties using dot notation.

Cross-validate Mdl using 10-fold cross-validation.

CVMdl = crossval(Mdl);

CVMdl is a ClassificationPartitionedECOC cross-validated ECOC classifier.

Estimate the classification error.

loss = kfoldLoss(CVMdl)
loss = 0.0400

The classification error is 4%, which indicates that the ECOC classifier generalizes fairly well.

Más acerca de

expandir todo

Algoritmos

expandir todo

Alternatives

You can use these alternative algorithms to train a multiclass model:

References

[1] Fürnkranz, Johannes. “Round Robin Classification.” Journal of Machine Learning Research, Vol. 2, 2002, pp. 721–747.

[2] Escalera, S., O. Pujol, and P. Radeva. “Separability of ternary codes for sparse designs of error-correcting output codes.” Pattern Recognition Letters, Vol. 30, Issue 3, 2009, pp. 285–297.

Capacidades ampliadas

Introducido en R2014b