Error-correcting output codes learner template
an error-correcting output codes (ECOC) classification learner template.
t = templateECOC()
If you specify a default template, then the software uses default values for all input arguments during training.
For example, you can specify a coding design, whether to fit posterior probabilities, or the types of binary learners.
If you display
t in the Command Window, then
all options appear empty (
), except those that
you specify using name-value pair arguments. During training, the
software uses default values for empty options.
templateECOC to create a default ECOC template.
t = templateECOC()
t = Fit template for classification ECOC. BinaryLearners: '' Coding: '' FitPosterior:  Options:  VerbosityLevel:  NumConcurrent:  Version: 1 Method: 'ECOC' Type: 'classification'
All properties of the template object are empty except for
Type. When you pass
testckfold, the software fills in the empty properties with their respective default values. For example, the software fills the
BinaryLearners property with
'SVM'. For details on other default values, see
t is a plan for an ECOC learner. When you create it, no computation occurs. You can pass
testckfold to specify a plan for an ECOC classification model to statistically compare with another model.
One way to select predictors or features is to train two models where one that uses a subset of the predictors that trained the other. Statistically compare the predictive performances of the models. If there is sufficient evidence that model trained on fewer predictors performs better than the model trained using more of the predictors, then you can proceed with a more efficient model.
Load Fisher's iris data set. Plot all 2-dimensional combinations of predictors.
load fisheriris d = size(meas,2); % Number of predictors pairs = combnk(1:d,2); figure; for j = 1:size(pairs,1); subplot(3,2,j); gscatter(meas(:,pairs(j,1)),meas(:,pairs(j,2)),species); xlabel(sprintf('meas(:,%d)',pairs(j,1))); ylabel(sprintf('meas(:,%d)',pairs(j,2))); legend off; end
Based on the scatterplot,
meas(:,4) seem like they separate the groups well.
Create an ECOC template. Specify to use a one-versus-all coding design.
t = templateECOC('Coding','onevsall');
By default, the ECOC model uses linear SVM binary learners. You can choose other, supported algorithms by specifying them using the
'Learners' name-value pair argument.
Test whether an ECOC model that is just trained using predictors 3 and 4 performs at most as well as an ECOC model that is trained using all predictors. Rejecting this null hypothesis means that the ECOC model trained using predictors 3 and 4 performs better than the ECOC model trained using all predictors. Suppose represents the classification error of the ECOC model trained using predictors 3 and 4 and represents the classification error of the ECOC model trained using all predictors, then the test is:
testckfold conducts a 5-by-2 k-fold F test, which is not appropriate as a one-tailed test. Specify to conduct a 5-by-2 k-fold t test.
rng(1); % For reproducibility [h,pValue] = testckfold(t,t,meas(:,pairs(1,:)),meas,species,... 'Alternative','greater','Test','5x2t')
h = logical 0
pValue = 0.8940
h = 0 indicates that there is not enough evidence to suggest that the model trained using predictors 3 and 4 is more accurate than the model trained using all predictors.
comma-separated pairs of
the argument name and
Value is the corresponding value.
Name must appear inside quotes. You can specify several name and value
pair arguments in any order as
'Coding','ternarycomplete','FitPosterior',true,'Learners','tree'specifies a ternary complete coding design, to transform scores to posterior probabilities, and to grow classification trees for all binary learners.
t— ECOC classification template
ECOC classification template, returned as a template object.
specify how to create an ECOC classifier whose predictive performance
you want to compare with another classifier.
If you display
t to the Command Window, then
all, unspecified options appear empty (
the software replaces empty options with their corresponding default
values during training.