This example shows how to perform transfer learning interactively using the Deep Network Designer app.
Transfer learning is the process of taking a pretrained deep learning network and fine-tuning it to learn a new task. Using transfer learning is usually faster and easier than training a network from scratch. You can quickly transfer learned features to a new task using a smaller amount of data.
Use Deep Network Designer to perform transfer learning for image classification by following these steps:
Open the Deep Network Designer app and choose a pretrained network.
Import the new data set.
Replace the final layers with new layers adapted to the new data set.
Set learning rates so that learning is faster in the new layers than in the transferred layers.
Train the network using Deep Network Designer, or export the network for training at the command line.
In the workspace, extract the MathWorks Merch data set. This is a small data set containing 75 images of MathWorks merchandise, belonging to five different classes (cap, cube, playing cards, screwdriver, and torch).
To open Deep Network Designer, on the Apps tab, under Machine Learning and Deep Learning, click the app icon. Alternatively, you can open the app from the command line:
Deep Network Designer provides a selection of pretrained image classification networks that have learned rich feature representations suitable for a wide range of images. Transfer learning works best if your images are similar to the images originally used to train the network. If your training images are natural images like those in the ImageNet database, then any of the pretrained networks is suitable. For a list of available networks and how to compare them, see Pretrained Deep Neural Networks.
If your data is very different from the ImageNet data—for example, if you have tiny images, spectrograms, or nonimage data—training a new network might be better. For an example showing how to train a network using nonimage data, see Create Simple Sequence Classification Network Using Deep Network Designer.
SqueezeNet does not require an additional support package. For other pretrained networks, if you do not have the required support package installed, then the app provides the Install option.
Select SqueezeNet from the list of pretrained networks and click Open.
Deep Network Designer displays a zoomed-out view of the whole network in the Designer pane.
Explore the network plot. To zoom in with the mouse, use Ctrl+scroll wheel. To pan, use the arrow keys, or hold down the scroll wheel and drag the mouse. Select a layer to view its properties. Deselect all layers to view the network summary in the Properties pane.
To load the data into Deep Network Designer, on the Data tab, click Import Data. The Import Data dialog box opens.
In the Data source list, select Folder. Click Browse and select the extracted MerchData folder.
You can choose to apply image augmentation to your training data. The Deep Network Designer app provides the following augmentation options:
Random reflection in the x-axis
Random reflection in the y-axis
Random horizontal translation
Random vertical translation
You can effectively increase the amount of training data by applying randomized augmentation to your data. Augmentation also enables you to train networks to be invariant to distortions in image data. For example, you can add randomized rotations to input images so that a network is invariant to the presence of rotation in input images.
For this example, apply a random reflection in the x-axis, a random rotation from the range [-90,90] degrees, and a random rescaling from the range [1,2].
You can also choose to import validation data either by splitting it from the training data, or by importing it from another source. Validation estimates model performance on new data compared to the training data, and helps you to monitor performance and protect against overfitting.
For this example, use 30% of the images for validation.
Click Import to import the data into Deep Network Designer.
Using Deep Network Designer, you can visually inspect the distribution of the training and validation data in the Data pane. You can see that, in this example, there are five classes in the data set.
Edit the network in the Designer pane to specify a new number of classes in your data. To prepare the network for transfer learning, replace the last learnable layer and the final classification layer.
To use a pretrained network for transfer learning, you must change the number of classes to match your new data set. First, find the last learnable layer in the network. For SqueezeNet, the last learnable layer is the last convolutional layer, '
conv10'. In this case, replace the convolutional layer with a new convolutional layer with the number of filters equal to the number of classes.
Drag a new
convolutional2dLayer onto the canvas. To match the original convolutional layer, set
NumFilters property defines the number of classes for classification problems. Change
NumFilters to the number of classes in the new data, in this example,
Change the learning rates so that learning is faster in the new layer than in the transferred layers by setting set
BiasLearnRateFactor to 10.
Delete the last 2-D convolutional layer and connect your new layer instead.
For transfer learning, you need to replace the output layer. Scroll to the end of the Layer Library and drag a new
classificationLayer onto the canvas. Delete the original
output layer and connect your new layer in its place.
For a new output layer, you do not need to set the
OutputSize. At training time, Deep Network Designer automatically sets the output classes of the layer from the data.
To check that the network is ready for training, click Analyze. If the Deep Learning Network Analyzer reports zero errors, then the edited network is ready for training.
The Deep Network Designer app enables you to train image classification networks imported or created in the app. For other types of data, you can construct the network in Deep Network Designer and then export the network for training. For an example of showing how to use Deep Network Designer to construct a sequence network and then export the results for training, see Create Simple Sequence Classification Network Using Deep Network Designer.
To train the network with the default settings, on the Training tab, click Train. The default training options are better suited for large data sets, for small data sets reduce
If you want greater control over the training, click Training Options and choose the settings to train with.
InitialLearnRate to a small value to slow down learning in the transferred layers.
ValidationFrequency so that the accuracy on the validation data is calculated once every epoch.
Specify a small number of epochs. An epoch is a full training cycle on the entire training data set. For transfer learning, you do not need to train for as many epochs.
Specify the mini-batch size, that is, how many images to use in each iteration. To ensure the whole data set is used during each epoch, set the mini-batch size to evenly divide the number of training samples.
For this example, set
8. As there are 55 observations, set
MiniBatchSize to 11 to divide the training data evenly and ensure you use the whole data set during each epoch. For more information on selecting training options, see
To train the network with the specified training options, click Close and then click Train.
Deep Network Designer allows you to visualize and monitor training progress. You can then edit the training options and retrain the network, if required.
To export the network architecture with the trained weights, on the Training tab, select Export > Export Trained Network and Results. Deep Network Designer exports the trained network as the variable
trainedNetwork_1 and the training info as the variable
trainInfoStruct_1 = struct with fields: TrainingLoss: [1×40 double] TrainingAccuracy: [1×40 double] ValidationLoss: [4.8267 NaN NaN NaN 2.1034 NaN NaN NaN NaN 1.2332 NaN NaN NaN NaN 0.6614 NaN NaN NaN NaN 0.4312 NaN NaN NaN NaN 0.2878 NaN NaN NaN NaN 0.2472 NaN NaN NaN NaN 0.2647 NaN NaN NaN NaN 0.2893] ValidationAccuracy: [10 NaN NaN NaN 10 NaN NaN NaN NaN 40 NaN NaN NaN NaN 80 NaN NaN NaN NaN 90 NaN NaN NaN NaN 95 NaN NaN NaN NaN 95 NaN NaN NaN NaN 95 NaN NaN NaN NaN 95] BaseLearnRate: [1×40 double] FinalValidationLoss: 0.2893 FinalValidationAccuracy: 95
You can also generate MATLAB code, which recreates the network and the training options used. On the Training tab, select Export > Generate Code for Training. Examine the MATLAB code to learn how to programmatically prepare the data for training, create the network architecture, and train the network.
Load a new image to classify using the trained network.
I = imread("MerchDataTest.jpg");
Deep Network Designer resizes the images during training to match the network input size. To view the network input size, go to the Designer pane and select the
imageInputLayer (first layer). This network has an input size of 227-by-227.
Resize the test image to match the network input size.
I = imresize(I, [227 227]);
Classify the test image using the trained network.
[YPred,probs] = classify(trainedNetwork_1,I); imshow(I) label = YPred; title(string(label) + ", " + num2str(100*max(probs),3) + "%");