Build Networks with Deep Network Designer

Build and edit deep learning networks interactively using the Deep Network Designer app. Using this app, you can import networks or build a network from scratch, view and edit layer properties, combine networks, and generate code to create the network architecture. You can then train your network using Deep Network Designer, or export the network for training at the command line.

You can use Deep Network Designer for a range of network construction tasks:

• Transfer Learning

• Image Classification

• Sequence Classification

• Numeric Data Classification

• Convert Classification Network into Regression Network

• Multiple-Input and Multiple-Output Networks

• Deep Networks

• Advanced Deep Learning Applications

• dlnetwork for Custom Training Loops

Assemble a network by dragging blocks from the Layer Library and connecting them. To quickly search for layers, use the Filter layers search box in the Layer Library pane.

You can add layers from the workspace to the network in the Designer pane.

1. Click New.

2. Pause on From Workspace and click Import.

3. Choose the layers or network to import and click OK.

4. Click Add to add the layers or network to the Designer pane.

You can also load pretrained networks by clicking New and selecting them from the start page.

To view and edit layer properties, select a layer. Click the help icon next to the layer name for information on the layer properties.

For information on all layer properties, click the layer name in the table on the List of Deep Learning Layers page.

Once you have constructed your network, you can analyze it to check for errors. For more information, see Check Network.

Transfer Learning

Transfer learning is commonly used in deep learning applications. You can take a pretrained network and use it as a starting point to learn a new task. Fine-tuning a network with transfer learning is usually much faster and easier than training a network with randomly initialized weights from scratch. You can quickly transfer learned features to a new task using a smaller number of training images.

Deep Network Designer has a selection of pretrained networks suitable for transfer learning with image data.

Load Pretrained Network

Open the app and select a pretrained network. You can also load a pretrained network by selecting the Designer tab and clicking New. If you need to download the network, pause on the network and click Install to open the Add-On Explorer.

Tip

To get started, try choosing one of the faster networks, such as SqueezeNet or GoogLeNet. Once you gain an understanding of which settings work well, try a more accurate network, such as Inception-v3 or a ResNet, and see if that improves your results. For more information on selecting a pretrained network, see Pretrained Deep Neural Networks.

Adapt Pretrained Network

To prepare the network for transfer learning, replace the last learnable layer and the final classification layer.

• If the last learnable layer is a 2-D convolutional layer (for example, the 'conv10' layer in SqueezeNet):

  • Drag a new convolution2dLayer onto the canvas. Set the NumFilters property to the new number of classes and FilterSize to 1,1.

  • Change the learning rates so that learning is faster in the new layer than in the transferred layers by increasing the WeightLearnRateFactor and BiasLearnRateFactor values.

  • Delete the last convolution2dLayer and connect your new layer instead.

• If the last learnable layer is a fully connected layer (most pretrained networks, for example, GoogLeNet):

  • Drag a new fullyConnectedLayer onto the canvas and set the OutputSize property to the new number of classes.

  • Change the learning rates so that learning is faster in the new layer than in the transferred layers by increasing the WeightLearnRateFactor and BiasLearnRateFactor values.

  • Delete the last fullyConnectedLayer and connect your new layer instead.

Next, delete the classification output layer. Then, drag a new classificationLayer onto the canvas and connect it instead. The default settings for the output layer mean it will learn the number of classes during training.

To check that the network is ready for training, on the Designer tab, click Analyze.

For an example showing how to retrain a pretrained network to classify new images, see Transfer Learning with Deep Network Designer. If you have Audio Toolbox™, you can also load pretrained networks suitable for audio tasks. For an example showing how to retrain a pretrained network to classify a new set of audio signals, see Transfer Learning with Pretrained Audio Networks in Deep Network Designer.

You can also use pretrained networks and transfer learning for regression tasks. For more information, see Convert Classification Network into Regression Network.

Image Classification

You can build an image classification network using Deep Network Designer by dragging layers from the Layer Library and connecting them. You can also create the network at the command line and then import the network into Deep Network Designer.

For example, create a network to train for image classification on a data set of 28-by-28 images divided into 10 classes.

inputSize = [28 28 1];
numClasses = 10;

layers = [
    imageInputLayer(inputSize)
    convolution2dLayer(5,20)
    batchNormalizationLayer
    reluLayer
    fullyConnectedLayer(numClasses)
    softmaxLayer
    classificationLayer];

deepNetworkDesigner(layers)

To adapt this network to your own data, set the InputSize of the image input layer to match your image input size and set the OutputSize of the fully connected layer to the number of classes in your data. For more complex classification tasks, create a deeper network. For more information, see Deep Networks.

For an example showing how to create and train an image classification network, see Create Simple Image Classification Network Using Deep Network Designer.

Sequence Classification

You can use Deep Network Designer to build a sequence network from scratch, or you can use one of the prebuilt untrained networks from the start page. Open the Deep Network Designer start page. Pause on Sequence-to-Label and click Open. Doing so opens a prebuilt network suitable for sequence classification problems.

You can adapt this sequence network for training with your data. Suppose you have data with 12 features and 9 classes. To adapt this network, select sequenceInputLayer and set the InputSize to 12.

Then, select the fullyConnectedLayer and set the OutputSize to 9, the number of classes.

The network is now ready to train. To train the network in Deep Network Designer, create a CombinedDatastore containing the predictors and responses. For more information, see Import Data into Deep Network Designer. For an example showing how to create a combined datastore and train a sequence-to-sequence regression network using Deep Network Designer, see Train Network for Time Series Forecasting Using Deep Network Designer. For an example showing how to export a network built in Deep Network Designer and train using command line functions, see Create Simple Sequence Classification Network Using Deep Network Designer.

Numeric Data Classification

If you have a data set of numeric features (for example, a collection of numeric data without spatial or time dimensions), then you can train a deep learning network using a feature input layer. For more information about the feature input layer, see featureInputLayer.

You can construct a suitable network using Deep Network Designer, or you can create the network at the command line and import the network into Deep Network Designer.

For example, create a network for numeric data with 10 classes, where each observation consists of 20 features.

inputSize = 20;
numClasses = 10;

layers = [
featureInputLayer(inputSize,'Normalization','zscore')
fullyConnectedLayer(50)
batchNormalizationLayer
reluLayer
fullyConnectedLayer(numClasses)
softmaxLayer
classificationLayer];

deepNetworkDesigner(layers)

To adapt this network to your own data, set the InputSize of the feature input layer to match the number of features in your data and set the OutputSize of the fully connected layer to the number of classes in your data. For more complex classification tasks, create a deeper network. For more information, see Deep Networks.

To train a network in Deep Network Designer using data in a table, you must first convert your data into a suitable datastore. For example, start by converting your table into arrays containing the predictors and responses. Then, convert the arrays into arrayDatastore objects. Finally, combine the predictor and response array datastores into a CombinedDatastore object. You can then use the combined datastore to train in Deep Network Designer. For more information, see Import Data into Deep Network Designer. You can also train with tabular data and the trainNetwork function by exporting the network to the workspace.

Convert Classification Network into Regression Network

You can convert a classification network into a regression network by replacing the final layers of the network. Conversion is useful when you want to take a pretrained classification network and retrain it for regression tasks.

For example, suppose you have a GoogLeNet pretrained network. To convert this network into a regression network with a single response, replace the final fully connected layer, the softmax layer, and the classification output layer with a fully connected layer with OutputSize set to 1 (the number of responses) and a regression layer.

If your output has multiple responses, change the OutputSize value of the fully connected layer to the number of responses.

Multiple-Input and Multiple-Output Networks

Multiple Inputs

You can define a network with multiple inputs if the network requires data from multiple sources or in different formats. For example, some networks require image data captured from multiple sensors at different resolutions.

Using Deep Network Designer, you can control the inputs and outputs of each layer. For example, to create a network with multiple image inputs, create two branches, each starting with an image input layer.

You can train a multi-input network with the same type of input, for example, images from two difference sources, using Deep Network Designer and a datastore object. For networks with data in multiple formats, for example, image and sequence data, train the network using a custom training loop. For more information, see dlnetwork for Custom Training Loops.

Multiple Outputs

You can define networks with multiple outputs for tasks requiring multiple responses in different formats, for example, tasks requiring both categorical and numeric output.

Using Deep Network Designer, you can control the outputs of each layer.

To train a multi-output network, you must use a custom training loop. Custom training loops must use a dlnetwork object that does not contain any output layers. For more information, see dlnetwork for Custom Training Loops.

Deep Networks

Building large networks can be difficult, you can use Deep Network Designer to speed up construction. You can work with blocks of layers at a time. Select multiple layers, then copy and paste or delete. For example, you can use blocks of layers to create multiple copies of groups of convolution, batch normalization, and ReLU layers.

For trained networks, copying layers also copies the weights and the biases.

You can also copy sub-networks from the workspace to connect up easily using the app. To import a network or layers into the app, click New > Import from workspace. Click Add to add the layers to the current network.

Advanced Deep Learning Applications

You can use Deep Network Designer to build and train networks for advanced applications, such as computer vision or image processing tasks.

Create Semantic Segmentation Network

Semantic segmentation describes the process of associating each pixel of an image with a class label. Applications for semantic segmentation include road segmentation for autonomous driving and cancer cell segmentation for medical diagnosis.

Create a semantic segmentation network by dragging layers from the Layer Library to the Designer pane or creating the network at the command-line and importing the network into Deep Network Designer.

For example, create a simple semantic segmentation network based on a downsampling and upsampling design.

inputSize = [32 32 1];

layers = [
    imageInputLayer(inputSize)
    convolution2dLayer([3,3],64,'Padding',[1,1,1,1])
    reluLayer
    maxPooling2dLayer([2,2],'Stride',[2,2])
    convolution2dLayer([3,3],64,'Padding',[1,1,1,1])
    reluLayer
    transposedConv2dLayer([4,4],64,'Stride',[2,2],'Cropping',[1,1,1,1])
    convolution2dLayer([1,1],2)
    softmaxLayer
    pixelClassificationLayer
    ];

deepNetworkDesigner(layers) 

For more information on constructing and training a semantic segmentation network, see Train Simple Semantic Segmentation Network in Deep Network Designer (Computer Vision Toolbox).

Create Image-to-Image Regression Network

Image-to-image regression involves taking an input image and producing an output image, often of the same size. This type of network is useful for super-resolution, colorization, or image deblurring.

You can create image-to-image regression networks using Deep Network Designer. For example, create a simple network architecture suitable for image-to-image regression using the unetLayers function from Computer Vision Toolbox™. This function provides a network suitable for semantic segmentation, that can be easily adapted for image-to-image regression.

Create a network with input size 28-by-28-by-1 pixels.

layers = unetLayers([28,28,1],2,'encoderDepth',2);
deepNetworkDesigner(layers);

In the Designer pane, replace the softmax and pixel classification layers with a regression layer from the Layer Library.

Select the final convolutional layer and set the NumFilters property to 1.

For more information on constructing and training an image-to-image regression network, see Image-to-Image Regression in Deep Network Designer.

dlnetwork for Custom Training Loops

You can build and analyze dlnetwork objects using Deep Network Designer. A dlnetwork object enables support for custom training loops using automatic differentiation. Use custom training loops when the built-in training options do not provide the training options that you need for your task.

To check that your network is ready for training using a custom training loop, click Analyze > Analyze for dlnetwork. For more information, see Check Network.

Training with a custom training loop is not supported in Deep Network Designer. To train your network using a custom training loop, first export the network to the workspace and convert it to a dlnetwork object. You can then train the network using the dlnetwork object and a custom training loop. For more information, see Train Network Using Custom Training Loop.

Check Network

To check your network and examine the layers in further detail, on the Designer tab, click Analyze. Investigate problems and examine the layer properties to resolve size mismatches in the network. Return to Deep Network Designer to edit layers, then check the results by clicking Analyze again. If Deep Learning Network Analyzer reports zero errors, then the edited network is ready for training.

You can also analyze networks for custom training workflows. Click Analyze > Analyze for dlnetwork to analyze the network for usage with dlnetwork objects. For example, the Network Analyzer checks that the layer graph does not have any output layers.