softmaxLayer

Softmax layer

Description

A softmax layer applies a softmax function to the input.

Creation

Syntax

layer = softmaxLayer

layer = softmaxLayer('Name',Name)

Description

layer = softmaxLayer creates a softmax layer.

layer = softmaxLayer('Name',Name) creates a softmax layer and sets the optional Name property using a name-value pair. For example, softmaxLayer('Name','sm1') creates a softmax layer with the name 'sm1'. Enclose the property name in single quotes.

Properties

Name — Layer name

'' (default) | character vector | string scalar

Layer name, specified as a character vector or a string scalar. To include a layer in a layer graph, you must specify a nonempty unique layer name. If you train a series network with the layer and Name is set to '', then the software automatically assigns a name to the layer at training time.

Data Types: char | string

NumInputs — Number of inputs

1 (default)

Number of inputs of the layer. This layer accepts a single input only.

Data Types: double

InputNames — Input names

{'in'} (default)

Input names of the layer. This layer accepts a single input only.

Data Types: cell

NumOutputs — Number of outputs

1 (default)

Number of outputs of the layer. This layer has a single output only.

Data Types: double

OutputNames — Output names

{'out'} (default)

Output names of the layer. This layer has a single output only.

Data Types: cell

Examples

Create Softmax Layer

Create a softmax layer with the name 'sm1'.

layer = softmaxLayer('Name','sm1')

layer = 
  SoftmaxLayer with properties:

    Name: 'sm1'

Include a softmax layer in a Layer array.

layers = [ ...
    imageInputLayer([28 28 1])
    convolution2dLayer(5,20)
    reluLayer
    maxPooling2dLayer(2,'Stride',2)
    fullyConnectedLayer(10)
    softmaxLayer
    classificationLayer]

layers = 
  7x1 Layer array with layers:

     1   ''   Image Input             28x28x1 images with 'zerocenter' normalization
     2   ''   Convolution             20 5x5 convolutions with stride [1  1] and padding [0  0  0  0]
     3   ''   ReLU                    ReLU
     4   ''   Max Pooling             2x2 max pooling with stride [2  2] and padding [0  0  0  0]
     5   ''   Fully Connected         10 fully connected layer
     6   ''   Softmax                 softmax
     7   ''   Classification Output   crossentropyex

More About

Softmax Layer

A softmax layer applies a softmax function to the input.

For classification problems, a softmax layer and then a classification layer must follow the final fully connected layer.

The output unit activation function is the softmax function:

$$y_r\left(x\right)=\frac{\exp \left(a_r\left(x\right)\right)}{{\displaystyle \sum _{j=1}^k\exp \left(a_j\left(x\right)\right)}},$$

where $$0\le y_r\le 1$$ and $${\displaystyle \sum _{j=1}^k{y}_j=1}$$.

The softmax function is the output unit activation function after the last fully connected layer for multi-class classification problems:

$$P\left(c_r|x,\theta \right)=\frac{P\left(x,\theta |c_r\right)P\left(c_r\right)}{{\displaystyle \sum _{j=1}^{k}P\left(x,\theta |c_j\right)P\left(c_j\right)}}=\frac{\exp \left(a_r\left(x,\theta \right)\right)}{{\displaystyle \sum _{j=1}^k\exp \left(a_j\left(x,\theta \right)\right)}},$$

where $$0\le P\left(c_r|x,\theta \right)\le 1$$ and $${\displaystyle \sum _{j=1}^{k}P\left(c_j|x,\theta \right)=1}$$. Moreover, $$a_r=\ln \left(P\left(x,\theta |c_r\right)P\left(c_r\right)\right)$$, $$P\left(x,\theta |c_r\right)$$ is the conditional probability of the sample given class r, and $$P\left(c_r\right)$$ is the class prior probability.

The softmax function is also known as the normalized exponential and can be considered the multi-class generalization of the logistic sigmoid function [1].

References

[1] Bishop, C. M. Pattern Recognition and Machine Learning. Springer, New York, NY, 2006.

Extended Capabilities

GPU Code Generation
Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.