Do I need the derivative of loss function at every layer for a customized loss function?

Hi,

I am very new to use the neural network toolbox in matlab, I am trying to implement my own customized loss function but my loss function is complicated. My question is, is there any easy way to build my neural network other than determining the derivative of my loss function at every layer in my network, or determining once is enough?

Regards,

Jubeyer

Answer 1

[Raunak Gupta]

Hi,

In my understanding for define custom loss function you need to change the last layer of the model which can be a classificationLayer or a regressionLayer. For defining custom loss function, you need to declare a custom layer which required declaring forward loss and backward loss for backpropagation. So, if calculation of derivative of the loss function is feasible you may declare that in backwardLoss otherwise you may only declare the forwardLoss using the functions that supports dlarray.

Calculating the derivative of loss at each layer for backpropagation is not required as only last layer is the one which is using something custom. For defining the custom layers for different problem statements, you may refer to the following. Note that for using above functions you require Deep Learning Toolbox also.

Define Custom Training Loops, Loss Functions, and Networks

Define Custom Classification Output Layer

Define Custom Regression Output Layer

Define Custom Deep Learning Layer with Learnable Parameters