The custom loss function, which is trained with the dlarray structure, reports an error when using sgdmupdate to update, and cannot be assigned, because this type of variable

% 定义 LSTM 网络结构
numFeatures = size(X_train, 2); % 特征数量，这里是 3
numHiddenUnits = 100; % LSTM 隐藏单元数量

layers = [ ...
    sequenceInputLayer(numFeatures)
    lstmLayer(numHiddenUnits)
    fullyConnectedLayer(numFeatures)
    
    ];

maxEpochs = 50;
miniBatchSize = 3;
learningRate = 0.01;

% 初始化 LSTM 模型
net = dlnetwork(layers); % 在 CPU 上训练

% 训练循环
numObservations = size(X_train, 1);
numMiniBatches = floor(numObservations / miniBatchSize);

averageGrad = [];
averageSqGrad = [];
iteration = 0;
Velocities = zeros(size(net));
momentum = 0.9;

for epoch = 1:maxEpochs
    % 打乱训练数据
    idx = randperm(numObservations);
    X_train_shuffled = X_train(idx, :);
    Y_train_shuffled = Y_train(idx, :);
    
    totalLoss = 0;
    
    for i = 1:numMiniBatches
        idxBatch = (i - 1) * miniBatchSize + 1 : i * miniBatchSize;
        X_batch = X_train_shuffled(idxBatch, :);
        Y_batch = Y_train_shuffled(idxBatch, :);

        iteration = iteration + 1;
        
        % 前向传播计算预测
        Y_pred = predict(net, X_batch');
        Y_pred = Y_pred';

        Y_pred = dlarray(Y_pred, 'TS');
        Y_batch = dlarray(Y_batch, 'TS');
        
        %loss = mean(( Y_batch - Y_pred).^2, 'all');
        [loss_1, gradients] = dlfeval(@customLoss, Y_batch, Y_pred);
        
        gradients = extractdata(gradients);
       

        [net, Velocities] = sgdmupdate(net, gradients, Velocities,learningRate,momentum);

        % 计算损失
        % loss = customLoss(Y_batch, Y_pred);
        % totalLoss = totalLoss + loss;
        
        % 反向传播更新梯度
        %dL_dY = 2 * (Y_pred - Y_batch) / miniBatchSize;
        %gradients = dlgradient(loss, net.LearnableParameters);
        %net = updateParameters(net, gradients, learningRate);
    end
    
    % 显示每个 epoch 的平均损失
    avgLoss = totalLoss / numMiniBatches;
    fprintf('Epoch %d, Average Loss: %.4f\n', epoch, avgLoss);
end

无法执行赋值，因为此类型的变量不支持使用点进行索引。

出错 deep.internal.recording.containerfeval>iProcessNetwork_Nout_Nin (第 382 行)

protoTable.Value = zeros(height(protoTable),0);

出错 deep.internal.recording.containerfeval>iDispatch_Nout_Nin (第 214 行)

outputs = iProcessNetwork_Nout_Nin(fun, paramFun, numOut, ...

出错 deep.internal.recording.containerfeval (第 38 行)

outputs = iDispatch_Nout_Nin(allowNetInput, fun, paramFun, numOut, ...

出错 deep.internal.networkContainerFixedArgsFun (第 29 行)

varargout = deep.internal.recording.containerfeval(...

出错 sgdmupdate (第 126 行)

[p, vel] = deep.internal.networkContainerFixedArgsFun(func, ...

Answer 1

[praguna manvi]

Hi,

According to the documentation on “dlfeval” :

https://www.mathworks.com/help/deeplearning/ref/dlfeval.html?s_tid=doc_ta#mw_ea840021-31a5-4c7d-b535-4a317bf496d4

You can use a “dlnetwork” object as a function argument to evaluate gradients for deep learning. To do this, you should perform the network's forward pass within the "fun" function handle.

To implement the custom loss in the above network, refer to the function “modelGradients” from the corrected code below. Compute “gradients = dlgradient(loss, net.Learnables);” and then use “sgdmupdate(net, ...)” to update the network.

numObservations = 100;
numFeatures = 3;
numResponses = 3;
sequenceLength = 1; 

X_train = rand(numFeatures, sequenceLength, numObservations); 
Y_train = rand(numResponses, sequenceLength, numObservations); 


numHiddenUnits = 100;

layers = [ ...
    sequenceInputLayer(numFeatures)
    lstmLayer(numHiddenUnits, 'OutputMode', 'sequence')
    fullyConnectedLayer(numResponses)
    ];

maxEpochs = 50;
miniBatchSize = 3;
learningRate = 0.01;


net = dlnetwork(layers); 

numMiniBatches = floor(numObservations / miniBatchSize);

averageGrad = [];
averageSqGrad = [];
iteration = 0;
Velocities = [];
momentum = 0.9;

for epoch = 1:maxEpochs
    idx = randperm(numObservations);
    X_train_shuffled = X_train(:, :, idx);
    Y_train_shuffled = Y_train(:, :, idx);
    totalLoss = 0;
    for i = 1:numMiniBatches
        idxBatch = (i - 1) * miniBatchSize + 1 : i * miniBatchSize;
        X_batch = X_train_shuffled(:, :, idxBatch);
        Y_batch = Y_train_shuffled(:, :, idxBatch);
        iteration = iteration + 1;
        dlX_batch = dlarray(X_batch, 'CBT');
        dlY_batch = dlarray(Y_batch, 'CBT');
        [loss, gradients] = dlfeval(@modelGradients, net, dlX_batch, dlY_batch);
        
        [net, Velocities] = sgdmupdate(net, gradients, Velocities, learningRate, momentum);
        
        totalLoss = totalLoss + loss;
    end
    
    avgLoss = totalLoss / numMiniBatches;
    fprintf('Epoch %d, Average Loss: %.4f\n', epoch, avgLoss);
end

function [loss, gradients] = modelGradients(net, dlX, dlY)
    dlYPred = forward(net, dlX);
    loss = mean((dlYPred - dlY).^2, 'all');
    gradients = dlgradient(loss, net.Learnables);
end

Additionally, please refer to this example on how to train a network using custom loss :

https://www.mathworks.com/help/deeplearning/ug/train-generative-adversarial-network.html?searchHighlight=gans&s_tid=srchtitle_support_results_1_gans#TrainGenerativeAdversarialNetworkGANExample-9