Resultados de
I'm working on training neural networks without backpropagation / automatic differentiation, using locally derived analytic forms of update rules. Given that this allows a direct formula to be derived for the update rule, it removes alot of the overhead that is otherwise required from automatic differentiation.
However, matlab's functionalities for neural networks are currently solely based around backpropagation and automatic differentiation, such as the dlgradient function and requiring everything to be dlarrays during training.
I have two main requests, specifically for functions that perform a single operation within a single layer of a neural network, such as "dlconv", "fullyconnect", "maxpool", "avgpool", "relu", etc:
- these functions should also allow normal gpuArray data instead of requiring everything to be dlarrays.
- these functions are currently designed to only perform the forward pass. I request that these also be designed to perform the backward pass if user requests. There can be another input user flag that can be "forward" (default) or "backward", and then the function should have all the necessary inputs to perform that operation (e.g. for "avgpool" forward pass it only needs the avgpool input data and the avgpool parameters, but for the "avgpool" backward pass it needs the deriviative w.r.t. the avgpool output data, the avgpool parameters, and the original data dimensions). I know that there is a maxunpool function that achieves this for maxpool, but it has significant issues when trying to use it this way instead of by backpropagation in a dlgradient type layer, see (https://www.mathworks.com/matlabcentral/answers/2179587-making-a-custom-way-to-train-cnns-and-i-am-noticing-that-avgpool-is-significantly-faster-than-maxpo?s_tid=srchtitle).
I don't know how many people would benefit from this feature, and someone could always spend their time creating these functionalities themselves by matlab scripts, cuDNN mex, etc., but regardless it would be nice for matlab to have this allowable for more customizable neural net training.
Why is RoBERTa not available as a pretrained model? It is superior to BERT in many fields and has become more popular in the literature. For faster inference, you should offer DistilBERT, which is more modern than BERT but smaller/faster. The respository hasn't been updated in two years, which is a lifetime in the field of deep learning.
https://github.com/matlab-deep-learning/transformer-models
The MATLAB Online Training Suite has been updated in the areas of Deep Learning and traditional Machine Learning! These are great self-paced courses that can get you from zero to hero pretty quickly.
Deep Learning Onramp (Free to everyone!) has been updated to use the dlnetwork workflow, such as the trainnet function, which became the preferred method for creating and training deep networks in R2024a.
- Content streamlined to reduce the focus on data processing and feature extraction, and emphasize the machine learning workflow.
- Course example simplified by using a sample of the original data.
- Classification Learner used in the course where appropriate.
The rest of the updates are for subscribers to the full Online Training Suite
The Deep Learning Techniques in MATLAB for Image Applications learning path teaches skills to tackle a variety of image applications. It is made up of the following four short courses:
- Explore Convolutional Neural Networks
- Tune Deep Learning Training Options
- Regression with Deep Learning
- Object Detection with Deep Learning
Two more deep learning short courses are also available:
The Machine Learning Techniques in MATLAB learning path helps learners build their traditional machine learning skill set:
Hi I want to track a animal in my recorded video. I tried computer vision toolbox but it is not very accurate for this type of tracking. The recording is from top and the animal runs in a maze. I want to track body and head. The next step is classifying the movements in video using deep learning but again we do not have a trained network.
