Detect Objects Using YOLOX-Tiny Network Deployed to FPGA
R2026bSince R2026b
This example shows how to deploy a trained You Only Look Once X-tiny object detector (YOLOX-Tiny) to a target FPGA board.
Create YOLOX-Tiny Detector Object
In this example, you use a pretrained YOLOX object detector. Create the YOLOx object detector by using the yoloxObjectDetector function and extract the underlying dlnetwork object for FPGA deployment:
networkName = "tiny-coco";
detector = yoloxObjectDetector(networkName);
net = detector.createDLNetwork()net =
dlnetwork with properties:
Layers: [265×1 nnet.cnn.layer.Layer]
Connections: [296×2 table]
Learnables: [314×3 table]
State: [148×3 table]
InputNames: {'inputLayer'}
OutputNames: {'head:finalCatLayer'}
Initialized: 1
View summary with summary.
To verify that the detector works correctly before deploying it to the FPGA, test the detector on a sample image.
I = imread("carsonroad.png"); [bboxesML,scoresML,labelsML] = detect(detector,I); detectedImg = insertObjectAnnotation(I,"Rectangle",bboxesML,labelsML); figure imshow(detectedImg)
Prepare Network for FPGA Deployment
Use the prepareNetworkForDLHDL helper function to prepare the network for deployment. This function removes layers that Deep Learning HDL Toolbox™ does not directly support. The function:
Replaces
Resize2DLayerlayers with the half-pixel geometric transform modeRemoves the final layer
Inserts identity
1-by-1convolution layers before eachRevisedFlattenLayerlayer
netFPGA = prepareNetworkForDLHDL(net);
Configure the Deep Learning Processor
To generate a custom bitstream, configure the deep learning processor by using a dlhdl.ProcessorConfig object. Set the target frequency to 220 MHz. Disable segmentation block generation for the convolution and fully connected modules. To enable support for the Resize2D and SwishLayer layers, enable Resize2D and SwishLayer in the custom module of the deep learning processor configuration..
hPC = dlhdl.ProcessorConfig; hPC.TargetFrequency = 220; hPC.setModuleProperty('conv','SegmentationBlockGeneration','off'); hPC.setModuleProperty('fc','ModuleGeneration','off'); hPC.setModuleProperty('custom','Resize2D','on'); hPC.setModuleProperty('custom','SwishLayer','on');
Generate a custom bitstream from the deep learning processor configuration by using the dlhdl.buildProcessor function.
dlhdl.buildProcessor(hPC);
Deploy Single-Precision Network to FPGA
Define the target FPGA board programming interface by using the dlhdl.Target object. Specify that the interface is for a Xilinx board with an Ethernet interface.
hTarget = dlhdl.Target("Xilinx",Interface="Ethernet");
Prepare the network for deployment by creating a dlhdl.Workflow object. Specify the network, the bitstream name, and the target. Verify that the bitstream name matches the data type and the FPGA board. In this example, the target FPGA board is the Xilinx Zynq® UltraScale+™ MPSoC ZCU102 board. The bitstream uses the single data type.
hW = dlhdl.Workflow(Network=netFPGA,Bitstream="dlprocessor.bit",Target=hTarget);To compile the network and generate the instructions, weights, and biases for deployment, run the compile method of the dlhdl.Workflow object.
compile(hW)
To deploy the network on the Xilinx ZCU102 SoC hardware, run the deploy method of the dlhdl.Workflow object. The deploy method programs the FPGA device with the bitstream, downloads the network weights and biases to the external memory on the FPGA board, and configures the deep learning processor registers. The method displays progress messages and the time required to deploy the network.
hW.deploy
### Programming FPGA Bitstream using Ethernet... ### Attempting to connect to the hardware board at 192.168.1.101... ### Connection successful ### Programming FPGA device on Xilinx SoC hardware board at 192.168.1.101... ### Attempting to connect to the hardware board at 192.168.1.101... ### Connection successful ### Copying FPGA programming files to SD card... ### Searching for "/mnt/devicetree_dlhdl.dtb" on target. ### Found "/mnt/devicetree_dlhdl.dtb" on target. ### Setting FPGA bitstream and devicetree for boot... # Copying Bitstream dlprocessor.bit to /mnt/hdlcoder_rd/ # Set Bitstream to hdlcoder_rd//dlprocessor.bit # Copying Devicetree devicetree_dlhdl.dtb to /mnt/hdlcoder_rd/ # Set Devicetree to hdlcoder_rd//devicetree_dlhdl.dtb # Set up boot for Reference Design: 'AXI-Stream DDR Memory Access : 3-AXIM' ### Programming done. The system will now reboot for persistent changes to take effect. ### Rebooting Xilinx SoC at 192.168.1.101... ### Reboot may take several seconds... ### Attempting to connect to the hardware board at 192.168.1.101... ### Failed to connect to the hardware board at 192.168.1.101 with error: Error connecting to SSH server at 192.168.1.101. ### Attempting to ping the hardware board at 192.168.1.101... ### Ping successful ### Attempting to connect to the hardware board at 192.168.1.101... ### Connection successful ### Programming successfull! ### Programming the FPGA bitstream has been completed successfully. ### Loading weights to Conv Processor. ### Conv Weights loaded. Current time is 01-Jul-2026 22:20:40
Preprocess Input Image
Preprocess the input image by using letterbox resizing to resize the input image to the network input size. Letterbox resizing preserves the aspect ratio of the original image by padding the shorter dimension with gray pixels. This resizing prevents distortion of object shapes and improves detection accuracy.
networkInputSize = detector.InputSize(1:2); [imProcessed,resizeFactor] = visualinspection.detection.yolox.resizeLetterbox(I,networkInputSize);
Run Prediction on FPGA
Get the activations of the network by using the predict method of the dlhdl.Workflow object. The predict method writes the preprocessed input image to the FPGA external memory, triggers the deep learning processor to execute the network, and returns the output activations. Enable profiling to display the latency of each layer and the throughput in frames per second.
hwprediction = cell(size(netFPGA.OutputNames'));
[hwprediction{:},speed] = hW.predict(imProcessed,"Profile","on");### Finished writing input activations.
### Running single input activation.
Deep Learning Processor Profiler Performance Results
LastFrameLatency(cycles) LastFrameLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 92754097 0.42161 1 88142528 2.5
inputLayer_norm_add 1213208 0.00551
inputLayer_norm 1213507 0.00552
TopLevelModule_backbone_backbone_stem_conv_conv_2_modified 1760878 0.00800
TopLevelModule_backbone_backbone_stem_conv_act 719018 0.00327
TopLevelModule_backbone_backbone_dark2_0_conv 1463164 0.00665
TopLevelModule_backbone_backbone_dark2_0_act 359252 0.00163
TopLevelModule_backbone_backbone_dark2_1_conv1_conv 1119543 0.00509
TopLevelModule_backbone_backbone_dark2_1_conv2_conv 1188314 0.00540
TopLevelModule_backbone_backbone_dark2_1_conv1_act 289462 0.00132
TopLevelModule_backbone_backbone_dark2_1_conv2_act 290775 0.00132
TopLevelModule_backbone_backbone_dark2_1_m_0_conv1_conv 678659 0.00308
TopLevelModule_backbone_backbone_dark2_1_m_0_conv1_act 180255 0.00082
TopLevelModule_backbone_backbone_dark2_1_m_0_conv2_conv 622814 0.00283
TopLevelModule_backbone_backbone_dark2_1_m_0_conv2_act 180145 0.00082
addition 454967 0.00207
TopLevelModule_backbone_backbone_dark2_1_conv3_conv 2008863 0.00913
TopLevelModule_backbone_backbone_dark2_1_conv3_act 359612 0.00163
TopLevelModule_backbone_backbone_dark3_0_conv 1145249 0.00521
TopLevelModule_backbone_backbone_dark3_0_act 180213 0.00082
TopLevelModule_backbone_backbone_dark3_1_conv1_conv 1015473 0.00462
TopLevelModule_backbone_backbone_dark3_1_conv2_conv 1132499 0.00515
TopLevelModule_backbone_backbone_dark3_1_conv1_act 235650 0.00107
TopLevelModule_backbone_backbone_dark3_1_conv2_act 175115 0.00080
TopLevelModule_backbone_backbone_dark3_1_m_0_conv1_conv 563040 0.00256
TopLevelModule_backbone_backbone_dark3_1_m_0_conv1_act 89770 0.00041
TopLevelModule_backbone_backbone_dark3_1_m_0_conv2_conv 501499 0.00228
TopLevelModule_backbone_backbone_dark3_1_m_0_conv2_act 89938 0.00041
addition_1 227569 0.00103
TopLevelModule_backbone_backbone_dark3_1_m_1_conv1_conv 501260 0.00228
TopLevelModule_backbone_backbone_dark3_1_m_1_conv1_act 89975 0.00041
TopLevelModule_backbone_backbone_dark3_1_m_1_conv2_conv 501608 0.00228
TopLevelModule_backbone_backbone_dark3_1_m_1_conv2_act 90112 0.00041
addition_2 227519 0.00103
TopLevelModule_backbone_backbone_dark3_1_m_2_conv1_conv 501308 0.00228
TopLevelModule_backbone_backbone_dark3_1_m_2_conv1_act 89955 0.00041
TopLevelModule_backbone_backbone_dark3_1_m_2_conv2_conv 501306 0.00228
TopLevelModule_backbone_backbone_dark3_1_m_2_conv2_act 89921 0.00041
addition_3 227469 0.00103
TopLevelModule_backbone_backbone_dark3_1_conv3_conv 1885346 0.00857
TopLevelModule_backbone_backbone_dark3_1_conv3_act 180052 0.00082
TopLevelModule_backbone_backbone_dark4_0_conv 1253567 0.00570
TopLevelModule_backbone_backbone_dark4_0_act 90341 0.00041
TopLevelModule_backbone_backbone_dark4_1_conv1_conv 886218 0.00403
TopLevelModule_backbone_backbone_dark4_1_conv2_conv 958901 0.00436
TopLevelModule_backbone_backbone_dark4_1_conv1_act 115785 0.00053
TopLevelModule_backbone_backbone_dark4_1_conv2_act 100818 0.00046
TopLevelModule_backbone_backbone_dark4_1_m_0_conv1_conv 516150 0.00235
TopLevelModule_backbone_backbone_dark4_1_m_0_conv1_act 45136 0.00021
TopLevelModule_backbone_backbone_dark4_1_m_0_conv2_conv 460065 0.00209
TopLevelModule_backbone_backbone_dark4_1_m_0_conv2_act 45206 0.00021
addition_4 114100 0.00052
TopLevelModule_backbone_backbone_dark4_1_m_1_conv1_conv 459851 0.00209
TopLevelModule_backbone_backbone_dark4_1_m_1_conv1_act 45089 0.00020
TopLevelModule_backbone_backbone_dark4_1_m_1_conv2_conv 459667 0.00209
TopLevelModule_backbone_backbone_dark4_1_m_1_conv2_act 45052 0.00020
addition_5 113760 0.00052
TopLevelModule_backbone_backbone_dark4_1_m_2_conv1_conv 459790 0.00209
TopLevelModule_backbone_backbone_dark4_1_m_2_conv1_act 45013 0.00020
TopLevelModule_backbone_backbone_dark4_1_m_2_conv2_conv 459867 0.00209
TopLevelModule_backbone_backbone_dark4_1_m_2_conv2_act 45238 0.00021
addition_6 113880 0.00052
TopLevelModule_backbone_backbone_dark4_1_conv3_conv 1705859 0.00775
TopLevelModule_backbone_backbone_dark4_1_conv3_act 89770 0.00041
TopLevelModule_backbone_backbone_dark5_0_conv 917013 0.00417
TopLevelModule_backbone_backbone_dark5_0_act 44924 0.00020
TopLevelModule_backbone_backbone_dark5_1_conv1_conv 879005 0.00400
TopLevelModule_backbone_backbone_dark5_1_conv1_act 22608 0.00010
TopLevelModule_backbone_backbone_dark5_1_m_0 76082 0.00035
TopLevelModule_backbone_backbone_dark5_1_m_1 118263 0.00054
TopLevelModule_backbone_backbone_dark5_1_m_2 253650 0.00115
TopLevelModule_backbone_backbone_dark5_1_conv2_conv 3383807 0.01538
TopLevelModule_backbone_backbone_dark5_1_conv2_act 45132 0.00021
TopLevelModule_backbone_backbone_dark5_2_conv1_conv 879114 0.00400
TopLevelModule_backbone_backbone_dark5_2_conv2_conv 915259 0.00416
TopLevelModule_backbone_backbone_dark5_2_conv1_act 51781 0.00024
TopLevelModule_backbone_backbone_dark5_2_conv2_act 51612 0.00023
TopLevelModule_backbone_backbone_dark5_2_m_0_conv1_conv 485515 0.00221
TopLevelModule_backbone_backbone_dark5_2_m_0_conv1_act 22468 0.00010
TopLevelModule_backbone_backbone_dark5_2_m_0_conv2_conv 448966 0.00204
TopLevelModule_backbone_backbone_dark5_2_m_0_conv2_act 22597 0.00010
TopLevelModule_backbone_backbone_dark5_2_conv3_conv 1710800 0.00778
TopLevelModule_backbone_backbone_dark5_2_conv3_act 45165 0.00021
neck:TopLevelModule_backbone_lateral_conv0_conv 879212 0.00400
neck:TopLevelModule_backbone_lateral_conv0_act 22542 0.00010
neck:layer 74962 0.00034
neck:TopLevelModule_backbone_C3_p4_conv1_conv 2179215 0.00991
neck:TopLevelModule_backbone_C3_p4_conv2_conv 2253050 0.01024
neck:TopLevelModule_backbone_C3_p4_conv1_act 110594 0.00050
neck:TopLevelModule_backbone_C3_p4_conv2_act 100834 0.00046
neck:TopLevelModule_backbone_C3_p4_m_0_conv1_conv 516131 0.00235
neck:TopLevelModule_backbone_C3_p4_m_0_conv1_act 44947 0.00020
neck:TopLevelModule_backbone_C3_p4_m_0_conv2_conv 460128 0.00209
neck:TopLevelModule_backbone_C3_p4_m_0_conv2_act 45047 0.00020
neck:TopLevelModule_backbone_C3_p4_conv3_conv 1705969 0.00775
neck:TopLevelModule_backbone_C3_p4_conv3_act 90038 0.00041
neck:TopLevelModule_backbone_reduce_conv1_conv 886112 0.00403
neck:TopLevelModule_backbone_reduce_conv1_act 44926 0.00020
neck:layer_1 150086 0.00068
neck:TopLevelModule_backbone_C3_p3_conv1_conv 1904770 0.00866
neck:TopLevelModule_backbone_C3_p3_conv2_conv 2033845 0.00924
neck:TopLevelModule_backbone_C3_p3_conv1_act 224488 0.00102
neck:TopLevelModule_backbone_C3_p3_conv2_act 175582 0.00080
neck:TopLevelModule_backbone_C3_p3_m_0_conv1_conv 563117 0.00256
neck:TopLevelModule_backbone_C3_p3_m_0_conv1_act 89981 0.00041
neck:TopLevelModule_backbone_C3_p3_m_0_conv2_conv 501518 0.00228
neck:TopLevelModule_backbone_C3_p3_m_0_conv2_act 89842 0.00041
neck:TopLevelModule_backbone_C3_p3_conv3_conv 1885422 0.00857
neck:TopLevelModule_backbone_C3_p3_conv3_act 180132 0.00082
neck:TopLevelModule_backbone_bu_conv2_conv 700909 0.00319
head:TopLevelModule_head_stems_0_conv 1972278 0.00896
neck:TopLevelModule_backbone_bu_conv2_act 153434 0.00070
head:TopLevelModule_head_stems_0_act 179875 0.00082
head:TopLevelModule_head_cls_convs_0_0_conv 1872014 0.00851
head:TopLevelModule_head_reg_convs_0_0_conv 1987583 0.00903
neck:TopLevelModule_backbone_C3_n3_conv1_conv 998415 0.00454
neck:TopLevelModule_backbone_C3_n3_conv2_conv 959271 0.00436
head:TopLevelModule_head_cls_convs_0_0_act 324406 0.00147
head:TopLevelModule_head_reg_convs_0_0_act 295730 0.00134
neck:TopLevelModule_backbone_C3_n3_conv1_act 115876 0.00053
neck:TopLevelModule_backbone_C3_n3_conv2_act 153436 0.00070
head:TopLevelModule_head_cls_convs_0_1_conv 1960540 0.00891
head:TopLevelModule_head_reg_convs_0_1_conv 1987255 0.00903
neck:TopLevelModule_backbone_C3_n3_m_0_conv1_conv 516013 0.00235
head:TopLevelModule_head_cls_convs_0_1_act 324215 0.00147
head:TopLevelModule_head_reg_convs_0_1_act 243693 0.00111
neck:TopLevelModule_backbone_C3_n3_m_0_conv1_act 153496 0.00070
head:TopLevelModule_head_cls_preds_0 1682413 0.00765
head:TopLevelModule_head_reg_preds_0 219204 0.00100
head:TopLevelModule_head_obj_preds_0 218542 0.00099
neck:TopLevelModule_backbone_C3_n3_m_0_conv2_conv 459741 0.00209
neck:TopLevelModule_backbone_C3_n3_m_0_conv2_act 154376 0.00070
head:flattenLayer1_conv 1710102 0.00777
neck:TopLevelModule_backbone_C3_n3_conv3_conv 1706046 0.00775
neck:TopLevelModule_backbone_C3_n3_conv3_act 89953 0.00041
neck:TopLevelModule_backbone_bu_conv1_conv 491710 0.00224
head:TopLevelModule_head_stems_1_conv 922864 0.00419
neck:TopLevelModule_backbone_bu_conv1_act 58002 0.00026
head:TopLevelModule_head_stems_1_act 45096 0.00020
head:TopLevelModule_head_cls_convs_1_0_conv 459747 0.00209
head:TopLevelModule_head_reg_convs_1_0_conv 516000 0.00235
neck:TopLevelModule_backbone_C3_n4_conv1_conv 952009 0.00433
neck:TopLevelModule_backbone_C3_n4_conv2_conv 915842 0.00416
head:TopLevelModule_head_cls_convs_1_0_act 100269 0.00046
head:TopLevelModule_head_reg_convs_1_0_act 103411 0.00047
neck:TopLevelModule_backbone_C3_n4_conv1_act 52262 0.00024
neck:TopLevelModule_backbone_C3_n4_conv2_act 57954 0.00026
head:TopLevelModule_head_cls_convs_1_1_conv 496272 0.00226
head:TopLevelModule_head_reg_convs_1_1_conv 516015 0.00235
neck:TopLevelModule_backbone_C3_n4_m_0_conv1_conv 506691 0.00230
head:TopLevelModule_head_cls_convs_1_1_act 100789 0.00046
head:TopLevelModule_head_reg_convs_1_1_act 93702 0.00043
neck:TopLevelModule_backbone_C3_n4_m_0_conv1_act 58042 0.00026
head:TopLevelModule_head_cls_preds_1 425233 0.00193
head:TopLevelModule_head_reg_preds_1 51143 0.00023
head:TopLevelModule_head_obj_preds_1 51309 0.00023
neck:TopLevelModule_backbone_C3_n4_m_0_conv2_conv 449144 0.00204
neck:TopLevelModule_backbone_C3_n4_m_0_conv2_act 57932 0.00026
head:flattenLayer2_conv 427997 0.00195
neck:TopLevelModule_backbone_C3_n4_conv3_conv 1710854 0.00778
neck:TopLevelModule_backbone_C3_n4_conv3_act 45064 0.00020
head:TopLevelModule_head_stems_2_conv 463259 0.00211
head:TopLevelModule_head_stems_2_act 11304 0.00005
head:TopLevelModule_head_cls_convs_2_0_conv 122877 0.00056
head:TopLevelModule_head_reg_convs_2_0_conv 141764 0.00064
head:TopLevelModule_head_cls_convs_2_0_act 26121 0.00012
head:TopLevelModule_head_reg_convs_2_0_act 26211 0.00012
head:TopLevelModule_head_cls_convs_2_1_conv 141449 0.00064
head:TopLevelModule_head_reg_convs_2_1_conv 141820 0.00064
head:TopLevelModule_head_cls_convs_2_1_act 26311 0.00012
head:TopLevelModule_head_reg_convs_2_1_act 25836 0.00012
head:TopLevelModule_head_cls_preds_2 122598 0.00056
head:TopLevelModule_head_reg_preds_2 17099 0.00008
head:TopLevelModule_head_obj_preds_2 17006 0.00008
head:flattenLayer3_conv 105460 0.00048
* The clock frequency of the DL processor is: 220MHz
Postprocess FPGA Output
The FPGA returns predictions from multiple output heads as separate feature maps. Concatenate the predictions across the second dimension to combine all detection scales into a single prediction matrix.
hwpredictionFPGA = cat(2,hwprediction{:});
Use the postprocess function to decode the raw network output into bounding boxes, confidence scores, and class labels. The postprocessing applies sigmoid activation to objectness and class scores and decodes the bounding box coordinates from the anchor offsets. It then applies confidence thresholding and performs non-maximum suppression to eliminate overlapping detections. The ResizeFactor argument accounts for the letterbox resizing applied during preprocessing and maps the bounding boxes back to the original image coordinates.
[bboxesFPGA,scoresFPGA,labelsFPGA] = visualinspection.detection.yolox.postprocess( ...
detector,hwpredictionFPGA,ResizeFactor=resizeFactor);
Display Detection Results
Annotate the original image with the detected bounding boxes and class labels, then display the result.
detectedImg = insertObjectAnnotation(I,"Rectangle",bboxesFPGA,labelsFPGA);
figure
imshow(detectedImg)
Helper Functions
function netModified = prepareNetworkForDLHDL(net) % prepareNetworkForDLHDL Modify network for Deep Learning HDL deployment. % netModified = prepareNetworkForDLHDL(net) replaces Resize2DLayers with % half-pixel geometric transform mode, removes the final layer, and % inserts an identity 1x1 conv layer before each RevisedFlattenLayer. % Replace Resize2DLayers with half-pixel geometric transform mode for i = 1:length(net.Layers) if isa(net.Layers(i), 'nnet.cnn.layer.Resize2DLayer') resizeLayer = net.Layers(i); newReSizeLayer = resize2dLayer("GeometricTransformMode", "half-pixel", ... "Method", resizeLayer.Method, ... "EnableReferenceInput", resizeLayer.EnableReferenceInput, ... "Name", resizeLayer.Name); net = replaceLayer(net, resizeLayer.Name, newReSizeLayer); end end % Convert to layer graph for structural modifications lgraph = layerGraph(net); sz = length(net.Layers); % Remove the final layer finalLayer = net.Layers(end); lgraph = removeLayers(lgraph, finalLayer.Name); % Determine intermediate layer sizes using a sample forward pass inputSize = net.Layers(1).InputSize; X = dlarray(ones([inputSize 1], 'single'), 'SSCB'); net = initialize(net); % Insert identity 1x1 conv layer before each RevisedFlattenLayer for i = 1:sz currLayer = net.Layers(i); if isa(currLayer, 'visualinspection.layer.internal.RevisedFlattenLayer') % Determine parent layer output size parentIdx = dnnfpga.compiler.optimizations.findParent(lgraph, currLayer.Name); parentLayer = lgraph.Layers(parentIdx); parentAct = predict(net, X, 'Outputs', {parentLayer.Name}); [~, ~, c, ~] = size(parentAct); % Create a 1x1 convolution layer with identity weights convName = [currLayer.Name '_conv']; convLayer = convolution2dLayer(1, c, 'NumChannels', c, ... 'Padding', 'same', 'Name', convName); Weights = zeros(1, 1, c, c, 'single'); Weights(1, 1, :, :) = reshape(eye(c, 'single'), 1, 1, c, c); convLayer.Weights = Weights; convLayer.Bias = zeros(1, 1, c, 'single'); % Insert conv layer between parent and RevisedFlattenLayer lgraph = disconnectLayers(lgraph, parentLayer.Name, currLayer.Name); lgraph = addLayers(lgraph, convLayer); lgraph = connectLayers(lgraph, parentLayer.Name, convName); lgraph = connectLayers(lgraph, convName, currLayer.Name); end end netModified = dlnetwork(lgraph); end
