Export Labeled Data from Signal Labeler for AI-Based Spectrum Sensing Applications
R2026bThe ubiquitousness of wireless devices and scarcity of licensed spectrum bands has increased the demand for efficiency in wireless spectrum usage. There are opportunities for improving the efficiency of spectrum usage beyond that afforded by conventional methods by using spectrum-sensing techniques that involve machine learning and deep learning algorithms. Spectrum sensing enables systems to determine the availability of frequency resources, which is an important step in cognitive radio as well as applications such as radar-centric integrated sensing and radar target detection and classification.
This example shows how to use deep learning to train both a semantic segmentation network and an object detection network, then use them to detect RF frames. You use the Signal Labeler app to create labeled training data from complex RF I/Q signals in the time-frequency domain. This example trains neural networks to identify three different types of Bluetooth® frames, WLAN frames, and collisions between two or more types of frames.
Computer vision uses semantic segmentation and object detection techniques to identify objects and their locations in an image or video. In wireless communication and radar applications, the objects of interest are radio waveforms coming from different wireless standards, radar pulses, or unknown sources. The locations of the objects are the time and frequency regions these waveforms occupy in the time-frequency domain.
Because this example requires an object detection network that accepts an image as input and labels regions of interest in it, this example uses a YOLOX network.
Prepare Training Data
Load Training Data
This example uses the Spectrogram Data Set for Deep Learning Based RF-Frame Detection [1]. The data set contains 20,000 augmented signal segments. This example uses a subset of the data set such that each signal has a random number of different Wi-Fi® and Bluetooth frames. The size of this data set is 4,831 signals.
% Define the location in which to store the dataset. By default, this % example uses the temporary folder for the system. baseDir = tempdir; baseDataDir = fullfile(baseDir, "SpectrogramRFFrameDetectionData"); baseNetworkDir = fullfile(baseDir, "SpectrogramRFFrameDetectionNetwork"); unzip("https://ssd.mathworks.com/supportfiles/SPT/data/SpectrogramRFFrameDetectionData.zip", ... baseDir)
Format Signals for Labeling in Signal Labeler
To open the Signal Labeler app, on the Apps tab of the MATLAB Toolstrip, in the Signal Processing and Audio section of the Apps section, select Signal Labeler. Alternatively, you can open the app from the MATLAB command line.
signalLabeler
On the Labeler tab of the app toolstrip, click Import and, in the Members section, select From Folders. In the Import Members From Folder dialog box, specify Name as the path to the SpectrogramRFFrameDetectionData folder inside the temporary folder. Set Extension to MAT files. Specify the sample rate of the data set in the Time information section by selecting Working in, then selecting Time from the drop-down. For the options that appear, set the first drop-down to Sample Rate, specify a value of 25 in the field, and set the units drop-down to MHz. This specifies a sample rate of 25 MHz. Click Import.
The imported files appear in the Label Signal Set Members pane.
Plot the first signal by selecting the check box next in the Plot column for that signal.
On the Display tab of the app toolstrip, in the Views section, select Spectrogram. The app displays a set of axes with the signal spectrogram and adds a Spectrogram tab to the app toolstrip.
Modify the spectrogram parameters to get a clear spectrogram where the waveforms of interest are easy to identify. In the Spectrogram tab of the app in the toolstrip, select a desired Resolution Type. The spectrogram settings used in this example include a Resolution Type of Window Length, Hann window of length 256, an overlap of 50%, and a minimum power limit of –80 dB. The minimum power limit reduces the power dynamic range of the spectrogram making it easier to identify regions that contain waveforms. Ensure that each waveform is clear in each member, as these settings are locked when labeling areas of interest in the app. Ensure that many members have clear waveforms by cycling through members in the toolstrip. In the Labeler tab, select the Next or Previous in the Member Navigation section.
To label data in the Signal Labeler app, first create a label definition. On the Labeler tab of the app toolstrip, in the Label Definition section, click Add to open the Add Label Definition dialog box. Specify Label name as RFFrame, and set Label type to Time-Frequency ROI. Set Data type to categorical and, in the Categories box, add the categories BLE_1MHz, BLE_2MHz, BT_classic, WLAN, and collision, which are the labels for the three different Bluetooth waveforms, the WLAN waveforms, and the waveform collisions, respectively.
In the Time-frequency Transform: Spectrogram section, select Options to inspect the spectrogram options associated with the label definition. Note that the settings correspond to those specified in the app toolstrip.
The label definition uses the specified options to create spectrograms associated with all labels, enforcing a tight connection between the I/Q signals and the spectrograms used for labeling. To accept these settings, in the Specify Spectrogram Options dialog box, click OK. To finalize and create the label definition, in the Add Label Definition dialog box, click OK.
An information dialog box appears upon label definition creation, indicating that the specified spectrogram parameters are locked while labeling. Click OK to accept that the parameters are locked and continue to labeling.
Manually Label RF Frames in Signal Labeler
Upon label creation, the RFFrame label definition appears in the Label Definition pane, already selected, and Draw Labels in the Set Label Value section of the Labeler tab is is automatically active.
For convenience, this example includes a labeledSignalSet object, labeledLss, that already has the manual labels for each member of this data set.
Label the signals one at a time:
In the spectrogram plot, click and drag to select a region, represented as a thick, animated dashed line around a shaded region.
Move and resize the active region until it encloses a radio signal. To zoom in on a region of the spectrogram for more accurate placement, you can choose a zoom action at the top-right corner of the spectrogram plot, or from the Display tab of the app toolstrip, select Panner and use the signal panner to focus on a specific region of the signal. If you use the zoom tools directly in the spectrogram plot, the app turns off Draw Labels mode by default.
Use the drop-down next to your label to select the appropriate label definition category for the signal. The first frame contains only one WLAN signal. You can also specify the category for the label before you create it by using the Value parameter in the Set Label Value section of the Labeler tab of the app toolstrip.
To finalize the label, press Enter, double-click the label, or select Accept from the Options section of the Labeler tab of the app toolstrip.
Repeat this process for all other radio signals in the member, and then for each member in the data set.
Before labeling the next member, you must remove the current signal from the plot by clearing the check box in its row of the Plot column in the Labeled Signal Set Members pane. You cannot plot multiple signals while the spectrogram display is active.
Export Labeled Signal Set
Once you have labeled the signals in all of the members, export the labeled data set as a new labeledSignalSet object. You can export the labeled signal set to the MATLAB workspace or to a MAT file. On the Labeler tab of the app toolstrip, click Export and, in the Labeled Signal Set section, select To Workspace. In the Export Labeled Signal Set To Workspace dialog box, specify Name as labeledLss, optionally add a short description, and click Export. For more information on how Signal Labeler exports labeled signal sets, see Export Data and Create Data Sets.
Next, use the data and labels stored in the exported labeled signal set to train detection and segmentation deep learning models.
Train YOLOX Object Detection Network
Load the labeled signal set. Set the alternate file system roots to the folder in which you downloaded the files so that you can use the labeled signal set in your machine.
load(fullfile(baseDataDir,"preLabeledSignalSet.mat")) setAlternateFileSystemRoots(labeledLss, ... [labeledLss.Source.Folders baseDataDir])
Prepare Data for Training
For this example, you want to use an object detection network that accepts an image as input and finds and labels regions of interest in it. Each label instance describes the type of waveform detected, either BLE_1MHz, BLE_2MHz, BT_classic, WLAN, or collision.
The exported labeled signal set contains all of the signals in the data set and all the created labels. The labels have regions of interest defined by limits in both time and frequency. To train a network, you must first convert the signals from the time domain into a spectrogram, and then further from a spectrogram into an image. You must also transform the labels from time and frequency limits to a format that the network understands. The createDatastores object function of the labeledLss object helps with this process.
The target YOLOX tiny-coco object detection network expects images of size 256-by-256-by-3. Because the unmodified spectrograms created by the labeledLss object are greater than these dimensions, you must resize them to fit the image dimensions the network expects.
imageSize = [256 256];
The YOLOX network defines rectangular regions of interest in an image as axis-aligned rectangles, expressed in spatial coordinates as an M-by-4 numeric matrix with rows of the form [x y w h], where:
M is the number of axis-aligned rectangles.
x and
yspecify the upper-left corner of the rectangle.w specifies the width of the rectangle, which is its length along the x-axis.
h specifies the height of the rectangle, which is its length along the y-axis.
Both the training data and the detected results from the network must define regions of interest as axis-aligned rectangles.
To get images of the desired size and corresponding region labels, use the createDatastores object function to create a transformed datastore, which generates images from the spectrograms, and an array datastore, which contains the label information in the correct format. Specify the TimeFrequencyMapFormat argument as "image" to transform the spectrogram to a uint8 image. Specify the TimeFrequencyImageSize argument to produce spectrogram images of size 256-by-256. To return an array datastore that defines the label borders as axis-aligned rectangles, specify the TimeFrequencyLabelFormat argument as "xywh". This array datastore also contains the label values as a categorical.
labelDefinitionName = "RFFrame"; [odSds,odLds] = createDatastores(labeledLss, labelDefinitionName, ... TimeFrequencyLabelFormat="xywh", ... TimeFrequencyMapFormat="image", ... TimeFrequencyImageSize=[imageSize(1) imageSize(2)], ... TimeFrequencyIncludePartialBins=true);
As an alternative to computing spectrograms on-the-fly, you can precompute and save the spectrogram images to files. See Creating Stored Data Set for more information.
When you use the read function of the TransformedDatastore object odSds, it computes the spectrogram using the settings specified in the label definition. The datastore transforms the spectrogram into a three-dimensional image whose first two dimensions are specified by imageSize.
The ArrayDatastore object odLds contains the label information in the axis-aligned rectangle form, mapped to the dimensions specified by imageSize. The label values stored in odLds have been transformed from the time and frequency limits in the labeledLss object to pixel values of the resized spectrogram.
Inspect Labels
Inspect the first spectrogram and its associated labels within the datastores. For each label, plot the boundaries of the label as a rectangle and label the rectangle with the label value.
Get the first spectrogram image and its associated labels.
[specImage,specInfo] = read(odSds); reset(odSds) specLabel = preview(odLds);
Plot the image of the spectrogram specImage.
imagesc(specImage)
%NOTE% I don't think I can add alt text to these output images, and the review has alt texts added to them.
The specInfo structure contains the information required to display the axes limits and ticks, as well as the options used to create the spectrogram from the time domain signal.
The specLabel cell array contains the locations and values of the labels.
Plot the spectrogram image and the associated labels with the helperObjectDetectionPlotter function.
catsObjectDetection = ["BLE_1MHz","BLE_2MHz","BT_classic","WLAN","collision"]; helperObjectDetectionPlotter(specImage,specInfo,specLabel,catsObjectDetection)
Split Training, Validation, and Test Sets
Split the data sets into training, validation, and test sets. Allocate 80% of the data for training, 10% for validation, and 10% for test. Use a random number stream with a specified seed for repeatability of the shuffled file order.
parts = [80 10 10];
s = RandStream("mt19937ar",Seed=0);
numFiles = labeledLss.NumMembers;
shuffledIndices = randperm(s,numFiles);
numTrain = floor(numFiles*parts(1)/100);
numVal = floor(numFiles*parts(2)/100);
odSdsTrain = subset(odSds,shuffledIndices(1:numTrain));
odSdsVal = subset(odSds,shuffledIndices(numTrain+(1:numVal)));
odSdsTest = subset(odSds,shuffledIndices((numTrain+numVal+1):end));
odLdsTrain = subset(odLds,shuffledIndices(1:numTrain));
odLdsVal = subset(odLds,shuffledIndices(numTrain+(1:numVal)));
odLdsTest = subset(odLds,shuffledIndices((numTrain+numVal+1):end));Combine the datastores containing the spectrogram and the label information into a single datastore for training.
odCdsTrain = combine(odSdsTrain,odLdsTrain); odCdsVal = combine(odSdsVal,odLdsVal); odCdsTest = combine(odSdsTest,odLdsTest);
Train Deep Neural Network for Object Detection
This example uses the YOLOX object model for the detection problem because it is suited for quickly identifying waveforms in input spectrograms. The YOLOX model is a single-stage, anchor-free technique, which significantly reduces the model size and improves computation speed compared to previous YOLO models. Instead of using memory-intensive predefined anchor boxes, YOLOX localizes objects directly by finding object centers.
Because training a network can require several hours, this example includes a pretrained network. To train the network yourself, specify trainNow as true. To instead download a MAT file containing the pretrained network, set trainNow to false.
trainNow =false; if trainNow frameDetector = yoloxObjectDetector("tiny-coco",catsObjectDetection, ... InputSize=[imageSize(1) imageSize(2) 3]); end
Configure training using the trainOptions (Deep Learning Toolbox) function to specify the stochastic descent with momentum (SGDM) optimization algorithm and the hyper-parameters used for SGDM. Train the object detector for a maximum of 100 epochs. Specify the ValidationData name-value argument as the validation data as odCdsVal. Set OutputNetwork to "best-validation-loss" to obtain the network with the lowest validation loss during training when the training finishes.
options = trainingOptions("sgdm", ... InitialLearnRate=5e-4, ... LearnRateSchedule="piecewise", ... LearnRateDropFactor=0.99, ... LearnRateDropPeriod=1, ... MiniBatchSize=20, ... MaxEpochs=100, ... ExecutionEnvironment="auto", ... Shuffle="every-epoch", ... VerboseFrequency=25, ... ValidationFrequency=100, ... ValidationData=odCdsVal, ... ResetInputNormalization=false, ... OutputNetwork="best-validation-loss", ... GradientThreshold=30, ... L2Regularization=5e-4);
Train the detector by using the trainYOLOXObjectDetector (Visual Inspection Toolbox) function.
if trainNow frameDetector = trainYOLOXObjectDetector(cdsTrain,frameDetector, ... options,FreezeSubNetwork="none"); modelDateTime = string(datetime("now",Format="yyyy-MM-dd-HH-mm-ss")); save(fullfile(baseNetworkDir,"trainedRFFrameDetectorYoloX"+modelDateTime), ... "frameDetector") else load(fullfile(baseNetworkDir,"networks.mat"),"frameDetector") end
Evaluate Detector
Detect the bounding boxes for all test images. Set the detection threshold to a low value to detect as many objects as possible. Evaluate the detection performance across a full range of detection score values.
detectionThresh = 0.1;
detectorResults = detect(frameDetector,odCdsTest, ...
AutoResize=false,MiniBatchSize=1,Threshold=detectionThresh);Calculate object detection metrics on the test set detection results by using the evaluateObjectDetection (Computer Vision Toolbox) function on the test partition of the data set.
metricsObjectDetection = evaluateObjectDetection(detectorResults,odCdsTest);
Calculate and display the average precision (AP) score for each class. Average precision quantifies the ability of the detector to classify objects correctly.
AP = averagePrecision(metricsObjectDetection); disp(table(catsObjectDetection',AP,VariableNames=["Category","AP"]))
Category AP
____________ _______
"BLE_1MHz" 0.85924
"BLE_2MHz" 0.81538
"BT_classic" 0.6855
"WLAN" 0.86357
"collision" 0.24459
The detector performs well when detecting the Bluetooth and Wi-Fi frames, but struggles to capture areas of collision correctly. This is expected, as collisions make it harder to determine the correct signal types and the region over which each signal is present.
Evaluate Detector Errors Using Confusion Matrix
The confusion matrix helps quantify the performance of the detector object across different classes by providing a detailed breakdown of the detection errors. Investigate the types of classification errors made by the detector at a selected detection score threshold by using the confusionMatrix (Computer Vision Toolbox) object function. Use the detection score threshold you determined from the precision recall analysis to discard predictions below that threshold value.
Compute the confusion matrix at a specified score.
[confMat,confusionClassNames] = confusionMatrix(metricsObjectDetection, ...
scoreThreshold=detectionThresh);Display the confusion matrix as a confusion chart.
figure
confusionchart(confMat{1},confusionClassNames)
The results show the network can accurately predict areas in the spectrogram image that contain WLAN and Bluetooth 1 and 2 MHz signals. The network does not classify the collision between two signals well, as there are a significant number of WLAN signals mislabeled as a collision and collisions mislabeled as WLAN signals.
Train a Semantic Segmentation Network
Prepare Data for Training
Semantic segmentation networks assign a value to every pixel of an input image. The network trained in this example assigns values based on the type of waveform detected in each pixel, either BLE_1MHz, BLE_2MHz, BT_classic, WLAN, collision, or no waveform detected. Both the data used to train the network and the output of the trained network must use this format.
Add a category, undefined, to the labeled signal set that represents no waveform detected.
catsSemanticSegmentation = ["BLE_1MHz","BLE_2MHz","BT_classic","WLAN", ... "collision","undefined"]; editLabelDefinition(labeledLss, labelDefinitionName, ... Categories=catsSemanticSegmentation);
This example uses transfer learning on a few existing networks included with MATLAB. The resnet18, resnet50, and mobilenetv2 models all require input image sizes of at least 224-by-224. This example constrains the image dimensions to 256-by-256-by-3 to meet this criterion. You can adjust the image size to observe changes in training speed and overall performance.
imageSize = [256 256];
The labels used to train the network must consist of categorical arrays that match the size of the spectrogram images. Each value within the categorical array corresponds to a single pixel in the associated spectrogram image.
Use the createDatastores object function of the labeledLss object to create a transformed datastore, which contains a signal datastore, and an array datastore, which contains the label information expected by the network for training a semantic segmentation problem. Specify the TimeFrequencyMapFormat argument as "image" to return a uint8 image. Specify the TimeFrequencyImageSize argument to ensure the transformed datastore returns an image of the spectrogram of size 256-by-256. To return an array datastore that defines the labels in each image as a categorical array of the same dimensions as the corresponding spectrogram image, specify the TimeFrequencyLabelFormat argument as "mask".
[ssSds,ssLds] = createDatastores(labeledLss,labelDefinitionName, ... TimeFrequencyLabelFormat="mask", ... TimeFrequencyMapFormat="image", ... TimeFrequencyImageSize=[imageSize(1) imageSize(2)], ... TimeFrequencyIncludePartialBins=true);
As an alternative to computing spectrograms on-the-fly, you can precompute and save the spectrogram images to files. See Creating Stored Data Set for more information.
Transform the array datastore ssLds that contains the labels by using the transform object function, specifying the transformation function as the replaceUnknowns helper function included with this example. This function replaces any missing data values returned by the datastore with the sixth category in the label definition, "undefined".
ssLds = transform(ssLds,@replaceUnknowns);
Inspect Labels
Inspect the first spectrogram and its associated labels within the datastores. Plot the spectrogram and the mask image that identifies the labeled areas of the spectrogram.
Get the first spectrogram and its associated labels.
[specImage,specInfo] = read(ssSds); reset(ssSds) specLabel = preview(ssLds);
Plot the spectrogram and its mask images using the helperSemanticSegmentationPlotter helper function.
helperSemanticSegmentationPlotter(specImage,specInfo,specLabel,catsSemanticSegmentation)
Prepare Training, Validation, and Test Sets
Split the data sets into training, validation, and test sets. Allocate 80% of the data for training, 10% for validation, and 10% for test. Use a random number stream with a specified seed for repeatability of the shuffled file order.
parts = [80 10 10];
s = RandStream("mt19937ar",Seed=0);
numFiles = labeledLss.NumMembers;
shuffledIndices = randperm(s,numFiles);
numTrain = floor(numFiles*parts(1)/100);
numVal = floor(numFiles*parts(2)/100);
ssSdsTrain = subset(ssSds,shuffledIndices(1:numTrain));
ssSdsVal = subset(ssSds,shuffledIndices(numTrain+(1:numVal)));
ssSdsTest = subset(ssSds,shuffledIndices((numTrain+numVal+1):end));
ssLdsTrain = subset(ssLds,shuffledIndices(1:numTrain));
ssLdsVal = subset(ssLds,shuffledIndices(numTrain+(1:numVal)));
ssLdsTest = subset(ssLds,shuffledIndices((numTrain+numVal+1):end));Combine the datastores containing the spectrogram and the label information into a single datastore to use for training.
ssCdsTrain = combine(ssSdsTrain,ssLdsTrain); ssCdsVal = combine(ssSdsVal,ssLdsVal); ssCdsTest = combine(ssSdsTest,ssLdsTest);
Balance Classes Using Class Weighting
Use class weighting to improve the training when classes in the training set are not balanced. Compute the pixel label counts using the countEachPixelLabel helper function, and use the computed counts to calculate the median frequency class weights.
tbl = countEachPixelLabel(ssLdsTrain); numClasses = numel(catsSemanticSegmentation); imageFreq = tbl.PixelCount./tbl.ImagePixelCount; classWeights = median(imageFreq)./imageFreq; classWeights = classWeights/(sum(classWeights)+eps(class(classWeights))); if length(classWeights) < numClasses classWeights = [classWeights; zeros(numClasses-length(classWeights),1)]; end
Select Training Options
Configure the training options by using the trainingOptions (Deep Learning Toolbox) function.
mbs = 40; opts = trainingOptions("sgdm", ... MiniBatchSize=mbs, ... MaxEpochs=20, ... LearnRateSchedule="piecewise", ... InitialLearnRate=0.02, ... LearnRateDropPeriod=10, ... LearnRateDropFactor=0.1, ... ValidationData=ssCdsVal, ... ValidationPatience=5, ... Shuffle="every-epoch", ... OutputNetwork="best-validation-loss", ... Plots="training-progress");
Train Deep Neural Network for Semantic Segmentation
Train the network using the combined training data store ssCdsTrain. The combined training data store contains spectrogram images and true pixel labels. Use weighted cross-entropy loss together with a custom normalization to update the network during training. This example includes a custom loss function, lossFunction, that the crossentropy (Deep Learning Toolbox) loss function and applies custom normalization.
Because training a network can require several hours, this example includes a pretrained network. To train the network yourself, specify trainNow as true. To instead download a MAT file containing the pretrained network, set trainNow to false.
trainNow =false; baseNetwork =
"resnet50"; layers = deeplabv3plus([imageSize(1) imageSize(2)], ... numel(catsSemanticSegmentation),baseNetwork); if trainNow [frameSegmenter,trainInfo] = trainnet(ssCdsTrain,layers, ... @(ypred,ytrue) lossFunction(ypred,ytrue,classWeights),opts); modelDateTime = string(datetime("now",Format="yyyy-MM-dd-HH-mm-ss")); save(fullfile(baseNetworkDir,"trainedRFFrameSemanticSegmenter"+modelDateTime+".mat"), ... "frameSegmenter", "trainInfo") else load(fullfile(baseNetworkDir, "networks.mat"), "frameSegmenter") end
Evaluate Neural Network
Test the network signal identification performance using the test datastore ssSdsTest. Use the semanticseg (Computer Vision Toolbox) function to get the pixel estimates of the spectrogram images in the test datastore. Use the evaluateSemanticSegmentation (Computer Vision Toolbox) function to compute various metrics to evaluate the quality of the semantic segmentation results.
ldsPredictedResults = semanticseg(ssSdsTest,frameSegmenter, ... MiniBatchSize=mbs, ... WriteLocation=tempdir, ... Classes=catsSemanticSegmentation);
Running semantic segmentation network ------------------------------------- * Processed 484 images.
metricsSemanticSeg = evaluateSemanticSegmentation(ldsPredictedResults,ssLdsTest);
Evaluating semantic segmentation results
----------------------------------------
* Selected metrics: global accuracy, class accuracy, IoU, weighted IoU, BF score.
* Processed 484 images.
* Finalizing... Done.
* Data set metrics:
GlobalAccuracy MeanAccuracy MeanIoU WeightedIoU MeanBFScore
______________ ____________ _______ ___________ ___________
0.83011 0.90123 0.59601 0.75272 0.71357
Plot the normalized confusion matrix for all test frames.
figure cm = confusionchart(metricsSemanticSeg.ConfusionMatrix.Variables, ... catsSemanticSegmentation,Normalization="row-normalized")
cm =
ConfusionMatrixChart with properties:
NormalizedValues: [6×6 double]
ClassLabels: ["BLE_1MHz" "BLE_2MHz" "BT_classic" "WLAN" "collision" "undefined"]
Show all properties
cm.Title = "Confusion Matrix - Synthetic";
The results are accurate in correctly identifying each of three Bluetooth classes.
Plot Results of Network Segmentation
Inspect the first spectrogram and associated labels within the datastores. Plot the spectrogram as well as the mask.
Get the first spectrogram and its associated labels.
[specImage,specInfo] = read(ssSdsTest); reset(ssSdsTest); predictedSpecLabel = preview(ldsPredictedResults); actualSpecLabel = preview(ssLdsTest);
Plot the spectrogram and mask images using the helperSpectrumSensingPredictedPlotter helper function.
helperSpectrumSensingPredictedPlotter(specImage,specInfo, ...
predictedSpecLabel,actualSpecLabel,catsSemanticSegmentation)
This visualization shows that the model finds sections within WLAN signals as collisions, even when no collision between signals exists. The model has a hard time differentiating overlapping WLAN signals with collision. The model does a good job detecting signals from undefined noise and retains much of the shape of the ground-truth labels.
Creating Stored Data Set
Rather than exporting the spectrograms from the Signal Labeler app to the MATLAB workspace, you can save the spectrograms to files. This method has the advantage that you must only compute the spectrogram images once, as you can perform all training using the stored images.
From the Signal Labeler app toolstrip in the Data Set section of the Labeler tab, select Create Data Set. In the Create Data Set Files dialog box, specify Folder as the directory in which to create the data set. For problems that are targeting object detection, select [x y w h] for the labels format. For problems that are targeting semantic segmentation, select Mask. Set the Time-frequency map type to Image (RGB), and choose a desired image size with the width and height numeric text boxes and click Create.
After the data set has been created, create a signal datastore for the spectrograms and the labels.
Inspect the READ file in the exported data set location. This file indicates the variable names inside each MAT file.
To create a signalDatastore to use for targeting a semantic segmentation problem, use the timeFrequencyMap and maskLabels variables.
% semanticSegmentationDataSetDatastore = signalDatastore(fullfile(cd,"*.mat"), ... % SampleRate=25e6,SignalVariableNames=["timeFrequencyMap","maskLabels"]);
To create a datastore to use for targeting an object detection problem, use the timeFrequencyMap and the axisAlignedRectangleLabels variables. The label locations and values must be returned in a cell array format. To organize the datastores so that the read function returns a 1-by-3 cell array, use the transform method of the datastore such that the first cell contains the spectrogram image, the second cell contains the label region limits in the axis-aligned rectangle format, and the third cell contains a vector of the label values.
% objectDetectionDataSetDatastore = signalDatastore(... % fullfile(cd, "exportedDataset", "*.mat"), ... % SignalVariableName=["timeFrequencyMap","axisAlignedRectangleLabels"], ... % ReadOutputOrientation="row"); % objectDetectionDataSetDatastore = transform(@(x) [a(1), a{2}], ... % objectDetectionDataSetDatastore);
You can use either type of datastore object in this state to train networks with the trainnet function and trainYOLOXObjectDetector function, as shown in this example.
Conclusions
This example shows how to use the Signal Labeler app to label time-frequency regions of signal spectrograms. You can use the resulting labeled data set to train object detection and semantic segmentation models.
Manually labeling on top of spectrograms can take a long time on larger data sets. To expedite this process through automation, consider this example: Automated Labeling of Time-Frequency Regions for AI-Based Spectrum Sensing Applications.
Supporting Functions
function helperObjectDetectionPlotter(specImage,specInfo,specLabel,categories) hFig = figure; hAx = axes(hFig); image(specImage,Parent=hAx) % Use the specInfo output from the transformed datastore to assign tick labels % on the x-axes and y-axes. yidx = linspace(1,size(specInfo.Frequencies,1),9); yticks(hAx,yidx) yticklabels(hAx, flipud(string(specInfo.Frequencies(round(yidx))/1e6))) ylabel(hAx, "Frequency (MHz)") xidx = linspace(1,size(specInfo.Time,2),11); xticks(hAx,xidx) xticklabels(hAx,string(specInfo.Time(round(xidx))*1e3)) xlabel(hAx, "Time (ms)") % Draw a rectangle on the axis for each label. Assign the associated label % and color by label value. categoryColors = ["red","k","yellow","green","blue","magenta"]; for i = 1:numel(specLabel{2}) hR = drawrectangle(hAx, Position=specLabel{1}(i,:)); hR.InteractionsAllowed = "none"; hR.Label = string(specLabel{2}(i)); hR.Color = categoryColors(hR.Label == categories); hR.FaceAlpha = 0; end end function helperSemanticSegmentationPlotter(specImage,specInfo,specLabel, ... categories) figure tiledlayout("vertical",TileSpacing="tight") hAx = nexttile; image(specImage,Parent=hAx); % Use the specInfo output from the transformed datastore to assign tick % labels on the x-axes and y-axes. yidx = linspace(1,size(specInfo.Frequencies,1),9); yticks(hAx,yidx) yticklabels(hAx,flipud(string(specInfo.Frequencies(round(yidx))/1e6))) ylabel(hAx, "Frequency (MHz)") xidx = linspace(1, size(specInfo.Time,2),11); xticks(hAx,xidx) xticklabels(hAx,string(specInfo.Time(round(xidx))*1e3)) xlabel(hAx, "Time (ms)") % Add a colorbar. colorbar title("Spectrogram Image") % Draw the label output mask. hAx = nexttile; labelValues = double(specLabel{1}); im = imagesc(labelValues,[1 numel(categories)]); % Use the specInfo output from the transformed datastore to assign tick labels % on the x-axes and y-axes. yidx = linspace(1,size(specInfo.Frequencies,1),9); yticks(hAx,yidx) yticklabels(hAx,flipud(string(specInfo.Frequencies(round(yidx))/1e6))) ylabel(hAx, "Frequency (MHz)") xidx = linspace(1,size(specInfo.Time,2),11); xticks(hAx,xidx) xticklabels(hAx,string(specInfo.Time(round(xidx))*1e3)) xlabel(hAx, "Time (ms)") % Add a colorbar that identifies the label category by color. cmap = cool(numel(categories)); ticks = 1:numel(categories); im.Parent.Colormap = cmap; colorbar(TickLabels=cellstr(categories),Ticks=ticks, ... TickLength=0,TickLabelInterpreter="none") title("Ground Truth Labels") end function helperSpectrumSensingPredictedPlotter(specImage,specInfo, ... predictedSpecLabel,actualSpecLabel,categories) figure tiledlayout("vertical",TileSpacing="tight") hAx = nexttile; image(specImage,Parent=hAx) title("Example Spectrogram") % Use the specInfo output from the transformed datastore to assign tick labels on % the x-axes and y-axes. yidx = linspace(1,size(specInfo.Frequencies,1),5); yticks(hAx,yidx) yticklabels(hAx,flipud(string(specInfo.Frequencies(round(yidx))/1e6))) ylabel(hAx, "Frequency (MHz)") xidx = linspace(1, size(specInfo.Time,2),11); xticks(hAx,xidx) xticklabels(hAx,string(specInfo.Time(round(xidx))*1e3)) xlabel(hAx, "Time (ms)") % Add a colorbar. colorbar % Draw the actual label mask. hAx = nexttile; labelValues = double(actualSpecLabel{1}); im = imagesc(labelValues,[1 numel(categories)]); title("Actual Labels") % Use the specInfo output from the transformed datastore to assign tick labels on % the x and y axes. yidx = linspace(1,size(specInfo.Frequencies,1),5); yticks(hAx,yidx) yticklabels(hAx,flipud(string(specInfo.Frequencies(round(yidx))/1e6))) ylabel(hAx, "Frequency (MHz)") xidx = linspace(1, size(specInfo.Time,2),11); xticks(hAx,xidx) xticklabels(hAx,string(specInfo.Time(round(xidx))*1e3)) xlabel(hAx, "Time (ms)") % Add a colorbar that identifies the label category by color. cmap = cool(numel(categories)); ticks = 1:numel(categories); im.Parent.Colormap = flipud(cmap); colorbar(TickLabels=cellstr(categories),Ticks=ticks, ... TickLength=0,TickLabelInterpreter="none") % Draw the predicted label mask. hAx = nexttile; labelValues = double(predictedSpecLabel); im = imagesc(labelValues,[1 numel(categories)]); title("Predicted Labels") % Use the specInfo output from the transformed datastore to assign tick labels on % the x-axes and y-axes axes. yidx = linspace(1,size(specInfo.Frequencies,1),5); yticks(hAx,yidx) yticklabels(hAx,flipud(string(specInfo.Frequencies(round(yidx))/1e6))) ylabel(hAx, "Frequency (MHz)") xidx = linspace(1, size(specInfo.Time,2),11); xticks(hAx,xidx) xticklabels(hAx,string(specInfo.Time(round(xidx))*1e3)) xlabel(hAx, "Time (ms)") % Add a colorbar that identifies the label category by color. cmap = cool(numel(categories)); ticks = 1:numel(categories); im.Parent.Colormap = flipud(cmap); colorbar(TickLabels=cellstr(categories),Ticks=ticks, ... TickLength=0,TickLabelInterpreter="none") end function [labelsOut,info] = replaceUnknowns(dataIn,info) % This function replaces the missing values of a categorical array with % the string "undefined". missingLabels = ismissing(dataIn{1}); labels = dataIn{1}; labels(missingLabels) = "undefined"; labelsOut = {labels}; end function tbl = countEachPixelLabel(lds) labels = read(lds); reset(lds); labels = labels{1}; % Initialize counter cats = categories(labels(1)); s = struct; for i = 1:numel(cats) s.(cats{i}) = 0; end st = s; labels = readall(lds); for i = 1:numel(labels) label = labels{i}; for j = 1:numel(cats) cat = cats{j}; numPixels = sum(label(:) == cat); s.(cat) = s.(cat) + numPixels; if numPixels > 0 st.(cat) = st.(cat) + numel(label(:)); end end end tbl = table(cats,zeros(numel(cats),1),zeros(numel(cats),1), ... VariableNames=["Name","PixelCount","ImagePixelCount"]); for i = 1:numel(cats) tbl.PixelCount(i) = s.(cats{i}); tbl.ImagePixelCount(i) = st.(cats{i}); end end function loss = lossFunction(ypred,yactual,weights) % Compute weighted cross-entropy loss. cdim = strfind(dims(ypred), "C"); loss = crossentropy(ypred,yactual,weights,WeightsFormat="C", ... NormalizationFactor="none"); wn = shiftdim(weights(:)',-(cdim-2)); wnT = extractdata(yactual).*wn; normFac = sum(wnT(:)) + eps("single"); loss = loss/normFac; end
References
[1] Wicht, Jakob, Ulf Wetzker, and Vineeta Jain. “Spectrogram Data Set for Deep-Learning-Based RF Frame Detection.” Data 7, no. 12 (2022): 168. https://doi.org/10.3390/data7120168
See Also
Apps
Objects
labeledSignalSet|yoloxObjectDetector(Visual Inspection Toolbox)
