objectInsertionDatastore

R2026b

Create synthetic labeled datastore for training an instance segmentation or object detection network

Since R2025a

expand all in page

Description

The objectInsertionDatastore object blends datastores of labeled images containing foreground objects with destination images containing the background. Using this object, you can create a synthetic labeled datastore for training an instance segmentation or object detection network.

To insert an object into a single background image at a randomized or specified location, use the insertObjectInImage function.

Note

This functionality requires Deep Learning Toolbox™.

Creation

Syntax

dsNew = objectInsertionDatastore(dsDestination,dsObject,numNewImages)
dsNew = objectInsertionDatastore(___,PropertyName=Value)

Description

dsNew = objectInsertionDatastore(dsDestination,dsObject,numNewImages) creates an augmented datastore dsNew by inserting objects from the source datastore dsObject into the destination datastore containing image data, dsDestination. The objects from the source images are inserted at randomized locations on the destination images. You must specify the number of images synthetic images to create, numNewImages.

dsNew = objectInsertionDatastore(___,PropertyName=Value) sets writeable properties using one or more name-value arguments, in addition to the input argument from the previous syntax. For example, NumObjectsToInsert=2 specifies the number of objects to insert as 2.

example

Input Arguments

expand all

Destination datastore of images, specified as a datastore such as an ImageDatastore or CombinedDatastore object.

To insert objects at specific image locations using a region constraint mask, specify the dsDestination argument as a datastore that returns a cell array with two columns, {Image ObjectMask}, when called with the read function. Image is an array of RGB or grayscale images, and ObjectMask is a logical array of masks which span object regions to insert.

Datastore of source images containing objects, specified as a datastore such as an ImageDatastore or CombinedDatastore object. You must set up your data so that calling the read and readall functions on the datastore returns a cell array with these columns.

Image DataBoxesLabelsMasks

RGB or grayscale image containing objects to insert, specified as an H-by-W-by-3 or H-by-W-by-1 numeric array, respectively.

Bounding boxes, defined in spatial coordinates as an M-by-4 numeric matrix with rows of the form [x y w h], where:

  • M is the number of axis-aligned rectangles.

  • x and y specify the upper-left corner of the rectangle.

  • w specifies the width of the rectangle, which is its length along the x-axis.

  • h specifies the height of the rectangle, which is its length along the y-axis.

Object class names, specified as an M-by-1 categorical vector, where M is the number of specified bounding boxes. All categorical data read from the datastore must contain the same categories.

Binary masks, specified as a logical array of size H-by-W-by-M, where M is the number of specified bounding boxes. Each channel is a mask, and each mask is the segmentation of one object in the image.

Number of augmented images to create, specified as a positive integer. In every call to the datastore with the read function, the objectInsertionDatastore object inserts random object from the datastore dsObject into a random image from the datastore dsDestination.

Properties

expand all

Number of objects to insert from the source datastore, dsObject, specified as a positive integer or a range of positive integers [minNum,maxNum]. If you specify a range of positive integers [minNum,maxNum], the objectInsertionDatastore object inserts a random number of objects into each image in the destination datastore dsDestination.

Output format of the data returned by calling the augmented datastore dsNew with the read function, specified as one of these options:

  • "InstanceSegmentation" – Instance segmentation data containing image data, bounding boxes, masks, and labels

  • "ObjectDetection" – Object detection data containing image data, bounding boxes and labels

  • "SceneClassification" – Scene classification, or semantic segmentation data, containing image data and masks

Maximum object overlap between an object in an image from the datastore dsObject and a previously inserted object, specified as a positive scalar in range [0, 1]. Specify the ObjectsInSceneMaxOverlap value as 1 to insert new objects on top of existing objects. Decrease this value to decrease the overlap between the newly inserted objects and existing objects.

Boundary constraint mode used to constrain objects at the boundaries of images in the destination datastore dsDestination, specified as one of these options:

ValueEdge Detection Mechanism
"inbounds-bbox"

To insert an object, the entire bounding box containing the object must be completely within the destination image.

"inbounds-exact"To insert an object, the object mask must be completely within the destination image.
"center"Objects are inserted when the object centroid lies inside the destination image, so objects can be partially visible and clipped at the image boundary.

Maximum insertion attempts to satisfy the overlap value, specified as a positive scalar. The overlap value is represented by the ObjectsInSceneMaxOverlap property. If no insertion location can be found on the destination image after MaxInsertionAttempts iterations, the object from the source datastore image will not be inserted into the destination datastore image.

Blend method to use for inserting objects from the images in the source datastore dsObject into the images in the destination datastore dsDestination, specified as a one of these options:

BlendMethod ValueBlend Method
"guidedfilter"

Guided filter blending [1] – Blur the object mask of the source image at the mask-object boundary. Guided image filtering uses edge context from the source image to filter the mask prior to blending the object image with the destination image.

"poisson"

Poisson blending [2] – Blend the source object into the destination image by matching the color distribution of the source object to that of the boundary between the destination image and source object mask.

"none"

No blending – At insertion locations of the destination image, replace pixels of the destination image by the source image pixel values.

Geometric augmentation, specified as a function handle that returns an affinetform2d object. The geometric augmentation warps a source image and the mask associated with each object in the image before it is inserted into the destination image. For example, specify the GeometricAugmentation value as @() randomAffine2d("Scale",[0.9,1.1]) to apply a randomized scale transformation to each inserted object.

Examples

collapse all

Open Live Script

Create a datastore containing synthetic anomalous images by using normal images of industrial parts from the RobustAD data set, attached to this example as a folder of supporting files.

Create Destination Datastore

Create a combined datastore for the destination background images paired with their region constraint masks. The region constraint masks define the valid areas on the plate surface at which to insert anomalies, preventing placement on the background.

destFiles = dir(fullfile("PlateData","normal","Normal_*.jpg"));
dsDest = imageDatastore(fullfile({destFiles.folder},{destFiles.name}));

maskDestFiles = dir(fullfile("PlateData","normalMasks","NormalMask_*.png"));
dsRegionMask = imageDatastore(fullfile({maskDestFiles.folder},{maskDestFiles.name}));

dsDestination = combine(dsDest,dsRegionMask);

Visualize Sample from Destination Datastore

Read a sample from the destination datastore to visualize the normal plate image and its corresponding region constraint mask. Reset the datastore after reading to return the read pointer to the beginning for subsequent operations.

sample = read(dsDestination);
figure
tiledlayout(1,2)
nexttile
imshow(sample{1})
title("Normal Plate Image")
nexttile
imshow(sample{2},[])
title("Region Constraint Mask")

Figure contains 2 axes objects. Hidden axes object 1 with title Normal Plate Image contains an object of type image. Hidden axes object 2 with title Region Constraint Mask contains an object of type image.

reset(dsDestination)

Create Source Object Datastore

Create a combined datastore for the source objects. The source datastore must return four elements per read: the source image, bounding boxes, categorical labels, and a 3-D logical mask array.

srcFiles = dir(fullfile("PlateData","anomalyImages","Image_*.jpg"));
dsSrcImages = imageDatastore(fullfile({srcFiles.folder},{srcFiles.name}));

Compute Bounding Boxes and Per-Object Masks

Extract individual object bounding boxes and per-object masks from each anomaly mask by using connected component analysis. Save a separate annotation MAT file for each anomaly image.

maskFiles = dir(fullfile("PlateData","anomalyMasks","anomalyMask_*.png"));
matDir = fullfile(pwd,"anomalyAnnotations");
if ~isfolder(matDir)
    mkdir(matDir);
end

numImages = numel(maskFiles);
for i = 1:numImages
    [~,name] = fileparts(maskFiles(i).name);
    matFilePath = fullfile(matDir,name + ".mat");
    [boxes,labels,masks3D] = extractObjectData(fullfile(maskFiles(i).folder,maskFiles(i).name));
    save(matFilePath,"boxes","labels","masks3D");
end

Create File Datastore for Object Annotations

Use a single fileDatastore to load annotations from the saved MAT files. Then use the transform function to convert the datastore to the four-element cell array format required by the objectInsertionDatastore object.

dsAnnotations = fileDatastore(fullfile(matDir,"*.mat"),ReadFcn=@(f) load(f));
dsCombined = combine(dsSrcImages,dsAnnotations);
dsSource = transform(dsCombined,@(data) {data{1},data{2}.boxes,data{2}.labels,data{2}.masks3D});

Visualize Sample from Source Datastore

Read a sample from the source datastore to visualize the anomaly image and its per-object mask overlay. Reset the datastore after reading so that the next read starts from the first image.

sample = read(dsSource);
srcImage = sample{1};
srcBoxes = sample{2};
srcMask = any(sample{4},3);
figure
tiledlayout(1,2)
nexttile
annotated = insertObjectAnnotation(srcImage,"rectangle",srcBoxes,"Anomaly");
imshow(annotated)
title("Anomaly Image with Bounding Boxes")
nexttile
imshow(srcMask)
title("Anomaly Mask")

Figure contains 2 axes objects. Hidden axes object 1 with title Anomaly Image with Bounding Boxes contains an object of type image. Hidden axes object 2 with title Anomaly Mask contains an object of type image.

reset(dsSource)

Create Synthetic Datastore

Create a datastore of synthetic anomaly images by using the objectInsertionDatastore object to generate eight synthetic images. Specify Poisson blending for seamless defect insertion into the plate surface.

numSyntheticImages = 8;
dsNew = objectInsertionDatastore(dsDestination,dsSource,numSyntheticImages, ...
    BlendMethod="poisson");
dsNew
dsNew = 
  objectInsertionDatastore with properties:

                    NumReads: 8
          NumObjectsToInsert: 1
                OutputFormat: "InstanceSegmentation"
      BoundaryConstraintMode: "inbounds-bbox"
    ObjectsInSceneMaxOverlap: 1
        MaxInsertionAttempts: 10
                 BlendMethod: "poisson"
       GeometricAugmentation: @()affinetform2d()

Visualize Synthetic Data

Read a synthetic anomaly image from the datastore, and visualize the generated image with annotated bounding boxes around the inserted anomalies.

figure
result = read(dsNew);
synImage = result{1};
boxes = result{2};
labels = result{3};
annotated = insertObjectAnnotation(synImage,"rectangle",boxes,string(labels));
imshow(annotated)
title("Synthetic Image")

Figure contains an axes object. The hidden axes object with title Synthetic Image contains an object of type image.

Next Steps

You can now use the synthetic datastore to train an object or anomaly detector. Reset the datastore so that a training function can iterate over all synthetic images.

reset(dsNew)

Helper Functions

extractObjectData

Extract individual defect bounding boxes, categorical labels, and per-object 3-D logical masks from a binary anomaly mask file by using connected component analysis.

function [boxes,labels,masks3D] = extractObjectData(maskFilePath)
    m = imread(maskFilePath);
    cc = bwconncomp(m);
    numObj = cc.NumObjects;
    props = regionprops(cc,"BoundingBox");
    boxes = vertcat(props.BoundingBox);
    labels = repmat(categorical("Anomaly"),numObj,1);
    L = labelmatrix(cc);
    masks3D = false(size(m,1),size(m,2),numObj);
    for j = 1:numObj
        masks3D(:,:,j) = (L == j);
    end
end

References

[1] He, Kaiming, Jian Sun, and Xiaoou Tang. “Guided Image Filtering.” IEEE Transactions on Pattern Analysis and Machine Intelligence 35, no. 6 (June 2013): 1397–1409. doi:10.1109/TPAMI.2012.213.

[2] Pérez, Patrick, Michel Gangnet, and Andrew Blake. “Poisson Image Editing.” In ACM SIGGRAPH 2003 Papers, 313–18. San Diego California: ACM, 2003. doi:10.1145/1201775.882269.

Version History

Introduced in R2025a

expand all

See Also

| | | | |

Topics