How use BLOCKPROC with SEMANTICSEG -> ERROR "Out of memory on device. To view more detail about available memory on the GPU"

Dear all,

I hope this message finds you well. I am in need of using the SEMANTICSEG command to segment a large image with dimensions (21828 x 54644 x 3) uint8. However, as this image is quite large, the SEMANTICSEG function cannot process the entire image at once. Consequently, I have employed the BLOCKPROC function to send blocks for processing by SEMANTICSEG. Please find the code below:

% Load the original image (21828 x 54644 x 3)
I = imread('img_lote3.tif');

% Extract information about the image size
[num_rows, num_cols, num_channels] = size(I);

% Calculate the Greatest Common Divisor (GCD) between the two values
gcd_value = gcd(num_rows, num_cols);

% Calculate the block size to be used in BLOCKPROC (5457 x 13661)
block_size = [(num_rows ./ gcd_value) (num_cols ./ gcd_value)];

% Create the SEMANTICSEG function, adjusting the output with RENAMECATS, STRING, DOUBLE, UINT8 functions
processing_function = @(block) uint8(double(string((renamecats(semanticseg(block.data, network),{'0','1','2','3','4','5','6','9'})))));
% processing_function = @(block) semanticseg(block.data, network); without adjusting the output

% Call for the processing
Result = blockproc(I, block_size, processing_function); 

Upon running this code, the following error messages are presented:

BLOCKPROC encountered an error while evaluating the user-supplied function handle, FUN.

The cause of the error was:

Error using @(block)uint8(double(string((renamecats(semanticseg(block.data,net),{'0','1','2','3','4','5','6','9'})))))

Out of memory on device. To view more detail about available memory on the GPU, use 'gpuDevice()'.

If the problem persists, reset the GPU by calling 'gpuDevice(1)'.

Error in blockprocFunDispatcher (line 13) output_block = fun(block_struct);

Error in blockprocInMemory (line 80) [ul_output fun_nargout] = blockprocFunDispatcher(fun,block_struct,...

Error in blockproc (line 251) result_image = blockprocInMemory(source,fun,options);

I would like to understand what I'm doing wrong with this procedure. Could someone please assist me?

My Laptop Configuration:

Processor: 12th Gen Intel(R) Core(TM) i7-12700H 2.30 GHz

Installed RAM: 32.0 GB (usable: 31.7 GB)

System Type: 64-bit operating system, x64-based processor

MatLab Version: R2023b

CUDADevice with properties:

Name: 'NVIDIA GeForce RTX 3070 Ti Laptop GPU'

Index: 1

ComputeCapability: '8.6'

SupportsDouble: 1

GraphicsDriverVersion: '532.09'

DriverModel: 'WDDM'

ToolkitVersion: 11.8000

MaxThreadsPerBlock: 1024

MaxShmemPerBlock: 49152 (49.15 KB)

MaxThreadBlockSize: [1024 1024 64]

MaxGridSize: [2.1475e+09 65535 65535]

SIMDWidth: 32

TotalMemory: 8589410304 (8.59 GB)

AvailableMemory: 7258959872 (7.26 GB)

CachePolicy: 'balanced'

MultiprocessorCount: 46

ClockRateKHz: 1485000

ComputeMode: 'Default'

GPUOverlapsTransfers: 1

KernelExecutionTimeout: 1

CanMapHostMemory: 1

DeviceSupported: 1

DeviceAvailable: 1

DeviceSelected: 1

Thank you for your attention.

Answer 1

[Shubham]

Hey Airton,

It seems that you are trying to use "semanticseg" for segmenting a large image using "blockproc" function to process the image in blocks.

Since uint8 takes 1 byte of memory, the image is as size can be calculated as follows:

% Size in MB
MB = (21828 * 54644 * 3) / (1024 * 1024) 

MB = 3.4125e+03

% Size in GB
GB = MB / 1024

GB = 3.3326

You are using gcd for dividing the image in blocks. The gcd is calculated as follows:

GCD = gcd(21828,54644)

GCD = 4

I understand that using gcd for dividing the images into block would help in maintaining the aspect ratio of the image. However, in this case, the block size for image is very large:

% block size in MB
block_size = MB/GCD

block_size = 853.1350

The total memory consumption for the segmentation would be dependent on the size of the network, memory used by the network to perform the computations and the input size. Just dividing the image into 4 blocks may not be sufficient.

For dividing the image into blocks, figure out a block size as per your requirement and then create blocks after padding the image. In this way you can control the block size for performing any computations. For more information, refer to the following MATLAB Answer which also provides a helper function for creating image blocks: https://www.mathworks.com/matlabcentral/answers/385273-why-do-i-get-maximum-variable-size-allowed-on-the-device-is-exceeded-error-when-running-the-sema

Another approach could be to downsize the image. You can use "imresize" function in MATLAB to downsize the image. The documentation also provides various interpolation methods used by the "imresize" function: https://www.mathworks.com/help/matlab/ref/imresize.html

Note: the solution would vary as per your use case.

1. You can divide the image into blocks where underlying information(pixels) are highly repetitive. Each segment of the image can be treated as a new image without the hampering the accuracy. For example the "triangleTest.jpg" image used in the documentation of "semanticseg" : https://www.mathworks.com/help/vision/ref/semanticseg.html#:~:text=I%20%3D%20imread(%22triangleTest.jpg%22)%3B
2. Downsize the image if it contains a large object. In such a case, breaking the image into chunks would not provide accurate results. Segmenting a human being from background could be an example for this case.

You may also refer to the following MATLAB Answers post:

https://www.mathworks.com/matlabcentral/answers/427468-how-does-semanticseg-command-work-on-images-larger-than-what-the-network-was-trained-with

I hope this helps!

[Shubham]

Hey @Airton Gaio Junior, did you find the above answer helpful? If so, could you mark this as accepted?