Detect when no GPU is selected

In the documentation on gpuDevice() it says that,

"gpuDevice([]), with an empty argument (as opposed to no argument), deselects the GPU device and clears its memory of gpuArray and CUDAKernel variables. . This leaves no GPU device selected as the current device."

Is there a way to detect whether this 'no GPU' state is the current state? The intuitive way to check would be the following,

    g=gpuDevice;
    isempty(g.Index)

This does not work, however, because a call to gpuDevice with no input arguments always results in the selection of a device. Thus, g.Index can never be empty.

[Matt J]

Maybe some elaboration is in order. Our lab has GPGPU code that calls the GPU from MATLAB entirely through shared libraries involving a lot of our own CUDA code. An M-coded handle class object is used to invoke these libraries as needed.

Currently, when the handle object is cleared, the class delete() method is called and issues a clearing command to the GPU similar (but not indentical) to invoking gpuDevice([]). However, problems arise when there are also gpuArray variables present in the workspace. Any subsequent attempt to use the gpuArray variables after the handle object is deleted then crash the GPU.

So the question is, during deletion of the handle object, what needs to be done to the GPU (what state does it need to be left in) to safely let go of the graphics card and to ensure that the gpuArray variables continue to be valid?

Answer 1

[Joss Knight]

You can query device properties without selecting it using parallel.gpu.GPUDevice.getDevice. So, for instance, you can find the selected device without selecting it like this:

   selectedDevice = [];  % Nothing selected
   for i = 1:gpuDeviceCount
       if parallel.gpu.GPUDevice.getDevice(i).DeviceSelected
           selectedDevice = i;
           break;
       end
   end

For your purposes, you can use this handy internal:

   parallel.internal.gpu.isAnyDeviceSelected

Since you're writing your own CUDA code (using MEX, presumably), you can also just query device properties using the CUDA runtime API (e.g. cudaGetDeviceProperties), or the driver API (e.g. cuDeviceGetProperties).

In answer to your 'elaboration' question, I'd need to know what you're actually doing. If you are resetting the device (using, for instance, cudaDeviceReset) then there's nothing you can do to save your gpuArrays - the device memory has been cleared and every pointer into GPU memory invalidated. Since your GPU is crashing when you subsequently attempt to read those gpuArrays it seems like this is what you're doing. Your only hope would be to copy everything to the CPU and back after the reset. For instance, you could use the mex API to call out to a MATLAB function which walks through the workspace, finds all gpuArrays, copies them to CPU arrays, and then another to copy back afterwards. That's a bit crazy because it could be a huge task to check every workspace variable and its contents.

Perhaps all you really want to do is ensure your card doesn't crash. In that case MATLAB needs to know you reset the device, so call reset(gpuDevice) or gpuDevice([]) in MATLAB instead of doing it your own code.

[Matt J]

That looks like what we needed, Joss. Thanks a lot.