How to use GPU and CPU simultaneously in a for loop ?

% n be the number of iterations : 6
% A be the two dimensional matrix of size 10*10 stored in three dimension.

for i=1:n
    A(:,:,i)=rand(10,10);
    [Q(:,:,i),R(:,:,i)]=qr(A(:,:,i));
end

1)Here, n is the number of iterations and each iterations can run independently, I wanted to use both CPU and GPU i.e if (i==1) is running on CPU, (i==2) should run on GPU. If for example, GPU completes before CPU, then it should automatically carry on with the next iteration which is (i==3). Once the CPU completes, (i==1), it should proceed with (i==4) and so on. Both GPU and CPU should run asynchronously, where GPU itself should run in an asynchronous manner. How should I implement this ?

2) If i have 2 gpu's then same as the above method, if (i==1) is running on CPU, (i==2) should run on GPU1 and (i==3)should un on GPU2. Which ever completes work faster should take next iteration and so on. How to implement this ?

Answer 1

[Matt J]

You can't make the CPU and GPU do alternating loop iterations, but you can divide the slices of A into two batches, and make the CPU and GPU work asynchronously on their respective batch, as below.

Acpu=A(:,:,1:n/2);                  %chunk #1 : send to CPU
Agpu=gpuArray(A(:,:,n/2+1:end));    %chunk #2 : send to GPU with device index 1

p=parpool(2);
F(1)=parFeval(p, @deploy,2,Acpu);
F(2)=parFeval(p, @deploy,2,Agpu,1);

[Q,R] = fetchOutputs(F,'UniformOutput',false); % Blocks until complete

 Q=cat(3,Q{1},gather(Q{2}));
 R=cat(3,R{1},gather(R{2}));
 
function [q,r]=deploy(a,Id)

     if nargin>1, gpuDevice(Id);end
     for i=size(a,3):-1:1  
         [q(:,:,i),r(:,:,i)]=qr(A(:,:,i));
     end
 
end

Naturally, you should pass the gpuDevice ID numbers that are actually assigned to the graphics cards on your machine. Above, I've assumed they are numbered 1,2, etc...

[Matt J]

For two GPUs it is quite similar,

Acpu=A(:,:,1:n/3);                 %chunk #1 : send to CPU
Agpu1=gpuArray(A(:,:,n/3+1:2*n/3));%chunk #2 : send to GPU with device index 1
Agpu2=gpuArray(A(:,:,2*n/3+1:n));  %chunk #3 : send to GPU with device index 2


p=parpool(3);

F(1)=parFeval(p, @deploy,2,Acpu);
F(2)=parFeval(p, @deploy,2,Agpu1,1);
F(3)=parFeval(p, @deploy,2,Agpu2,2);

[Q,R] = fetchOutputs(F,'UniformOutput',false); % Blocks until complete

 Q=cat(3,Q{1},gather(Q{2}), gather(Q{3}) );
 R=cat(3,R{1},gather(R{2}), gather(R{3}) );

[Srinidhi Ganeshan]

Thank you for your Answer @Matt j . when i execute the above code i am getting an error

"fetchOutputs must be called either with a single input argument, or with two additional arguments specifying 'UniformOutput' as a logical scalar", when i come to the line

[Q,R] = fetchOutputs(F,'UniformOutput',0); % Blocks until complete 

I am not able to figure this out as i am not familiar with the fetchOutput function.

[Matt J]

I guess it wants this,

[Q,R] = fetchOutputs(F,'UniformOutput',false);

In any case, here is the documentation on fetchOutputs.

[Srinidhi Ganeshan]

F(2)=parfeval(p, @deploy,2,Agpu1,1);
F(3)=parfeval(p, @deploy,2,Agpu2,2);

Apparently, giving id as 1,2 isn't working. Is there any other way, one could define Gpu device Index in parfeval such that it can work ?

[Matt J]

You should pass the gpuDevice ID numbers that are actually assigned to the graphics cards that you are trying to use.

 for i=1:gpuDeviceCount
    gpuDevice(i),
 end

[Rachel Chen]

@Matt, can I use parfor with CPU and GPU like this as well?