Client to worker communication in Matlab

Is there a way to let the cleint send data back to parallel workers in Matlab. Using a queue make the data sending from worker to the client easy, but vice versa is proving diffcult.

I am trying to use a ascii file that is written by the client for the workers to read, but simulataneous accessing of the file is giving me an error.

Error using startSimulation>mainCore (line 440)

"File temp/current.txt is in use by another process or thread. You should be able to load data once the other process or thread has released the file."

Is there a way to wait for the file to be readable (waitfor doesn't work), or implemeting a easy semaphore code to do this? I am open to implementing any ideas, there has to be a communication between client and workers both ways.

[Edric Ellis]

It would be helpful if you could post a minimal reproduction to demonstrate the problem. I presume you must be using parfeval to trigger execution on the workers - because otherwise the client would be blocked executing the parfor or spmd block otherwise... Could you perhaps only schedule the parfeval calls after the client has completed?

[Anurag Kamal]

Well, I am using parfor, adn this is how the pseudo code looks like

mainscript()
    initialize queue;
    aftereach(queue,@purgefunc)
    parfor i=1:3
        results(i)=start_sim(params, queue)
    end
end mainscript()

purgefunc(data)
    if data==!
        plot(results)
    end if
end purgefunc

startsim(params, queue)
    while t<tf // final simulation time
        send.queue(data)
    end
end

Now as the loop on workers is continuously running, how do I send data from client to worker.

Answer 1

[Walter Roberson]

https://www.mathworks.com/help/parallel-computing/parallel.pool.pollabledataqueue.html

"You can construct the queue on the workers and send it back to the client to enable communication in the reverse direction. However, you cannot send a queue from one worker to another. Use spmd, labSend, or labReceive instead."

[Anurag Kamal]

if the workers are in a loop, how can they send a variable back to the client. Sending the pollable queue by the queue doesn't work, I tried it.

Answer 2

[Davey Gregg]

I have been hitting my head against this problem for a while too. There does not seem to be an easy way to do this. I found this way works - it's not ideal because it takes some time to bounce the command to the hard drive and get it back. But it will let you send a command to the workers from anywhere beit the main client, a function, or even a seperate script. Basically you write a file to tempdir where you can save your command and then have the workers access the file to receive it.

%Make a matObj file in the temp directory
G = 1;
matObj = matfile([tempdir,'G.mat'],'Writable',true);
matObj.G = G;

% Setup a queue to receive data from workers
dataOut = parallel.pool.DataQueue;
afterEach(dataOut,@sendIT);

spmd (4)
    if labindex == 1
        t = 1;
        while G == 1
            G = matObj.G; % Access the matObj to recieve data from client
            pause(1);
            t = t+1;
            if t == 5
                disp('sent')
                send(dataOut,0)
            end
        end
    end
end  

delete([tempdir,'G.mat']);

% update matObj with command from client
function sendIT(data)
    matObj = matfile([tempdir,'G'],'Writable',true);
    matObj.G = data;
end

Answer 3

[Sahil Islam]

%load("data.txt);

data = readtable("data.txt);

data = table2array(data);

Instead of loading, using 'readtable' solved the problem for me.