How can I send a variable from the client to a running function on a worker?

If I have a function running on a worker via parfeval or a job, can I have it run in a loop until I send a stop signal to it from the client?

Answer 1

[Ed Mitchell]

Here is an example that demonstrates how you can use PollableDataQueue to send the command across to the function running a while loop on a worker via parfeval

%open parallel pool or grab handle to existing pool
p=gcp;

%create data queue to send data to display from worker to client
queueDisp=parallel.pool.DataQueue;
afterEach(queueDisp,@disp);

%create data queue to send the worker's data queue back to the client
queue1 = parallel.pool.PollableDataQueue; 

%call the function
f = parfeval(@test, 0, queue1,queueDisp); 

%receieve the data queue created on the worker back to the client 
ok=false;

while ~ok
    [queue2,ok] = poll(queue1,1)
end
disp("queue acquired")

%wait 10 seconds then send stop command.  This send command could also be
%run directly from the command window
pause(10)
send(queue2,"stop")

function test(queue1,queueDisp) 

    %create queue on worker to send data to client
    queue2 = parallel.pool.PollableDataQueue;

    %use queue1 to send reference to queue2 back to the client
    send(queue1,queue2);

    a="ready";
    while ~strcmp(a,"stop")
        send(queueDisp,a);%send data created on worker back to client
        [a,ok]=poll(queue2,1);%check to see if stop command sent from client;
    end
end 

If you need similar functionality on a batch job instead of an interactive pool here is an example of how you can use ValueStore to send a stop command to function running a while loop on a worker via batch

%get cluster object
c=parcluster("Processes");

%call the function
j = batch(c,@test, 2); 

%get ValueStore object
store=j.ValueStore;
store("stop_command")="go";

%wait 6 seconds then send stop command. This could also be done from the
%command window.

pause(6)
store("stop_command")="stop";

wait(j);
outputs=fetchOutputs(j);

function [a,b] = test() 
    %get handle to valuestore
    store=getCurrentValueStore;
    a=0;
    while ~strcmp(store("stop_command"),"stop")
        pause(1)
        a=a+1;
    end
    b="complete"
end 

Answer 2

[Thomas Falch]

Starting in R2025a, it's possible to use the new "any-destination" PollableDataQueue to greatly simplify Ed's solution. The any-destination queue makes it much simpler to send data from the client to a worker (or from one worker to another).

The new queue is documented here: PollableDataQueue - Send and poll data between client and workers - MATLAB

The code to send a stop signal to a worker could look like this:

queue = parallel.pool.PollalbleDataQueue(Destination="any");

f = parfeval(@runUntilStopSignalReceived, 0, queue);

% Let the function run for a while, then stop it
pause(10);
queue.send("stop")

function runUntilStopSignalReceived(queue)

keepGoing = true
while keepGoing
    doSomeWork()
    [data, didReceive] = queue.poll()
    if didReceive && strcmp(data, "stop")
        % We got the stop signal, stop
        break;
    end
end