Out of memory smpd parallel computing toolbox eig on complex matrix

10 views (last 30 days)
Hi, I have an 11x11 matrix from a system of 11 ODEs (hence the complexity). Matlab's eig was unable to solve the matrix without running out of memory, so I'm trying out the parallel computing toolbox. I haven't been able to find any clear instructions, so I may be doing very obvious things wrong. My code is:
parpool
spmd
tic
[l1,l2] = eig(D);
time = gop(@max, toc) % Time for all labs in the pool to complete.
end
delete(gcp)
and D is my matrix. I set up my local profile so that I use 6 workers. My question is: am I using the toolbox/functions incorrectly, or is my system just too complex to solve? In case relevant, I am running Matlab 2016b on win10 64, 4 ghz cpu, 24 gb ram. It gets up to almost 100% memory usage and about 80% cpu usage with this code.
The error:
Starting parallel pool (parpool) using the 'local' profile ... connected to 6 workers.
ans =
Pool with properties:
Connected: true
NumWorkers: 6
Cluster: local
AttachedFiles: {}
IdleTimeout: 30 minute(s) (30 minutes remaining)
SpmdEnabled: true
Warning: An error has occurred during SPMD execution. An attempt has been made to interrupt execution on the workers. If this situation persists, it may be necessary to interrupt execution using CTRL-C and then deleting and
restarting the parallel pool.
The error that occurred on worker 3 is:
Error using symengine
Out of memory.
Error in sym/mupadmexnout (line 981)
out = mupadmex(fcn,args{:});
Error in sym/eig (line 54)
[Vsym,Dsym,p] = mupadmexnout('symobj::eigenvectors',A);
Error in eigen_test1 (line 50)
spmd
.
> In spmdlang.RemoteSpmdExecutor/maybeWarnIfInterruptedAndWaiting (line 300)
In spmdlang.RemoteSpmdExecutor/isComputationComplete (line 131)
In spmdlang.spmd_feval_impl (line 19)
In spmd_feval (line 8)
In eigen_test1 (line 50)
Error using eigen_test1 (line 50)
Error detected on worker 3.
Caused by:
Error using symengine
Out of memory.
Exception in thread "Explorer NavigationContext request queue" IdleTimeout has been reached.
Parallel pool using the 'local' profile is shutting down.
  1 Comment
Edric Ellis
Edric Ellis on 3 Feb 2017
In this case, unfortunately I think you are simply replicating the same computation on each worker. Inside an spmd block, to get parallelism, you need to use codistributed arrays. But it looks here like you are using symbolic computation.

Sign in to comment.

Accepted Answer

Prannay Jain
Prannay Jain on 7 Feb 2017
I understand that you are running into memory issues even after parallelizing the computation using spmd and parpool.
Moving to Parallel Computing Toolbox (PCT) while remaining on the same machine will never resolve a memory issue. By splitting the computation across multiple workers makes the memory situation worse by having extra MATLABs open. You will either need a bigger machine with more memory or try considering out a cluster where you can appropriately distribute the problem across multiple machines' memory using MATLAB Distributed Computing Server (MDCS) and PCT.
  1 Comment
Renee
Renee on 8 Feb 2017
Thank you for the clarification. I should have read the docs more carefully.

Sign in to comment.

More Answers (0)

Categories

Find more on Parallel Computing Fundamentals in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by