Parallel Computing Toolbox™ supports distributed arrays to partition large arrays across multiple
MATLAB® workers. You operate on the entire array as
a single entity, however, workers operate only on their part of the array, and automatically
transfer data between themselves when necessary. Simultaneous execution is supported by the
single program multiple data (
spmd) language construct to facilitate
communication between workers. Use distributed-enabled matrix operations and functions to
work directly with these arrays without further modification. You can use distributed arrays
in Parallel Computing Toolbox to run big data applications using the combined memory of your cluster.
|Create distributed array from data in the client workspace or a datastore|
|Transfer distributed array or gpuArray to local workspace|
|Execute code in parallel on workers of parallel pool|
|Create Composite object|
|Build parallel.pool.Constant from data or function handle|
|Create codistributed array from replicated local data|
|Create parallel pool on cluster|
|Shut down parallel pool|
|Redistribute codistributed array with another distribution scheme|
|Create codistributed array from distributed data|
|Local portion of codistributed array|
|Global indices for local part of codistributed array|
|Global operation across all workers|
|Write distributed data to an output location|
|Apply function to each page of distributed array or gpuArray|
|Access elements of distributed arrays from client|
|Access elements of arrays distributed among workers in parallel pool|
|Access nondistributed variables on multiple workers from client|
|1-D distribution scheme for codistributed array|
|2-D block-cyclic distribution scheme for codistributed array|
|Parallel pool of workers|
Examples and How To
When your data array is too big to fit into the memory of a single machine,
you can create a
MATLAB functions that operate on distributed arrays
create distributed arrays and partition the data among your workers
spmd statements to run the
same code on multiple datasets and control codistributed arrays
Composite objects in the MATLAB client session let you directly access data values on the workers.
Train Network in Parallel with Custom Training Loop (Deep Learning Toolbox)
This example shows how to set up a custom training loop to train a network in parallel.
In this example, we look at the
gop function and the functions that build on it:
This example shows the basics of working with spmd statements, and how they provide an interactive means of performing parallel computations.
Compare and contrast
spmd against other parallel computing
functionality such as
Learn about starting and stopping parallel pools, pool size, and cluster selection.
Specify your preferences, and automatically create a parallel pool.
Describes the various types of arrays used in communicating jobs
Describes how to use codistributed arrays for calculation
Describes how to program a
with codistributed arrays
Work with remote data in Amazon S3™, Azure® Blob Storage, or HDFS™.