Concepts of Parallel Computing in Statistics and Machine Learning Toolbox

Subtleties in Parallel Computing

There are two main subtleties in parallel computations:

• Nested parallel evaluations (see No Nested parfor Loops). Only the outermost parfor loop runs in parallel, the others run serially.

• Reproducible results when using random numbers (see Reproducibility in Parallel Statistical Computations). How can you get exactly the same results when repeatedly running a parallel computation that uses random numbers?

Vocabulary for Parallel Computation

• worker — An independent MATLAB^® session that runs code distributed by the client.

• client — The MATLAB session with which you interact, and that distributes jobs to workers.

• parfor — A Parallel Computing Toolbox™ function that distributes independent code segments to workers (see Working with parfor).

• random stream — A pseudorandom number generator, and the sequence of values it generates. MATLAB implements random streams with the RandStream class.

• reproducible computation — A computation that can be exactly replicated, even in the presence of random numbers (see Reproducibility in Parallel Statistical Computations).