Create and Discover Clusters

Create a Cloud Cluster

If you are new to cloud clusters, see Getting Started with Cloud Center.

1. Sign in to Cloud Center at https://cloudcenter.mathworks.com. If you are in MATLAB^®, you can also select Parallel > Create and Manage Clusters in the Environment section in the Home tab to go to Cloud Center.

2. If you have never used Cloud Center before, there is a one-time setup to link your cloud account with Cloud Center. See Link Your Cloud Account to Cloud Center.

3. To start any cloud resources, click the Cloud Resources tab.

4. Next to MATLAB Parallel Server™, click Create.

   

5. On the Create Cloud Resource page, configure your cluster settings. The settings for the cloud machine configuration in Cloud Center are adapted from the following reference architecture available on GitHub^®: MATLAB Parallel Server on Amazon Web Services.

   Tip

   Click Start to try a default cluster. Cloud Center prompts you if you need to create a new SSH key. You might want to configure other cluster settings, such as cluster size, machine types and storage settings. For example, for deep learning, choose a machine type with GPUs such as the P3, G4dn, or G5 instances.

   Configuration Setting	Description
   Name	Specify a name.
   Release	Select the MATLAB release with the same version as your local desktop client MATLAB.
   Provider	Select Amazon® AWS®.
   Account	Select an AWS account in which to launch your cluster.
   Operating System	Select the operating system of the instances on your cluster.
   Cloud Location	Select the Region to run your cluster. Consider your location and connectivity.
   Cluster Profile Name	Name of the cluster used when you discover clusters from MATLAB. This attribute is read-only.
   Shared State	Select Personal Cluster or Shareable Cluster.
   	Personal Cluster (default) - A personal cluster is one that is accessible only by you. Auto-Manage Cluster Access - This attribute is checked by default. Clear this option if you prefer to manage cluster access to and from MATLAB manually. MATLAB Drive Access - Your MATLAB Drive files are available to the cluster’s workers in the /MATLAB Drive file system path. The files available on MATLAB Drive are read-only. You can access your MATLAB Drive files by following steps at Access Files in Your MATLAB Drive (MATLAB).
   	Shareable Cluster - Shareable clusters can be shared with other people. You can share the cluster with anyone by sharing the cluster profile. Anyone who has the cluster profile can import the cluster profile into MATLAB, allowing them to submit jobs to or interact with the cluster from MATLAB. They can also reshare the cluster profile. Note Although you can share the cluster with anyone, they can only access it if their IP address is allowed to access the cluster. See the Auto-Manage Cluster Access option for details on how to provide them access to the cluster. Auto-Manage Cluster Access — This attribute is cleared when you specify a Shareable Cluster. Check this option if you prefer to have Cloud Center manage the access to and from MATLAB.
   Auto-Manage Cluster Access	Option indicating whether to automatically update AWS Virtual Machine (VM) inbound firewall rules to permit the end user's current public IP address. Enable this option if you want to access your cluster using MATLAB Online™. You can also manage active inbound firewall rules on the details page for this cluster. For more details, see Manage Cluster Access.
   	Selected - Allows Cloud Center to manage a cluster’s inbound firewall rules.
   	Cleared - Manually manage a cluster's inbound firewall rules. By default, only the IP address range specified in Default IP Address can access the cluster.
   Headnode Machine Type	AWS Virtual Machine (VM) type to use for the headnode, which runs MATLAB Job Scheduler (MJS). No workers are started on this node, so this can be a smaller VM type than the worker nodes. See https://aws.amazon.com/ec2/instance-types/ for a list of instance types. Recommended default c6id.2xlarge.
   Worker Machine Type	AWS Virtual Machine (VM) to use for the workers. See https://aws.amazon.com/ec2/instance-types/ for a list of VM types. Recommended default is r7a.8xlarge. Tip For deep learning, choose a machine type with GPUs such as the P3, G4dn, or G5 instances. P3s have GPUs with high performance for general computation. G4dn and G5 instances have GPUs with high single-precision performance for deep learning, image processing, computer vision, and automated driving simulations.
   Allow cluster to auto-resize	Enabled - The number of machines in your cluster can shrink or grow depending on the amount of work submitted to the cluster. Set Workers in Cluster to the maximum number of workers you want in the cluster. For more information, see Resize Clusters Automatically.
   	Disabled (default) - The number of machines in your cluster remains fixed at the number of workers set by Workers in Cluster.
   Workers in Cluster	Choose the number of workers, using the Upper Limit menu. If you select a number greater than the Workers per Machine, the Machines in Cluster value updates to show more than one machine. Cloud Center supports a maximum of 1024 workers per cluster. The Initial Count field shows the number of workers your cluster starts with. If Allow cluster to auto-resize is disabled, the Initial Count field matches your Upper Limit selection. If Allow cluster to auto-resize is enabled, the Upper Limit menu sets the maximum number of workers for your cluster, in increments of Workers per Machine. The Initial Count field is zero. Your cluster starts with zero workers and can resize up to the maximum number of workers. For more information, see Resize Clusters Automatically.
   Workers per Machine	Number of MATLAB workers to start on each Virtual Machine (VM). Specify 1 worker per physical CPU core, or 1 per GPU for GPU computing. For example, a r7a.8xlarge instance has 32 physical cores, so can support 32 MATLAB workers. See https://aws.amazon.com/ec2/instance-types/ for details on each VM type.
   Auto-Shutdown (hrs)	Select a timeout for the cluster so that it shuts down automatically. Choose Always On, On Idle (based on worker activity), or the number of hours in which to automatically stop your cluster machines. Choosing On Idle stops the cluster machines if cluster is idle for about 10 minutes (30 minutes at startup). After the timeout period, the cluster machines stop but the data volumes are still running and incurring costs. To avoid costs, stop these resources and save a data backup of the cluster in your cloud account by clicking Stop All Resources.
   Log Level	Change the cluster log level. The log level controls the amount of detail in the generated logs, ranging from 0-Off (no logging aside from system essential messages) to 4-Medium-High. If you need to diagnose cluster issues with support engineers, increase the log level for more detail. Log levels above 3-Medium can negatively impact performance.
   VPC	ID of an existing Virtual Private Cloud (VPC) in which to deploy this resource.
   Subnet	ID of an existing public Subnet in which to deploy this resource. Select a VPC and Subnet that meet the requirements for Connecting a Desktop Computer (Client Machine) to MATLAB Parallel Server Running on AWS. For more information, see Configure AWS VPC for Cloud Center.
   Enable Shared Storage	Select to enable shared storage. This storage is shared among all worker nodes and persists between cluster runs. Mounted at /shared/persisted. If you enable shared storage, the MATLAB Job Scheduler database is included in the shared storage. Otherwise, the MATLAB Job Scheduler database volume is set to 100 GB.
   Shared Storage Size (GB)	Size in GB of the persisted storage to mount across the cluster. The storage has a default and minimum capacity of 32 GB and can expand to 100 TB. If set to 0 or less than 100, no persisted storage is created.
   Name of SSH Key	If you do not have a key, Cloud Center prompts you to create one. AWS requires an SSH key in that region to start EC2 instances. Click Create new, in the dialog box, enter a name, and click Download Key. Your browser might require you to identify a location. You get a root access key file with the extension .pem. Store this file in a safe place, because you cannot download it again. However, you can always create a new key, and download its key file. You can specify the same SSH key for multiple clusters. If you want to log in as ubuntu (root) to your cloud cluster machines, you need the machine SSH key. Cluster machines have no password, so you use a key to log in using SSH. If you have existing keys, select from the keys for the specified region of your AWS account, or create a new key. Otherwise, Cloud Center uses the previously selected key or the first key listed alphabetically in the AWS account.
   Default IP Address	Initial IP address range that can connect to this cluster from MATLAB and via SSH. You can add a list of IP address ranges separated by commas in this field. The IP address must be formatted in Classless Inter-Domain Routing (CIDR) notation as <ip_address>/<mask>. E.g. 10.0.0.1/32. You can find your public IP address by searching for 'what is my ip address' on the web. The mask determines the number of IP addresses to include. A mask of 32 is a single IP address. You can use this calculator to build a specific range: https://www.ipaddressguide.com/cidr. You might need to contact your IT administrator to determine which address is appropriate.

6. Click Start to create and start your cluster machines. The cluster starts a number of machines (instances) determined by your choices of number of workers and workers per machine. Cloud Center displays the cluster status Starting.

   It can take up to several minutes for a cluster to completely start up. The status indicates the stages of the process. To get status on any individual cluster machine, under Cluster Details, click Headnode or Worker expanders.

   When the cluster is started and ready for use, Cloud Center displays the cluster status as Running.

Tip

For next steps using your new cluster, discover the running cluster from MATLAB. See Discover Clusters.

This figure shows an example of Create Cluster settings for a personal cluster.

If the cluster fails to start completely, see the warning icon for more details. Often you can shut down your failed cluster and attempt to start it again.

Discover Clusters on Local Machine

To access a running cluster created in your account, use Discover Clusters from MATLAB. For more details, see Discover Clusters.

If you have a shared cluster, you can download and share the cluster profile with others. When your cloud cluster is starting or online, download the cluster profile using the Download cluster profile option under the Actions column or under the three dots beside it. Share the cluster profile with others to allow them to access the cluster from MATLAB and the Cluster Profile Manager. For more details, see Import Cluster Profiles and Access Cloud Clusters.