To resolve this issue, you can use the resource management functionality of the scheduler. Generally speaking, you should be able to define MATLAB Parallel Server license seats as a consumable resource of the scheduler so that it can be aware of the total seat count and queue jobs accordingly. All MATLAB jobs need to consume this resource.
The example listed below is specific to LSF.
Inside LSF, you can define a dynamic resource to handle the available license number count for MATLAB Parallel Server. For this example, we will abbreviate the resource name to "mdcs". Here are the steps to create this resource inside LSF:
1. In lsf.share, define the resource:
Begin Resource\nRESOURCENAME TYPE INTERVAL INCREASING DESCRIPTION # Keywords\nmdcs Numeric 30 N (MATLAB Parallel Server License)\nEnd Resource
2. In lsf.cluster.$name (where $name is the name of the cluster), define the resource location:
Begin ResourceMap \nRESOURCENAME LOCATION \nmdcs [all] \nEnd ResourceMap
Defining the LOCATION value as "all" indicates that the mdcs resource is a global resource instead of a host-based resource.
3. LSF needs to continually query the number of available MATLAB Parallel Server license seats as a resource. This can be done using an elim executable:
#!/bin/sh \ntotal_licenses=`$MATLAB_ROOT/etc/$ARCH/ParallelServerLicenseCheck -c PORT_NUMBER@HOST | /bin/grep issued | /bin/cut -f2 -d:` \nwhile true \ndo \nused_licenses=`$MATLAB_ROOT/etc/$ARCH/ParallelServerLicenseCheck -c PORT_NUMBER@HOST | /bin/grep use | /bin/cut -f2 -d:` \navailable_licenses=$((total_licenses - used_licenses)) \n/bin/echo "1 mdcs $available_licenses" \n/bin/sleep 30 \ndone
This script provides the number of available licenses for MATLAB Parallel Server.
4. Restart the necessary LSF services to apply your changes:
lsadmin reconfig \nbadmin mbdrestart
5. You can check the status of available seats using the following command:
lsload -l \nHOST_NAME status r15s r1m r15m ut pg io ls it tmp swp mem mdcs \ncompute-0-1 ok 0.0 0.0 0.0 0% 2.4 42 1 7328 7440M 3968M 274M 8 \ncompute-0-0 ok 0.0 0.0 0.0 0% 2.9 53 0 11624 7448M 4000M 499M 8 \nocs41 ok 0.4 0.0 0.0 1% 19.2 327 1 1 5779M 4000M 485M 8
This will give the available seat count for MATLAB Parallel Server jobs under the "mdcs" column.
6. Create a queue for MATLAB with the name of "matlab". In lsb.queue define:
Begin Queue \nQUEUE_NAME = matlab \nRES_REQ=rusage[matlab-parallel-server=1:duration=1] \n... \nEnd Queue
Inside the MATLAB configuration for this cluster, specify "-q matlab" in your submission command.
Once fully configured, any jobs that do not have enough available MATLAB Parallel Server license seats upon submission will stay in a "pending" state until enough seats become available.
NOTE: The above example was setup to work in a MathWorks environment. For any further support please contact your scheduler vendor and/or MathWorks Support.
