-

This site is deprecated and will be decommissioned shortly. For current information regarding HPC visit our new site: hpc.njit.edu

GettingStartedWithSerialAndParallelMATLABOnLochness

From NJIT-ARCS HPC Wiki
Jump to: navigation, search

This page is under construction, Will be completed by 6/30 Following this procedure a user will be able to submit jobs to lochness or stheno from Matlab running locally on the user's computer. The version of Matlab on the user's computer must be the same as on the cluster, currently 2021a.

Installing the Add-On

From the Matlab window, click on "Add-ons" and select "Get Add-Ons."

ClickOnAddons.png

In the search box enter "slurm" and click on the magnifying glass icon.

Select "Parallel Computing Toolbox plugin for MATLAB Parallel Server with Slurm"

Alternatively, this Add-On can be downloaded directly from the Mathworks site.

SlurmAddOn.png

Click on "Install."

ClickOnInstall.png

The installation of the Add-On is complete. Click on "OK" the start the "Generic Profile Wizard for Slurm."

InstallationComplete.png

Creating a Profile for Lochness or Stheno

The following steps will create a profile for lochness (or stheno). Click "Next" to begin.

GenericProfile1.png

In the "Operating System" screen "Unix" is already selected. Click "Next" to continue.

GenericProfile2.png

This "Submission Mode" screen determines whether or not to use a "shared" or "nonshared" submission mode. Since Matlab installed on your personal computer or laptop does not use a shared job location storage, select "No" where indicated and click "Next" to continue.


GenericProfile3.png

Click "Next" to continue.

GenericProfile4.png

In the "Connection Details" screen, enter the cluster host, either "lochness.njit.edu" or "stheno.njit.edu." Enter your UCID for the username.

Select "No" for the "Do you want to use an identity file to log in to the cluster" option and click next to continue.

GenericProfile5.png

In the "Cluster Details" screen enter the full path to the directory on lochness to store the Matlab job files. In the case the directory is $HOME/MDCS. MDCS stands for Matlab Distributed Computing Server. It is not necessary to name this directory MDCS. This directory can be named anything you wish. To determine the value of $HOME, log onto lochness and run the following:

login-1-45 ~ >: echo $HOME
/home/g/guest24

Make sure to check the box "Use unique subfolders."

Click "Next" to continue.

GenericProfile6.png

In the "Workers" screen enter "512" for the number of workers and "/opt/site/apps/matlab/R2021a" for "MATLAB installation folders for workers." Click "Next" to continue.

GenericProfile7 1.png

In the "License" screen make sure to select "Network license manager" and click "Next" to continue.

GenericProfile8.png

In the "Profile Details" screen enter either "Lochness" or "Stheno" depending on which cluster you are making a profile for. The "Cluster description" is optional and may be left blank. Click "Next" to continue.

GenericProfile9.png

In the "Summary" screen make sure everything is correct and click "Create."

GenericProfile10 1.png

In the "Profile Created Successfully" screen, check the "Set the new profile as default" box and click on "Finish."

GenericProfile11.png

Submitting a Serial Job

This section will demonstrate how to create a cluster object and submit a simple job to the cluster. The job will run the 'hostname' command on the node assigned to the job. The output will indicate clearly that the job ran on the cluster and not on the local computer.

The hostname.m file used in this demonstration can be downloaded here.

In the Matlab window enter:

 >> c=parcluster 

C=parcluster 1.png

Certain arguments need to be passe dto SLURM in order for the job to run properly. Her we will set values for partion, mem-per-cpu and time. In the Matlab window enter:

 >> c.AdditionalProperties.AdditionalSubmitArgs=['--partition=public --mem-per-cpu=10G --time=2-00:00:00'] 

To make this persistent between Matlab sessions these arguments need to be saved to the profile. In the Matlab window enter:

 >> c.saveProfile 

AdditionalArguments.png

We will now submit the hostname.m function to the cluster. In the Matlab window enter the following:

>> j=c.batch(@hostname, 1, {}, 'AutoAddClientPath', false); 

@: Submitting a function.
1: The number of output arguments from the evaluated function.
{}: Cell array of input arguments to the function. In this case empty.
AutoAddClientPath', false: The client path is not available on the cluster.

When the job is submitted, you will be prompted for your password.

BatchEnterPasswd.png BatchHostname.png MonitorJobs.png

Submitting a Parallel Function

SubmitParallel.png JobFinished.png FetchOutputs.png

Submitting a Script Requiring a GPU

GpuSubmitArgs.png GpuSubmit.png GpuDiary.png

Load and Plotting Results from A Job

PlotDemoSub.png Load x.png Plot x.png