-

This site is deprecated and will be decommissioned shortly. For current information regarding HPC visit our new site: hpc.njit.edu

Difference between pages "UserAccess" and "UserEnvironment"

From NJIT-ARCS HPC Wiki
(Difference between pages)
Jump to: navigation, search
(Importing text file)
 
(Importing text file)
 
Line 1: Line 1:
−
This page provides information on methods of connecting to NJIT's HPC hosts from Linux, MacOS, and Windows platforms.
 
  
−
==Getting a Login==
 
−
Faculty can obtain a login to NJIT's HPC & BD systems by sending an email to ARCS@njit.edu.  Students can obtain a login either by taking a class that
 
−
uses one of the sytems or by asking their faculty adviser email ARCS@njit.edu on their behalf.  Your login and password is the same as for any NJIT AFS
 
−
system.
 
  
−
==Secure Shell (SSH)==
+
== Shell ==
 +
The default shell for all users is bash. [http://www.gnu.org/software/bash/ GNU bash].
  
−
You must be on the NJIT network (this includes VPN) to ssh directly into the headnode of an HPC cluster.
+
== Home Directories ==
  
−
If you are outside of the NJIT network, you cam ssh into afsconnect1.njit.edu or afsconnect2.njit.edu,
+
<table width="40%" style="border:1px solid black; border-collapse:collapse;">
−
and from there ssh into the headnode of an HPC cluster.
+
<tr>
 +
  <th style="border:1px solid black;">HPC Host</th>
 +
  <th style="border:1px solid black;">Home directory</th>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">Cnrdp.arcs </td>
 +
<td style="border:1px solid black;"> AFS </td>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">Kong </td>
 +
<td style="border:1px solid black;"> local </td>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">Gorgon </td>
 +
<td style="border:1px solid black;"> AFS </td>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">Phi </td>
 +
<td style="border:1px solid black;"> AFS </td>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">Stheno </td>
 +
<td style="border:1px solid black;"> local </td>
 +
</tr>
 +
</table>
 +
== Modules ==
 +
The Environment Modules package provides for the dynamic modification of a user's environment via
 +
modulefiles.
  
−
===Linux and MacOS===
+
Each modulefile contains the information needed to configure the shell for an application.
 +
Once the Modules package is initialized, the environment can be modified on a per-module
 +
basis using the module command which interprets modulefiles. Typically modulefiles
 +
instruct the module command to alter or set shell environment variables such as PATH, MANPATH, etc.
 +
modulefiles may be shared by many users on a system and users may have their own collection to
 +
supplement or replace the shared modulefiles.
 +
[http://modules.sourceforge.net/ Modules]
  
−
====Linux====
+
<table width="90%" style="border:1px solid black; border-collapse:collapse;">
 +
<tr>
 +
  <th style="border:1px solid black;">Command</th>
 +
  <th style="border:1px solid black;">Description</th>
 +
  <th style="border:1px solid black;">Example</th>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">module available (av) </td>
 +
<td style="border:1px solid black;"> lists available modules </td>
 +
<td style="border:1px solid black;"> module av </td>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">module available <em>string</em> </td>
 +
<td style="border:1px solid black;"> lists available modules matching <em>string</em> </td>
 +
<td style="border:1px solid black;"> module av gcc </td>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">module load <em>modulefile</em> </td>
 +
<td style="border:1px solid black;"> loads <em>modulefile</em> </td>
 +
<td style="border:1px solid black;"> module load gcc </td>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">module list </td>
 +
<td style="border:1px solid black;"> lists currently loaded modules </td>
 +
<td style="border:1px solid black;"> module list </td>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">module unload <em>modulefile</em> </td>
 +
<td style="border:1px solid black;"> unloads <em>modulefile</em> </td>
 +
<td style="border:1px solid black;"> module unload gcc </td>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">module switch <em>modulefile</em> </td>
 +
<td style="border:1px solid black;"> loads different version of currently loaded module file </td>
 +
<td style="border:1px solid black;"> module load gcc5/5.2.0<br />module switch gcc5/5.3.0 </td>
 +
</tr>
 +
<tr>
 +
<td style="border:1px solid black;">module purge </td>
 +
<td style="border:1px solid black;"> unloads all loaded modules. Restores environment to state<br />before any modules were loaded </td>
 +
<td style="border:1px solid black;"> module purge </td>
 +
</tr>
 +
</table>
  
−
There are different methods to open a terminal window on the various Linux distributions.  Search the web for instructions specific to the distribution
+
<h5>Documentation</h5>
−
you are using.  Once a terminal window is opened, enter :
+
<code>"man module"</code>, or view the manpage [http://modules.sourceforge.net/man/module.html online].
  
−
<pre code> localhost> ssh -X -Y ucid@HPC_HOST.njit.edu </pre>  
+
<strong>Note</strong><br />
 +
All of the applications listed on the [https://wiki.hpc.arcs.njit.edu/web/applications.php Applications]
 +
page have an associated module file. This module file must be loaded to set the user environment for
 +
the application. If you frequently use a particular application it is convenient
 +
to load the module every login. This can easily be done by adding the module command in your
 +
~/.bashrc file - i.e., "module load <em>application</em>".
 +
<h5>Modules in submit scripts</h5>
  
−
You will be prompted for your password. Enter your <b>AFS</b> password.  You can omit the -X -Y if you are not using a graphic interface.  Substitute "Kong" or "Stheno" for "HPC_HOST".
+
If a module is needed to run a command on a compute node, include the following just
−
 
+
before the command :
−
====MacOS====
+
−
 
+
−
Open a Terminal window. Applications -> Utilities -> Terminal
+
−
 
+
−
At the terminal prompt enter :
+
−
 
+
−
<pre code> localhost> ssh -X -Y ucid@HPC_HOST.njit.edu </pre>
+
−
 
+
−
You will be prompted for your password. Enter your <b>AFS</b> password.  You can omit the -X -Y if you are not using a graphic interface. Substitute "Kong" or "Stheno" for "HPC_HOST".
+
−
 
+
−
===Windows===
+
−
 
+
−
A free Secure Shell client can be obtained from the [http://mobaxterm.mobatek.net MobaXterm Site]. In addition to SSH, MobaXterm provides an X windows
+
−
server, VNC client and other useful utilities. Download, install, and open MobaXterm.
+
−
 
+
−
[[File: MobaXterm.open.png|100px]]
+
−
 
+
−
At the terminal prompt enter :
+
−
 
+
−
<pre code> localhost> ssh -X -Y ucid@HPC_HOST.njit.edu </pre>
+
−
 
+
−
[[File:MobaXterm.ssh.initial.png|100px]]
+
−
 
+
−
You will be prompted your your password. Enter your '''AFS''' password.  You can omit the -X -Y if not using a graphical interface on Kong. Substitute "Kong" or "Stheno" for "HPC_HOST".
+
−
 
+
−
==Virtual Network Computing (VNC)==
+
−
 
+
−
VNC software creates an image of a remote host's desktop on the user's local workstation or laptop, permitting access to the remote host with a graphical
+
−
user interface (GUI).
+
−
 
+
−
Using VNC, it is possible to use a GUI environment from a remote host as if you were at the console of that host. In addition, it is also possible to
+
−
start long-running jobs, and then disconnect without your jobs terminating.
+
−
 
+
−
===Establish a VNC Server Session on an HPC host===
+
−
 
+
−
*Using the SSH client on your local workstation or laptop, log onto an HPC host.
+
−
*Run ''krenew'' with the arguments below to maintain your Kerberos and AFS credentials
+
−
*Establish the VNC environment and set a VNC session password.
+
−
**Enter ''vncserver''.  You will be prompted for a password, and then a session will be started.
+
−
 
+
−
<pre class=code>
+
−
hpc-host-NM ucid>: krenew -t -b -K 60
+
−
hpc-host-NM ucid>: vncserver
+
−
You will require a password to access your desktops.
+
−
Password:
+
−
Verify:
+
−
 
+
−
New 'hpc-host:8 (username)' desktop is hpc-host:8
+
−
 
+
−
Creating default startup script /home/ucid/.vnc/xstartup
+
−
Starting applications specified in /home/ucid/.vnc/xstartup
+
−
Log file is /home/ucid/.vnc/hpc-host:8.log
+
−
</pre>
+
−
 
+
−
*Note and record your assigned VNC Server desktop number.
+
−
*This VNC session will continue to run unless the process is terminated either by the user terminating the process, or by a host reboot or other abnormal
+
−
operation. '''Do not start more than one VNC Server session'''.
+
−
*Note that if the system is rebooted or your VNC session is otherwise terminated you will need to establish a new session. VNC sessions are stateless,
+
−
which means that desktop numbers are not preserved across logins. You will get a new new desktop number at the next login. 
+
−
 
+
−
*Unless you alter the files created by the VNC server, you will not be asked to set a password again.
+
−
 
+
−
=== Connecting to an established VNC Server Session on an HPC host ===
+
−
 
+
−
Independent of the operating system on your workstation or laptop, the best way to connect to an established VNC Server
+
−
session is through an SSH "tunnel". Connecting to an established VNC server session will require both an SSH client and a VNC client.
+
−
 
+
−
====Linux and MacOS====
+
−
 
+
−
Using the SSH client on the local computer create an SSH tunnel between the local computer and the hpc-host. The
+
−
command is
+
−
 
+
−
ssh -L <<i>local-port</i>>:localhost:<<i>remote-port</i>>  <i>ucid</i>@hpc-host.njit.edu
+
−
 
+
−
The local port should be set to 5900 and the remote port is 5900 + (the desktop number assigned when the VNC session was established.)
+
  
 
<pre code>
 
<pre code>
−
localhost> ssh -L 5900:localhost:5901 ucid@hpc-host.
+
. /opt/Modules/init/bash
−
</pre> 
+
module load <modulefile>
−
 
+
−
=====Linux=====
+
−
 
+
−
After the SSH tunnel is created, open a terminal and enter ''vncviewer localhost''. A window will appear and you will be prompted for a password. Enter
+
−
the password created when the VNC session was established.
+
−
 
+
−
<pre code>localhost> vncviewer localhost
+
 
</pre>
 
</pre>
−
 
−
[[File:VNC_Authentication_VncAuth_.png]]
 
−
 
−
You are now connected to your VNC session.
 
−
 
−
=====MacOS=====
 
−
 
−
There are several VNC clients available for Mac. This example will use
 
−
[http://sourceforge.net/apps/mediawiki/tigervnc/index.php?title=Welcome_to_TigerVNC TigerVNC].
 
−
[http://sourceforge.net/projects/tigervnc/files/ Download] and install TigerVNC according to the instructions.
 
−
 
−
After the SSH tunnel is created, open TigerVNC.
 
−
 
−
Application -> TigerVNC Viewer 1.3.0
 
−
 
−
Enter "localhost" for the VNC viewer.
 
−
 
−
[[File:VNC Viewer Connection Details.png]]
 
−
 
−
Click "Connect". A new window will appear prompting for a password. Enter the password created when the VNC
 
−
session was established. Click 'OK.'
 
−
 
−
[[File:VNC_authentication.png]]
 
−
 
−
You are now connected to your VNC session.
 
−
 
−
====Windows====
 
−
 
−
[http://mobaxterm.mobatek.net MobXterm] can be used as both an SSH and VNC client.
 
−
 
−
Select 'New Session.'
 
−
[[File:MobaXterm.clk.nw.sess.png|100px]]
 
−
 
−
Choose "VNC" and click "OK"
 
−
 
−
[[File:MobaXterm.choose.vnc.png|100px]]
 
−
 
−
Enter the hostname that the VNC server is established on. The Port number is : 5900 + (the desktop number obtained when the VNC session was established).
 
−
Select "Connect through SSH gateway (jump host)". Enter the hostname that the VNC server is established on for the "Gateway SSH server", 22 for the
 
−
"Port", and your UCID for "User". Click "OK".
 
−
 
−
[[File:MobaXterm.vnc.sess.setup.1.png|100px]]
 
−
 
−
The first password prompt will be for the SSH tunnel. Enter your AFS password.
 
−
 
−
[[File:MobaXterm.vnc.sess.pwd.1.png|100px]]
 
−
 
−
The second password will be for "vncuser on localhost". Enter the password created when the VNC session was established.
 
−
 
−
[[File:MobaXterm.vnc.sess.pwd.2.png|100px]]
 
−
 
−
You are now connected to you VNC session.
 
−
 
−
[[File:MobaXterm.vnc.session.png|100px]]
 
−
 
−
==Ending a VNC Server session==
 
−
 
−
*Using ssh from your local workstation or laptop, log on to the HPC host where the vncserver is running.
 
−
 
−
*Enter ''vncserver -kill <desktop number>'' Where ''<desktop number>'' is the desktop number assigned when the VNC Server Session was established.
 
−
 
−
<pre class=Code>
 
−
hpc-host-NM ucid>: vncserver -kill :8
 
−
Killing Xvnc process ID 31460
 
−
</pre>
 
−
 
−
[[Category:HPC and BD Wiki Users]]
 

Latest revision as of 16:36, 5 October 2020


Shell

The default shell for all users is bash. GNU bash.

Home Directories

HPC Host Home directory
Cnrdp.arcs AFS
Kong local
Gorgon AFS
Phi AFS
Stheno local

Modules

The Environment Modules package provides for the dynamic modification of a user's environment via modulefiles.

Each modulefile contains the information needed to configure the shell for an application. Once the Modules package is initialized, the environment can be modified on a per-module basis using the module command which interprets modulefiles. Typically modulefiles instruct the module command to alter or set shell environment variables such as PATH, MANPATH, etc. modulefiles may be shared by many users on a system and users may have their own collection to supplement or replace the shared modulefiles. Modules

Command Description Example
module available (av) lists available modules module av
module available string lists available modules matching string module av gcc
module load modulefile loads modulefile module load gcc
module list lists currently loaded modules module list
module unload modulefile unloads modulefile module unload gcc
module switch modulefile loads different version of currently loaded module file module load gcc5/5.2.0
module switch gcc5/5.3.0
module purge unloads all loaded modules. Restores environment to state
before any modules were loaded 		module purge
Documentation

"man module", or view the manpage online.

Note
All of the applications listed on the Applications page have an associated module file. This module file must be loaded to set the user environment for the application. If you frequently use a particular application it is convenient to load the module every login. This can easily be done by adding the module command in your ~/.bashrc file - i.e., "module load application".

Modules in submit scripts

If a module is needed to run a command on a compute node, include the following just before the command : 

. /opt/Modules/init/bash
module load <modulefile>
Retrieved from "https://wiki.hpc.arcs.njit.edu/index.php?title=UserAccess&oldid=201"