ModulesOnLochness

From NJIT-ARCS HPC Wiki
Revision as of 16:34, 5 October 2020 by Hpcwiki1 dept.admin (Talk | contribs) (Importing text file)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Lmod

Lmod is used to to manage modules on Lochness and Stheno. The software stack on Lochness and Stheno is new. Submit scripts will need to be adjusted to account for the new lmod system.

For more information on Lmod check the user manual.

module available

Lmod presents a hierarchal view of the available modules. This means that the view of available modules changes depending on the module loaded by the user.



$ module available

----------------------------------------------------- /opt/site/modulefiles ------------------------------------------------------
   cuda/10.0.130        julia/1.0.4     matlab/2019a           polyflow/20.1.0 (D)    python/3.8 (D)
   cuda/10.1.243        matlab/2018a    matlab/2020a    (D)    python/3.6             raspa2/2.0
   cuda/10.2.89  (D)    matlab/2018b    polyflow/19.4.0        python/3.7             vim/8.2

-------------------------------------------------- /opt/ohpc/pub/modulefiles ----------------------------------------------------
   EasyBuild/3.9.2       cmake/3.15.4    gnu8/8.3.0              papi/5.7.0    singularity/3.4.1
   autotools             gnu/5.4.0       intel/19.0.4.243        pmix/2.2.2    valgrind/3.15.0
   charliecloud/0.9.7    gnu7/7.3.0      intel/19.1.0.166 (D)    prun/1.3

There are 3 versions of gnu compilers and 2 versions of intel compilers. Load the 'gnu8' compiler

$ module load gnu8

$ module available

------------------------------------------------- /opt/ohpc/pub/moduledeps/gnu8 --------------------------------------------------
   R/3.5.3        impi/2019.4.243        metis/5.1.0       ocr/1.0.1         pdtoolkit/3.25      scotch/6.0.6
   gsl/2.5        impi/2019.6.166 (D)    mpich/3.3.1       openblas/0.3.5    plasma/2.8.0        superlu/5.2.1
   hdf5/1.10.5    likwid/4.3.4           mvapich2/2.3.1    openmpi3/3.1.4    py2-numpy/1.15.3

----------------------------------------------------- /opt/site/modulefiles ------------------------------------------------------
   cuda/10.0.130        julia/1.0.4     matlab/2019a           polyflow/20.1.0 (D)    python/3.8 (D)
   cuda/10.1.243        matlab/2018a    matlab/2020a    (D)    python/3.6             raspa2/2.0
   cuda/10.2.89  (D)    matlab/2018b    polyflow/19.4.0        python/3.7             vim/8.2

--------------------------------------------------- /opt/ohpc/pub/modulefiles ----------------------------------------------------
   EasyBuild/3.9.2       cmake/3.15.4    gnu8/8.3.0       (L)    papi/5.7.0    singularity/3.4.1
   autotools             gnu/5.4.0       intel/19.0.4.243        pmix/2.2.2    valgrind/3.15.0
   charliecloud/0.9.7    gnu7/7.3.0      intel/19.1.0.166 (D)    prun/1.3

Notice that various new modules are now available including various MPI distributions.

module spider

module spider provides information on the software installed and how to load the modules necessary to run the software. For example to find netcdf:

 $ module spider netcdf

------------------------------------------------------------------------------------------------------------------------------
  netcdf:
------------------------------------------------------------------------------------------------------------------------------
    Description:
      C Libraries for the Unidata network Common Data Form

     Versions:
        netcdf/4.5.0
        netcdf/4.6.1
        netcdf/4.6.3
     Other possible modules matches:
        netcdf-cxx  netcdf-fortran  pnetcdf

------------------------------------------------------------------------------------------------------------------------------
  To find other possible module matches execute:

      $ module -r spider '.*netcdf.*'

------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "netcdf" module (including how to load the modules) use the module's full name.
  For example:

     $ module spider netcdf/4.6.3
------------------------------------------------------------------------------------------------------------------------------

Use the full version to get more information.

$ module spider netcdf/4.6.3

------------------------------------------------------------------------------------------------------------------------------
  netcdf: netcdf/4.6.3
------------------------------------------------------------------------------------------------------------------------------
    Description:
      C Libraries for the Unidata network Common Data Form


    You will need to load all module(s) on any one of the lines below before the "netcdf/4.6.3" module is available to load.

      gnu8/8.3.0  impi/2019.4.243
      gnu8/8.3.0  impi/2019.6.166
      gnu8/8.3.0  mpich/3.3.1
      gnu8/8.3.0  mvapich2/2.3.1
      gnu8/8.3.0  openmpi3/3.1.4
      intel/19.0.4.243  impi/2019.4.243
      intel/19.0.4.243  impi/2019.6.166
      intel/19.0.4.243  mpich/3.3.1
      intel/19.0.4.243  mvapich2/2.3.1
      intel/19.1.0.166  impi/2019.4.243
      intel/19.1.0.166  impi/2019.6.166
      intel/19.1.0.166  mpich/3.3.1
      intel/19.1.0.166  mvapich2/2.3.1

    Help:

      This module loads the NetCDF C API built with the gnu8 compiler
      toolchain and the mpich MPI stack.

      Note that this build of NetCDF leverages the HDF I/O library and requires linkage
      against hdf5. Consequently, the phdf5 package is loaded automatically with this module.
      A typical compilation step for C applications requiring NetCDF is as follows:

      $CC -I$NETCDF_INC app.c -L$NETCDF_LIB -lnetcdf -L$HDF5_LIB -lhdf5

      Version 4.6.3

Following the instructions above , to load netcdf/4.6.3 compiled with openmpi3 based on the gnu8 compilers, the command would be:

$ module load gnu8/8.3.0  openmpi3/3.1.4 netcdf/4.6.3

$ module list

Currently Loaded Modules:
  1) gnu8/8.3.0   2) openmpi3/3.1.4   3) phdf5/1.10.5   4) netcdf/4.6.3

Note the phd5/1.10.5 is automatically loaded as a dependency.


It is useful to use regular expressions to search through the hierarchy to find software packages. For example:


$ module -r spider '.*netcdf.*'

------------------------------------------------------------------------------------------------------------------------------
  netcdf:
------------------------------------------------------------------------------------------------------------------------------
    Description:
      C Libraries for the Unidata network Common Data Form

     Versions:
        netcdf/4.5.0
        netcdf/4.6.1
        netcdf/4.6.3

------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "netcdf" module (including how to load the modules) use the module's full name.
  For example:

     $ module spider netcdf/4.6.3
------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------
  netcdf-cxx:
------------------------------------------------------------------------------------------------------------------------------
    Description:
      C++ Libraries for the Unidata network Common Data Form

     Versions:
        netcdf-cxx/4.3.0

------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "netcdf-cxx" module (including how to load the modules) use the module's full name.
  For example:


     $ module spider netcdf-cxx/4.3.0
------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------
  netcdf-fortran:
------------------------------------------------------------------------------------------------------------------------------
    Description:
      Fortran Libraries for the Unidata network Common Data Form

     Versions:
        netcdf-fortran/4.4.4
        netcdf-fortran/4.4.5

------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "netcdf-fortran" module (including how to load the modules) use the module's full name.
  For example:

     $ module spider netcdf-fortran/4.4.5
------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------
  pnetcdf:
------------------------------------------------------------------------------------------------------------------------------
    Description:
      A Parallel NetCDF library (PnetCDF)

     Versions:
        pnetcdf/1.8.1
        pnetcdf/1.9.0
        pnetcdf/1.12.0

------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "pnetcdf" module (including how to load the modules) use the module's full name.
  For example:


     $ module spider pnetcdf/1.12.0
------------------------------------------------------------------------------------------------------------------------------