-

This site is deprecated and will be decommissioned shortly. For current information regarding HPC visit our new site: hpc.njit.edu

SharingData

From NJIT-ARCS HPC Wiki
Revision as of 16:35, 5 October 2020 by Hpcwiki1 dept.admin (Talk | contribs) (Importing text file)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

There are many instances in which researchers wish to share data with other members of research projects. Methods of sharing data vary, depending on whether the data is stored in :

  • NFS
  • AFS

Data stored in NFS

  • Users on the kong.njit.edu and stheno.njit.edu clusters are assigned home directories, which reside on NFS-mounted disk.
    Currently, kong home directory data is separate from stheno home directory data - kong data cannot be accessed from stheno, and vice versa. However, it is planned to implement a system in which a user's (single) home directory is shared between kong and stheno.
  • Users on kong or stheno can give other users on those clusters access to their home directories via a Linux group. To do this, contact arcs@njit.edu.
  • HPC resources other than kong and stheno do not have NFS-mounted disk.

Data stored in AFS

The following applies to all data stored in AFS, including very large files (up to 2TB).

  • All HPC resources mount AFS (/afs/cad.njit.edu/....); thus users on those resources have access to everything in AFS (depending on the user's AFS permission in a given directory).
  • Users can give other UCIDs permissions in AFS directories that they own - e.g., in /afs/cad/research/<dept>/<userUCID>. More info
  • Outside collaborators
    • Arrangements are made on a case-by-case basis to provide outside collaborators with a guest AFS account, so that they can log in remotely to NJIT AFS systems to access data there.
    • It is possible to set things up so that outside collaborators at institutions that support AFS to mount the NJIT cad.njit.edu AFS cell. This would have the advantage that the cad.njit.edu would appear to be a local directory for the collaborator - i.e., the collaborator would access the cad.njit.edu cell diectly from his/her local AFS client wokstation; there would be no need for the collaborator to log into an NJIT AFS system to access data.

      The reverse is also true : a researcher at NJIT could access AFS-aware foreign cells as if they were local.