HPCandBDSurvey2017

HPC and BD Survey Structure

The survey covers several topics. The survey taker can address any combination of these topics.

Topics:

1. HPC hardware resources
2. HPC storage
3. BD resources
• BD layer 1: Repository (Figure 1 BDC white paper)
• BD layer 2: Technological Infrastructure

[Note: layer 3, BD Applications, is not addressed in this survey, since this layer represents the outcome of using the resources in layers 1 and 2]

4. Software environment
• HPC
• BD
5. Internet bandwidth; Science DMZ
6. Consultation with Academic and Research Computing Systems (ARCS)

Terminology used in the survey

• Terminology applying to multiple sections, presented at those sections:
• On-premise shared The resource is located at NJIT, is provided by NJIT, and is shared amongst its users
• On-premise dedicated The resource is located at NJIT, and is dedicated to the purchaser of the resource
• Off-premise shared The resource is not located at NJIT, is provided by NJIT, and is shared amongst its users
• Off-premise dedicated The resource is not located at NJIT, and is dedicated to the purchaser of the resource
• External The resource is publically-available (e.g., at a national supercomputing center); successful proposal by researcher is required
• Other, section-specific, terminology is presented if the user reaches that section

1. HPC computational resources

1.1 CPU, GPU, max RAM per node, max number of cores that can be simultaneously used, scratch space, node interconnect network

• Rating of adequacy
• Adequate
• Moderate increase needed
• Large increase needed
• Don't know
• If inadequate, how to address
• Increase shared on-premise - resource use restrictions apply
• Increase dedicated on-premise - researcher purchases resources
• Increase shared off-premise - resource use restrictions apply
• Increase dedicated off-premise - researcher purchases resources
• Use external grid computing, e.g., Xsede, Open Science Grid

1.2 HPC Documentation

• Rating of adequacy

1.3 Comments

2. HPC storage

2.1 Adequacy of base allocations

• Adequate
• Moderate increase needed
• Large increase needed
• Don't know

2.1.2 If inadequate, how to address:

• Increase on-premise shared - resource use restrictions apply
• Increase on-premise dedicated - researcher purchases resources
• Increase off-premise shared - resource use restrictions apply
• Increase off-premise dedicated - researcher purchases resources

2.2 Importance of platform-independent access to HPC storage

• Very important
• Moderately important
• Not important
• Don't know

2.3 Adequacy of HPC storage documentation at the HPC and BD wiki

• Adequate
• Somewhat more needed
• Much more needed
• Don't know

2.4 Comments

3. BD layer 1: Repository

3.1 Assessment of adequacy of BD resources

3.1.1 Resource: Shared storage for: raw data; metadata, markup data; analysis results; models, views, tables; forms, animations; workflow templates, provenance data; see figure

• Rating of adequacy
• If inadequate, how to address
• increase on-premise shared
• increase on-premise dedicated - researcher purchase
• increase off-premise shared
• increase off-premise dedicated - researcher purchase

3.2 Comments

4. BD layer 2: Technological Infrastructure

4.1 Assessment of adequacy of BD resources

4.1.1 Resource: Hadoop/Spark nodes, max RAM per node, HDFS storage

• Rating of adequacy
• If inadequate, how to address
• increase on-premise shared
• increase on-premise dedicated - researcher purchase
• increase off-premise shared
• increase off-premise dedicated - researcher purchase

4.2 Comments

5. Software environment

5.1 Assessment of adequacy of software environment

• Excellent
• Good
• Fair
• Poor
• Don't know

5.2 Identify software not currently available; is there an associated cost; expected use

Expected use:

• Research - low, medium, high
• Teaching - low, mediumm, high

5.3 Rating of adequacy of software documentation

5.4 Comments

6. Internet bandwidth, Science DMZ

6.1 Suitability of Internet bandwidth, including Internet 2 if applicable, for your work

• Excellent
• Good
• Fair
• Poor
• Don't know

6.2 Rating of the desirability of implementing a Science DMZ at NJIT as it relates to your work

• High
• Moderate
• Low
• Don't know

6.3 Comments

7. Consultation

7.1 Rating of the effectiveness of consultation in your work

• Excellent
• Good
• Fair
• Poor
• Don't know

7.2 Comments