-

This site is deprecated and will be decommissioned shortly. For current information regarding HPC visit our new site: hpc.njit.edu

Difference between pages "HPCLocMtx" and "HPCOffPremiseCosts"

From NJIT-ARCS HPC Wiki
(Difference between pages)
Jump to: navigation, search
(Importing text file)
 
(Importing text file)
 
Line 1: Line 1:
−
== HPC Location Matrix ==
+
<div class="noautonum">__TOC__</div>
  
−
This matrix lists the pros and cons of types of HPC hardware ownership and hardware locations.
+
== HPC Off-Premise Costs Document ==
  
 +
=== Summary ===
  
−
<table width="100%" style="border:1px solid black; border-collapse:collapse;">
+
Cost estimates for hosting the NJIT HPC infrastructure off-premise (in the cloud)
 +
for a three-year period are presented for Penguin Computing, and are in preparation for Amazon
 +
Web Services. Cost estimates for Azure are forthcoming.
  
−
<caption>HPC Location Matrix</caption>
+
=== Purpose of the report ===
−
<tr bgcolor="#dddddd">
+
 
−
  <th style="border:1px solid black;">Item</th>
+
This report is the start of a costs-benefits analysis of hosting some portion - as yet to be
−
  <th style="border:1px solid black;">Category</th>
+
determined, and likely to be fluid - of NJIT's HPC infrastructure off-premise.
−
  <th style="border:1px solid black;">Subcategory A</th>
+
 
−
  <th style="border:1px solid black;">Subcategory B</th>
+
=== Report ===
−
  <th style="border:1px solid black;">Pros</th>
+
<ul>
−
  <th style="border:1px solid black;">Cons</th>
+
<li> [https://wiki.hpc.arcs.njit.edu/external/off-premise/HPC3year.xlsx Three-year cost estimates, as of 10/31/2017]</li>
−
  <th style="border:1px solid black;">Notes</th>
+
 
−
</tr>
+
<li> [https://wiki.hpc.arcs.njit.edu/external/off-premise/TartanOnSiteCostProjection.xlsx Seven-year HPC expansion cost estimate]</li>
−
<tr>
+
 
−
<td style="border:1px solid black;">1</td>
+
<li> A cost estimate for hosting the HPC cluster infrastructure at Azure for 3 years
−
<td style="border:1px solid black;">Hardware owned<br />by institution </td>
+
            is expected by EOD 11/03/2017</li>
−
<td style="border:1px solid black;"> </td>
+
  </li>
−
<td style="border:1px solid black;"> </td>
+
 
−
<td style="border:1px solid black;"> </td>
+
<li> Cost estimates for other vendors - Google Cloud Platform, IBM Bluemix,
−
<td style="border:1px solid black;"> </td>
+
            Oracle Cloud - may be obtained later, on an as-needed basis.
−
<td style="border:1px solid black;"> </td>
+
  </li>
−
<td style="border:1px solid black;"></td>
+
 
−
</tr>
+
<li> Big data hardware off-premise cost estimates are not included in the current phase; to
−
<tr>
+
    be addressed later.
−
<td style="border:1px solid black;">2</td>
+
</li>
−
<td style="border:1px solid black;"></td>
+
</ul>
−
<td style="border:1px solid black;"> On-campus </td>
+
 
−
<td style="border:1px solid black;"> </td>
+
=== Proposal ===
−
<td style="border:1px solid black;"> </td>
+
Assuming that it is cost effective to move HPC off premise, ARCS recommends using the Penguin Computing On Demand
−
<td style="border:1px solid black;"> </td>
+
(POD) service for the following reasons:
−
<td style="border:1px solid black;"> </td>
+
<ul>
−
<td style="border:1px solid black;"></td>
+
<li>POD is entirely hardware-based</li>
−
</tr>
+
<li>Penguin Computing provides much of all of the software needed by NJIT, including compilers, utilities, and scheduler.
−
<tr>
+
This is not the case for AWS or Azure</li>
−
<td style="border:1px solid black;">3</td>
+
<li>POD includes the Luster parallel file system on the high-speed Omnipath node interconnect</li>
−
<td style="border:1px solid black;"></td>
+
<li>POD is flexible with respect to accommodating user needs, e.g., backups</li>
−
<td style="border:1px solid black;"> </td>
+
</ul>
−
<td style="border:1px solid black;"> </td>
+
 
−
<td style="border:1px solid black;"> Strongly preferred by NJIT researchers using HPC </td>
+
A trial project(s), involving one or more researchers, may be useful in gathering data on the deployment and use of off-premise HPC.
−
<td style="border:1px solid black;"> Hardware will not stay current without funded timely re-fresh </td>
+
−
<td style="border:1px solid black;">  </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">4</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Provides maximum control for researchers and administrators </td>
+
−
<td style="border:1px solid black;"> Expansion is limited by physical space, and power and HVAC requirements </td>
+
−
<td style="border:1px solid black;">  </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">5</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> [[ HPCBaseline ]] provides guaranteed capability for researchers without and with their own funding </td>
+
−
<td style="border:1px solid black;"> Expansion of resources takes months, compared to typically less than 1 hour for cloud </td>
+
−
<td style="border:1px solid black;">  </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">6</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Researchers who own hardware have continued use of that hardware when their funding runs out </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;">  </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">7</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> For HPC use patterns, less expensive than commercial cloud </td>
+
−
<td style="border:1px solid black;">  </td>
+
−
<td style="border:1px solid black;">  </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">8</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Easily integrated into the NJIT AFS software environment </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;">  </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">9</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Can be expanded into the cloud as needed and funded </td>
+
−
<td style="border:1px solid black;">  </td>
+
−
<td style="border:1px solid black;">  </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">10</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Colorado School of Mines (peer institution) [https://docs.google.com/document/d/1QW83O9n26o2mQmhxq5T6pDEzrh2mROLmu0HMPeqnBwE/edit?ts=5c6c2ed2 stance] </td>
+
−
<td style="border:1px solid black;">  </td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">11</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> [https://wiki.hpc.arcs.njit.edu/external/surveys/campchamp/Jan2019.results.pdf Campus Champions On/off-premise HPC] </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> See: 1)Last bullet in "Section 1 responses"<br />2) Next to last bullet in "Section 2 responses"  </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">12</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> Off-campus </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">13</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Co-location </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">14</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Differences from "On-campus": Item 4, Diminished control for administrators; Item 8, Integration may be problematical </td>
+
−
<td style="border:1px solid black;"> Differences from "On-campus" None </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">15</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Commercial </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">16</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Differences from "On-campus": Item 4, Significantly diminished control for administrators; Item 8, Integration likely to be problematical </td>
+
−
<td style="border:1px solid black;"> Differences from "On-campus" Items 3, 4, 5 do not apply </td>
+
−
<td style="border:1px solid black;"> Costs are unlnown </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">17</td>
+
−
<td style="border:1px solid black;">Hardware not owned<br />by institution </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">18</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> Free cloud [1] </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">19</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> "Evergreen" - new processors readily available </td>
+
−
<td style="border:1px solid black;"> Successful proposal needed </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">20</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Flexible configuration </td>
+
−
<td style="border:1px solid black;">  Access ends when proposal period ends </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">21</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Expandable, within propsal limits </td>
+
−
<td style="border:1px solid black;"> Loss of common software environment compared to owned </td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">22</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> Commercial cloud [2] </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">23</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> "Evergreen" - new processors readily available </td>
+
−
<td style="border:1px solid black;"> Generally more expensive for HPC than owned hardware </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">24</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Flexible configuration </td>
+
−
<td style="border:1px solid black;"> Loss of common software environment compared to owned </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
<tr>
+
−
<td style="border:1px solid black;">25</td>
+
−
<td style="border:1px solid black;"></td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"> Rapidly expandable </td>
+
−
<td style="border:1px solid black;"> Access ends when funds run out </td>
+
−
<td style="border:1px solid black;"> </td>
+
−
<td style="border:1px solid black;"></td>
+
−
</tr>
+
−
</table>
+
−
<p></p>
+
−
<strong>Legend</strong><br />
+
−
[1] E.g., XSEDE
+
−
<br />
+
−
[2] E.g., AWS, Azure, GCP, Penguin
+
−
----
+

Latest revision as of 16:33, 5 October 2020

HPC Off-Premise Costs Document

Summary

Cost estimates for hosting the NJIT HPC infrastructure off-premise (in the cloud) for a three-year period are presented for Penguin Computing, and are in preparation for Amazon Web Services. Cost estimates for Azure are forthcoming.

Purpose of the report

This report is the start of a costs-benefits analysis of hosting some portion - as yet to be determined, and likely to be fluid - of NJIT's HPC infrastructure off-premise.

Report

  • Three-year cost estimates, as of 10/31/2017
  • Seven-year HPC expansion cost estimate
  • A cost estimate for hosting the HPC cluster infrastructure at Azure for 3 years is expected by EOD 11/03/2017
  • Cost estimates for other vendors - Google Cloud Platform, IBM Bluemix, Oracle Cloud - may be obtained later, on an as-needed basis.
  • Big data hardware off-premise cost estimates are not included in the current phase; to be addressed later.

Proposal

Assuming that it is cost effective to move HPC off premise, ARCS recommends using the Penguin Computing On Demand (POD) service for the following reasons:

  • POD is entirely hardware-based
  • Penguin Computing provides much of all of the software needed by NJIT, including compilers, utilities, and scheduler. This is not the case for AWS or Azure
  • POD includes the Luster parallel file system on the high-speed Omnipath node interconnect
  • POD is flexible with respect to accommodating user needs, e.g., backups

A trial project(s), involving one or more researchers, may be useful in gathering data on the deployment and use of off-premise HPC.

Retrieved from "https://wiki.hpc.arcs.njit.edu/index.php?title=HPCLocMtx&oldid=73"