https://wiki.hpc.arcs.njit.edu/index.php?title=FAQ&feed=atom&action=history
FAQ - Revision history
2024-03-29T11:13:51Z
Revision history for this page on the wiki
MediaWiki 1.26.2
https://wiki.hpc.arcs.njit.edu/index.php?title=FAQ&diff=860&oldid=prev
Hpcwiki1 dept.admin: Importing text file
2023-08-04T11:22:22Z
<p>Importing text file</p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;' lang='en'>
<td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 11:22, 4 August 2023</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l125" >Line 125:</td>
<td colspan="2" class="diff-lineno">Line 125:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div><ol></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div><ol></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>  <li>Use rclone : [[https://rclone.org/ rclone]]</li></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>  <li>Use rclone : [[https://rclone.org/ rclone]]</li></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>  <li>rsync to a local disk - contact <em><del class="diffchange diffchange-inline">arcs_help</del>@njit.edu</em> for assistance</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>  <li>rsync to a local disk - contact <em><ins class="diffchange diffchange-inline">hpc</ins>@njit.edu</em> for assistance</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div></ol></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div></ol></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
</table>
Hpcwiki1 dept.admin
https://wiki.hpc.arcs.njit.edu/index.php?title=FAQ&diff=859&oldid=prev
Hpcwiki1 dept.admin: Importing text file
2023-08-04T11:17:41Z
<p>Importing text file</p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;' lang='en'>
<td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 11:17, 4 August 2023</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l125" >Line 125:</td>
<td colspan="2" class="diff-lineno">Line 125:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div><ol></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div><ol></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>  <li>Use rclone : [[https://rclone.org/ rclone]]</li></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>  <li>Use rclone : [[https://rclone.org/ rclone]]</li></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>  <li>rsync to a local disk - contact <em><del class="diffchange diffchange-inline">arcs</del>@njit.edu</em> for assistance</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>  <li>rsync to a local disk - contact <em><ins class="diffchange diffchange-inline">arcs_help</ins>@njit.edu</em> for assistance</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div></ol></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div></ol></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l144" >Line 144:</td>
<td colspan="2" class="diff-lineno">Line 144:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>1. Getting files from your local computer to your local cluster home directory</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>1. Getting files from your local computer to your local cluster home directory</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div><ol type="A"></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div><ol type="A"></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div> <li>make your local computer an AFS client; contact <em><del class="diffchange diffchange-inline">arcs</del>@njit.edu</em> for help with this</li></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> <li>make your local computer an AFS client; contact <em><ins class="diffchange diffchange-inline">arsc_help</ins>@njit.edu</em> for help with this</li></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div> <li>copy the relevant files from your local computer into AFS space that you have write access to,</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div> <li>copy the relevant files from your local computer into AFS space that you have write access to,</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>   usually a research directory</li></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>   usually a research directory</li></div></td></tr>
<tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l150" >Line 150:</td>
<td colspan="2" class="diff-lineno">Line 150:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div> && aklog" if you don't have a token</li></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div> && aklog" if you don't have a token</li></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div> <li>use <em>tar</em> or <em>cp</em> to get the files in AFS into your local cluster home  </div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div> <li>use <em>tar</em> or <em>cp</em> to get the files in AFS into your local cluster home  </div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>   directory. Contact <em><del class="diffchange diffchange-inline">arcs</del>@njit.edu</em> if you need help</li></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>   directory. Contact <em><ins class="diffchange diffchange-inline">arsc_help</ins>@njit.edu</em> if you need help</li></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div></ol></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div></ol></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>2. Accessing your files in AFS directly from compute nodes</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>2. Accessing your files in AFS directly from compute nodes</div></td></tr>
<tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l241" >Line 241:</td>
<td colspan="2" class="diff-lineno">Line 241:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>===How do I request software that is not currently available ?===</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>===How do I request software that is not currently available ?===</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>This is done by sending mail to <em><del class="diffchange diffchange-inline">arcs</del>@njit.edu</em>. The software must meet these</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>This is done by sending mail to <em><ins class="diffchange diffchange-inline">arsc_help</ins>@njit.edu</em>. The software must meet these</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>criteria :</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>criteria :</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div><ul></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div><ul></div></td></tr>
</table>
Hpcwiki1 dept.admin
https://wiki.hpc.arcs.njit.edu/index.php?title=FAQ&diff=242&oldid=prev
Hpcwiki1 dept.admin: Importing text file
2021-01-05T14:31:34Z
<p>Importing text file</p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;' lang='en'>
<td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 14:31, 5 January 2021</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l254" >Line 254:</td>
<td colspan="2" class="diff-lineno">Line 254:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If the software is not free, a funding source will need to be identified.</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>If the software is not free, a funding source will need to be identified.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>===<del class="diffchange diffchange-inline">The Stheno GCC version 4.1.2. Can a newer version be installed ?</del>===</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>===<ins class="diffchange diffchange-inline">Running Jupyter Notebook</ins>===  </div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div> </div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline"><ul></ins></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">The problem is that the Stheno operating system, Scientific Linux SL release</del></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline"> </ins><<ins class="diffchange diffchange-inline">li</ins>><ins class="diffchange diffchange-inline">copy </ins>/<ins class="diffchange diffchange-inline">opt</ins>/<ins class="diffchange diffchange-inline">site</ins>/<ins class="diffchange diffchange-inline">examples</ins>/<ins class="diffchange diffchange-inline">jupyter_notebook</ins>/<ins class="diffchange diffchange-inline">jupyter_submit</ins>.<ins class="diffchange diffchange-inline">sbatch</ins>.<ins class="diffchange diffchange-inline">sh to your directory<</ins>/<ins class="diffchange diffchange-inline">li></ins></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">5.5, is too old.</del></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline"> <li>Modify jupyter_submit</ins>.<ins class="diffchange diffchange-inline">sbatch</ins>.<ins class="diffchange diffchange-inline">sh as needed - instructions are provided in jupyter_submit</ins>.<ins class="diffchange diffchange-inline">sbatch</ins>.<ins class="diffchange diffchange-inline">sh<</ins>/<ins class="diffchange diffchange-inline">li></ins></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div> </div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline"> </ins><<ins class="diffchange diffchange-inline">li</ins>><ins class="diffchange diffchange-inline">Submit jupyter_submit</ins>.<ins class="diffchange diffchange-inline">sbatch</ins>.<ins class="diffchange diffchange-inline">sh</ins><<ins class="diffchange diffchange-inline">/li</ins>></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">Although several newer versions of gcc are installed in AFS :</del></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></<ins class="diffchange diffchange-inline">ul</ins>></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><<del class="diffchange diffchange-inline">pre code</del>></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div> </div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">  "module avail gcc" gives :</del></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">  </del></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">  ------------------------------- </del>/<del class="diffchange diffchange-inline">afs</del>/<del class="diffchange diffchange-inline">cad.njit.edu</del>/<del class="diffchange diffchange-inline">ucs</del>/<del class="diffchange diffchange-inline">modulefiles</del></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">  gcc</del>/<del class="diffchange diffchange-inline">4</del>.<del class="diffchange diffchange-inline">8</del>.<del class="diffchange diffchange-inline">1 gcc</del>/<del class="diffchange diffchange-inline">4</del>.<del class="diffchange diffchange-inline">8</del>.<del class="diffchange diffchange-inline">2 gcc/4</del>.<del class="diffchange diffchange-inline">9</del>.<del class="diffchange diffchange-inline">2 gcc</del>/<del class="diffchange diffchange-inline">5.2.0 gcc/5.3.0 gcc/5.4.0 gcc/6.1.0</del></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline"> </del><<del class="diffchange diffchange-inline">/pre</del>></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline"> </del></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">none of them will load on Stheno, due to its operating system</del>.</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div> </div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">See [[Roadmap]] for scheduled Stheno operating system upgrade</del>.</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div> </div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">The operating system on Kong.njit.edu, Scientific Linux release 6.2,</del></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">does support the gcc installations in AFS.</del></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div> </div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">Users can get a login on Kong upon request to </del><<del class="diffchange diffchange-inline">em</del>><del class="diffchange diffchange-inline">arcs@njit.edu</del></<del class="diffchange diffchange-inline">em</del>> <del class="diffchange diffchange-inline">from a faculty member.</del></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>==<strong><font color="#9966cc">Submit script behaves strangely</font></strong>==</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>==<strong><font color="#9966cc">Submit script behaves strangely</font></strong>==</div></td></tr>
</table>
Hpcwiki1 dept.admin
https://wiki.hpc.arcs.njit.edu/index.php?title=FAQ&diff=241&oldid=prev
Hpcwiki1 dept.admin: Importing text file
2021-01-04T14:19:49Z
<p>Importing text file</p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;' lang='en'>
<td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 14:19, 4 January 2021</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l120" >Line 120:</td>
<td colspan="2" class="diff-lineno">Line 120:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>  location in AFS to which the user has write access - e.g., a research directory or AFS home directory.</li></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>  location in AFS to which the user has write access - e.g., a research directory or AFS home directory.</li></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div></ul></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div></ul></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">==<strong><font color="#9966cc">Archiving data</font></strong>==</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">===How do I archive data from local storage on a cluster or AFS ?===</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"><ol></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"> <li>Use rclone : [[https://rclone.org/ rclone]]</li></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"> <li>rsync to a local disk - contact <em>arcs@njit.edu</em> for assistance</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ol></ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>==<strong><font color="#9966cc">Getting Job Status</font></strong>==</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>==<strong><font color="#9966cc">Getting Job Status</font></strong>==</div></td></tr>
<tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l147" >Line 147:</td>
<td colspan="2" class="diff-lineno">Line 154:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>2. Accessing your files in AFS directly from compute nodes</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>2. Accessing your files in AFS directly from compute nodes</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div><ul></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div><ul></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div> <li>Use ksub [<del class="diffchange diffchange-inline">https://wiki.hpc.arcs.njit.edu/index.php/</del>UsingKsub <del class="diffchange diffchange-inline">Using ksub</del>]</li></div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> <li>Use ksub [<ins class="diffchange diffchange-inline">[</ins>UsingKsub<ins class="diffchange diffchange-inline">]</ins>]</li></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div></ul></div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div></ul></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
</table>
Hpcwiki1 dept.admin
https://wiki.hpc.arcs.njit.edu/index.php?title=FAQ&diff=56&oldid=prev
Hpcwiki1 dept.admin: Importing text file
2020-10-05T16:32:53Z
<p>Importing text file</p>
<p><b>New page</b></p><div><br />
==<strong><font color="#9966cc">Accessing AFS Space</font></strong>==<br />
===How can I access my AFS space from HPC cluster compute nodes ?===<br />
Jobs submitted by the SGE scheduler to compute nodes have to get the user's<br />
AFS token in order to access space in AFS that that user has access to.<br />
The current method for doing this is via "ksub". See [[UsingKsub]]. <br />
<br />
Just before running <em>ksub</em>, be <strong>sure</strong> you have your Kerberos<br />
ticket and AFS token by :<br />
<pre code>kinit && aklog</pre><br />
<br />
An improved method for doing what ksub does, "auks", is in the process of being implemented.<br />
<br />
==<strong><font color="#9966cc">Compilers</font></strong>==<br />
===What compilers are available ?===<br />
In addition to the compilers listed by "module av", the GNU compilers that are<br />
part of the standard operating system installation are available.<br />
<br />
See [[SoftwareModulesAvailable]]<br />
<br />
==<strong><font color="#9966cc">Compiling CUDA code</font></strong>==<br />
===How do I compile CUDA programs ?===<br />
<ol><br />
<li><br />
Get an interactive login a GPU node, e,g., node151 or node152<br />
<pre code>qlogin node151</pre><br />
</li><br />
<br />
<li><br />
Load gcc and CUDA modules, e.g.,<br />
<pre code>module load gcc/5.4.0 cuda</pre><br />
</li><br />
<br />
<li><br />
Compile your code<br />
</li><br />
<br />
<li><br />
Log out of the GPU node<br />
</li><br />
<br />
<li><br />
Submit your job to the <strong>gpu</strong> queue using a submit script<br />
</li><br />
</ol><br />
<br />
See [[RunningCUDASamplesOnKong]] and [[KongQueuesTable]]<br />
<br />
==<strong><font color="#9966cc">Conserving disk space</font></strong>==<br />
===What are the best ways of conserving local space on the HPC clusters ?===<br />
<ol><br />
<li>Remove unnecessary files</li><br />
<li>If you have a research directory located at /afs/cad/research/.., <br />
move (and better yet compress) files into a sub-directory of that<br />
directory. This not only frees up space on on the cluster, but keeps the <br />
user's results available even after their cluster account is removed.<br />
</li><br />
<li>Compress files with <em>bzip2</em> or <em>gzip</em>. bzip2 is usually more effective</li><br />
</ol><br />
<br />
==<strong><font color="#9966cc">"Error message: vsu_ClientInit: Could not get afs tokens, running unauthenticated" from qsub</font></strong>==<br />
===I get this message when running <em>qsub</em>. Is it something I should be concerned about ?===<br />
This message indicates that you do not have your AFS token. The message can be safely<br />
ignored when running qsub.<br />
<br />
==<strong><font color="#9966cc">Error message from a node</font></strong>==<br />
===I get this message repeatedly in a terminal window when logged into Kong. Is it something I should be concerned about ?===<br />
This message indicates that there is a problem with a certain node. The message can be safely ignored, unless your job was <br />
using that node. The ARCS staff will take the offending node out of service as soon as possible.<br />
<pre code><br />
Message from syslogd@nodeXXX at timestamp<br />
kernel: Code: 48 8d 45 d0 4c 89 4d f8 c7 45 b0 10 00 00 00 48 89 45 c0 e8 38 ff ff ff c9 c3 90 90 90 90 90 90 44 8d 46 3f 85 f6 55 44<br />
0f 49 c6 <31> d2 48 89 e5 41 c1 f8 06 45 85 c0 7e 24 48 83 3f 00 48 89 f8 <br />
</pre><br />
<br />
==<strong><font color="#9966cc">Getting access to HPC clusters</font></strong>==<br />
===Who is eligible ?===<br />
All NJIT researchers; courses using HPC. See [[UserAccess]].<br />
<br />
==<strong><font color="#9966cc">Getting AFS token using qlogin</font></strong>==<br />
===How do I get my AFS token when using <em>qlogin</em> ?===<br />
After qlogin :<br />
<pre code><br />
kinit && aklog && tokens<br />
</pre><br />
<br />
<em>kinit</em> gets your Kerberos ticket when you supply the correct password<br />
<br />
<em>aklog</em> gets your AFS token from your Kerberos ticket<br />
<br />
<em>tokens</em> displays your AFS token status<br />
<br />
==<strong><font color="#9966cc"> Kerberos ticket and AFS token status message</font></strong>==<br />
===<strong><font color="#9966cc">What does the "Your Kerberos ticket and AFS token status" message mean</font></strong> ?===<br />
<br />
A message such as the following is informational, telling the user :<br />
<ul><br />
<li>Until what date thier Kerberos tickect ca be renewed</li><br />
<li>When their AFS token will expire, unless refreshed</li><br />
</ul><br />
<br />
<pre code><br />
=== === === Your Kerberos ticket and AFS token status === === ===<br />
Kerberos : Renew until 02/25/17 07:36:21, Flags: FRI<br />
AFS : User's (AFS ID 22964) tokens for afs@cad.njit.edu [Expires Jan 31 17:36]<br />
</pre><br />
<br />
==<strong><font color="#9966cc">Getting cluster resource usage history</font></strong>==<br />
===How can I view the historical usage of resources on the HPC clusters ?===<br />
Use [[Ganglia]]<br />
<br />
==<strong><font color="#9966cc">Getting files into your local directory on kong or stheno</font></strong>==<br />
===How do I get files from outside kong or stheno into my local directory on those machines ?===<br />
The easiest way to to this is to copy the files from AFS. <br />
<ul><br />
<li>Since kong and stheno are AFS clients, a user logged into kong or stheno can copy files from any location in <br />
AFS that the user has read access into their local kong or stheno directory.</li><br />
<br />
<li>Conversely, a user can copy files from their local kong or stheno directory to any<br />
location in AFS to which the user has write access - e.g., a research directory or AFS home directory.</li><br />
</ul><br />
<br />
==<strong><font color="#9966cc">Getting Job Status</font></strong>==<br />
===How can I get the status of jobs I have submitted to SGE ?===<br />
Use "qtsat" on a head node in various formats. See [[SonOfGridEngine]].<br />
<br />
===How can I see an overall summary of queue activity ?===<br />
Use "qsummary" on a head node. For usage : "qsummary -h". See [[SonOfGridEngine]].<br />
<br />
==<strong><font color="#9966cc">Getting local files onto or asccessible from an HPC cluster</font></strong>==<br />
===How do I get files that are local to my computer onto or accessible from the kong or stheno clusters ?===<br />
Programs running on compute nodes can access files that are :<br />
<ol><br />
<li>in the user's local cluster home directory</li><br />
<li>in an AFS directory the user has access to</li><br />
</ol><br />
1. Getting files from your local computer to your local cluster home directory<br />
<ol type="A"><br />
<li>make your local computer an AFS client; contact <em>arcs@njit.edu</em> for help with this</li><br />
<li>copy the relevant files from your local computer into AFS space that you have write access to,<br />
usually a research directory</li><br />
<li>log in to the cluster headnode; make sure you have your AFS token, via <em>tokens</em>; use "kinit<br />
&& aklog" if you don't have a token</li><br />
<li>use <em>tar</em> or <em>cp</em> to get the files in AFS into your local cluster home <br />
directory. Contact <em>arcs@njit.edu</em> if you need help</li><br />
</ol><br />
2. Accessing your files in AFS directly from compute nodes<br />
<ul><br />
<li>Use ksub [https://wiki.hpc.arcs.njit.edu/index.php/UsingKsub Using ksub]</li><br />
</ul><br />
<br />
==<strong><font color="#9966cc">Languages</font></strong>==<br />
===What languages are available ?===<br />
In addition to the languages listed by "module av", the languages that are<br />
part of the standard operating system installation are available - e.g.,<br />
/usr/bin/perl, /usr/bin/python. See [[SoftwareModulesAvailable]].<br />
<br />
==<strong><font color="#9966cc">Performance, local machine vs. cluster</font></strong>==<br />
===An application runs faster on my own computer than it does on the HPC cluster ...?===<br />
This can happen for a variety of reasons, including :<br />
<ul><br />
<li>your local computer has faster CPU, more RAM than cluster nodes</li><br />
<li>your job on the cluster node shares the nodes's reosurces with other jobs</li><br />
<li>the disk I/O on your local computer is faster than on the cluster</li><br />
</ul><br />
It should be noted that even in such cases users can get significantly better theoughput using<br />
a cluster by :<br />
<ul><br />
<li>running many serial jobs simultaneously</li><br />
<li>running jobs in parallel</li><br />
</ul><br />
<br />
==<strong><font color="#9966cc">Program runs on head node but not on compute nodes</font></strong>==<br />
===My program runs OK when I run it on the head node, but produces different results, or doesn't run at all, on compute nodes. Why ?===<br />
There remain differences - scheduled to be remedied - between the head node and compute nodes. Code<br />
compiled on the head node may not run the same way on compute nodes. Fix : use <em>qlogin</em> to log in<br />
to an arbitrary compute node, and compile your code on that node.<br />
<br />
==<strong><font color="#9966cc">Resources for big data analysis</font></strong>==<br />
===What resources exist for big data analysis ?===<br />
An Hadoop cluster for research and teaching came on-line 09 Dec 2015. Documentation on<br />
its use is being developed.<br />
<br />
==<strong><font color="#9966cc">Resources writeups</font></strong>==<br />
===What writeups are available that describe NJIT's high performance computing (HPC) and big data (BD) resources ?===<br />
IST support for researchers [[ISTResearcherSupport]]<br />
<br />
Short Overview [[ShortOverview]]<br />
<br />
Very Short Overview [[VeryShortOverview]]<br />
<br />
Detailed hardware specs [https://web.njit.edu/topics/hpc/specs Cluster specs]<br />
<br />
==<strong><font color="#9966cc">Running Matlab</font></strong>==<br />
===How do I specify matlab jobs to use a specific queue ?===<br />
A queue can be specified within a matlab input file, e.g. :<br />
<pre code><br />
ClusterInfo.setQueue ('medium')<br />
</pre> <br />
<br />
===Is there support for Matlab Distributed Computing Srever (MDSC) on the HPC clusters ?===<br />
Yes. See [[GettingStartedWithSerialAndParallelMATLABOnKongAndStheno]].<br />
<br />
==<strong><font color="#9966cc">Software</font></strong>==<br />
===What software is available on the HPC clusters ?===<br />
The simplest way to see what software is available is to enter :<br />
<pre code><br />
module available<br />
</pre><br />
This will produce a list of most - but not all - available software.<br />
<br />
You can refine your search to list specific software. E.g., to list<br />
all of the GCC compilers avaialable, :<br />
<pre code><br />
module available gcc<br />
</pre><br />
<br />
Recent list of modules [[SoftwareModulesAvailable]]<br />
<br />
The "module available" command can be used on the kong and stheno<br />
head nodes, and on all public-access AFS Linux clients. The shortest<br />
form of this command is "module av".<br />
<br />
===What is the "module" Command ?===<br />
This command is used to set the environment variables for the software you<br />
want to use. see [https://en.wikipedia.org/wiki/Environment_Modules_(software) Environt modules],<br />
and "man module".<br />
<br />
===How do I use the "module load <software>" command ?===<br />
Generally, this command is placed in your submit script, as in :<br />
<pre code><br />
module load matlab<br />
</pre><br />
<br />
===How do I request software that is not currently available ?===<br />
This is done by sending mail to <em>arcs@njit.edu</em>. The software must meet these<br />
criteria :<br />
<ul><br />
<li>requested by faculty or staff</li><br />
<li>free</li><br />
<li>compatible with the current Linux version on the HPC clusters</li><br />
<li>used for research and/or courses</li><br />
</ul><br />
In general, such software will be installed in AFS, and will be accessible to<br />
all AFS Linux clients.<br />
<br />
If the software is not free, a funding source will need to be identified.<br />
<br />
===The Stheno GCC version 4.1.2. Can a newer version be installed ?===<br />
<br />
The problem is that the Stheno operating system, Scientific Linux SL release<br />
5.5, is too old.<br />
<br />
Although several newer versions of gcc are installed in AFS :<br />
<pre code><br />
<br />
"module avail gcc" gives :<br />
<br />
------------------------------- /afs/cad.njit.edu/ucs/modulefiles<br />
gcc/4.8.1 gcc/4.8.2 gcc/4.9.2 gcc/5.2.0 gcc/5.3.0 gcc/5.4.0 gcc/6.1.0<br />
</pre><br />
<br />
none of them will load on Stheno, due to its operating system.<br />
<br />
See [[Roadmap]] for scheduled Stheno operating system upgrade.<br />
<br />
The operating system on Kong.njit.edu, Scientific Linux release 6.2,<br />
does support the gcc installations in AFS.<br />
<br />
Users can get a login on Kong upon request to <em>arcs@njit.edu</em> from a faculty member.<br />
<br />
==<strong><font color="#9966cc">Submit script behaves strangely</font></strong>==<br />
===The statements in my submit script look correct, but it appears that the script is not being read correctly. Why is that ?===<br />
This behavior can happen if your submit script contains control characters that confuse<br />
the SGE job scheduler. The most common cause of the problem is the presence of Windows<br />
DOS line feed characters in the submit script. One way to remove these characters is via<br />
the Linux <em>dos2unix</em> command. See "man dos2unix".<br />
<br />
==<strong><font color="#9966cc">Submitting jobs</font></strong>==<br />
===How are jobs submitted to the HPC cluster compute nodes ?===<br />
Jobs must be submitted using a "submit script". See [[SonOfGridEngine]].<br />
<br />
===What are the valid queue names ?===<br />
Valid queue names are : "short", "medium", and "long", and others. See [[KongQueuesTable]] and [[SthenoQueuesTable]].<br />
<br />
===Can I run jobs on a head node ?===<br />
Jobs run on an HPC cluster head node that use non-negligible resources will be<br />
automatically terminated, with email to that effect sent to the owner of the job.<br />
<br />
===How can I run an interactive job on an HPC cluster compute node ?===<br />
Use "qlogin" on a head node. See [[KongQueues]].<br />
<br />
==<strong><font color="#9966cc">Testing if binaries will run</font></strong>==<br />
===I have binaries, but no source code. How can I tell if the binaries should run on an HPC cluster ?===<br />
You can use the <em>library.check</em> (/usr/ucs/bin/library.check) utility, to check for missing <br />
libraries an/or GLIBC_* versions. For usage : "library.check -h".<br />
<br />
==<strong><font color="#9966cc">Using GPUs</font></strong>==<br />
===What graphical processing units (GPUs) are available in the HPC clusters ?===<br />
See [https://web.njit.edu/topics/hpc/specs Cluster specs]<br />
<br />
===How do I use those GPUs ?===<br />
See [[RunningCUDASamplesOnKong]], [[MatlabGPUOnStheno]]<br />
<br />
==<strong><font color="#9966cc">Using GPUs in parallel on Kong</font></strong>==<br />
===What is the maximum number of GPUs that can be used in parallel on Kong ?===<br />
As of April 2016, there are 2 GPU nodes, each with 2 GPUs, so the maximum number<br />
of GPUs that can be used in parallel is <strong>4</strong>.<br />
<br />
==<strong><font color="#9966cc">Using local scratch for large writes and reads</font></strong>==<br />
===How do I use the large scratch space local to each compute node to significantlty improve throughput ?===<br />
In your submit script, put something like the following :<br />
<pre code><br />
######################################<br />
# copy files needed to scratch<br />
<br />
mkdir -p /scratch/ucid/work<br />
cp <all needed files> /scratch/ucid/work<br />
cd /scratch/ucid/work<br />
#######################################<br />
# Run your program <br />
<br />
/full/path/to/cmd and arguments<br />
########################################<br />
# Copy results to local home directory<br />
<br />
cp <all results and needed files home> ~/results_directory<br />
#########################################<br />
# Delete scratch directory<br />
<br />
rm -rf /scratch/ucid<br />
</pre><br />
<br />
==<strong><font color="#9966cc">Using multithreading</font></strong>==<br />
===My code uses multithreading. How do I tell the submit script to do multithreading ?===<br />
To use multithreading you need to use the threaded parallel environment. In the submit script :<br />
<pre code><br />
#$ -pe threaded NUMBER_OF_CORES<br />
</pre><br />
<br />
NUMBER_OF_CORES should not exceed 8 for the short, medium or long queues.<br />
<br />
For the <strong>smp</strong> queue, up to 32 cores can be used. In the submit script :<br />
<pre code><br />
#$ -pe threaded 32<br />
#$ -q smp<br />
</pre></div>
Hpcwiki1 dept.admin