StorageUseCase

1. Usman Roshan use case characteristics
1. Primary characteristic is a large (several TB) allocation is purchased without backup
2. Very large dataset files downloaded from other institutions or research entities. The downloads can be repeated if the files are lost
3. Research workflow reuses data as algorithm is developed and tuned
4. Research effort creates very large output files, but these are considered expendable as only more recent versions are kept for the purposes of tuning the algorithm, and even if lost, they can be recreated by re-running the workflow>/li>
5. The researcher commits to storing the the algorithm, workflow, and final = results (including source code, makefiles, executables, submit scripts, analysis documents, etc., in AFS or other backed up storage
6. The researcher has confirmed in writing (email) that he understands no backups means files lost, for any reason, cannot be restored
7. The researcher's stated primary motivation for no backup is cost reduction
2. Storage and backup cost URL
The Storage and backup cost URL is at the HPC wiki, which is referenced at various places on the IST website
3. Iulian Neamtiu disk

Two of Neamtiu's servers in GITC 4320 contain large amounts of disk:

• silo100.hpcnet.campus, 80TB
• jd, 80TB, (expected in a few weeks) in GITC 4320

The systems are enterprise-grade (as opposed to consumer-grade or mid-grade)

The following applies to silo100 and most likely to jd also.

• The motherboard/RAID/PWS/case is a manufactured by Supermicro and assembled/sold/supported by Microway
• Equipped with enterprise-grade HDs Ultrastar He10
• Has fiber-optic NIC installed (but not utilized; this is also relevant if connecting this server to Kong)
4. Connecting low-cost disk to Kong
• It is theoretically possible to NFS-mount disk in 4320 to Kong (e.g., disk on silo100 and/or jd), especially since 4320 is directly below 5302, permitting cable to be run between the rooms so traffic is isolated from the campus network. Non-enterprise storage in GITC 5302 could also be NFS-mounted on Kong. NST would be responsible for such implementations
• More broadly, an AFS (AuriStor) fileserver could be set up with a large amount of inexpensive disk that is not backed up This storage would be automatically accessible from all AFS clients, including the HPC clusters, without any administrative intervention once thw fileserver is on-line
• Researchers could purchase inespensive disk in the same way they now purchase enterprise disk