Research Computing | Condo2017 Storage

Transcript or Alternate URL: 

This video will cover the differences in the storage architecture between condo and condo2017.

Condo2017 uses new home directories. This is necessary because the new system uses a newer OS, a different job scheduler, and new modules. Startup files, often called dot files, in your old home directory could interfere with proper login and job startup.

If you have files you would like to move from your old condo home directory it is available in a read-only form at /oldhome/yournetid. Feel to copy any of your old files; but as we said, we discourage copying of your startup files (.bashrc, .profile, etc)

As before a large parallel tmp directory /ptmp is available. It is currently 230TB but will be combined with old condo /ptmp when all users have been moved away from that system. Remember /ptmp is temporary storage. It should not be used for permanent file storage as files are automatically deleted 90 days after their creation date.

Also as before each compute node provides local scratch space. Local scratch space is 2.5TB and will be deleted at the end of each job.
If you are moving large amounts of data you are encouraged to use our data transfer node, condodtn. Condodtn is a dedicated machine configured for fast data transfer and to keep data transfer loads off of the head node and compute nodes.

Use of /work directories is now mandatory. /dataXXX folders no longer exist. If you need to access files from myfiles they are available at /myfiles/Users/YourNetId Other myfiles folders like las/research can be accessed as /myfiles/las/research The new Large Scale Storage (LSS) can be accessed at /lss/research/pinetid-lab (where PINetID is the NetID of the PI for your group). This storage is not yet ready for general use, we'll send out an announcement soon about general availability.

Neither myfiles or lss storage is available on the compute nodes. They are only available on condodtn and the condo2017 headnode. Any files that will be needed for a job need to be copied to your groups /work directory or /ptmp.

This has been a review of storage on the condo2017 cluster. Further information is available on the HPC website at: