How to use the Data Transfer Node on Condo

Page

The DTN is used to transfer data to and from the cluster. On Condo The DTN is named “condodtn”. Condodtn has 10 Gbit/second connections to the campus network and to the internet allowing for fast transfer of data to on-campus and off-campus machines. 

To use the DTN ssh from the head node using the same password used to logon to the head node. All files and directories on the DTN are identical to those on the head node. For security, one must use scp and rsync only on the DTN and not on some other machine. If you want to copy many files and/or directories and/or large files, rsync is recommended over scp. An advantage of using rsync instead of scp is that rsync will only copy files if they are different. The following are typical examples using scp and rsync. In the following examples one can replace “scp” or “scp -r” by “rsync -ae ssh”.  

Example 1: To copy ~username/mydir/prog.c from hpc-class to ./mydir/prog.c in your current directory on the DTN, issue the following on the DTN:

scp username@hpc-class.its.iastate.edu:mydir/prog.c ./mydir/prog.c

Example 2: To copy the entire “mydir” directory in example 1, issue the following on the DTN:

scp -r username@hpc-class.its.iastate.edu:mydir ./mydir

Example 3: To copy all the files in “mydir” that end in .c from hpc-class to DTN, issue the following on the DTN:

scp username@hpc-class.its.iastate.edu:mydir/*.c ./newdir

Example 4: To copy prog1.f90 in your current directory on the DTN to prog2.f90 in your home directory on hpc-class issue the following on the DTN:

scp prog1.f90 username@hpc-class.its.iastate.edu:prog2.f90

Grid FTP needs to be installed and configured both on the cluster and on the client side using root privileges. It is the fastest way to transfer data to and from HPC machines.

Reference: "How to use the Condo and CyEnce Cluster" by Gleen R. Luecke, Summer 2015

Category: