Skip to main content

Transferring data

Where is your data?

Your home directory, /users/your-user-name/, is shared across all CARC machines, meaning that once your data has been uploaded to your home directory it is accessible regardless of which machine you are logged in to. Refer to our documentation page on CARC for details on CARC systems.

Graphical User Interface (GUI) options

There are several options available for data transfer that employ a GUI for ease of use. Some popular open source options for various OSs are listed below.

GUI-based programs are very user friendly and well-suited to those that are less comfortable in Linux Command-line interface. Unfortunately, the programs listed above, and other GUI-based programs, employ File Transfer Protocol (FTP), which has a relatively low transfer speed and is suitable for smaller file sizes. CARC supports and encourages using the encrypted version, SFTP, which is supported by all tools discussed on this page.

Command-line interface (CLI) options

GUI's are useful when using an unfamiliar software because it is easy to explore a tool by examiling menus and other visual queues. However, command line based tools are significantly more powerful because they can be scripted and therefor automated. Tools for file transfers are no different, and furthermore, these programs have several benefits over their GUI-based counterparts, such as higher transfer rates and the ability to continue a transfer if it is interrupted without having to restart from the beginning. Below are several popular options with example commands and links for more advanced usage.

Secure Copy (SCP)

Transfer from local machine to CARC:

$ scp /your-file

Transfer from CARC to local machine:

$ scp /target-directory/

Remote Sync (RSYNC)

Transfer from local machine to CARC

$ rsync -vhatP /your-file

Transfer from CARC to local machine

$ rsync -vhatP /target-directory/

The -vhatP flag are instructions to rsync print out the progress of the transfer verbosely and use human-readable units.

BaBar Copy (BBCP)

Transfer from local machine to CARC

$ bbcp /your-file

Transfer from CARC to local machine

$ bbcp /target-directory/

As you can see, the syntax for using the various programs is very similar, however, the options for advanced usage are unique to each program. The examples provided above are for very basic data transfers, but you should refer to the links provided in order to optimize each for maximum data transfer efficiency and speed. The necessity for transfer optimization increases as file size increases.