-
Notifications
You must be signed in to change notification settings - Fork 3
Transferring Data to Brain
Brain's storage system isn't intended for long-term storage of data. As a result, you will need to copy your data and code to Brain, and then copy any results from Brain to some other long-term storage system. In addition, Brain's storage system is not backed up. Thus, Brain must not be the sole storage location of any data you wish to keep.
SFTP will likely be your most common method of data transfer. Using SFTP, you can easily transfer data to/from other DEOHS systems to the cluster. In general, there are two main ways you can perform the transfer. The first being to initiate the transfer using a SFTP client installed on the cluster, with the second being an SFTP client on your DEOHS desktop (or laptop).
When establishing an SFTP session from the cluster, it's best to do so from the head node. This offer the best performance when accessing files on the cluster's storage system. For SFTP connections, you can use the builtin "sftp" command line client. Here's how:
Connecting to a remote host:
sftp USERNAME@HOST
Change remote directory (change which directory you are in on the remote host):
cd /go/to/dir
Change local directory (change which directory you are in on Brain):
lcd /go/to/dir
Download the file:
get file.txt
Download a directory (recursive download):
get -r dir_i_want
Upload a file:
put file.txt
Upload a directory (recursive upload):
put -r dir_i_want
Exit the SFTP session:
quit
Brain's desktop environment includes a builtin file manager, that can speak multiple protocols including SFTP. With it, you can transfer files to remote systems in a graphical manner.
- Click on the "Start" or Application menu (in the lower left corner of the screen)
- Navigate to Programs > Accessories > Files
- Within "Files", click on "Other Locations"
- Look for "Connect To Server", and enter "sftp://server.you.want.to.access/"
- Click on the "Connect" button. You will be prompted for credentials, and to accept the server's SSH fingerprint.
- Once connected, press Ctrl + L, in order to edit the "Path" field.
- With the "Path" field editable, enter the full path to the files you wish to access.
The "Files" app will keep the SFTP connection open, as long as the window itself is running. Transferring files is as simple as drag and drop, or copy and paste between folders within "Files".
Using the SFTP client of your choice (we typically install FileZilla or CyberDuck), you'll want to establish an SFTP connection to brain. Most desktop SFTP clients work the same, with a simple drag and drop interface, much like Windows Explorer. You'll find the connection settings below:
- Protocol: SFTP
- Host (or Server): brain.deohs.washington.edu
- Port: 22
- Username: UW NetID
- Password: UW NetID Password
Below are some tutorials / guides for common desktop SFTP clients:
The Aspera Desktop Client is installed on the cluster's head node or frontend. The client supports both graphical and command-line modes of operation. However, the graphical mode is the primary mode of operation. For the graphical mode, you'll need to establish an X2Go session with the cluster head node. Once connected, open a Terminal window, and run the following:
asperascp
Since Aspera offers many options for transfers, please review the user's manual for further details.
While the Desktop Client's command-line mode may lack some features, it may meet your data transfer needs. Please see the CLI section of the manual for a description of features, and how to initiate a transfer.