Transferring Files Using a Graphical User Interface
For many CARC users, the most convenient way to transfer files between their computer and CARC systems is to use a GUI (graphical user interface)-based SFTP client. The SFTP client is installed and run on your computer. It works by connecting to the SFTP server running on CARC systems, enabling you to transfer files back and forth. You can connect to the CARC's hpc-transfer1 or hpc-transfer2 nodes, which are dedicated, high-speed data transfer nodes.
Along with USC login credentials, Duo two-factor authentication (2FA) is required for CARC access. Most third-party SSH clients can be configured to use 2FA, and to maintain an open connection so as to minimize the number of authentication requests during a transfer session. Refer to your preferred client's documentation for how to do this.
You are welcome to use any SFTP client you wish. Cyberduck, FileZilla, WinSCP, and MobaXterm are popular choices. Instructions for using each of these clients to transfer data between your local computer and CARC systems are described below.
Note: Due to security risks, please be mindful of the type of information being transferred. Where possible, omit all information that may be considered confidential. For examples of confidential information that requires additional consideration, visit http://itservices.usc.edu/security/sensitive-info.
You can download Cyberduck from the vendor website at https://cyberduck.io/.
Go to the Bookmark menu and choose New Bookmark. Save these settings:
- Server: discovery.usc.edu (or endeavour.usc.edu for condo nodes)
- Username: Your USC NetID/username
- (Optional) SSH Private Key: The path to your private key (usually ~/.ssh/id_rsa)
To configure Duo compatibility, expand More Options and set the Transfer Files field to "Use browser connection".
Clicking the red X in the upper right saves your settings. Highlighting the bookmark and pressing Enter on your keyboard will open the connection.
Click Allow to accept the unknown fingerprint, and you will get the login popup.
It won't log you in right away - you need one more step.
This last screen is where you enter "1" to have the Duo push sent to your phone. Once that succeeds, you are logged in to your Linux home directory on the CARC systems.
Note: If you encounter connection timeout errors, you can increase the timeout window: Preferences > Connection > Timeouts. Change the "Timeout for opening connections (seconds)" value to 120, for example.
You can download FileZilla from the vendor website at https://filezilla-project.org.
Make sure to download the FileZilla client, not the FileZilla server. Once the installation is complete, you will need to create a CARC profile on the client. You can do so by going to the Site Manager and clicking on "New site":
For convenience, you can click the Rename button to name the site something memorable. Apply these settings:
- Protocol: SFTP – SSH File Transfer Protocol
- Host: discovery.usc.edu (or endeavour.usc.edu for condo nodes)
- Logon Type: Interactive
- User: Your USC NetID/username
After the General tab settings have been filled out, select the Transfer Settings tab:
- Check "Limit number of simultaneous connections"
- Maximum number of connections: 1
These settings will keep a single connection open so you will not have to re-authenticate.
Note: If you encounter connection timeout errors, you can increase the timeout window: Edit > Settings > Connection > Timeout. Change the "Timeout in seconds" value to 120, for example.
Click Connect and the password prompt will pop up. Enter your USC NetID password:
You will be prompted for a password again, but what the program really wants is the Duo push method. Choose the number corresponding to your preference, such as a "1" to have a push sent to your phone:
Once the authentication goes through, you will be logged in to your home directory, with your local laptop or PC folders on the left, and your Linux home directory structure on the right.
Note: Upon connecting for the first time, you may receive a pop-up asking you to accept a server key. Accept the server key if you encounter this.
You can download WinSCP from the vendor website at https://winscp.net.
After installing, click on New Session and enter your relevant details. Click Login to continue.
Next you will be asked to select your Duo authentication method. Enter the number of your choice.
After you are logged in, you will see a drag-and-drop layout with your local system on one side and your Linux directory structure on the other. The tool offers extensive additional functionality for multiple transfers, file synchronizing, and more. Experiment to find your most productive workflow.
Note: If you encounter connection timeout errors, you can increase the timeout window: Login Dialog > Advanced... > Connection > Timeouts. Change the "Server response timeout" value to 120, for example.
You can download MobaXterm from the vendor website at https://mobaxterm.mobatek.net/download-home-edition.html.
To both transfer and modify (e.g., delete) files, use SFTP with MobaXterm (instead of SCP).
To begin, instead of using SSH, click the "Session" icon on the top left corner (or, Sessions > New Session from the top menu). Click "SFTP" when asked to choose a session type. Enter hpc-transfer1.usc.edu or hpc-transfer2.usc.edu under "Remote Host" and enter your USC NetID as your username. Keep the port as "22". You can choose to save these options as a shortcut under "Bookmark Settings." Lastly, click OK.
You will first be asked to enter your password and then to answer the Duo 2FA prompt.
When successfully entering the SFTP session, your local files will be on the left and your remote CARC files will be on the right. Use the top left file menu to navigate your local files (displayed underneath), and use the buttons on the top right side to navigate your remote files. Drag and drop between the two menus to copy files between your local computer and CARC systems. To modify files on CARC systems, use the buttons at the top.
Note: If you encounter connection timeout errors, click the "Settings" icon from the top menu. Then click to the SSH tab and under "SSH Settings", make sure to check "SSH keepalive" and click OK. Restart MobaXterm and try again.