UNL has come to an arrangement with Box.com to provide unlimited cloud storage to every student, staff, and faculty member. This can be useful when used with jobs to automatically upload results when the job has completed. Combined with Box Sync, the uploaded files can be sync'd to your laptop or desktop upon job completion. The upload and download speed of Box is about 20 to 30 MB/s in good network traffic conditions. Users can use a tool called lftp to transfer files between HCC clusters and their Box accounts.
Step-by-step guide for Lftp
- You need to create your UNL Box.com account here.
- Since we are going to be using webdav protocol to access your Box.com storage, you need to create an External Password. In the Box.com interface, you can create it at Account Settings > Create External Password.
After logging into the cluster of your choice, load the
lftpmodule by entering the command below at the prompt:
Connect to Box using your full email as the username and external password you created:
Test the connection by running the
lscommand. You should see a listing of your Box files. Assuming it works, add a bookmark named "box" to use when connecting later:
quit. To reconnect later, use bookmark name:
To upload or download files, use the
putcommands. For example:
To download directories, use the
mirrorcommand. To upload directories, use the
mirrorcommand with the
-Roption. For example, to download a directory named
my_box_dirto your current directory:
To upload a directory named
my_hcc_dirto Box, use
Lftp also supports using scripts to transfer files. This can be used to automatic downloading or uploading files during jobs. For example, create a file called "transfer.sh" with the following lines:
To run this script, do: