Data Transfer to YODA or LDS from ALICE (irods)

The iCommands are used to connect to an iRods based storage. Examples are Yoda and Leiden Data Store (LDS). The iCommands need to be configured to connect to the right environment via a configuration file in your home directory. If the file does not exists on your system, the configuration file can be setup in the following way:

Create the .irods directory in your Linux home directory (~ is short for /home/<username>/ under Linux):

$ mkdir ~/.irods

Create the configuration file ~/.irods/irods_environment.json using your preferred text editor:

{
"irods_authentication_scheme": "pam_password",
"irods_client_server_negotiation": "request_server_negotiation",
"irods_client_server_policy": "CS_NEG_REQUIRE",
"irods_default_resource": "sc_rds",
"irods_encryption_algorithm": "AES-256-CBC",
"irods_encryption_key_size": 32,
"irods_encryption_num_hash_rounds": 16,
"irods_encryption_salt_size": 8,
"irods_home": "/lds_mainzone/home/<USERNAME>",
"irods_host": "provider.lds.services.universiteitleiden.nl",
"irods_port": 1247,
"irods_ssl_verify_server": "cert",
"irods_user_name": "<USERNAME>",
"irods_zone_name": "lds_mainzone"
}

{
    "irods_host": "leiden-yoda.irods.surfsara.nl",
    "irods_port": 1247,
    "irods_home": "/leiden/home",
    "irods_user_name": "b.van.dijk@issc.leidenuniv.nl",
    "irods_zone_name": "leiden",
    "irods_authentication_scheme": "pam",
    "irods_encryption_algorithm": "AES-256-CBC",
    "irods_encryption_key_size": 32,
    "irods_encryption_num_hash_rounds": 16,
    "irods_encryption_salt_size": 8,
    "irods_client_server_negotiation": "request_server_negotiation"
}

Notes:

The iRODS username <USERNAME> should be replaced by your UL username (LDS) or UL email address (Yoda).
Your username is case-sensitive, make sure to use all lowercase letters!

To see if everything was properly configured, you can now start the session by invoking the iinit command which will initialize your connection to the RDMS:

$ iinit
Enter your current PAM password:

If you do not see an error after specifying your password, you are set to start using iCommands. See the section below for more information on using iCommands including some examples.

Note:
The iinit command will not just initialize your connection to the RDMS, but will also store your password in an obfuscated format under ~/.irods/.irodsA. Therefore, you do not have to re-type your password each time a subsequent command is executed during the same session. With --ttl the session time can be changed from the default.

Transferring (many) small files is less efficient. If possible use tar to bundle small files.

More information is on the LDS manual and FAQ pages:
https://pubappslu.atlassian.net/wiki/x/EgC9W?xpis=eyJicmlkZ2UiOiJzbWFydExpbmtzIiwiaWQiOiIxNzgyNzM2MDU5MTg0Iiwic291cmNlIjoiY29uZmx1ZW5jZSJ9
https://pubappslu.atlassian.net/wiki/x/HAA_Zg?xpis=eyJicmlkZ2UiOiJzbWFydExpbmtzIiwiaWQiOiIxNzgyNzM2MDU5MTg0Iiwic291cmNlIjoiY29uZmx1ZW5jZSJ9

Links:

LDS manual: https://pubappslu.atlassian.net/wiki/x/EgC9W
iCommands manual https://pubappslu.atlassian.net/wiki/x/EwCOOw

FAQ ALICE-Leiden Data Store: https://pubappslu.atlassian.net/wiki/x/HAA_Zg

(deprecated) Getting started with the iRODS iCommands - RICC