S3 file storage
===============

To share files S3 (Simple Storage Service) is used. In this tutorial we use `MinIO client <https://docs.min.io/docs/minio-client-quickstart-guide.html>`_ to work with it. 

You can get access, browse and download files using your browser, using this link:  

`ATHENA S3 Space <https://dtn01.sdcc.bnl.gov:9000/minio/eictest/ATHENA/>`_

**The LOGIN and PASSWORD for read access are available for all members** (but can't be published at open places like this site). Please ask on 
`#software-helpdesk Slack channel <https://eicip6.slack.com/archives/C02267ZDARF>`_ or email one of the SWG members. 

One can access S3 directly using TFile like this

.. code:: C++

    TFile* file = TFile::Open("s3https://dtn01.sdcc.bnl.gov:9000/eictest/ATHENA/<path>.root","AUTH=<login>:<password>");

Or set environment variables `S3_ACCESS_KEY` and `S3_SECRET_KEY` instead of using AUTH and putting login and password to code;

.. code:: bash

    export S3_ACCESS_KEY=<login> 
    export S3_SECRET_KEY=<password>

If you encounter `Forbidden (403)` errors at BNL RCF, try adding the `NOPROXY` option:

.. code:: C++

    TFile* file = TFile::Open("s3https://dtn01.sdcc.bnl.gov:9000/eictest/ATHENA/<path>.root","NOPROXY AUTH=<login>:<password>");


XRootD access
-------------

One can utilize XRootD server to get access to files without credentials using: 

.. code:: 

    root://sci-xrootd.jlab.org//osgpool/eic/ATHENA/<path to file>

.. code:: c++

    std::string fileName = "RECO/JETS/crossDivNrgCrab/DIS_NC_Q2gt10_crossDivNrgCrab_25mRad_18x275_v1.0001.root";
    TFile* file = TFile::Open("root://sci-xrootd.jlab.org//osgpool/eic/ATHENA/" + fileName);
    // ...

(!) Currently (July 2021) only RECO is on xrootd (since we only have a 2.5 TB limit there)

Installation
------------

Installing MinIO client:

.. code:: bash

    wget https://dl.min.io/client/mc/release/linux-amd64/mc
    mkdir -p $HOME/bin && mv mc $HOME/bin/mc && chmod +x  $HOME/bin/mc
    alias mc="$HOME/bin/mc"
    mc --autocompletion

> You may name the command diffirently if mc aka midnight commander which also uses `mc` is installed

Getting access
--------------

Public read-only access

.. code:: bash

    mc config host add S3 https://dtn01.sdcc.bnl.gov:9000 <login> <password>


Private write-enabled access

.. code:: bash

    mc config host add S3rw https://dtn01.sdcc.bnl.gov:9000/ $user $key


Private write-enabled access at BNL

.. code:: bash

    mc config host add S3rw http://eicoss01.sdcc.bnl.local:9000 $user $key


Operations
----------    

Listing contents (the eictest is historical and may change)

.. code:: bash

    mc tree S3/eictest/ATHENA
    mc ls S3/eictest/ATHENA
    mc find S3/eictest/ATHENA --name '*.hepmc*'
    mc du S3/eictest/ATHENA

Copying contents out

.. code:: bash
    
    mc cp --insecure S3/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc .

Mirroring directories

.. code:: bash
    
    mc mirror --insecure S3/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/ crossDivNrgCrab/

Streaming contents out

.. code:: bash

    mc cat --insecure S3/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc | head -n 20
    mc cat S3/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc.gz | gunzip -c | head -n 20

Copying contents in

.. code:: bash

    mc cp /gpfs02/eic/bpage/home/eicBeamSimu/Pythia8/headonTestJin/test_crossDivNrgCrab_* S3rw/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab

Streaming contents in

.. code:: bash

    cat /gpfs02/eic/bpage/home/eicBeamSimu/Pythia8/headonTestJin/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc | mc pipe S3rw/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc
    gzip -c /gpfs02/eic/bpage/home/eicBeamSimu/Pythia8/headonTestJin/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc | mc pipe S3rw/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc.gz