S3 file storage

To share files S3 (Simple Storage Service) is used. In this tutorial we use MinIO client to work with it.

You can get access, browse and download files using your browser, using this link:

ATHENA S3 Space

The LOGIN and PASSWORD for read access are available for all members (but can’t be published at open places like this site). Please ask on #software-helpdesk Slack channel or email one of the SWG members.

One can access S3 directly using TFile like this

TFile* file = TFile::Open("s3https://dtn01.sdcc.bnl.gov:9000/eictest/ATHENA/<path>.root","AUTH=<login>:<password>");

Or set environment variables S3_ACCESS_KEY and S3_SECRET_KEY instead of using AUTH and putting login and password to code;

export S3_ACCESS_KEY=<login>
export S3_SECRET_KEY=<password>

If you encounter Forbidden (403) errors at BNL RCF, try adding the NOPROXY option:

TFile* file = TFile::Open("s3https://dtn01.sdcc.bnl.gov:9000/eictest/ATHENA/<path>.root","NOPROXY AUTH=<login>:<password>");

XRootD access

One can utilize XRootD server to get access to files without credentials using:

root://sci-xrootd.jlab.org//osgpool/eic/ATHENA/<path to file>
std::string fileName = "RECO/JETS/crossDivNrgCrab/DIS_NC_Q2gt10_crossDivNrgCrab_25mRad_18x275_v1.0001.root";
TFile* file = TFile::Open("root://sci-xrootd.jlab.org//osgpool/eic/ATHENA/" + fileName);
// ...

(!) Currently (July 2021) only RECO is on xrootd (since we only have a 2.5 TB limit there)

Installation

Installing MinIO client:

wget https://dl.min.io/client/mc/release/linux-amd64/mc
mkdir -p $HOME/bin && mv mc $HOME/bin/mc && chmod +x  $HOME/bin/mc
alias mc="$HOME/bin/mc"
mc --autocompletion

> You may name the command diffirently if mc aka midnight commander which also uses mc is installed

Getting access

Public read-only access

mc config host add S3 https://dtn01.sdcc.bnl.gov:9000 <login> <password>

Private write-enabled access

mc config host add S3rw https://dtn01.sdcc.bnl.gov:9000/ $user $key

Private write-enabled access at BNL

mc config host add S3rw http://eicoss01.sdcc.bnl.local:9000 $user $key

Operations

Listing contents (the eictest is historical and may change)

mc tree S3/eictest/ATHENA
mc ls S3/eictest/ATHENA
mc find S3/eictest/ATHENA --name '*.hepmc*'
mc du S3/eictest/ATHENA

Copying contents out

mc cp --insecure S3/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc .

Mirroring directories

mc mirror --insecure S3/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/ crossDivNrgCrab/

Streaming contents out

mc cat --insecure S3/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc | head -n 20
mc cat S3/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc.gz | gunzip -c | head -n 20

Copying contents in

mc cp /gpfs02/eic/bpage/home/eicBeamSimu/Pythia8/headonTestJin/test_crossDivNrgCrab_* S3rw/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab

Streaming contents in

cat /gpfs02/eic/bpage/home/eicBeamSimu/Pythia8/headonTestJin/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc | mc pipe S3rw/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc
gzip -c /gpfs02/eic/bpage/home/eicBeamSimu/Pythia8/headonTestJin/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc | mc pipe S3rw/eictest/ATHENA/EVGEN/JETS/crossDivNrgCrab/test_crossDivNrgCrab_25mRad_18x275_v1.hepmc.gz