CoEPP RC
 

Processing ROOT files from the Tier 3

ROOT files / datasets stored on a Tier 2 disk

One can access files stored on a Tier 2 disk via xrootd (file access protocol). You will need the following to do so

  1. Valid grid proxy certificate
  2. physical location of the files

Issue the following command on a grid enabled UI to gain a valid grid proxy certificate

au-voms-proxy-init

If you don't have a grid certificate then go to this page Grid Computing Start.

To get the physical location of the files in the dataset that you wish to run over you'll need to setup GRID dq2 tools, and then run dq2-ls with some special settings. An example is shown below

setupATLAS --quiet
localSetupDQ2Client
dq2-ls --pfn --files --site=<SITENAME> <DatasetNAME>

Possible sitenames include

AUSTRALIA-ATLAS_LOCALGROUPDISK
AUSTRALIA-ATLAS_SCRATCHDISK
AUSTRALIA-ATLAS_DATADISK

An example

dq2-ls --pfn --files  --site=AUSTRALIA-ATLAS_LOCALGROUPDISK mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937/
...
srm://agh3.atlas.unimelb.edu.au/dpm/atlas.unimelb.edu.au/home/atlas/atlaslocalgroupdisk/rucio/mc11_7TeV/d7/cf/NTUP_TOP.758242._000971.root.1
...

The string reporting the location of the file has to be slightly edited to allow xrootd to find the file. Replace “srm” with “root” and add an extra forward slash “/” after the first instance of “.au”. Therefore the new string able to be interpreted by ROOT should read

root://agh3.atlas.unimelb.edu.au//dpm/atlas.unimelb.edu.au/home/atlas/atlaslocalgroupdisk/rucio/mc11_7TeV/d7/cf/NTUP_TOP.758242._000971.root.1

To open files within an interactive ROOT session you need to explicitly use the “TFile::Open” method because it decodes the argument such that it knows to use xrootd if the filename begins with “root:”. Here's how to open the file in an interactive ROOT session

root [0]  TFile* f = TFile::Open("root://agh3.atlas.unimelb.edu.au//dpm/atlas.unimelb.edu.au/home/atlas/atlasloc
Warning in <TClass::TClass>: no dictionary for class AttributeListLayout is available
Warning in <TClass::TClass>: no dictionary for class pair<string,string> is available

root [1] f->ls()
TXNetFile**        root://agh3.atlas.unimelb.edu.au//dpm/atlas.unimelb.edu.au/home/atlas/atlaslocalgroupdisk/rucio/mc11_7TeV/d7/cf/NTUP_TOP.758242._000971.root.1    StreamNTUP_TOP
 TXNetFile*        root://agh3.atlas.unimelb.edu.au//dpm/atlas.unimelb.edu.au/home/atlas/atlaslocalgroupdisk/rucio/mc11_7TeV/d7/cf/NTUP_TOP.758242._000971.root.1    StreamNTUP_TOP
  KEY: AttributeListLayout    Schema;1
  KEY: TDirectoryFile    physicsMeta;1    physicsMeta
  KEY: TDirectoryFile    Lumi;1    Lumi
  KEY: TTree    CollectionTree;1    CollectionTree
  KEY: TTree    physics;1    physics

Of course you'll want to run over files using a batch script, and to do that you'll have to put your GRID proxy certificate in a place accessible by the worker nodes in the batch queue. Once you obtain a grid proxy the following environment variable is set

echo $X509_USER_PROXY
/tmp/x509up_u702

When you obtain your proxy a directory /tmp/x509up_u702 is created. Copy this directory into an area visible to the batch queue worker nodes. On the cloud batch queue /data is visible to the worker nodes and so below I copy the grid proxy certificate there

rsync -avz --partial --progress /tmp/x509up_u702 /data/antonio/.

Below is an example job script that loads one file, processes the events in the tree and makes a plot of a histogram.

 
#!/bin/bash                                                                                                                                                                              
                                                                                                                                                                                          
export X509_USER_PROXY=/home/antonio/x509up_u702
setupATLAS
localSetupROOT

(
cat <<EOF                                                                                                                                                                               
void run()                                                                                                                                                                                
{                                                                                                                                                                                         
TFile* f      = TFile::Open("root://agh3.atlas.unimelb.edu.au//dpm/atlas.unimelb.edu.au/home/atlas/atlaslocalgroupdisk/rucio/mc11_7TeV/d7/cf/NTUP_TOP.758242._000971.root.1");            
                                                                                                                                                                                          
TH1D* h_Met   = new TH1D("MET",                                 "E^{T}_{miss} [GeV]",  20,      0.   ,   200.    );                                                                       
                                                                                                                                                                                          
TCanvas* c    = new TCanvas("c1","c1",600,600);                                                                                                                                           
                                                                                                                                                                                          
TTree* physics = (TTree*) f->Get("physics");                                                                                                                                              
                                                                                                                                                                                          
physics->Draw("MET_RefFinal_em_tightpp_et/1000.>>MET","","",1000);                                                                                                                        
                                                                                                                                                                                          
c->SaveAs("MET.pdf");                                                                                                                                                                     
}                                                                                                                                                                                         
EOF                                                                                                                                                                                       
) >> run.cc

root -b -q run.cc

The above job script can be sent to a queue. For example

chmod 755 xrdtest_tier2_rootscript.sh
qsub -q syd_short  xrdtest_tier2_rootscript.sh

Naturally you can also read the files interactively in a python session

Python 2.7.3 (default, Oct 25 2012, 12:19:07)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ROOT import *
>>> f = TFile.Open("root://agh3.atlas.unimelb.edu.au//dpm/atlas.unimelb.edu.au/home/atlas/atlaslocalgroupdisk/rucio/mc11_7TeV/d7/cf/NTUP_TOP.758242._000971.root.1")
TClass::TClass:0: RuntimeWarning: no dictionary for class AttributeListLayout is available
TClass::TClass:0: RuntimeWarning: no dictionary for class pair<string,string> is available
>>> f.ls()
TXNetFile**        root://agh3.atlas.unimelb.edu.au//dpm/atlas.unimelb.edu.au/home/atlas/atlaslocalgroupdisk/rucio/mc11_7TeV/d7/cf/NTUP_TOP.758242._000971.root.1    StreamNTUP_TOP
 TXNetFile*        root://agh3.atlas.unimelb.edu.au//dpm/atlas.unimelb.edu.au/home/atlas/atlaslocalgroupdisk/rucio/mc11_7TeV/d7/cf/NTUP_TOP.758242._000971.root.1    StreamNTUP_TOP
  KEY: AttributeListLayout    Schema;1
  KEY: TDirectoryFile    physicsMeta;1    physicsMeta
  KEY: TDirectoryFile    Lumi;1    Lumi
  KEY: TTree    CollectionTree;1    CollectionTree
  KEY: TTree    physics;1    physics
>>>

ROOT files / datasets stored on a Tier 3 disk

One can access files stored on a Tier 3 disk via xrootd (file access protocol). In contrast to accessing ROOT files stored on the Tier 2 all you need to know is the location of the files. The catch is that you have access to Tier 3 storage located in Adelaide, Brisbane, Melbourne and Sydney.

Xrootd redirectors can be used to access files in each of these locations. In the example above you will see that the hostname between the first and second instances of \/\/ (e.g. agh3.atlas.unimelb.edu.au). In Australia we have a redirector at each site hosting data. They are:

  • Adelaide
xrdadlsr.adl.coepp.org.au
  • Brisbane
xrdqldsr.qld.coepp.org.au
  • Melbourne
xrdmelsr.mel.coepp.org.au
  • Sydney
xrdsydsr.syd.coepp.org.au
  • National redirector
xrootd.coepp.org.au

If one wishes to run over data located in Sydney whilst being based in Melbourne, the first step is to find it. Here's how. Connect to the Sydney site redirector. Once is in an xrd session you can navigate the directory tree.

melui1:./~>xrd xrdsydsr.syd.coepp.org.au
(C) 2004-2010 by the Xrootd group. Xrootd version: v3.3.2
Welcome to the xrootd command line interface.
Type 'help' for a list of available commands.
root://sydui.physics.usyd.edu.au:1094//> ls /coepp/local/antonio/tmp2/mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937_tid758240_00
Error 3011: Unable to open directory /coepp/local/antonio/tmp2/mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937_tid758240_00; no such file or directory

In server sydui.physics.usyd.edu.au:1094 or in some of its child nodes.
-r--(016)    814309737 2014-01-11 07:04:08 /coepp/local/antonio/tmp2/mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937_tid758240_00/NTUP_TOP.758240._000001.root.1
-r--(016)    834402407 2014-01-11 05:59:11 /coepp/local/antonio/tmp2/mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937_tid758240_00/NTUP_TOP.758240._000002.root.1
-r--(016)    778814217 2014-01-11 13:45:43 /coepp/local/antonio/tmp2/mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937_tid758240_00/NTUP_TOP.758240._000003.root.1
-r--(016)    776202771 2014-01-11 11:59:10 /coepp/local/antonio/tmp2/mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937_tid758240_00/NTUP_TOP.758240._000004.root.1
-r--(016)    843196757 2014-01-11 15:12:08 /coepp/local/antonio/tmp2/mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937_tid758240_00/NTUP_TOP.758240._000005.root.1
-r--(016)    858529137 2014-01-11 12:01:29 /coepp/local/antonio/tmp2/mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937_tid758240_00/NTUP_TOP.758240._000006.root.1
-r--(016)    803882279 2014-01-11 10:12:37 /coepp/local/antonio/tmp2/mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937_tid758240_00/NTUP_TOP.758240._000007.root.1
-r--(016)    851217479 2014-01-11 06:30:32 /coepp/local/antonio/tmp2/mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937_tid758240_00/NTUP_TOP.758240._000008.root.1
-r--(016)    805065742 2014-01-11 07:43:34 /coepp/local/antonio/tmp2/mc11_7TeV.105200.T1_McAtNlo_Jimmy.merge.NTUP_TOP.e835_s1272_s1274_r3043_r2993_p937_tid758240_00/NTUP_TOP.758240._000009.root.1

root://sydui.physics.usyd.edu.au:1094//>
tier3/faq/accessing_root_files.txt · Last modified: 2014/03/31 12:08 by antonio
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki