Tier-3 Software Management
setupATLAS
$ setupATLAS
(...)
$ printMenu
...Type localSetupAGIS to setup AGIS
...Type localSetupAtlantis to setup Atlantis
...Type localSetupDQ2Client to use DQ2 Client
...Type localSetupEIClient to setup EIClient
...Type localSetupEmi to use emi
...Type localSetupFAX to use FAX
...Type localSetupGanga to use Ganga
...Type localSetupGcc to use alternate gcc
...Type localSetupPacman to use Pacman
...Type localSetupPandaClient to use Panda Client
...Type localSetupPyAMI to setup pyAMI
...Type localSetupPoD to setup Proof-on-Demand
...Type localSetupROOT to setup (standalone) ROOT
...Type localSetupRucioClients to setup rucio-clients
...Type localSetupSFT to setup SFT packages
...Type localSetupXRootD to setup XRootD
...Type showVersions to show versions of installed software
...Type asetup to setup a release (changeASetup to change asetup version)
...Type rcSetup to setup an ASG release (changeRCSetup to change rcSetup ver.)
...Type diagnostics for diagnostic tools
...Type helpMe for more help
...Type printMenu to show this menu
$ localSetupROOT --help
Usage: localSetupROOT [options] [version]
This sets up the ROOT environment. It is meant to be used outside
Athena so do not source this file if you intend to run Athena jobs.
You need to set the environment variable ATLAS_LOCAL_ROOT_BASE first.
Options (to override defaults) are:
-h --help Print this help message
--skipConfirm Skip the confirmation dialogue
--quiet Print no output (also sets skipConfirm)
--xrdVer=STRING xrootd version to setup
it will setup the approproate OS/platform version
--davixVer=STRING davix version to setup
This is dev so default is current
Note: if you run h2root, you may see an error about a missing shared
library libshift.so (used by Castor). If you see this, do
export LD_LIBRARY_PATH=$LD_LIBARY_PATH:<path to an ATLAS kit>/external/slc3compat/2.0.0/i686-slc3-gcc323/lib
Note: root 5.32 and newer will setup external xrootd package.
To disable this, do prior to setup
export ALRB_SKIP_XDR=1
ROOT versions:
---> 5.28.00g-i686-slc5-gcc43-opt
---> 5.28.00g-x86_64-slc5-gcc43-opt
(...)
$ localSetupROOT 5.34.25-x86_64-slc6-gcc48-opt
************************************************************************
Setting up ROOT ...
************************************************************************
Setting up ROOT version 5.34.25-x86_64-slc6-gcc48-opt
Current GCC version: gcc (GCC) 4.8.1
Python version in your path is 2.7.4
Python version used to build ROOT is 2.7
Setting up gccxml version 0.9.0_20120309p2-x86_64-slc6-gcc48-opt
Setting up gsl version 1.10-x86_64-slc6-gcc48-opt
Setting up fftw version 3.1.2-x86_64-slc6-gcc48-opt
Setting up XRootD version 3.3.6.p1-x86_64-slc6-gcc48-opt
Setting up davix version 0.4.0-x86_64-slc6
DAVIX root is $ALRB_DAVIX_ROOT
************************************************************************
Tip for _this_ standalone ROOT and grid (ie prun) submission:
avoid --athenaTag if you do not need athena
use --rootVer=5.34/25 --cmtConfig=x86_64-slc6-gcc48-opt
************************************************************************
$ which root
/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.25-x86_64-slc6-gcc48-opt/bin/root
$ which python
/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/python/2.7.4-x86_64-slc6-gcc48/sw/lcg/external/Python/2.7.4/x86_64-slc6-gcc48-opt/bin/python
$ which gcc
/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Gcc/gcc481_x86_64_slc6/slc6/gcc48/bin/gcc
setupSW
$ setupSW
(...)
$ printMenu
...Type localSetupAGIS to setup AGIS
...Type localSetupAtlantis to setup Atlantis
...Type localSetupDQ2Client to use DQ2 Client
...Type localSetupEIClient to setup EIClient
...Type localSetupEmi to use emi
...Type localSetupFAX to use FAX
...Type localSetupGanga to use Ganga
...Type localSetupGcc to use alternate gcc
...Type localSetupPacman to use Pacman
...Type localSetupPandaClient to use Panda Client
...Type localSetupPyAMI to setup pyAMI
...Type localSetupPoD to setup Proof-on-Demand
...Type localSetupROOT to setup (standalone) ROOT
...Type localSetupRucioClients to setup rucio-clients
...Type localSetupSFT to setup SFT packages
...Type localSetupXRootD to setup XRootD
...Type showVersions to show versions of installed software
...Type asetup to setup a release (changeASetup to change asetup version)
...Type rcSetup to setup an ASG release (changeRCSetup to change rcSetup ver.)
...Type diagnostics for diagnostic tools
...Type helpMe for more help
...Type printMenu to show this menu
People from theory may be specially interested in the localSetupSFT command since it makes available a wide set of MC generators and 3rd party pieces of software like blas, boost, cernlib, clhep, and much more.
$ localSetupSFT --help
Usage: localSetupSFT [options] [version]
This sets up the ATLAS environment for LCG externals packages
You need to set the environment variable ATLAS_LOCAL_ROOT_BASE first.
Options (to override defaults) are:
-h --help Print this help message
--skipConfirm Skip the confirmation dialogue
--quiet Print no output
--lcgExtSW=STRING LCG External software to setup
This is from the list below.
Separate multiple values with commas.
--cmtConfig=STRING CMTCONFIG version to use
Default is Athena or Standalone ROOT's cmtconfig
For more details, please see
https://twiki.atlas-canada.ca/bin/view/AtlasCanada/LocalSetupSFT
sft versions:
bjam/3.1.13
blas/20070405
blas/20110419
Boost/1.42.0_python2.5
(...)
$ localSetupSFT MCGenerators_lcgcmt67b/herwig++/2.7.1
************************************************************************
Current GCC version: gcc (GCC) 4.8.1
Python version in your path is 2.7
Setting up herwig++ 2.7.1 ...
The env $SFT_HOME_herwigPP is the home dir
************************************************************************
Modules
<OS VERSION>/<COMPILER VERSION>/<PYTHON VERSION>/<NAME>-<VERSION>.<ARCH>
Usage
sl6/gcc447/texlive-2014.x86_64
1) Check the current latex version
# which latex
/usr/bin/latex
# rpm -qf /usr/bin/latex
texlive-latex-2007-57.el6_2.x86_64
--- * ---
2) Check which software modules are available
# module avail
----------- /usr/share/Modules/modulefiles ------------
sl6/gcc447/bbcp-15.02.03.00.1.x86_64
sl6/gcc447/texlive-2014.x86_64
sl6/gcc480/python274/coepp/pytools-1.0.0.x86_64
sl6/gcc480/python274/madanalysis5-v1.1.11_patch1b.x86_64
----------- /etc/modulefiles --------------------------
openmpi-x86_64
--- * ---
3) Load the **sl6/gcc447/texlive-2014.x86_64** module
# module load sl6/gcc447/texlive-2014.x86_64
# module list
Currently Loaded Modulefiles:
1) sl6/gcc447/texlive-2014.x86_64
# which latex
/cvmfs/experimental.cloud.coepp.org.au/sw/sl6/gcc447/x86_64/texlive/2014/bin/x86_64-linux/latex
--- * ---
4) Go back to the default latex version
# module unload sl6/gcc447/texlive-2014.x86_64
$ module list
No Modulefiles Currently Loaded.
# which latex
/usr/bin/latex
sl6/gcc447/bbcp-15.02.03.00.1.x86_64
1) Check which software modules are available
# module avail
----------- /usr/share/Modules/modulefiles ------------
sl6/gcc447/bbcp-15.02.03.00.1.x86_64
sl6/gcc447/texlive-2014.x86_64
sl6/gcc480/python274/coepp/pytools-1.0.0.x86_64
sl6/gcc480/python274/madanalysis5-v1.1.11_patch1b.x86_64
----------- /etc/modulefiles --------------------------
openmpi-x86_64
--- * ---
2) Load the sl6/gcc447/bbcp-15.02.03.00.1.x86_64 module so that the bbcp command becomes available
$ module load sl6/gcc447/bbcp-15.02.03.00.1.x86_64
$ which bbcp
/cvmfs/experimental.cloud.coepp.org.au/sw/sl6/gcc447/x86_64/bbcp/15.02.03.00.1/bin/bbcp
--- * ---
3) List the current loaded modules
$ module list
Currently Loaded Modulefiles:
1) sl6/gcc447/bbcp-15.02.03.00.1.x86_64
--- * ---
4) Unload the module
$ module unload sl6/gcc447/bbcp-15.02.03.00.1.x86_64
$ module list
No Modulefiles Currently Loaded.
Basic bbcp Manual
Advanced bbcp Example: Copies using tar / pipes
The –N option enables copying via a combination of named pipes or programs.
The arguments to –N indicate whether the source (i) is a named pipe or program or the target (o) is a named pipe or program.
When the source is a named pipe or program, only one source may be specified. Determination of whether the data source or target is a pipe or program is done at copy-time on each node. If a pipe is detected, it is treated as a regular file with no specified size.
bbcp -P 10 -s 8 -e -E md5 -N io 'tar cvf - <source directory>' root@<remote_host>:'tar -C <destiny dir> -xvf -'
# Does not unpacks the tar at the destiny (good for lustre)
bbcp -P 10 -s 8 -e -E md5 -N i "tar cvf - <source directory>" root@<remote_host>:<destiny dir>
References
1) Load ROOT
# setupATLAS (or setupSW)
# localSetupROOT
$ which root
/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.25-x86_64-slc6-gcc48-opt/bin/root
$ which gcc
/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Gcc/gcc481_x86_64_slc6/slc6/gcc48/bin/gcc
$ which python
/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/python/2.7.4-x86_64-slc6-gcc48/sw/lcg/external/Python/2.7.4/x86_64-slc6-gcc48-opt/bin/python
--- * ---
2) Check that numpy and scipy are not available
$ python
Python 2.7.4 (default, May 5 2013, 17:33:10)
[GCC 4.8.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named numpy
>>> import scipy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named scipy
>>>
--- * ---
3) Load the 'sl6/gcc480/python274/coepp/pytools-1.0.0.x86_64' module, and retry to import numpy and scipy
$ module load sl6/gcc480/python274/coepp/pytools-1.0.0.x86_64
$ module list
Currently Loaded Modulefiles:
1) sl6/gcc480/python274/coepp/pytools-1.0.0.x86_64
$ python
python 2.7.4 (default, May 5 2013, 17:33:10)
[GCC 4.8.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import scipy
>>>
--- * ---
4) Unload the module
$ module unload sl6/gcc480/python274/coepp/pytools-1.0.0.x86_64
Note on virtualenv
However, users have the possibility to create their own virtual envs in some personnel / private areas for specific things. The problem is that, by default, there is no inheritance of multiple virtualenv, and by enabling a personnel virtualenv, the user looses access to the coepp pytools. VirtualEnvs are there to isolate environments, and there is no option at the command line level, and at VirtualEnv creation time, to allow inheritance.
To enable private virtualenvs without loosing access to the central coepp pytools virtualenv, users will have to activate their private environments but do not use the python binary deployed there. To achieve this, users have to:
In the following example, I demonstrate how I can use django, deployed under my private virtual env, and still use the python software deployed through coepp pytools virtualenv:
-bash-4.1$ python
Python 2.7.4 (default, May 5 2013, 17:33:10)
[GCC 4.8.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> activate_this = '/home/goncalo/coepp/mypersonnelvirtualenv/bin/activate_this.py'
>>> execfile(activate_this, dict(__file__=activate_this))
>>> import django
>>> import rootpy
sl6/gcc480/python274/madanalysis5-v1.1.11_patch1b.x86_64
1) Load ROOT
# setupATLAS (or setupSW)
# localSetupROOT
$ which root
/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.25-x86_64-slc6-gcc48-opt/bin/root
$ which gcc
/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Gcc/gcc481_x86_64_slc6/slc6/gcc48/bin/gcc
$ which python
/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/python/2.7.4-x86_64-slc6-gcc48/sw/lcg/external/Python/2.7.4/x86_64-slc6-gcc48-opt/bin/python
--- * ---
2) Load 'sl6/gcc480/python274/madanalysis5-v1.1.11_patch1b.x86_64' module. Check that 'sl6/gcc480/python274/coepp/pytools-1.0.0.x86_64' is loaded automatically
$ module load sl6/gcc480/python274/madanalysis5-v1.1.11_patch1b.x86_64
$ module list
Currently Loaded Modulefiles:
1) sl6/gcc480/python274/coepp/pytools-1.0.0.x86_64 2) sl6/gcc480/python274/madanalysis5-v1.1.11_patch1b.x86_64
$ which ma5
/cvmfs/experimental.cloud.coepp.org.au/sw/sl6/gcc480/python274/x86_64/madanalysis5/v1.1.11_patch1b/bin/ma5
--- * ---
3) Run madanalysis
$ ma5
*************************************************************
* *
* W E L C O M E to M A D A N A L Y S I S 5 *
* ______ ______ *
* /'\_/`\/\ __ \/\ ___\ *
* /\ \ \ \_\ \ \ \__/ *
* \ \ \__\ \ \ __ \ \___``\ *
* \ \ \_/\ \ \ \/\ \/\ \_\ \ *
* \ \_\\ \_\ \_\ \_\ \____/ *
* \/_/ \/_/\/_/\/_/\/___/ *
* *
* MA5 release : 1.1.11 2014/09/15 *
* *
* Comput. Phys. Commun. 184 (2013) 222-256 *
* J. Phys. Conf. Ser. 123 (2014) 012032 *
* arXiv:1405.3982 [hep-ph] *
* *
* The MadAnalysis Development Team - Please visit us at *
* https://launchpad.net/madanalysis5 *
* *
* Type 'help' for in-line help. *
* *
*************************************************************
Platform: Linux 2.6.32-504.3.3.el6.x86_64 [Linux mode]
Reading user settings ...
Checking mandatory packages:
- python [OK]
- python library: numpy [OK]
- g++ [OK]
- GNU Make [OK]
- Root [OK]
- PyRoot libraries [OK]
Checking optional packages:
- pdflatex [OK]
- latex [OK]
- dvipdf [OK]
- zlib [OK]
- FastJet [OK]
- Delphes [OK]
- Delphes-MA5tune [OK]
- Recasting tools [OK]
Checking the MadAnalysis library:
=> MadAnalysis libraries found.
=> MadAnalysis test program works.
*************************************************************
MadGraph 5 NOT found:
=> Particle labels from input/particles_name_default.txt
=> 87 particles successfully exported.
=> Multiparticle labels from madanalysis/input/multiparticles_default.txt
=> Creation of the label 'invisible' (-> missing energy).
=> Creation of the label 'hadronic' (-> jet energy).
=> 8 multiparticles successfully exported.
ma5>exit
--- * ---
4) Unload the module
$ module unload sl6/gcc480/python274/madanalysis5-v1.1.11_patch1b.x86_64
$ module list
No Modulefiles Currently Loaded.