CoEPP RC
 

Grid FAQ

External Python Module (ex: numpy) not found

  • Users have to be carefull about their setup options, because they overwrite default configuration. To exemplify this problem, here is the example of a user which complained that numpy was not found although it was installed in the machine.
$ python
Python 2.7.4 (default, May  5 2013, 17:33:10)
[GCC 4.8.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named numpy
  • This problem was caused by the fact that:
    1. user was configuring his environment with localSetupROOT.
    2. localSetupROOT fetches python 2.7.4-x86_64-slc6-gcc48 (from CVMFS).
    3. python 2.7.4-x86_64-slc6-gcc48 (from CVMFS) does not supports numpy.
  • The solution is to use localSetupSFT. This sets up the ATLAS environment for LCG externals packages. It allows the user to set up all other python analysis tools that you need for your current python version. That is done with:
$ localSetupSFT pyanalysis/1.4_python2.7
************************************************************************
  Current GCC version: gcc (GCC) 4.8.1
  Python version in your path is 2.7
Setting up pyanalysis 1.4_python2.7 ...
 The env $SFT_HOME_pyanalysis is the home dir
************************************************************************

$ python
Python 2.7.4 (default, May  5 2013, 17:33:10)
[GCC 4.8.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>>

Your build job fails (and all other jobs are cancelled)

Look at the job out log (ie. http://panda.cern.ch/server/pandamon/query?job=1875869714#ErrorDetails) we notice:

15 Jul 11:53:32| runJob.py   | Executing job command 1/1: export FRONTIER_ID=\"[1875869714]\";export CMSSW_VERSION=$FRONTIER_ID;export RUCIO_APPID=\"prun\";source /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/cmtsite/asetup.sh 17.0.4.2,notest,AtlasProduction --cmtconfig i686-slc5-gcc43-opt ;export X509_USER_PROXY=/home/pilatl08/home_cream_739346599/cream_739346599.proxy;./buildGen-00-00-01 -i sources.7b0595d6-ed59-4f0b-9bc7-d10736265476.tar.gz -o user.nsoni.0715115033.254260.lib._001847.lib.tgz -r . --sourceURL https://voatlas178.cern.ch:25443 --useRootCore --cmtConfig i686-slc5-gcc43-opt  --oldPrefix \"srm://agh3.atlas.unimelb.edu.au(:8446/srm/managerv2?SFN=)*\" --newPrefix rfio:// --lfcHost prod-lfc-atlas.cern.ch 1>prun_stdout.txt 2>prun_stderr.txt
15 Jul 11:54:28| runJob.py   | Job command 1/1 failed: res = (768, '')
15 Jul 11:54:28| runJob.py   | Job CPU usage: 10 kSI2kseconds
15 Jul 11:54:28| runJob.py   | Job CPU conversion factor: 1.0000000000
15 Jul 11:54:28| runJob.py   | Original exit code: 768
15 Jul 11:54:28| runJob.py   | Exit code: 3 (returned from OS)

We can see we need to look in prun_stderr or prun_stdout for more details about why the build failed (http://voatlas140.cern.ch:25880/monitor/logs/b11f76a9-e444-4789-9055-082ad8978d0a/tarball_PandaJob_1875869714_ANALY_AUSTRALIA/prun_stdout.txt):

Compiling ../Root/TPileupReweighting.cxx
In file included from /usr/include/stdio.h:929,
                 from /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/LCGCMT/LCGCMT_60d/InstallArea/i686-slc5-gcc43-opt/bin/../../../../../sw/lcg/app/releases/ROOT/5.28.00g/i686-slc5-gcc43-opt/root/include/Rtypes.h:33,
                 from /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/LCGCMT/LCGCMT_60d/InstallArea/i686-slc5-gcc43-opt/bin/../../../../../sw/lcg/app/releases/ROOT/5.28.00g/i686-slc5-gcc43-opt/root/include/TObject.h:31,
                 from /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/LCGCMT/LCGCMT_60d/InstallArea/i686-slc5-gcc43-opt/bin/../../../../../sw/lcg/app/releases/ROOT/5.28.00g/i686-slc5-gcc43-opt/root/include/TNamed.h:26,
                 from ../PileupReweighting/TPileupReweighting.h:22,
                 from ../Root/TPileupReweighting.cxx:14:
/usr/include/bits/stdio.h: In function '__ssize_t getline(char**, size_t*, FILE*)':
/usr/include/bits/stdio.h:118: error: '__getdelim' was not declared in this scope
In file included from /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/LCGCMT/LCGCMT_60d/InstallArea/i686-slc5-gcc43-opt/bin/../../../../../sw/lcg/app/releases/ROOT/5.28.00g/i686-slc5-gcc43-opt/root/include/Rtypes.h:34,
                 from /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/LCGCMT/LCGCMT_60d/InstallArea/i686-slc5-gcc43-opt/bin/../../../../../sw/lcg/app/releases/ROOT/5.28.00g/i686-slc5-gcc43-opt/root/include/TObject.h:31,
                 from /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/LCGCMT/LCGCMT_60d/InstallArea/i686-slc5-gcc43-opt/bin/../../../../../sw/lcg/app/releases/ROOT/5.28.00g/i686-slc5-gcc43-opt/root/include/TNamed.h:26,
                 from ../PileupReweighting/TPileupReweighting.h:22,
                 from ../Root/TPileupReweighting.cxx:14:
/usr/include/string.h: At global scope:
/usr/include/string.h:548: error: '__locale_t' has not been declared
/usr/include/string.h:549: error: nonnull argument references non-pointer operand (argument 1, operand 3)
/usr/include/string.h:552: error: '__locale_t' has not been declared
/usr/include/string.h:553: error: nonnull argument references non-pointer operand (argument 1, operand 4)
In file included from /cvmfs/atlas.cern.ch/repo/sw/atlas-gcc/435/x86_64/slc5/x86_64-slc5-gcc43-opt/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.3.5/../../../../include/c++/4.3.5/bits/localefwd.h:47,
                 from /cvmfs/atlas.cern.ch/repo/sw/atlas-gcc/435/x86_64/slc5/x86_64-slc5-gcc43-opt/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.3.5/../../../../include/c++/4.3.5/string:50,
                 from /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/LCGCMT/LCGCMT_60d/InstallArea/i686-slc5-gcc43-opt/bin/../../../../../sw/lcg/app/releases/ROOT/5.28.00g/i686-slc5-gcc43-opt/root/include/TSchemaHelper.h:7,
                 from /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/LCGCMT/LCGCMT_60d/InstallArea/i686-slc5-gcc43-opt/bin/../../../../../sw/lcg/app/releases/ROOT/5.28.00g/i686-slc5-gcc43-opt/root/include/TGenericClassInfo.h:17,
                 from /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/LCGCMT/LCGCMT_60d/InstallArea/i686-slc5-gcc43-opt/bin/../../../../../sw/lcg/app/releases/ROOT/5.28.00g/i686-slc5-gcc43-opt/root/include/Rtypes.h:263,
                 from /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/LCGCMT/LCGCMT_60d/InstallArea/i686-slc5-gcc43-opt/bin/../../../../../sw/lcg/app/releases/ROOT/5.28.00g/i686-slc5-gcc43-opt/root/include/TObject.h:31,
                 from /cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.4/LCGCMT/LCGCMT_60d/InstallArea/i686-slc5-gcc43-opt/bin/../../../../../sw/lcg/app/releases/ROOT/5.28.00g/i686-slc5-gcc43-opt/root/include/TNamed.h:26,
                 from ../PileupReweighting/TPileupReweighting.h:22,
                 from ../Root/TPileupReweighting.cxx:14:
/cvmfs/atlas.cern.ch/repo/sw/atlas-gcc/435/x86_64/slc5/x86_64-slc5-gcc43-opt/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.3.5/../../../../include/c++/4.3.5/x86_64-unknown-linux-gnu/32/bits/c++locale.h:57: error: 'uselocale' was not declared in this scope
/cvmfs/atlas.cern.ch/repo/sw/atlas-gcc/435/x86_64/slc5/x86_64-slc5-gcc43-opt/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.3.5/../../../../include/c++/4.3.5/x86_64-unknown-linux-gnu/32/bits/c++locale.h:57: error: invalid type in declaration before ';' token
/cvmfs/atlas.cern.ch/repo/sw/atlas-gcc/435/x86_64/slc5/x86_64-slc5-gcc43-opt/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.3.5/../../../../include/c++/4.3.5/x86_64-unknown-linux-gnu/32/bits/c++locale.h: In function 'int std::__convert_from_v(__locale_struct* const&, char*, int, const char*, ...)':
/cvmfs/atlas.cern.ch/repo/sw/atlas-gcc/435/x86_64/slc5/x86_64-slc5-gcc43-opt/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.3.5/../../../../include/c++/4.3.5/x86_64-unknown-linux-gnu/32/bits/c++locale.h:77: error: '__gnu_cxx::__uselocale' cannot be used as a function
/cvmfs/atlas.cern.ch/repo/sw/atlas-gcc/435/x86_64/slc5/x86_64-slc5-gcc43-opt/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.3.5/../../../../include/c++/4.3.5/x86_64-unknown-linux-gnu/32/bits/c++locale.h:102: error: '__gnu_cxx::__uselocale' cannot be used as a function
make: *** [../obj/TPileupReweighting.o] Error 1

This looks suspiciously like we might have SL6 compatibility issues in the code.

Looking at the ATLAS SL6 Compatibility page (https://twiki.cern.ch/twiki/bin/view/Atlas/RPMCompatSLC6) it advises to use latest releases of most branch to get the SL6 fixes.

A solution has also been suggested at: hn-atlas-dist-analys@cern.ch mailing list

grid/faq.txt · Last modified: 2014/12/30 11:04 by goncalo
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki