Using pyhf on lxplus with root

[johnmcgowan1]

Hi pyhf developers,

I would like to use pyhf to fit some data stored in root histograms on lxplus. I've been having some trouble setting up an environment in which I can use root and pyhf. I have been unable to import pyROOT successfully in a virtualenv. I am able to use root in a virtualenv after setting up a python view:

setupATLAS
lsetup "views LCG_98python3 x86_64-centos7-gcc8-opt"

but this has some incompatibilities with pyhf (matplotlib among many other packages).

Is there an existing view that is pyhf compatible. Or is there another simple way to use root and pyhf simultaneously on lxplus?

Thank you in advance,
John

[matthewfeickert]

@johnmcgowan1 I moved this to a GitHub Discussion instead of an Issue as I think this will be a recurring question that will have an answer that will change through time. In the future pyhf will be available in an LCG view, but it might not receive the same level of support as we can't advise users to have all of the correct dependencies needed if the LCG view doesn't provide them.

For the time being, to get full control over a virtual environment you're probably going to need to do something like the following in your .bashrc

function setupENV {
    # Setup default LCG view
    local default_LCG_release="LCG_98python3"
    local default_LCG_platform="x86_64-centos7-gcc8-opt"
    printf "\nlsetup 'views ${default_LCG_release} ${default_LCG_platform}'\n"
    lsetup "views ${default_LCG_release} ${default_LCG_platform}"
    # strip out LGG view site-packages from PYTHONPATH to allow venv control of pip
    IFS=':' read -ra _LCG_paths <<< "${PYTHONPATH}"
    for idx in "${!_LCG_paths[@]}"
    do
        [[ "${_LCG_paths[$idx]}" == *"site-packages"* ]] && unset -v "_LCG_paths[${idx}]"
    done
    export PYTHONPATH=$(IFS=:; echo "${_LCG_paths[*]}")
}

You can then do something like

$ setupENV
$ which python
/cvmfs/sft.cern.ch/lcg/views/LCG_98python3/x86_64-centos7-gcc8-opt/bin/python
$ python --version --version
Python 3.7.6 (default, Aug 12 2020, 09:46:40) 
[GCC 8.3.0]
$ python -m venv example
$ . example/bin/activate
(example) $ python -m pip install --upgrade pip setuptools wheel
(example) $ python -m pip list
Package    Version
---------- -------
pip        21.0.1 
pyspark    2.4.6  
setuptools 56.0.0
wheel      0.36.2 
(example) $ root --version
ROOT Version: 6.22/00
Built for linuxx8664gcc on Jun 14 2020, 15:54:05
From tags/v6-22-00@v6-22-00
(example) $ python -c "from ROOT import TH1F; hist = TH1F('hist', '', 10, 0, 10); print(hist)"
Name: hist Title:  NbinsX: 10
(example) $ python -m pip install pyhf[contrib]
(example) $ python -m pip list
Package            Version  
------------------ ---------
attrs              20.3.0
certifi            2020.12.5
chardet            4.0.0
click              7.1.2
cycler             0.10.0
idna               2.10
importlib-metadata 3.10.1
jsonpatch          1.32
jsonpointer        2.1
jsonschema         3.2.0
kiwisolver         1.3.1
matplotlib         3.4.1
numpy              1.20.2
Pillow             8.2.0
pip                21.0.1
pyhf               0.6.1
pyparsing          2.4.7
pyrsistent         0.17.3
pyspark            2.4.6
python-dateutil    2.8.1
PyYAML             5.4.1
requests           2.25.1
scipy              1.6.2
setuptools         56.0.0
six                1.15.0
tqdm               4.60.0
typing-extensions  3.7.4.3
urllib3            1.26.4
wheel              0.36.2
zipp               3.4.1
(example) $  pyhf --version
pyhf, version 0.6.1

I'll note that this is pretty fragile and if you lsetup other things this can break. I think that @kratsg might have a more robust solution here.

[johnmcgowan1]

Thank you @matthewfeickert. This solution suits my purposes for the time being!