Get python packages from conda-forge, use mamba #2933
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Cartopy and Shapely feel very difficult to install from PyPI, as they have no binary wheels and require a specific version of proj and geos from apt. But the R `sf` package also requires a specific version of proj from apt, and this sort of puts us in an impossible situation as cartopy 0.20.* requires a newer version than sf. So we start using conda-forge to install some of these base scientific packages instead. This reduces the scope for a lot of these conflicts, and lets us peacefully use apt just for R. I'm trying to only have pip packages depend on conda, not the other way around, so I've moved some of the 'base' scientific python packages to be installed from conda. Ideally, we should basically move everything. Ref berkeley-dsep-infra#2824
yuvipanda
force-pushed
the
cartopy-again
branch
from
4f2776a to
f88ffce
Compare
cartopy gets it from conda-forge now
yuvipanda
changed the title
Gives me a whole lot of bogus conflicts that should not be conflicts, like Package typing conflicts for: bokeh=2.3 -> typing_extensions[version='>=3.7.4'] -> typing[version='>=3.7.4'] spacy=3.1 -> typing_extensions[version='>=3.7.4,<4.0.0.0'] -> typing[version='>=3.7.4']
I don't think this will help
yuvipanda
force-pushed
the
cartopy-again
branch
from
eb2cfd7 to
b4362e0
Compare
Mamba is faster and provides error messages that are actually comprehendable on conflicts
These were causing conflicts conda couldn't solve, and there is no reason to move these right now.
From looking at package-popularity data, I don't see it be imported: https://grafana.datahub.berkeley.edu/d/1EJ3WNi7z/python-popularity-contest-copy?orgId=1&from=now-30d&to=now&viewPanel=8 It fails to install with setuptools>58 - see jamescasbon/PyVCF#332.
Not miniforge
yuvipanda
changed the title
This was referenced Nov 2, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cartopy and Shapely feel very difficult to install from PyPI,
as they have no binary wheels and require a specific version of
proj and geos from apt. But the R
sfpackage also requiresa specific version of proj from apt, and this sort of puts us
in an impossible situation as cartopy 0.20.* requires a newer
version than sf.
So we start using conda-forge to install some of these base
scientific packages instead. This reduces the scope for a lot
of these conflicts, and lets us peacefully use apt just for R.
I'm trying to only have pip packages depend on conda, not
the other way around, so I've moved some of the 'base'
scientific python packages to be installed from conda.
Ideally, we should basically move everything.
I've also moved everyone to using mambaforge,
mostly because mamba provided a much more useful error
message than conda when the PR ran into conflicting packages.
We should just use mamba eventually to install packages, as it
is faster.
Also fixes #2925
The pyvcf package is uninstalled, as it is incompatible with newer version
of setuptools (jamescasbon/PyVCF#332). I do not
see any imports of it in our python popularity dashboard (https://grafana.datahub.berkeley.edu/d/1EJ3WNi7z/python-popularity-contest-copy?orgId=1&from=now-30d&to=now&viewPanel=8)
Ref #2824