Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why not uv? #4

Closed
MarcSkovMadsen opened this issue Nov 23, 2024 · 21 comments · Fixed by #5
Closed

Why not uv? #4

MarcSkovMadsen opened this issue Nov 23, 2024 · 21 comments · Fixed by #5

Comments

@MarcSkovMadsen
Copy link
Contributor

MarcSkovMadsen commented Nov 23, 2024

Some users might be in an organisation with a very restricted environment. pixi is just harder to install than uv. uv can be installed with pip. If you are a windows user you will need to install pixi via a windows installer which might not be allowed or possible. I think there are other ways. But the pixi web site describes so many different ways of installing that it appears complicated. There is no simple, single way to install no matter what.

Furthermore uv is a very well known and widely adapted tool already while pixi appears more niche. For example Ray Bell writes he uses pixi for those 10% of projects that are very hard to install and uv for the rest.

Please consider migrating from pixi to uv to ease adaption.

@MarcSkovMadsen MarcSkovMadsen changed the title Make it possible to use template for users in restricted environment Ease adoption with uv Nov 23, 2024
@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 23, 2024

Besides not being able to easily install on windows, I also have challenges on Linux because I'm running on a Jupyterhub inside a Docker container where /home/jovyan/.bashrc is readonly. Thus I get

touch: cannot touch '/home/jovyan/.bashrc': Permission denied

when trying to install pixi. This just complicates things.

I have to manually add

export PATH=$PATH:~/.pixi/bin

to my /home/jovyan/<user-initials>/.bashrc file.

I never have those problems with uv. It just works.

@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 23, 2024

Ahh. Now I need to start a stronger server because pixi uses too many resources and my pod is killed. I never have to do that with any of my other tools or projects:

image

image

@MarcSkovMadsen
Copy link
Contributor Author

Running pytest takes for ever because it needs to update (?) the environments.

image

And then my pod is killed again

image

@MarcSkovMadsen MarcSkovMadsen changed the title Ease adoption with uv Why not uv? Nov 23, 2024
@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 24, 2024

Ok. I would like to add pandas as a test dependency and narwhals as a package dependency.. Should I add pandas to dev environment in pyproject.toml? Should I add it to test dependencies in pixi.toml? How? How does that lock file behave and should it be included in my git history?

@MarcSkovMadsen
Copy link
Contributor Author

Ok. I figured out that to select my python interpreter in VS Code I need to manually select .pixi/envs/default/bin/python.

@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 24, 2024

Ok. I want to jump into a shell. But its a pretty slow experience as it starts updating all the environments. It ended up taking 60 secs!

image

@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 24, 2024

Ok. I tried to install autocompletion as described in the pixi guide

echo 'eval "$(pixi completion --shell bash)"' >> /home/jovyan/<my-user-initials>/.bashrc 

It corrupted my .bashrc. When I open a terminal I see:

fi': not a valid identifier

The problem is that it does not append to a new line

image

@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 24, 2024

Ok there is a VS Code pixi extension that should make life easier. But unfortunately its not in the open-vsx.org Market Place that Code Server is using. And I'm working on code-server.

image

@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 24, 2024

Now I run pytest again. Have to wait 60 s again.

image

@MarcSkovMadsen
Copy link
Contributor Author

No.It killed my pod with up to 16 cores and 128 GB RAM!

image

@MarcSkovMadsen
Copy link
Contributor Author

I don't have time for pixi. Will just do uv pip install -e .[dev] and go from there.

@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 24, 2024

I think the problem is that pixi (with our configuration?) works perfectly for ci/ cd but not for personal development. The question is how it can be configured to also work for personal development?

@ahuang11
Copy link
Contributor

I don't have a strong preference to pixi as I do to mkdocs because I personally just use conda & pip for my own dev envs.

As far as I understand, uv is a drop in replacement to pip? I'm wondering if there will be any issues with uv compiling Panel JSComponents? I'm also wondering whether hatch is a better replacement?

I don't have much experience in all these beyond pip and conda. I would like to hear what @hoxbro thinks since he brought it into the HoloViz repos.

@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 24, 2024

I don't have a strong preference to pixi as I do to mkdocs because I personally just use conda & pip for my own dev envs.

As far as I understand, uv is a drop in replacement to pip? I'm wondering if there will be any issues with uv compiling Panel JSComponents? I'm also wondering whether hatch is a better replacement?

I don't have much experience in all these beyond pip and conda. I would like to hear what @hoxbro thinks since he brought it into the HoloViz repos.

I don't think uv, conda or pixi is compiling panel jscomponents. Its done with Panel and Node. The challenge can be to get Node installed. You can do that with conda or pixi. Not uv.

@ahuang11
Copy link
Contributor

Right, I think that's the supporting argument for pixi, but as you've experienced maybe it's excessive to set up on Windows, and maybe conda is sufficient.

@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 24, 2024

Keeping the pixi.lock file up to date is also a mystery for me. How do I do that? I ended up deleting it and running pixi install again.

@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 24, 2024

Then pixi does not install non-conda packages by default and the error message is not great. For example mkdocs-pycafe is not a conda package. Took me some time to figure out cause and solution.

Maybe its obvious for someone working with conda every day. I'm not though.

image

@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 24, 2024

Ok. My docs can't build.

Looks like my package is not installed in the docs environment?

image

To build docs locally I need

image

Otherwise auto doc won't find my code.

Originally this was src.panel_copy_paste. Maybe it needs to be that in ci/cd?

@MarcSkovMadsen
Copy link
Contributor Author

MarcSkovMadsen commented Nov 24, 2024

I've changed it back to src.panel_copy_paste. Then my docs can build in actions. But docstrings are still not collected.

I've tried to search and figure out how to get panel_copy_paste installed in the docs pixi environment. But I've not been able to. The docs look stylish. But they are really hard to navigate.

@MarcSkovMadsen
Copy link
Contributor Author

Ok. Figured it out. had to set the mkdocstrings path:

image

@MarcSkovMadsen MarcSkovMadsen linked a pull request Nov 24, 2024 that will close this issue
Hofer-Julian added a commit to Hofer-Julian/pixi that referenced this issue Nov 25, 2024
We currently give people a one-liner to setup completions.

That is convenient, but also fragile.
If people's startup script doesn't end in a newline, it [breaks](panel-extensions/copier-template-panel-extension#4 (comment))
If people run it multiple times it will be duplicated.

Let's follow starship.rs and give users instructions to add it themselves
@hoxbro
Copy link

hoxbro commented Nov 25, 2024

I just want to clarify that I use Pixi as part of my local development, and I'm pretty sure I heard @philippjfr say something similar. We only transitioned to Pixi earlier this year, and before that, we all did what @ahuang11 did and just used a combination of conda/pip because pyctdev was that bad to use. Habits die hard, and I would not force people to use Pixi if they have something that works.


I will only go over some of the comments left in this issue. Some of them have already been resolved, but I still want to add some clarification.

If you are a windows user you will need to install pixi via a windows installer which might not be allowed or possible. I think there are other ways. But the pixi web site describes so many different ways of installing that it appears complicated. There is no simple, single way to install no matter what.

There is a simple way for Windows users to install Pixi: use the installer. However, if you are in a corporate setting that bans this method, you need to dive deeper. I agree it is not as easy as pip install, I don't know why it is unavailable on PyPi. Maybe nobody has asked them?

For example Ray Bell writes he uses pixi for those 10% of projects that are very hard to install and uv for the rest.

As you later found out, in our case, we need to install non-Python tools, e.g., node and esbuild. This puts us in the 10% projects, in my opinion. Even if not all templates will need this, it doesn't merit the extra work of doubling the work of having two different workflows.

I'm running on a Jupyterhub inside a Docker container where /home/jovyan/.bashrc is readonly. ... I never have those problems with uv. It just works.

I think this is because UV uses the python environment. Maybe this could be helped by Pixi adding a symlink to ~/.local/bin during the installation for Linux users and/or a PyPI package.

Now I need to start a stronger server because pixi uses too many resources and my pod is killed

This is a big problem! An open issue is here: prefix-dev/pixi#2458, with a well-known repo.

I think the problem is that pixi (with our configuration?) works perfectly for ci/ cd but not for personal development. The question is how it can be configured to also work for personal development?

As I mentioned at the start, it works for personal development. I would be very careful in extrapolating your experience to the general experience, and likewise, I will do the same with my experience.

I would also say it is a new tool and requires some learning. From the look of it, you only use uv's pip interface, not their higher-level API. Reading the documentation could explain some of the problems mentioned.

FWIW, I have opened a PR on HoloViews and I am discussing with @maximlt around adding a dev environment. The environment should be more aligned with the conda/pip workflow, where you should be able to enable it with pixi shell -e dev (still need to update docs).

I would like to hear what @hoxbro thinks since he brought it into the HoloViz repos.

Mostly because our project depends on some non-Python-only packages, and I wanted to have a unified experience across our repos. These non-packages can roughly be split into: NodeJS (Panel, GeoViews), GPU-support (Datashader, HoloViews, hvplot), and Geo-packages (GeoViews - Not as essential anymore).

Hofer-Julian added a commit to prefix-dev/pixi that referenced this issue Nov 25, 2024
We currently give people a one-liner to setup completions.

That is convenient, but also fragile.
If people's startup script doesn't end in a newline, it
[breaks](panel-extensions/copier-template-panel-extension#4 (comment)).
If people run it multiple times it will be duplicated.

Let's follow https://starship.rs and give users instructions to add it
themselves
jjjermiah pushed a commit to jjjermiah/pixi that referenced this issue Nov 30, 2024
We currently give people a one-liner to setup completions.

That is convenient, but also fragile.
If people's startup script doesn't end in a newline, it
[breaks](panel-extensions/copier-template-panel-extension#4 (comment)).
If people run it multiple times it will be duplicated.

Let's follow https://starship.rs and give users instructions to add it
themselves
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants