EPIC: Support running on macOS

[weiji14]

To allow Clay to run on not just Linux operating systems, but OSX as well!

This comes from user feedback during DevSeed's Team Week, where we found out that the folks developing the Clay model are predominantly Linux-based, but users were using other operating systems like macOS.

Non-OS specific:

• Document how to install mamba (using brew on OSX, or add non-OS specific instructions)
• Extra steps to handle mamba commands on zsh
• Create a docker image for Clay (@leothomas) Add a docker container (and docker-compose file) to run the model/notebooks in a containerize environment #166

OSX (Intel chips):

• Solve installation on OSX by removing cuda part which is Linux specific (@chuckwondo)
• Get CI running on OSX 10.14+
• Mention minimum version of pytorch needed is 2.1?

OSX-ARM64 (M1/M2/M3 chips):

• Get CI running on macos-14 (Add osx-arm64 platform to conda-lock.yml file and GitHub Actions CI #164) Figure out why torchdata~=0.7.1 and torchvision~=0.16.1 cannot be installed on osx-arm64, maybe Rebuild for increased compatibility and build for python 3.12 conda-forge/torchvision-feedstock#89 will help?

[mikeskaug]

I would be happy to help with this if there is anything I can do. A few weeks ago I started working on getting things set up and running on my Mac. I was interested in experimenting with the project and my typical approach is to start local. My first goal was to get to the point where I could train and predict on a small data set and then move on from there. I haven't finished but I would be happy to share my approach or issues I've encountered.

I've made a few minor changes but the most significant departure has been to use pip instead of mamba to install requirements. I don't know if that's useful or just a bad idea, but I thought it would be worth a try and it seems to work. (Although I guess I'm not 100% sure until I have the various cli commands running, which currently seem to be failing because of data type support on cpu.)

[weiji14]

Hi @mikeskaug, thanks for offering to help! If you can post some of the error messages you're getting (open a bug report issue), that would be great for us to keep track. There's been lots of dtype related bugs (e.g. #134) due to different operating systems and GPU/CPU devices being used, and we'd like to iron those out as much as possible!

If you're keen to contribute, we're open to help with improving our documentation, or you can also jump in and review some of our outstanding PRs (e.g. #164 and #166 are macOS related ones). Or if you have a use-case/dataset you'd like to share and collaborate on, we can discuss more too at https://github.com/Clay-foundation/model/discussions/categories/ideas 😄

[ritwikvashistha]

Hi, I tried using the model on my Mac M1 Pro as well as on Binder. But I ran into the same issue with both of them. When I was trying to install locally, I got an error "The following package could not be------installed: pytorch ~=2.1.0 cuda12 does not exist (perhaps a typo or a missing channel)." With Binder, I got the same error as it was loading. I would appreciate any suggestions on how to resolve this issue.

[weiji14]

"The following package could not be------installed: pytorch ~=2.1.0 cuda12 does not exist

Ah yes, that's because Mac M1 devices don't have CUDA. You'll need to install from the lock-file instead from #164 which adds support for ARM OSX devices. Gimme a few minutes to merge that PR, and then if you do a git pull on the main branch, you should be able to install with:

mamba create --name claymodel python=3.11 conda-lock=2.5.1
conda-lock install --mamba --name claymodel conda-lock.yml

Which reminds me, I should update the installation instructions on that branch before merging.

[weiji14]

@ritwikvashistha, I've merged #164 now, so installation macOS ARM devices should work 🤞. Can you try following the conda-lock installation instructions I mentioned above (or at https://clay-foundation.github.io/model/installation.html#advanced) and see if it installs on your device? Let us know if you still get any errors!

[ritwikvashistha]

I followed the steps you mentioned and got the following error after the conda-lock line:

"(claymodel) ritwikvashistha@ritwiks-mbp model % conda-lock install --micromamba --name claymodel conda-lock.yml
Traceback (most recent call last):
File "/Users/ritwikvashistha/micromamba/envs/claymodel/bin/conda-lock", line 10, in
sys.exit(main())
^^^^^^
File "/Users/ritwikvashistha/micromamba/envs/claymodel/lib/python3.11/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ritwikvashistha/micromamba/envs/claymodel/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/Users/ritwikvashistha/micromamba/envs/claymodel/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ritwikvashistha/micromamba/envs/claymodel/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ritwikvashistha/micromamba/envs/claymodel/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ritwikvashistha/micromamba/envs/claymodel/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ritwikvashistha/micromamba/envs/claymodel/lib/python3.11/site-packages/conda_lock/conda_lock.py", line 1483, in click_install
install(
File "/Users/ritwikvashistha/micromamba/envs/claymodel/lib/python3.11/site-packages/conda_lock/conda_lock.py", line 1518, in install
_conda_exe = determine_conda_executable(conda, mamba=mamba, micromamba=micromamba)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/ritwikvashistha/micromamba/envs/claymodel/lib/python3.11/site-packages/conda_lock/invoke_conda.py", line 59, in determine_conda_executable
if determine_micromamba_version(str(candidate)) < LooseVersion("0.17"):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'Version' and 'LooseVersion' "

[weiji14]

Error message looks the same as conda/conda-lock#596, looks like an upgrade to conda-lock>=2.5.3 is needed.

[weiji14]

Ok, trying to fix that TypeError: '<' not supported between instances of 'Version' and 'LooseVersion' " in #183. Could you try:

mamba env remove -n claymodel
mamba create --name claymodel python=3.11 conda-lock=2.5.6
conda-lock install --mamba --name claymodel conda-lock.yml

Note the new version of conda-lock=2.5.6 is used.

[ritwikvashistha]

I just did that, and I didn't get any errors. But it seems like the dependencies didn't get downloaded since, on trying to run some basic codes, I am getting errors related to missing modules. I think I will just download all those missing modules manually since they don't seem like a lot, but I just wanted to mention the issue for reference.

[weiji14]

I am getting errors related to missing modules. I think I will just download all those missing modules manually since they don't seem like a lot, but I just wanted to mention the issue for reference.

Could you file a separate bug report (https://github.com/Clay-foundation/model/issues/new) for those missing modules? Would be good for us to keep track of what dependencies are missing and needs to be added to environment.yml/conda-lock.yml.

[chuckwondo]

Ok, trying to fix that TypeError: '<' not supported between instances of 'Version' and 'LooseVersion' " in #183. Could you try:

mamba env remove -n claymodel
mamba create --name claymodel python=3.11 conda-lock=2.5.6
conda-lock install --mamba --name claymodel conda-lock.yml

Note the new version of conda-lock=2.5.6 is used.

Note that you need to activate the env before running conda-lock, so the steps should be the following:

mamba env remove -n claymodel
mamba create --name claymodel python=3.11 conda-lock=2.5.6
mamba activate claymodel
conda-lock install --mamba --name claymodel conda-lock.yml

[ritwikvashistha]

It works now, and I just tried the 'Burn scar analysis using embeddings from partial inputs' tutorial. I was able to reproduce it. I will continue experimenting with it to get embeddings from partial inputs. Thanks, @chuckwondo and @weiji14 for the help and the promptness in your replies!