Include M1 runners in CI and update install instructions accordingly

[niksirbi]

Description

What is this PR

• Bug fix
• Addition of a new feature
• Other

Why is this PR needed?

• Many of our users will use M1/2 Macs
• We want to double-check if simple pip install works on M1/2 Macs (or if we have to revert to conda, see Remove pytables as a conda dependency #104)

References

closes #123

How has this PR been tested?

The real test are the CI checks in this PR.

Is this a breaking change?

No

Does this PR require an update to the documentation?

No

Checklist:

• The code has been tested locally
• Tests have been added to cover all new functionality
• The documentation has been updated to reflect any changes
• The code has been formatted with pre-commit

EDIT 2024-02-26

The CI tests were failing on M1 runners, because the hdf5 binaries could not be found (see discussion below). Therefore, this PR has been amended to also change movement's installation procedure, which should now be:

conda create -n movement-env -c conda-forge python=3.10 hdf5
conda activate movement-env
pip install movement

The above should work cross-platform.
I've renamed the PR from "Include CI tests on macOS 14 M1 runner" to "Include M1 runners in CI and update install instructions accordingly".

[niksirbi]

@neuroinformatics-unit/behaviour I think our fears have been confirmed, installation specifically fails on macos-14 (M1) runners, seemingly due to the HDF5 hack we discussed in #91 and implemented in #104.

The relevant error:

× Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [12 lines of output]
      /var/folders/1k/qq3pcbf12vb6vyblh81736p40000gn/T/H5close02d79slb.c:1:10: fatal error: 'H5public.h' file not found
      #include "H5public.h"
               ^~~~~~~~~~~~
      1 error generated.
      cpuinfo failed, assuming no CPU features: 'flags'
      * Using Python 3.10.11 (v3.10.11:7d4cc5aa85, Apr  4 [202](https://github.com/neuroinformatics-unit/movement/actions/runs/8021671030/job/21915041655?pr=125#step:3:215)3, 19:05:19) [Clang 13.0.0 (clang-1300.0.29.30)]
      * Found cython 3.0.8
      * USE_PKGCONFIG: True
      .. ERROR:: Could not find a local HDF5 installation.
         You may need to explicitly state where your local HDF5 headers and
         library can be found by setting the ``HDF5_DIR`` environment
         variable or by using the ``--hdf5`` command-line option.
      [end of output]

This doesn't occur on Linux, Windows and Intel MacOS (macos-latest aka macos-12) runners.

I'm considering to revert #104.

@willGraham01 if the same hack is also implemented for some BrainGlobe packages, we should also test on M1 runners there.

[willGraham01]

This is a problem with M1 macs in general, not just GitHub runners. A brew-installed version of HDF5 goes into opt/homebrew/opt and not the usual header directory.

brew install hdf5
export HDF5_DIR=/opt/homebrew/opt/hdf5

will allow the HDF5 headers to be found by the C++ compiler.

(Also, not wanting to be too pedantic, but it looks like the source code is #include "H5public.h" rather than #include <H5public.h>, which means the compiler is searching for a local version of H5 - within the source location - rather than the system include directories. This shouldn't affect anything since if the compiler cannot locate the file locally, it should default back to the system directories, but worth double-checking what the intention is here. If HDF5 is being included as a submodule, then ignore me 😅 )

[niksirbi]

This is a problem with M1 macs in general, not just GitHub runners. A brew-installed version of HDF5 goes into opt/homebrew/opt and not the usual header directory.

😞 so we can forget about a one-line installation. So we either instruct people to brew install hdf5 or we go back to the previous installation instructions with conda install pytables.

[willGraham01]

conda install pytables

If we do go back to non-oneline, why not conda install hdf5 instead? I would need to double check the recipe, but I would assume that conda puts the headers in a sensible place, unlike brew.

(Recipe: https://github.com/conda-forge/hdf5-feedstock/blob/d2771a298d3d3efe279422aad7533cefbafae8c1/recipe/build.sh#L4 - looks like it will put the header files wherever the system library directory is, which should be the correct place)

[niksirbi]

If we do go back to non-oneline, why not conda install hdf5 instead? I would need to double check the recipe, but I would assume that conda puts the headers in a sensible place, unlike brew

True, I might try that instead.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

[niksirbi]

Alright, conda install -c conda-forge hdf5 definitely does the trick, and I think we'll go with that for now. Anyway, the goal is to release movement on conda-forge soonish.

[niksirbi]

@sfmig I've tagged you as reviewer because you've also encountered the hdf5 problem on your M2 Mac.

[sfmig]

works for me! ✨

You may want to add a quick check after installation, to make sure everything works (like running Python and importing movement for example?).