Automatic discovery, SWIG wrappers, and pyproject toml

[stephematician]

Summary

I want to include a SWIG-generated wrapper module called cext (which wraps the internal _cext library that SWIG generates for me, too).

When I build the project I see that, although cext.py is generated, it is not added to the dist or wheel:

user@host:~/<project_folder>$ python -m build
...
swigging src/foo_module/cext.i to src/foo_module/cext_wrap.c
swig -python -I./src/foo_module/include -o src/foo_module/cext_wrap.c src/foo_module/cext.i
creating build/temp.linux-x86_64-cpython-312/src/foo_module
...
adding 'foo_module/__init__.py'
adding 'foo_module/_cext.cpython-312-x86_64-linux-gnu.so'
adding 'foo_module/cext.i'
adding 'foo_module/include/foo_module/foo_module.h'
adding 'foo_project-0.0.1.dev0.dist-info/METADATA'
adding 'foo_project-0.0.1.dev0.dist-info/WHEEL'
adding 'foo_project-0.0.1.dev0.dist-info/top_level.txt'
adding 'foo_project-0.0.1.dev0.dist-info/RECORD'
...

So when I try to import (obviously):

import foo_module.cext
# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
# ModuleNotFoundError: No module named 'foo_module.cext'
import foo_module._cext
# OK - but only internal module, doesn't have Python wrappers

What's the simplest (and most future-proof) way to get the desired result that cext.py is included?

Details of MWE

1. Broadly speaking I'm using src-layout for package discovery.
2. I have an "include" folder that will contain headers to be distributed with the package (similar to how Numpy might share headers).

<project_folder>
├── MANIFEST.in
├── pyproject.toml
└── src/
    └── foo_module/
        ├── __init__.py
        ├── cext.i
        └── include/
            └── foo_module/
                └── foo_module.h

A MANIFEST.in is required to ensure the headers are packaged:

recursive-include src/foo_module/include *.h

Here's how I think the pyproject.toml would look:

[project]
name = "foo_project"
version = "0.0.1.dev0"

[build-system]
requires = ["setuptools"]

[[tool.setuptools.ext-modules]]
name = "foo_module._cext"
sources = ["src/foo_module/cext.i"]
swig-opts = ["-I./src/foo_module/include"]
include-dirs = ["src/foo_module/include"]

[tool.setuptools.packages.find]
where = ["src"]

The contents of cext.i are:

%module cext
%{
#include "foo_module/foo_module.h"
%}

%include "foo_module/foo_module.h"

So, basically, I want to wrap all the structures that I declare in foo_module.h and a few enums, and so forth. But for a minimal example, can add anything to foo_module.h that we like, e.g.

/* foo_module.h */
#define bar 0

[stephematician]

See also: https://stackoverflow.com/questions/50239473/building-a-module-with-setuptools-and-swig

[stephematician]

My guess is that, for now, I can switch the ordering of the build_ext and build_py commands as per the stackoverflow example, and then use a custom build backend that wraps build_meta as in: https://setuptools.pypa.io/en/latest/build_meta.html

[stephematician]

For now, adding the following setup.py is the simplest solution I could find (no need to modify the build 'backend' or make other changes to pyproject.toml or MANIFEST.in)

from setuptools import setup
from setuptools.command.build_py import build_py as _build_py    

class build_py(_build_py):
    def run(self):
        self.run_command("build_ext")
        return super().run()

setup(cmdclass = {'build_py' : build_py})