fix: Export samtools wrappers

[msto]

pysam does not currently export the various wrapper functions in the samtools module, resulting in type-checking errors.

e.g.

45: error: Module has no attribute "faidx"  [attr-defined]

I've followed the pattern implemented in the other modules, exporting attributes in a module-specific __all__ declaration and then adding this list to the package's __all__.

I believe this PR addresses the problem, but I'm unable to verify - when I install the updated package locally with pip install -e, mypy cannot find the type stubs for pysam. (As an aside - is there a CONTRIBUTING or other documentation for local install/testing?)

Thanks!

[jmarshall]

Ideally such documentation would be in doc/developer.rst or would be unnecessary, but neither seems to be the case.

Me, I generally build with python3.xx setup.py build and have /path/to/pysam/build/lib.platform-arch-cpython-3xx listed in a $USER_SITE/localcustomize.pth path configuration file. This is a bit low-tech but suffices to run pytest and — from any directory other than the pysam source directory — to import pysam or run mypy.

With this I could verify whether your PR solves the problem. Unfortunately it does not — __all__ seems to be a red herring here, and to be honest I'm not sure we're using it correctly at all in pysam/__init__.py. What does appear to do the trick is instead adding explicit type hints to pysam/__init__.py, which is probably unsurprising in retrospect:

faidx: pysam.utils.PysamDispatcher
# etc

[jmarshall]

Personally I'd like to deprecate having the samtools subcommands in the root pysam namespace. The bcftools subcommands need to be used like this:

import pysam.bcftools
pysam.bcftools.view(…)

And the samtools subcommands can be used similarly:

import pysam.samtools
pysam.samtools.view(…)

(And both of the above are type hinted appropriately.)

Additionally at present the following also work for samtools:

import pysam
pysam.samtools.view(…)
pysam.view(…)

Thus the samtools subcommands are available after plain import pysam and also they are available directly as pysam.xyz(…). The latter is redundant and, especially for pysam.view(…) and other subcommands that exist in both samtools and bcftools, has the appearance of ambiguity. So I would like to deprecate that and eventually remove it. But while it's there it should probably be type hinted appropriately nonetheless.

If we also deprecated having import pysam make pysam.samtools.view(…) et al available, then AIUI plain import pysam would not need to load libcsamtools.so, which would be a small speedup and isolation win. Or that may not be worthwhile, in which case we should have plain import pysam make pysam.bcftools.xyz(…) available too.

[msto]

@tfenne and I are both in favor of that deprecation. 🙂

[jmarshall]

Cheers. In the meantime, I've figured out how to get the typechecking to work — alternative PR incoming soon.

[jmarshall]

Turns out this type checking is determined by pysam/samtools.py's definition of __all__ after all. This file originally defined __all__ unconditionally, but PR #1143 inadvertently removed it and PR #1171 put it back but only for Python ≤3.7.

Your PR puts it back, but mypy still doesn't understand it. I think this has something to do with the complexity of the code there, because PR #1315 fixes it by setting __all__ as simply as possible. 🤷

Thanks for reporting this issue (and trying to fix it). I'd prefer not to add the samtools subcommands to pysam/__init__.py's __all__ as they haven't been listed there previously, so I'll leave that part as the status quo.

Closing as superseded by PR #1315, which you may be interested in looking at.