Formalized tests and documentation for the GOST packages

[elbeejay]

Questions for the developers of the GOST packages which seem to be GOSTnets, GOST_Urban, and gostrocks (although the example notebook from GOSTNets_Urban seems to suggest those functions could be bundled up into a package as well).

1. Should software tests (unit tests and integration tests) be developed for these packages?
2. Should documentation of the packages (whether this is extensive to include examples or simply a more readable listing of functions and doc-strings) be created?
3. Should GOSTNets_Urban be turned into a proper Python package?
4. Should gostrocks be migrated to the worldbank GitHub organization?

@g4brielvs I'm tagging you here as you might know the proper folks to contact regarding these questions. I think these changes could be made while addressing the issues related to PEP 571 (worldbank/GOSTurban#7, worldbank/GOSTnets#40, worldbank/GOSTnetsraster#8).

[g4brielvs]

@elbeejay Thanks for summarizing what we talked about on this discussion. The primary users should be @andresfchamorro @bpstewar @bennyistanto (missing anybody?). For the overall package and documentation structure, it could be similar to BlackMarblePy. Using Jupyter Book/Sphinx we can display the docstrings (which can be improved) and create example notebooks. Also, I'd suggest improving the testing and the release cycle as well (it could be an Action).

[andresfchamorro]

I think these are all great initiatives! On 1/2 yes, it would be nice to get some guidance on testing modules. Some of the packages have docstrings but it has been mostly ad-hoc so I support the idea of doing a cleanup and adding documentation where needed.

On 3/4, Ben will have thoughts since he wrote most of the code there. My sense is that they are used often enough by project repos that they should be cleaned up and in the organization's account. There is also a lot of miscellaneous code from old years that might not be relevant.

[elbeejay]

it would be nice to get some guidance on testing modules.

I'd advocate for using pytest in conjunction with coverage to make it easier to track how much of the code bases are being tested. The base Python unittest.mock is pretty good for mocking complex objects or data structures, and would likely make sense for testing functions that take other objects or large datasets (for which there are no simple example inputs) as input parameters. Happy to help set up the structure for a test suite with GitHub Actions etc., or I can even dig in and start writing up some tests, let me know what'd be most helpful @andresfchamorro.

Some of the packages have docstrings but it has been mostly ad-hoc so I support the idea of doing a cleanup and adding documentation where needed.

It would definitely help with both readability of the code as well as the auto-documentation of functions if the doc-strings all followed a consistent and standard format.

[g4brielvs]

As far as style is concerned, I'd favor numpydoc (e.g, BlackMarblePy), but I'm eager to hear what colleagues think.

[rbanick]

Reaching from beyond the grave (hi everyone) I would like to second pytest and coverage in combination. If you want to go one step further I've found nox an elegant framework for packaging up tests, linting, and other CI/CD and allowing you to run them against various Python environments.

While I'm here I'll say I've found numpydoc style the most friendly as well. But that's personal preference.

[g4brielvs]

@elbeejay @bpstewar @andresfchamorro If I may jump in, a good first step would be to migrate the current packages to PEP 621 and create the documentation on GitHub Pages (either as the Template and BlackMarblePy or another framework the team chooses). Then, we can work on targeted issues to improve the packages. For example, we could address worldbank/GOSTnets#41 and add PuLP and other optional dependencies to the pyproject.toml (instead of the current usage).

[elbeejay]

Sure that sounds good to me - can we touch base sometime @g4brielvs about the PEP 571 part of this? I'd like to make sure we get on the same page about what aspects of it you'd like to incorporate - are we going to write custom build_wheel(), build_sdist(), etc. hooks to manage the build process or just adopt their guidance surrounding the formatting of the pyproject.toml file?

[g4brielvs]

Wrong PEP! It should have been pyproject.toml specification. The proposal is to migrate setup.py to pyproject.toml. Using PEP 518/621 aligns our project with the latest standards in the Python packaging ecosystem, ensuring compatibility with modern packaging tools.

[elbeejay]

Ok - that sounds far simpler, so we'd just be switching to pyproject.toml where relevant.