How do we add new packages?

[ivirshup]

We should have a document describing how we add new packages.

This should include (at least):

• How are packages selected
• Criteria for inclusion
• How is this decision made

[ivirshup]

My first proposal here would be:

Selection

The core team chooses packages to include. If we don't already own them, we ask the owners if they'd like to join

Criteria for inclusion

I'm less sure on this. I currently see two broad options

Nothing too formal

Packages in here are ones that the core members say they maintain. No specific criteria.

This would allow more experimental packages to live here.

Formal

Some more formal requirements:

• Scope: the package fall into one of these categories
  • "core tool", e.g. provides fundamental capabilities and is likely (or is) a dependency of other project
  • Infrastructure for core tools (e.g. docs, test data, ci tooling, websites, etc.)
• Code quality – must have gone through a code review
• Dev practices
  • Should conform to standard practices we define in the infrastructure working group. This not only provides some standard of quality (e.g. amount of test coverage), but also makes it easier for existing maintainers if broad workflows are similar.
  • Should have CI
• Defined maintainers
  • Possibly a requirement that they be official members of the org

How is this decision made

Vote of the core team

[Zethson]

Although I consider myself the most pragmatic and rather "move fast and break things quickly" guy among us I'd vote for your more formal approach. We have to have some quality standard and the one that you listed are reasonable. There's only one exception imo:

Code quality – must have gone through a code review

A code review of a full package is a monumental task and one that no one of us has time for. I'd much rather keep this subjective. If all of us with a couple of glances agree that the code is of sufficient quality it should be good enough. We need to trust the maintainers that we bring in anyways. If we don't we should not have added their package or the people behind them in the first place :)

[ivirshup]

A code review of a full package is a monumental task ... I'd much rather keep this subjective.

For sure. I would like to have some rough guidelines/ a process set-out though. Maybe things like:

• Some set of current maintainers must have had a meeting about it, and decided to pass it
• Check that core functionality (stuff in the main tutorial) is meaningfully tested, not just covered

There are a couple of main benefits I think we can get here:

This is an opportunity to fix up code whose maintenance would otherwise get deferred. If there's something in the core functionality/ doc builds that's really difficult to figure out, I think this review process could provide the impetus to get these things fixed. If we're promising to maintain something it would be good to make sure it's in a maintainable state.

Second, I think it's good chance to get some collaboration in. I think it would pay off to have some forced collaboration time early on, so the package owner and scverse team have some experience working with each other. Plus good chance to find any red-flags, and make sure new maintainers are up for maintenance.

Could be interesting to look through what a typical bioconductor package review is like.

[ivirshup]

Where do we document this?

[giovp]

agree with what you mentioned before, I think minimum req are:

• tests and CI
• docs
• some form of pre-commits checks

I would document this in website.

[grst]

This would allow more experimental packages to live here.

Some communities have a separate "incubator" namespace for packages that are experimental, or not yet up to the coding standards:

• Apache
• Conda

[gtca]

By the way, we might want to discuss if we should include a small chapter on package exclusion. I would expect it not to have strong statements but might be good to outline that we potentially archive superseded and unmaintained projects as well as projects that diverge from scverse Principles.

[giovp]

gdoc to draft and discuss here:
https://docs.google.com/document/d/1u_OJSX9Vfvx3xe_EDeSc4Ol4C3tzBgm3fnDlNP3gj_k/edit?usp=sharing

[ivirshup]

Paraphrasing some brainstorming from a governance meeting where this came up:

• Goal of process: ensure packages included as core continue to be maintained
• Code owners (e.g. lab) must be aligned, not one off python project
• Community participation of developers (e.g. active on forums, attends meetings)
• Is there a development plan moving forward? E.g. multiple developers, funding?
• Reasonably maintainable
• Important enough to maintain