Proposal: Add scores to Go Modules in GoCenter

[retgits]

Proposal: Add scores to Go Modules in GoCenter

Last update: 2019-06-22

Introduction

Starting with two quotes from Russ Cox as he explained his thoughts on software dependencies:

For decades, discussion of software reuse was far more common than actual software reuse. Today, the situation is reversed: developers reuse software written by others every day, in the form of software dependencies, and the situation goes mostly unexamined.

(...)

Software dependencies carry with them serious risks that are too often overlooked.

The difficulty is figuring out which criteria are important to modules and how well those criteria are taken care of in a specific Go module. I propose to add a module score to GoCenter based on the criteria the community deems important calculated in using a formula that is open to discussion and updates as both Go evolves and requirements on modules evolve.

Benefits to the Go community

Adding a score based on predetermined criteria will benefit developers in seeing how well a module is perceived and enables authors of modules to identify possible areas of improvement.

Determining the score

The score will be calculated by GoCenter once per week, with the ability on the UI to trigger an update. The criteria that will make up the score are:

Criterium	Why	Scale
Number of dependents	A dependency which is highly popular is usually a good indicator for having good quality	5 dependent projects equals 1 point
Average score of direct dependencies	High quality nodules will use high quality dependencies themselves	Average score of dependencies
Number of GitHub stars	While GitHub stars are considered vanity metrics, they do give some indication into the popularity of the module	100 stars equals 1 point
Number of downloads	Modules that get downloaded frequently and are thus trusted and used by others, means the module is high quality	100 downloads equals 1 point
Deleted versions	While modules will remain in GoCenter, not having the version available in the version control system is an indication of untrustworthy modules	0 points if versions exist in GoCenter but not in the version control system, otherwise 5 points
Deleted module	While modules will remain in GoCenter, not having the project available in the version control system is an indication of untrustworthy modules	5 points if the project exists, total score will automatically be 0 if the project is deleted
Documentation	Knowing how to use a module is important	5 points when a README exists, 0 points if none exists
Maintainers	Having active maintainers for a module is important to make sure updates can be done	5 points for a number of maintainers higher than 1, 0 points otherwise
Linting score	Properly formatted and linted code is important for the readability and thus quality of the code	Similar to Go ReportCard points will be awarded depending on the scored percentage. 5 points -> score of 97% or higher, 4 points -> score between 81% and 96%, 3 points -> score between 61% and 80%, 2 points -> score between 50% and 60%, 1 point -> score between 31% and 50%, 0 points otherwise
Activity	Projects with a high number of commits, and active development are likely to be better quality modules	5 points when commits are made in the last two weeks, one point is deducted for every two weeks without a commit
Releases	Projects that actively release new versions are likely to be better quality	5 points when releases are made in the last two months, one point is deducted for every two months without a release
Tests	Successful and stable projects have tests and have a high test coverage	5 points are awarded to a test coverage score of over 80%, one point is deducted for every 10% less than that

The score of the module is determined by grading it against a normal distribution of all scores modules that GoCenter knows about.

Percentile range	Score
96th - 100th percentile	A+
86th - 95th percentile	A
61sh - 85th percentile	B
31st - 60th percentile	C
0th - 30th percentile	D

Community feedback

At JFrog's user conference, we've had a board with some of these criteria listed (see image below) and the feedback we got was:

• Average score of direct dependencies: 7 upvotes, 1 downvote
• Number of GitHub stars: 7 upvotes
• Number of downloads: 8 upvotes, 1 downvote
• Deleted versions (tags): 1 upvote
• Documentation: 17 upvotes, 3 downvotes
• Maintainers: 2 upvotes
• Linting score: 1 upvote, 2 downvotes

[klasyap]

Based on the Go community's feedback, we are going to incorporate module scores for all Go modules in GoCenter. You can view the proposal here. We invite the Go community to comment and/or submit a pull request against the scoring formula.