crates.io: Package Groups

[reem]

Motivation

We want to support "special" or "blessed" packages and would also like to limit the contention for important names in a flat namespace.

Overview

Cargo will support groups as package namespaces with special permissions. Any user can create a new group whose name is not taken and begin adding packages and owners to it. Packages in a group can be referenced in the [dependencies] section and on crates.io as <group>/<package>. Only owners of a group can add new packages to the group or update existing packages.

Details

New cargo commands:

cargo group create <name>

Creates a new blank group with the passed-in name, make yourself an owner, and push it.

cargo group edit <name>

Download and open the group's configuration as a toml file and edit it with $EDITOR. This includes description, members, etc. Only owners of a group may run this command.

Adding a package to a group is as easy as adding group = "<name>" to the [package] section of that package's Cargo.toml and running cargo publish. Permission to do this action is checked during cargo publish.

Expressing a dependency on a package within a group is the same as with a normal package, except the package is referred to as <group>/<package> in Cargo.toml. Notably, the name of the package does not change, and you still access the crate by doing extern crate <package>. This has the potential to cause some naming conflicts, but cargo could provide a solution to this by adding optional alias metadata to a dependency that tells cargo what name it should use when linking that dependency.

Each group gets its own page on crates.io that shows all the packages within it, and all packages in a group share owners.

Group configuration

cargo group edit <name> brings up the configuration file for a group, which contains the following structure in toml:

Necessary:

name
owners

Optional:

description
homepage

More fields could be added if/when they are needed.

[ioquatix]

Have you considered making "group" to be a full blown domain name?

[mangecoeur]

@ioquatix Like in Java? Please god no!

[ioquatix]

@mangecoeur Perhaps you can provide some actual logical reasons?

If you use domain names you have a pre-existing system for name dispute resolution - i.e. domain registries. It isn't completely stupid IMHO.

[reem]

I think domains would be a bad idea, since it significantly raises the
barrier to entry and vastly inflates the length of group names.

Forcing users to pay someone to create a project group makes this idea a
non-starter in my book.
On Tue, Nov 25, 2014 at 4:07 AM Samuel Williams [email protected]
wrote:

@mangecoeur https://github.com/mangecoeur Perhaps you can provide some
actual logical reasons?

If you use domain names you have a pre-existing system for name dispute
resolution - i.e. domain registries. It isn't completely stupid IMHO.

—
Reply to this email directly or view it on GitHub
#975 (comment).

[mangecoeur]

@ioquatix so the java habit is particularly bad (making endless sub-folders for each part of the domain) - but let's say you at least avoid that problem, you then have the case where you are forced to associate a group with a domain - so the group name is defined by the original group ownership/availability of domain (which may be taken up by something totally unrelated)/willingness of someone to pay for it. If the project changes hands/gets a new website you then either break everyone's stuff by updating the domain to match or let the group name and group website be out of sync (this has happened to several java libs I've used).
You also force a bunch of unnecessary textual noise into every name (many places have composite tlds like .co.jp or co.uk which just looks messy). Finally, you don't really solve the name dispute issue, since you can have technically have foo.co.uk and foo.ac.uk belong to totally different people with different projects and this would be acceptable, but the situation is still confusing for users - so you would probably want a naming guideline anyway where you ask people to honor existing projects by avoiding using too similar names.

[arthurprs]

Can't we just make group = username?

[ioquatix]

Groups make sense in many ways, except they are still a limited commodity - who owns the "HTML" group? Who owns the "OpenGL" group? The name contention problem still exists, it's still an essentially flat namespace.

By prefixing packages with a group name, e.g. "html-client", "html-server", you'd achieve almost exactly the same thing. If you need to have additional metadata, have a top level package "html" with that if really necessary.

[pwoolcoc]

@arthurprs +1

[ghost]

@arthurprs +1 - Something like so: username -> package -> files; use ecogiko::math::Float;

[pwoolcoc]

@ecogiko I agree with the part of the proposal that would leave off the username/groupname in use declarations, I would rather put ecogiko/math in my Cargo.toml dependencies, but still use math::Float in my code. It just seems that adding a group name, when a package is already associated with a username, is unnecessary.

[arthurprs]

@ecogiko please no. The crate group (or username if we go that way) shouldn't affect the crate code namespaces.

[blaenk]

I agree that if we have namespacing, then the groups mentioned here should just be the usernames. If an "organization" wants to create a namespace they could just register the username, like "iron" could be one, this way packages don't clash by default.

After using Clojure, I'm pretty accustomed to using username/project to reference a dependency. The nice thing is that if the username and the project are the same, then it can be used by itself, e.g. iron/iron would just be iron. In my opinion, using username/project in Cargo.toml is a small price to pay to avoid name clashing and incentivizing name squatting etc. It's claimed that this should not be an issue because it supposedly isn't in Ruby et al, but I anticipate that Rust may be much more diversely used (not just primarily the realm of web-apps), and it would be a huge shame if many years later the community is pretty much forced to adopt a de facto pseudo-namespacing technique like user_project

[pythonesque]

+1 to username for namespace. I don't find counterarguments compelling and it drastically reduces the name contention problem (which I do find to be a problem with Rubygems).

[Nercury]

Wait, a username? I think simply an additional namespace would be enough. With a similar uniqueness resolution as package name.

How someone else solved the problem (HUGE repository): https://packagist.org/about

[blaenk]

Sure, but "additional namespace" is overly general. The OP proposal (group)
and username both qualify as an additional namespace, so we're debating
over which may be more practical. Choosing the username to serve this
purpose is not unlike github, and has the advantage of already existing (it
can be taken from github, since the system is already based on github).

On Tuesday, November 25, 2014, Nerijus Arlauskas [email protected]
wrote:

Wait, a username? I think simply an additional namespace would be enough.
With a similar uniqueness resolution as package name.

How someone else solved the problem (HUGE repository):
https://packagist.org/about

—
Reply to this email directly or view it on GitHub
#975 (comment).

• Jorge Israel Peña

[arthurprs]

The username is a safe and meaningfull namespace, we don't even need an additional attribute in the toml.

[tomjakubowski]

I think Clojure and its tooling take the right approach for this sort of thing. Leiningen, the project orchestration tool equivalent to Cargo, uses Maven for dependency resolution and specification, so it has the same concepts of "groups" and "packages". But conventionally, for Clojure projects hosted on Clojars anyway, the canonical group name is just the name of the overall project. Users also get a personal namespace for forks, throwaway experiments, alpha versions, and the like under org.clojars.username. As an example, my Clojure library weasel is in the group weasel, and if I were to release plugins or extensions for it I would consider placing them in the weasel group with it.

To specify a dependency in project.clj, you'd write something like [group-name/package-name "x.y.z"], but for the common case of a group and package name being the same you can just write [package-name "x.y.z"]. So Leiningen users who wish to depend on weasel can just write [weasel "0.4.2"] instead of spelling out both the group and package names.

More information here: https://github.com/ato/clojars-web/wiki/Groups

[tomjakubowski]

It's also important when we consider and dread the conventional use of reverse domain names in Java that the deep src/org/example/foo/bar/baz directories for package sources are orthogonal to Maven's group names. Just because they both use RDNS doesn't mean they're related.

[Nercury]

If I wanted another group, would I have to create another username? For example, a community project.

[arthurprs]

You don't get groups, you get your username (or organization name) as a namespace.

[alexcrichton]

I commented in rust-lang/crates.io#58 as well, but I just want to reiterate that the policy here is under development. We're going to be discussing this at the work week next week and I'll be sure to keep everyone here posted with what we're thinking as well.

[ghost]

I like the idea of having some sort of namespace or group and also like the idea of usernames for that. Not so sure about having "blessed" groups/packages though. Do we really need that if we have a good way to gauge package popularity and similar statistics? Who decides which things are blessed? Would that mean those packages have some sort of official status or is it just something more like "recommended"?

[reem]

By "blessed" I meant packages that are the "official" packages for a group, such as iron's router.

[ghost]

Ah, okay. That sounds like a good idea then. I was thinking about something closer to the way the haskell platform (e.g., "blessed packages") works for hackage which I'm not a big fan of.

[liigo]

Uses username for namespace is not a good idea. An username of github.com? or an username of crates.io? or any else? Not all of the crates host in github.

I prefer uses URL as namespace:
github.com/liigo/project
bitbucket.org/liigo/pkg
code.google.com/p/foo
code.csdn.net/liigo/bar
You're not required to register an unique global username all over the world. Url is always unique.

[blaenk]

crates.io currently logs you in using your github identity, so yes, from github.

[steveklabnik]

The team has discussed these issues at some length and we've written up a more formal explanation of our package group policy at http://discuss.rust-lang.org/t/crates-io-package-policies/1041. Further discussion should happen there.

[sergey-shandar]

@steveklabnik the link http://discuss.rust-lang.org/t/crates-io-package-policies/1041 is broken.

[steveklabnik]

It's https://internals.rust-lang.org/t/crates-io-package-policies/1041 now :)