crates.io clone --filter-tree=0, 9sec instead of 108sec ?

[soloturn]

would it be possible to do partial clone for the crates index @cuviper @alexcrichton ? no blobs are downloaded, and the builds usually do not need most of them.

❯ time git clone --filter=tree:0 --bare https://github.com/rust-lang/crates.io-index 2
Cloning into bare repository '2'...
remote: Enumerating objects: 74170, done.
remote: Counting objects: 100% (241/241), done.
remote: Compressing objects: 100% (238/238), done.
remote: Total 74170 (delta 3), reused 241 (delta 3), pack-reused 73929
Receiving objects: 100% (74170/74170), 11.83 MiB | 3.12 MiB/s, done.
Resolving deltas: 100% (159/159), done.

________________________________________________________
Executed in    9.70 secs    fish           external
   usr time    1.81 secs    0.00 millis    1.81 secs
   sys time    0.37 secs    1.58 millis    0.37 secs

❯ time git clone --bare https://github.com/rust-lang/crates.io-index 3
Cloning into bare repository '3'...
remote: Enumerating objects: 499358, done.
remote: Counting objects: 100% (1187/1187), done.
remote: Compressing objects: 100% (599/599), done.
remote: Total 499358 (delta 665), reused 1086 (delta 564), pack-reused 498171
Receiving objects: 100% (499358/499358), 286.00 MiB | 3.88 MiB/s, done.
Resolving deltas: 100% (347226/347226), done.

________________________________________________________
Executed in  108.35 secs    fish           external
   usr time  112.60 secs   50.01 millis  112.55 secs
   sys time   10.07 secs   13.68 millis   10.05 secs

to get a file one possibility would be:

❯ git -C 2 show HEAD:se/rd/serde

i saw you guys commented / implemented on rust-lang/cargo#4015.

[Turbo87]

https://rust-lang.github.io/rfcs/2789-sparse-index.html should resolve the need for this :)

[soloturn]

wow, it got already merged as well:
rust-lang/cargo#9069

CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse whatever-command

makes it already snappy :)

[Eh2406]

We would also need support for filter-tree in libgit2 or in gitoxide (when we use it).