feat(assist): import organizer revamping

[Conaclos]

Summary

This long awaited feature is finally here! It is one of the most difficult features I have implemented. It took me one failed attempt and many hours. I am very happy with the result.

This PR implements #3177 with some divergences:

• Import merging is not implemented yet.
  This is left to a future PR.
• Import groups support glob exceptions.
  This drastically simplifies import matching and makes it more powerful.

This new import organizer provides the following features:

• It orders named specifiers using a natural order

  - import { A10, A9 } from "";
  + import { A9, A10 } from "";
  
  - export { A10, A9 } from "";
  + export { A9, A10 } from "";

• It orders import attributes according to their keys using a natural order

  - import {} from "" with { "k10": 0, "k9": 0 };
  + import {} from "" with { "k9": 0, "k10": 0 };
  
  - export {} from "" with { "k10": 0, "k9": 0 };
  + export {} from "" with { "k9": 0, "k10": 0 };

• Imports and exports are separated into chunks before to be sorted.
  A new chunk starts every time we met a side effect (bare) import, a distinct statement kind, or a detached comment.
  A detached comment is separated by a blank line of the import/export.

  // Chunk 1
  import { A } from "a";
  
  import { B } from "b";
  // attached comment
  import { C } from "c";
  import "side-ffect"
  // Chunk 2
  import { D } from "d";
  
  // Chunk 3 (detached comment)
  
  import { E } from "e";
  
  // Chunk 4
  export { F } from "f";
  export { G } from "g";
  
  function f() {}
  
  // Chunk 5
  export { FH } from "h";

• Chunk of imports or exports are first sorted according to the group they belong to.
  By default, there is a single group: the implicit group.
  Users can define their own groups and define an order between them.
  The implicit group always comes last.
  Groups are either predefined (like :BUN: and :NODE:) or globs with exceptions.

• Imports of a group are sorted according to other parameters, notably their import source.
  See the RFC and the implementation for more details.

• The implementation takes care of removing and adding newlines where required.
  Chunks are separated by a blank line except for side effect imports when other imports surround them.
  Comments are either attached or detached to an import/export.
  A detached comment (separated by a blank line of the import/export) creates a new chunk and is kept above the chunk.
  I added an exception for the header comment in a file: this comment is never attached. This ensures that Copyright notice are never moved.

Limitation / Possible improvements

• We could improve our glob handling:
  • @scope/lib doesn't match the glob @scope/lib/**.
    This could create some frictions.
  • It is currently not possible to write globs like node:**
    This could be easily fixed.
    As a workaround, users have to match against both node:* and node:*/**
• Trivia handling when sorting named specifiers and attributes should be improved. (Done)
• Some data structures could be shared for sorting distinct named specifiers and import attributes. (Not worth the complexity?)

Implementation strategy

The run procedure checks if everything is organized and register any "issues" that it founds.
An issue can report an unorganized import/export or an unsorted chunk (more precisely unsorted prefix of a chunk).
The action procedure is responsible for fixing the found issues.

This is different of the previous implementation where the entire sorting logic was in run.
This new approach pursuits two objectives:

1. Make the happy path (The imports and exports are already organized) as fast as possible.
2. Try to reduce the work to do for sorting.
   Notably all the chunk logic is in run.
   action doesn't bother about it.

Also, by dividing chunk sorting and individual import/export organizer, we are able to avoid a costly chunk sort when only named specifiers of an import are unsorted.

I think there is room for improving the code. However, it is already a big PR and I would like to ship it before we release Biome 2.0 beta.

Test Plan

I added some tests.

[github-actions]

Parser conformance results on

js/262

Test result	main count	This PR count	Difference
Total	50431	50431	0
Passed	49116	49116	0
Failed	1315	1315	0
Panics	0	0	0
Coverage	97.39%	97.39%	0.00%

jsx/babel

Test result	main count	This PR count	Difference
Total	40	40	0
Passed	37	37	0
Failed	3	3	0
Panics	0	0	0
Coverage	92.50%	92.50%	0.00%

symbols/microsoft

Test result	main count	This PR count	Difference
Total	6645	6645	0
Passed	2229	2229	0
Failed	4416	4416	0
Panics	0	0	0
Coverage	33.54%	33.54%	0.00%

ts/babel

Test result	main count	This PR count	Difference
Total	798	798	0
Passed	706	706	0
Failed	92	92	0
Panics	0	0	0
Coverage	88.47%	88.47%	0.00%

ts/microsoft

Test result	main count	This PR count	Difference
Total	18664	18664	0
Passed	14314	14314	0
Failed	4350	4350	0
Panics	0	0	0
Coverage	76.69%	76.69%	0.00%

[codspeed-hq]

CodSpeed Performance Report

Merging #5364 will not alter performance

_{Comparing conaclos/import-organizer-revamping-2 (5f535ae) with main (772dcf5)}

Summary

✅ 95 untouched benchmarks

[dyc3]

Not a full review, looks great at a glance!

[arendjr]

Nice work! Do you still want to include this into the beta?

[Conaclos]

Nice work! Do you still want to include this into the beta?

Yes. I can add something to the blog post.

[ematipico]

The documentation must be updated to reflect the new options. Which means that:

• we need to document legacy and how it works (probably not needed but it needs to be highlighted)
• we need to document importGroups and provide enough examples to explain the new behaviour

I'm not going to block the PR, but this is something we must do before the beta, if we plan to release it for the beta

[arendjr]

I think we’re getting there, right? 😊

[Conaclos]

I think we’re getting there, right? 😊

I have just committed the blank-line between groups feature.
I think we have enough features for the beta.

I have to document the new system before merging the PR.

[Conaclos]

The documentation is committed. Feel free to review.

[ematipico]

This is an outstanding feature; thank you, @Conaclos! I'm sure a lot of users will be pleased. I hope you're proud of what you just did.

I left various comments around the docs. Many are grammar fixes. Here are some high-level pointers:

• consider a glossary to explain specific terms to place at the beginning
• Let's first discuss our defaults without mentioning custom groups. Then, once all the important information is listed, we can have "custom groups" at the very end. A user needs to know about default groups, blank lines, comments, etc., to create groups.
• Before showing the example, could you explain it and describe its outcome, too? I know the codegen shows a diagnostic, but it's important for accessibility too.

[ematipico]

Suggested change

	/// Chunks also ends as soon as a statement or a side-effect import (also called bare import) is encountered.
	/// The following example contains three chunks.
	///
	/// ```js,ignore
	/// import A from "a";
	/// import "x";
	/// import * as B from "b";
	/// function f() {}
	/// import * as C from "c";
	/// Chunks also end when a statement or a side-effect import (or bare import) is encountered.
	/// The following example contains three chunks.
	///
	/// ```js,ignore
	/// import A from "a";
	/// import "x";
	/// import * as B from "b";
	/// function f() {}
	/// import * as C from "c";

In this example, it isn't how the chunks are divided; I suggest explaining them. For instance, I don't understand if import "x" belongs to a chunk or not, and if it does, to which chunk belongs to.

Also, I suggest adding some exports so we keep the examples consistent.

[ematipico]

Suggested change

	/// The import and export organizer ensures that chunks are separated from each other with a blank lines.
	/// Only side-effect imports adjacent to a chunk are not separated by a blank line.
	/// The action ensures that chunks are separated from each other with blank lines.
	/// Only side-effect imports adjacent to a chunk are not separated by a blank line.

We should start using the word "action". This is an action/rule like all the others.

Also, regarding the second line, maybe we should provide an example.

[ematipico]

We used the word "source" many times , and the explanation comes only at this point of the docs. We should explain it at the very beginning.

Also, I know you decided to use the word "source" because it might be more beginner-friendly, but I wonder if we should at least mention that it is technically called specifier.

[ematipico]

This explanation goes in contrast with the previous explanation. The previous explanation said that a detached comment needs to be surrounded by blank lines. Or maybe the previous explanation, with the example, confused me.

[ematipico]

Suggested change

	/// Note that there is an exception at the rule.
	/// If a comment appears at the top of the file, it is considered as detached even if no blank lien follows.
	/// This ensures that copyright notice and file header comment stay at the top of the file.
	/// However, there is an exception to the rule.
	/// If a comment appears at the top of the file, it is considered detached even if no blank line follows.
	/// This ensures that copyright notice and file header comments stay at the top of the file.

Be careful!! The first example that talked about attached comments had a comment at the top of the file, which is considered an attached comment 😄

[ematipico]

I know that I am reviewing only the docs now, but I wonder if I should narrow down the expectation only to multiline comments. What do you think?