Skip to content

Commit

Permalink
fix: don't handle TOKEN_ERROR as trivial tokens. (#159)
Browse files Browse the repository at this point in the history
* fix: don't handle TOKEN_ERROR as trivial tokens.

Instead handle them and fail during parsing. The parser produces an error which also indicates which token would be expected instead

* add: one more testcase; bare_tilde
  • Loading branch information
hsjobeki authored Dec 16, 2024
1 parent 01e6582 commit b93d6bb
Show file tree
Hide file tree
Showing 11 changed files with 38 additions and 1 deletion.
2 changes: 1 addition & 1 deletion src/kinds.rs
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,6 @@ impl SyntaxKind {
/// Returns true if this token is a comment, whitespace, or similar, and
/// should be skipped over by the parser.
pub fn is_trivia(self) -> bool {
matches!(self, TOKEN_COMMENT | TOKEN_ERROR | TOKEN_WHITESPACE)
matches!(self, TOKEN_COMMENT | TOKEN_WHITESPACE)
}
}
5 changes: 5 additions & 0 deletions test_data/parser/error/path_bare_tilde.expect
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
error: unexpected TOKEN_ERROR at 0..1, wanted any of [TOKEN_L_PAREN, TOKEN_REC, TOKEN_L_BRACE, TOKEN_L_BRACK, TOKEN_STRING_START, TOKEN_IDENT]
[email protected]
[email protected]
[email protected] "~"

1 change: 1 addition & 0 deletions test_data/parser/error/path_bare_tilde.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
~
12 changes: 12 additions & 0 deletions test_data/parser/error/path_interp_trailing_slash.expect
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
error: unexpected token at 8..9
[email protected]
[email protected]
[email protected] "./"
[email protected]
[email protected] "${"
[email protected]
[email protected] "foo"
[email protected] "}"
[email protected]
[email protected] "/"

1 change: 1 addition & 0 deletions test_data/parser/error/path_interp_trailing_slash.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
./${foo}/
5 changes: 5 additions & 0 deletions test_data/parser/error/path_tilde.expect
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
error: unexpected TOKEN_ERROR at 0..2, wanted any of [TOKEN_L_PAREN, TOKEN_REC, TOKEN_L_BRACE, TOKEN_L_BRACK, TOKEN_STRING_START, TOKEN_IDENT]
[email protected]
[email protected]
[email protected] "~p"

1 change: 1 addition & 0 deletions test_data/parser/error/path_tilde.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
~p
5 changes: 5 additions & 0 deletions test_data/parser/error/path_tilde_slash.expect
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
error: unexpected TOKEN_ERROR at 0..2, wanted any of [TOKEN_L_PAREN, TOKEN_REC, TOKEN_L_BRACE, TOKEN_L_BRACK, TOKEN_STRING_START, TOKEN_IDENT]
[email protected]
[email protected]
[email protected] "~/"

1 change: 1 addition & 0 deletions test_data/parser/error/path_tilde_slash.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
~/
5 changes: 5 additions & 0 deletions test_data/parser/error/path_trailing_slash.expect
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
error: unexpected TOKEN_ERROR at 0..3, wanted any of [TOKEN_L_PAREN, TOKEN_REC, TOKEN_L_BRACE, TOKEN_L_BRACK, TOKEN_STRING_START, TOKEN_IDENT]
[email protected]
[email protected]
[email protected] "/a/"

1 change: 1 addition & 0 deletions test_data/parser/error/path_trailing_slash.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/a/

0 comments on commit b93d6bb

Please sign in to comment.