Parser error on Asynchronous iteration and Asynchronous generators

[adams85]

The parser has issues with the aforementioned ES2018 features:

• Asynchronous iteration works only when ParserOptions.Tolerant is set to false, otherwise it results in ParserException. Sample input:

  for await (const x of syncToAsyncIterable(['a', 'b'])) {
    console.log(x);
  }

  results in error as expected but is parsed fine in async contexts:

   async function f() {
     for await (const x of syncToAsyncIterable(['a', 'b'])) {
       console.log(x);
     }
   }

• Asynchronous generators results in in ParserException regardless the ParserOptions.Tolerant setting. Sample input:

  async function* asyncGen() {
    yield someValue;
  }

Library version: 2.0.2

[lahma]

I think this is a problem also in the original Esprima JS project so maybe need to file an issue there.

[adams85]

@lahma You mean it has to be fixed in the original Esprima first? That'd be a bummer because this is kinda blocking for my lib...

BTW, have you seen this comment? Could you re-open that issue? (Unfortunately I can't.)

[lahma]

@lahma You mean it has to be fixed in the original Esprima first? That'd be a bummer because this is kinda blocking for my lib...

Can be fixed on either side first, just a good practice to get the fix ported also to the other side to keep code consistent between the implementations.

BTW, have you seen this comment? Could you re-open that issue? (Unfortunately I can't.)

Would you like to submit a PR to fix remaining issue(s)?

(sorry I fat-fingered and edited your comment first)

[adams85]

@lahma I can take the other issue (the minor strict mode bug) but I'm not familiar enough with the parser to tackle this one.

[lahma]

Seems that there's already a work item on Esprima side.

[lahma]

By a cursory look it's enough to change:

var isGenerator = isAsync ? false : Match("*");

to

var isGenerator = Match("*");

in two places. Breaks two (now invalid) test cases and lacks test cases and JSON trees for results.

Edit: requires some more changes for async generator methods.

[adams85]

@lahma Now this is the only blocking issue left for me. Are you working on this? If not, I might give it a shot.

[lahma]

I have some work done, it should be pretty close to be PR ready against the JS version. Got to get my thoughts back together and check where I left the work..

[adams85]

Great news!

BTW, in the meantime I realized that I was mistaken about the Asynchronous iterations bug. It's parsed alright in async contexts, just the error message is not too helpful. I'll update the issue description.

However, I noticed that the parser skates over invalid usages of await in the case of for and for...in loops even when tolerant parsing is switched off:

(async () => { 
  for await (;;) { }
})()

is accepted without an error.

I've been making some progress on this though, I'll submit it as a PR soon.

[lahma]

Here's the draft PR on C# side, I'm working on both JS and C# to see how the implementation works: #190

[sebastienros]

It's not even implemented on esprima's side?

[lahma]

Hey welcome back @sebastienros ! Not yet implemented, I'm working on both sides to see what is missing: jquery/esprima#2101

[sebastienros]

That's awesome!

[lahma]

I've completed the changes on JS side and also have the relevant code on .NET side ready (actually developed on .NET sided as debugging is waaaay easier). Waiting for acceptance there.

[adams85]

@lahma May I expect a 2.x release after these changes get accepted and merged? I'd like to know that to be able to decide whether or not to stall releasing one of my libs depending on Esprima.NET.

[lahma]

I'd be happy to have a 2.x release with these fixes that have landed and including this too (unless @sebastienros objects). If we don't hear back from JS side for a while we can also consider merging beforehand and fixing later if needed.

[adams85]

Excellent, thank you!

[lahma]

I've merged in the changes as the JS PR seems to be in standstill. Your three examples provided in summary now parse without errors. Let's iterate more later if needed...

[lahma]

And the JS PR was also just merged.