Marketplace getNotifications test timeout

[cuongdo]

I get this error intermittently when running npm run test:jsw and in Travis CI:

  1) Marketplace Resource
       getNotifications
         should return notifications:
     Error: Timeout of 10000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/Users/cdoorigin/dev/origin-js/test/resource_marketplace.test.js)

Could you please take a look @tyleryasaka?

(This can wait until after feature freeze, since we're all pressed for time.)

[cuongdo]

Possibly related intermittent failure:

  2) Marketplace Resource
       setNotification
         should allow notifications to be marked as read:
     Error: Timeout of 10000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/Users/cdoorigin/dev/origin-js/test/resource_marketplace.test.js)

[tyleryasaka]

@cuongdo Yes I was just mentioning this to @micahalcorn . I have been experiencing the same thing. (To reproduce, just leave npm run test:jsw open while triggering multiple runs of the tests by changing files. Each subsequent run of the notification tests will take a little longer, and soon they will cross the 10-second threshold.)

So our some of our tests (at least the notification tests) take longer to run each time when npm run test:jsw is used. This is because

(1) We are not restarting the entire blockchain after each test, combined with
(2) (I think) that we are having to search through more events on each test run, since the blockchain has been running longer and more transactions have occurred. (Most of the data we fetch is from events.)

I'm not sure what the best solution is here, but we should keep this issue open until we find one.

I had a solution for this with our old contracts. It was test/helpers/contract-service-helper. This would (and still does) redeploy the long-running marketplace contracts so that each test can run with a fresh marketplace. This used to essentially act as a reset. Now, since we are depending on searching through events rather than just quering a contract, this no longer works.

[tyleryasaka]

Ok, looks like this is an even bigger problem. TravisCI tests are frequently failing with this issue. Not always, but randomly failing tests are a very bad thing. 😬

[cuongdo]

I haven't seen this locally since merging #486. I'm not sure what that'd be the case though.

[tyleryasaka]

@cuongdo Glad your initial issue seems to be fixed. Unfortunately the tests do take longer after each execution, so the problem I described is still there. 😦