Some queries DoS the backend, frontend hangs

[tuukka]

Some of our queries seem to crash the backend, which results in a denial-of-service for all users. Also, in this case the frontend looks like the query is still executing (the spinner keeps turning).

Here's links to some of our problematic queries: tom-james-watson/wikitrivia#26

[hannahbast]

@tuukka @nicolaes Thanks a lot for pointing this out to us! I looked at the query, and it's a combination of three problems:

1. We were making some experiments and temporarily configured QLever to operate with 70 GB of main memory, but on the machine only ~ 50 GB of main memory were available. So any query going to the limits would crash the backend. I have fixed that now. (When QLever crashes it is almost always because it wants to allocate memory that the machine does not have, and this, in turn, can only happen if QLever is wrongly configured.)

2. The query https://qlever.cs.uni-freiburg.de/wikidata/aFFkcp involves several very large predicates and currently uses a lot of main memory. It also outputs an unusually large number of columns. We will investigate and try to figure out how to improve performance. We are anyways currently working on an issue related to this. Anyway, thanks for pointing this out to us, we are very eager to learn about hard queries that QLever cannot process although it should, in principle, be able to.

3. The QLever UI normally shows a proper error message when QLever aborts the query processing because of insufficient memory. I am not sure why it does not in this case, we will investigate.

[tuukka]

Thank you for the quick reply and help!

Re 1: Could you additionally configure the backend service so that it automatically restarts after a crash?

Re 2: Here's another one of the queries that crashed - is it the same issue or something else causing the crash here? https://qlever.cs.uni-freiburg.de/wikidata/uBQWSn

Re 3: If I recall correctly, the server replied to the query with a 404 error. Perhaps the frontend code couldn't handle that?

[hannahbast]

@1: That is the normal configuration, it just experimentally ran otherwise in the last days (and still does currently), but we will revert to the auto-restart mode today or tomorrow.

@2: We will investigate, thanks!

@3: Yup, I think that's the problem, will fix that!

[hannahbast]

@tuukka @nicolaes Here is a quick update

@ Romanian query: The problem QLever had with that query ran a bit deeper and we are still working on that. But what's probably more interesting for you is that your original query did not make sense because it gives an empty result because a Wikidata entity can have a subset of the many properties in the query, but never all of them. A more meaningful variant of the query is the following: https://qlever.cs.uni-freiburg.de/wikidata/m76Lrg . This takes 23 seconds on QLever. It's not super fast, but it's also a very hard query (and Blazegraph or Virtuoso both time out on it).

@ English variant of the query: You can just replace the two "ro" by "en", and the query will still work, but now with the English Wikipedia pages: https://qlever.cs.uni-freiburg.de/wikidata/luxplf . The running time is essentially the same. The simpler query https://qlever.cs.uni-freiburg.de/wikidata/uBQWSn you posted above also works and takes around 7 seconds. The reason it didn't work when you tried it was probably because the backend was close to dying (or already dead) because of the other query.

@ QLever UI: The QLever UI failed to show a proper error message when the backend did not reply (it did not matter what the status code was). This is now fixed and the UI should report a meaningful message in all cases.

Please let us know if there are any more problems.

[tuukka]

Is this a typo in the code?

Error processing query

Unknown error

Your query was:
 jqXHR.reponseJSON was empty

[hannahbast]

For which query (on which instance) did you get this result?

[tuukka]

For which query (on which instance) did you get this result?

This was last night, approximately an hour before midnight UTC. Unfortunately I didn't save more details.

[hannahbast]

First step: Reload https://qlever.cs.uni-freiburg.de (F12 to open the Developer Console, Network -> Disable Cache, F5).

Let us know when you encounter the error again.

[hannahbast]

Fixed by 743cb9ecced5bdc7378ddee94b15dd091c06a757

[nicolaes]

@hannahbast thank you for looking into this 🙏