[quirrel.dev service] Occasional 400 from api.quirrel.dev

[thecodedrift]

Bug Report

Current Behavior
Occasionally, a queue fails to post a handler success response back to api.quirrel.dev. The lack of a response causes the Quirrel service to think the call failed.

Expected behavior/code
The communication with api.quirrel.dev has built in retries, or at least on failure throws a useful error since the 400 shows up in quirrel.dev as an empty string ""

Environment

package	version
Quirrel	1.7.4
next.js	12.0.7
node	lts (16)
yarn	3.0.2

• Quirrel version: [e.g. v1.0.3]
• Node/npm version: [e.g. Node 14/npm 6]

Possible Solution
As a workaround, I can pass my own fetch implementation into the route, wrapping with fetch-retry or an equivalent library. However, it feels like some amount of basic retrying should be baked into the client library itself instead of throwing on a 400.

I prototyped a few API alternatives to get fetch exposed at the point of Queue creation. I think the most reasonable place to put this if we don't want to change the way the Quirrel client works is to expose fetch in EnqueueJobOptions ref which defaults to (options) => fetch(options), allowing for a custom fetching implementation.

Or we can add retries to makeRequest, enqueue, enqueueMany, etc. In that case, I would strongly recommend using something like fetch-retry with sensible defaults.

Additional context/Screenshots

2022-01-03T21:04:04.230Z	e3ee4716-20fd-452c-91b2-3d153315278f	
ERROR	Unhandled Promise Rejection
{
  "errorType": "Runtime.UnhandledPromiseRejection",
  "errorMessage": "FetchError: request to https://api.quirrel.dev/queues/<queue> failed, reason: read ECONNRESET",
  "reason": {
    "errorType": "FetchError",
    "errorMessage": "request to https://api.quirrel.dev/queues/<queue> failed, reason: read ECONNRESET",
    "code": "ECONNRESET",
    "message": "request to https://api.quirrel.dev/queues/<queue> failed, reason: read ECONNRESET",
    "type": "system",
    "errno": "ECONNRESET",
    "stack": [
      "FetchError: request to https://api.quirrel.dev/queues/<queue> failed, reason: read ECONNRESET",
      "    at ClientRequest.<anonymous> (/var/task/node_modules/node-fetch/lib/index.js:1461:11)",
      "    at ClientRequest.emit (events.js:400:28)",
      "    at TLSSocket.socketErrorListener (_http_client.js:475:9)",
      "    at TLSSocket.emit (events.js:400:28)",
      "    at emitErrorNT (internal/streams/destroy.js:106:8)",
      "    at emitErrorCloseNT (internal/streams/destroy.js:74:3)",
      "    at processTicksAndRejections (internal/process/task_queues.js:82:21)"
    ]
  },
  "promise": {},
  "stack": [
    "Runtime.UnhandledPromiseRejection: FetchError: request to https://api.quirrel.dev/queues/<queue> failed, reason: read ECONNRESET",
    "    at process.<anonymous> (/var/runtime/index.js:35:15)",
    "    at process.emit (events.js:412:35)",
    "    at processPromiseRejections (internal/process/promises.js:245:33)",
    "    at processTicksAndRejections (internal/process/task_queues.js:96:32)"
  ]
}

[ERROR] [1641243844254] LAMBDA_RUNTIME Failed to post handler success response. Http response code: 400.
RequestId: 0eac4614-0a52-4a16-b13e-f33138398502 Error: Runtime exited with error: exit status 128
Runtime.ExitError

[Skn0tt]

Hey @jakobo! This is a sensible request. In fact, it's so sensible I already have a PR for that open: #882 😁 Will try to get that merged soon.

[Skn0tt]

released in https://github.com/quirrel-dev/quirrel/releases/tag/v1.8.1!