Error: Invalid JSON RPC response: ""

[wbt]

I have a set of simple contracts that I am trying to deploy via truffle migrate --reset to a private PoA chain running on geth 1.9.10 (amd64/Win10). I have the node with the RPC API open running in one terminal window and am running the migrate command in another window (in addition to other network nodes running elsewhere). I am using Truffle's HDWalletProvider and referencing the server by IP 127.0.0.1 with port as appropriate to the RPC-running node.

The migration deploys several contracts and has a 20 second gap between them, which is considerably greater than the 5 second gap between blocks on the running chain.

The account the contracts are being deployed from is richly prefunded in the genesis block.

The first 2-3 contracts reliably deploy successfully, and in the node terminal I see a line saying "Submitted contract creation" with fullhash and contract values matching the transaction hash and contract address reported by Truffle.

However, soon after, I get a large group of errors reporting Invalid JSON RPC response: ""

The stack trace says the following (where dir is actually much longer):

    at Object.InvalidResponse (dir\node_modules\web3-core-helpers\src\errors.js:42:16)
    at XMLHttpRequest.request.onreadystatechange (dir\node_modules\web3-providers-http\src\index.js:92:32)
    at XMLHttpRequestEventTarget.dispatchEvent (dir\node_modules\xhr2-cookies\xml-http-request-event-target.ts:44:13)
    at XMLHttpRequest._setReadyState (dir\node_modules\xhr2-cookies\xml-http-request.ts:219:8)
    at XMLHttpRequest._onHttpRequestError (dir\node_modules\xhr2-cookies\xml-http-request.ts:379:8)
    at ClientRequest.<anonymous> (dir\node_modules\xhr2-cookies\xml-http-request.ts:266:37)
    at ClientRequest.emit (events.js:159:13)
    at Socket.socketErrorListener (_http_client.js:389:9)
    at Socket.emit (events.js:159:13)
    at emitErrorNT (internal/streams/destroy.js:64:8)
    at process._tickCallback (internal/process/next_tick.js:152:19)

Versions

Solidity ^0.5.0 in contracts and in truffle-config.js
truffle version:
Truffle v5.1.12 (core: 5.1.12)
Solidity - ^0.5.0 (solc-js)
Node v9.3.0
Web3.js v1.2.1

I don't think it's an issue where Truffle can't reach the server at all, because the first few contracts seem to deploy fine. Also it doesn't seem like this traffic with the 20 second gaps should be enough to overwhelm the RPC server. The server console does not indicate any problems.

[cgewecke]

Hi @wbt

Some questions...

• are you using Truffle migrations' block confirmations setting to stage the transactions or a manual delay? If the latter, does using confirmations instead have any effect?
• do the deployments fail when using a single geth node (e.g in --dev mode)?
• do the deployments fail with an insta-mining ganache instance?
• do the deployments fail with a ganache instance set up to mine at a 5s block interval?

[wbt]

Thanks for responding with useful questions!

I have one answer, at least:

• I am not using the block confirmations setting, but a time-based delay (with setTimeout()). However, based on what I was seeing in the geth node logs, I don't think using confirmations would have any effect as a low number of confirmations (e.g. 2) gets reached before the time-based delay expires.

I found a workaround to getting a successful migration in downgrading the globally installed version of Truffle to v4.1.16. As testing with insta-mining Ganache (prior to finding your comment) revealed, the code-base where this was originally observed had some issues causing the migration to fail much later in the process than contract deployment, because it needs a bit of work to come up to Truffle v5 standards.

After a wide range of interventions, including restarting the machine, restarting Geth nodes, removing the build folder a few times, and uninstalling & reinstalling Truffle a few times, my smaller reproducible example no longer reproduces this specific issue: the contracts are deployed successfully to the local geth network, using truffle v5.1.12 as reported. Even returning back to the original code while still running v5.1.12 after all those interventions, the contract deployments are all succeeding, even when the time-based wait is dropped to 2 seconds. (The migration still fails because of the other changes it needs to get up to truffle v5, but that's not core to this issue.)

Unfortunately, at this point I can't easily test for answers to the other questions. However, if I get back to observing this issue, I will try to be a little more systematically careful about testing the interventions to try to narrow this issue down. I'm convinced there is still an issue here, but it's just hard to readily isolate.

---

While looking into this, I opened several tabs of folks reporting similar issues elsewhere. As of finding the workaround, some were still open as not yet fully investigated to see if they might yield insight, and I include them here in case I or others can benefit from them later:
Web3 #1470
Loomchain #1558
Truffle #852
StackOverflow #48217795
Truffle #894b
Infura #78

[cgewecke]

@wbt

Thanks for collecting all those issues. Read through a bit seems like it might be traceable to metamask/web3-provider-engine's block polling.

There is a recommendation for improving Web3's error message in Infura 78 which might be appropriate since it clarifies whether or not a disconnection has happened...

The only suggestion I have here is that Web3JS should enhance the error text to something like:

"Invalid JSON RPC response. Server dropped the connection unexpectedly resulting in an empty response from the server."

[wbt]

"Invalid JSON RPC response. Server dropped the connection unexpectedly resulting in an empty response from the server."

That would, of course, depend on Web3 detecting that the empty response is due to a dropped connection. It would also be helpful to be able to capture those errors in a reliable programmatic way (e.g. with a fixed code) so that a process can pause issuance of additional transactions, give the server some minutes to cool down, and then try again.

While I definitely agree that a more informative error message, assuming it is accurate, would help, and completely address some of the other issues I found along the way, I don't think it would cover this one because it leaves open the question of why the server disconnection is happening, even with such a modest level of activity directed at the server and a rock-solid network connection to localhost. This is especially observing the server process produce no errors and successfully process contract deployments immediately before and a few minutes after the drops/disconnections.

[cgewecke]

@wbt Closing in favor of #3425 which has a PR associated with it. Hope that's ok, please ping if you feel there's something in this issue that isn't encompassed by that ticket.

[HamishAndrew]

I've got this in Cardanos KEVM TestNet .via Mallet...... Error: Invalid JSON RPC response: “404 page not found\n” NOT sure where that lies but WEB3 seems to be mentioned alot ..

Note that this issue has been addressed by the recent PR #5179 to replace the broken xhr2-cookies for the modern fetch API.