Unsuccessfull geth dump

[rllola]

I have 3 nodes for which I tried to do a geth dump but thye all failed because of missing trie node. How can I fix this ? How can I get a dump of the state from a relatively recent blocks ?

[holiman]

try geth snapshot dump. Much faster, if it works, since it uses the snapshot flat data instead of iterating the trie.
Also, if you're willing to provide the output from the failed command, that'd be good, maybe we can spot some error

[rllola]

Thanks for your answer.

The error that I have using geth dump is

missing trie node 6a9286d49a985db2aa2d5c2dc38ca7f2c69d27e7dd016ab88db1aa79f2acddd9 (path ) state 0x6a9286d49a985db2aa2d5c2dc38ca7f2c69d27e7dd016ab88db1aa79f2acddd9 is not available

Everytime it is a different trie and I tried specifying block numbers but always missing a trie.

I tried the geth snapshot dump and this time it is saying it is missing the header

full logs

lola@R320-Lola:~$ ./geth-linux-amd64-1.14.12-293a300d/geth snapshot dump --nostorage 21373450
INFO [12-10|12:23:13.939] Maximum peer count                       ETH=50 total=50
INFO [12-10|12:23:13.942] Smartcard socket not found, disabling    err="stat /run/pcscd/pcscd.comm: no such file or directory"
INFO [12-10|12:23:13.945] Set global gas cap                       cap=50,000,000
INFO [12-10|12:23:13.946] Initializing the KZG library             backend=gokzg
INFO [12-10|12:23:13.987] Using pebble as the backing database
INFO [12-10|12:23:13.987] Allocated cache and file handles         database=/home/lola/.ethereum/geth/chaindata cache=512.00MiB handles=524,288
INFO [12-10|12:23:14.343] Opened ancient database                  database=/home/lola/.ethereum/geth/chaindata/ancient/chain readonly=true
header for block 21373450 not found

I setup this node this week end. I will try on 2 others see if I have the same result.

[tskoyo]

Do you have your node synced with the latest state? The missing trie node and header not found errors usually mean that the node hasn’t fully synced up to that block or hasn’t retained the necessary historical state data.

[rllola]

I have 2 nodes that are not synced because I believe they cant catch up. And everytime I stop the state healing start from 0.

Here the log of one of them

Dec 11 05:54:25 R320-Lola geth[2876943]: INFO [12-11|05:54:25.448] Forkchoice requested sync to new head    number=21,378,799 hash=24a4ce..6f003d finalized=21,378,713
Dec 11 05:54:26 R320-Lola geth[2876943]: INFO [12-11|05:54:26.066] Syncing: chain download in progress      synced=100.00% chain=832.40GiB headers=21,378,[email protected] bodies=21,378,[email protected] receipts=21,378,[email protected] eta=20m18.563s
Dec 11 05:54:29 R320-Lola geth[2876943]: INFO [12-11|05:54:29.822] Syncing: state healing in progress       accounts=297,[email protected] slots=512,[email protected] [email protected]    nodes=4,881,[email protected]    pending=24756
Dec 11 05:54:34 R320-Lola geth[2876943]: INFO [12-11|05:54:34.070] Syncing: chain download in progress      synced=100.00% chain=832.40GiB headers=21,378,[email protected] bodies=21,378,[email protected] receipts=21,378,[email protected] eta=20m18.720s
Dec 11 05:54:37 R320-Lola geth[2876943]: INFO [12-11|05:54:37.113] Forkchoice requested sync to new head    number=21,378,800 hash=151f70..2a2bdf finalized=21,378,713
Dec 11 05:54:38 R320-Lola geth[2876943]: INFO [12-11|05:54:38.154] Syncing: state healing in progress       accounts=297,[email protected] slots=512,[email protected] [email protected]    nodes=4,881,[email protected]    pending=25089
Dec 11 05:54:42 R320-Lola geth[2876943]: INFO [12-11|05:54:42.075] Syncing: chain download in progress      synced=100.00% chain=832.40GiB headers=21,378,[email protected] bodies=21,378,[email protected] receipts=21,378,[email protected] eta=20m30.826s
Dec 11 05:54:46 R320-Lola geth[2876943]: INFO [12-11|05:54:46.629] Syncing: state healing in progress       accounts=297,[email protected] slots=512,[email protected] [email protected]    nodes=4,882,[email protected]    pending=25241
Dec 11 05:54:49 R320-Lola geth[2876943]: INFO [12-11|05:54:49.029] Forkchoice requested sync to new head    number=21,378,801 hash=99dcfc..17fbbc finalized=21,378,713
Dec 11 05:54:50 R320-Lola geth[2876943]: INFO [12-11|05:54:50.079] Syncing: chain download in progress      synced=100.00% chain=832.40GiB headers=21,378,[email protected] bodies=21,378,[email protected] receipts=21,378,[email protected] eta=20m42.936s
Dec 11 05:54:56 R320-Lola geth[2876943]: INFO [12-11|05:54:56.036] Syncing: state healing in progress       accounts=297,[email protected] slots=513,[email protected] [email protected]    nodes=4,883,[email protected]    pending=25418
Dec 11 05:54:58 R320-Lola geth[2876943]: INFO [12-11|05:54:58.083] Syncing: chain download in progress      synced=100.00% chain=832.40GiB headers=21,378,[email protected] bodies=21,378,[email protected] receipts=21,378,[email protected] eta=20m43.095s
Dec 11 05:55:01 R320-Lola geth[2876943]: INFO [12-11|05:55:01.197] Forkchoice requested sync to new head    number=21,378,802 hash=4617c7..b5f4a4 finalized=21,378,713
Dec 11 05:55:04 R320-Lola geth[2876943]: INFO [12-11|05:55:04.036] Syncing: state healing in progress       accounts=297,[email protected] slots=513,[email protected] [email protected]    nodes=4,883,[email protected]    pending=25488
Dec 11 05:55:06 R320-Lola geth[2876943]: INFO [12-11|05:55:06.089] Syncing: chain download in progress      synced=100.00% chain=832.40GiB headers=21,378,[email protected] bodies=21,378,[email protected] receipts=21,378,[email protected] eta=20m55.209s

The other node is one of a friend and yes it is synced. Also I have to stop the node to do a dump so it is annoying.
But same result I have missing tries or when doing snapshot I have missing headers message.