Return parsed token accounts in simulateBundle

[mcintyre94]

Rewrite of #592 on latest master

Problem

If you call simulateBundle and pass a token account in pre/postExecutionAccountsConfigs, then the token accounts are never returned in JSON. You will always get the base64 encoded data.

This is caused by attempting to encode the data without fetching the number of decimals for the mint, which causes a fallback to base64 data.

This was fixed for simulateTransaction upstream, but simulateBundle has the same issue.

Upstream issue (with more details of the code path causing this): solana-labs/solana#34693
Upstream fix to simulateTransaction: solana-labs/solana#34619

Summary of Changes

The changes are based on the upstream changes in simulateTransaction.

Instead of directly encoding the post-account data, we call get_encoded_account (added in the upstream PR), which will correctly parse a token account. As this function was written for transaction post-accounts it also takes overwrite_accounts: Option<&HashMap<Pubkey, AccountSharedData>>, so that account state can be overridden if a transaction has changed it.

In order to provide this, we now maintain a AccountOverrides object in rpc_bundle_result_from_bank_result. This is the same data structure, but has some useful helper functions and I think makes sense to use here. After each transaction, this is updated (using upsert_account_overrides) with the post_accounts from the transaction. This is then passed when we encode the post-accounts for that transaction.

The result is that rpc_bundle_result_from_bank_result (and the simulateBundle RPC call) now correctly JSON encodes token accounts, using data from the transactions, or the bank if required.

End-to-end test on localhost

• Build this branch: cargo build

• Create a directory /Users/username/scratch/solana-validator-tokens with the following files:

wallet.json

{
    "pubkey": "2hSzAdfBYSx3q9duv2VwxVoapcbHy14qaTfkPmHn4gbY",
    "account": {
      "data": [
        "",
        "base64"
      ],
      "executable": false,
      "lamports": 100000000,
      "owner": "11111111111111111111111111111111",
      "rentEpoch": 0
    }
  }

token-mint.json

{
    "pubkey": "Gh9ZwEmdLJ8DscKNTkTqPbNwLNNBjuSzaG9Vp2KGtKJr",
    "account": {
        "data": [
            "AQAAAOkoOVUJZf/U1krKr0bUXfcxjltPV8kMSH1gYl2Cm4N7iOvPHkm44BgGAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==",
            "base64"
        ],
        "executable": false,
        "lamports": 58203456600,
        "owner": "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA",
        "rentEpoch": 18446744073709551615,
        "space": 82
    }
}

receive-token.json

{
    "pubkey": "D9EYGf9etBGpPLHwiKMpxzn8gcHZapP6uvakAiCkdapK",
    "account": {
        "data": [
            "6Sg5VQll/9TWSsqvRtRd9zGOW09XyQxIfWBiXYKbg3vbLzrNfN5YZjAai4m3oURtvycuDlQ8wTPq3YIlZUPQ9KiqxS0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
            "base64"
        ],
        "executable": false,
        "lamports": 2039280,
        "owner": "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA",
        "rentEpoch": 18446744073709551615,
        "space": 165
    }
}

send-token.json

{
    "pubkey": "Dg5iatJR92aPAZ6E9kBCrdb2DDeaPoQdexjC48MrvNGf",
    "account": {
        "data": [
            "6Sg5VQll/9TWSsqvRtRd9zGOW09XyQxIfWBiXYKbg3sZOE0jrk6GS84ZX4OU7iFB1zRTqOg8ThrPzCdGFT/NlwDh9QUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
            "base64"
        ],
        "executable": false,
        "lamports": 2039280,
        "owner": "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA",
        "rentEpoch": 18446744073709551615,
        "space": 165
    }
}

• Run the local validator: ./target/debug/solana-test-validator --account-dir /Users/username/scratch/solana-validator-tokens

• Run the following curl command:

curl command

curl --location 'http://localhost:8899/' \
--header 'accept: application/json' \
--header 'content-type: application/json; charset=utf-8' \
--data '{
    "id": 1,
    "jsonrpc": "2.0",
    "method": "simulateBundle",
    "params": [
        {
            "encodedTransactions": [
"AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAQHGThNI65OhkvOGV+DlO4hQdc0U6joPE4az8wnRhU/zZe0aDLGa0cy16lBWIf9ij+BU6eobQjtSBAP3ZRNOzSRTLxPOmj5LXZb8EVEBvpSsLG9P9kYCYqlowhFVOefwFCYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADpKDlVCWX/1NZKyq9G1F33MY5bT1fJDEh9YGJdgpuDewVKU1qZKSEGTSTocWDaOHx8NbXdvJK7geQfqEBBBUSNBt324ddloZPZy+FGzut5rBy0he1fWzeROoz1hX7/AKmOYFcfFz0Vu4XMA4jr8WltA1R3YRYSEipAl9FD4qy+FgMFAAY0RkhkVEwDAgAADAIAAAAAAAAAAAAAAAYEAgQBAAoMoIYBAAAAAAAG"
            ]
        },
        {
            "commitment": "confirmed",
            "encoding": "jsonParsed",
            "skipSigVerify": true,
            "replaceRecentBlockhash": true,
            "preExecutionAccountsConfigs": [{
                "addresses": [
                    "D9EYGf9etBGpPLHwiKMpxzn8gcHZapP6uvakAiCkdapK",
                    "Dg5iatJR92aPAZ6E9kBCrdb2DDeaPoQdexjC48MrvNGf"
                ],
                "encoding": "jsonParsed"
            }],
            "postExecutionAccountsConfigs": [{
                "addresses": [
                    "D9EYGf9etBGpPLHwiKMpxzn8gcHZapP6uvakAiCkdapK",
                    "Dg5iatJR92aPAZ6E9kBCrdb2DDeaPoQdexjC48MrvNGf"
                ],
                "encoding": "jsonParsed"
            }]
        }
    ]
}'

Or, just the JSON RPC body

{
    "id": 1,
    "jsonrpc": "2.0",
    "method": "simulateBundle",
    "params": [
        {
            "encodedTransactions": [
"AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAQHGThNI65OhkvOGV+DlO4hQdc0U6joPE4az8wnRhU/zZe0aDLGa0cy16lBWIf9ij+BU6eobQjtSBAP3ZRNOzSRTLxPOmj5LXZb8EVEBvpSsLG9P9kYCYqlowhFVOefwFCYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADpKDlVCWX/1NZKyq9G1F33MY5bT1fJDEh9YGJdgpuDewVKU1qZKSEGTSTocWDaOHx8NbXdvJK7geQfqEBBBUSNBt324ddloZPZy+FGzut5rBy0he1fWzeROoz1hX7/AKmOYFcfFz0Vu4XMA4jr8WltA1R3YRYSEipAl9FD4qy+FgMFAAY0RkhkVEwDAgAADAIAAAAAAAAAAAAAAAYEAgQBAAoMoIYBAAAAAAAG"
            ]
        },
        {
            "commitment": "confirmed",
            "encoding": "jsonParsed",
            "skipSigVerify": true,
            "replaceRecentBlockhash": true,
            "preExecutionAccountsConfigs": [{
                "addresses": [
                    "D9EYGf9etBGpPLHwiKMpxzn8gcHZapP6uvakAiCkdapK",
                    "Dg5iatJR92aPAZ6E9kBCrdb2DDeaPoQdexjC48MrvNGf"
                ],
                "encoding": "jsonParsed"
            }],
            "postExecutionAccountsConfigs": [{
                "addresses": [
                    "D9EYGf9etBGpPLHwiKMpxzn8gcHZapP6uvakAiCkdapK",
                    "Dg5iatJR92aPAZ6E9kBCrdb2DDeaPoQdexjC48MrvNGf"
                ],
                "encoding": "jsonParsed"
            }]
        }
    ]
}

You'll get back JSON parsed details for the token accounts, before and after.

If you do the same on the current master branch then you'll get base64 encoded token accounts.

Note that simulateTransaction works correctly because my equivalent upstream fix is already merged into Jito.

[buffalu]

do you need to use account overrides here? you should have access to the accounts in the execution results

[mcintyre94]

It comes from the post accounts, but needs to be converted to the shape get_encoded_account expects, which is exactly your AccountOverrides type. Upstream doesn't have that type, but it's the same shape.

Are you suggesting writing a version of get_encoded_account that takes the Jito post_accounts: Vec<(Pubkey, AccountSharedData)> directly? I thought it'd be a smaller change to re-use get_encoded_account from upstream by reshaping the post_accounts to match it.

I think I'd also need to write upsert logic to combine the outputs of the transactions in the bundle since we encode the accounts after each transaction and not just at the end. I thought it was better to re-use AccountOverrides::upsert_account_overrides instead of writing custom logic for that.