Atomic VM Refactor #885

ceyonur · 2025-03-24T22:28:31Z

Why this should be merged

This PR moves atomic logic to atomic pkg and adds atomic VM that wraps the evm/vm.

AvalancheGo PR: ava-labs/avalanchego#3702

How this works

Defines InnerVM/ExtensibleVM interfaces to extend and wrap plugin/vm with atomic logic and capabilities. Implement InnerVM/ExtensibleVM in evm/VM.

How this was tested

Extended unit tests
moved existing test to atomic pkg
e2e tests should cover
bootstrapped fuji

Need to be documented?

No

Need to update RELEASES.md?

No

Signed-off-by: Ceyhun Onur <[email protected]>

Co-authored-by: Quentin McGaw <[email protected]> Signed-off-by: Ceyhun Onur <[email protected]>

…move-atomic-gossip

… into libevm-atomic-refactor-2

ceyonur · 2025-04-22T14:39:53Z

The e2e test seems unrelated: #928

I'm marking this r4r

alarso16

Some readability questions, but non-blocking

alarso16 · 2025-04-23T13:11:18Z

plugin/evm/atomic/export_tx.go

-
-	return nil
-}
+func (utx *UnsignedExportTx) Visit(v Visitor) error { return v.ExportTx(utx) }


This name is a little strange to me - why call it Visit and Visitor?

I believe this is the standard naming for "Visitor" pattern (we also use the same pattern in avalanchego). While we don't have many other operations to support more stuff for visitor pattern, I think this fits well here to keep things separate as possible

plugin/evm/atomic/state/atomic_trie.go

alarso16 · 2025-04-23T13:19:38Z

plugin/evm/atomic/state/atomic_trie_iterator.go

 	"github.com/ava-labs/libevm/trie"
 )

-const atomicTrieKeyLen = wrappers.LongLen + common.HashLength
-
 // atomicTrieIterator is an implementation of types.AtomicTrieIterator that serves


This isn't a type anymore, should probably delete the comment

alarso16 · 2025-04-23T15:40:50Z

plugin/evm/atomic/vm/vm.go

+	case errors.Is(err, vmerrors.ErrMakeNewBlockFailed):
+		log.Debug("discarding txs due to error making new block", "err", err)
+		vm.mempool.DiscardCurrentTxs()
+	case err != nil:


This seems confusing - if there is an unidentified error, then we issue the atomic txs?

good catch. I also realized we were not using ErrMakeNewBlockFailed. I refactored how we handle atomic tx extraction and should be fixed now.

alarso16 · 2025-04-23T15:44:29Z

plugin/evm/message/syncable.go

-	acceptImpl func(SyncSummary) (block.StateSyncMode, error)
+type Syncable interface {
+	block.StateSummary
+	GetBlockHash() common.Hash


The other getters don't define it like this, can we just call it Hash() or BlockHash()?

They conflicts with field names, and I think we can be a little more explicit here. Syncable is a bit generic and do not immediately tell anything about Block

alarso16 · 2025-04-23T15:56:16Z

plugin/evm/vmerrors/errors.go

Didn't we just get rid of this file in libevm 2.5?

This is required between atomic pkg and evm pkg (or we can remove it if we let atomic pkg import evm pkg)

michaelkaplan13 · 2025-04-24T14:55:57Z

plugin/evm/atomic/vm/vm.go

+	networkCodec, err := message.NewCodec(atomicsync.AtomicSyncSummary{})
+	if err != nil {
+		return fmt.Errorf("failed to create codec manager: %w", err)
+	}
+
+	// Create the atomic extension structs
+	// some of them need to be initialized after the inner VM is initialized
+	blockExtension := newBlockExtension(extDataHashes, vm)
+	syncExtender := &atomicsync.AtomicSyncExtender{}
+	syncProvider := &atomicsync.AtomicSummaryProvider{}
+	// Create and pass the leaf handler to the atomic extension
+	// it will be initialized after the inner VM is initialized
+	leafHandler := NewAtomicLeafHandler()
+	atomicLeafTypeConfig := &extension.LeafRequestConfig{
+		LeafType:   atomicsync.AtomicTrieNode,
+		MetricName: "sync_atomic_trie_leaves",
+		Handler:    leafHandler,
+	}
+	vm.mempool = &txpool.Mempool{}
+
+	extensionConfig := &extension.Config{
+		NetworkCodec:               networkCodec,
+		ConsensusCallbacks:         vm.createConsensusCallbacks(),
+		BlockExtension:             blockExtension,
+		SyncableParser:             atomicsync.NewAtomicSyncSummaryParser(),
+		SyncExtender:               syncExtender,
+		SyncSummaryProvider:        syncProvider,
+		ExtraSyncLeafHandlerConfig: atomicLeafTypeConfig,
+		ExtraMempool:               vm.mempool,
+		Clock:                      &vm.clock,
+	}
+	if err := innerVM.SetExtensionConfig(extensionConfig); err != nil {
+		return fmt.Errorf("failed to set extension config: %w", err)
+	}
+
+	// Initialize inner vm with the provided parameters
+	if err := innerVM.Initialize(
+		ctx,
+		chainCtx,
+		db,
+		genesisBytes,
+		upgradeBytes,
+		configBytes,
+		toEngine,
+		fxs,
+		appSender,
+	); err != nil {
+		return fmt.Errorf("failed to initialize inner VM: %w", err)
+	}
+
+	// Now we can initialize the mempool and so
+	err = vm.mempool.Initialize(chainCtx, innerVM.MetricRegistry(), defaultMempoolSize, vm.verifyTxAtTip)
+	if err != nil {
+		return fmt.Errorf("failed to initialize mempool: %w", err)
+	}
+
+	// initialize bonus blocks on mainnet
+	var (
+		bonusBlockHeights map[uint64]ids.ID
+	)
+	if vm.ctx.NetworkID == constants.MainnetID {
+		bonusBlockHeights, err = readMainnetBonusBlocks()
+		if err != nil {
+			return fmt.Errorf("failed to read mainnet bonus blocks: %w", err)
+		}
+	}
+
+	// initialize atomic repository
+	lastAcceptedHash, lastAcceptedHeight, err := innerVM.ReadLastAccepted()
+	if err != nil {
+		return fmt.Errorf("failed to read last accepted block: %w", err)
+	}
+	vm.atomicTxRepository, err = atomicstate.NewAtomicTxRepository(innerVM.VersionDB(), atomic.Codec, lastAcceptedHeight)
+	if err != nil {
+		return fmt.Errorf("failed to create atomic repository: %w", err)
+	}
+	vm.atomicBackend, err = atomicstate.NewAtomicBackend(
+		vm.ctx.SharedMemory, bonusBlockHeights,
+		vm.atomicTxRepository, lastAcceptedHeight, lastAcceptedHash,
+		innerVM.Config().CommitInterval,
+	)
+	if err != nil {
+		return fmt.Errorf("failed to create atomic backend: %w", err)
+	}
+
+	// Atomic backend is available now, we can initialize structs that depend on it
+	syncProvider.Initialize(vm.atomicBackend.AtomicTrie())
+	syncExtender.Initialize(vm.atomicBackend, vm.atomicBackend.AtomicTrie(), innerVM.Config().StateSyncRequestSize)
+	leafHandler.Initialize(vm.atomicBackend.AtomicTrie().TrieDB(), atomicstate.AtomicTrieKeyLength, networkCodec)
+	vm.secpCache = secp256k1.RecoverCache{
+		LRU: cache.LRU[ids.ID, *secp256k1.PublicKey]{
+			Size: secpCacheSize,
+		},
+	}
+
+	// so [vm.baseCodec] is a dummy codec use to fulfill the secp256k1fx VM
+	// interface. The fx will register all of its types, which can be safely
+	// ignored by the VM's codec.
+	vm.baseCodec = linearcodec.NewDefault()
+	return vm.fx.Initialize(vm)


Suggested change

innerVM := vm.InnerVM

vm.ctx = chainCtx

var extDataHashes map[common.Hash]common.Hash

// Set the chain config for mainnet/fuji chain IDs

switch chainCtx.NetworkID {

case constants.MainnetID:

extDataHashes = mainnetExtDataHashes

case constants.FujiID:

extDataHashes = fujiExtDataHashes

}

// Free the memory of the extDataHash map

fujiExtDataHashes = nil

mainnetExtDataHashes = nil

networkCodec, err := message.NewCodec(atomicsync.AtomicSyncSummary{})

if err != nil {

return fmt.Errorf("failed to create codec manager: %w", err)

}

// Create the atomic extension structs

// some of them need to be initialized after the inner VM is initialized

blockExtension := newBlockExtension(extDataHashes, vm)

syncExtender := &atomicsync.AtomicSyncExtender{}

syncProvider := &atomicsync.AtomicSummaryProvider{}

// Create and pass the leaf handler to the atomic extension

// it will be initialized after the inner VM is initialized

leafHandler := NewAtomicLeafHandler()

atomicLeafTypeConfig := &extension.LeafRequestConfig{

LeafType: atomicsync.AtomicTrieNode,

MetricName: "sync_atomic_trie_leaves",

Handler: leafHandler,

}

vm.mempool = &txpool.Mempool{}

extensionConfig := &extension.Config{

NetworkCodec: networkCodec,

ConsensusCallbacks: vm.createConsensusCallbacks(),

BlockExtension: blockExtension,

SyncableParser: atomicsync.NewAtomicSyncSummaryParser(),

SyncExtender: syncExtender,

SyncSummaryProvider: syncProvider,

ExtraSyncLeafHandlerConfig: atomicLeafTypeConfig,

ExtraMempool: vm.mempool,

Clock: &vm.clock,

}

if err := innerVM.SetExtensionConfig(extensionConfig); err != nil {

return fmt.Errorf("failed to set extension config: %w", err)

}

// Initialize inner vm with the provided parameters

if err := innerVM.Initialize(

ctx,

chainCtx,

db,

genesisBytes,

upgradeBytes,

configBytes,

toEngine,

fxs,

appSender,

); err != nil {

return fmt.Errorf("failed to initialize inner VM: %w", err)

}

// Now we can initialize the mempool and so

err = vm.mempool.Initialize(chainCtx, innerVM.MetricRegistry(), defaultMempoolSize, vm.verifyTxAtTip)

if err != nil {

return fmt.Errorf("failed to initialize mempool: %w", err)

}

// initialize bonus blocks on mainnet

var (

bonusBlockHeights map[uint64]ids.ID

)

if vm.ctx.NetworkID == constants.MainnetID {

bonusBlockHeights, err = readMainnetBonusBlocks()

if err != nil {

return fmt.Errorf("failed to read mainnet bonus blocks: %w", err)

}

}

// initialize atomic repository

lastAcceptedHash, lastAcceptedHeight, err := innerVM.ReadLastAccepted()

if err != nil {

return fmt.Errorf("failed to read last accepted block: %w", err)

}

vm.atomicTxRepository, err = atomicstate.NewAtomicTxRepository(innerVM.VersionDB(), atomic.Codec, lastAcceptedHeight)

if err != nil {

return fmt.Errorf("failed to create atomic repository: %w", err)

}

vm.atomicBackend, err = atomicstate.NewAtomicBackend(

vm.ctx.SharedMemory, bonusBlockHeights,

vm.atomicTxRepository, lastAcceptedHeight, lastAcceptedHash,

innerVM.Config().CommitInterval,

)

if err != nil {

return fmt.Errorf("failed to create atomic backend: %w", err)

}

// Atomic backend is available now, we can initialize structs that depend on it

syncProvider.Initialize(vm.atomicBackend.AtomicTrie())

syncExtender.Initialize(vm.atomicBackend, vm.atomicBackend.AtomicTrie(), innerVM.Config().StateSyncRequestSize)

leafHandler.Initialize(vm.atomicBackend.AtomicTrie().TrieDB(), atomicstate.AtomicTrieKeyLength, networkCodec)

vm.secpCache = secp256k1.RecoverCache{

LRU: cache.LRU[ids.ID, *secp256k1.PublicKey]{

Size: secpCacheSize,

},

}

// so [vm.baseCodec] is a dummy codec use to fulfill the secp256k1fx VM

// interface. The fx will register all of its types, which can be safely

// ignored by the VM's codec.

vm.baseCodec = linearcodec.NewDefault()

return vm.fx.Initialize(vm)

vm.ctx = chainCtx

var extDataHashes map[common.Hash]common.Hash

// Set the chain config for mainnet/fuji chain IDs

switch chainCtx.NetworkID {

case constants.MainnetID:

extDataHashes = mainnetExtDataHashes

case constants.FujiID:

extDataHashes = fujiExtDataHashes

}

// Free the memory of the extDataHash map

fujiExtDataHashes = nil

mainnetExtDataHashes = nil

networkCodec, err := message.NewCodec(atomicsync.AtomicSyncSummary{})

if err != nil {

return fmt.Errorf("failed to create codec manager: %w", err)

}

// Create the atomic extension structs

// some of them need to be initialized after the inner VM is initialized

blockExtension := newBlockExtension(extDataHashes, vm)

syncExtender := &atomicsync.AtomicSyncExtender{}

syncProvider := &atomicsync.AtomicSummaryProvider{}

// Create and pass the leaf handler to the atomic extension

// it will be initialized after the inner VM is initialized

leafHandler := NewAtomicLeafHandler()

atomicLeafTypeConfig := &extension.LeafRequestConfig{

LeafType: atomicsync.AtomicTrieNode,

MetricName: "sync_atomic_trie_leaves",

Handler: leafHandler,

}

vm.mempool = &txpool.Mempool{}

extensionConfig := &extension.Config{

NetworkCodec: networkCodec,

ConsensusCallbacks: vm.createConsensusCallbacks(),

BlockExtension: blockExtension,

SyncableParser: atomicsync.NewAtomicSyncSummaryParser(),

SyncExtender: syncExtender,

SyncSummaryProvider: syncProvider,

ExtraSyncLeafHandlerConfig: atomicLeafTypeConfig,

ExtraMempool: vm.mempool,

Clock: &vm.clock,

}

if err := vm.SetExtensionConfig(extensionConfig); err != nil {

return fmt.Errorf("failed to set extension config: %w", err)

}

// Initialize inner vm with the provided parameters

if err := vm.InnerVM.Initialize(

ctx,

chainCtx,

db,

genesisBytes,

upgradeBytes,

configBytes,

toEngine,

fxs,

appSender,

); err != nil {

return fmt.Errorf("failed to initialize inner VM: %w", err)

}

// Now we can initialize the mempool and so

err = vm.mempool.Initialize(chainCtx, vm.MetricRegistry(), defaultMempoolSize, vm.verifyTxAtTip)

if err != nil {

return fmt.Errorf("failed to initialize mempool: %w", err)

}

// initialize bonus blocks on mainnet

var (

bonusBlockHeights map[uint64]ids.ID

)

if vm.ctx.NetworkID == constants.MainnetID {

bonusBlockHeights, err = readMainnetBonusBlocks()

if err != nil {

return fmt.Errorf("failed to read mainnet bonus blocks: %w", err)

}

}

// initialize atomic repository

lastAcceptedHash, lastAcceptedHeight, err := vm.ReadLastAccepted()

if err != nil {

return fmt.Errorf("failed to read last accepted block: %w", err)

}

vm.atomicTxRepository, err = atomicstate.NewAtomicTxRepository(vm.VersionDB(), atomic.Codec, lastAcceptedHeight)

if err != nil {

return fmt.Errorf("failed to create atomic repository: %w", err)

}

vm.atomicBackend, err = atomicstate.NewAtomicBackend(

vm.ctx.SharedMemory, bonusBlockHeights,

vm.atomicTxRepository, lastAcceptedHeight, lastAcceptedHash,

vm.Config().CommitInterval,

)

if err != nil {

return fmt.Errorf("failed to create atomic backend: %w", err)

}

// Atomic backend is available now, we can initialize structs that depend on it

syncProvider.Initialize(vm.atomicBackend.AtomicTrie())

syncExtender.Initialize(vm.atomicBackend, vm.atomicBackend.AtomicTrie(), vm.Config().StateSyncRequestSize)

leafHandler.Initialize(vm.atomicBackend.AtomicTrie().TrieDB(), atomicstate.AtomicTrieKeyLength, networkCodec)

vm.secpCache = secp256k1.RecoverCache{

LRU: cache.LRU[ids.ID, *secp256k1.PublicKey]{

Size: secpCacheSize,

},

}

// so [vm.baseCodec] is a dummy codec use to fulfill the secp256k1fx VM

// interface. The fx will register all of its types, which can be safely

// ignored by the VM's codec.

vm.baseCodec = linearcodec.NewDefault()

return vm.fx.Initialize(vm)

I don't think we need to explicitly call the innerVM methods, and it could be more error prone if we override/extend those for the atomic.VM in the future.

This is also the case for other methods below that use innerVM that I think we should clean up.

I agree - I think this is more intuitive to read as well.

I changed them to implicit calls, but we should still be careful that we don't introduce any unwanted recursions.

Co-authored-by: Austin Larson <[email protected]> Signed-off-by: Ceyhun Onur <[email protected]>

… into libevm-atomic-refactor-2

ceyonur · 2025-05-02T20:32:27Z

plugin/evm/atomic/vm/vm.go

+)
+
+type VM struct {
+	// TODO: decide if we want to directly import the evm package and VM struct


We can still do this, but I cannot decide if keeping evm pkg separate as possible so that it's clear what is needed to wrap the VM (innervm/evm.VM)

or if it's better to keep it cleaner by removing any unnecessary interfaces and directly import plugin/evm.

ceyonur and others added 30 commits December 11, 2024 16:57

new atomic pkg

bc26c83

Merge branch 'master' into move-atomic-txs

fe45c37

bump avago

9c3e284

bump versions

ddbbab7

move attomic gossip

0ecd789

lint

019bb04

change newimport clk to time

aa50ce6

move utils

52081a9

remove extra heaps files

d93257b

move database to own pkg

0b10cc4

move atomic trie/state/backend to separaet pkg

8eb3056

remove multicoin/deprecated apis

aad2f5f

Merge branch 'seperate-atomic-pkg-base' into move-atomic-txs

397c535

bump avago

1815741

bump avago

2b13c8e

go mod tidy

2798ad6

Merge branch 'move-atomic-txs' into move-atomic-gossip

6bf2ff2

Signed-off-by: Ceyhun Onur <[email protected]>

update releases md

278e055

use address methods from avago

e5f2c27

bump avago

71babe6

bump e2e avago version

aab47b6

Merge branch 'move-atomic-txs' into move-atomic-gossip

b9b67bc

Update plugin/evm/atomic/gossip_test.go

6f1adc8

Co-authored-by: Quentin McGaw <[email protected]> Signed-off-by: Ceyhun Onur <[email protected]>

Update plugin/evm/atomic/mempool.go

876716f

Co-authored-by: Quentin McGaw <[email protected]> Signed-off-by: Ceyhun Onur <[email protected]>

Update plugin/evm/config/config.go

682e4bc

Co-authored-by: Quentin McGaw <[email protected]> Signed-off-by: Ceyhun Onur <[email protected]>

Update plugin/evm/atomic/tx_heap.go

38ad868

Co-authored-by: Quentin McGaw <[email protected]> Signed-off-by: Ceyhun Onur <[email protected]>

Update plugin/evm/config/config.go

de328e9

Co-authored-by: Quentin McGaw <[email protected]> Signed-off-by: Ceyhun Onur <[email protected]>

fix reviews

55c0be9

Merge branch 'move-atomic-gossip' of github.com:ava-labs/coreth into …

3ac1a1a

…move-atomic-gossip

decouple atomic logic from statesyncers

f17d97c

bump avalanchego dep to atomic-refactor compatible version

e79f334

ceyonur mentioned this pull request Mar 27, 2025

Atomic vm refactor #768

Closed

ceyonur added 6 commits March 27, 2025 19:42

reviews from 768

efb2ae6

Merge branch 'libevm' into libevm-atomic-refactor-2

69a09d4

bump avago

1c43fe5

go mod tidy

46ea1e4

Merge branch 'libevm' into libevm-atomic-refactor-2

0907904

Merge branch 'libevm-atomic-refactor-2' of github.com:ava-labs/coreth…

5f7ad88

… into libevm-atomic-refactor-2

ceyonur changed the base branch from libevm to master April 18, 2025 16:18

ceyonur changed the title ~~Libevm atomic refactor 2~~ Atomic VM Refactor Apr 19, 2025

Merge branch 'master' into libevm-atomic-refactor-2

2f02ab7

ceyonur mentioned this pull request Apr 19, 2025

use atomicvm version for coreth ava-labs/avalanchego#3900

Draft

ceyonur added 2 commits April 20, 2025 16:44

update avalanchego

d476769

update action

f3871f6

ceyonur marked this pull request as ready for review April 22, 2025 14:39

ceyonur requested a review from a team as a code owner April 22, 2025 14:39

Merge branch 'master' into libevm-atomic-refactor-2

902d827

alarso16 approved these changes Apr 23, 2025

View reviewed changes

michaelkaplan13 reviewed Apr 24, 2025

View reviewed changes

alarso16 mentioned this pull request Apr 30, 2025

refactor(plugin,sync): parallelize syncers #927

Open

ceyonur and others added 6 commits May 1, 2025 20:15

Update plugin/evm/atomic/state/atomic_trie.go

c7cd72f

Co-authored-by: Austin Larson <[email protected]> Signed-off-by: Ceyhun Onur <[email protected]>

use block extension for wrapped blocks

bb583c6

call innerVM functions implicitly

47c03b7

Merge branch 'libevm-atomic-refactor-2' of github.com:ava-labs/coreth…

2446b6d

… into libevm-atomic-refactor-2

Merge branch 'master' into libevm-atomic-refactor-2

c7e0876

remove todos

b8979ba

ceyonur commented May 2, 2025

View reviewed changes

ceyonur added 2 commits May 5, 2025 13:55

Merge branch 'master' into libevm-atomic-refactor-2

1ca1e49

update ci version

f5031aa

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Atomic VM Refactor #885

Atomic VM Refactor #885

ceyonur commented Mar 24, 2025 •

edited

Loading

ceyonur commented Apr 22, 2025

alarso16 left a comment

alarso16 Apr 23, 2025

ceyonur May 1, 2025

alarso16 Apr 23, 2025

alarso16 Apr 23, 2025

ceyonur May 2, 2025

alarso16 Apr 23, 2025

ceyonur May 2, 2025

alarso16 Apr 23, 2025

ceyonur May 2, 2025

michaelkaplan13 Apr 24, 2025

alarso16 Apr 30, 2025

ceyonur May 2, 2025

ceyonur May 2, 2025

Atomic VM Refactor #885

Are you sure you want to change the base?

Atomic VM Refactor #885

Conversation

ceyonur commented Mar 24, 2025 • edited Loading

Why this should be merged

How this works

How this was tested

Need to be documented?

Need to update RELEASES.md?

ceyonur commented Apr 22, 2025

alarso16 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ceyonur commented Mar 24, 2025 •

edited

Loading