feat: cpu and store gas updates #3054

piux2 · 2024-10-31T04:53:38Z

Summary:

This PR updates the CPU and Store gas based on results from the benchmarking tool:#2241

For CPU gas, the measurement is in nanoseconds per opcode execution.
For storage gas, the measurement is in nanoseconds per byte for each type of Gno store access.

Changes:

We moved the gas meter from the underlying store to the upper Gno store to capture accurate resource consumption for VM transactions. At the same time, we retain the original gas store and gas meter for the Auth Keeper to handle regular blockchain transactions that do not necessarily involve the VM.

We also updated the gas-wanted in the integration test to reflect actual gas usage. This can serve as a flag to alert us to future changes that might increase gas assumptions.

Additional reasons for these changes include:

The Gno VM store processes additional complex encoding and decoding of data structures, rather than simply reading and writing bytes to disk.
For the above reason, we benchmarked gas for store access at the Gno store level.
We want to avoid consuming gas at two points for a single store access during a VM transaction.

Here are the diagrams to explain the store access gas before and after changes

Before:

After:

Contributors' checklist...

Added new tests
Provided an example (e.g. screenshot) to aid review or the PR is self-explanatory
Updated the official documentation or not needed
No breaking changes were made, or a BREAKING CHANGE: xxx message was included in the description
Added references to related issues and PRs
Provided any useful hints for running manual tests

gno.land/pkg/sdk/vm/gas_test.go

thehowl · 2024-10-31T13:01:27Z

gno.land/cmd/gnoland/testdata/addpkg_namespace.txtar

can we just make the txtars all just use a -gas-wanted of 10M?

just because they'll have to be updated again. I'd want for us to keep track of "golden" gas values just in one place.

I understand that we need to update for gas changes. This provides an opportunity to verify the changes and understand where the gas impact comes from, especially if there is a significant increase in gas usage in our implementation. We can add a 50% buffer to the gas usage as a safeguard. I am also open to other solutions.

thehowl · 2024-10-31T13:05:06Z

gnovm/pkg/gnolang/machine.go

@@ -1154,127 +1154,130 @@ func (m *Machine) incrCPU(cycles int64) {
 }

 const (
+	// CPU cycles


Can we add a reference to the benchmarks and methodology?

Yes please 🙏

I'm a bit worried about the new gas values, and a comparison would help us make a fair assessment of the gas bumps

I'm concerned that this approach renders ugnot nearly valueless, as it results in excessively long and difficult-to-read numbers.

We should either adopt a strategy that sacrifices some precision, such as dividing by N (which could be a parameter), or consider introducing another level of gnot conversion, like ngnot for gas, to maintain the relevance of ugnot.

@moul @zivkovicmilos I totally agree; the high gas fees create a poor user experience. Let me think about how we can address this later. I think we could maintain precision in the program and round it up at the UI layer.

thehowl · 2024-10-31T13:08:51Z

gno.land/pkg/sdk/vm/keeper.go

@@ -205,6 +205,7 @@ var gnoStoreContextKey gnoStoreContextKeyType
 func (vm *VMKeeper) newGnoTransactionStore(ctx sdk.Context) gno.TransactionStore {
 	base := ctx.Store(vm.baseKey)
 	iavl := ctx.Store(vm.iavlKey)
+	vm.gnoStore.SetGasMeter(ctx.GasMeter())


This is unsafe, because vm.gnoStore is shared, and this would set the gas meter globally; ie. not scoped to the transaction.

It sounds reasonable to switch to start with per transaction and later move to per machine if we also have machine concurrency.

@thehowl good catch! I will pass the gasMeter to gnoStore.BeginTransaction() instead

thehowl · 2024-10-31T13:09:09Z

gnovm/pkg/gnolang/store.go

+func (ds *defaultStore) SetGasMeter(gm store.GasMeter) {
+	ds.gasMeter = gm
+}


I think this ought to go in BeginTransaction, and the store shouldn't use a gas store otherwise.

It makes sense to count gas for the store independently since future features will dissociate CPU gas from storage gas, allowing costs to cover cpuCost + storageCost + futureStorageRent. Additionally, realms could have different storage cost ratios—some sponsored by the chain, others transitioned to 'archives,' or even storage classes with varying configurations based on price tiers.

I think this ought to go in BeginTransaction, and the store shouldn't use a gas store otherwise.

@thehowl Agreed, I will assign the gasMeter in BeginTransaction and remove this method.

thehowl · 2024-10-31T13:09:45Z

gnovm/pkg/gnolang/store.go

+func DefaultGasConfig() GasConfig {
+	return GasConfig{
+		GasGetObject:       16,   // per byte cost
+		GasSetObject:       16,   // per byte cost
+		GasGetType:         52,   // per byte cost
+		GasSetType:         52,   // per byte cost
+		GasGetPackageRealm: 524,  // per byte cost
+		GasSetPackageRealm: 524,  // per byte cost
+		GasAddMemPackage:   8,    // per byte cost
+		GasGetMemPackage:   8,    // per byte cost
+		GasDeleteObject:    3715, // flat cost
+	}
+}


These are constants for the VM. Can these also not be constants?

I think it is better to keep the gas constant until we implement other ways to update the value, such as through genesis or GovDAO with the parameter module. For now, whenever we change the value, we should modify this section of the code and go through code reviews again.

thehowl · 2024-10-31T13:11:38Z

tm2/pkg/sdk/context.go

+func (c Context) Store(key store.StoreKey) store.Store {
+	return c.MultiStore().GetStore(key)
+}
+


I can understand the rename to GasStore, but I don't think we should expose a gas-less Store method, especially when getting the underlying store is still trivial and exported.

Suggested change

func (c Context) Store(key store.StoreKey) store.Store {

return c.MultiStore().GetStore(key)

}

func (c Context) GaslessStore(key store.StoreKey) store.Store {

return c.MultiStore().GetStore(key)

}

?

I can understand the rename to GasStore, but I don't think we should expose a gas-less Store method, especially when getting the underlying store is still trivial and exported.

The purpose here is not to rename the store but to separate gas store access from non-gas store access in the context. This separation is necessary because we need to wrap the gas meter at different levels for Auth Keeper gas consumption and VM gas consumption.

Please refer to the diagram above. In Auth Keeper, the GasStore wraps the gas meter at the underlying base store level. In VM Keeper, the gas meter is wrapped above the underlying store in the gnoStore, allowing us to capture activity to persist realm objects. The ctx.Store is used by the VM Keeper to access the underlying store without the gas meter, so it can wrap it at a higher layer.

The context.Store abstracts away the underlying MultiStore concept. This approach is consistent with how context, app, and keeper store access are managed.

zivkovicmilos

I think the PR looks fine, apart from the comments left by @thehowl, which should be addressed about the global gas meter setting in the VM keeper 🙏

I'm temporarily just putting a pause on the PR, to allow for us to discuss whether these new gas values make sense, since they are a huge change. I also want to see some more reasoning and gas comparisons, apart from #2241

If we're fine with the bumps, I am also cool with the PR moving forward as is ✅

cc @moul @notJoon

gno.land/cmd/gnoland/testdata/issue_1786.txtar

zivkovicmilos · 2024-11-19T12:56:10Z

gnovm/pkg/gnolang/machine.go

@@ -1154,127 +1154,130 @@ func (m *Machine) incrCPU(cycles int64) {
 }

 const (
+	// CPU cycles


Yes please 🙏

I'm a bit worried about the new gas values, and a comparison would help us make a fair assessment of the gas bumps

moul

Overall, it looks good. In this PR or a future one, I suggest we:

Add a global multiplier/divider parameter to avoid using ngnot while keeping the numbers relatively readable.
Create a benchmarking tool that runs all the gnocode in the repository and generates a CSV or TXT file with the gas details we commit periodically (there's no need for this on every commit, in my opinion). This way, whenever we want to fine-tune a parameter, we can run the same script again and compare the results.

zivkovicmilos

Unblocking after @moul is fine with the bumps

Gno2D2 · 2024-12-06T09:54:00Z

I'm a bot that assists the Gno Core team in maintaining this repository. My role is to ensure that contributors understand and follow our guidelines, helping to streamline the development process.

The following requirements must be fulfilled before a pull request can be merged.
Some requirement checks are automated and can be verified by the CI, while others need manual verification by a staff member.

These requirements are defined in this configuration file.

Automated Checks

🟢 Maintainers must be able to edit this pull request (more info)
🟢 The pull request head branch must be up-to-date with its base (more info)

Manual Checks

No manual checks match this pull request.

Debug

Automated Checks
Maintainers must be able to edit this pull request (more info)
If
🟢 Condition met
└── 🟢 On every pull request
Then
🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request
The pull request head branch must be up-to-date with its base (more info)
If
🟢 Condition met
└── 🟢 On every pull request
Then
🟢 Requirement satisfied
└── 🟢 Head branch (piux2:feat_gas_ops) is up to date with base (master): behind by 0 / ahead by 5

codecov · 2024-12-06T23:12:19Z

Codecov Report

Attention: Patch coverage is 80.32787% with 12 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
gno.land/pkg/keyscli/addpkg.go	0.00%	4 Missing and 2 partials ⚠️
gnovm/pkg/gnolang/store.go	87.50%	4 Missing and 1 partial ⚠️
tm2/pkg/sdk/auth/keeper.go	80.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

github-actions bot assigned piux2 Oct 31, 2024

github-actions bot added 📦 🤖 gnovm Issues or PRs gnovm related 📦 🌐 tendermint v2 Issues or PRs tm2 related 📦 ⛰️ gno.land Issues or PRs gno.land package related labels Oct 31, 2024

piux2 requested review from jaekwon, thehowl and zivkovicmilos October 31, 2024 04:54

thehowl reviewed Oct 31, 2024

View reviewed changes

Kouteki added the in focus Core team is prioritizing this work label Nov 1, 2024

Kouteki added this to the 🚀 Mainnet launch milestone Nov 11, 2024

Kouteki mentioned this pull request Nov 19, 2024

[WIP] Test6 launch #3142

Open

zivkovicmilos requested changes Nov 19, 2024

View reviewed changes

Kouteki requested a review from moul November 20, 2024 11:37

moul approved these changes Nov 25, 2024

View reviewed changes

This was referenced Nov 27, 2024

Minutes: Core Staff Weekly Syncs [every Monday] gnolang/meetings#36

Open

[chain] Gas Fee Equation MVP #1917

Open

zivkovicmilos approved these changes Dec 4, 2024

View reviewed changes

piux2 force-pushed the feat_gas_ops branch from 42990a4 to fce2e4e Compare December 6, 2024 22:53

piux2 added 2 commits December 6, 2024 15:10

feat: cpu and store gas updates

292f670

fix gas meter being global

9f197d1

piux2 force-pushed the feat_gas_ops branch from fce2e4e to 9f197d1 Compare December 6, 2024 23:10

piux2 added 3 commits December 6, 2024 15:24

fix test

e298282

fix test

67d5564

fix gas

a749455

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: cpu and store gas updates #3054

feat: cpu and store gas updates #3054

piux2 commented Oct 31, 2024 •

edited

Loading

thehowl Oct 31, 2024

piux2 Dec 6, 2024 •

edited

Loading

thehowl Oct 31, 2024

zivkovicmilos Nov 19, 2024

moul Nov 25, 2024

piux2 Dec 6, 2024 •

edited

Loading

thehowl Oct 31, 2024

moul Nov 25, 2024

piux2 Dec 6, 2024

thehowl Oct 31, 2024

moul Nov 25, 2024

piux2 Dec 6, 2024 •

edited

Loading

thehowl Oct 31, 2024

piux2 Dec 6, 2024

thehowl Oct 31, 2024

moul Nov 25, 2024 •

edited

Loading

piux2 Dec 6, 2024

zivkovicmilos left a comment

zivkovicmilos Nov 19, 2024

moul left a comment

zivkovicmilos left a comment

Gno2D2 commented Dec 6, 2024 •

edited

Loading

If

Then

If

Then

codecov bot commented Dec 6, 2024 •

edited

Loading

feat: cpu and store gas updates #3054

Are you sure you want to change the base?

feat: cpu and store gas updates #3054

Conversation

piux2 commented Oct 31, 2024 • edited Loading

Summary:

Changes:

Before:

After:

Choose a reason for hiding this comment

piux2 Dec 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

piux2 Dec 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

piux2 Dec 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moul Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zivkovicmilos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moul left a comment

Choose a reason for hiding this comment

zivkovicmilos left a comment

Choose a reason for hiding this comment

Gno2D2 commented Dec 6, 2024 • edited Loading

Automated Checks

Manual Checks

If

Then

If

Then

codecov bot commented Dec 6, 2024 • edited Loading

Codecov Report

piux2 commented Oct 31, 2024 •

edited

Loading

piux2 Dec 6, 2024 •

edited

Loading

piux2 Dec 6, 2024 •

edited

Loading

piux2 Dec 6, 2024 •

edited

Loading

moul Nov 25, 2024 •

edited

Loading

Gno2D2 commented Dec 6, 2024 •

edited

Loading

codecov bot commented Dec 6, 2024 •

edited

Loading