[sharded 4] migrate statevalueindex

[areshand]

Description

create new statekey schema and populated with fast sync and db tailing

Type of Change

• New feature
• Bug fix
• Breaking change
• Performance improvement
• Refactoring
• Dependency update
• Documentation update
• Tests

Which Components or Systems Does This Change Impact?

• Validator Node
• Full Node (API, Indexer, etc.)
• Move/Aptos Virtual Machine
• Aptos Framework
• Aptos CLI/SDK
• Developer Infrastructure
• Other (specify)

How Has This Been Tested?

1. updated integration test
2. added a new smoke-test for fast-sync scenario

Key Areas to Review

Checklist

• I have read and followed the CONTRIBUTING doc
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I identified and added all stakeholders and component owners affected by this change as reviewers
• I tested both happy and unhappy path of the functionality
• I have made corresponding changes to the documentation

[trunk-io]

⏱️ 20h 10m total CI duration on this PR

Job	Cumulative Duration	Recent Runs
test-fuzzers	5h 57m	🟩 🟩 🟩 🟩 🟩 (+5 more)
rust-targeted-unit-tests	4h 52m	🟥 🟥 🟥 🟥 🟩 (+10 more)
rust-move-tests	1h 58m	🟩 🟩 🟩 🟩 🟩 (+5 more)
rust-smoke-tests	1h 10m	🟩 🟩
rust-lints	57m	🟩 🟩 🟩 🟩 🟩 (+5 more)
execution-performance / single-node-performance	50m	🟥 🟩
forge-e2e-test / forge	45m	🟩 🟥 🟩
run-tests-main-branch	44m	🟩 🟩 🟩 🟩 🟩 (+5 more)
forge-framework-upgrade-test / forge	34m	🟩 🟩
forge-compat-test / forge	26m	🟩 🟩
rust-images / rust-all	24m	🟩 🟩
general-lints	18m	🟩 🟩 🟩 🟩 🟩 (+5 more)
cli-e2e-tests / run-cli-tests	16m	🟩 🟩
test-target-determinator	11m	🟩 🟩
execution-performance / test-target-determinator	10m	🟩 🟩
check-dynamic-deps	10m	🟩 🟩 🟩 🟩 🟩 (+5 more)
rust-build-cached-packages	9m	🟩 🟩
check	7m	🟩 🟩
semgrep/ci	5m	🟩 🟩 🟩 🟩 🟩 (+5 more)
node-api-compatibility-tests / node-api-compatibility-tests	2m	🟩 🟩
file_change_determinator	2m	🟩 🟩 🟩 🟩 🟩 (+5 more)
file_change_determinator	2m	🟩 🟩 🟩 🟩 🟩 (+5 more)
permission-check	32s	🟩 🟩 🟩 🟩 🟩 (+5 more)
permission-check	30s	🟩 🟩 🟩 🟩 🟩 (+5 more)
permission-check	26s	🟩 🟩 🟩 🟩 🟩 (+5 more)
permission-check	26s	🟩 🟩 🟩 🟩 🟩 (+5 more)
file_change_determinator	19s	🟩 🟩
determine-docker-build-metadata	7s	🟩 🟩
permission-check	6s	🟩 🟩

🚨 4 jobs on the last run were significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
rust-move-tests	16m	8m
execution-performance / single-node-performance	27m	15m
rust-targeted-unit-tests	24m	13m
forge-framework-upgrade-test / forge	19m	14m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

[github-actions]

✅ Forge suite compat success on f648076a280621dbfd4e73b1ca83e3a3f52878ed ==> bcc33e72b891c31a85eb17c94d75f5e2ebb02453

Compatibility test results for f648076a280621dbfd4e73b1ca83e3a3f52878ed ==> bcc33e72b891c31a85eb17c94d75f5e2ebb02453 (PR)
1. Check liveness of validators at old version: f648076a280621dbfd4e73b1ca83e3a3f52878ed
compatibility::simple-validator-upgrade::liveness-check : committed: 8147.594929607059 txn/s, latency: 3906.647759387675 ms, (p50: 2900 ms, p90: 6600 ms, p99: 25300 ms), latency samples: 305720
2. Upgrading first Validator to new version: bcc33e72b891c31a85eb17c94d75f5e2ebb02453
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 3010.7834876938036 txn/s, latency: 10313.395303514377 ms, (p50: 10200 ms, p90: 14000 ms, p99: 14300 ms), latency samples: 125200
3. Upgrading rest of first batch to new version: bcc33e72b891c31a85eb17c94d75f5e2ebb02453
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 3299.9628988933177 txn/s, latency: 9286.836031841394 ms, (p50: 9800 ms, p90: 13900 ms, p99: 14200 ms), latency samples: 133160
4. upgrading second batch to new version: bcc33e72b891c31a85eb17c94d75f5e2ebb02453
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 4133.8740649276115 txn/s, latency: 7642.67601239942 ms, (p50: 5700 ms, p90: 21100 ms, p99: 25600 ms), latency samples: 151620
5. check swarm health
Compatibility test for f648076a280621dbfd4e73b1ca83e3a3f52878ed ==> bcc33e72b891c31a85eb17c94d75f5e2ebb02453 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite framework_upgrade success on f648076a280621dbfd4e73b1ca83e3a3f52878ed ==> bcc33e72b891c31a85eb17c94d75f5e2ebb02453

Compatibility test results for f648076a280621dbfd4e73b1ca83e3a3f52878ed ==> bcc33e72b891c31a85eb17c94d75f5e2ebb02453 (PR)
Upgrade the nodes to version: bcc33e72b891c31a85eb17c94d75f5e2ebb02453
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1204.6984516924547 txn/s, submitted: 1207.6611985467873 txn/s, failed submission: 2.96274685433256 txn/s, expired: 2.96274685433256 txn/s, latency: 2527.132415815361 ms, (p50: 1800 ms, p90: 4500 ms, p99: 7500 ms), latency samples: 105720
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1102.6953885761727 txn/s, submitted: 1105.289411089932 txn/s, failed submission: 2.5940225137591746 txn/s, expired: 2.5940225137591746 txn/s, latency: 2934.874743370402 ms, (p50: 2100 ms, p90: 5400 ms, p99: 9200 ms), latency samples: 93520
5. check swarm health
Compatibility test for f648076a280621dbfd4e73b1ca83e3a3f52878ed ==> bcc33e72b891c31a85eb17c94d75f5e2ebb02453 passed
Upgrade the remaining nodes to version: bcc33e72b891c31a85eb17c94d75f5e2ebb02453
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 956.576836670198 txn/s, submitted: 959.3455105824128 txn/s, failed submission: 2.7686739122147555 txn/s, expired: 2.7686739122147555 txn/s, latency: 3311.1623010130247 ms, (p50: 2100 ms, p90: 6900 ms, p99: 10500 ms), latency samples: 82920
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite realistic_env_max_load success on bcc33e72b891c31a85eb17c94d75f5e2ebb02453

two traffics test: inner traffic : committed: 8366.087526038898 txn/s, latency: 4687.014674433672 ms, (p50: 4500 ms, p90: 5400 ms, p99: 11100 ms), latency samples: 3611860
two traffics test : committed: 99.94457843821091 txn/s, latency: 2168.1310344827584 ms, (p50: 2000 ms, p90: 2400 ms, p99: 6500 ms), latency samples: 1740
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.220, avg: 0.211", "QsPosToProposal: max: 0.254, avg: 0.231", "ConsensusProposalToOrdered: max: 0.314, avg: 0.290", "ConsensusOrderedToCommit: max: 0.388, avg: 0.372", "ConsensusProposalToCommit: max: 0.676, avg: 0.662"]
Max round gap was 1 [limit 4] at version 1529106. Max no progress secs was 5.036104 [limit 15] at version 1529106.
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking