mirror of
https://github.com/ethereum/go-ethereum.git
synced 2026-08-19 18:32:23 +00:00
Squash the main verkle PR ahead of rebase don't call Bytes() in GetTreeKey (#137) trie: avoid endianness conversion in GetTreeKey (#140) * trie/utils: add concrete expected value in trie key generation test Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * mod: update to latest go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/utils: avoid endianness conversions Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * apply review changes & update to official go-verkle version Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> upgrade go-verkle to CoW version and get TestProcessVerkle to build (#138) updating ci to use self-hosted machine (#143) fix: storage offset in non-header group + reuse of value buffer (#145) dedup call to ChunkifyCode, same as replay branch (#156) * dedup call to ChunkifyCode, same as replay branch * fix some linter issues fix code offset in tree update (#157) fix REVERT in state processor test execution (#158) * fix code offset in tree update * fix REVERT in test execution save on key hashing: lump code size update with first code chunk group (#159) fix code chunk key calculation and storage key calculation (#161) * fix codeKey calculation * Remove * fix storageOffset * fix the fix to the fix to the offset fix * Remove copy/pasted, unused code in test * fix linter --------- Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com> fix: infinite loop when calling extcodecopy on empty code (#151) upgrade to latest go-verkle fix: only update code in the tree if it's dirty (#174) fix: read-touch the code size and Keccak of the origin (#175) List of changes for converting a sepolia database (#182) * naive conversion rebased on top of beverly hills * changes for the sepolia shadow fork conversion * fixes to please the linter * fixes to please the linter Unified point cache (#180) * Unified point cache * Use cache for Try*Account * alter Trie interface to use caching for slots (#181) * alter Trie interface to use caching for slots * fix: use a lock to protect the point cache (#185) * use fastest non-master go-verkle version & pull trie/Verkle.go changes to use new api (#184) * mod: update to fastest go-verkle version today Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/verkle: use new batch serialization api Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> * fix: TryDelete signature in unit tests --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> trie/utils: fix potential overflow (#191) * trie/utils: fix potential overflow Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/utils: receive storage key as a byte slice Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * revert formatter changes Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/utils: fix mod 256 Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> trie/utils: fix incorrect bigint assignment (#193) Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> upgrade precomp link to fix CI fix: add missing code size&keccak leaves in empty accounts (#192) fixes to use the latest go-verkle@master (#197) * fixes to use the latest go-verkle@master * linter fixes * linter fixes for tests * fix: use jsign's go-verkle fix refactor: remove unused (*StateDB).GetXLittleEndian methods (#204) fix gas accounting issue in state_processor_test.go (#207) update go-verkle not to use StatelessNode anymore (#206) * update go-verkle not to use StatelessNode anymore * update go-verkle to latest refactor: move verkle gas accounting to its own block in TransitionDB (#208) fix a panic in deserializeVerkleProof if GetProofItems returns a nil ProofElements use the cachingDB instead of a custom VerkleDB (#209) * use the cachingDB instead of a custom VerkleDB * fix stack trace in LES remove holiman from CODEOWNERS as he gets too many emails read from tree in state object if the snapshot is nil (#205) add missing error checks for the root node type (#214) implement OpenStorageTrie for verkle trees (#210) * implement OpenStorageTrie for verkle trees * add a few comments for future maintenance * fix linter issue fix: copy balance leaf to new buffer in TryGetAccount (#217) implement some heretofore unimplemented iterator methods (#219) params: move verkle params to their own file (#228) fix: proper number of chunk evals (#215) overlay transition (#244) * overlay transition Fix some bugs identified in the code review Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> Include base -> overlay key-values migration logic (#199) * mod: add go-verkle version with key-value migration new apis Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * core/stateprocessor: use constant for max number of migrated key-values Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * core: add base->overlay key-values migration logic Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * core: fix some compiler errors Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie: consider removing transition trie api in the future Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * mod: use latest go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> fix some unit tests errors get convresion block from file fix compilation issues fix initialization issue in migrator fix: changes needed to run the first 28 blocks important sutff: fix the banner fix: use nonce instead of balance in nonce leaf (#202) fixes for performing the overlay transition (#203) * fixes for performing the overlay transition * fixes for the full replay * fix: deletion-and-recreation of EoA * fixes to replay 2M+ blocks * upgrade to go-verkle@master * fix: proper number of chunk evals * rewrite conversion loop to fix known issues changes to make replay work with the overlay method (#216) * fixes for performing the overlay transition fixes for the full replay fix: deletion-and-recreation of EoA fixes to replay 2M+ blocks upgrade to go-verkle@master fix: proper number of chunk evals rewrite conversion loop to fix known issues changes to make replay work with the overlay method fixes to replay 2M+ blocks update to latest go-verkle@master * use a PBSS-like scheme for internal nodes (#221) * use a PBSS-like scheme for internal nodes * a couple of fixes coming from debugging replay * fix: use an error to notify the transition tree that a deleted account was found in the overlay tree (#222) * fixes for pbss replay (#227) * fixes for pbss replay * trie/verkle: use capped batch size (#229) * trie/verkle: use capped batch size Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/verkle: avoid path variable allocation per db.Put Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * don't keep more than 32 state root conversions in RAM (#230) --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com> * cleanup some code * mod: update go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * re-enable snapshot (#231) * re-enable cancun block / snapshot (#226) * clear storage conversion key upon translating account (#234) * clear storage conversion key upon translating account * mod: use latest go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> * fix: self-deadlock with translated root map mutex (#236) * return compressed commitment as root commitment (#237) --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> fix first panic in *TransitionTrie.Copy() upgrade go-verkle to latest master mod: update go-verkle (#239) Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> core: print state root every 100 blocks (#240) Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> fix: only Commit the account trie (#242) fixes to get TestProcessVerkle to work with the overlay branch (#238) * fixes to get TestProcessVerkle to work with the overlay branch * fix all panics in verkle state processor test * fix proof verification move transition management to cachingDB * fix: mark the verkle transition as started if it's ended without being started * fix the verkle state processing test * fix linter errors * Add a function to clear verkle params for replay * fix: handle TransitionTrie in OpenStorageTrie * fix linter issue * fix the deleted account error (#247) * code cleanup (#248) * fix: don't error on a missing conversion.txt (#249) * Overlay Tree preimages exporting and usage (#246) * export overlay preimages tool Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * use preimages flat file in overlay tree migration logic Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * cmd/geth: add --roothash to overlay tree preimage exporting command Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * cleanup Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * review feedback Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * fix: reduce the PR footprint (#250) * fix: don't fail when preimages.bin is missing (#251) * fix: don't fail when preimages.bin is missing * fix: don't open the preimages file when outside of transition --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> review changes remove replay-specific code
294 lines
10 KiB
Go
294 lines
10 KiB
Go
// Copyright 2015 The go-ethereum Authors
|
|
// This file is part of the go-ethereum library.
|
|
//
|
|
// The go-ethereum library is free software: you can redistribute it and/or modify
|
|
// it under the terms of the GNU Lesser General Public License as published by
|
|
// the Free Software Foundation, either version 3 of the License, or
|
|
// (at your option) any later version.
|
|
//
|
|
// The go-ethereum library is distributed in the hope that it will be useful,
|
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
// GNU Lesser General Public License for more details.
|
|
//
|
|
// You should have received a copy of the GNU Lesser General Public License
|
|
// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
package trie
|
|
|
|
import (
|
|
"github.com/ethereum/go-ethereum/common"
|
|
"github.com/ethereum/go-ethereum/core/types"
|
|
"github.com/ethereum/go-ethereum/rlp"
|
|
"github.com/ethereum/go-ethereum/trie/trienode"
|
|
)
|
|
|
|
// SecureTrie is the old name of StateTrie.
|
|
// Deprecated: use StateTrie.
|
|
type SecureTrie = StateTrie
|
|
|
|
// NewSecure creates a new StateTrie.
|
|
// Deprecated: use NewStateTrie.
|
|
func NewSecure(stateRoot common.Hash, owner common.Hash, root common.Hash, db *Database) (*SecureTrie, error) {
|
|
id := &ID{
|
|
StateRoot: stateRoot,
|
|
Owner: owner,
|
|
Root: root,
|
|
}
|
|
return NewStateTrie(id, db)
|
|
}
|
|
|
|
// StateTrie wraps a trie with key hashing. In a stateTrie trie, all
|
|
// access operations hash the key using keccak256. This prevents
|
|
// calling code from creating long chains of nodes that
|
|
// increase the access time.
|
|
//
|
|
// Contrary to a regular trie, a StateTrie can only be created with
|
|
// New and must have an attached database. The database also stores
|
|
// the preimage of each key if preimage recording is enabled.
|
|
//
|
|
// StateTrie is not safe for concurrent use.
|
|
type StateTrie struct {
|
|
trie Trie
|
|
preimages *preimageStore
|
|
hashKeyBuf [common.HashLength]byte
|
|
secKeyCache map[string][]byte
|
|
secKeyCacheOwner *StateTrie // Pointer to self, replace the key cache on mismatch
|
|
}
|
|
|
|
// NewStateTrie creates a trie with an existing root node from a backing database.
|
|
//
|
|
// If root is the zero hash or the sha3 hash of an empty string, the
|
|
// trie is initially empty. Otherwise, New will panic if db is nil
|
|
// and returns MissingNodeError if the root node cannot be found.
|
|
func NewStateTrie(id *ID, db *Database) (*StateTrie, error) {
|
|
if db == nil {
|
|
panic("trie.NewStateTrie called without a database")
|
|
}
|
|
trie, err := New(id, db)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return &StateTrie{trie: *trie, preimages: db.preimages}, nil
|
|
}
|
|
|
|
// MustGet returns the value for key stored in the trie.
|
|
// The value bytes must not be modified by the caller.
|
|
//
|
|
// This function will omit any encountered error but just
|
|
// print out an error message.
|
|
func (t *StateTrie) MustGet(key []byte) []byte {
|
|
return t.trie.MustGet(t.hashKey(key))
|
|
}
|
|
|
|
// GetStorage attempts to retrieve a storage slot with provided account address
|
|
// and slot key. The value bytes must not be modified by the caller.
|
|
// If the specified storage slot is not in the trie, nil will be returned.
|
|
// If a trie node is not found in the database, a MissingNodeError is returned.
|
|
func (t *StateTrie) GetStorage(_ common.Address, key []byte) ([]byte, error) {
|
|
enc, err := t.trie.Get(t.hashKey(key))
|
|
if err != nil || len(enc) == 0 {
|
|
return nil, err
|
|
}
|
|
_, content, _, err := rlp.Split(enc)
|
|
return content, err
|
|
}
|
|
|
|
// GetAccount attempts to retrieve an account with provided account address.
|
|
// If the specified account is not in the trie, nil will be returned.
|
|
// If a trie node is not found in the database, a MissingNodeError is returned.
|
|
func (t *StateTrie) GetAccount(address common.Address) (*types.StateAccount, error) {
|
|
res, err := t.trie.Get(t.hashKey(address.Bytes()))
|
|
if res == nil || err != nil {
|
|
return nil, err
|
|
}
|
|
ret := new(types.StateAccount)
|
|
err = rlp.DecodeBytes(res, ret)
|
|
return ret, err
|
|
}
|
|
|
|
// GetAccountByHash does the same thing as GetAccount, however it expects an
|
|
// account hash that is the hash of address. This constitutes an abstraction
|
|
// leak, since the client code needs to know the key format.
|
|
func (t *StateTrie) GetAccountByHash(addrHash common.Hash) (*types.StateAccount, error) {
|
|
res, err := t.trie.Get(addrHash.Bytes())
|
|
if res == nil || err != nil {
|
|
return nil, err
|
|
}
|
|
ret := new(types.StateAccount)
|
|
err = rlp.DecodeBytes(res, ret)
|
|
return ret, err
|
|
}
|
|
|
|
// GetNode attempts to retrieve a trie node by compact-encoded path. It is not
|
|
// possible to use keybyte-encoding as the path might contain odd nibbles.
|
|
// If the specified trie node is not in the trie, nil will be returned.
|
|
// If a trie node is not found in the database, a MissingNodeError is returned.
|
|
func (t *StateTrie) GetNode(path []byte) ([]byte, int, error) {
|
|
return t.trie.GetNode(path)
|
|
}
|
|
|
|
// MustUpdate associates key with value in the trie. Subsequent calls to
|
|
// Get will return value. If value has length zero, any existing value
|
|
// is deleted from the trie and calls to Get will return nil.
|
|
//
|
|
// The value bytes must not be modified by the caller while they are
|
|
// stored in the trie.
|
|
//
|
|
// This function will omit any encountered error but just print out an
|
|
// error message.
|
|
func (t *StateTrie) MustUpdate(key, value []byte) {
|
|
hk := t.hashKey(key)
|
|
t.trie.MustUpdate(hk, value)
|
|
t.getSecKeyCache()[string(hk)] = common.CopyBytes(key)
|
|
}
|
|
|
|
// UpdateStorage associates key with value in the trie. Subsequent calls to
|
|
// Get will return value. If value has length zero, any existing value
|
|
// is deleted from the trie and calls to Get will return nil.
|
|
//
|
|
// The value bytes must not be modified by the caller while they are
|
|
// stored in the trie.
|
|
//
|
|
// If a node is not found in the database, a MissingNodeError is returned.
|
|
func (t *StateTrie) UpdateStorage(_ common.Address, key, value []byte) error {
|
|
hk := t.hashKey(key)
|
|
v, _ := rlp.EncodeToBytes(value)
|
|
err := t.trie.Update(hk, v)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
t.getSecKeyCache()[string(hk)] = common.CopyBytes(key)
|
|
return nil
|
|
}
|
|
|
|
// UpdateAccount will abstract the write of an account to the secure trie.
|
|
func (t *StateTrie) UpdateAccount(address common.Address, acc *types.StateAccount) error {
|
|
hk := t.hashKey(address.Bytes())
|
|
data, err := rlp.EncodeToBytes(acc)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if err := t.trie.Update(hk, data); err != nil {
|
|
return err
|
|
}
|
|
t.getSecKeyCache()[string(hk)] = address.Bytes()
|
|
return nil
|
|
}
|
|
|
|
func (t *StateTrie) UpdateContractCode(_ common.Address, _ common.Hash, _ []byte) error {
|
|
return nil
|
|
}
|
|
|
|
// MustDelete removes any existing value for key from the trie. This function
|
|
// will omit any encountered error but just print out an error message.
|
|
func (t *StateTrie) MustDelete(key []byte) {
|
|
hk := t.hashKey(key)
|
|
delete(t.getSecKeyCache(), string(hk))
|
|
t.trie.MustDelete(hk)
|
|
}
|
|
|
|
// DeleteStorage removes any existing storage slot from the trie.
|
|
// If the specified trie node is not in the trie, nothing will be changed.
|
|
// If a node is not found in the database, a MissingNodeError is returned.
|
|
func (t *StateTrie) DeleteStorage(_ common.Address, key []byte) error {
|
|
hk := t.hashKey(key)
|
|
delete(t.getSecKeyCache(), string(hk))
|
|
return t.trie.Delete(hk)
|
|
}
|
|
|
|
// DeleteAccount abstracts an account deletion from the trie.
|
|
func (t *StateTrie) DeleteAccount(address common.Address) error {
|
|
hk := t.hashKey(address.Bytes())
|
|
delete(t.getSecKeyCache(), string(hk))
|
|
return t.trie.Delete(hk)
|
|
}
|
|
|
|
// GetKey returns the sha3 preimage of a hashed key that was
|
|
// previously used to store a value.
|
|
func (t *StateTrie) GetKey(shaKey []byte) []byte {
|
|
if key, ok := t.getSecKeyCache()[string(shaKey)]; ok {
|
|
return key
|
|
}
|
|
if t.preimages == nil {
|
|
return nil
|
|
}
|
|
return t.preimages.preimage(common.BytesToHash(shaKey))
|
|
}
|
|
|
|
// Commit collects all dirty nodes in the trie and replaces them with the
|
|
// corresponding node hash. All collected nodes (including dirty leaves if
|
|
// collectLeaf is true) will be encapsulated into a nodeset for return.
|
|
// The returned nodeset can be nil if the trie is clean (nothing to commit).
|
|
// All cached preimages will be also flushed if preimages recording is enabled.
|
|
// Once the trie is committed, it's not usable anymore. A new trie must
|
|
// be created with new root and updated trie database for following usage
|
|
func (t *StateTrie) Commit(collectLeaf bool) (common.Hash, *trienode.NodeSet, error) {
|
|
// Write all the pre-images to the actual disk database
|
|
if len(t.getSecKeyCache()) > 0 {
|
|
if t.preimages != nil {
|
|
preimages := make(map[common.Hash][]byte)
|
|
for hk, key := range t.secKeyCache {
|
|
preimages[common.BytesToHash([]byte(hk))] = key
|
|
}
|
|
t.preimages.insertPreimage(preimages)
|
|
}
|
|
t.secKeyCache = make(map[string][]byte)
|
|
}
|
|
// Commit the trie and return its modified nodeset.
|
|
return t.trie.Commit(collectLeaf)
|
|
}
|
|
|
|
// Hash returns the root hash of StateTrie. It does not write to the
|
|
// database and can be used even if the trie doesn't have one.
|
|
func (t *StateTrie) Hash() common.Hash {
|
|
return t.trie.Hash()
|
|
}
|
|
|
|
// Copy returns a copy of StateTrie.
|
|
func (t *StateTrie) Copy() *StateTrie {
|
|
return &StateTrie{
|
|
trie: *t.trie.Copy(),
|
|
preimages: t.preimages,
|
|
secKeyCache: t.secKeyCache,
|
|
}
|
|
}
|
|
|
|
// NodeIterator returns an iterator that returns nodes of the underlying trie.
|
|
// Iteration starts at the key after the given start key.
|
|
func (t *StateTrie) NodeIterator(start []byte) (NodeIterator, error) {
|
|
return t.trie.NodeIterator(start)
|
|
}
|
|
|
|
// MustNodeIterator is a wrapper of NodeIterator and will omit any encountered
|
|
// error but just print out an error message.
|
|
func (t *StateTrie) MustNodeIterator(start []byte) NodeIterator {
|
|
return t.trie.MustNodeIterator(start)
|
|
}
|
|
|
|
// hashKey returns the hash of key as an ephemeral buffer.
|
|
// The caller must not hold onto the return value because it will become
|
|
// invalid on the next call to hashKey or secKey.
|
|
func (t *StateTrie) hashKey(key []byte) []byte {
|
|
h := newHasher(false)
|
|
h.sha.Reset()
|
|
h.sha.Write(key)
|
|
h.sha.Read(t.hashKeyBuf[:])
|
|
returnHasherToPool(h)
|
|
return t.hashKeyBuf[:]
|
|
}
|
|
|
|
// getSecKeyCache returns the current secure key cache, creating a new one if
|
|
// ownership changed (i.e. the current secure trie is a copy of another owning
|
|
// the actual cache).
|
|
func (t *StateTrie) getSecKeyCache() map[string][]byte {
|
|
if t != t.secKeyCacheOwner {
|
|
t.secKeyCacheOwner = t
|
|
t.secKeyCache = make(map[string][]byte)
|
|
}
|
|
return t.secKeyCache
|
|
}
|
|
|
|
func (t *StateTrie) IsVerkle() bool {
|
|
return false
|
|
}
|