go-ethereum/core/vm/interpreter.go
Guillaume Ballet ac152143e1 Verkle tree-based state with overlay transition
Squash the main verkle PR ahead of rebase

don't call Bytes() in GetTreeKey (#137)

trie: avoid endianness conversion in GetTreeKey (#140)

* trie/utils: add concrete expected value in trie key generation test

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* mod: update to latest go-verkle

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* trie/utils: avoid endianness conversions

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* apply review changes & update to official go-verkle version

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

upgrade go-verkle to CoW version and get TestProcessVerkle to build (#138)

updating ci to use self-hosted machine (#143)

fix: storage offset in non-header group + reuse of value buffer (#145)

dedup call to ChunkifyCode, same as replay branch (#156)

* dedup call to ChunkifyCode, same as replay branch

* fix some linter issues

fix code offset in tree update (#157)

fix REVERT in state processor test execution (#158)

* fix code offset in tree update

* fix REVERT in test execution

save on key hashing: lump code size update with first code chunk group (#159)

fix code chunk key calculation and storage key calculation (#161)

* fix codeKey calculation

* Remove

* fix storageOffset

* fix the fix to the fix to the offset fix

* Remove copy/pasted, unused code in test

* fix linter

---------

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

fix: infinite loop when calling extcodecopy on empty code (#151)

upgrade to latest go-verkle

fix: only update code in the tree if it's dirty (#174)

fix: read-touch the code size and Keccak of the origin (#175)

List of changes for converting a sepolia database (#182)

* naive conversion rebased on top of beverly hills

* changes for the sepolia shadow fork conversion

* fixes to please the linter

* fixes to please the linter

Unified point cache (#180)

* Unified point cache

* Use cache for Try*Account

* alter Trie interface to use caching for slots (#181)

* alter Trie interface to use caching for slots

* fix: use a lock to protect the point cache (#185)

* use fastest non-master go-verkle version & pull trie/Verkle.go changes to use new api (#184)

* mod: update to fastest go-verkle version today

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* trie/verkle: use new batch serialization api

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com>

* fix: TryDelete signature in unit tests

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com>

trie/utils: fix potential overflow (#191)

* trie/utils: fix potential overflow

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* trie/utils: receive storage key as a byte slice

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* revert formatter changes

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* trie/utils: fix mod 256

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

trie/utils: fix incorrect bigint assignment (#193)

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

upgrade precomp link to fix CI

fix: add missing code size&keccak leaves in empty accounts (#192)

fixes to use the latest go-verkle@master (#197)

* fixes to use the latest go-verkle@master

* linter fixes

* linter fixes for tests

* fix: use jsign's go-verkle fix

refactor: remove unused (*StateDB).GetXLittleEndian methods (#204)

fix gas accounting issue in state_processor_test.go (#207)

update go-verkle not to use StatelessNode anymore (#206)

* update go-verkle not to use StatelessNode anymore

* update go-verkle to latest

refactor: move verkle gas accounting to its own block in TransitionDB (#208)

fix a panic in deserializeVerkleProof if GetProofItems returns a nil ProofElements

use the cachingDB instead of a custom VerkleDB (#209)

* use the cachingDB instead of a custom VerkleDB

* fix stack trace in LES

remove holiman from CODEOWNERS as he gets too many emails

read from tree in state object if the snapshot is nil (#205)

add missing error checks for the root node type (#214)

implement OpenStorageTrie for verkle trees (#210)

* implement OpenStorageTrie for verkle trees

* add a few comments for future maintenance

* fix linter issue

fix: copy balance leaf to new buffer in TryGetAccount (#217)

implement some heretofore unimplemented iterator methods (#219)

params: move verkle params to their own file (#228)

fix: proper number of chunk evals (#215)

overlay transition (#244)

* overlay transition

Fix some bugs identified in the code review

Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com>

Include base -> overlay key-values migration logic (#199)

* mod: add go-verkle version with key-value migration new apis

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* core/stateprocessor: use constant for max number of migrated key-values

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* core: add base->overlay key-values migration logic

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* core: fix some compiler errors

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* trie: consider removing transition trie api in the future

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* mod: use latest go-verkle

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

fix some unit tests errors

get convresion block from file

fix compilation issues

fix initialization issue in migrator

fix: changes needed to run the first 28 blocks

important sutff: fix the banner

fix: use nonce instead of balance in nonce leaf (#202)

fixes for performing the overlay transition (#203)

* fixes for performing the overlay transition

* fixes for the full replay

* fix: deletion-and-recreation of EoA

* fixes to replay 2M+ blocks

* upgrade to go-verkle@master

* fix: proper number of chunk evals

* rewrite conversion loop to fix known issues

changes to make replay work with the overlay method (#216)

* fixes for performing the overlay transition

fixes for the full replay

fix: deletion-and-recreation of EoA

fixes to replay 2M+ blocks

upgrade to go-verkle@master

fix: proper number of chunk evals

rewrite conversion loop to fix known issues

changes to make replay work with the overlay method

fixes to replay 2M+ blocks

update to latest go-verkle@master

* use a PBSS-like scheme for internal nodes (#221)

* use a PBSS-like scheme for internal nodes

* a couple of fixes coming from debugging replay

* fix: use an error to notify the transition tree that a deleted account was found in the overlay tree (#222)

* fixes for pbss replay (#227)

* fixes for pbss replay

* trie/verkle: use capped batch size (#229)

* trie/verkle: use capped batch size

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* trie/verkle: avoid path variable allocation per db.Put

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* don't keep more than 32 state root conversions in RAM (#230)

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

* cleanup some code

* mod: update go-verkle

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* re-enable snapshot (#231)

* re-enable cancun block / snapshot (#226)

* clear storage conversion key upon translating account (#234)

* clear storage conversion key upon translating account

* mod: use latest go-verkle

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com>

* fix: self-deadlock with translated root map mutex (#236)

* return compressed commitment as root commitment (#237)

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com>

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com>

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com>

fix first panic in *TransitionTrie.Copy()

upgrade go-verkle to latest master

mod: update go-verkle (#239)

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

core: print state root every 100 blocks (#240)

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

fix: only Commit the account trie (#242)

fixes to get TestProcessVerkle to work with the overlay branch (#238)

* fixes to get TestProcessVerkle to work with the overlay branch

* fix all panics in verkle state processor test

* fix proof verification

move transition management to cachingDB

* fix: mark the verkle transition as started if it's ended without being started

* fix the verkle state processing test

* fix linter errors

* Add a function to clear verkle params for replay

* fix: handle TransitionTrie in OpenStorageTrie

* fix linter issue

* fix the deleted account error (#247)

* code cleanup (#248)

* fix: don't error on a missing conversion.txt (#249)

* Overlay Tree preimages exporting and usage (#246)

* export overlay preimages tool

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* use preimages flat file in overlay tree migration logic

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* cmd/geth: add --roothash to overlay tree preimage exporting command

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* cleanup

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* review feedback

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* fix: reduce the PR footprint (#250)

* fix: don't fail when preimages.bin is missing (#251)

* fix: don't fail when preimages.bin is missing

* fix: don't open the preimages file when outside of transition

---------

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com>

review changes

remove replay-specific code
2024-05-08 13:24:03 +02:00

271 lines
9.5 KiB
Go

// Copyright 2014 The go-ethereum Authors
// This file is part of the go-ethereum library.
//
// The go-ethereum library is free software: you can redistribute it and/or modify
// it under the terms of the GNU Lesser General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// The go-ethereum library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public License
// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
package vm
import (
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/math"
"github.com/ethereum/go-ethereum/crypto"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/trie"
"github.com/ethereum/go-ethereum/trie/utils"
"github.com/gballet/go-verkle"
"github.com/holiman/uint256"
)
// Config are the configuration options for the Interpreter
type Config struct {
Tracer EVMLogger // Opcode logger
NoBaseFee bool // Forces the EIP-1559 baseFee to 0 (needed for 0 price calls)
EnablePreimageRecording bool // Enables recording of SHA3/keccak preimages
ExtraEips []int // Additional EIPS that are to be enabled
}
// ScopeContext contains the things that are per-call, such as stack and memory,
// but not transients like pc and gas
type ScopeContext struct {
Memory *Memory
Stack *Stack
Contract *Contract
}
// EVMInterpreter represents an EVM interpreter
type EVMInterpreter struct {
evm *EVM
table *JumpTable
hasher crypto.KeccakState // Keccak256 hasher instance shared across opcodes
hasherBuf common.Hash // Keccak256 hasher result array shared aross opcodes
readOnly bool // Whether to throw on stateful modifications
returnData []byte // Last CALL's return data for subsequent reuse
}
// NewEVMInterpreter returns a new instance of the Interpreter.
func NewEVMInterpreter(evm *EVM) *EVMInterpreter {
// If jump table was not initialised we set the default one.
var table *JumpTable
switch {
case evm.chainRules.IsCancun:
table = &cancunInstructionSet
case evm.chainRules.IsShanghai:
table = &shanghaiInstructionSet
case evm.chainRules.IsMerge:
table = &mergeInstructionSet
case evm.chainRules.IsLondon:
table = &londonInstructionSet
case evm.chainRules.IsBerlin:
table = &berlinInstructionSet
case evm.chainRules.IsIstanbul:
table = &istanbulInstructionSet
case evm.chainRules.IsConstantinople:
table = &constantinopleInstructionSet
case evm.chainRules.IsByzantium:
table = &byzantiumInstructionSet
case evm.chainRules.IsEIP158:
table = &spuriousDragonInstructionSet
case evm.chainRules.IsEIP150:
table = &tangerineWhistleInstructionSet
case evm.chainRules.IsHomestead:
table = &homesteadInstructionSet
default:
table = &frontierInstructionSet
}
var extraEips []int
if len(evm.Config.ExtraEips) > 0 {
// Deep-copy jumptable to prevent modification of opcodes in other tables
table = copyJumpTable(table)
}
for _, eip := range evm.Config.ExtraEips {
if err := EnableEIP(eip, table); err != nil {
// Disable it, so caller can check if it's activated or not
log.Error("EIP activation failed", "eip", eip, "error", err)
} else {
extraEips = append(extraEips, eip)
}
}
evm.Config.ExtraEips = extraEips
return &EVMInterpreter{evm: evm, table: table}
}
// Run loops and evaluates the contract's code with the given input data and returns
// the return byte-slice and an error if one occurred.
//
// It's important to note that any errors returned by the interpreter should be
// considered a revert-and-consume-all-gas operation except for
// ErrExecutionReverted which means revert-and-keep-gas-left.
func (in *EVMInterpreter) Run(contract *Contract, input []byte, readOnly bool) (ret []byte, err error) {
// Increment the call depth which is restricted to 1024
in.evm.depth++
defer func() { in.evm.depth-- }()
// Make sure the readOnly is only set if we aren't in readOnly yet.
// This also makes sure that the readOnly flag isn't removed for child calls.
if readOnly && !in.readOnly {
in.readOnly = true
defer func() { in.readOnly = false }()
}
// Reset the previous call's return data. It's unimportant to preserve the old buffer
// as every returning call will return new data anyway.
in.returnData = nil
// Don't bother with the execution if there's no code.
if len(contract.Code) == 0 {
return nil, nil
}
var (
op OpCode // current opcode
mem = NewMemory() // bound memory
stack = newstack() // local stack
callContext = &ScopeContext{
Memory: mem,
Stack: stack,
Contract: contract,
}
// For optimisation reason we're using uint64 as the program counter.
// It's theoretically possible to go above 2^64. The YP defines the PC
// to be uint256. Practically much less so feasible.
pc = uint64(0) // program counter
cost uint64
// copies used by tracer
pcCopy uint64 // needed for the deferred EVMLogger
gasCopy uint64 // for EVMLogger to log gas remaining before execution
logged bool // deferred EVMLogger should ignore already logged steps
res []byte // result of the opcode execution function
debug = in.evm.Config.Tracer != nil
chunkEvals [][]byte
)
// Don't move this deferred function, it's placed before the capturestate-deferred method,
// so that it get's executed _after_: the capturestate needs the stacks before
// they are returned to the pools
defer func() {
returnStack(stack)
}()
contract.Input = input
if debug {
defer func() {
if err != nil {
if !logged {
in.evm.Config.Tracer.CaptureState(pcCopy, op, gasCopy, cost, callContext, in.returnData, in.evm.depth, err)
} else {
in.evm.Config.Tracer.CaptureFault(pcCopy, op, gasCopy, cost, callContext, in.evm.depth, err)
}
}
}()
}
// Evaluate one address per group of 256, 31-byte chunks
if in.evm.chainRules.IsCancun && !contract.IsDeployment {
contract.Chunks = trie.ChunkifyCode(contract.Code)
// number of extra stems to evaluate after the header stem
extraEvals := (len(contract.Chunks) + 127) / verkle.NodeWidth
chunkEvals = make([][]byte, extraEvals+1)
for i := 1; i < extraEvals+1; i++ {
chunkEvals[i] = utils.GetTreeKeyCodeChunkWithEvaluatedAddress(contract.AddressPoint(), uint256.NewInt(uint64(i)*256))
}
// Header account is already known, it's the header account
chunkEvals[0] = utils.GetTreeKeyVersionWithEvaluatedAddress(contract.AddressPoint())
}
// The Interpreter main run loop (contextual). This loop runs until either an
// explicit STOP, RETURN or SELFDESTRUCT is executed, an error occurred during
// the execution of one of the operations or until the done flag is set by the
// parent context.
for {
if debug {
// Capture pre-execution values for tracing.
logged, pcCopy, gasCopy = false, pc, contract.Gas
}
if contract.Chunks != nil {
// if the PC ends up in a new "chunk" of verkleized code, charge the
// associated costs.
contract.Gas -= touchChunkOnReadAndChargeGas(contract.Chunks, pc, chunkEvals, contract.Code, in.evm.TxContext.Accesses, contract.IsDeployment)
}
// Get the operation from the jump table and validate the stack to ensure there are
// enough stack items available to perform the operation.
op = contract.GetOp(pc)
operation := in.table[op]
cost = operation.constantGas // For tracing
// Validate stack
if sLen := stack.len(); sLen < operation.minStack {
return nil, &ErrStackUnderflow{stackLen: sLen, required: operation.minStack}
} else if sLen > operation.maxStack {
return nil, &ErrStackOverflow{stackLen: sLen, limit: operation.maxStack}
}
if !contract.UseGas(cost) {
return nil, ErrOutOfGas
}
if operation.dynamicGas != nil {
// All ops with a dynamic memory usage also has a dynamic gas cost.
var memorySize uint64
// calculate the new memory size and expand the memory to fit
// the operation
// Memory check needs to be done prior to evaluating the dynamic gas portion,
// to detect calculation overflows
if operation.memorySize != nil {
memSize, overflow := operation.memorySize(stack)
if overflow {
return nil, ErrGasUintOverflow
}
// memory is expanded in words of 32 bytes. Gas
// is also calculated in words.
if memorySize, overflow = math.SafeMul(toWordSize(memSize), 32); overflow {
return nil, ErrGasUintOverflow
}
}
// Consume the gas and return an error if not enough gas is available.
// cost is explicitly set so that the capture state defer method can get the proper cost
var dynamicCost uint64
dynamicCost, err = operation.dynamicGas(in.evm, contract, stack, mem, memorySize)
cost += dynamicCost // for tracing
if err != nil || !contract.UseGas(dynamicCost) {
return nil, ErrOutOfGas
}
// Do tracing before memory expansion
if debug {
in.evm.Config.Tracer.CaptureState(pc, op, gasCopy, cost, callContext, in.returnData, in.evm.depth, err)
logged = true
}
if memorySize > 0 {
mem.Resize(memorySize)
}
} else if debug {
in.evm.Config.Tracer.CaptureState(pc, op, gasCopy, cost, callContext, in.returnData, in.evm.depth, err)
logged = true
}
// execute the operation
res, err = operation.execute(&pc, in, callContext)
if err != nil {
break
}
pc++
}
if err == errStopToken {
err = nil // clear stop token error
}
return res, err
}