go-ethereum/core/vm/eips.go
Jonny Rhea 6e62cc5aa8
core/vm: compute stack operations in place (#35156)
The stack primitives pop by value: pop() returns the 32-byte value
itself, so every popped operand is copied out of the stack arena before
it is used. The result side was already in place, peek returns a pointer
and binary ops write into the new stack top. This PR fixes the operand
side: pointer-returning primitives (popPtr, popPtrPeek, etc), with the
handlers rewritten to read operands directly from their arena slots.
Every popped operand paid the copy, whatever the op went on to do with
it, so this optimization covers the arithmetic and comparison ops as
much as JUMP, MSTORE, SSTORE and RETURN.

The copy is visible in the assembly. On arm64, master's opLt spends four
instructions moving the popped value through the frame, and the
comparison then reads it back from there:

LDP (R5), (R6, R7) ; load words 0 and 1 of the popped value from the
arena
    LDP  16(R5), (R5, R8)            ; load words 2 and 3
STP (R6, R7), vm.~r0-64(SP) ; store words 0 and 1 into a frame slot
    STP  (R5, R8), vm.~r0-48(SP)     ; store words 2 and 3

With popPtrPeek those four instructions are gone, the frame shrinks from
locals=0x58 to locals=0x18, and the function from 336 to 288 bytes. The
compiler cannot remove the copy itself: uint256.Int is a four-element
array, and Go's SSA does not promote arrays longer than one element to
registers, so a by-value pop pays this round trip no matter how far
inlining gets, for LT exactly as for ADD.

The CALL and CREATE families are deliberately not converted: a child
frame reuses the same stack arena, so parent pointers into popped slots
die when the child pushes. The rule is recorded on the primitives:
pointers stay valid until the next push or any sub call. Converting the
call family safely means materializing scalars before the child call,
left for later work with a call-heavy benchmark to justify it.

### Benchmarks

Measured with the benchmark suite from #35144 (the evm-bench contract
workloads and the block import benchmark), which is not part of this
PR's diff. Apple M4 Max, fixed iteration counts, n=10, all p=0.000. B/op
and allocs/op are statistically identical on every benchmark:

| benchmark | master | PR | vs master |
|---|---|---|---|
| Snailtracer | 60.0 ms | 54.1 ms | -9.8% |
| TenThousandHashes | 13.2 ms | 12.2 ms | -7.8% |
| ERC20Transfer | 11.7 ms | 11.0 ms | -5.5% |
| ERC20Mint | 7.49 ms | 7.02 ms | -6.2% |
| ERC20ApprovalTransfer | 8.92 ms | 8.44 ms | -5.4% |

This PR is independent of #35144 but plays nicely with it: the generated
dispatch there splices these handler bodies, so the in-place forms land
in its fast path too, where they measure larger.

### Testing

The rewritten handlers run on the interpreter's only execution path, so
correctness rests on references outside the change:

- **Consensus fixtures.** The full tests package passes: state tests,
the execution-spec families, blockchain tests.
- **Opcode testcases.** The JSON testcases compare individual opcode
results against committed expected values.
- **Tracer fixtures.** The tracetest reference files pin exact log and
return data shapes, covering the rewritten LOG and RETURN paths.
- **Cross-build differential.** A goevmlab campaign running this
branch's evm against master's evm over generated state tests across four
forks (Prague, Cancun, London, Osaka) with full trace comparison:
160,566 tests, zero divergences.

---------

Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
2026-06-16 07:47:05 -05:00

593 lines
17 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

// Copyright 2019 The go-ethereum Authors
// This file is part of the go-ethereum library.
//
// The go-ethereum library is free software: you can redistribute it and/or modify
// it under the terms of the GNU Lesser General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// The go-ethereum library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public License
// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
package vm
import (
"fmt"
"math"
"sort"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/core/tracing"
"github.com/ethereum/go-ethereum/params"
)
var activators = map[int]func(*JumpTable){
5656: enable5656,
6780: enable6780,
3855: enable3855,
3860: enable3860,
3529: enable3529,
3198: enable3198,
2929: enable2929,
2200: enable2200,
1884: enable1884,
1344: enable1344,
1153: enable1153,
4762: enable4762,
7702: enable7702,
7939: enable7939,
8024: enable8024,
7843: enable7843,
8037: enable8037,
}
// EnableEIP enables the given EIP on the config.
// This operation writes in-place, and callers need to ensure that the globally
// defined jump tables are not polluted.
func EnableEIP(eipNum int, jt *JumpTable) error {
enablerFn, ok := activators[eipNum]
if !ok {
return fmt.Errorf("undefined eip %d", eipNum)
}
enablerFn(jt)
return nil
}
func ValidEip(eipNum int) bool {
_, ok := activators[eipNum]
return ok
}
func ActivateableEips() []string {
var nums []string
for k := range activators {
nums = append(nums, fmt.Sprintf("%d", k))
}
sort.Strings(nums)
return nums
}
// enable1884 applies EIP-1884 to the given jump table:
// - Increase cost of BALANCE to 700
// - Increase cost of EXTCODEHASH to 700
// - Increase cost of SLOAD to 800
// - Define SELFBALANCE, with cost GasFastStep (5)
func enable1884(jt *JumpTable) {
// Gas cost changes
jt[SLOAD].constantGas = params.SloadGasEIP1884
jt[BALANCE].constantGas = params.BalanceGasEIP1884
jt[EXTCODEHASH].constantGas = params.ExtcodeHashGasEIP1884
// New opcode
jt[SELFBALANCE] = &operation{
execute: opSelfBalance,
constantGas: GasFastStep,
minStack: minStack(0, 1),
maxStack: maxStack(0, 1),
}
}
func opSelfBalance(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
scope.Stack.get().Set(evm.StateDB.GetBalance(scope.Contract.Address()))
return nil, nil
}
// enable1344 applies EIP-1344 (ChainID Opcode)
// - Adds an opcode that returns the current chains EIP-155 unique identifier
func enable1344(jt *JumpTable) {
// New opcode
jt[CHAINID] = &operation{
execute: opChainID,
constantGas: GasQuickStep,
minStack: minStack(0, 1),
maxStack: maxStack(0, 1),
}
}
// opChainID implements CHAINID opcode
func opChainID(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
scope.Stack.get().SetFromBig(evm.chainConfig.ChainID)
return nil, nil
}
// enable2200 applies EIP-2200 (Rebalance net-metered SSTORE)
func enable2200(jt *JumpTable) {
jt[SLOAD].constantGas = params.SloadGasEIP2200
jt[SSTORE].dynamicGas = gasSStoreEIP2200
}
// enable2929 enables "EIP-2929: Gas cost increases for state access opcodes"
// https://eips.ethereum.org/EIPS/eip-2929
func enable2929(jt *JumpTable) {
jt[SSTORE].dynamicGas = gasSStoreEIP2929
jt[SLOAD].constantGas = 0
jt[SLOAD].dynamicGas = gasSLoadEIP2929
jt[EXTCODECOPY].constantGas = params.WarmStorageReadCostEIP2929
jt[EXTCODECOPY].dynamicGas = gasExtCodeCopyEIP2929
jt[EXTCODESIZE].constantGas = params.WarmStorageReadCostEIP2929
jt[EXTCODESIZE].dynamicGas = gasEip2929AccountCheck
jt[EXTCODEHASH].constantGas = params.WarmStorageReadCostEIP2929
jt[EXTCODEHASH].dynamicGas = gasEip2929AccountCheck
jt[BALANCE].constantGas = params.WarmStorageReadCostEIP2929
jt[BALANCE].dynamicGas = gasEip2929AccountCheck
jt[CALL].constantGas = params.WarmStorageReadCostEIP2929
jt[CALL].dynamicGas = gasCallEIP2929
jt[CALLCODE].constantGas = params.WarmStorageReadCostEIP2929
jt[CALLCODE].dynamicGas = gasCallCodeEIP2929
jt[STATICCALL].constantGas = params.WarmStorageReadCostEIP2929
jt[STATICCALL].dynamicGas = gasStaticCallEIP2929
jt[DELEGATECALL].constantGas = params.WarmStorageReadCostEIP2929
jt[DELEGATECALL].dynamicGas = gasDelegateCallEIP2929
// This was previously part of the dynamic cost, but we're using it as a constantGas
// factor here
jt[SELFDESTRUCT].constantGas = params.SelfdestructGasEIP150
jt[SELFDESTRUCT].dynamicGas = gasSelfdestructEIP2929
}
// enable3529 enabled "EIP-3529: Reduction in refunds":
// - Removes refunds for selfdestructs
// - Reduces refunds for SSTORE
// - Reduces max refunds to 20% gas
func enable3529(jt *JumpTable) {
jt[SSTORE].dynamicGas = gasSStoreEIP3529
jt[SELFDESTRUCT].dynamicGas = gasSelfdestructEIP3529
}
// enable3198 applies EIP-3198 (BASEFEE Opcode)
// - Adds an opcode that returns the current block's base fee.
func enable3198(jt *JumpTable) {
// New opcode
jt[BASEFEE] = &operation{
execute: opBaseFee,
constantGas: GasQuickStep,
minStack: minStack(0, 1),
maxStack: maxStack(0, 1),
}
}
// enable1153 applies EIP-1153 "Transient Storage"
// - Adds TLOAD that reads from transient storage
// - Adds TSTORE that writes to transient storage
func enable1153(jt *JumpTable) {
jt[TLOAD] = &operation{
execute: opTload,
constantGas: params.WarmStorageReadCostEIP2929,
minStack: minStack(1, 1),
maxStack: maxStack(1, 1),
}
jt[TSTORE] = &operation{
execute: opTstore,
constantGas: params.WarmStorageReadCostEIP2929,
minStack: minStack(2, 0),
maxStack: maxStack(2, 0),
}
}
// opTload implements TLOAD opcode
func opTload(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
loc := scope.Stack.peek()
hash := common.Hash(loc.Bytes32())
val := evm.StateDB.GetTransientState(scope.Contract.Address(), hash)
loc.SetBytes(val.Bytes())
return nil, nil
}
// opTstore implements TSTORE opcode
func opTstore(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
if evm.readOnly {
return nil, ErrWriteProtection
}
loc, val := scope.Stack.pop2()
evm.StateDB.SetTransientState(scope.Contract.Address(), loc.Bytes32(), val.Bytes32())
return nil, nil
}
// opBaseFee implements BASEFEE opcode
func opBaseFee(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
scope.Stack.get().SetFromBig(evm.Context.BaseFee)
return nil, nil
}
// enable3855 applies EIP-3855 (PUSH0 opcode)
func enable3855(jt *JumpTable) {
// New opcode
jt[PUSH0] = &operation{
execute: opPush0,
constantGas: GasQuickStep,
minStack: minStack(0, 1),
maxStack: maxStack(0, 1),
}
}
// opPush0 implements the PUSH0 opcode
func opPush0(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
scope.Stack.get().Clear()
return nil, nil
}
// enable3860 enables "EIP-3860: Limit and meter initcode"
// https://eips.ethereum.org/EIPS/eip-3860
func enable3860(jt *JumpTable) {
jt[CREATE].dynamicGas = gasCreateEip3860
jt[CREATE2].dynamicGas = gasCreate2Eip3860
}
// enable5656 enables EIP-5656 (MCOPY opcode)
// https://eips.ethereum.org/EIPS/eip-5656
func enable5656(jt *JumpTable) {
jt[MCOPY] = &operation{
execute: opMcopy,
constantGas: GasFastestStep,
dynamicGas: gasMcopy,
minStack: minStack(3, 0),
maxStack: maxStack(3, 0),
memorySize: memoryMcopy,
}
}
// opMcopy implements the MCOPY opcode (https://eips.ethereum.org/EIPS/eip-5656)
func opMcopy(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
dst, src, length := scope.Stack.pop3()
// These values are checked for overflow during memory expansion calculation
// (the memorySize function on the opcode).
scope.Memory.Copy(dst.Uint64(), src.Uint64(), length.Uint64())
return nil, nil
}
// opBlobHash implements the BLOBHASH opcode
func opBlobHash(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
index := scope.Stack.peek()
if index.LtUint64(uint64(len(evm.TxContext.BlobHashes))) {
blobHash := evm.TxContext.BlobHashes[index.Uint64()]
index.SetBytes32(blobHash[:])
} else {
index.Clear()
}
return nil, nil
}
// opBlobBaseFee implements BLOBBASEFEE opcode
func opBlobBaseFee(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
scope.Stack.get().SetFromBig(evm.Context.BlobBaseFee)
return nil, nil
}
// opCLZ implements the CLZ opcode (count leading zero bits)
func opCLZ(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
x := scope.Stack.peek()
x.SetUint64(256 - uint64(x.BitLen()))
return nil, nil
}
// enable4844 applies EIP-4844 (BLOBHASH opcode)
func enable4844(jt *JumpTable) {
jt[BLOBHASH] = &operation{
execute: opBlobHash,
constantGas: GasFastestStep,
minStack: minStack(1, 1),
maxStack: maxStack(1, 1),
}
}
// enable7939 enables EIP-7939 (CLZ opcode)
func enable7939(jt *JumpTable) {
jt[CLZ] = &operation{
execute: opCLZ,
constantGas: GasFastStep,
minStack: minStack(1, 1),
maxStack: maxStack(1, 1),
}
}
// enable7516 applies EIP-7516 (BLOBBASEFEE opcode)
func enable7516(jt *JumpTable) {
jt[BLOBBASEFEE] = &operation{
execute: opBlobBaseFee,
constantGas: GasQuickStep,
minStack: minStack(0, 1),
maxStack: maxStack(0, 1),
}
}
// enable6780 applies EIP-6780 (deactivate SELFDESTRUCT)
func enable6780(jt *JumpTable) {
jt[SELFDESTRUCT] = &operation{
execute: opSelfdestruct6780,
dynamicGas: gasSelfdestructEIP3529,
constantGas: params.SelfdestructGasEIP150,
minStack: minStack(1, 0),
maxStack: maxStack(1, 0),
}
}
// enable8024 applies EIP-8024 (DUPN, SWAPN, EXCHANGE)
func enable8024(jt *JumpTable) {
jt[DUPN] = &operation{
execute: opDupN,
constantGas: GasFastestStep,
minStack: minStack(1, 0),
maxStack: maxStack(0, 1),
}
jt[SWAPN] = &operation{
execute: opSwapN,
constantGas: GasFastestStep,
minStack: minStack(2, 0),
maxStack: maxStack(0, 0),
}
jt[EXCHANGE] = &operation{
execute: opExchange,
constantGas: GasFastestStep,
minStack: minStack(2, 0),
maxStack: maxStack(0, 0),
}
}
func opExtCodeCopyEIP4762(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
var (
stack = scope.Stack
a, memOffset, codeOffset, length = stack.pop4()
)
uint64CodeOffset, overflow := codeOffset.Uint64WithOverflow()
if overflow {
uint64CodeOffset = math.MaxUint64
}
addr := common.Address(a.Bytes20())
code := evm.StateDB.GetCode(addr)
paddedCodeCopy, copyOffset, nonPaddedCopyLength := getDataAndAdjustedBounds(code, uint64CodeOffset, length.Uint64())
consumed, wanted := evm.AccessEvents.CodeChunksRangeGas(addr, copyOffset, nonPaddedCopyLength, uint64(len(code)), false, scope.Contract.Gas.RegularGas)
scope.Contract.chargeRegular(consumed, evm.Config.Tracer, tracing.GasChangeUnspecified)
if consumed < wanted {
return nil, ErrOutOfGas
}
scope.Memory.Set(memOffset.Uint64(), length.Uint64(), paddedCodeCopy)
return nil, nil
}
// opPush1EIP4762 handles the special case of PUSH1 opcode for EIP-4762, which
// need not worry about the adjusted bound logic when adding the PUSHDATA to
// the list of access events.
func opPush1EIP4762(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
var (
codeLen = uint64(len(scope.Contract.Code))
elem = scope.Stack.get()
)
*pc += 1
if *pc < codeLen {
elem.SetUint64(uint64(scope.Contract.Code[*pc]))
if !scope.Contract.IsDeployment && !scope.Contract.IsSystemCall && *pc%31 == 0 {
// touch next chunk if PUSH1 is at the boundary. if so, *pc has
// advanced past this boundary.
contractAddr := scope.Contract.Address()
consumed, wanted := evm.AccessEvents.CodeChunksRangeGas(contractAddr, *pc+1, uint64(1), uint64(len(scope.Contract.Code)), false, scope.Contract.Gas.RegularGas)
scope.Contract.chargeRegular(wanted, evm.Config.Tracer, tracing.GasChangeUnspecified)
if consumed < wanted {
return nil, ErrOutOfGas
}
}
} else {
elem.Clear()
}
return nil, nil
}
func makePushEIP4762(size uint64, pushByteSize int) executionFunc {
return func(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
var (
codeLen = len(scope.Contract.Code)
start = min(codeLen, int(*pc+1))
end = min(codeLen, start+pushByteSize)
)
scope.Stack.get().SetBytes(
common.RightPadBytes(
scope.Contract.Code[start:end],
pushByteSize,
))
if !scope.Contract.IsDeployment && !scope.Contract.IsSystemCall {
contractAddr := scope.Contract.Address()
consumed, wanted := evm.AccessEvents.CodeChunksRangeGas(contractAddr, uint64(start), uint64(pushByteSize), uint64(len(scope.Contract.Code)), false, scope.Contract.Gas.RegularGas)
scope.Contract.chargeRegular(consumed, evm.Config.Tracer, tracing.GasChangeUnspecified)
if consumed < wanted {
return nil, ErrOutOfGas
}
}
*pc += size
return nil, nil
}
}
func enable4762(jt *JumpTable) {
jt[SSTORE] = &operation{
dynamicGas: gasSStore4762,
execute: opSstore,
minStack: minStack(2, 0),
maxStack: maxStack(2, 0),
}
jt[SLOAD] = &operation{
dynamicGas: gasSLoad4762,
execute: opSload,
minStack: minStack(1, 1),
maxStack: maxStack(1, 1),
}
jt[BALANCE] = &operation{
execute: opBalance,
dynamicGas: gasBalance4762,
minStack: minStack(1, 1),
maxStack: maxStack(1, 1),
}
jt[EXTCODESIZE] = &operation{
execute: opExtCodeSize,
dynamicGas: gasExtCodeSize4762,
minStack: minStack(1, 1),
maxStack: maxStack(1, 1),
}
jt[EXTCODEHASH] = &operation{
execute: opExtCodeHash,
dynamicGas: gasExtCodeHash4762,
minStack: minStack(1, 1),
maxStack: maxStack(1, 1),
}
jt[EXTCODECOPY] = &operation{
execute: opExtCodeCopyEIP4762,
dynamicGas: gasExtCodeCopyEIP4762,
minStack: minStack(4, 0),
maxStack: maxStack(4, 0),
memorySize: memoryExtCodeCopy,
}
jt[CODECOPY] = &operation{
execute: opCodeCopy,
constantGas: GasFastestStep,
dynamicGas: gasCodeCopyEip4762,
minStack: minStack(3, 0),
maxStack: maxStack(3, 0),
memorySize: memoryCodeCopy,
}
jt[SELFDESTRUCT] = &operation{
execute: opSelfdestruct6780,
dynamicGas: gasSelfdestructEIP4762,
constantGas: params.SelfdestructGasEIP150,
minStack: minStack(1, 0),
maxStack: maxStack(1, 0),
}
jt[CREATE] = &operation{
execute: opCreate,
constantGas: params.CreateNGasEip4762,
dynamicGas: gasCreateEip3860,
minStack: minStack(3, 1),
maxStack: maxStack(3, 1),
memorySize: memoryCreate,
}
jt[CREATE2] = &operation{
execute: opCreate2,
constantGas: params.CreateNGasEip4762,
dynamicGas: gasCreate2Eip3860,
minStack: minStack(4, 1),
maxStack: maxStack(4, 1),
memorySize: memoryCreate2,
}
jt[CALL] = &operation{
execute: opCall,
dynamicGas: gasCallEIP4762,
minStack: minStack(7, 1),
maxStack: maxStack(7, 1),
memorySize: memoryCall,
}
jt[CALLCODE] = &operation{
execute: opCallCode,
dynamicGas: gasCallCodeEIP4762,
minStack: minStack(7, 1),
maxStack: maxStack(7, 1),
memorySize: memoryCall,
}
jt[STATICCALL] = &operation{
execute: opStaticCall,
dynamicGas: gasStaticCallEIP4762,
minStack: minStack(6, 1),
maxStack: maxStack(6, 1),
memorySize: memoryStaticCall,
}
jt[DELEGATECALL] = &operation{
execute: opDelegateCall,
dynamicGas: gasDelegateCallEIP4762,
minStack: minStack(6, 1),
maxStack: maxStack(6, 1),
memorySize: memoryDelegateCall,
}
jt[PUSH1] = &operation{
execute: opPush1EIP4762,
constantGas: GasFastestStep,
minStack: minStack(0, 1),
maxStack: maxStack(0, 1),
}
for i := 1; i < 32; i++ {
jt[PUSH1+OpCode(i)] = &operation{
execute: makePushEIP4762(uint64(i+1), i+1),
constantGas: GasFastestStep,
minStack: minStack(0, 1),
maxStack: maxStack(0, 1),
}
}
}
// enable7702 the EIP-7702 changes to support delegation designators.
func enable7702(jt *JumpTable) {
jt[CALL].dynamicGas = gasCallEIP7702
jt[CALLCODE].dynamicGas = gasCallCodeEIP7702
jt[STATICCALL].dynamicGas = gasStaticCallEIP7702
jt[DELEGATECALL].dynamicGas = gasDelegateCallEIP7702
}
// opSlotNum enables the SLOTNUM opcode
func opSlotNum(pc *uint64, evm *EVM, scope *ScopeContext) ([]byte, error) {
scope.Stack.get().SetUint64(evm.Context.SlotNum)
return nil, nil
}
// enable7843 enables the SLOTNUM opcode as specified in EIP-7843.
func enable7843(jt *JumpTable) {
jt[SLOTNUM] = &operation{
execute: opSlotNum,
constantGas: GasQuickStep,
minStack: minStack(0, 1),
maxStack: maxStack(0, 1),
}
}
// enable8037 enables the multidimensional-metering as specified in EIP-8037.
func enable8037(jt *JumpTable) {
jt[CREATE].constantGas = params.CreateGasAmsterdam
jt[CREATE2].constantGas = params.CreateGasAmsterdam
jt[SELFDESTRUCT].dynamicGas = gasSelfdestruct8037
jt[SSTORE].dynamicGas = gasSStore8037
}