I recently went on a longer flight and started profiling the geth block
production pipeline.
This PR contains a bunch of individual fixes split into separate
commits.
I can drop some if necessary.
Benchmarking is not super easy, the benchmark I wrote is a bit
non-deterministic.
I will try to write a better benchmark later
```
goos: linux
goarch: amd64
pkg: github.com/ethereum/go-ethereum/miner
cpu: Intel(R) Core(TM) Ultra 7 155U
│ /tmp/old.txt │ /tmp/new.txt │
│ sec/op │ sec/op vs base │
BuildPayload-14 141.5µ ± 3% 146.0µ ± 6% ~ (p=0.346 n=200)
│ /tmp/old.txt │ /tmp/new.txt │
│ B/op │ B/op vs base │
BuildPayload-14 188.2Ki ± 4% 177.4Ki ± 4% -5.71% (p=0.018 n=200)
│ /tmp/old.txt │ /tmp/new.txt │
│ allocs/op │ allocs/op vs base │
BuildPayload-14 2.703k ± 4% 2.453k ± 5% -9.25% (p=0.000 n=200)
```
The core part of this PR that we need to adopt is to move the code and
nonce change hook invocations to occur at tx finalization, instead of
when the selfdestruct opcode is called.
Additionally:
* remove `SelfDestruct6780` now that it is essentially the same as
`SelfDestruct` just gated by `is new contract`
* don't duplicate `BalanceIncreaseSelfdestruct` (transfer to recipient
of selfdestruct) in the hooked statedb and in the opcode handler for the
selfdestruct opcode.
* balance is burned immediately when the beneficiary of the selfdestruct
is the sender, and the contract was created in the same transaction.
Previously we emit two balance increases to the recipient (see above
point), and a balance decrease from the sender.
---------
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
We still default to legacy txes for methods like eth_sendTransaction,
eth_signTransaction. We can default to 0x2 and if someone would like to
stay on legacy they can do so by setting the `gasPrice` field.
cc @deffrian
While updating to latest Geth, I noticed `OnCodeChangeV2` was not
properly handled in `SelfDestruct/6780`, this PR fixes this and bring a
unit test. Let me know if it's deemed more approriate to merge the tests
with the other one.
This pr implements https://github.com/ethereum/go-ethereum/issues/32733
to make StateProcessor more customisable.
## Compatibility notes
This introduces a breaking change to users using geth EVM as a library.
The `NewStateProcessor` function now takes one parameter which has the
chainConfig embedded instead of 2 parameters.
Introduces a new tracer which returns the preimages
of evm KECCAK256 hashes.
See #32570.
---------
Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
before:
go test -run=^$ -bench=. ./eth/... 827.57s user 23.80s system 361% cpu
3:55.49 total
after:
go test -run=^$ -bench=. ./eth/... 281.62s user 13.62s system 245% cpu
2:00.49 total
This disables the tx gaslimit cap for eth_call and related RPC operations.
I don't like how this fix works. Ideally we'd be checking the tx
gaslimit somewhere else, like in the block validator, or any other place
that considers block transactions. Doing the check in StateTransition
means it affects all possible ways of executing a message.
The challenge is finding a place for this check that also triggers
correctly in tests where it is wanted. So for now, we are just combining
this with the EOA sender check for transactions. Both are disabled for
call-type messages.
Fixes a prestateTracer test case covering 7702 delegation.
---------
Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
The TestTraceChain function was missing a defer backend.teardown() call,
which is required to properly release blockchain resources after test
completion.
---------
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
**Problem:** Including full account code in prestateTracer response
significantly increases response payload size.
**Solution:** Add codeHash field to the response. This will allow
client-side bytecode caching and is a non-breaking change.
**Note:** codeHash for EoAs is excluded to save space.
---------
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Fixes#32175.
This fixes the scenario where the blockhash opcode would return 0x0
during RPC simulations when using BlockOverrides with a future block
number. The root cause was that BlockOverrides.Apply() only modified the
vm.BlockContext, but GetHashFn() depends on the actual
types.Header.Number to resolve valid historical block hashes. This
caused a mismatch and resulted in incorrect behavior during trace and
call simulations.
---------
Co-authored-by: shantichanal <158101918+shantichanal@users.noreply.github.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
Improves the SSTORE gas calculation a bit. Previously we would pull up
the state object twice. This is okay for existing objects, since they
are cached, however non-existing objects are not cached, thus we needed
to go through all 128 diff layers as well as the disk layer twice, just
for the gas calculation
```
goos: linux
goarch: amd64
pkg: github.com/ethereum/go-ethereum/core/vm
cpu: AMD Ryzen 9 5900X 12-Core Processor
│ /tmp/old.txt │ /tmp/new.txt │
│ sec/op │ sec/op vs base │
Interpreter-24 1118.0n ± 2% 602.8n ± 1% -46.09% (p=0.000 n=10)
```
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
In this pull request, the original `CacheConfig` has been renamed to `BlockChainConfig`.
Over time, more fields have been added to `CacheConfig` to support
blockchain configuration. Such as `ChainHistoryMode`, which clearly extends
beyond just caching concerns.
Additionally, adding new parameters to the blockchain constructor has
become increasingly complicated, since it’s initialized across multiple
places in the codebase. A natural solution is to consolidate these arguments
into a dedicated configuration struct.
As a result, the existing `CacheConfig` has been redefined as `BlockChainConfig`.
Some parameters, such as `VmConfig`, `TxLookupLimit`, and `ChainOverrides`
have been moved into `BlockChainConfig`. Besides, a few fields in `BlockChainConfig`
were renamed, specifically:
- `TrieCleanNoPrefetch` -> `NoPrefetch`
- `TrieDirtyDisabled` -> `ArchiveMode`
Notably, this change won't affect the command line flags or the toml
configuration file. It's just an internal refactoring and fully backward-compatible.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
The prestateTracer had the intention of excluding accounts that were
empty prior to execution from the prestate. This was being done only for
created contracts. This PR makes it so all such empty accounts are
excluded. This behavior is configurable using the `includeEmpty: true`
flag introduced in #31855.
---------
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
This PR introduces a flag that enables returning of newly created state
objects in the prestateTracer.
**Rationale**
Having this information is useful because local execution can more
easily distinguish between newly created objects and system contracts.
---------
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
This pull request introduces a new test suite in workload framework, for
transaction tracing.
**test generation**
`go run . tracegen --trace-tests trace-test.json http://host:8545`
and you can choose to store the trace result in a specific folder
`go run . tracegen --trace-tests trace-test.json --trace-output
./trace-result http://host:8545`
**test run**
`./workload test -run Trace/Transaction --trace-invalid ./trace-invalid
http://host:8545`
The mismatched trace result will be saved in the specific folder for
further investigation.
This PR introduces a new native tracer for AA bundlers. Bundlers participating in the alternative
mempool will need to validate userops. This tracer will return sufficient information for them to
decide whether griefing is possible. Resolves#30546
---------
Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
This PR introduces an allocation-free version of the
Transaction.EffectiveGasTip method to improve performance by reducing
memory allocations.
## Changes
- Added a new `EffectiveGasTipInto` method that accepts a destination
parameter to avoid memory allocations
- Refactored the existing `EffectiveGasTip` method to use the new
allocation-free implementation
- Updated related methods (`EffectiveGasTipValue`, `EffectiveGasTipCmp`,
`EffectiveGasTipIntCmp`) to use the allocation-free approach
- Added tests and benchmarks to verify correctness and measure
performance improvements
## Motivation
In high-transaction-volume environments, the `EffectiveGasTip` method is
called frequently. Reducing memory allocations in this method decreases
garbage collection pressure and improves overall system performance.
## Benchmark Results
As-Is
BenchmarkEffectiveGasTip/Original-10 42089140 27.45 ns/op 8 B/op 1
allocs/op
To-Be
BenchmarkEffectiveGasTip/IntoMethod-10 72353263 16.73 ns/op 0 B/op 0
allocs/op
## Summary of Improvements
- **Performance**: ~39% faster execution (27.45 ns/op → 16.73 ns/op)
- **Memory**: Eliminated all allocations (8 B/op → 0 B/op)
- **Allocation count**: Reduced from 1 to 0 allocations per operation
This optimization follows the same pattern successfully applied to other
methods in the codebase, maintaining API compatibility while improving
performance.
## Safety & Compatibility
This optimization has no side effects or adverse impacts because:
- It maintains functional equivalence as confirmed by comprehensive
tests
- It preserves API compatibility with existing callers
- It follows clear memory ownership patterns with the destination
parameter
- It maintains thread safety by only modifying the caller-provided
destination parameter
This optimization follows the same pattern successfully applied to other
methods in the codebase, providing better performance without
compromising stability or correctness.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
Fixes methods debug_standardTraceBlockToFile
and debug_standardTraceBadBlockToFile which were
outputting empty files.
---------
Co-authored-by: maskpp <maskpp266@gmail.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Fixes https://github.com/ethereum/go-ethereum/issues/31732.
This logic was removed in the recent refactoring in the txindexer to
handle history cutoff (#31393). It was first introduced in this PR:
https://github.com/ethereum/go-ethereum/pull/28908.
I have tested it and it works as an alternative to #31745.
This PR packs 3 changes to the flow of fetching txs from the API:
- It caches the indexer tail after each run is over to avoid hitting the
db all the time as was done originally in #28908.
- Changes `backend.GetTransaction`. It doesn't return an error anymore
when tx indexer is in progress. It shifts the responsibility to the
caller to check the progress. The reason is that in most cases we anyway
check the txpool for the tx. If it was indeed a pending tx we can avoid
the indexer progress check.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This pull request fixes a broken unit test
```
=== CONT TestTracingWithOverrides
api_test.go:1012: result: {"gas":21167,"failed":false,"returnValue":"0x0000000000000000000000000000000000000000000000000000000000000002","structLogs":[{"pc":0,"op":"PUSH1","gas":24978860,"gasCost":3,"depth":1,"stack":[]},{"pc":2,"op":"CALLDATALOAD","gas":24978857,"gasCost":3,"depth":1,"stack":["0x0"]},{"pc":3,"op":"PUSH1","gas":24978854,"gasCost":3,"depth":1,"stack":["0x1"]},{"pc":5,"op":"ADD","gas":24978851,"gasCost":3,"depth":1,"stack":["0x1","0x1"]},{"pc":6,"op":"PUSH1","gas":24978848,"gasCost":3,"depth":1,"stack":["0x2"]},{"pc":8,"op":"MSTORE","gas":24978845,"gasCost":6,"depth":1,"stack":["0x2","0x0"]},{"pc":9,"op":"PUSH1","gas":24978839,"gasCost":3,"depth":1,"stack":[]},{"pc":11,"op":"PUSH1","gas":24978836,"gasCost":3,"depth":1,"stack":["0x20"]},{"pc":13,"op":"RETURN","gas":24978833,"gasCost":0,"depth":1,"stack":["0x20","0x0"]}]}
api_test.go:1013: test 10, result mismatch, have
{21167 false 0x0000000000000000000000000000000000000000000000000000000000000002}
, want
{21167 false 0000000000000000000000000000000000000000000000000000000000000002}
api_test.go:1012: result: {"gas":25664,"failed":false,"returnValue":"0x000000000000000000000000c6e93f4c1920eaeaa1e699f76a7a8c18e3056074","structLogs":[]}
api_test.go:1013: test 11, result mismatch, have
{25664 false 0x000000000000000000000000c6e93f4c1920eaeaa1e699f76a7a8c18e3056074}
, want
{25664 false 000000000000000000000000c6e93f4c1920eaeaa1e699f76a7a8c18e3056074}
```
This is a **breaking change** to the opcode tracer. The top-level
`returnValue` field of a trace will be now hex-encoded. If the return
data is empty, this field will contain "0x".
Fixes#31196
This PR does a few things including:
- Remove `ContractRef` interface
- Remove `vm.AccountRef` which implements `ContractRef` interface
- Maintain the `jumpDests` struct in EVM for sharing between call frames
- Simplify the delegateCall context initialization
This removes the method `TestingTTDBlock` introduced by #30744. It was
added to make the beacon consensus engine aware of the merge block in
tests without relying on the total difficulty. However, tracking the
merge block this way is very annoying. We usually configure forks in the
`ChainConfig`, but the method is on the consensus engine, which isn't
always created in the same place. By sidestepping the `ChainConfig` we
don't get the usual fork-order checking, so it's possible to enable the
merge before the London fork, for example. This in turn can lead to very
hard-to-debug outputs and validation errors.
So here I'm changing the consensus engine to check the
`MergeNetsplitBlock` instead. Alternatively, we assume a network is
merged if it has a `TerminalTotalDifficulty` of zero, which is a very
common configuration in tests.
Here we add some more changes for live tracing API v1.1:
- Hook `OnSystemCallStartV2` was introduced with `VMContext` as parameter.
- Hook `OnBlockHashRead` was introduced.
- `GetCodeHash` was added to the state interface
- The new `WrapWithJournal` construction helps with tracking EVM reverts in the tracer.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR changes the signature of `CalcExcessBlobGas` to take in just
the header timestamp instead of the whole object. It also adds a sanity
check for the parent->child block order to `VerifyEIP4844Header`.
A clarification was made to EIP-7691 stating that at the fork boundary
it is required to use the target blob count associated with the head
block, rather than the parent as implemented here.
See for more: https://github.com/ethereum/EIPs/pull/9249
This PR defines the Osaka fork. An easy first step to start our work on
the next hardfork
(This is needed for EOF testing as well)
---------
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
The total difficulty is the sum of all block difficulties from genesis
to a certain block. This value was used in PoW for deciding which chain
is heavier, and thus which chain to select. Since PoS has a different
fork selection algorithm, all blocks since the merge have a difficulty
of 0, and all total difficulties are the same for the past 2 years.
Whilst the TDs are mostly useless nowadays, there was never really a
reason to mess around removing them since they are so tiny. This
reasoning changes when we go down the path of pruned chain history. In
order to reconstruct any TD, we **must** retrieve all the headers from
chain head to genesis and then iterate all the difficulties to compute
the TD.
In a world where we completely prune past chain segments (bodies,
receipts, headers), it is not possible to reconstruct the TD at all. In
a world where we still keep chain headers and prune only the rest,
reconstructing it possible as long as we process (or download) the chain
forward from genesis, but trying to snap sync the head first and
backfill later hits the same issue, the TD becomes impossible to
calculate until genesis is backfilled.
All in all, the TD is a messy out-of-state, out-of-consensus computed
field that is overall useless nowadays, but code relying on it forces
the client into certain modes of operation and prevents other modes or
other optimizations. This PR completely nukes out the TD from the node.
It doesn't compute it, it doesn't operate on it, it's as if it didn't
even exist.
Caveats:
- Whenever we have APIs that return TD (devp2p handshake, tracer, etc.)
we return a TD of 0.
- For era files, we recompute the TD during export time (fairly quick)
to retain the format content.
- It is not possible to "verify" the merge point (i.e. with TD gone, TTD
is useless). Since we're not verifying PoW any more, just blindly trust
it, not verifying but blindly trusting the many year old merge point
seems just the same trust model.
- Our tests still need to be able to generate pre and post merge blocks,
so they need a new way to split the merge without TTD. The PR introduces
a settable ttdBlock field on the consensus object which is used by tests
as the block where originally the TTD happened. This is not needed for
live nodes, we never want to generate old blocks.
- One merge transition consensus test was disabled. With a
non-operational TD, testing how the client reacts to TTD is useless, it
cannot react.
Questions:
- Should we also drop total terminal difficulty from the genesis json?
It's a number we cannot react on any more, so maybe it would be cleaner
to get rid of even more concepts.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
