This pr implements https://github.com/ethereum/go-ethereum/issues/32733
to make StateProcessor more customisable.
## Compatibility notes
This introduces a breaking change to users using geth EVM as a library.
The `NewStateProcessor` function now takes one parameter which has the
chainConfig embedded instead of 2 parameters.
The TestTraceChain function was missing a defer backend.teardown() call,
which is required to properly release blockchain resources after test
completion.
---------
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Fixes#32175.
This fixes the scenario where the blockhash opcode would return 0x0
during RPC simulations when using BlockOverrides with a future block
number. The root cause was that BlockOverrides.Apply() only modified the
vm.BlockContext, but GetHashFn() depends on the actual
types.Header.Number to resolve valid historical block hashes. This
caused a mismatch and resulted in incorrect behavior during trace and
call simulations.
---------
Co-authored-by: shantichanal <158101918+shantichanal@users.noreply.github.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
In this pull request, the original `CacheConfig` has been renamed to `BlockChainConfig`.
Over time, more fields have been added to `CacheConfig` to support
blockchain configuration. Such as `ChainHistoryMode`, which clearly extends
beyond just caching concerns.
Additionally, adding new parameters to the blockchain constructor has
become increasingly complicated, since it’s initialized across multiple
places in the codebase. A natural solution is to consolidate these arguments
into a dedicated configuration struct.
As a result, the existing `CacheConfig` has been redefined as `BlockChainConfig`.
Some parameters, such as `VmConfig`, `TxLookupLimit`, and `ChainOverrides`
have been moved into `BlockChainConfig`. Besides, a few fields in `BlockChainConfig`
were renamed, specifically:
- `TrieCleanNoPrefetch` -> `NoPrefetch`
- `TrieDirtyDisabled` -> `ArchiveMode`
Notably, this change won't affect the command line flags or the toml
configuration file. It's just an internal refactoring and fully backward-compatible.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
Fixes methods debug_standardTraceBlockToFile
and debug_standardTraceBadBlockToFile which were
outputting empty files.
---------
Co-authored-by: maskpp <maskpp266@gmail.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Fixes https://github.com/ethereum/go-ethereum/issues/31732.
This logic was removed in the recent refactoring in the txindexer to
handle history cutoff (#31393). It was first introduced in this PR:
https://github.com/ethereum/go-ethereum/pull/28908.
I have tested it and it works as an alternative to #31745.
This PR packs 3 changes to the flow of fetching txs from the API:
- It caches the indexer tail after each run is over to avoid hitting the
db all the time as was done originally in #28908.
- Changes `backend.GetTransaction`. It doesn't return an error anymore
when tx indexer is in progress. It shifts the responsibility to the
caller to check the progress. The reason is that in most cases we anyway
check the txpool for the tx. If it was indeed a pending tx we can avoid
the indexer progress check.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This pull request fixes a broken unit test
```
=== CONT TestTracingWithOverrides
api_test.go:1012: result: {"gas":21167,"failed":false,"returnValue":"0x0000000000000000000000000000000000000000000000000000000000000002","structLogs":[{"pc":0,"op":"PUSH1","gas":24978860,"gasCost":3,"depth":1,"stack":[]},{"pc":2,"op":"CALLDATALOAD","gas":24978857,"gasCost":3,"depth":1,"stack":["0x0"]},{"pc":3,"op":"PUSH1","gas":24978854,"gasCost":3,"depth":1,"stack":["0x1"]},{"pc":5,"op":"ADD","gas":24978851,"gasCost":3,"depth":1,"stack":["0x1","0x1"]},{"pc":6,"op":"PUSH1","gas":24978848,"gasCost":3,"depth":1,"stack":["0x2"]},{"pc":8,"op":"MSTORE","gas":24978845,"gasCost":6,"depth":1,"stack":["0x2","0x0"]},{"pc":9,"op":"PUSH1","gas":24978839,"gasCost":3,"depth":1,"stack":[]},{"pc":11,"op":"PUSH1","gas":24978836,"gasCost":3,"depth":1,"stack":["0x20"]},{"pc":13,"op":"RETURN","gas":24978833,"gasCost":0,"depth":1,"stack":["0x20","0x0"]}]}
api_test.go:1013: test 10, result mismatch, have
{21167 false 0x0000000000000000000000000000000000000000000000000000000000000002}
, want
{21167 false 0000000000000000000000000000000000000000000000000000000000000002}
api_test.go:1012: result: {"gas":25664,"failed":false,"returnValue":"0x000000000000000000000000c6e93f4c1920eaeaa1e699f76a7a8c18e3056074","structLogs":[]}
api_test.go:1013: test 11, result mismatch, have
{25664 false 0x000000000000000000000000c6e93f4c1920eaeaa1e699f76a7a8c18e3056074}
, want
{25664 false 000000000000000000000000c6e93f4c1920eaeaa1e699f76a7a8c18e3056074}
```
This is a **breaking change** to the opcode tracer. The top-level
`returnValue` field of a trace will be now hex-encoded. If the return
data is empty, this field will contain "0x".
Fixes#31196
The total difficulty is the sum of all block difficulties from genesis
to a certain block. This value was used in PoW for deciding which chain
is heavier, and thus which chain to select. Since PoS has a different
fork selection algorithm, all blocks since the merge have a difficulty
of 0, and all total difficulties are the same for the past 2 years.
Whilst the TDs are mostly useless nowadays, there was never really a
reason to mess around removing them since they are so tiny. This
reasoning changes when we go down the path of pruned chain history. In
order to reconstruct any TD, we **must** retrieve all the headers from
chain head to genesis and then iterate all the difficulties to compute
the TD.
In a world where we completely prune past chain segments (bodies,
receipts, headers), it is not possible to reconstruct the TD at all. In
a world where we still keep chain headers and prune only the rest,
reconstructing it possible as long as we process (or download) the chain
forward from genesis, but trying to snap sync the head first and
backfill later hits the same issue, the TD becomes impossible to
calculate until genesis is backfilled.
All in all, the TD is a messy out-of-state, out-of-consensus computed
field that is overall useless nowadays, but code relying on it forces
the client into certain modes of operation and prevents other modes or
other optimizations. This PR completely nukes out the TD from the node.
It doesn't compute it, it doesn't operate on it, it's as if it didn't
even exist.
Caveats:
- Whenever we have APIs that return TD (devp2p handshake, tracer, etc.)
we return a TD of 0.
- For era files, we recompute the TD during export time (fairly quick)
to retain the format content.
- It is not possible to "verify" the merge point (i.e. with TD gone, TTD
is useless). Since we're not verifying PoW any more, just blindly trust
it, not verifying but blindly trusting the many year old merge point
seems just the same trust model.
- Our tests still need to be able to generate pre and post merge blocks,
so they need a new way to split the merge without TTD. The PR introduces
a settable ttdBlock field on the consensus object which is used by tests
as the block where originally the TTD happened. This is not needed for
live nodes, we never want to generate old blocks.
- One merge transition consensus test was disabled. With a
non-operational TD, testing how the client reacts to TTD is useless, it
cannot react.
Questions:
- Should we also drop total terminal difficulty from the genesis json?
It's a number we cannot react on any more, so maybe it would be cleaner
to get rid of even more concepts.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR refactors the structlog a bit, making it so that it can be used
in a streaming mode.
-------------
OBS: this PR makes a change in the input `config` config, the third
input-parem field to `debug.traceCall`. Previously, seteting it to e.g.
` {"enableMemory": true, "limit": 1024}` would mean that the response
was limited to `1024` items. Since an 'item' may include both memory and
storage, the actual size of the response was undertermined.
After this change, the response will be limited to `1024` __`bytes`__
(or thereabouts).
-----------
The commandline usage of structlog now uses the streaming mode, leaving
the non-streaming mode of operation for the eth_Call.
There are two benefits of streaming mode
1. Not have to maintain a long list of operations,
2. Not have to duplicate / n-plicate data, e.g. memory / stack /
returndata so that each entry has their own private slice.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
When a tx/block was being traced through the API the state hooks weren't
being called as they should. This is due to #30745 moving the hooked
statedb one level up in the state processor. This PR fixes that.
---------
Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This change relocates the EVM tx context switching to the ApplyMessage function.
With this change, we can remove a lot of EVM.SetTxContext calls before
message execution.
### Tracing API changes
- This PR replaces the `GasPrice` field of the `VMContext` struct with
`BaseFee`. Users may instead take the effective gas price from
`tx.EffectiveGasTipValue(env.BaseFee)`.
---------
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
This pull request refactors the EVM constructor by removing the
TxContext parameter.
The EVM object is frequently overused. Ideally, only a single EVM
instance should be created and reused throughout the entire state
transition of a block, with the transaction context switched as needed
by calling evm.SetTxContext.
Unfortunately, in some parts of the code, the EVM object is repeatedly
created, resulting in unnecessary complexity. This pull request is the
first step towards gradually improving and simplifying this setup.
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR changes how sidechains are handled.
Before the merge, it was possible to import a chain with lower td and not set it as canonical. After the merge, we expect every chain that we get via InsertChain to be canonical. Non-canonical blocks can still be inserted
with InsertBlockWIthoutSetHead.
If during the InsertChain, the existing chain is not canonical anymore, we mark it as a sidechain and send the SideChainEvents normally.
This fixes an issue for `debug_traceBlock*` methods where the BASEFEE opcode was returning always 0. This caused the method return invalid results.
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Here we add a Go API for running tracing plugins within the main block import process.
As an advanced user of geth, you can now create a Go file in eth/tracers/live/, and within
that file register your custom tracer implementation. Then recompile geth and select your tracer
on the command line. Hooks defined in the tracer will run whenever a block is processed.
The hook system is defined in package core/tracing. It uses a struct with callbacks, instead of
requiring an interface, for several reasons:
- We plan to keep this API stable long-term. The core/tracing hook API does not depend on
on deep geth internals.
- There are a lot of hooks, and tracers will only need some of them. Using a struct allows you
to implement only the hooks you want to actually use.
All existing tracers in eth/tracers/native have been rewritten to use the new hook system.
This change breaks compatibility with the vm.EVMLogger interface that we used to have.
If you are a user of vm.EVMLogger, please migrate to core/tracing, and sorry for breaking
your stuff. But we just couldn't have both the old and new tracing APIs coexist in the EVM.
---------
Co-authored-by: Matthieu Vachon <matthieu.o.vachon@gmail.com>
Co-authored-by: Delweng <delweng@gmail.com>
Co-authored-by: Martin HS <martin@swende.se>
This change simplifies the logic for indexing transactions and enhances the UX when transaction is not found by returning more information to users.
Transaction indexing is now considered as a part of the initial sync, and `eth.syncing` will thus be `true` if transaction indexing is not yet finished. API consumers can use the syncing status to determine if the node is ready to serve users.
The Go authors updated golang/x/ext to change the function signature of the slices sort method.
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just
picked a new version that some other dep depends on, causing our code to fail building.
This PR updates the dep on our code too and does all the refactorings to follow upstream...
This PR modifies the interface for the results of `debug_traceBlock` and `debug_traceCall` by adding the `txHash`, allowing users to identify which transaction's trace result corresponds to.
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
Use the new atomic types in package eth/tracers
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Here, the core.Message interface turns into a plain struct and
types.Message gets removed.
This is a breaking change to packages core and core/types. While we do
not promise API stability for package core, we do for core/types. An
exception can be made for types.Message, since it doesn't have any
purpose apart from invoking the state transition in package core.
types.Message was also marked deprecated by the same commit it
got added in, 4dca5d4db7 (November 2016).
The core.Message interface was added in December 2014, in commit
db494170dc, for the purpose of 'testing' state transitions. It's the
same change that made transaction struct fields private. Before that,
the state transition used *types.Transaction directly.
Over time, multiple implementations of the interface accrued across
different packages, since constructing a Message is required whenever
one wants to invoke the state transition. These implementations all
looked very similar, a struct with private fields exposing the fields
as accessor methods.
By changing Message into a struct with public fields we can remove all
these useless interface implementations. It will also hopefully
simplify future changes to the type with less updates to apply across
all of go-ethereum when a field is added to Message.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR removes the notion of fakeStorage from the state objects, and instead, for any state modifications that are needed, it simply makes the changes.
Currently calling `debug_TraceTransaction` with a transaction hash that doesn't exist returns a confusing error: `genesis is not traceable`. This PR changes the behaviour to instead return an error message saying `transaction not found`
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
* ethclient/gethclient: improve time-sensitive flaky test
* eth/catalyst: fix (?) flaky test
* core: stop blockchains in tests after use
* core: fix dangling blockchain instances
* core: rm whitespace
* eth/gasprice, eth/tracers, consensus/clique: stop dangling blockchains in tests
* all: address review concerns
* core: goimports
* eth/catalyst: fix another time-sensitive test
* consensus/clique: add snapshot test run function
* core: rename stop() to stopWithoutSaving()
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR simplifies the logic of chain tracer and also adds the unit tests.
The most important change has been made in this PR is the state management. Whenever a tracing state is acquired there is a corresponding release function be returned as well. It must be called once the state is used up, otherwise resource leaking can happen.
And also the logic of state management has been simplified a lot. Specifically, the state provider(eth backend, les backend) should ensure the state is available and referenced. State customers can use the state according to their own needs, or build other states based on the given state. But once the release function is called, there is no guarantee of the availability of the state.
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
This PR adds support for block overrides when doing debug_traceCall.
- Previously, debug_traceCall against pending erroneously used a common.Hash{} stateroot when looking up the state, meaning that a totally empty state was used -- so it always failed,
- With this change, we reject executing debug_traceCall against pending.
- And we add ability to override all evm-visible header fields.
* eth/tracers: refactor traceTx to separate out struct logging
review fix
Update eth/tracers/api.go
Co-authored-by: Martin Holst Swende <martin@swende.se>
Mv ExecutionResult type to logger package
review fix
impl GetResult for StructLogger
make formatLogs private
confused exit and end..
account for intrinsicGas in structlogger, fix TraceCall test
Add Stop method to logger
Simplify traceTx
Fix test
rm logger from blockchain test
account for refund in structLogger
* use tx hooks in struct logger
* minor
* avoid executionResult in struct logger
* revert blockchain test changes
This adds a check to verify that a sender-account does not have code, which means that the codehash is either `emptyCodeHash` _OR_ not present. The latter occurs IFF the sender did not previously exist, a situation which can only occur with zero cost gasprices.
When processing a transaction with London fork rules, EIP-1559 mandates
checking that the sender must have sufficient balance to cover gas * gasFeeCap.
In the EIP's pseudocode, this check happens after the value transferred by the
transaction has already been deducted. However, in go-ethereum, the balance
has not yet been updated when the check happens, and therefore needs to be
added explicitly.
Co-authored-by: Martin Holst Swende <martin@swende.se>
