- Pre-BAL path keeps StateDB synchronous read-time accumulators
(AccountReads, StorageReads, CodeReads, CodeLoaded, CodeLoadBytes)
so Execution = ptime - reads stays well-formed under single-thread
execution.
- BAL path drops aggregate reader read-times; under parallel workers
they sum across goroutines and aren't a wall-clock proxy.
- Delete dead PrefetchReadTimes/WaitPrefetch forwarders on *reader and
the now-unused ReadTimer/ReadDurations scaffolding.
- Add regression test for EIP-7702 delegation clear: empty []byte code
in the BAL must reset CodeHash to EmptyCodeHash.
Populates per-block state read/write counts in slow-block JSON for BAL
blocks (which #34892 left as TBD), and adds reader-level read timing.
Builds on top of bal-devnet-3 — most of the PR's earlier slow-block log
infrastructure was adapted into upstream by that commit, so this change
is now scoped to the metric population that the BAL alone can derive.
- BAL helpers: BlockAccessList.{UniqueAccountCount, UniqueStorageSlotCount,
WrittenCounts}. WrittenCounts walks the BAL once and returns the
block-aggregate write counts.
- Reader-level read timing: *reader times all synchronous Account/Storage/
Code/CodeSize calls via atomic counters; exposed via ReadTimes()
ReadDurations and the new state.ReadTimer interface. Replaces StateDB-
level AccountReads/StorageReads/CodeReads tracking (the StateDB shouldn't
time its dependencies — the reader is where the I/O happens).
- Reader-level code-load dedup: *reader.codeLoaded sync.Map records the
first-seen byte length per address; CodeLoads() returns (count, bytes).
Exposed via state.CodeLoadTracker. Replaces StateDB CodeLoaded/
CodeLoadBytes tracking and the SnapshotCodeLoads aggregation pattern.
- BALStateTransition: caches BlockAccessList.WrittenCounts() once at
construction; tracks accountDeleted/storageDeleted atomics for the
parallel root-pass (the BAL alone can't distinguish a selfdestruct from
a balance/nonce reset). Exposes Deletions() DeletionCounts. Drops the
older accountUpdated/storageUpdated/codeUpdated/codeUpdateBytes counters
(now derived from WrittenCounts).
- BAL block stats path (blockchain.go): populates StateCounts directly —
AccountUpdated = WrittenCounts.Accounts - Deletions.Accounts (same for
storage). AccountLoaded/StorageLoaded come from BAL. CodeLoaded/
CodeLoadBytes come from the shared *reader (deduplicated across phase
StateDBs naturally because they share one reader instance).
- Non-BAL block stats path: read durations come from the reader; counts
from StateDB fields. StorageUpdated/StorageDeleted unified to int width.
- Hard type assertions: state.ReadTimer / state.CodeLoadTracker /
state.ReaderStater consumers use direct casts (no silent zero
fallback) — every Reader chain in production satisfies these
interfaces.
- Meter alignment: account/storage Updated meters subtract Deletions to
avoid double-reporting blocks under both Update and Delete dashboards.
Adapts some of the changes from
https://github.com/ethereum/go-ethereum/pull/34861 . Some other metrics
which are recorded manually during execution in that PR, but can be
deduced from the BAL are TBD.
I've added two bal feature flags:
* `--bal.prefetchworkers <uint>`: this tunes the number of concurrent
go-routines that will be used to perform state fetching tasks by the BAL
prefetcher. Default is `runtime.NumCPUs`, the current behavior in
`bal-devnet-3`.
* `--bal.blockingprefetch`: If set, state prefetching will block the
execution of transactions and state root update.
---------
Co-authored-by: CPerezz <cperezz19@pm.me>
This PR extends the statistics of contract code read by adding these
fields:
- **CacheHitBytes**: the total number of bytes served by cache
- **CacheMissBytes**: the total number of bytes read on cache miss
- **CodeReadBytes**: the total number of bytes for contract code read
In order to reduce the amount of code that is embedded into the keeper
binary, I am removing all the verkle code that uses go-verkle and
go-ipa. This will be followed by further PRs that are more like stubs to
replace code when the keeper build is detected.
I'm keeping the binary tree of course. This means that you will still
see `isVerkle` variables all over the codebase, but they will be renamed
when code is touched (i.e. this is not an invitation for 30+ AI slop
PRs).
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
## Description
This PR fixes incorrect contract code state metrics by ensuring
duplicate codes are not counted towards the reported results.
## Rationale
The contract code metrics don't consider database deduplication. The
current implementation assumes that the results are only **slightly
inaccurate**, but this is not true, especially for data collection
efforts that started from the genesis block.
This PR introduces a new debug feature, logging the slow blocks with
detailed performance statistics, such as state read, EVM execution and
so on.
Notably, the detailed performance statistics of slow blocks won't be
logged during the sync to not overwhelm users. Specifically, the statistics
are only logged if there is a single block processed.
Example output
```
########## SLOW BLOCK #########
Block: 23537063 (0xa7f878611c2dd27f245fc41107d12ebcf06b4e289f1d6acf44d49a169554ee09) txs: 248, mgasps: 202.99
EVM execution: 63.295ms
Validation: 1.130ms
Account read: 6.634ms(648)
Storage read: 17.391ms(1434)
State hash: 6.722ms
DB commit: 3.260ms
Block write: 1.954ms
Total: 99.094ms
State read cache: account (hit: 622, miss: 26), storage (hit: 1325, miss: 109)
##############################
```
This is broken off of #31730 to only focus on testing networks that
start with verkle at genesis.
The PR has seen a lot of work since its creation, and it now targets
creating and re-executing tests for a binary tree testnet without the
transition (so it starts at genesis). The transition tree has been moved
to its own package. It also replaces verkle with the binary tree for
this specific application.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This add some of the changes that were missing from #31634. It
introduces the `TransitionTrie`, which is a façade pattern between the
current MPT trie and the overlay tree.
---------
Signed-off-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
This is the first part of #31532
It maintains a series of conversion maker which are to be updated by the
conversion code (in a follow-up PR, this is a breakdown of a larger PR
to make things easier to review). They can be used in this way:
- During the conversion, by storing the conversion markers when the
block has been processed. This is meant to be written in a function that
isn't currently present, hence [this
TODO](https://github.com/ethereum/go-ethereum/pull/31634/files#diff-89272f61e115723833d498a0acbe59fa2286e3dc7276a676a7f7816f21e248b7R384).
Part of https://github.com/ethereum/go-ethereum/issues/31583
---------
Signed-off-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This pull request introduces a mechanism to expose statistics from the
state reader, specifically related to cache utilization during state prefetching.
To improve state access performance, a pair of state readers is constructed
with a shared local cache. One reader to execute transactions ahead of time
to warm up the cache. The other reader is used by the actual chain processing
logic, which can benefit from the prefetched states.
This PR adds visibility into how effective the cache is by exposing relevant
usage statistics.
---------
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
Co-authored-by: Csaba Kiraly <csaba.kiraly@gmail.com>
As https://github.com/ethereum/go-ethereum/pull/31769 defined a global
hash pool, so we can reuse it, and also remove the unnecessary
KeccakState buffering
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR creates a global hasher pool that can be used by all packages.
It also removes a bunch of the package local pools.
It also updates a few locations to use available hashers or the global
hashing pool to reduce allocations all over the codebase.
This change should reduce global allocation count by ~1%
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This pull request enhances the block prefetcher by executing transactions
in parallel to warm the cache alongside the main block processor.
Unlike the original prefetcher, which only executes the next block and
is limited to chain syncing, the new implementation can be applied to any
block. This makes it useful not only during chain sync but also for regular
block insertion after the initial sync.
---------
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
This PR introduces a `ContractCodeReader` interface with functions defined:
type ContractCodeReader interface {
Code(addr common.Address, codeHash common.Hash) ([]byte, error)
CodeSize(addr common.Address, codeHash common.Hash) (int, error)
}
This interface can be implemented in various ways. Although the codebase
currently includes only one implementation, additional implementations
could be created for different purposes and scenarios, such as a code
reader designed for the Verkle tree approach or one that reads code from
the witness.
*Notably, this interface modifies the function’s semantics. If the
contract code is not found, no error will be returned. An error should
only be returned in the event of an unexpected issue, primarily for
future implementations.*
The original state.Reader interface is extended with ContractCodeReader
methods, it gives us more flexibility to manipulate the reader with additional
logic on top, e.g. Hooks.
type Reader interface {
ContractCodeReader
StateReader
}
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This pull request introduces a state.Reader interface for state
accessing.
The interface could be implemented in various ways. It can be pure trie
only reader, or the combination of trie and state snapshot. What's more,
this interface allows us to have more flexibility in the future, e.g.
the
archive reader (for accessing archive state).
Additionally, this pull request removes the following metrics
- `chain/snapshot/account/reads`
- `chain/snapshot/storage/reads`
