The upstream libray has removed the assembly-based implementation of
keccak. We need to maintain our own library to avoid a peformance
regression.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
Implement standardized JSON format for slow block logging to enable
cross-client performance analysis and protocol research.
This change is part of the Cross-Client Execution Metrics initiative
proposed by Gary Rong: https://hackmd.io/dg7rizTyTXuCf2LSa2LsyQ
The standardized metrics enabled data-driven analysis like the EIP-7907
research: https://ethresear.ch/t/data-driven-analysis-on-eip-7907/23850
JSON format includes:
- block: number, hash, gas_used, tx_count
- timing: execution_ms, total_ms
- throughput: mgas_per_sec
- state_reads: accounts, storage_slots, bytecodes, code_bytes
- state_writes: accounts, storage_slots, bytecodes
- cache: account/storage/code hits, misses, hit_rate
This should come after merging #33522
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This is a tweak to the wasm build, that expects the `geth_io` namespace
to expect a `geth_io` module, providing a `len` and `read` methods. This
will be provided by the WASM interface in sp1. This forces an API change
on the OpenVM side, but the interface on their side is still being
designed, so we should proceed with this change, and we'll make a
different tag for OpenVM if this can't work for them.
Co-authored-by: wakabat <wakabat@protonmail.com>
This adds support for Grafana Pyroscope, a continuous profiling solution.
The client is configured similarly to metrics, i.e. run
geth --pyroscope --pyroscope.server=https://...
This commit is a resubmit of #33261 with some changes.
---------
Co-authored-by: Carlos Bermudez Porto <cbermudez.dev@gmail.com>
Adding an RPC flag to limit the block range size for eth_getLogs and
eth_newFilter requests.
closing https://github.com/ethereum/go-ethereum/issues/24508
---------
Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
Add Open Telemetry tracing inside the RPC server to help attribute runtime costs within `handler.handleCall()`. In particular, it allows us to distinguish time spent decoding arguments, invoking methods via reflection, and actually executing the method and constructing/encoding JSON responses.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
### Description
Add a new `OnStateUpdate` hook which gets invoked after state is
committed.
### Rationale
For our particular use case, we need to obtain the state size metrics at
every single block when fuly syncing from genesis. With the current
state sizer, whenever the node is stopped, the background process must
be freshly initialized. During this re-initialization, it can skip some
blocks while the node continues executing blocks, causing gaps in the
recorded metrics.
Using this state update hook allows us to customize our own data
persistence logic, and we would never skip blocks upon node restart.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR removes the version-check command and its associated logic as
discussed in issue #31222.
Removed versionCheckCommand from misccmd.go and main.go.
Deleted version_check.go and its corresponding tests.
Cleaned up testdata/vcheck directory (~800 lines of JSON/signatures
removed).
Verified build with make geth
In order to reduce the amount of code that is embedded into the keeper
binary, I am removing all the verkle code that uses go-verkle and
go-ipa. This will be followed by further PRs that are more like stubs to
replace code when the keeper build is detected.
I'm keeping the binary tree of course. This means that you will still
see `isVerkle` variables all over the codebase, but they will be renamed
when code is touched (i.e. this is not an invitation for 30+ AI slop
PRs).
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR introduces a new debug feature, logging the slow blocks with
detailed performance statistics, such as state read, EVM execution and
so on.
Notably, the detailed performance statistics of slow blocks won't be
logged during the sync to not overwhelm users. Specifically, the statistics
are only logged if there is a single block processed.
Example output
```
########## SLOW BLOCK #########
Block: 23537063 (0xa7f878611c2dd27f245fc41107d12ebcf06b4e289f1d6acf44d49a169554ee09) txs: 248, mgasps: 202.99
EVM execution: 63.295ms
Validation: 1.130ms
Account read: 6.634ms(648)
Storage read: 17.391ms(1434)
State hash: 6.722ms
DB commit: 3.260ms
Block write: 1.954ms
Total: 99.094ms
State read cache: account (hit: 622, miss: 26), storage (hit: 1325, miss: 109)
##############################
```
Recently in #31630 we removed support for overriding the network id in
preset networks. While this feature is niche, it is useful for shadow
forks. This PR proposes we add the functionality back, but in a simpler
way.
Instead of checking whether the flag is set in each branch of the
network switch statement, simply apply the network flag after the switch
statement is complete. This retains the following behavior:
1. Auto network id based on chain id still works, because `IsSet` only
returns true if the flag is _actually_ set. Not if it just has a default
set.
2. The preset networks will set their network id directly and only if
the network id flag is set is it overridden. This, combined with the
override genesis flag is what allows the shadow forks.
3. Setting the network id to the same network id that the preset _would
have_ set causes no issues and simply emits the `WARN` that the flag is
being set explicitly. I don't think people explicitly set the network id
flag often.
```
WARN [10-22|09:36:15.052] Setting network id with flag id=10
```
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
Enable blocktest to read filenames from stdin when no path argument is
provided, matching the existing statetest behavior. This allows
efficient batch processing of blockchain tests.
Usage:
- Single file: evm blocktest <path>
- Batch mode: find tests/ -name "*.json" | evm blocktest
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
Co-authored-by: Felix Lange <fjl@twurst.com>
No matter what value of P2P.DiscoveryV4 or DiscoveryV5 is set in config file,
it will be overwritten by the CLI flag, even if the flag is not set. This fixes it
to apply the flag only if set.
Bumps
[github.com/consensys/gnark-crypto](https://github.com/consensys/gnark-crypto)
from 0.18.0 to 0.18.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/consensys/gnark-crypto/releases">github.com/consensys/gnark-crypto's
releases</a>.</em></p>
<blockquote>
<h2>v0.18.1</h2>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/Consensys/gnark-crypto/compare/v0.18.0...v0.18.1">https://github.com/Consensys/gnark-crypto/compare/v0.18.0...v0.18.1</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/Consensys/gnark-crypto/blob/master/CHANGELOG.md">github.com/consensys/gnark-crypto's
changelog</a>.</em></p>
<blockquote>
<h2>[v0.18.1] - 2025-10-28</h2>
<h3>Docs</h3>
<ul>
<li>add CHANGELOG for 0.18.1</li>
</ul>
<h3>Perf</h3>
<ul>
<li>limit memory allocation during Vector deserialization (<a
href="https://redirect.github.com/Consensys/gnark-crypto/issues/759">#759</a>)</li>
</ul>
<p><!-- raw HTML omitted --><!-- raw HTML omitted --></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="fb04e95c3b"><code>fb04e95</code></a>
docs: add CHANGELOG for 0.18.1</li>
<li><a
href="0a4d04ae62"><code>0a4d04a</code></a>
perf: limit memory allocation during Vector deserialization (<a
href="https://redirect.github.com/consensys/gnark-crypto/issues/759">#759</a>)</li>
<li>See full diff in <a
href="https://github.com/consensys/gnark-crypto/compare/v0.18.0...v0.18.1">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ethereum/go-ethereum/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Adds a flag to specify how many blobs a node is willing to include in
their locally build block as specified in
https://eips.ethereum.org/EIPS/eip-7872
I deviated from the EIP in one case, I allowed for specifying 0 as the
minimum blobs/block
Fix#33212.
This PR remove `github.com/olekukonko/tablewriter` from dependencies and
use a naive stub implementation.
`github.com/olekukonko/tablewriter` is used to format database inspection
output neatly. However, it requires custom adjustments for TinyGo and is
incompatible with the latest version.
---------
Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
This is broken off of #31730 to only focus on testing networks that
start with verkle at genesis.
The PR has seen a lot of work since its creation, and it now targets
creating and re-executing tests for a binary tree testnet without the
transition (so it starts at genesis). The transition tree has been moved
to its own package. It also replaces verkle with the binary tree for
this specific application.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR is an alternative to #32556.
Instead of trying to be smart and reuse `geth init`, we can introduce a
new flag `--genesis` that loads the `genesis.json` from file into the
`Genesis` object in the same path that the other network flags currently
work in.
Question: is something like `--genesis` enough to start deprecating
`geth init`?
--
```console
$ geth --datadir data --hoodi
..
INFO [10-06|22:37:11.202] - BPO2: @1762955544
..
$ geth --datadir data --genesis genesis.json
..
INFO [10-06|22:37:27.988] - BPO2: @1862955544
..
```
Pull the genesis [from the
specs](https://raw.githubusercontent.com/eth-clients/hoodi/refs/heads/main/metadata/genesis.json)
and modify one of the BPO timestamps to simulate a shadow fork.
---------
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Uses the go module's `replace` directive to delegate keccak computation
to precompiles.
This is still in draft because it needs more testing. Also, it relies on
a PR that I created, that hasn't been merged yet.
_Note that this PR doesn't implement the stateful keccak state
structure, and it reverts to the current behavior. This is a bit silly
since this is what is used in the tree root computation. The runtime
doesn't currently export the sponge. I will see if I can fix that in a
further PR, but it is going to take more time. In the meantime, this is
a useful first step_
New RPC method eth_sendRawTransactionSync(rawTx, timeoutMs?) that
submits a signed tx and blocks until a receipt is available or a timeout
elapses.
Two CLI flags to tune server-side limits:
--rpc.txsync.defaulttimeout (default wait window)
--rpc.txsync.maxtimeout (upper bound; requests are clamped)
closes https://github.com/ethereum/go-ethereum/issues/32094
---------
Co-authored-by: aodhgan <gawnieg@gmail.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Fix logging in the verkle dump path to report the actual key being
processed.
Previously, the loop always logged keylist[0], which misled users when
expanding multiple keys and made debugging harder. This change aligns
the
log with the key passed to root.Get, improving traceability and
diagnostics.
This PR adds a `filterfuzz` subcommand to the workload tester that
generates requests similarly to `filtergen` (though with a much smaller
block length limit) and also verifies the results by retrieving all
block receipts in the range and locally filtering out relevant results.
Unlike `filtergen` that operates on the finalized chain range only,
`filterfuzz` does check the head region, actually it seeds a new query
at every new chain head.
In both `TestSimultaneousRequests` and `TestSameRequestID`, we send two
concurrent requests. The client under test is free to respond in either
order, so we need to handle responses both ways.
Also fixes an issue where some generated blob transactions didn't have
any blobs.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
These functions were previously ignoring the error returned by both
`statedb.Commit()` and the subsequent `state.New()`,
which could silently fail and cause panics later when the `statedb` is
used.
This change adds proper error checking and panics with a descriptive
error
message if state creation fails.
While unlikely in normal operation, this can occur if there are database
corruption issues or if invalid root hashes are provided, making
debugging
significantly easier when such issues do occur.
This issue was encountered and fixed in
https://github.com/gballet/go-ethereum/pull/552
where the error handling proved essential for debugging
cc: @gballet as this was discussed in a call already.
The parent header was missing the BaseFee field when calculating the
reserve price for EIP-7918 in the Osaka fork, causing a nil pointer
dereference. This fix ensures BaseFee is properly set from ParentBaseFee
in the environment.
Added regression test case 34 to verify Osaka fork blob gas calculation
works correctly with parent base fee.
This change replaces wrapping a stale outer err with the iterator’s own
error after Next(), and switches the post-BlockAndReceipts() check to
use the returned err. According to internal/era iterator contract,
Error() should be consulted immediately after Next() to surface
iteration errors, while decoding errors from Block/Receipts are returned
directly. The previous code could hide the real failure (using nil or
unrelated err), leading to misleading diagnostics and missed iteration
errors.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
https://go.dev/ref/mod#go-work-file advises against checking `go.work`
files because they can interfere with local development. We added the
workspace file in order to make `go test` and other tools work across
multiple modules. But it seems to cause weird issues with the
`go.work.sum` file being modified, etc.
So with this PR, we instead run all the `ci.go` commands for all modules
in the workspace manually.
Fixes a crash when loading the beacon chain config if new fields like
`BLOB_SCHEDULE: []` are present.
Previously, the config loader assumed all values were strings, causing
errors such as:
```
Fatal: Could not load beacon chain config '/network-configs/config.yaml': failed to parse beacon chain config file: yaml: unmarshal errors:
line 242: cannot unmarshal !!seq into string
```
This PR updates the parsing logic to handle non-string values correctly
and adds explicit validation for fork fields.
Add cli configurable limit for the number of addresses allowed in
eth_getLogs filter criteria:
https://github.com/ethereum/go-ethereum/issues/32264
Key changes:
- Added --rpc.getlogmaxaddrs CLI flag (default: 1000) to configure the
maximum number of addresses
- Updated ethconfig.Config with FilterMaxAddresses field for
configuration management
- Modified filter system to use the configurable limit instead of the
hardcoded maxAddresses constant
- Enhanced test coverage with new test cases for address limit
validation
- Removed hardcoded validation from JSON unmarshaling, moving it to
runtime validation
Please notice that I remove the check at FilterCriteria UnmarshalJSON
because the runtime config can not pass into this validation.
Please help review this change!
---------
Co-authored-by: zsfelfoldi <zsfelfoldi@gmail.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
ZKVMs are constrained environments that liberally allocate memory and
never release it. In this context, using the GC is only going to cause
issues down the road, and slow things down in any case.
