The stack primitives pop by value: pop() returns the 32-byte value
itself, so every popped operand is copied out of the stack arena before
it is used. The result side was already in place, peek returns a pointer
and binary ops write into the new stack top. This PR fixes the operand
side: pointer-returning primitives (popPtr, popPtrPeek, etc), with the
handlers rewritten to read operands directly from their arena slots.
Every popped operand paid the copy, whatever the op went on to do with
it, so this optimization covers the arithmetic and comparison ops as
much as JUMP, MSTORE, SSTORE and RETURN.
The copy is visible in the assembly. On arm64, master's opLt spends four
instructions moving the popped value through the frame, and the
comparison then reads it back from there:
LDP (R5), (R6, R7) ; load words 0 and 1 of the popped value from the
arena
LDP 16(R5), (R5, R8) ; load words 2 and 3
STP (R6, R7), vm.~r0-64(SP) ; store words 0 and 1 into a frame slot
STP (R5, R8), vm.~r0-48(SP) ; store words 2 and 3
With popPtrPeek those four instructions are gone, the frame shrinks from
locals=0x58 to locals=0x18, and the function from 336 to 288 bytes. The
compiler cannot remove the copy itself: uint256.Int is a four-element
array, and Go's SSA does not promote arrays longer than one element to
registers, so a by-value pop pays this round trip no matter how far
inlining gets, for LT exactly as for ADD.
The CALL and CREATE families are deliberately not converted: a child
frame reuses the same stack arena, so parent pointers into popped slots
die when the child pushes. The rule is recorded on the primitives:
pointers stay valid until the next push or any sub call. Converting the
call family safely means materializing scalars before the child call,
left for later work with a call-heavy benchmark to justify it.
### Benchmarks
Measured with the benchmark suite from #35144 (the evm-bench contract
workloads and the block import benchmark), which is not part of this
PR's diff. Apple M4 Max, fixed iteration counts, n=10, all p=0.000. B/op
and allocs/op are statistically identical on every benchmark:
| benchmark | master | PR | vs master |
|---|---|---|---|
| Snailtracer | 60.0 ms | 54.1 ms | -9.8% |
| TenThousandHashes | 13.2 ms | 12.2 ms | -7.8% |
| ERC20Transfer | 11.7 ms | 11.0 ms | -5.5% |
| ERC20Mint | 7.49 ms | 7.02 ms | -6.2% |
| ERC20ApprovalTransfer | 8.92 ms | 8.44 ms | -5.4% |
This PR is independent of #35144 but plays nicely with it: the generated
dispatch there splices these handler bodies, so the in-place forms land
in its fast path too, where they measure larger.
### Testing
The rewritten handlers run on the interpreter's only execution path, so
correctness rests on references outside the change:
- **Consensus fixtures.** The full tests package passes: state tests,
the execution-spec families, blockchain tests.
- **Opcode testcases.** The JSON testcases compare individual opcode
results against committed expected values.
- **Tracer fixtures.** The tracetest reference files pin exact log and
return data shapes, covering the rewritten LOG and RETURN paths.
- **Cross-build differential.** A goevmlab campaign running this
branch's evm against master's evm over generated state tests across four
forks (Prague, Cancun, London, Osaka) with full trace comparison:
160,566 tests, zero divergences.
---------
Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
rjl493456442 pointed out that the cleared figure logged at the end of
`geth db prune-history` only counted the body/receipt flat-file delta
via prunableFreezerSize() and silently dropped the tx-index work done
just before TruncateTail. Anyone reading the log line saw a number
that under-reported the actual on-disk savings.
PruneTransactionIndex now returns the logical bytes deleted (sum of
key + value lengths for each removed entry) by accumulating during the
existing DeleteAllTxLookupEntries iteration. cmd/geth folds that into
the StorageSize total before logging. The figure is uncompacted — the
KV store doesn't expose per-prefix on-disk usage without a full
compaction sweep — but it's the only signal available without changing
the prune sequence, and it matches the granularity rjl asked for.
Existing rawdb test (TestPruneTransactionIndex) is unchanged; Go's
untyped-discard semantics keep it compiling against the new signature.
Implements https://eips.ethereum.org/EIPS/eip-8037
mainly done in order to judge the complexity of the EIP
and to act as a jumping off point, since the eip will likely
change.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR introduces a cache for GetBlobs request.
The main purpose of this PR is to reduce the getBlobs latency by reading and
decoding blobs from the pool in advance of the actual query. This is important
especially in the context of a sparse blobpool, since it may be necessary to
recover blobs from cells on a getBlobs request.
Previously, the Engine API read and decoded blobs from the pool on every call.
Now those calls check the cache and only fall back to the pool on a miss.
The cache has two modes:
- In topK mode (default), it wakes up periodically, picks the most profitable
pending blob transactions up to the current fork's maxBlobsPerBlock, and loads
their blobs. The selection logic is shared with the miner's block-building
logic. The selection size is derived from eip4844.MaxBlobsPerBlock at the
current head.
- When the CL calls HasBlobs, the cache switches to hasBlobs mode and tries to
pin the set it just reported as available. Cache updates (read, decode, and
optionally conversion in the future) run in background goroutines.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
Adds snap/2 (EIP-8189), a block-access-list (BAL) based state sync, and
wires it to run side by side with snap/1. It's opt-in (for now) behind a
new --snap.v2 flag and chosen at startup.
https://eips.ethereum.org/EIPS/eip-8189
---------
Co-authored-by: Toni Wahrstätter <info@toniwahrstaetter.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This is a PR that removes all correctly flagged typos, in order to stop
an onslaught of slop PRs in its tracks. It should be followed by #34994
but the latter needs more configuration work and I want to limit the
stem of PRs right now.
This PR is trying to stem further slop PRs by going over all incorrect
strings and fixing them.
---------
Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
## Overview
This PR fixes a race condition during blockchain shutdown where snapshot
generation could continue accessing the trie database after it has been
closed, leading to iterator errors. We noticed this in one of our nodes
on https://github.com/ava-labs/avalanchego, which relies on an older
version of geth with the same issue (so this behavior does happen!).
During node shutdown, the following sequence occurs:
1. `BlockChain.Stop()` calls `snaps.Release()` to clean up snapshot
resources
2. `Release()` only resets the cache but doesn't stop the generator
goroutine
3. The trie database is then closed via `triedb.Close()`
4. The still-running generator attempts to iterate storage tries
5. Iterator fails because the database is closed (`"Generator failed to
iterate storage trie"`)
## Problem
There are three related bugs:
1. `Release()` doesn't stop generation: The `diskLayer.Release()` method
only resets the cache without stopping ongoing snapshot generation,
leaving the generator goroutine running after database closure.
2. `stopGeneration()` has an incorrect completion check: The
`stopGeneration()` method checks `genMarker != nil` to determine if
generation is running. However, `genMarker` is set to nil when
generation completes successfully, even though the generator goroutine
is still waiting for the abort signal at the end of `generate()`. See
line 705 in `generate.go`:
eaaa5b716d/core/state/snapshot/generate.go (L699-L707)
This means `stopGeneration()` returns early without sending the abort
signal.
3. Node shutdown doesn't stop generation: During shutdown, no code path
calls `stopGeneration()` or sends the abort signal to the generator,
causing the generator to access a closed database and error.
## Fix
- Modified `diskLayer.Release()` to call `stopGeneration()` before
releasing resources
- Added cancelation architecture, removing reliance on someone having to
wait
- Fixed `stopGeneration()` to properly and safely stop snapshot
generation
- Added `TestGenerateGoroutineLeak` to verify the fix and prevent
regression. The test fails without the fix and passes with it.
- The test creates a snapshot with active generation, waits for
completion, then calls `Release()`, and uses `go.uber.org/goleak` to
assert no generator goroutine survives.
- Without the fix, the test fails: `Release()` returns without stopping
the generator, which stays parked at `generate.go:705` waiting for an
abort signal that never comes:
```
--- FAIL: TestGenerateGoroutineLeak (0.88s)
generate_test.go: found unexpected goroutines:
[Goroutine 6 in state chan receive, with
core/state/snapshot.(*diskLayer).generate on top of the stack:
core/state/snapshot.(*diskLayer).generate(...)
core/state/snapshot/generate.go:705
created by core/state/snapshot.generateSnapshot
core/state/snapshot/generate.go:79 ]
```
- With the fix, the test passes: `Release()` -> `stopGeneration()`
blocks until the generator goroutine has fully exited, so nothing leaks
Note that this fix follows the same pattern used in `Tree.Disable()` in
https://github.com/ethereum/go-ethereum/pull/30040, which introduced
`stopGeneration()` for use in `Disable()` and `Rebuild()` but didn't
address the shutdown path.
The test follows the same pattern used in
`TestCheckSimBackendGoroutineLeak`
Rewrites triedb.GenerateTrie as a single partitioned pass that
reconciles stale account.Root fields and rebuilds the trie at the same
time, with 16-way parallelism and crash resume baked in.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
The per-call SERVER span ended inside `handleCall()`, so the JSON-RPC
response write happened after the span closed. For large responses like
`engine_getBlobsV*`, that write time was missing from traces.
- Extend the SERVER span past `writeJSON`.
- For batches, add a top-level `jsonrpc.batch` SERVER span (with `rpc.batch.size`) covering the whole batch including `callBuffer.write`.
- Add `rpc.writeJSON` span around the non-batch response write.
- Add `rpc.writeJSONBatch` span around the batch response write.
- Add `rpc.httpWrite` span around the actual HTTP write, separating JSON encoding from network write.
- Add additional telemetry helpers.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR implements flat-file storage for finalized block access lists,
specifically:
* The freezer is extended with the notion of tail groups, allowing
different groups within a single freezer instance to maintain
independent tails, while all tables within the same group remain
tail-aligned.
* The freezer can now dynamically attach new tables to an existing
freezer instance, with both the table head and tail initialized to the
freezer's common head.
* A new freezer table, **bals**, has been added to the chain freezer
with its own dedicated tail group, preserving the flexibility to deploy
a tail-pruning policy different from the main chain data group.
Additionally, the BALs in the key-value store will be migrated to the
freezer instance once they are finalized or there are at least 90K block
confirmations on top acting as a "soft finalization". This freezing
policy is same with all chain segment data.
Because the UBT doesn't differentiate slots from accounts, the content
of the tree can not be exported as a `GenesisAlloc`, which means that
`evm t8n` can not intergrate it. We have tried integrating the new
format into execution-specs, but this is very hard to maintain because
the team doesn't see it as a priority and their own repository is seeing
a lot of churn. This PR adds the ability to capture the structure of
what is being inserted in the tree, so that the information isn't lost
and it can be dumped in the t8n context.
---------
Co-authored-by: felipe <fselmo2@gmail.com>
- Adds tracing to the `GetBlobsV1/V2/V3`
- Adds `blobs.requested` and `blobs.filled` attributes to
`GetBlobsV1/V2/V3` spans.
- Adds tracing to `BlobTxPool().GetBlobs()`
This PR:
- Adds `engine_newPayloadWithWitnessV5`. The codebase already supports
the previous `VX`, so only `V5` was missing.
- Make the consensus witness format use the field [ordering defined in
the
spec](8d7e68f4b7/src/ethereum/forks/amsterdam/stateless_host_exec_witness.py (L175-L176))
to make it canonical.
cc @gballet
---------
Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
Updates the static validation logic to cover additional edge cases
(reflecting the state of the latest devnet branch, except cleaned up
slightly).
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR implements the serving side of the eth71 BAL exchange messages.
Until commit 4cd7092 also contained the requesting side, but since that
part still needs more work, I'm splitting it out into a separate PR.
The test injects BALs directly into rawdb. This can be removed once BAL
generation is integrated into the chain maker.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR finally lands EIP-7928, collecting the block accessList during
the block execution and verifying against the block header.
---------
Co-authored-by: jwasinger <j-wasinger@hotmail.com>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
This PR fixes a bug in the current blobpool `Reset` function where it
used the Transaction type instead of blobTxForPool.
Decoding transactions fetched from the pool as Transaction type
caused an error because the blobpool stores blobTxForPool types.
This PR extends the journal to track the pre-transaction values of
mutated balances, nonces, and code.
At the end of the transaction, these values are used to filter out no-op
changes, such as balance transitions from a-> b->a. These changes are
excluded from the block-level access list.
Additionally, there is a dedicated `bal.ConstructionBlockAccessList`
objects for gathering the state reads and writes within the current
transaction. These state writes will be keyed by the block accessList
index.
---------
Co-authored-by: jwasinger <j-wasinger@hotmail.com>
This PR introduces OnGasChangeV2 tracing hook, as the pre-requisite for landing
EIP-8037.
---------
Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
This PR introduces a separate transaction pool type for sparse blobpool.
In sparse blobpool, PooledTransactions message delivers transactions without
blobs, partial or full cells are downloaded by Cells message. Blobpool no longer
stores transactions with complete sidecars, and it stores transactions without
blobs, along with the corresponding cells. Because of this, a dedicated type
distinct from types.Transaction is required.
This PR introduces a type called `BlobTxForPool` and stores each sidecar field
independently, in order to bypass the assumption that a sidecar always exists as
a complete unit.
Reintroducing the conversion queue was considered, but was ultimately omitted
because type conversion should be sufficiently fast. With sparse blobpool, blob
-> cell computation would take about ~13ms per blob. Not sure whether this is
fast enough, but otherwise we can add the conversion queue later on the sparse
blobpool branch.
This is a refactoring PR to wrap all pre/post-execution system calls as
the exported functions, eliminating the duplicated system calls across
the codebase.
There are a few things unchanged but worths highlight:
- ChainMaker is left as unchanged, a significant rewrite is required
- BeaconRoot in header should be non-nil if Cancun is enabled
---------
Co-authored-by: jwasinger <j-wasinger@hotmail.com>
`GetBlobs` returned early when `CellProofsAt` reported
corrupted/out-of-bounds proofs, dropping every blob already collected
and aborting the remaining hashes — a single bad sidecar killed the
whole Engine API batch for consensus clients. Replaced the `return nil,
nil, nil, err` with `log.Error + continue` so the slot stays `nil` per
the sparse-array contract, matching the store/RLP/nil-sidecar branches a
few lines above.
This PR makes a small update to the `Pending()` method in the legacy
pool. By changing the lock from exclusive to read-only, it aims to
improve concurrency performance.
---------
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
the gapped queue cap was effectively per-sender rather than total — a
sender pool spread across enough distinct addresses could grow
`p.gapped` well past `maxGapped`, defeating the resource bound.
`maxGapped` was being compared against `len(p.gapped)`, which is a
`map[address][]tx` and counts unique senders, not queued txs. Switched
the check to `len(p.gappedSource)` (keyed by tx hash, so its length is
the real total). Also wired up a `blobpool/gapped/count` gauge plus
`promoted`, `evicted`, and `gappedfull` meters so queue size and churn
are actually observable in prod.
This PR addresses one of the biggest performance issue with binary
tries: storing each internal node individually bloats the index, the
disk, and triggers a lot of write amplifications. To fix this issue,
this PR serializes groups of nodes together.
Because we are still looking for the ideal group size, the "depth" of
the group tree is made a parameter, but that will be removed in the
future, once the perfect size is known.
This is a rebase of #33658
---------
Co-authored-by: Copilot <copilot@github.com>
