eth/72 announces blob transactions without the blob payload while eth/71
includes it, so the same transaction is announced with different
sizes. Compare each announcement against the size expected for that
peer's version instead of a single size so that honest peers on either version
are no longer dropped on delivery.
`BlobPool.Init` builds the lookup, index and store without holding
`p.lock`, and `eth/backend.go` publishes the pool to `blobpool.NewCache`
before `txpool.New` runs Init. Cache.update spawns a goroutine calling
`getByVhash`, so it can read the pool mid-construction:
```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28]
blobpool.(*BlobPool).getByVhash(...) blobpool.go:1303
blobpool.(*Cache).update.func1() cache.go:429
```
`Init` fills the lookup inside `billy.Open`, whose index callback runs
`parseTransaction` -> `trackTransaction`, and assigns `p.store` only
once `billy.Open` returns. For that window a lookup hit names a
transaction whose store does not exist yet, and `p.store.Get`
dereferences nil.
Build the cache after `txpool.New` so no reader exists while `Init` runs
-- `getByVhash` has exactly one caller -- and have `getByVhash` read the
store under the same lock as the lookup, returning nil when it is not
set.
A benchmark harness that restarts the client once per test hit this on
1-4% of starts; a node that starts once rarely will.
This PR explicitly clears the referenced block access list at the end of
each transaction. This is essential because accounts are shadow-copied
into the global block access list, and it prevents accidental mutations
through stale references in StateDB.
Fixes#35418.
After a restart, the syncer sets the finalized marker to head−64 and the
safe marker to head−32 on every downloader sync event, ignoring epoch
boundaries. Since this service is registered unconditionally and
`api_backend` serves `CurrentFinalBlock()` verbatim,
`eth_getBlockByNumber("finalized")` can return a mid-epoch block that
was never finalized and can still be reorged.
The synthesized markers exist to make `finalized`/`safe` usable when no
consensus client is attached. This change scopes them to exactly that
case: they are only set when an explicit sync target has been specified
(`--synctarget`), which is the only mode where no CL supplies real
checkpoints. Normal nodes now never override consensus-client
checkpoints.
`TestSyncerDoesNotInventFinalityMarkers` syncs a node to head with no CL
finality input and asserts the chain's finalized/safe markers stay
unset; it fails on master (finalized invented at head−64) and passes
with this change.
---------
Co-authored-by: SillyZir <269283839+SillyZir@users.noreply.github.com>
Co-authored-by: Jonny Rhea <5555162+jrhea@users.noreply.github.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
StateSize is a development tool for calculating and tracking the size of
the state. It was requested by ethPandaOps a while ago, but it has since
been superseded by another approach that captures state changes via a
live tracer. Therefore, this tool can be retired to reduce maintenance
overhead.
Close the started signal only after the fetchers are registered on
cancelWg, so suspend's Cancel actually waits for them instead of racing
the registration.
`testing_buildBlockV1` decodes `targetGasLimit` from the payload
attributes but does not pass it to `miner.BuildPayloadArgs`. The miner
then falls back to its configured gas ceiling, so the field is silently
ignored. The engine API path (`forkchoiceUpdated`) already passes it
through.
This matters for fixture generation in ethereum/execution-apis, where
`testing_buildBlockV1` builds Amsterdam test blocks and the gas limit
must honor the CL-provided target (see ethereum/execution-apis#857 and
ethereum/execution-apis#862).
The new test builds an Amsterdam block with a target inside the
per-block adjustment bound and checks the payload hits it exactly.
Since #35386 blocks are delivered without waiting for their access list,
a reserved BAL task can outlive its block and hit the stale branch of
`reserveHeaders`, which logs at error level — demote that case to debug.
---------
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
This PR implements the BAL downloader. Once the Amsterdam fork is
enabled, BALs are scheduled for download for BAL-eligible blocks.
Unlike mandatory components such as block bodies, BALs are optional and
are downloaded on a best-effort basis. If a block's essential components
are ready for delivery before its BAL has been retrieved, the block will
be delivered without the BAL.
If snap sync v2 is re-enabled after a previous sync already completed
and committed its pivot, the leftover journal is now discarded instead
of resumed.
This PR aligns the `Cells` and `GetCells` message implementations with
the spec: https://github.com/ethereum/devp2p/blob/master/caps/eth.md#getcells-0x14
Previously, `GetCellsPacket` and `CellsPacket` embedded `GetCellsRequest`
and `CellsResponse`. This caused them to be encoded as nested lists, which
does not match the wire format defined by the spec. This PR inlines their fields
to flatten the RLP layout of `Cells` and `GetCells`.
Blocks heavy in precompile calls (e.g. Aztec's proof settlement txs)
spend most of their processing time on operations (ECMUL, pairings, KZG
point evaluation, MODEXP) that the state prefetcher has already computed
and thrown away.
This PR adds a precompile result cache shared between the prefetcher and
block processing (and the miner), following the JumpDestCache pattern.
Note that cached precompiles are keyed by address and input, with
entries partitioned by the active precompile set, so a fork that changes
the behavior behind an address can never be served results from before
it.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
commitPivotBlock updated committed without pivotLock, while other pivot
transitions serialize through that lock. Take the lock around the store
to keep pivot commitment consistent with pivotHeader updates.
This PR coordinates the prefetcher with the main tx executor. Block
processing publishes the index of the transaction it is executing,
prefetch workers skip anything already reached and transactions above 1M
gas are promoted to the front of the prefetch queue while the rest keeps
block order.
The deep-reorg check used depth >= maxReorgDepth, rejecting reorgs at
exactly the configured limit. Use > so a depth equal to maxReorgDepth is
still accepted.
When a peer drop removes the last waiter for a hash, waitlist and
waittime were cleaned up but partial could retain a stale entry. Delete
it too so the hash is fully forgotten.
This PR improves the sync mode management, disabling the snap mode once
the pivot block is committed.
Originally the mode will only be flipped from snap to full once the entire sync
cycle is completed. However, it's theoretically possible to cancel the sync cycle
after committing the pivot but before inserting the remaining blocks. It drags
the following sync cycle back to the snap mode.
The snap mode should only be used if the head state is missing and unrecoverable.
Enabling the snap mode for multiple times should be strictly prevented.
What's more, it brings a fix to pivot block management, that once the
pivot block is committed, the pivot block marker should be no longer advanced,
preventing the situation that the real pivot state is below the pivot marker.
blobFetcherQueueingHashes used len(announces), which is the number of
peers with pending announces, not the number of queued hashes. Count
unique hashes across peers instead.
---------
Co-authored-by: Bosul Mun <bsbs8645@snu.ac.kr>
Commit 1f87331fb moved the known-transaction marking in
`sendPooledTransactionHashes` to after a successful send, so hashes are
not marked known to the peer if the announcement fails to go out.
The sparse blobpool change (d91b71fb3) reintroduced the original
track-before-send ordering when adding the eth/72 packet variant,
causing failed announcements to suppress future re-announcements of the
same hashes to that peer.
This restores the send-first ordering for both eth/71 and eth/72 packet
versions, and adds a regression test covering success and failure paths
on both protocol versions.
## Checklist
- [x] Restored mark-known-after-send for ETH71 and ETH72
- [x] Added `TestSendPooledTransactionHashes` covering success and
closed-pipe failure
---------
Co-authored-by: Bosul Mun <bsbs8645@snu.ac.kr>
This is the implementation of EIP-8070 Sparse Blobpool. It introduces
protocol version eth/72 which relays blob transaction cells instead of
full blobs.
The blobpool now store 'incomplete' transactions, where only some of
the cells are provided. The stored cell indexes are taken from the
custody bitmap, which is provided by the consensus layer in
forkchoiceUpdatedV4. This method will be called once Glamsterdam
activates, and the default custody is full custody, so for now there
is no change in the amount of stored cells for now.
The main entities added are the BlobBuffer and BlobFetcher, which work
together to track and fetch missing cells from connected peers. The
partial transactions become available for inclusion in blocks when
they are covered by enough peers that hold all cells.
This change also introduces engine_getBlobsV4, which allows for
cell-based responses (and partial blobs with only some of the cells).
We maintain backward compatibility with getBlobsV3 which expects full
blob responses by recovering the blob from available cells. Since this
process is resource-intensive, we proactively cache the conversion so
it is ready in time for getBlobsV3 calls. This mechanism will be
removed once support for getBlobsV4 is universal across all consensus
layer implementations.
devp2p tests for eth/72 are not part of this initial change. This is
to avoid breaking test success status for execution clients that do
not have eth/72 implemented yet. The tests will be added in a
subsequent change.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
- Move `msg.Discard` defer ahead of the max message size check in
`handleMessage`.
- Ensures oversized messages are released when the handler returns
early.
The peer dropper periodically disconnects random peers to create churn.
This was previously blind to peer quality.
This PR adds peer-score based peer protection, handling the
multi-dimensionality problem of peer scoring through the concept of
protected peer pools.
---------
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
Co-authored-by: healthykim <bsbs8645@snu.ac.kr>
This PR does a few things:
- reject `debug_setHead` if the target is even before the pivot block
(if non-nil)
- reject `debug_setHead` if in path mode, the target is not recoverable
- decouple the chain rewinding and state recovery in path mode and
recover the state in one shot
---------
Co-authored-by: jonny rhea <5555162+jrhea@users.noreply.github.com>
This PR addresses an issue in the eth71 `BlockAccessListsMsg` handler,
specifically:
- if the requested bal is not accessible in the server side, 0x80
(EmptyString) will be returned as the marker
- at the client side, old message definition
`rlp.RawList[RawBlockAccessList]` assumes all the elements are List
- the message with 0x80 (kind = string) won't be decoded correctly
- the peer will be disconnected
The message definition has been changed to `rlp.RawList[rlp.RawValue]`,
which is aligned with the one in SNAP/2 protocol.
This PR addresses the panic in tests. As the eventLoop is spun up when
the downloader was closed, the sub will be nil and make the panic
happens.
```
goroutine 421 [running]:
github.com/ethereum/go-ethereum/eth/downloader.(*DownloaderAPI).eventLoop(0xcb0e4d0)
/opt/actions-runner/_work/go-ethereum/go-ethereum/eth/downloader/api.go:91 +0x127
created by github.com/ethereum/go-ethereum/eth/downloader.NewDownloaderAPI in goroutine 352
/opt/actions-runner/_work/go-ethereum/go-ethereum/eth/downloader/api.go:50 +0xf2
```
This PR improves the block download used by snap sync. Specifically,
blocks and their associated data (receipts and canonical hash mappings)
are now written directly to the database without checking existence.
The current implementation could fail in cases where the block header
and body were already present (has.Block returns true), but the
corresponding canonical hash mapping was missing. One possible scenario
is when a newPayload event is processed without a subsequent
forkChoiceUpdate.
It is still unclear why Geth may re-enter snap sync after Engine API
events have been processed after the sync. Anyway, bypassing the
existence is a reasonable change.
What's more, in the downloader, the presence of canonical hash is also
considered for deciding the range of blocks to be downloaded.
Specifically:
- in the full sync, the block with header and body available but
canonical hash missing will be re-inserted;
- in the snap sync, the block with header, body and receipt available
but canonical hash missing will be re-inserted;
