This PR improves the speed of Disc/v4 and Disc/v5 based discovery by
adding a prefetch buffer to discovery sources, eliminating slowdowns
due to timeouts and rate mismatch between the two processes.
Since we now want to filter the discv4 nodes iterator, it is being removed
from the default discovery mix in p2p.Server. To keep backwards-compatibility,
the default unfiltered discovery iterator will be utilized by the server when
no protocol-specific discovery is configured.
---------
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
This fixes a regression introduced by #29158 where receipts of empty blocks
were stored into the database as an empty byte array, instead of an RLP empty list.
Fixes#31938
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
Fixes an issue where querying logs for block ranges starting from 0 would fail with an irrelevant
error on a pruned node. Now the correct "history is pruned" error will be returned.
This situation was failing quietly for me recently when I had a partial
data corruption issue. Changing the log level to Error would increase
visibility for me.
This implements a backing store for chain history based on era1 files.
The new store is integrated with the freezer. Queries for blocks and receipts
below the current freezer tail are handled by the era store.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
This changes the API backend to return null for not-found blocks. This behavior
is required by the RPC When `BlockByNumberOrHash` always returned an error
for this case ever since being added in https://github.com/ethereum/go-ethereum/pull/19491.
The backend method has a couple of call sites, and all of them handle a `nil`
block result because `BlockByNumber` returns `nil` for not-found.
The only case where this makes a real difference is for `eth_getBlockReceipts`,
which was changed in #31361 to actually forward the error from `BlockByNumberOrHash`
to the caller.
This PR introduces a new native tracer for AA bundlers. Bundlers participating in the alternative
mempool will need to validate userops. This tracer will return sufficient information for them to
decide whether griefing is possible. Resolves#30546
---------
Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
This PR implements eth/69. This protocol version drops the bloom filter
from receipts messages, reducing the amount of data needed for a sync
by ~530GB (2.3B txs * 256 byte) uncompressed. Compressed this will
be reduced to ~100GB
The new version also changes the Status message and introduces the
BlockRangeUpdate message to relay information about the available history
range.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
In this pull request, snapshot generation in pathdb has been ported from
the legacy state snapshot implementation. Additionally, when running in
path mode, legacy state snapshot data is now managed by the pathdb
based snapshot logic.
Note: Existing snapshot data will be re-generated, regardless of whether
it was previously fully constructed.
This PR contains three refactors:
- refactor the latest fork check that we use quite extensively
- refactor the nil checks in NewPayloads
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
This PR introduces an allocation-free version of the
Transaction.EffectiveGasTip method to improve performance by reducing
memory allocations.
## Changes
- Added a new `EffectiveGasTipInto` method that accepts a destination
parameter to avoid memory allocations
- Refactored the existing `EffectiveGasTip` method to use the new
allocation-free implementation
- Updated related methods (`EffectiveGasTipValue`, `EffectiveGasTipCmp`,
`EffectiveGasTipIntCmp`) to use the allocation-free approach
- Added tests and benchmarks to verify correctness and measure
performance improvements
## Motivation
In high-transaction-volume environments, the `EffectiveGasTip` method is
called frequently. Reducing memory allocations in this method decreases
garbage collection pressure and improves overall system performance.
## Benchmark Results
As-Is
BenchmarkEffectiveGasTip/Original-10 42089140 27.45 ns/op 8 B/op 1
allocs/op
To-Be
BenchmarkEffectiveGasTip/IntoMethod-10 72353263 16.73 ns/op 0 B/op 0
allocs/op
## Summary of Improvements
- **Performance**: ~39% faster execution (27.45 ns/op → 16.73 ns/op)
- **Memory**: Eliminated all allocations (8 B/op → 0 B/op)
- **Allocation count**: Reduced from 1 to 0 allocations per operation
This optimization follows the same pattern successfully applied to other
methods in the codebase, maintaining API compatibility while improving
performance.
## Safety & Compatibility
This optimization has no side effects or adverse impacts because:
- It maintains functional equivalence as confirmed by comprehensive
tests
- It preserves API compatibility with existing callers
- It follows clear memory ownership patterns with the destination
parameter
- It maintains thread safety by only modifying the caller-provided
destination parameter
This optimization follows the same pattern successfully applied to other
methods in the codebase, providing better performance without
compromising stability or correctness.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
Fixes methods debug_standardTraceBlockToFile
and debug_standardTraceBadBlockToFile which were
outputting empty files.
---------
Co-authored-by: maskpp <maskpp266@gmail.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Fixes https://github.com/ethereum/go-ethereum/issues/31732.
This logic was removed in the recent refactoring in the txindexer to
handle history cutoff (#31393). It was first introduced in this PR:
https://github.com/ethereum/go-ethereum/pull/28908.
I have tested it and it works as an alternative to #31745.
This PR packs 3 changes to the flow of fetching txs from the API:
- It caches the indexer tail after each run is over to avoid hitting the
db all the time as was done originally in #28908.
- Changes `backend.GetTransaction`. It doesn't return an error anymore
when tx indexer is in progress. It shifts the responsibility to the
caller to check the progress. The reason is that in most cases we anyway
check the txpool for the tx. If it was indeed a pending tx we can avoid
the indexer progress check.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This changes the filtermaps to only pull up the raw receipts, not the
derived receipts which saves a lot of allocations.
During normal execution this will reduce the allocations of the whole
geth node by ~15%.
This PR changes the chain view update mechanism of the log filter.
Previously the head updates were all wired through the indexer, even in
unindexed mode. This was both a bit weird and also unsafe as the
indexer's chain view was updates asynchronously with some delay, making
some log related tests flaky. Also, the reorg safety of the indexed
search was integrated with unindexed search in a weird way, relying on
`syncRange.ValidBlocks` in the unindexed case too, with a special
condition added to only consider the head of the valid range but not the
tail in the unindexed case.
In this PR the current chain view is directly accessible through the
filter backend and unindexed search is also chain view based, making it
inherently safe. The matcher sync mechanism is now only used for indexed
search as originally intended, removing a few ugly special conditions.
The PR is currently based on top of
https://github.com/ethereum/go-ethereum/pull/31642
Together they fix https://github.com/ethereum/go-ethereum/issues/31518
and replace https://github.com/ethereum/go-ethereum/pull/31542
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
The API `eth_feeHistory` returns
`{"jsonrpc":"2.0","id":1,"error":{"code":-32603,"message":"json:
unsupported value: NaN"}}`, when we query `eth_feeHistory` with a old
block that without a blob, or when the field
`config.blobSchedule.cancun.max` in genesis.config is 0 (that happens
for some projects fork geth but they don't have blob).
So here we specially handle the case when maxBlobGas == 0 to prevent
this issue from happening.
This pull request improves error handling for local transaction submissions.
Specifically, if a transaction fails with a temporary error but might be
accepted later, the error will not be returned to the user; instead, the
transaction will be tracked locally for resubmission.
However, if the transaction fails with a permanent error (e.g., invalid
transaction or insufficient balance), the error will be propagated to the user.
These errors returned in the legacyPool are regarded as temporary failure:
- `ErrOutOfOrderTxFromDelegated`
- `txpool.ErrInflightTxLimitReached`
- `ErrAuthorityReserved`
- `txpool.ErrUnderpriced`
- `ErrTxPoolOverflow`
- `ErrFutureReplacePending`
Notably, InsufficientBalance is also treated as a permanent error, as
it’s highly unlikely that users will transfer funds into the sender account
after submitting the transaction. Otherwise, users may be confused—seeing
their transaction submitted but unaware that the sender lacks sufficient funds—and
continue waiting for it to be included.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
closes#31401
---------
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
BroadcastTransactions needs the Sender address to route message flows
from the same Sender address consistently to the same random subset of
peers. It however spent considerable time calculating the Sender
addresses, even if the Sender address was already calculated and cached
in other parts of the code.
Since we only need the mapping, we can use any signer, and the one that
had already been used is a better choice because of cache reuse.
I added the history mode configuration in eth/ethconfig initially, since
it seemed like the logical place. But it turns out we need access to the
intended pruning setting at a deeper level, and it actually needs to be
integrated with the blockchain startup procedure.
With this change applied, if a node previously had its history pruned,
and is subsequently restarted **without** the `--history.chain
postmerge` flag, the `BlockChain` initialization code will now verify
the freezer tail against the known pruning point of the predefined
network and will restore pruning status. Note that this logic is quite
restrictive, we allow non-zero tail only for known networks, and only
for the specific pruning point that is defined.
As of now, Geth disconnects peers only on protocol error or timeout,
meaning once connection slots are filled, the peerset is largely fixed.
As mentioned in https://github.com/ethereum/go-ethereum/issues/31321,
Geth should occasionally disconnect peers to ensure some churn.
What/when to disconnect could depend on:
- the state of geth (e.g. sync or not)
- current number of peers
- peer level metrics
This PR adds a very slow churn using a random drop.
---------
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
before this changes, this will result in numerous test failures:
```
> go test -run=Eth2AssembleBlock -c
> stress ./catalyst.test
```
The reason is that after creating/inserting the test chain, there is a
race between the txpool head reset and the promotion of txs added from
tests.
Ensuring that the txpool state is up to date with the head of the chain
before proceeding fixes these flaky tests.
Fixes#31169
The TestTransactionForgotten test was flaky due to real time
dependencies. This PR:
- Replaces real time with mock clock for deterministic timing control
- Adds precise state checks at timeout boundaries
- Verifies underpriced cache states and cleanup
- Improves test reliability by controlling transaction timestamps
- Adds checks for transaction re-enqueueing behavior
The changes ensure consistent test behavior without timing-related
flakiness.
---------
Co-authored-by: Zsolt Felfoldi <zsfelfoldi@gmail.com>
This pull request introduces two constraints in the blobPool:
(a) If the sender has a pending authorization or delegation, only one
in-flight
executable transaction can be cached.
(b) If the authority address in a SetCode transaction is already
reserved by
the blobPool, the transaction will be rejected.
These constraints mitigate an attack where an attacker spams the pool
with
numerous blob transactions, evicts other transactions, and then cancels
all
pending blob transactions by draining the sender’s funds if they have a
delegation.
Note, because there is no exclusive lock held between different subpools
when processing transactions, it's totally possible the SetCode
transaction
and blob transactions with conflict sender and authorities are accepted
simultaneously. I think it's acceptable as it's very hard to be
exploited.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
This pull request introduces new sync logic for pruning mode. The downloader will now skip
insertion of block bodies and receipts before the configured history cutoff point.
Originally, in snap sync, the header chain and other components (bodies and receipts) were
inserted separately. However, in Proof-of-Stake, this separation is unnecessary since the
sync target is already verified by the CL.
To simplify the process, this pull request modifies `InsertReceiptChain` to insert headers
along with block bodies and receipts together. Besides, `InsertReceiptChain` doesn't have
the notion of reorg, as the common ancestor is always be found before the sync and extra
side chain is truncated at the beginning if they fall in the ancient store. The stale
canonical chain flags will always be rewritten by the new chain. Explicit reorg logic is
no longer required in `InsertReceiptChain`.
This is an alternative to #31309
With eth/68, transaction announcement must have transaction type and
size. So in announceTransactions, we need to query the transaction from
transaction pool with its hash. This creates overhead in case of blob
transaction which needs to load data from billy and RLP decode. This
commit creates a lightweight lookup from transaction hash to transaction
size and a function GetMetadata to query transaction type and
transaction size given the transaction hash.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR adds `rawdb.SafeDeleteRange` and uses it for range deletion in
`core/filtermaps`. This includes deleting the old bloombits database,
resetting the log index database and removing index data for unindexed
tail epochs (which previously weren't properly implemented for the
fallback case).
`SafeDeleteRange` either calls `ethdb.DeleteRange` if the node uses the
new path based state scheme or uses an iterator based fallback method
that safely skips trie nodes in the range if the old hash based state
scheme is used. Note that `ethdb.DeleteRange` also has its own iterator
based fallback implementation in `ethdb/leveldb`. If a path based state
scheme is used and the backing db is pebble (as it is on the majority of
new nodes) then `rawdb.SafeDeleteRange` uses the fast native range
delete.
Also note that `rawdb.SafeDeleteRange` has different semantics from
`ethdb.DeleteRange`, it does not automatically return if the operation
takes a long time. Instead it receives a `stopCallback` that can
interrupt the process if necessary. This is because in the safe mode
potentially a lot of entries are iterated without being deleted (this is
definitely the case when deleting the old bloombits database which has a
single byte prefix) and therefore restarting the process every time a
fixed number of entries have been iterated would result in a quadratic
run time in the number of skipped entries.
When running in safe mode, unindexing an epoch takes about a second,
removing bloombits takes around 10s while resetting a full log index
might take a few minutes. If a range delete operation takes a
significant amount of time then log messages are printed. Also, any
range delete operation can be interrupted by shutdown (tail uinindexing
can also be interrupted by head indexing, similarly to how tail indexing
works). If the last unindexed epoch might have "dirty" index data left
then the indexed map range points to the first valid epoch and
`cleanedEpochsBefore` points to the previous, potentially dirty one. At
startup it is always assumed that the epoch before the first fully
indexed one might be dirty. New tail maps are never rendered and also no
further maps are unindexed before the previous unindexing is properly
cleaned up.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR adds an extra condition to the log indexer initialization in
order to avoid initializing with block 0 as target head. Previously this
caused the indexer to initialize without a checkpoint. Later, when the
real chain head was set, it indexed the entire history, then unindexed
most of it if only the recent history was supposed to be indexed. Now
the init only happens when there is an actual synced chain head and
therefore the index is initialized at the most recent checkpoint and
only the last year is indexed according to the default parameters.
During checkpoint initialization the best available checkpoint is also
checked against the history cutoff point and fails if the indexing would
have to start from a block older than the cutoff. If initialization
fails then the indexer reverts to unindexed mode instead of retrying
because the the failure conditions cannot be expected to recover later.
This pull request fixes a broken unit test
```
=== CONT TestTracingWithOverrides
api_test.go:1012: result: {"gas":21167,"failed":false,"returnValue":"0x0000000000000000000000000000000000000000000000000000000000000002","structLogs":[{"pc":0,"op":"PUSH1","gas":24978860,"gasCost":3,"depth":1,"stack":[]},{"pc":2,"op":"CALLDATALOAD","gas":24978857,"gasCost":3,"depth":1,"stack":["0x0"]},{"pc":3,"op":"PUSH1","gas":24978854,"gasCost":3,"depth":1,"stack":["0x1"]},{"pc":5,"op":"ADD","gas":24978851,"gasCost":3,"depth":1,"stack":["0x1","0x1"]},{"pc":6,"op":"PUSH1","gas":24978848,"gasCost":3,"depth":1,"stack":["0x2"]},{"pc":8,"op":"MSTORE","gas":24978845,"gasCost":6,"depth":1,"stack":["0x2","0x0"]},{"pc":9,"op":"PUSH1","gas":24978839,"gasCost":3,"depth":1,"stack":[]},{"pc":11,"op":"PUSH1","gas":24978836,"gasCost":3,"depth":1,"stack":["0x20"]},{"pc":13,"op":"RETURN","gas":24978833,"gasCost":0,"depth":1,"stack":["0x20","0x0"]}]}
api_test.go:1013: test 10, result mismatch, have
{21167 false 0x0000000000000000000000000000000000000000000000000000000000000002}
, want
{21167 false 0000000000000000000000000000000000000000000000000000000000000002}
api_test.go:1012: result: {"gas":25664,"failed":false,"returnValue":"0x000000000000000000000000c6e93f4c1920eaeaa1e699f76a7a8c18e3056074","structLogs":[]}
api_test.go:1013: test 11, result mismatch, have
{25664 false 0x000000000000000000000000c6e93f4c1920eaeaa1e699f76a7a8c18e3056074}
, want
{25664 false 000000000000000000000000c6e93f4c1920eaeaa1e699f76a7a8c18e3056074}
```
This is a **breaking change** to the opcode tracer. The top-level
`returnValue` field of a trace will be now hex-encoded. If the return
data is empty, this field will contain "0x".
Fixes#31196
