1
0
Fork 0
forked from forks/go-ethereum
Commit graph

2814 commits

Author SHA1 Message Date
Marcel
1b18ba2423
logs(indexer)Clean up log format in head index progress messages (#31761)
updates the log entries in `core/filtermaps/indexer.go` to remove double
quotes around keys like "first block" and "last block", changing them to
`firstblock` and `lastblock`. This brings them in line with the general
logging style used across the codebase, where log keys are unquoted
single words.

For example, the log:
`  INFO [...] "first block"=..., "last block"=...`

Is now rendered as:
`  INFO [...] firstblock=..., lastblock=...`

This change improves readability and maintains consistency with logs
such as:
`  INFO [...] number=2 sealhash=... uncles=0 txs=0 ...`

No functional behavior is changed — this is purely a formatting cleanup
for better developer experience.
2025-05-05 10:09:58 +08:00
Sina M
615d29f7c2
core: reduce load on txindexer from API (#31752)
Fixes https://github.com/ethereum/go-ethereum/issues/31732.

This logic was removed in the recent refactoring in the txindexer to
handle history cutoff (#31393). It was first introduced in this PR:
https://github.com/ethereum/go-ethereum/pull/28908.

I have tested it and it works as an alternative to #31745.

This PR packs 3 changes to the flow of fetching txs from the API:

- It caches the indexer tail after each run is over to avoid hitting the
db all the time as was done originally in #28908.

- Changes `backend.GetTransaction`. It doesn't return an error anymore
when tx indexer is in progress. It shifts the responsibility to the
caller to check the progress. The reason is that in most cases we anyway
check the txpool for the tx. If it was indeed a pending tx we can avoid
the indexer progress check.

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2025-05-05 10:07:55 +08:00
Miro
2d86a54000
core/txpool/legacypool: fix data race of pricedList access (#31758) 2025-05-04 10:16:33 +08:00
Felföldi Zsolt
8868ad6d6e
core/filtermaps: fix log index initialization (#31750)
This PR fixes an initialization bug that in some cases caused the map
renderer to leave the last, partially rendered map as is and resume
rendering from the next map. At initialization we check whether the
existing rendered maps are consistent with the current chain view and
revert them if necessary. Until now this happened through an ugly hacky
solution, a "limited" chain view that was supposed to trigger a rollback
of some maps in the renderer logic if necessary. This whole setup worked
under assumptions that just weren't true any more. As a result it always
tried to revert the last map but also it did not shorten the indexed
range, only set `headIndexed` to false which indicated to the renderer
logic that the last map is fully populated (which it wasn't).
Now an explicit rollback of any unusable (reorged) maps happens at
startup, which also means that no hacky chain view is necessary, as soon
as the new `FilterMaps` is returned, the indexed range and view are
consistent with each other.

In the first commit an extra check is also added to `writeFinishedMaps`
so that if there is ever again a bug that would result in a gapped index
then it will not break the db with writing the incomplete data. Instead
it will return an indexing error which causes the indexer to revert to
unindexed mode and print an error log instantly. Hopefully this will not
ever happen in the future, but in order to test this safeguard check I
manually triggered the bug with only the first commit enabled, which
caused an indexing error as expected. With the second commit added (the
actual fix) the same operation succeeded without any issues.

Note that the database version is also bumped in this PR in order to
enforce a full reindexing as any existing database might be potentially
broken.

Fixes https://github.com/ethereum/go-ethereum/issues/31729
2025-05-03 18:40:24 +02:00
Felföldi Zsolt
341929ab96
core/filtermaps: fix log value search range (#31734)
This PR fixes the out-of-range block number logic of `getBlockLvPointer`
which sometimes caused searches to fail if the head was updated in the
wrong moment. This logic ensures that querying the pointer of a future
block returns the pointer after the last fully indexed block (instead of
failing) and therefore an async range update will not cause the search
to fail. Earier this behaviour only worked when `headIndexed` was true
and `headDelimiter` pointed to the end of the indexed range. Now it also
works for an unfinished index.

This logic is also moved from `FilterMaps.getBlockLvPointer` to
`FilterMapsMatcherBackend.GetBlockLvPointer` because it is only required
by the search anyways. `FilterMaps.getBlockLvPointer` now only returns a
pointer for existing blocks, consistently with how it is used in the
indexer/renderer.

Note that this unhandled case has been present in the code for a long
time but went unnoticed because either one of two previously fixed bugs
did prevent it from being triggered; the incorrectly positive
`tempRange.headIndexed` (fixed in
https://github.com/ethereum/go-ethereum/pull/31680), though caused other
problems, prevented this one from being triggered as with a positive
`headIndexed` no database read was triggered in `getBlockLvPointer`.
Also, the unnecessary `indexLock` in `synced()` (fixed in
https://github.com/ethereum/go-ethereum/pull/31708) usually did prevent
the search seeing the temp range and therefore avoided noticeable
issues.
2025-05-02 17:50:22 +02:00
rjl493456442
79807bc3b1
core, eth/gasestimator: introduce MaxGasUsed for estimation (#31735)
This PR improves gas estimation for data-heavy transactions which hit the floor data gas cost.
2025-05-02 17:43:06 +02:00
Marius van der Wijden
7612872761
core/filtermaps: do not derive full receipts during rendering (#31716)
This changes the filtermaps to only pull up the raw receipts, not the
derived receipts which saves a lot of allocations.

During normal execution this will reduce the allocations of the whole
geth node by ~15%.
2025-04-30 15:23:08 +08:00
Marius van der Wijden
a511553e44
core: apply overrides to mainnet chainconfig (#31733)
This PR applies the config overrides to the new config as well,
otherwise they will not be applied to defined configs, making
shadowforks impossible.

To test:
```
>  ./build/bin/geth --override.prague 123 --dev --datadir /tmp/geth
INFO [04-28|21:20:47.009]  - Prague:                      @123
> ./build/bin/geth --override.prague 321 --dev --datadir /tmp/geth
INFO [04-28|21:23:59.760]  - Prague:                      @321
``
2025-04-29 08:29:56 +02:00
Marius van der Wijden
004526762b
core/txpool/legacypool: refactor truncatePending (#31715)
TruncatePending shows up bright red on our nodes, because it computes
the length of a map multiple times.
I don't know why this is so expensive, but around 20% of our time is
spent on this, which is super weird.

```
//PR: BenchmarkTruncatePending-24    	   17498	     69397 ns/op	   32872 B/op	       3 allocs/op
//Master: BenchmarkTruncatePending-24    	    9960	    123954 ns/op	   32872 B/op	       3 allocs/op
```

```
benchmark                       old ns/op     new ns/op     delta
BenchmarkTruncatePending-24     123954        69397         -44.01%

benchmark                       old allocs     new allocs     delta
BenchmarkTruncatePending-24     3              3              +0.00%

benchmark                       old bytes     new bytes     delta
BenchmarkTruncatePending-24     32872         32872         +0.00%
```
This simple PR is a 44% improvement over the old state


``` 
OUTINE ======================== github.com/ethereum/go-ethereum/core/txpool/legacypool.(*LegacyPool).truncatePending in github.com/ethereum/go-ethereum/core/txpool/legacypool/legacypool.go
     1.96s     18.02s (flat, cum) 19.57% of Total
         .          .   1495:func (pool *LegacyPool) truncatePending() {
         .          .   1496:	pending := uint64(0)
      60ms      2.99s   1497:	for _, list := range pool.pending {
     250ms      5.48s   1498:		pending += uint64(list.Len())
         .          .   1499:	}
         .          .   1500:	if pending <= pool.config.GlobalSlots {
         .          .   1501:		return
         .          .   1502:	}
         .          .   1503:
         .          .   1504:	pendingBeforeCap := pending
         .          .   1505:	// Assemble a spam order to penalize large transactors first
         .      510ms   1506:	spammers := prque.New[int64, common.Address](nil)
     140ms      2.50s   1507:	for addr, list := range pool.pending {
         .          .   1508:		// Only evict transactions from high rollers
      50ms      5.08s   1509:		if uint64(list.Len()) > pool.config.AccountSlots {
         .          .   1510:			spammers.Push(addr, int64(list.Len()))
         .          .   1511:		}
         .          .   1512:	}
         .          .   1513:	// Gradually drop transactions from offenders
         .          .   1514:	offenders := []common.Address{}
```

```go
// Benchmarks the speed of batch transaction insertion in case of multiple accounts.
func BenchmarkTruncatePending(b *testing.B) {
	// Generate a batch of transactions to enqueue into the pool
	pool, _ := setupPool()
	defer pool.Close()
	b.ReportAllocs()
	batches := make(types.Transactions, 4096+1024+1)
	for i := range len(batches) {
		key, _ := crypto.GenerateKey()
		account := crypto.PubkeyToAddress(key.PublicKey)
		pool.currentState.AddBalance(account, uint256.NewInt(1000000), tracing.BalanceChangeUnspecified)
		tx := transaction(uint64(0), 100000, key)
		batches[i] = tx
	}
	for _, tx := range batches {
		pool.addRemotesSync([]*types.Transaction{tx})
	}
	b.ResetTimer()
	// benchmark truncating the pending
	for range b.N {
		pool.truncatePending()
	}
}
```
2025-04-28 14:26:27 +08:00
rjl493456442
b6bdd698a0
core/filtermaps: fix deadlock in filtermap callback (#31708)
This PR fixes a deadlock situation is deleteTailEpoch that might arise
when
range delete is running in iterator based fallback mode (either using
leveldb
database or the hashdb state storage scheme). 

In this case a stopCb callback is called periodically that does check
events,
including matcher sync requests, in which case it tries to acquire
indexLock
for read access, while deleteTailEpoch already held it for write access.

This pull request removes the indexLock acquiring in
`FilterMapsMatcherBackend.synced`
as this function is only called in the indexLoop.

Fixes https://github.com/ethereum/go-ethereum/issues/31700
2025-04-25 11:57:38 +02:00
colin
846c14e21f
ethclient: allow passing AuthorizationList to calls (#31198)
This PR adds the `AuthorizationList` field to the `CallMsg` interface to support `eth_call`
and `eth_estimateGas` of set-code transactions.

---------

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2025-04-23 21:01:03 +02:00
Felföldi Zsolt
14f15430bb
core/filtermaps: clone cached slices, fix tempRange (#31680)
This PR ensures that caching a slice or a slice of slices will never
affect the original version by always cloning a slice fetched from cache
if it is not used in a guaranteed read only way.
2025-04-21 09:27:24 +02:00
Felföldi Zsolt
5a7bbb423f
beacon/params, core/filtermaps: update checkpoints (#31674)
This PR updates checkpoints for blsync and filtermaps.
2025-04-20 12:54:40 +02:00
Felföldi Zsolt
7f574372d5
eth/filters, core/filtermaps: safe chain view update (#31590)
This PR changes the chain view update mechanism of the log filter.
Previously the head updates were all wired through the indexer, even in
unindexed mode. This was both a bit weird and also unsafe as the
indexer's chain view was updates asynchronously with some delay, making
some log related tests flaky. Also, the reorg safety of the indexed
search was integrated with unindexed search in a weird way, relying on
`syncRange.ValidBlocks` in the unindexed case too, with a special
condition added to only consider the head of the valid range but not the
tail in the unindexed case.

In this PR the current chain view is directly accessible through the
filter backend and unindexed search is also chain view based, making it
inherently safe. The matcher sync mechanism is now only used for indexed
search as originally intended, removing a few ugly special conditions.

The PR is currently based on top of
https://github.com/ethereum/go-ethereum/pull/31642
Together they fix https://github.com/ethereum/go-ethereum/issues/31518
and replace https://github.com/ethereum/go-ethereum/pull/31542

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2025-04-20 09:48:49 +02:00
Gabriel-Trintinalia
1296cdb748
core: fail execution if system call fails to execute (#31639)
see: https://github.com/ethereum/pm/issues/1450#issuecomment-2800911584
2025-04-19 13:42:54 +02:00
Felföldi Zsolt
4c9e7d1b18
core/filtermaps: make ChainView thread safe (#31671)
This PR makes `filtermaps.ChainView` thread safe because it is used
concurrently both by the indexer and multiple matcher threads. Even
though it represents an immutable view of the chain, adding a mutex lock
to the `blockHash` function is necessary because it does so by extending
its list of non-canonical hashes if the underlying blockchain is
changed.

The unsafe concurrency did cause a panic once after running the unit
tests for several hours and it could also happen during live operation.
2025-04-18 20:00:11 +08:00
Felföldi Zsolt
2e0ad2cb4d
core/filtermaps: only use common ancestor snapshots (#31668)
This PR makes the conditions for using a map rendering snapshot stricter
so that whenever a reorg happens, only a snapshot of a common ancestor
block can be used. The issue fixed in
https://github.com/ethereum/go-ethereum/pull/31642 originated from using
a snapshot that wasn't a common ancestor. For example in the following
reorg scenario: `A->B`, then `A->B2`, then `A->B2->C2`, then `A->B->C`
the last reorg triggered a render from snapshot `B` saved earlier. Now
this is possible under certain conditions but extra care is needed, for
example if block `B` crosses a map boundary then it should not be
allowed. With the latest fix the checks are sufficient but I realized I
would just feel safer if we disallowed this rare and risky scenario
altogether and just render from snapshot `A` after the last reorg in the
example above. The performance difference if a few milliseconds and it
occurs rarely (about once a day on Holesky, probably much more rare on
Mainnet).
Note that this PR only makes the snapshot conditions stricter and
`TestIndexerRandomRange` does check that snapshots are still used
whenever it's obviously possible (adding blocks after the current head
without a reorg) so this change can be considered safe. Also I am
running the unit tests and the fuzzer and everything seems to be fine.
2025-04-18 13:39:11 +02:00
rjl493456442
9089f9461c
eth: add tx to locals only if it has a chance of acceptance (#31618)
This pull request improves error handling for local transaction submissions.

Specifically, if a transaction fails with a temporary error but might be
accepted later, the error will not be returned to the user; instead, the
transaction will be tracked locally for resubmission. 

However, if the transaction fails with a permanent error (e.g., invalid
transaction or insufficient balance), the error will be propagated to the user.

These errors returned in the legacyPool are regarded as temporary failure:

- `ErrOutOfOrderTxFromDelegated`
- `txpool.ErrInflightTxLimitReached`
- `ErrAuthorityReserved`
- `txpool.ErrUnderpriced`
- `ErrTxPoolOverflow`
- `ErrFutureReplacePending`

Notably, InsufficientBalance is also treated as a permanent error, as
it’s highly unlikely that users will transfer funds into the sender account
after submitting the transaction. Otherwise, users may be confused—seeing
their transaction submitted but unaware that the sender lacks sufficient funds—and
continue waiting for it to be included.

---------

Co-authored-by: lightclient <lightclient@protonmail.com>
2025-04-17 21:27:48 +02:00
lightclient
13b157a461
core,params: add fork readiness indicator in logs (#31340)
closes #31310 

This has been requested a few times in the past and I think it is a nice
quality-of-life improvement for users. At a predetermined interval,
there will now be a "Fork ready" log when a future fork is scheduled,
but not yet active.

It can only possibly print after block import, which kinda avoids the
scenario where the client isn't progressing or is syncing and the user
thinks it's "ready" because it sees a ready log.

New output:

```console
INFO [03-08|21:32:57.472] Imported new potential chain segment     number=7 hash=aa24ee..f09e62 blocks=1 txs=0 mgas=0.000 elapsed="874.916µs" mgasps=0.000 snapdiffs=973.00B triediffs=7.05KiB triedirty=0.00B
INFO [03-08|21:32:57.473] Ready for fork activation                fork=Prague date="18 Mar 25 19:29 CET" remaining=237h57m0s timestamp=1,742,322,597
INFO [03-08|21:32:57.475] Chain head was updated                   number=7 hash=aa24ee..f09e62 root=19b0de..8d32f2 elapsed="129.125µs"
```

Easiest way to verify this behavior is to apply this patch and run `geth
--dev --dev.period=12`

```patch
diff --git a/params/config.go b/params/config.go
index 9c7719d901..030c4f80e7 100644
--- a/params/config.go
+++ b/params/config.go
@@ -174,7 +174,7 @@ var (
                ShanghaiTime:            newUint64(0),
                CancunTime:              newUint64(0),
                TerminalTotalDifficulty: big.NewInt(0),
-               PragueTime:              newUint64(0),
+               PragueTime:              newUint64(uint64(time.Now().Add(time.Hour * 300).Unix())),
                BlobScheduleConfig: &BlobScheduleConfig{
                        Cancun: DefaultCancunBlobConfig,
                        Prague: DefaultPragueBlobConfig,
```
2025-04-17 10:46:47 +02:00
lightclient
50b5f3125b
params: add prague timestamp for mainnet (#31535)
https://eips.ethereum.org/EIPS/eip-7600#activation

Timestamp: `1746612311`
Fork id: `0xc376cf8b`
2025-04-17 10:46:00 +02:00
Sina M
e444823394
core: fix sync reset in pruned nodes (#31638)
This is an attempt at fixing #31601. I think what happens is the startup
logic will try to get the full block body (it's `bc.loadLastState`) and
fail because genesis block has been pruned from the freezer. This will
cause it to keep repeating the reset logic, causing a deadlock.

This can happen when due to an unsuccessful sync we don't have the state
for the head (or any other state) fully, and try to redo the snap sync.

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2025-04-17 16:32:40 +08:00
Felix Lange
cb21177aa8
core: fix history pruning initialization for empty DB (#31656)
This fixes an issue where running geth with `--history.chain postmerge`
would not work on an empty database.

```
ERROR[04-16|23:11:12.913] Chain history database is pruned to unknown block tail=0
Fatal: Failed to register the Ethereum service: unexpected database tail
```
2025-04-17 10:39:21 +08:00
Miro
87974974a7
core/txpool/legacypool: fix data race of txlookup access (#31641) 2025-04-17 10:36:53 +08:00
maskpp
846d578cc3
core/state: fix log format (#31610)
Log `key` in hexadecimal string format.
2025-04-16 23:46:38 +02:00
Felföldi Zsolt
ebb3eb29d3
core/filtermaps: fix map renderer reorg issue (#31642)
This PR fixes a bug in the map renderer that sometimes used an obsolete
block log value pointer to initialize the iterator for rendering from a
snapshot. This bug was triggered by chain reorgs and sometimes caused
indexing errors and invalid search results. A few other conditions are
also made safer that were not reported to cause issues yet but could
potentially be unsafe in some corner cases. A new unit test is also
added that reproduced the bug but passes with the new fixes.

Fixes https://github.com/ethereum/go-ethereum/issues/31593
Might also fix https://github.com/ethereum/go-ethereum/issues/31589
though this issue has not been reproduced yet, but it appears to be
related to a log index database corruption around a specific block,
similarly to the other issue.

Note that running this branch resets and regenerates the log index
database. For this purpose a `Version` field has been added to
`rawdb.FilterMapsRange` which will also make this easier in the future
if a breaking database change is needed or the existing one is
considered potentially broken due to a bug, like in this case.
2025-04-16 23:30:13 +02:00
Felix Lange
48ec86abbb
core: initialize history pruning in BlockChain (#31636)
I added the history mode configuration in eth/ethconfig initially, since
it seemed like the logical place. But it turns out we need access to the
intended pruning setting at a deeper level, and it actually needs to be
integrated with the blockchain startup procedure.

With this change applied, if a node previously had its history pruned,
and is subsequently restarted **without** the `--history.chain
postmerge` flag, the `BlockChain` initialization code will now verify
the freezer tail against the known pruning point of the predefined
network and will restore pruning status. Note that this logic is quite
restrictive, we allow non-zero tail only for known networks, and only
for the specific pruning point that is defined.
2025-04-15 14:32:46 +02:00
Luis Ayuso
0c2ad07673
core/txpool: allow tx and authority regardless of admission order (#31373)
This PR proposes a change to the authorizations' validation introduced
in commit cdb66c8. These changes make the expected behavior independent
of the order of admission of authorizations, improving the
predictability of the resulting state and the usability of the system
with it.

The current implementation behavior is dependent on the transaction
submission order: This issue is related to authorities and the sender of
a transaction, and can be reproduced respecting the normal nonce rules.

The issue can be reproduced by the two following cases:
**First case**
- Given an empty pool.
- Submit transaction `{ from: B, auths [ A ] }`: is accepted.
- Submit transaction `{ from: A }`: Is accepted: it becomes the one
in-flight transaction allowed.

**Second case**
- Given an empty pool.
- Submit transaction `{ from: A }`:  is accepted
- Submit transaction `{ from: B, auths [ A ] }`: is rejected since there
is already a queued/pending transaction from A.

The expected behavior is that both sequences of events would lead to the
same sets of accepted and rejected transactions.

**Proposed changes** 
The queued/pending transactions issued from any authority of the
transaction being validated have to be counted, allowing one transaction
from accounts submitting an authorization.

- Notice that the expected behavior was explicitly forbidden in the case
"reject-delegation-from-pending-account", I believe that this behavior
conflicts to the definition of the limitation, and it is removed in this
PR. The expected behavior is tested in
"accept-authorization-from-sender-of-one-inflight-tx".
- Replacement tests have been separated to improve readability of the
acceptance test.
- The test "allow-more-than-one-tx-from-replaced-authority" has been
extended with one extra transaction, since the system would always have
accepted one transaction (but not two).
- The test "accept-one-inflight-tx-of-delegated-account" is extended to
clean-up state, avoiding leaking the delegation used into the other
tests. Additionally, replacement check is removed to be tested in its
own test case.

**Expected behavior** 
The expected behavior of the authorizations' validation shall be as
follows:

![image](https://github.com/user-attachments/assets/dbde7a1f-9679-4691-94eb-c197a0cbb823)
Notice that replacement shall be allowed, and behavior shall remain
coherent with the table, according to the replaced transaction.

---------

Co-authored-by: lightclient <lightclient@protonmail.com>
2025-04-09 19:11:24 -06:00
lightclient
60b922fd52
core/txpool: add notice to Clear that is not meant for production code (#31567)
The `Sync(..)` function explicitly says not to rely on in production
code, but it is used in `Clear(..)` so should add a similar mention.
2025-04-09 14:19:28 -06:00
Marius van der Wijden
5cc9137c9c
core/vm: optimize push2 opcode (#31267)
During my benchmarks on Holesky, around 10% of all CPU time was spent in
PUSH2
```
ROUTINE ======================== github.com/ethereum/go-ethereum/core/vm.newFrontierInstructionSet.makePush.func1 in github.com/ethereum/go-ethereum/core/vm/instructions.go
    16.38s     20.35s (flat, cum) 10.31% of Total
     740ms      740ms    976:	return func(pc *uint64, interpreter *EVMInterpreter, scope *ScopeContext) ([]byte, error) {
         .          .    977:		var (
      40ms       40ms    978:			codeLen = len(scope.Contract.Code)
     970ms      970ms    979:			start   = min(codeLen, int(*pc+1))
     200ms      200ms    980:			end     = min(codeLen, start+pushByteSize)
         .          .    981:		)
     670ms      2.39s    982:		a := new(uint256.Int).SetBytes(scope.Contract.Code[start:end])
         .          .    983:
         .          .    984:		// Missing bytes: pushByteSize - len(pushData)
     410ms      410ms    985:		if missing := pushByteSize - (end - start); missing > 0 {
         .          .    986:			a.Lsh(a, uint(8*missing))
         .          .    987:		}
    12.69s     14.94s    988:		scope.Stack.push2(*a)
      10ms       10ms    989:		*pc += size
     650ms      650ms    990:		return nil, nil
         .          .    991:	}
         .          .    992:}
```

Which is quite crazy. We have a handwritten encoder for PUSH1 already,
this PR adds one for PUSH2.

PUSH2 is the second most used opcode as shown here:
https://gist.github.com/shemnon/fb9b292a103abb02d98d64df6fbd35c8 since
it is used by solidity quite significantly. Its used ~20 times as much
as PUSH20 and PUSH32.

# Benchmarks

```
BenchmarkPush/makePush-14         	94196547	        12.27 ns/op	       0 B/op	       0 allocs/op
BenchmarkPush/push-14             	429976924	         2.829 ns/op	       0 B/op	       0 allocs/op
``` 

---------

Co-authored-by: jwasinger <j-wasinger@hotmail.com>
2025-04-08 19:57:45 +02:00
rjl493456442
2e739fce58
core/txpool: add 7702 protection to blobpool (#31526)
This pull request introduces two constraints in the blobPool:

(a) If the sender has a pending authorization or delegation, only one
in-flight
executable transaction can be cached.
(b) If the authority address in a SetCode transaction is already
reserved by
the blobPool, the transaction will be rejected.

These constraints mitigate an attack where an attacker spams the pool
with
numerous blob transactions, evicts other transactions, and then cancels
all
pending blob transactions by draining the sender’s funds if they have a
delegation.

Note, because there is no exclusive lock held between different subpools
when processing transactions, it's totally possible the SetCode
transaction
and blob transactions with conflict sender and authorities are accepted
simultaneously. I think it's acceptable as it's very hard to be
exploited.

---------

Co-authored-by: lightclient <lightclient@protonmail.com>
2025-04-08 07:46:27 -06:00
rjl493456442
553183e5de
core, eth, node: use sync write option in pebble (#31519)
Fixes #31499
2025-04-03 16:03:03 +02:00
rjl493456442
90d44e715d
core, eth/downloader: implement pruning mode sync (#31414)
This pull request introduces new sync logic for pruning mode. The downloader will now skip
insertion of block bodies and receipts before the configured history cutoff point.

Originally, in snap sync, the header chain and other components (bodies and receipts) were
inserted separately. However, in Proof-of-Stake, this separation is unnecessary since the
sync target is already verified by the CL.

To simplify the process, this pull request modifies `InsertReceiptChain` to insert headers
along with block bodies and receipts together. Besides, `InsertReceiptChain` doesn't have
the notion of reorg, as the common ancestor is always be found before the sync and extra
side chain is truncated at the beginning if they fall in the ancient store. The stale
canonical chain flags will always be rewritten by the new chain. Explicit reorg logic is
no longer required in `InsertReceiptChain`.
2025-04-03 15:16:35 +02:00
minh-bq
ee30681a8d
core/txpool: add GetMetadata to transaction pool (#31433)
This is an alternative to #31309

With eth/68, transaction announcement must have transaction type and
size. So in announceTransactions, we need to query the transaction from
transaction pool with its hash. This creates overhead in case of blob
transaction which needs to load data from billy and RLP decode. This
commit creates a lightweight lookup from transaction hash to transaction
size and a function GetMetadata to query transaction type and
transaction size given the transaction hash.

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2025-04-02 15:47:56 +08:00
Felföldi Zsolt
7e3170fb5c
core/filtermaps: add metrics (#31511)
This PR adds metrics related to map rendering and pattern matching to
the `core/filtermaps` package.
2025-04-01 14:29:20 +02:00
Sina M
bc36f2de83
eth, eth/filters: implement API error code for pruned blocks (#31361)
Implements #31275

---------

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2025-04-01 13:42:01 +02:00
Felföldi Zsolt
14d576c002
core/filtermaps: hashdb safe delete range (#31525)
This PR adds `rawdb.SafeDeleteRange` and uses it for range deletion in
`core/filtermaps`. This includes deleting the old bloombits database,
resetting the log index database and removing index data for unindexed
tail epochs (which previously weren't properly implemented for the
fallback case).
`SafeDeleteRange` either calls `ethdb.DeleteRange` if the node uses the
new path based state scheme or uses an iterator based fallback method
that safely skips trie nodes in the range if the old hash based state
scheme is used. Note that `ethdb.DeleteRange` also has its own iterator
based fallback implementation in `ethdb/leveldb`. If a path based state
scheme is used and the backing db is pebble (as it is on the majority of
new nodes) then `rawdb.SafeDeleteRange` uses the fast native range
delete.
Also note that `rawdb.SafeDeleteRange` has different semantics from
`ethdb.DeleteRange`, it does not automatically return if the operation
takes a long time. Instead it receives a `stopCallback` that can
interrupt the process if necessary. This is because in the safe mode
potentially a lot of entries are iterated without being deleted (this is
definitely the case when deleting the old bloombits database which has a
single byte prefix) and therefore restarting the process every time a
fixed number of entries have been iterated would result in a quadratic
run time in the number of skipped entries.

When running in safe mode, unindexing an epoch takes about a second,
removing bloombits takes around 10s while resetting a full log index
might take a few minutes. If a range delete operation takes a
significant amount of time then log messages are printed. Also, any
range delete operation can be interrupted by shutdown (tail uinindexing
can also be interrupted by head indexing, similarly to how tail indexing
works). If the last unindexed epoch might have "dirty" index data left
then the indexed map range points to the first valid epoch and
`cleanedEpochsBefore` points to the previous, potentially dirty one. At
startup it is always assumed that the epoch before the first fully
indexed one might be dirty. New tail maps are never rendered and also no
further maps are unindexed before the previous unindexing is properly
cleaned up.

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2025-03-31 14:47:56 +02:00
Felföldi Zsolt
c8a9a9c091
core/filtermaps: revert to unindexed mode in case of indexing error (#31500)
This PR changes log indexer error handling so that if an indexing error
happens then it disables the indexer and reverts to unindexed more
without resetting the database (except in case of a failed database
init).
Resetting the database on the first error would probably be overkill as
a client update might fix this without having to reindex the entire
history. It would also make debugging very hard. On the other hand,
these errors do not resolve themselves automatically so constantly
retrying makes no sense either. With these changes a new attempt to
resume indexing is made every time the client is restarted.
The PR also fixes https://github.com/ethereum/go-ethereum/issues/31491
which originated from the tail indexer trying to resume processing a
failed map renderer.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2025-03-28 16:17:28 +01:00
rjl493456442
32f36a6749
core/txpool: fix nonce assignment in local tracker (#31496)
Fixes #31494
2025-03-28 12:32:24 +01:00
rekyyang
141968a48b
core/txpool/legacypool: fix data race in checkDelegationLimit (#31475) 2025-03-28 15:16:37 +08:00
jwasinger
cc273cef0b
Merge pull request #31228 from curvegrid/fix-sim-rollback
eth/catalyst: force sync of txpool before clearing subpools in Rollback
2025-03-27 18:40:15 +01:00
rjl493456442
5b4a743493
core/rawdb: remove LES database stats (#31495)
This removes DB schema for LES related db entries. LES has been non-functional
since the merge.
2025-03-26 12:48:04 +01:00
Delweng
c1ff2d8ba9
core/state: fix double-increment of accountLoaded counter (#31493) 2025-03-26 12:59:40 +08:00
Shude Li
4ff5093df1
all: use fmt.Appendf instead of fmt.Sprintf where possible (#31301) 2025-03-25 14:53:02 +01:00
nethoxa
21d36f7c37
core: process EL requests in GenerateVerkleChain (#31175) 2025-03-25 14:13:05 +01:00
rjl493456442
a14b8eca04
core/txpool: reject stale transaction for local tracking (#31473)
Fixes https://github.com/ethereum/go-ethereum/issues/31451
2025-03-25 11:16:26 +01:00
Felföldi Zsolt
cbe902d5da
core/filtermaps: fix log indexer init conditions (#31455)
This PR adds an extra condition to the log indexer initialization in
order to avoid initializing with block 0 as target head. Previously this
caused the indexer to initialize without a checkpoint. Later, when the
real chain head was set, it indexed the entire history, then unindexed
most of it if only the recent history was supposed to be indexed. Now
the init only happens when there is an actual synced chain head and
therefore the index is initialized at the most recent checkpoint and
only the last year is indexed according to the default parameters.
During checkpoint initialization the best available checkpoint is also
checked against the history cutoff point and fails if the indexing would
have to start from a block older than the cutoff. If initialization
fails then the indexer reverts to unindexed mode instead of retrying
because the the failure conditions cannot be expected to recover later.
2025-03-24 12:27:11 +01:00
Felix Lange
fd4049dc1e
core/rawdb: improve database stats output (#31463)
Instead of reporting all filtermaps stuff in one line, I'm breaking it
down into the three separate kinds of entries here.

```
+-----------------------+-----------------------------+------------+------------+
|       DATABASE        |          CATEGORY           |    SIZE    |   ITEMS    |
+-----------------------+-----------------------------+------------+------------+
| Key-Value store       | Log index filter-map rows   | 59.21 GiB  |  616077345 |
| Key-Value store       | Log index last-block-of-map | 12.35 MiB  |     269755 |
| Key-Value store       | Log index block-lv          | 421.70 MiB |   22109169 |
```

Also added some other changes to make it easier to debug:

- restored bloombits into the inspect output, so we notice if it doesn't
get deleted for some reason
- tracking of unaccounted key examples
2025-03-24 10:07:38 +01:00
Sina M
8fe09df54f
cmd/geth: add prune history command (#31384)
This adds a new subcommand 'geth prune-history' that removes the pre-merge history
on supported networks. Geth is not fully ready to work in this mode, please do not run
this command on your production node.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2025-03-21 13:12:56 +01:00
Sina M
1886922264
core: respect history cutoff in txindexer (#31393)
In #31384 we unindex TXes prior to the merge block. However when the
node starts up it will try to re-index those back if the config is to index the
whole chain. This change makes the indexer aware of the history cutoff block,
avoiding reindexing in that segment.

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2025-03-21 11:29:51 +01:00
Felföldi Zsolt
07cca7ab9f
core/bloombits: remove old bloombits logic and chain indexer (#31081)
This PR is #3 of a 3-part series that implements the new log index
intended to replace core/bloombits.
Based on https://github.com/ethereum/go-ethereum/pull/31079 and
https://github.com/ethereum/go-ethereum/pull/31080
Replaces https://github.com/ethereum/go-ethereum/pull/30370

This part removes the old bloombits package and the chain indexer that
was only used by bloombits. Deletes the old bloombits database.

FilterMaps data structure explanation:
https://gist.github.com/zsfelfoldi/a60795f9da7ae6422f28c7a34e02a07e

Log index generator code overview:
https://gist.github.com/zsfelfoldi/97105dff0b1a4f5ed557924a24b9b9e7

Search pattern matcher code overview:
https://gist.github.com/zsfelfoldi/5981735641c956afb18065e84f8aff34

Note that the possibility of a tree hashing scheme and remote proof
protocol are mentioned in the documents above but they are not exactly
specified yet. These specs are WIP and will be finalized after the local
log indexer/filter code is finalized and merged.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2025-03-21 10:47:58 +01:00