Adding an RPC flag to limit the block range size for eth_getLogs and
eth_newFilter requests.
closing https://github.com/ethereum/go-ethereum/issues/24508
---------
Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
This PR introduces a new debug feature, logging the slow blocks with
detailed performance statistics, such as state read, EVM execution and
so on.
Notably, the detailed performance statistics of slow blocks won't be
logged during the sync to not overwhelm users. Specifically, the statistics
are only logged if there is a single block processed.
Example output
```
########## SLOW BLOCK #########
Block: 23537063 (0xa7f878611c2dd27f245fc41107d12ebcf06b4e289f1d6acf44d49a169554ee09) txs: 248, mgasps: 202.99
EVM execution: 63.295ms
Validation: 1.130ms
Account read: 6.634ms(648)
Storage read: 17.391ms(1434)
State hash: 6.722ms
DB commit: 3.260ms
Block write: 1.954ms
Total: 99.094ms
State read cache: account (hit: 622, miss: 26), storage (hit: 1325, miss: 109)
##############################
```
New RPC method eth_sendRawTransactionSync(rawTx, timeoutMs?) that
submits a signed tx and blocks until a receipt is available or a timeout
elapses.
Two CLI flags to tune server-side limits:
--rpc.txsync.defaulttimeout (default wait window)
--rpc.txsync.maxtimeout (upper bound; requests are clamped)
closes https://github.com/ethereum/go-ethereum/issues/32094
---------
Co-authored-by: aodhgan <gawnieg@gmail.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Add cli configurable limit for the number of addresses allowed in
eth_getLogs filter criteria:
https://github.com/ethereum/go-ethereum/issues/32264
Key changes:
- Added --rpc.getlogmaxaddrs CLI flag (default: 1000) to configure the
maximum number of addresses
- Updated ethconfig.Config with FilterMaxAddresses field for
configuration management
- Modified filter system to use the configurable limit instead of the
hardcoded maxAddresses constant
- Enhanced test coverage with new test cases for address limit
validation
- Removed hardcoded validation from JSON unmarshaling, moving it to
runtime validation
Please notice that I remove the check at FilterCriteria UnmarshalJSON
because the runtime config can not pass into this validation.
Please help review this change!
---------
Co-authored-by: zsfelfoldi <zsfelfoldi@gmail.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
The format that is currently reported by the chain isn't very useful, as
it gives an average for ALL the nodes, and not only the leaves, which
skews the results.
Also, until now there was no way to activate the reporting of errors.
We also decided that metrics weren't the right tool to report this data,
so we decided to dump it to the console if the flag is enabled. A better
system should be built, but for now, printing to the logs does the job.
Add state size tracking and retrieve api, start geth with `--state.size-tracking`,
the initial bootstrap is required (around 1h on mainnet), after the bootstrap,
use `debug_stateSize()` RPC to retrieve the state size:
```
> debug.stateSize()
{
accountBytes: "0x39681967b",
accountTrienodeBytes: "0xc57939f0c",
accountTrienodes: "0x198b36ac",
accounts: "0x129da14a",
blockNumber: "0x1635e90",
contractCodeBytes: "0x2b63ef481",
contractCodes: "0x1c7b45",
stateRoot: "0x9c36a3ec3745d72eea8700bd27b90dcaa66de0494b187c5600750044151e620a",
storageBytes: "0x18a6e7d3f1",
storageTrienodeBytes: "0x2e7f53fae6",
storageTrienodes: "0x6e49a234",
storages: "0x517859c5"
}
```
---------
Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This implements a backing store for chain history based on era1 files.
The new store is integrated with the freezer. Queries for blocks and receipts
below the current freezer tail are handled by the era store.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
I added the history mode configuration in eth/ethconfig initially, since
it seemed like the logical place. But it turns out we need access to the
intended pruning setting at a deeper level, and it actually needs to be
integrated with the blockchain startup procedure.
With this change applied, if a node previously had its history pruned,
and is subsequently restarted **without** the `--history.chain
postmerge` flag, the `BlockChain` initialization code will now verify
the freezer tail against the known pruning point of the predefined
network and will restore pruning status. Note that this logic is quite
restrictive, we allow non-zero tail only for known networks, and only
for the specific pruning point that is defined.
Here I am adding a config option and geth flag (`--history.chain`) for
configuring history pruning. There are two options available:
- `--history.chain all` is the default and will keep all history like
before.
- `--history.chain postmerge` will configure the history cutoff point to
the merge block.
The option doesn't actually do anything right now, but we need it as a
precursor for other history pruning changes.
Lots of packages depend on eth/downloader just for the SyncMode type.
Since we have a dedicated package for eth protocol configuration, it
makes more sense to define SyncMode there, turning eth/downloader into
more of a leaf package.
This updates the message you get when trying to initialize Geth with
genesis.json that doesn't have `terminalTotalDifficulty`. The previous
message was a bit obscure, I had to check the code to find out what the
problem was.
rebased https://github.com/ethereum/go-ethereum/pull/29766 . The
downstream branch appears to have been deleted and I don't have perms to
push to that fork.
`TerminalTotalDifficultyPassed` is removed. `TerminalTotalDifficulty`
must now be non-nil, and it is expected that networks are already
merged: we can only import PoW/Clique chains, not produce blocks on
them.
---------
Co-authored-by: stevemilk <wangpeculiar@gmail.com>
This PR integrates witness-enabled block production, witness-creating
payload execution and stateless cross-validation into the `engine` API.
The purpose of the PR is to enable the following use-cases (for API
details, please see next section):
- Cross validating locally created blocks:
- Call `forkchoiceUpdatedWithWitness` instead of `forkchoiceUpdated` to
trigger witness creation too.
- Call `getPayload` as before to retrieve the new block and also the
above created witness.
- Call `executeStatelessPayload` against another client to
cross-validate the block.
- Cross validating locally processed blocks:
- Call `newPayloadWithWitness` instead of `newPayload` to trigger
witness creation too.
- Call `executeStatelessPayload` against another client to
cross-validate the block.
- Block production for stateless clients (local or MEV builders):
- Call `forkchoiceUpdatedWithWitness` instead of `forkchoiceUpdated` to
trigger witness creation too.
- Call `getPayload` as before to retrieve the new block and also the
above created witness.
- Propagate witnesses across the consensus libp2p network for stateless
Ethereum.
- Stateless validator validation:
- Call `executeStatelessPayload` with the propagated witness to
statelessly validate the block.
*Note, the various `WithWitness` methods could also *just be* an
additional boolean flag on the base methods, but this PR wanted to keep
the methods separate until a final consensus is reached on how to
integrate in production.*
---
The following `engine` API types are introduced:
```go
// StatelessPayloadStatusV1 is the result of a stateless payload execution.
type StatelessPayloadStatusV1 struct {
Status string `json:"status"`
StateRoot common.Hash `json:"stateRoot"`
ReceiptsRoot common.Hash `json:"receiptsRoot"`
ValidationError *string `json:"validationError"`
}
```
- Add `forkchoiceUpdatedWithWitnessV1,2,3` with same params and returns
as `forkchoiceUpdatedV1,2,3`, but triggering a stateless witness
building if block production is requested.
- Extend `getPayloadV2,3` to return `executionPayloadEnvelope` with an
additional `witness` field of type `bytes` iff created via
`forkchoiceUpdatedWithWitnessV2,3`.
- Add `newPayloadWithWitnessV1,2,3,4` with same params and returns as
`newPayloadV1,2,3,4`, but triggering a stateless witness creation during
payload execution to allow cross validating it.
- Extend `payloadStatusV1` with a `witness` field of type `bytes` if
returned by `newPayloadWithWitnessV1,2,3,4`.
- Add `executeStatelessPayloadV1,2,3,4` with same base params as
`newPayloadV1,2,3,4` and one more additional param (`witness`) of type
`bytes`. The method returns `statelessPayloadStatusV1`, which mirrors
`payloadStatusV1` but replaces `latestValidHash` with `stateRoot` and
`receiptRoot`.
Here we add a Go API for running tracing plugins within the main block import process.
As an advanced user of geth, you can now create a Go file in eth/tracers/live/, and within
that file register your custom tracer implementation. Then recompile geth and select your tracer
on the command line. Hooks defined in the tracer will run whenever a block is processed.
The hook system is defined in package core/tracing. It uses a struct with callbacks, instead of
requiring an interface, for several reasons:
- We plan to keep this API stable long-term. The core/tracing hook API does not depend on
on deep geth internals.
- There are a lot of hooks, and tracers will only need some of them. Using a struct allows you
to implement only the hooks you want to actually use.
All existing tracers in eth/tracers/native have been rewritten to use the new hook system.
This change breaks compatibility with the vm.EVMLogger interface that we used to have.
If you are a user of vm.EVMLogger, please migrate to core/tracing, and sorry for breaking
your stuff. But we just couldn't have both the old and new tracing APIs coexist in the EVM.
---------
Co-authored-by: Matthieu Vachon <matthieu.o.vachon@gmail.com>
Co-authored-by: Delweng <delweng@gmail.com>
Co-authored-by: Martin HS <martin@swende.se>
* eth: drop support for forward sync triggers and head block packets
* consensus, eth: enforce always merged network
* eth: fix tx looper startup and shutdown
* cmd, core: fix some tests
* core: remove notion of future blocks
* core, eth: drop unused methods and types
* cmd, les, tests: remove light client code
This commit removes the light client (LES) code.
Since the merge the light client has been broken and
it is hard to maintain it alongside the normal client.
We decided it would be best to remove it for now and
maybe rework and reintroduce it in the future.
* cmd, eth: remove some more mentions of light mode
* cmd: re-add flags and mark as deprecated
* cmd: warn the user about deprecated flags
* eth: better error message
* cmd, core: resolve scheme from a read-write database
* cmd, core, eth: move the scheme check in the ethereum constructor
* cmd/geth: dump should in ro mode
* cmd: reverts
* core/blobpool: implement txpool for blob txs
* core/txpool: track address reservations to notice any weird bugs
* core/txpool/blobpool: add support for in-memory operation for tests
* core/txpool/blobpool: fix heap updating after SetGasTip if account is evicted
* core/txpool/blobpool: fix eviction order if cheap leading txs are included
* core/txpool/blobpool: add note as to why the eviction fields are not inited in reinject
* go.mod: pull in inmem billy form upstream
* core/txpool/blobpool: fix review commens
* core/txpool/blobpool: make heap and heap test deterministic
* core/txpool/blobpool: luv u linter
* core/txpool: limit blob transactions to 16 per account
* core/txpool/blobpool: fix rebase errors
* core/txpool/blobpool: luv you linter
* go.mod: revert some strange crypto package dep updates
This change removes PoW header syncing related code from LES and also deletes
duplicated packages les/catalyst, les/downloader and les/fetcher. These package copies
were created because people wanted to make changes in their eth/ counterparts, but weren't
able to adapt LES code to the API changes.
The clean trie cache is persisted periodically, therefore Geth can
quickly warmup the cache in next restart.
However it will reduce the robustness of system. The assumption is
held in Geth that if the parent trie node is present, then the entire
sub-trie associated with the parent are all prensent.
Imagine the scenario that Geth rewinds itself to a past block and
restart, but Geth finds the root node of "future state" in clean
cache then regard this state is present in disk, while is not in fact.
Another example is offline pruning tool. Whenever an offline pruning
is performed, the clean cache file has to be removed to aviod hitting
the root node of "deleted states" in clean cache.
All in all, compare with the minor performance gain, system robustness
is something we care more.
* all: move main transaction pool into a subpool
* go.mod: remove superfluous updates
* core/txpool: review fixes, handle txs rejected by all subpools
* core/txpool: typos
* all: remove notion of trusted checkpoints in the post-merge world
* light: remove unused function
* eth/ethconfig, les: remove unused config option
* les: make linter happy
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR changes the API so that uint64 is used for fork timestamps.
It's a good choice because types.Header also uses uint64 for time.
Co-authored-by: Felix Lange <fjl@twurst.com>
--syncTarget is a feature for development purpose in post-merge world. Previously
it's added into eth.Config. But it turns out that's a stupid idea.
- syncTarget is a block object, which is hard to be put in config file(large)
- syncTarget is just a dev feature, doesn't make too much sense to add it in config file
So I remove it from the eth config object. And it also fixes the #26328
This PR adds a parameter to startup, --synctarget. The synctarget flag is a developer-flag, that can be useful in some scenarios as a replacement for a CL node. It defines a fixed block sync target:
geth --syncmode=full --synctarget=./block_15816882.hex_rlp
The --synctarget is only made available during syncmode=full
This adds a cache for block logs which is shared by all filters. The cache
size of is configurable using the `--cache.blocklogs` flag.
Co-authored-by: Felix Lange <fjl@twurst.com>
