TestBlobTxWithoutSidecar test could run infinitely in case of a client
not requesting the good transaction. This adds a timeout to make the
test fail in this case.
Fixes https://github.com/ethereum/go-ethereum/issues/32422
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
Add state size tracking and retrieve api, start geth with `--state.size-tracking`,
the initial bootstrap is required (around 1h on mainnet), after the bootstrap,
use `debug_stateSize()` RPC to retrieve the state size:
```
> debug.stateSize()
{
accountBytes: "0x39681967b",
accountTrienodeBytes: "0xc57939f0c",
accountTrienodes: "0x198b36ac",
accounts: "0x129da14a",
blockNumber: "0x1635e90",
contractCodeBytes: "0x2b63ef481",
contractCodes: "0x1c7b45",
stateRoot: "0x9c36a3ec3745d72eea8700bd27b90dcaa66de0494b187c5600750044151e620a",
storageBytes: "0x18a6e7d3f1",
storageTrienodeBytes: "0x2e7f53fae6",
storageTrienodes: "0x6e49a234",
storages: "0x517859c5"
}
```
---------
Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
As in #32060 we introduced the file based journal path, for the other
sub command(eg: snapshot, db), we should also pass the directory to the
triedb, else the subcommand(eg: `geth snapshot`) failed to run:
```bash
geth snapshot verify-state --datadir /geth-data
...
INFO [09-02|02:12:29.493] Allocated cache and file handles database=/geth-data/geth/chaindata cache=512.00MiB handles=524,288
INFO [09-02|02:12:32.746] Opened ancient database database=/geth-data/geth/chaindata/ancient/chain readonly=true
INFO [09-02|02:12:32.746] Opened Era store datadir=/geth-data/geth/chaindata/ancient/chain/era
INFO [09-02|02:12:32.758] State scheme set to already existing scheme=path
INFO [09-02|02:12:32.760] Load database journal from disk
INFO [09-02|02:12:32.764] Failed to load journal, discard it err="journal not found"
INFO [09-02|02:12:32.789] Opened ancient database database=/geth-data/geth/chaindata/ancient/state readonly=true
INFO [09-02|02:12:32.790] Initialized path database readonly=true triecache=0.00B statecache=0.00B buffer=0.00B history="entire chain"
ERROR[09-02|02:12:32.791] Failed to verify state root=c5458d..4cc785 err="unknown layer: c5458d476da0136a67ef24a93b909aa5c29efa5c5b885dbd1fbaed4e784cc785"
```
The main purpose of this change is to enforce the version setting when
constructing the blobSidecar, avoiding creating sidecar with wrong/default
version tag.
This is something interesting I came across during my benchmarks, we
spent ~3.8% of all allocations allocating the header number on the heap.
```
(pprof) list GetHeaderByHash
Total: 38197204475
ROUTINE ======================== github.com/ethereum/go-ethereum/core.(*BlockChain).GetHeaderByHash in github.com/ethereum/go-ethereum/core/blockchain_reader.go
0 5786566117 (flat, cum) 15.15% of Total
. . 79:func (bc *BlockChain) GetHeaderByHash(hash common.Hash) *types.Header {
. 5786566117 80: return bc.hc.GetHeaderByHash(hash)
. . 81:}
. . 82:
. . 83:// GetHeaderByNumber retrieves a block header from the database by number,
. . 84:// caching it (associated with its hash) if found.
. . 85:func (bc *BlockChain) GetHeaderByNumber(number uint64) *types.Header {
ROUTINE ======================== github.com/ethereum/go-ethereum/core.(*HeaderChain).GetHeaderByHash in github.com/ethereum/go-ethereum/core/headerchain.go
0 5786566117 (flat, cum) 15.15% of Total
. . 404:func (hc *HeaderChain) GetHeaderByHash(hash common.Hash) *types.Header {
. 1471264309 405: number := hc.GetBlockNumber(hash)
. . 406: if number == nil {
. . 407: return nil
. . 408: }
. 4315301808 409: return hc.GetHeader(hash, *number)
. . 410:}
. . 411:
. . 412:// HasHeader checks if a block header is present in the database or not.
. . 413:// In theory, if header is present in the database, all relative components
. . 414:// like td and hash->number should be present too.
(pprof) list GetBlockNumber
Total: 38197204475
ROUTINE ======================== github.com/ethereum/go-ethereum/core.(*HeaderChain).GetBlockNumber in github.com/ethereum/go-ethereum/core/headerchain.go
94438817 1471264309 (flat, cum) 3.85% of Total
. . 100:func (hc *HeaderChain) GetBlockNumber(hash common.Hash) *uint64 {
94438817 94438817 101: if cached, ok := hc.numberCache.Get(hash); ok {
. . 102: return &cached
. . 103: }
. 1376270828 104: number := rawdb.ReadHeaderNumber(hc.chainDb, hash)
. . 105: if number != nil {
. 554664 106: hc.numberCache.Add(hash, *number)
. . 107: }
. . 108: return number
. . 109:}
. . 110:
. . 111:type headerWriteResult struct {
(pprof) list ReadHeaderNumber
Total: 38197204475
ROUTINE ======================== github.com/ethereum/go-ethereum/core/rawdb.ReadHeaderNumber in github.com/ethereum/go-ethereum/core/rawdb/accessors_chain.go
204606513 1376270828 (flat, cum) 3.60% of Total
. . 146:func ReadHeaderNumber(db ethdb.KeyValueReader, hash common.Hash) *uint64 {
109577863 1281242178 147: data, _ := db.Get(headerNumberKey(hash))
. . 148: if len(data) != 8 {
. . 149: return nil
. . 150: }
95028650 95028650 151: number := binary.BigEndian.Uint64(data)
. . 152: return &number
. . 153:}
. . 154:
. . 155:// WriteHeaderNumber stores the hash->number mapping.
. . 156:func WriteHeaderNumber(db ethdb.KeyValueWriter, hash common.Hash, number uint64) {
```
Opening this to discuss the idea, I know that rawdb.EmptyNumber is not a
great name for the variable, open to suggestions
Introduce file-based state journal in path database, fixing
the Pebble restriction when the journal size exceeds 4GB.
---------
Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR updates the outdated documentation URL from docs.gnosis.io to
the new official docs.safe.global domain. The change reflects the
rebranding from Gnosis Safe to Safe and ensures that users are directed
to the current API documentation for transaction service reference.
This is a follow up PR after #32128 , Seems I've missed to add
--txlookuplimit as hidden. In hte meanwhile, I also add the other
deprecated flags into the output of `show-deprecated-flags`
Downloading from a range was failing because it would return and error
early with an error misinterpreting "start-end".
---------
Co-authored-by: shantichanal <158101918+shantichanal@users.noreply.github.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Replaced the outdated and broken link to the Web3 Secret Storage
Definition with the current official URL from ethereum.org in the Clef
README. This ensures users have access to up-to-date and accurate
documentation for the keystore file format.
In this pull request, the original `CacheConfig` has been renamed to `BlockChainConfig`.
Over time, more fields have been added to `CacheConfig` to support
blockchain configuration. Such as `ChainHistoryMode`, which clearly extends
beyond just caching concerns.
Additionally, adding new parameters to the blockchain constructor has
become increasingly complicated, since it’s initialized across multiple
places in the codebase. A natural solution is to consolidate these arguments
into a dedicated configuration struct.
As a result, the existing `CacheConfig` has been redefined as `BlockChainConfig`.
Some parameters, such as `VmConfig`, `TxLookupLimit`, and `ChainOverrides`
have been moved into `BlockChainConfig`. Besides, a few fields in `BlockChainConfig`
were renamed, specifically:
- `TrieCleanNoPrefetch` -> `NoPrefetch`
- `TrieDirtyDisabled` -> `ArchiveMode`
Notably, this change won't affect the command line flags or the toml
configuration file. It's just an internal refactoring and fully backward-compatible.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
With EOF removed from the Osaka fork, and no longer being tested, the
implementation will now just be bitrotting. I'm opting to remove it so
it doesn't get in the way of other changes.
This pull request introduces a new test suite in workload framework, for
transaction tracing.
**test generation**
`go run . tracegen --trace-tests trace-test.json http://host:8545`
and you can choose to store the trace result in a specific folder
`go run . tracegen --trace-tests trace-test.json --trace-output
./trace-result http://host:8545`
**test run**
`./workload test -run Trace/Transaction --trace-invalid ./trace-invalid
http://host:8545`
The mismatched trace result will be saved in the specific folder for
further investigation.
This changes the era1 downloader to place the files into the correct
location where geth will actually use them. Also adds integration with
the new --datadir.era flag.
This implements a backing store for chain history based on era1 files.
The new store is integrated with the freezer. Queries for blocks and receipts
below the current freezer tail are handled by the era store.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
This PR modifies the disclaimer/banner that is printed when starting up
Geth in dev mode:
* if the client is spun up in ephemeral dev mode with a keystore
override, the address of the first (prefunded) account is printed.
* if the client is spun up in ephemeral mode without a keystore
override, the genesis allocation contains a single static prefunded EOA
account. It's address and private key are logged.
* the banner is printed at the end of client initialization to make it
more prominent. Previously, it was logged towards the beginning of
client initialization and subsequent logging from startup filled the
terminal, pushing it out of view of the user.
Other change is that we now use a static prefunded dev account instead
of generating a random one when instantiating a new dev mode chain.
This is an example of what the banner looks like:
```
WARN [05-28|23:05:16.475] You are running Geth in --dev mode. Please note the following:
WARN [05-28|23:05:16.475]
WARN [05-28|23:05:16.475] 1. This mode is only intended for fast, iterative development without assumptions on
WARN [05-28|23:05:16.475] security or persistence.
WARN [05-28|23:05:16.475] 2. The database is created in memory unless specified otherwise. Therefore, shutting down
WARN [05-28|23:05:16.475] your computer or losing power will wipe your entire block data and chain state for
WARN [05-28|23:05:16.475] your dev environment.
WARN [05-28|23:05:16.475] 3. A random, pre-allocated developer account will be available and unlocked as
WARN [05-28|23:05:16.475] eth.coinbase, which can be used for testing. The random dev account is temporary,
WARN [05-28|23:05:16.475] stored on a ramdisk, and will be lost if your machine is restarted.
WARN [05-28|23:05:16.475] 4. Mining is enabled by default. However, the client will only seal blocks if transactions
WARN [05-28|23:05:16.475] are pending in the mempool. The miner's minimum accepted gas price is 1.
WARN [05-28|23:05:16.475] 5. Networking is disabled; there is no listen-address, the maximum number of peers is set
WARN [05-28|23:05:16.475] to 0, and discovery is disabled.
WARN [05-28|23:05:16.475]
WARN [05-28|23:05:16.475]
WARN [05-28|23:05:16.475] Running in ephemeral mode. The following account has been prefunded in the genesis:
WARN [05-28|23:05:16.475]
WARN [05-28|23:05:16.475] Account
WARN [05-28|23:05:16.475] ------------------
WARN [05-28|23:05:16.475] 0x71562b71999873db5b286df957af199ec94617f7 (10^49 ETH)
WARN [05-28|23:05:16.475]
WARN [05-28|23:05:16.475] Private Key
WARN [05-28|23:05:16.475] ------------------
WARN [05-28|23:05:16.475] 0xb71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291
WARN [05-28|23:05:16.475]
```
closes#31796
---------
Co-authored-by: jwasinger <j-wasinger@hotmail.com>
I added a test for BlockRangeUpdate in #29158 but forgot to enable it.
Here I'm adding two more tests for it. Also applied a small refactoring
to combine calls to `dial()` and `peer()` into a single function, since
these two calls are duplicated in each test.
This PR implements eth/69. This protocol version drops the bloom filter
from receipts messages, reducing the amount of data needed for a sync
by ~530GB (2.3B txs * 256 byte) uncompressed. Compressed this will
be reduced to ~100GB
The new version also changes the Status message and introduces the
BlockRangeUpdate message to relay information about the available history
range.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This adds a geth subcommand for downloading era1 files and placing them into
the correct location. The tool can be used even while geth is already running
on the datadir. Downloads are checked against a hard-coded list of checksums
for mainnet and sepolia.
```
./geth download-era --server $SERVER --block 333333
./geth download-era --server $SERVER --block 333333-444444
./geth download-era --server $SERVER --epoch 0-10
./geth download-era --server $SERVER --all
```
The implementation reuses the file downloader we already had for
fetching build tools. I've done some refactoring on it to make sure it
can support the new use case, and there are some changes to the build
here as well.
Add tests for GetBlockHeaders that verify client does not disconnect when unlikely block numbers are requested, e.g. max uint64.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
For PeerDAS, we need to compute cell proofs. Both ckzg and gokzg support
computing these cell proofs.
This PR does the following:
- Update the go-kzg library from "github.com/crate-crypto/go-kzg-4844"
to "github.com/crate-crypto/go-eth-kzg" which will be the new upstream
for go-kzg moving forward
- Update ckzg from v1.0.0 to v2.0.1 and switch to /v2
- Updates the trusted setup to contain the g1 points both in lagrange
and monomial form
- Expose `ComputeCells` to compute the cell proofs
