Commit graph

2305 commits

Author SHA1 Message Date
Marius van der Wijden
4639725b10 core/rawdb: allocate database keys with explicit size to avoid slice growth (#27772) 2023-11-10 11:09:46 -06:00
Martin Holst Swende
0aa81738e6 core/rawdb, ethdb/pebble: avoid fsync db in tests (#27836)
Adds an option to disable fsync for database operations.
This is to make tests faster.
2023-11-10 11:09:46 -06:00
Péter Szilágyi
22b083a163 cmd/utils: fix a startup issue on deleted chaindata but dangling ancients (#27989) 2023-11-10 11:09:46 -06:00
Péter Szilágyi
2aa9c9c87b core, eth, trie: expose more detailed dirty ram tracking for diff layers (#27971) 2023-11-10 11:09:46 -06:00
Felix Lange
b88138ad05 core: add BeaconRoot to header in GenerateChain (#27974)
Fixes an error in tests of internal/ethapi.
2023-11-10 11:09:46 -06:00
Martin Holst Swende
2e9477f46b core, params, beacon/engine: implement EIP 4788 BeaconRoot (#27849)
This change implements "EIP 4788 : Beacon block root in the EVM". It implements version-2 of EPI-4788, main difference being that the contract is an actual contract rather than a precompile, as in #27289.
2023-11-10 11:09:46 -06:00
Delweng
01eb95af0f core: ensure txindex will be triggered at least once (#27847)
Currently, we trigger the logic to (un)index transactions when the node receives a new
block. However, in some cases the node may not receive new blocks (eg, when the Geth node
is configured without peer discovery, or when it acts as an RPC node for historical-only
data).

In these situations, the Geth node user may not have previously configured txlookuplimit
(i.e. the default of around one year), but later realizes they need to index all
historical blocks. However, adding txlookuplimit=0 and restarting geth has no effect. This
change makes it check for required indexing work once, on startup, to fix the issue.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-11-10 11:09:46 -06:00
Delweng
4f92d3f516 internal/ethapi: add testcases for blobTx (#27818)
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-11-10 11:09:46 -06:00
Martin Holst Swende
10133ba062 core/txpool/legacypool: protect cache with mutex (#27898)
This change fixes the a potential race by using mutexes when the m.cache is read or modified.
2023-11-10 11:09:46 -06:00
lightclient
db5684444a core/forkid: correctly compute forkid when timestamp fork is activated in genesis (#27895)
This changes the forkID calculation to ignore time-based forks that occurred before the
genesis block. It's supposed to be done this way because the spec says:

> If a chain is configured to start with a non-Frontier ruleset already in its genesis, that is NOT considered a fork.
2023-11-10 11:09:46 -06:00
lightclient
93dbe130dd core/types: fix unmarshalling of BlobTx values (#27939)
FromBig returns true *when overflow occurs*
2023-11-10 11:09:46 -06:00
Felix Lange
278a4af91f core/types: support for optional blob sidecar in BlobTx (#27841)
This PR removes the newly added txpool.Transaction wrapper type, and instead adds a way
of keeping the blob sidecar within types.Transaction. It's better this way because most
code in go-ethereum does not care about blob transactions, and probably never will. This
will start mattering especially on the client side of RPC, where all APIs are based on
types.Transaction. Users need to be able to use the same signing flows they already
have.

However, since blobs are only allowed in some places but not others, we will now need to
add checks to avoid creating invalid blocks. I'm still trying to figure out the best place
to do some of these. The way I have it currently is as follows:

- In block validation (import), txs are verified not to have a blob sidecar.
- In miner, we strip off the sidecar when committing the transaction into the block.
- In TxPool validation, txs must have a sidecar to be added into the blobpool.
  - Note there is a special case here: when transactions are re-added because of a chain
    reorg, we cannot use the transactions gathered from the old chain blocks as-is,
    because they will be missing their blobs. This was previously handled by storing the
    blobs into the 'blobpool limbo'. The code has now changed to store the full
    transaction in the limbo instead, but it might be confusing for code readers why we're
    not simply adding the types.Transaction we already have.

Code changes summary:

- txpool.Transaction removed and all uses replaced by types.Transaction again
- blobpool now stores types.Transaction instead of defining its own blobTx format for storage
- the blobpool limbo now stores types.Transaction instead of storing only the blobs
- checks to validate the presence/absence of the blob sidecar added in certain critical places
2023-11-10 11:09:46 -06:00
Péter Szilágyi
1c411fcea2 all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-11-10 11:09:46 -06:00
DongXi Huang
dc448e7e2c core/types: remove duplication in eip2930 signer (#27860)
Remove duplication in signer
---------

Co-authored-by: GDdark <huangdongxi@minijoy.work>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-11-10 11:09:46 -06:00
imulmat4
e9a6ee6887 core/txpool/blobpool: fix metrics name for prometheus export (#27901) 2023-11-10 11:09:46 -06:00
rjl493456442
1053526246 all: activate pbss as experimental feature (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-11-10 11:09:46 -06:00
Martin Holst Swende
489b7c8265 eth/filters, core/rawdb: remove unused param, refactor filtering-loop (#27891)
This change removes a chainconfig parameter passed into rawdb.ReadLogs, which is not used nor needed.
It also modifies the filter loop slightly, avoiding a labeled break and instead using a method.

This change does not modify any behaviour.
2023-11-10 11:09:46 -06:00
Guillaume Ballet
075fd1c216 core/state: move UpdateContractCode before the trie hash is computed (#27853)
Context: The UpdateContractCode method was introduced for the state storage commitment
schemes that include the whole code for their commitment computation. It must therefore be called
before the root hash is computed at the end of IntermediateRoot.

This should have no impact on the MPT since, in this context, the method is a no-op.
2023-11-10 11:09:46 -06:00
Martin Holst Swende
0e17c0fd48 core/vm: update 4844 - point evaluation precompile address (#27874) 2023-11-10 11:09:46 -06:00
Felix Lange
932ffcbbf8 core/types: support yParity field in JSON transactions (#27744)
This adds support for the "yParity" field in transaction objects returned by RPC
APIs. We somehow forgot to add this field even though it has been in the spec for
a long time.
2023-11-10 11:09:46 -06:00
Darioush Jalali
b1d0e1e9d5 all: remove trailing whitespace (#27741) 2023-11-10 11:09:46 -06:00
Felix Lange
d8807c42b3 core/types: fix immutability guarantees in Block (#27844)
This change rearranges the accessor methods in block.go and fixes some minor issues with
the copy-on-write logic of block data. Fixed issues:

- Block.WithWithdrawals did not create a shallow copy of the block.

- Block.WithBody copied the header unnecessarily, and did not preserve withdrawals.

However, the bugs did not affect any code in go-ethereum because blocks are *always*
created using NewBlockWithHeader().WithBody().WithWithdrawals()
2023-11-10 11:09:46 -06:00
rjl493456442
b7ea2f2445 all: implement path-based state scheme (#25963)
* all: implement path-based state scheme

* all: edits from review

* core/rawdb, trie/triedb/pathdb: review changes

* core, light, trie, eth, tests: reimplement pbss history

* core, trie/triedb/pathdb: track block number in state history

* trie/triedb/pathdb: add history documentation

* core, trie/triedb/pathdb: address comments from Peter's review

Important changes to list:

- Cache trie nodes by path in clean cache
- Remove root->id mappings when history is truncated

* trie/triedb/pathdb: fallback to disk if unexpect node in clean cache

* core/rawdb: fix tests

* trie/triedb/pathdb: rename metrics, change clean cache key

* trie/triedb: manage the clean cache inside of disk layer

* trie/triedb/pathdb: move journal function

* trie/triedb/path: fix tests

* trie/triedb/pathdb: fix journal

* trie/triedb/pathdb: fix history

* trie/triedb/pathdb: try to fix tests on windows

* core, trie: address comments

* trie/triedb/pathdb: fix test issues

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-11-10 11:09:46 -06:00
Delweng
b2f7349a6b consensus/misc: move eip1559 into a package (#27828)
* consensus/misc: move eip1559 as a sub directory

Signed-off-by: jsvisa <delweng@gmail.com>

* consensus/misc: package name

Signed-off-by: jsvisa <delweng@gmail.com>

* all: eip1559

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-11-10 11:09:46 -06:00
rjl493456442
96cb2894b4 core/txpool/blobpool: remove error log when finalized block is nil (#27822)
* core/txpool/blobpool: remove error log when finalized block is nil

* core/txpool/blobpool: take peter's suggestion
2023-11-10 11:09:46 -06:00
Delweng
857b4f45c0 all: replace data gas to blob gas in comments (#27825)
* eth: excessDataGas -> excessBlobGas

* consensus: data gas -> blob gas

* core: data gas -> blob gas

* params: data gas -> blob gas
2023-11-10 11:09:46 -06:00
rjl493456442
426aec583b core, trie: track state change set with account address (#27815) 2023-11-10 11:09:46 -06:00
Delweng
cd2e754633 core: remove pointless check for excessBlobGas nilness (#27797)
* core: check excessBlobGas in front

Signed-off-by: jsvisa <delweng@gmail.com>

* core: no need to manual panic

Signed-off-by: jsvisa <delweng@gmail.com>

* core: no comment

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-11-10 11:09:46 -06:00
Mario Vega
ac8154a248 core/types: fix receipt blob fields marshaling (#27793) 2023-11-10 11:09:46 -06:00
Marius van der Wijden
23f7874257 core/types: put header fields in correct order (#27791) 2023-11-10 11:09:46 -06:00
Péter Szilágyi
50f1fb085b all: rename dataGas to blobGas (#27789) 2023-11-10 11:09:46 -06:00
Péter Szilágyi
c3b5e6ad26 core/txpool/blobpool: fix a merge conflict from a package rename (#27790) 2023-11-10 11:09:46 -06:00
Felix Lange
4121121152 core/types: add 4844 data gas fields in Receipt (#27743)
* core/types: add data gas fields in Receipt

* core/types: use BlobGas method of tx

* core: fix test

* core/types: fix receipt tests, add data gas used field test

---------

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-11-10 11:09:46 -06:00
Péter Szilágyi
16fd9c786e core/txpool/blobpool: 4844 blob transaction pool (#26940)
* core/blobpool: implement txpool for blob txs

* core/txpool: track address reservations to notice any weird bugs

* core/txpool/blobpool: add support for in-memory operation for tests

* core/txpool/blobpool: fix heap updating after SetGasTip if account is evicted

* core/txpool/blobpool: fix eviction order if cheap leading txs are included

* core/txpool/blobpool: add note as to why the eviction fields are not inited in reinject

* go.mod: pull in inmem billy form upstream

* core/txpool/blobpool: fix review commens

* core/txpool/blobpool: make heap and heap test deterministic

* core/txpool/blobpool: luv u linter

* core/txpool: limit blob transactions to 16 per account

* core/txpool/blobpool: fix rebase errors

* core/txpool/blobpool: luv you linter

* go.mod: revert some strange crypto package dep updates
2023-11-10 11:09:46 -06:00
Darioush Jalali
0c56bf33c0 core/types: deepcopy ExcessDataGas, DataGasUsed (#27767) 2023-11-10 11:09:46 -06:00
rjl493456442
108b77bca6 all: expose block number information to statedb (#27753)
* core/state: clean up

* all: add block number infomration to statedb

* core, trie: rename blockNumber to block
2023-11-10 11:09:46 -06:00
jwasinger
876a66cdfd core/state, core/vm: implement EIP 6780 (#27189)
EIP-6780: SELFDESTRUCT only in same transaction

>     SELFDESTRUCT will recover all funds to the caller but not delete the account, except when called in the same transaction as creation

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-11-10 11:09:46 -06:00
Martin Holst Swende
1481df630b core, tests: EIP-4844 transaction processing logic (#27721)
This updates the reference tests to the latest version and also adds logic
to process EIP-4844 blob transactions into the state transition. We are now
passing most Cancun fork tests.

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-11-10 11:09:46 -06:00
ucwong
31689cb6ad core/vm: use types.EmptyCodeHash (#27729) 2023-11-10 11:09:46 -06:00
jwasinger
50cb341607 core: replace instances of 'suicide' with 'selfdestruct' to improve code consistency. (#27716)
---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
2023-11-10 11:09:46 -06:00
ucwong
ddc0a3aedc core/blockchain: fast to snap in comments (#27722) 2023-11-10 11:09:46 -06:00
rjl493456442
2fabfa07cc core/rawdb: support freezer batch read with no size limit (#27687)
This change adds the ability to perform reads from freezer without size limitation. This can be useful in cases where callers are certain that out-of-memory will not happen (e.g. reading only a few elements). 

The previous API was designed to behave both optimally and secure while servicing a request from a peer, whereas this change should _not_ be used when an untrusted peer can influence the query size.
2023-11-10 11:09:46 -06:00
rjl493456442
5b3e45ba53 core/state: value diff tracking in StateDB (#27349)
This change makes the StateDB track the state key value diff of a block transition.
We already tracked current account and storage values for the purpose of updating
the state snapshot. With this PR, we now also track the original (pre-transition) values
of accounts and storage slots.
2023-11-10 11:09:46 -06:00
Delweng
9a984bc54e internal/blocktest: add package for shared test code (#27270) 2023-11-10 11:09:46 -06:00
Charles Cooper
b0a5097cea core/vm: implement EIP-5656, mcopy instruction (#26181)
Implements [EIP 5656](https://eips.ethereum.org/EIPS/eip-5656), MCOPY instruction, and enables it for Cancun. 

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-11-10 11:09:46 -06:00
Martin Holst Swende
c8f00777a6 core/vm: define cancun + enable 1153 (tstore/tload) in cancun (#27663) 2023-11-10 11:09:46 -06:00
Martin Holst Swende
373dfc6bcf core: remove outdated tests (#27662)
Back before #27178 , we spun up a number of ethash verifiers to verify headers. So we also had tests to ensure that we were indeed able to abort verification even if we had multiple workers running.

With PR #27178, we removed the parallelism in verification, and these tests are now failing, since we now just sequentially fire away the results as fast as possible on one routine.

This change removes the (sometimes failing) tests
2023-11-10 11:09:46 -06:00
jwasinger
7818ec3799 cmd/geth: implement dev mode for post-merge (#27327)
This change adds back the 'geth --dev' mode of operation, using a cl-mocker. 

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
2023-11-10 11:09:46 -06:00
Darioush Jalali
625e847b5b core: stop chain when tests are finished (#27660)
core (blockchain_test): add chain.Stop() to tests
2023-11-10 11:09:46 -06:00
rjl493456442
2c0c010df5 cmd, core, eth, graphql, trie: no persisted clean trie cache file (#27525)
The clean trie cache is persisted periodically, therefore Geth can
quickly warmup the cache in next restart.

However it will reduce the robustness of system. The assumption is
held in Geth that if the parent trie node is present, then the entire
sub-trie associated with the parent are all prensent.

Imagine the scenario that Geth rewinds itself to a past block and
restart, but Geth finds the root node of "future state" in clean
cache then regard this state is present in disk, while is not in fact.

Another example is offline pruning tool. Whenever an offline pruning
is performed, the clean cache file has to be removed to aviod hitting
the root node of "deleted states" in clean cache.

All in all, compare with the minor performance gain, system robustness
is something we care more.
2023-11-10 11:09:46 -06:00