forks/go-ethereum
1
0
Fork 1
mirror of https://github.com/ethereum/go-ethereum.git synced 2026-08-18 01:43:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

resolve conflict

Browse source
This commit is contained in:
Sina Mahmoodi 2025-02-04 18:21:37 +01:00
commit bca2e2c8b0
287 changed files with 5982 additions and 5088 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

24
.github/workflows/go.yml vendored
View file

@ -8,6 +8,30 @@ on:
workflow_dispatch:
jobs:
lint:
name: Lint
runs-on: self-hosted
steps:
- uses: actions/checkout@v4
# Cache build tools to avoid downloading them each time
- uses: actions/cache@v4
with:
path: build/cache
key: ${{ runner.os }}-build-tools-cache-${{ hashFiles('build/checksums.txt') }}
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: 1.23.0
cache: false
- name: Run linters
run: |
go run build/ci.go lint
go run build/ci.go check_tidy
go run build/ci.go check_baddeps
build:
runs-on: self-hosted
steps:

6
.golangci.yml
View file

@ -3,9 +3,6 @@
run:
timeout: 20m
tests: true
# default is true. Enables skipping of directories:
# vendor$, third_party$, testdata$, examples$, Godeps$, builtin$
skip-dirs-use-default: true
linters:
disable-all: true
@ -54,6 +51,9 @@ linters-settings:
exclude: [""]
issues:
# default is true. Enables skipping of directories:
# vendor$, third_party$, testdata$, examples$, Godeps$, builtin$
exclude-dirs-use-default: true
exclude-files:
- core/genesis_alloc.go
exclude-rules:

2
.travis.yml
View file

@ -25,7 +25,7 @@ jobs:
before_install:
- export DOCKER_CLI_EXPERIMENTAL=enabled
script:
- go run build/ci.go dockerx -platform "linux/amd64,linux/arm64,linux/riscv64" -upload ethereum/client-go
- go run build/ci.go dockerx -platform "linux/amd64,linux/arm64,linux/riscv64" -hub ethereum/client-go -upload
# This builder does the Linux Azure uploads
- stage: build

127
README.md
View file

@ -54,14 +54,14 @@ on how you can run your own `geth` instance.
Minimum:
* CPU with 2+ cores
* 4GB RAM
* CPU with 4+ cores
* 8GB RAM
* 1TB free storage space to sync the Mainnet
* 8 MBit/sec download Internet service
Recommended:
* Fast CPU with 4+ cores
* Fast CPU with 8+ cores
* 16GB+ RAM
* High-performance SSD with at least 1TB of free space
* 25+ MBit/sec download Internet service
@ -138,8 +138,6 @@ export your existing configuration:
$ geth --your-favourite-flags dumpconfig
```
*Note: This works only with `geth` v1.6.0 and above.*
#### Docker quick start
One of the quickest ways to get Ethereum up and running on your machine is by using
@ -180,14 +178,13 @@ HTTP based JSON-RPC API options:
* `--http.addr` HTTP-RPC server listening interface (default: `localhost`)
* `--http.port` HTTP-RPC server listening port (default: `8545`)
* `--http.api` API's offered over the HTTP-RPC interface (default: `eth,net,web3`)
* `--http.corsdomain` Comma separated list of domains from which to accept cross origin requests (browser enforced)
* `--http.corsdomain` Comma separated list of domains from which to accept cross-origin requests (browser enforced)
* `--ws` Enable the WS-RPC server
* `--ws.addr` WS-RPC server listening interface (default: `localhost`)
* `--ws.port` WS-RPC server listening port (default: `8546`)
* `--ws.api` API's offered over the WS-RPC interface (default: `eth,net,web3`)
* `--ws.origins` Origins from which to accept WebSocket requests
* `--ipcdisable` Disable the IPC-RPC server
* `--ipcapi` API's offered over the IPC-RPC interface (default: `admin,debug,eth,miner,net,personal,txpool,web3`)
* `--ipcpath` Filename for IPC socket/pipe within the datadir (explicit paths escape it)
You'll need to use your own programming environments' capabilities (libraries, tools, etc) to
@ -206,118 +203,14 @@ APIs!**
Maintaining your own private network is more involved as a lot of configurations taken for
granted in the official networks need to be manually set up.
#### Defining the private genesis state
Unfortunately since [the Merge](https://ethereum.org/en/roadmap/merge/) it is no longer possible
to easily set up a network of geth nodes without also setting up a corresponding beacon chain.
First, you'll need to create the genesis state of your networks, which all nodes need to be
aware of and agree upon. This consists of a small JSON file (e.g. call it `genesis.json`):
There are three different solutions depending on your use case:
```json
{
"config": {
"chainId": <arbitrary positive integer>,
"homesteadBlock": 0,
"eip150Block": 0,
"eip155Block": 0,
"eip158Block": 0,
"byzantiumBlock": 0,
"constantinopleBlock": 0,
"petersburgBlock": 0,
"istanbulBlock": 0,
"berlinBlock": 0,
"londonBlock": 0
},
"alloc": {},
"coinbase": "0x0000000000000000000000000000000000000000",
"difficulty": "0x20000",
"extraData": "",
"gasLimit": "0x2fefd8",
"nonce": "0x0000000000000042",
"mixhash": "0x0000000000000000000000000000000000000000000000000000000000000000",
"parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
"timestamp": "0x00"
}
```
The above fields should be fine for most purposes, although we'd recommend changing
the `nonce` to some random value so you prevent unknown remote nodes from being able
to connect to you. If you'd like to pre-fund some accounts for easier testing, create
the accounts and populate the `alloc` field with their addresses.
```json
"alloc": {
"0x0000000000000000000000000000000000000001": {
"balance": "111111111"
},
"0x0000000000000000000000000000000000000002": {
"balance": "222222222"
}
}
```
With the genesis state defined in the above JSON file, you'll need to initialize **every**
`geth` node with it prior to starting it up to ensure all blockchain parameters are correctly
set:
```shell
$ geth init path/to/genesis.json
```
#### Creating the rendezvous point
With all nodes that you want to run initialized to the desired genesis state, you'll need to
start a bootstrap node that others can use to find each other in your network and/or over
the internet. The clean way is to configure and run a dedicated bootnode:
```shell
# Use the devp2p tool to create a node file.
# The devp2p tool is also part of the 'alltools' distribution bundle.
$ devp2p key generate node1.key
# file node1.key is created.
$ devp2p key to-enr -ip 10.2.3.4 -udp 30303 -tcp 30303 node1.key
# Prints the ENR for use in --bootnode flag of other nodes.
# Note this method requires knowing the IP/ports ahead of time.
$ geth --nodekey=node1.key
```
With the bootnode online, it will display an [`enode` URL](https://ethereum.org/en/developers/docs/networking-layer/network-addresses/#enode)
that other nodes can use to connect to it and exchange peer information. Make sure to
replace the displayed IP address information (most probably `[::]`) with your externally
accessible IP to get the actual `enode` URL.
*Note: You could previously use the `bootnode` utility to start a stripped down version of geth. This is not possible anymore.*
#### Starting up your member nodes
With the bootnode operational and externally reachable (you can try
`telnet <ip> <port>` to ensure it's indeed reachable), start every subsequent `geth`
node pointed to the bootnode for peer discovery via the `--bootnodes` flag. It will
probably also be desirable to keep the data directory of your private network separated, so
do also specify a custom `--datadir` flag.
```shell
$ geth --datadir=path/to/custom/data/folder --bootnodes=<bootnode-enode-url-from-above>
```
*Note: Since your network will be completely cut off from the main and test networks, you'll
also need to configure a miner to process transactions and create new blocks for you.*
#### Running a private miner
In a private network setting a single CPU miner instance is more than enough for
practical purposes as it can produce a stable stream of blocks at the correct intervals
without needing heavy resources (consider running on a single thread, no need for multiple
ones either). To start a `geth` instance for mining, run it with all your usual flags, extended
by:
```shell
$ geth <usual-flags> --mine --miner.threads=1 --miner.etherbase=0x0000000000000000000000000000000000000000
```
Which will start mining blocks and transactions on a single CPU thread, crediting all
proceedings to the account specified by `--miner.etherbase`. You can further tune the mining
by changing the default gas limit blocks converge to (`--miner.targetgaslimit`) and the price
transactions are accepted at (`--miner.gasprice`).
* If you are looking for a simple way to test smart contracts from go in your CI, you can use the [Simulated Backend](https://geth.ethereum.org/docs/developers/dapp-developer/native-bindings#blockchain-simulator).
* If you want a convenient single node environment for testing, you can use our [Dev Mode](https://geth.ethereum.org/docs/developers/dapp-developer/dev-mode).
* If you are looking for a multiple node test network, you can set one up quite easily with [Kurtosis](https://geth.ethereum.org/docs/fundamentals/kurtosis).
## Contribution

18
accounts/abi/bind/util.go
View file

@ -30,12 +30,18 @@ import (
// WaitMined waits for tx to be mined on the blockchain.
// It stops waiting when the context is canceled.
func WaitMined(ctx context.Context, b DeployBackend, tx *types.Transaction) (*types.Receipt, error) {
return WaitMinedHash(ctx, b, tx.Hash())
}
// WaitMinedHash waits for a transaction with the provided hash to be mined on the blockchain.
// It stops waiting when the context is canceled.
func WaitMinedHash(ctx context.Context, b DeployBackend, hash common.Hash) (*types.Receipt, error) {
queryTicker := time.NewTicker(time.Second)
defer queryTicker.Stop()
logger := log.New("hash", tx.Hash())
logger := log.New("hash", hash)
for {
receipt, err := b.TransactionReceipt(ctx, tx.Hash())
receipt, err := b.TransactionReceipt(ctx, hash)
if err == nil {
return receipt, nil
}
@ -61,7 +67,13 @@ func WaitDeployed(ctx context.Context, b DeployBackend, tx *types.Transaction) (
if tx.To() != nil {
return common.Address{}, errors.New("tx is not contract creation")
}
receipt, err := WaitMined(ctx, b, tx)
return WaitDeployedHash(ctx, b, tx.Hash())
}
// WaitDeployedHash waits for a contract deployment transaction with the provided hash and returns the on-chain
// contract address when it is mined. It stops waiting when ctx is canceled.
func WaitDeployedHash(ctx context.Context, b DeployBackend, hash common.Hash) (common.Address, error) {
receipt, err := WaitMinedHash(ctx, b, hash)
if err != nil {
return common.Address{}, err
}

4
accounts/accounts.go
View file

@ -214,7 +214,9 @@ const (
// of starting any background processes such as automatic key derivation.
WalletOpened
// WalletDropped
// WalletDropped is fired when a wallet is removed or disconnected, either via USB
// or due to a filesystem event in the keystore. This event indicates that the wallet
// is no longer available for operations.
WalletDropped
)

25
accounts/usbwallet/hub.go
View file

@ -73,7 +73,7 @@ func NewLedgerHub() (*Hub, error) {
return newHub(LedgerScheme, 0x2c97, []uint16{
// Device definitions taken from
// https://github.com/LedgerHQ/ledger-live/blob/38012bc8899e0f07149ea9cfe7e64b2c146bc92b/libs/ledgerjs/packages/devices/src/index.ts
// https://github.com/LedgerHQ/ledger-live/blob/595cb73b7e6622dbbcfc11867082ddc886f1bf01/libs/ledgerjs/packages/devices/src/index.ts
// Original product IDs
0x0000, /* Ledger Blue */
@ -81,18 +81,14 @@ func NewLedgerHub() (*Hub, error) {
0x0004, /* Ledger Nano X */
0x0005, /* Ledger Nano S Plus */
0x0006, /* Ledger Nano FTS */
0x0007, /* Ledger Flex */
0x0015, /* HID + U2F + WebUSB Ledger Blue */
0x1015, /* HID + U2F + WebUSB Ledger Nano S */
0x4015, /* HID + U2F + WebUSB Ledger Nano X */
0x5015, /* HID + U2F + WebUSB Ledger Nano S Plus */
0x6015, /* HID + U2F + WebUSB Ledger Nano FTS */
0x0011, /* HID + WebUSB Ledger Blue */
0x1011, /* HID + WebUSB Ledger Nano S */
0x4011, /* HID + WebUSB Ledger Nano X */
0x5011, /* HID + WebUSB Ledger Nano S Plus */
0x6011, /* HID + WebUSB Ledger Nano FTS */
0x0000, /* WebUSB Ledger Blue */
0x1000, /* WebUSB Ledger Nano S */
0x4000, /* WebUSB Ledger Nano X */
0x5000, /* WebUSB Ledger Nano S Plus */
0x6000, /* WebUSB Ledger Nano FTS */
0x7000, /* WebUSB Ledger Flex */
}, 0xffa0, 0, newLedgerDriver)
}
@ -185,8 +181,11 @@ func (hub *Hub) refreshWallets() {
for _, info := range infos {
for _, id := range hub.productIDs {
// We check both the raw ProductID (legacy) and just the upper byte, as Ledger
// uses `MMII`, encoding a model (MM) and an interface bitfield (II)
mmOnly := info.ProductID & 0xff00
// Windows and Macos use UsageID matching, Linux uses Interface matching
if info.ProductID == id && (info.UsagePage == hub.usageID || info.Interface == hub.endpointID) {
if (info.ProductID == id || mmOnly == id) && (info.UsagePage == hub.usageID || info.Interface == hub.endpointID) {
devices = append(devices, info)
break
}

158
build/checksums.txt
View file

@ -5,88 +5,88 @@
# https://github.com/ethereum/execution-spec-tests/releases/download/v2.1.0/
ca89c76851b0900bfcc3cbb9a26cbece1f3d7c64a3bed38723e914713290df6c fixtures_develop.tar.gz
# version:golang 1.23.4
# version:golang 1.23.5
# https://go.dev/dl/
ad345ac421e90814293a9699cca19dd5238251c3f687980bbcae28495b263531 go1.23.4.src.tar.gz
459a09504f7ebf2cbcee6ac282c8f34f97651217b1feae64557dcdd392b9bb62 go1.23.4.aix-ppc64.tar.gz
0f4e569b2d38cb75cb2efcaf56beb42778ab5e46d89318fef39060fe36d7b9b7 go1.23.4.darwin-amd64.pkg
6700067389a53a1607d30aa8d6e01d198230397029faa0b109e89bc871ab5a0e go1.23.4.darwin-amd64.tar.gz
19c054eaf40c5fac65b027f7443c01382e493d3c8c42cf8b2504832ebddce037 go1.23.4.darwin-arm64.pkg
87d2bb0ad4fe24d2a0685a55df321e0efe4296419a9b3de03369dbe60b8acd3a go1.23.4.darwin-arm64.tar.gz
5e73dc89b44626677ec9d9aa4257d6d2ef1245502bc36a99385284910d0ade0a go1.23.4.dragonfly-amd64.tar.gz
8df26b1e71234756c1f0e82cfffba3f427c5a91a251844ada2c7694a6986c546 go1.23.4.freebsd-386.tar.gz
7de078d94d2af50ee9506ef7df85e4d12d4018b23e0b2cbcbc61d686f549b41a go1.23.4.freebsd-amd64.tar.gz
3f23e0a01cfe24e4160124cd7ab02bdd188264652074abdbce401c93f41e58a4 go1.23.4.freebsd-arm.tar.gz
986a20e7c94431f03b44b3c415abc698c7b4edc2ae8431f7ecae1c2429d4cfa2 go1.23.4.freebsd-arm64.tar.gz
25e39f005f977778ce963fc43089510fe7514f3cfc0358eab584de4ce9f181fb go1.23.4.freebsd-riscv64.tar.gz
7e1d52f93da68c3bab39e3d83f89944d7d151208e54fdc30b0eda2a3812661d7 go1.23.4.illumos-amd64.tar.gz
4a4a0e7587ef8c8a326439b957027f2791795e2d29d4ae3885b4091a48f843bc go1.23.4.linux-386.tar.gz
6924efde5de86fe277676e929dc9917d466efa02fb934197bc2eba35d5680971 go1.23.4.linux-amd64.tar.gz
16e5017863a7f6071363782b1b8042eb12c6ca4f4cd71528b2123f0a1275b13e go1.23.4.linux-arm64.tar.gz
1f1dda0dc7ce0b2295f57258ec5ef0803fd31b9ed0aa20e2e9222334e5755de1 go1.23.4.linux-armv6l.tar.gz
4f469179a335a1a7bb9f991ad0c567f3d3eeb9b412ecd192206ab5c3e1a52b5a go1.23.4.linux-loong64.tar.gz
86b68185bcc43ea07190e95137c3442f062acc7ae10c3f1cf900fbe23e07df24 go1.23.4.linux-mips.tar.gz
3a19245eec76533b3d01c90f3a40a38d63684028f0fd54d442dc9a9d03197891 go1.23.4.linux-mips64.tar.gz
b53a06fc8455f6a875329e8d2e24d39db298122c9cce6e948117022191f6c613 go1.23.4.linux-mips64le.tar.gz
66120a8105b8ba6559f4e6a13b1e39b433fb8032df9d1744e4486876fa1723ce go1.23.4.linux-mipsle.tar.gz
33be2bfb27f2821a65e9f6aba744c85ea7c5e233e16bac27bb3ec253bcd4e970 go1.23.4.linux-ppc64.tar.gz
65a303ef51e48ff77e004a6a5b4db6ce59495cd59c6af51b54bf4f786c01a1b9 go1.23.4.linux-ppc64le.tar.gz
7c40e9e0d722cef14ede765159ba297f4c6e3093bb106f10fbccf8564780049a go1.23.4.linux-riscv64.tar.gz
74aab82bf4eca7c26c830a5b0e2a31d193a4d5ba47045526b92473cc7188d7d7 go1.23.4.linux-s390x.tar.gz
dba009d8bf9928cb5a1e31fcbe0eb41335cce4fe63755d95cef6b5987df4ed5a go1.23.4.netbsd-386.tar.gz
54b081cc36355aa5ecb6db9544cf7e77366a7b08ce96cb98a45d043e393660c7 go1.23.4.netbsd-amd64.tar.gz
f05fec348c7c9f07e1ad4e436db4122e98de99ebcfbf6ac6176869785f334a02 go1.23.4.netbsd-arm.tar.gz
317878da2bface5a57a8eaf5c1fe2b40b1c82d8172a10453ad3eea36f6946bdb go1.23.4.netbsd-arm64.tar.gz
0d84350dfd72c505c6ad474e51676b04e95ffb748c614bd5bf8510026873059b go1.23.4.openbsd-386.tar.gz
cc62f5a14ea3d573d8edbce1833f70a8f99ca048a9db0fcc9e738fd48e950505 go1.23.4.openbsd-amd64.tar.gz
326aba6cf5bb9348fa3e41217abd2c84eac92608684e2fe8c5474fdab23a0db9 go1.23.4.openbsd-arm.tar.gz
51ea2a2588bf3da8e1476f3e2bd4d6724d74126e99f9c6ea9af4ebe389e64de6 go1.23.4.openbsd-arm64.tar.gz
44c5c82ab23e40225b2ba1e7d19150a5973ea58e93b4931e426e6e6f0d108872 go1.23.4.openbsd-ppc64.tar.gz
5fa31fc13d1e3c123a5e96ba38683fa2c947baed23ac9c7c341afcfe007c8993 go1.23.4.openbsd-riscv64.tar.gz
e5952fc93eeaa0094ef09a0e72a9f06f0621ce841a39f9637fb5b9062e77d67a go1.23.4.plan9-386.tar.gz
fb2a9ee3ae5a77e734862e257a9395b43e707ac45e060dfa84c5a40688e73170 go1.23.4.plan9-amd64.tar.gz
e1b95563b19fdebd6ea0d20c07641e69580976fa754e586c831ad7a3ae987140 go1.23.4.plan9-arm.tar.gz
088c282509fc9e1a8f29fc0dd16fe486854d05b8ceba08d077d17d11d6979a41 go1.23.4.solaris-amd64.tar.gz
e5865c1bfc3fee5d003819b2e2c800f598fe9994931bac63f573e8d05a10d91f go1.23.4.windows-386.msi
e544e0e356147ba998e267002bd0f2c4bf3370d495467a55baf2c63595a2026d go1.23.4.windows-386.zip
5f8cc5991eb8f4f96b6c611d839453cd11c9a2c3f23672a4188342c97ee159fa go1.23.4.windows-amd64.msi
16c59ac9196b63afb872ce9b47f945b9821a3e1542ec125f16f6085a1c0f3c39 go1.23.4.windows-amd64.zip
fc77c0531406d092c5356167e45c05a22d16bea84e3fa555e0f03af085c11763 go1.23.4.windows-arm.msi
1012cfd8ca7241c2beecb5c345dd61f01897c6f6baca80ea1bfed357035c868a go1.23.4.windows-arm.zip
8347c1aa4e1e67954d12830f88dbe44bd7ac0ec134bb472783dbfb5a3a8865d0 go1.23.4.windows-arm64.msi
db69cae5006753c785345c3215ad941f8b6224e2f81fec471c42d6857bee0e6f go1.23.4.windows-arm64.zip
a6f3f4bbd3e6bdd626f79b668f212fbb5649daf75084fb79b678a0ae4d97423b go1.23.5.src.tar.gz
8d8bc7d1b362dd91426da9352741db298ff73e3e0a3ccbe6f607f80ba17647a4 go1.23.5.aix-ppc64.tar.gz
d8b310b0b6bd6a630307579165cfac8a37571483c7d6804a10dd73bbefb0827f go1.23.5.darwin-amd64.tar.gz
d2b06bf0b8299e0187dfe2d8ad39bd3dd96a6d93fe4d1cfd42c7872452f4a0a2 go1.23.5.darwin-amd64.pkg
047bfce4fbd0da6426bd30cd19716b35a466b1c15a45525ce65b9824acb33285 go1.23.5.darwin-arm64.tar.gz
f819ed94939e08a5016b9a607ec84ebbde6cb3fe59750c59d97aa300c3fd02df go1.23.5.darwin-arm64.pkg
2dec52821e1f04a538d00b2cafe70fa506f2eea94a551bfe3ce1238f1bd4966f go1.23.5.dragonfly-amd64.tar.gz
7204e7bc62913b12f18c61afe0bc1a92fd192c0e45a54125978592296cb84e49 go1.23.5.freebsd-386.tar.gz
90a119995ebc3e36082874df5fa8fe6da194946679d01ae8bef33c87aab99391 go1.23.5.freebsd-amd64.tar.gz
255d26d873e41ff2fc278013bb2e5f25cf2ebe8d0ec84c07e3bb1436216020d3 go1.23.5.freebsd-arm.tar.gz
2785d9122654980b59ca38305a11b34f2a1e12d9f7eb41d52efc137c1fc29e61 go1.23.5.freebsd-arm64.tar.gz
8f66a94018ab666d56868f61c579aa81e549ac9700979ce6004445d315be2d37 go1.23.5.freebsd-riscv64.tar.gz
4b7a69928385ec512a4e77a547e24118adbb92301d2be36187ff0852ba9e6303 go1.23.5.illumos-amd64.tar.gz
6ecf6a41d0925358905fa2641db0e1c9037aa5b5bcd26ca6734caf50d9196417 go1.23.5.linux-386.tar.gz
cbcad4a6482107c7c7926df1608106c189417163428200ce357695cc7e01d091 go1.23.5.linux-amd64.tar.gz
47c84d332123883653b70da2db7dd57d2a865921ba4724efcdf56b5da7021db0 go1.23.5.linux-arm64.tar.gz
04e0b5cf5c216f0aa1bf8204d49312ad0845800ab0702dfe4357c0b1241027a3 go1.23.5.linux-armv6l.tar.gz
e1d14ac2207c78d52b76ba086da18a004c70aeb58cba72cd9bef0da7d1602786 go1.23.5.linux-loong64.tar.gz
d9e937f2fac4fc863850fb4cc31ae76d5495029a62858ef09c78604472d354c0 go1.23.5.linux-mips.tar.gz
59710d0782abafd47e40d1cf96aafa596bbdee09ac7c61062404604f49bd523e go1.23.5.linux-mips64.tar.gz
bc528cd836b4aa6701a42093ed390ef9929639a0e2818759887dc5539e517cab go1.23.5.linux-mips64le.tar.gz
a0404764ea1fd4a175dc5193622b15be6ed1ab59cbfa478f5ae24531bafb6cbd go1.23.5.linux-mipsle.tar.gz
db110284a0c91d4545273f210ca95b9f89f6e3ac90f39eb819033a6b96f25897 go1.23.5.linux-ppc64.tar.gz
db268bf5710b5b1b82ab38722ba6e4427d9e4942aed78c7d09195a9dff329613 go1.23.5.linux-ppc64le.tar.gz
d9da15778442464f32acfa777ac731fd4d47362b233b83a0932380cb6d2d5dc8 go1.23.5.linux-riscv64.tar.gz
14924b917d35311eb130e263f34931043d4f9dc65f20684301bf8f60a72edcdf go1.23.5.linux-s390x.tar.gz
7b8074102e7f039bd6473c44f58cb323c98dcda48df98ad1f78aaa2664769c8f go1.23.5.netbsd-386.tar.gz
1a466b9c8900e66664b15c07548ecb156e8274cf1028ac5da84134728e6dbbed go1.23.5.netbsd-amd64.tar.gz
901c9e72038926e37a4dbde8f03d1d81fcb9992850901a3da1da5a25ef93e65b go1.23.5.netbsd-arm.tar.gz
221f69a7c3a920e3666633ee0b4e5c810176982e74339ba4693226996dc636e4 go1.23.5.netbsd-arm64.tar.gz
42e46cbf73febb8e6ddf848765ce1c39573736383b132402cdc487eb6be3ad06 go1.23.5.openbsd-386.tar.gz
f49e81fce17aab21800fab7c4b10c97ab02f8a9c807fdf8641ccf2f87d69289f go1.23.5.openbsd-amd64.tar.gz
d8bd7269d4670a46e702b64822254a654824347c35923ef1c444d2e8687381ea go1.23.5.openbsd-arm.tar.gz
9cb259adff431d4d28b18e3348e26fe07ea10380675051dcfd740934b5e8b9f2 go1.23.5.openbsd-arm64.tar.gz
72a03223c98fcecfb06e57c3edd584f99fb7f6574a42f59348473f354be1f379 go1.23.5.openbsd-ppc64.tar.gz
c06432b859afb36657207382b7bac03f961b8fafc18176b501d239575a9ace64 go1.23.5.openbsd-riscv64.tar.gz
b1f9b12b269ab5cd4aa7ae3dd3075c2407c1ea8bb1211e6835261f98931201cc go1.23.5.plan9-386.tar.gz
45b4026a103e2f6cd436e2b7ad24b24a40dd22c9903519b98b45c535574fa01a go1.23.5.plan9-amd64.tar.gz
6e28e26f8c1e8620006490260aa5743198843aa0003c400cb65cbf5e743b21c7 go1.23.5.plan9-arm.tar.gz
0496c9969f208bd597f3e63fb27068ce1c7ed776618da1007fcc1c8be83ca413 go1.23.5.solaris-amd64.tar.gz
8441605a005ea74c28d8c02ca5f2708c17b4df7e91796148b9f8760caafb05c1 go1.23.5.windows-386.zip
39962346d8d0cb0cc8716489ee33b08d7a220c24a9e45423487876dd4acbdac6 go1.23.5.windows-386.msi
96d74945d7daeeb98a7978d0cf099321d7eb821b45f5c510373d545162d39c20 go1.23.5.windows-amd64.zip
03e11a988a18ad7e3f9038cef836330af72ba0a454a502cda7b7faee07a0dd8a go1.23.5.windows-amd64.msi
0005b31dcf9732c280a5cceb6aa1c5ab8284bc2541d0256c221256080acf2a09 go1.23.5.windows-arm.zip
a8442de35cbac230db8c4b20e363055671f2295dc4d6b2b2dfec66b89a3c4bce go1.23.5.windows-arm.msi
4f20c2d8a5a387c227e3ef48c5506b22906139d8afd8d66a78ef3de8dda1d1c3 go1.23.5.windows-arm64.zip
6f54fb46b669345c734936c521f7e0f55555e63ed6e11efbbaaed06f9514773c go1.23.5.windows-arm64.msi
# version:golangci 1.61.0
# version:golangci 1.63.4
# https://github.com/golangci/golangci-lint/releases/
# https://github.com/golangci/golangci-lint/releases/download/v1.61.0/
5c280ef3284f80c54fd90d73dc39ca276953949da1db03eb9dd0fbf868cc6e55 golangci-lint-1.61.0-darwin-amd64.tar.gz
544334890701e4e04a6e574bc010bea8945205c08c44cced73745a6378012d36 golangci-lint-1.61.0-darwin-arm64.tar.gz
e885a6f561092055930ebd298914d80e8fd2e10d2b1e9942836c2c6a115301fa golangci-lint-1.61.0-freebsd-386.tar.gz
b13f6a3f11f65e7ff66b734d7554df3bbae0f485768848424e7554ed289e19c2 golangci-lint-1.61.0-freebsd-amd64.tar.gz
cd8e7bbe5b8f33ed1597aa1cc588da96a3b9f22e1b9ae60d93511eae1a0ee8c5 golangci-lint-1.61.0-freebsd-armv6.tar.gz
7ade524dbd88bd250968f45e190af90e151fa5ee63dd6aa7f7bb90e8155db61d golangci-lint-1.61.0-freebsd-armv7.tar.gz
0fe3cd8a1ed8d9f54f48670a5af3df056d6040d94017057f0f4d65c930660ad9 golangci-lint-1.61.0-illumos-amd64.tar.gz
b463fc5053a612abd26393ebaff1d85d7d56058946f4f0f7bf25ed44ea899415 golangci-lint-1.61.0-linux-386.tar.gz
77cb0af99379d9a21d5dc8c38364d060e864a01bd2f3e30b5e8cc550c3a54111 golangci-lint-1.61.0-linux-amd64.tar.gz
af60ac05566d9351615cb31b4cc070185c25bf8cbd9b09c1873aa5ec6f3cc17e golangci-lint-1.61.0-linux-arm64.tar.gz
1f307f2fcc5d7d674062a967a0d83a7091e300529aa237ec6ad2b3dd14c897f5 golangci-lint-1.61.0-linux-armv6.tar.gz
3ad8cbaae75a547450844811300f99c4cd290277398e43d22b9eb1792d15af4c golangci-lint-1.61.0-linux-armv7.tar.gz
9be2ca67d961d7699079739cf6f7c8291c5183d57e34d1677de21ca19d0bd3ed golangci-lint-1.61.0-linux-loong64.tar.gz
90d005e1648115ebf0861b408eab9c936079a24763e883058b0a227cd3135d31 golangci-lint-1.61.0-linux-mips64.tar.gz
6d2ed4f49407115460b8c10ccfc40fd177e0887a48864a2879dd16e84ba2a48c golangci-lint-1.61.0-linux-mips64le.tar.gz
633089589af5a58b7430afb6eee107d4e9c99e8d91711ddc219eb13a07e8d3b8 golangci-lint-1.61.0-linux-ppc64le.tar.gz
4c1a097d9e0d1b4a8144dae6a1f5583a38d662f3bdc1498c4e954b6ed856be98 golangci-lint-1.61.0-linux-riscv64.tar.gz
30581d3c987d287b7064617f1a2694143e10dffc40bc25be6636006ee82d7e1c golangci-lint-1.61.0-linux-s390x.tar.gz
42530bf8100bd43c07f5efe6d92148ba6c5a7a712d510c6f24be85af6571d5eb golangci-lint-1.61.0-netbsd-386.tar.gz
b8bb07c920f6601edf718d5e82ec0784fd590b0992b42b6ec18da99f26013ed4 golangci-lint-1.61.0-netbsd-amd64.tar.gz
353a51527c60bd0776b0891b03f247c791986f625fca689d121972c624e54198 golangci-lint-1.61.0-netbsd-arm64.tar.gz
957a6272c3137910514225704c5dac0723b9c65eb7d9587366a997736e2d7580 golangci-lint-1.61.0-netbsd-armv6.tar.gz
a89eb28ff7f18f5cd52b914739360fa95cf2f643de4adeca46e26bec3a07e8d8 golangci-lint-1.61.0-netbsd-armv7.tar.gz
d8d74c43600b271393000717a4ed157d7a15bb85bab7db2efad9b63a694d4634 golangci-lint-1.61.0-windows-386.zip
e7bc2a81929a50f830244d6d2e657cce4f19a59aff49fa9000176ff34fda64ce golangci-lint-1.61.0-windows-amd64.zip
ed97c221596dd771e3dd9344872c140340bee2e819cd7a90afa1de752f1f2e0f golangci-lint-1.61.0-windows-arm64.zip
4b365233948b13d02d45928a5c390045e00945e919747b9887b5f260247541ae golangci-lint-1.61.0-windows-armv6.zip
595538fb64d152173959d28f6235227f9cd969a828e5af0c4e960d02af4ffd0e golangci-lint-1.61.0-windows-armv7.zip
# https://github.com/golangci/golangci-lint/releases/download/v1.63.4/
878d017cc360e4fb19510d39852c8189852e3c48e7ce0337577df73507c97d68 golangci-lint-1.63.4-darwin-amd64.tar.gz
a2b630c2ac8466393f0ccbbede4462387b6c190697a70bc2298c6d2123f21bbf golangci-lint-1.63.4-darwin-arm64.tar.gz
8938b74aa92888e561a1c5a4c175110b92f84e7d24733703e6d9ebc39e9cd5f8 golangci-lint-1.63.4-freebsd-386.tar.gz
054903339d620df2e760b978920100986e3b03bcb058f669d520a71dac9c34ed golangci-lint-1.63.4-freebsd-amd64.tar.gz
a19d499f961a02608348e8b626537a88edfaab6e1b6534f1eff742b5d6d750e4 golangci-lint-1.63.4-freebsd-armv6.tar.gz
00d616f0fb275b780ce4d26604bdd7fdbfe6bc9c63acd5a0b31498e1f7511108 golangci-lint-1.63.4-freebsd-armv7.tar.gz
d453688e0eabded3c1a97ff5a2777bb0df5a18851efdaaaf6b472e3e5713c33e golangci-lint-1.63.4-illumos-amd64.tar.gz
6b1bec847fc9f347d53712d05606a49d55d0e3b5c1bacadfed2393f3503de0e9 golangci-lint-1.63.4-linux-386.tar.gz
01abb14a4df47b5ca585eff3c34b105023cba92ec34ff17212dbb83855581690 golangci-lint-1.63.4-linux-amd64.tar.gz
51f0c79d19a92353e0465fb30a4901a0644a975d34e6f399ad2eebc0160bbb24 golangci-lint-1.63.4-linux-arm64.tar.gz
8d0a43f41e8424fbae10f7aa2dc29999f98112817c6dba63d7dc76832940a673 golangci-lint-1.63.4-linux-armv6.tar.gz
1045a047b31e9302c9160c7b0f199f4ac1bd02a1b221a2d9521bd3507f0cf671 golangci-lint-1.63.4-linux-armv7.tar.gz
933fe10ab50ce3bb0806e15a4ae69fe20f0549abf91dea0161236000ca706e67 golangci-lint-1.63.4-linux-loong64.tar.gz
45798630cbad5642862766051199fa862ef3c33d569cab12f01cac4f68e2ddd5 golangci-lint-1.63.4-linux-mips64.tar.gz
86ae25335ddb24975d2c915c1af0c7fad70dce99d0b4614fa4bee392de714aa2 golangci-lint-1.63.4-linux-mips64le.tar.gz
33dabd11aaba4b602938da98bcf49aabab55019557e0115cdc3dbcc3009768fa golangci-lint-1.63.4-linux-ppc64le.tar.gz
4e7a81230a663bcdf30bba5689ce96040abc76994dbc2003dce32c8dca8c06f3 golangci-lint-1.63.4-linux-riscv64.tar.gz
21370b49c7c47f4d9b8f982c952f940b01e65710174c3b4dad7b6452d58f92ec golangci-lint-1.63.4-linux-s390x.tar.gz
255866a6464c7e11bb7edd8e6e6ad54f11e1f01b82ba9ca229698ac788cd9724 golangci-lint-1.63.4-netbsd-386.tar.gz
2798c040ac658bda97224f204795199c81ac97bb207b21c02b664aaed380d5d2 golangci-lint-1.63.4-netbsd-amd64.tar.gz
b910eecffd0064103837e7e1abe870deb8ade22331e6dffe319f430d49399c8e golangci-lint-1.63.4-netbsd-arm64.tar.gz
df2693ef37147b457c3e2089614537dd2ae2e18e53641e756a5b404f4c72d3fa golangci-lint-1.63.4-netbsd-armv6.tar.gz
a28a533366974bd7834c4516cd6075bff3419a508d1ed7aa63ae8182768b352e golangci-lint-1.63.4-netbsd-armv7.tar.gz
368932775fb5c620b324dabf018155f3365f5e33c5af5b26e9321db373f96eea golangci-lint-1.63.4-windows-386.zip
184d13c2b8f5441576bec2a0d8ba7b2d45445595cf796b879a73bcc98c39f8c1 golangci-lint-1.63.4-windows-amd64.zip
4fabf175d5b05ef0858ded49527948eebac50e9093814979fd84555a75fb80a6 golangci-lint-1.63.4-windows-arm64.zip
e92be3f3ff30d4a849fb4b9a4c8d56837dee45269cb405a3ecad52fa034c781b golangci-lint-1.63.4-windows-armv6.zip
c71d348653b8f7fbb109bb10c1a481722bc6b0b2b6e731b897f99ac869f7653e golangci-lint-1.63.4-windows-armv7.zip
# This is the builder on PPA that will build Go itself (inception-y), don't modify!
#

39
build/ci.go
View file

@ -298,8 +298,8 @@ func doTest(cmdline []string) {
}
gotest := tc.Go("test")
// CI needs a bit more time for the statetests (default 10m).
gotest.Args = append(gotest.Args, "-timeout=30m")
// CI needs a bit more time for the statetests (default 45m).
gotest.Args = append(gotest.Args, "-timeout=45m")
// Enable CKZG backend in CI.
gotest.Args = append(gotest.Args, "-tags=ckzg")
@ -684,7 +684,8 @@ func maybeSkipArchive(env build.Environment) {
func doDockerBuildx(cmdline []string) {
var (
platform = flag.String("platform", "", `Push a multi-arch docker image for the specified architectures (usually "linux/amd64,linux/arm64")`)
upload = flag.String("upload", "", `Where to upload the docker image (usually "ethereum/client-go")`)
hubImage = flag.String("hub", "ethereum/client-go", `Where to upload the docker image`)
upload = flag.Bool("upload", false, `Whether to trigger upload`)
)
flag.CommandLine.Parse(cmdline)
@ -719,25 +720,33 @@ func doDockerBuildx(cmdline []string) {
tags = []string{"stable", fmt.Sprintf("release-%v", version.Family), "v" + version.Semantic}
}
// Need to create a mult-arch builder
build.MustRunCommand("docker", "buildx", "create", "--use", "--name", "multi-arch-builder", "--platform", *platform)
check := exec.Command("docker", "buildx", "inspect", "multi-arch-builder")
if check.Run() != nil {
build.MustRunCommand("docker", "buildx", "create", "--use", "--name", "multi-arch-builder", "--platform", *platform)
}
for _, spec := range []struct {
file string
base string
}{
{file: "Dockerfile", base: fmt.Sprintf("%s:", *upload)},
{file: "Dockerfile.alltools", base: fmt.Sprintf("%s:alltools-", *upload)},
{file: "Dockerfile", base: fmt.Sprintf("%s:", *hubImage)},
{file: "Dockerfile.alltools", base: fmt.Sprintf("%s:alltools-", *hubImage)},
} {
for _, tag := range tags { // latest, stable etc
gethImage := fmt.Sprintf("%s%s", spec.base, tag)
build.MustRunCommand("docker", "buildx", "build",
cmd := exec.Command("docker", "buildx", "build",
"--build-arg", "COMMIT="+env.Commit,
"--build-arg", "VERSION="+version.WithMeta,
"--build-arg", "BUILDNUM="+env.Buildnum,
"--tag", gethImage,
"--platform", *platform,
"--push",
"--file", spec.file, ".")
"--file", spec.file,
)
if *upload {
cmd.Args = append(cmd.Args, "--push")
}
cmd.Args = append(cmd.Args, ".")
build.MustRun(cmd)
}
}
}
@ -883,11 +892,17 @@ func ppaUpload(workdir, ppa, sshUser string, files []string) {
os.WriteFile(idfile, sshkey, 0600)
}
}
// Upload
// Upload. This doesn't always work, so try up to three times.
dest := sshUser + "@ppa.launchpad.net"
if err := build.UploadSFTP(idfile, dest, incomingDir, files); err != nil {
log.Fatal(err)
for i := 0; i < 3; i++ {
err := build.UploadSFTP(idfile, dest, incomingDir, files)
if err == nil {
return
}
log.Println("PPA upload failed:", err)
time.Sleep(5 * time.Second)
}
log.Fatal("PPA upload failed all attempts.")
}
func getenvBase64(variable string) []byte {

3
cmd/abigen/main.go
View file

@ -98,6 +98,9 @@ func abigen(c *cli.Context) error {
if c.String(pkgFlag.Name) == "" {
utils.Fatalf("No destination package specified (--pkg)")
}
if c.String(abiFlag.Name) == "" && c.String(jsonFlag.Name) == "" {
utils.Fatalf("Either contract ABI source (--abi) or combined-json (--combined-json) are required")
}
var lang bind.Lang
switch c.String(langFlag.Name) {
case "go":

6
cmd/clef/rules.md
View file

@ -7,7 +7,7 @@ It enables usecases like the following:
* I want to auto-approve transactions with contract `CasinoDapp`, with up to `0.05 ether` in value to maximum `1 ether` per 24h period
* I want to auto-approve transaction to contract `EthAlarmClock` with `data`=`0xdeadbeef`, if `value=0`, `gas < 44k` and `gasPrice < 40Gwei`
The two main features that are required for this to work well are;
The two main features that are required for this to work well are:
1. Rule Implementation: how to create, manage, and interpret rules in a flexible but secure manner
2. Credential management and credentials; how to provide auto-unlock without exposing keys unnecessarily.
@ -29,10 +29,10 @@ function asBig(str) {
// Approve transactions to a certain contract if the value is below a certain limit
function ApproveTx(req) {
var limit = big.Newint("0xb1a2bc2ec50000")
var limit = new BigNumber("0xb1a2bc2ec50000")
var value = asBig(req.transaction.value);
if (req.transaction.to.toLowerCase() == "0xae967917c465db8578ca9024c205720b1a3651a9") && value.lt(limit)) {
if (req.transaction.to.toLowerCase() == "0xae967917c465db8578ca9024c205720b1a3651a9" && value.lt(limit)) {
return "Approve"
}
// If we return "Reject", it will be rejected.

9
cmd/devp2p/dns_route53.go
View file

@ -17,6 +17,7 @@
package main
import (
"cmp"
"context"
"errors"
"fmt"
@ -292,13 +293,7 @@ func sortChanges(changes []types.Change) {
if a.Action == b.Action {
return strings.Compare(*a.ResourceRecordSet.Name, *b.ResourceRecordSet.Name)
}
if score[string(a.Action)] < score[string(b.Action)] {
return -1
}
if score[string(a.Action)] > score[string(b.Action)] {
return 1
}
return 0
return cmp.Compare(score[string(a.Action)], score[string(b.Action)])
})
}

31
cmd/devp2p/internal/ethtest/chain.go
View file

@ -28,7 +28,6 @@ import (
"os"
"path/filepath"
"slices"
"sort"
"strings"
"github.com/ethereum/go-ethereum/common"
@ -41,6 +40,7 @@ import (
"github.com/ethereum/go-ethereum/eth/protocols/eth"
"github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/rlp"
"golang.org/x/exp/maps"
)
// Chain is a lightweight blockchain-like store which can read a hivechain
@ -143,11 +143,7 @@ func (c *Chain) ForkID() forkid.ID {
// TD calculates the total difficulty of the chain at the
// chain head.
func (c *Chain) TD() *big.Int {
sum := new(big.Int)
for _, block := range c.blocks[:c.Len()] {
sum.Add(sum, block.Difficulty())
}
return sum
return new(big.Int)
}
// GetBlock returns the block at the specified number.
@ -166,11 +162,8 @@ func (c *Chain) RootAt(height int) common.Hash {
// GetSender returns the address associated with account at the index in the
// pre-funded accounts list.
func (c *Chain) GetSender(idx int) (common.Address, uint64) {
var accounts Addresses
for addr := range c.senders {
accounts = append(accounts, addr)
}
sort.Sort(accounts)
accounts := maps.Keys(c.senders)
slices.SortFunc(accounts, common.Address.Cmp)
addr := accounts[idx]
return addr, c.senders[addr].Nonce
}
@ -260,22 +253,6 @@ func loadGenesis(genesisFile string) (core.Genesis, error) {
return gen, nil
}
type Addresses []common.Address
func (a Addresses) Len() int {
return len(a)
}
func (a Addresses) Less(i, j int) bool {
return bytes.Compare(a[i][:], a[j][:]) < 0
}
func (a Addresses) Swap(i, j int) {
tmp := a[i]
a[i] = a[j]
a[j] = tmp
}
func blocksFromFile(chainfile string, gblock *types.Block) ([]*types.Block, error) {
// Load chain.rlp.
fh, err := os.Open(chainfile)

1
cmd/devp2p/internal/ethtest/protocol.go
View file

@ -13,6 +13,7 @@
//
// You should have received a copy of the GNU General Public License
// along with go-ethereum. If not, see <http://www.gnu.org/licenses/>.
package ethtest
import (

40
cmd/devp2p/internal/ethtest/suite.go
View file

@ -18,7 +18,6 @@ package ethtest
import (
"crypto/rand"
"math/big"
"reflect"
"github.com/ethereum/go-ethereum/common"
@ -74,7 +73,6 @@ func (s *Suite) EthTests() []utesting.Test {
{Name: "GetBlockBodies", Fn: s.TestGetBlockBodies},
// // malicious handshakes + status
{Name: "MaliciousHandshake", Fn: s.TestMaliciousHandshake},
{Name: "MaliciousStatus", Fn: s.TestMaliciousStatus},
// test transactions
{Name: "LargeTxRequest", Fn: s.TestLargeTxRequest, Slow: true},
{Name: "Transaction", Fn: s.TestTransaction},
@ -453,42 +451,6 @@ func (s *Suite) TestMaliciousHandshake(t *utesting.T) {
}
}
func (s *Suite) TestMaliciousStatus(t *utesting.T) {
t.Log(`This test sends a malicious eth Status message to the node and expects a disconnect.`)
conn, err := s.dial()
if err != nil {
t.Fatalf("dial failed: %v", err)
}
defer conn.Close()
if err := conn.handshake(); err != nil {
t.Fatalf("handshake failed: %v", err)
}
// Create status with large total difficulty.
status := &eth.StatusPacket{
ProtocolVersion: uint32(conn.negotiatedProtoVersion),
NetworkID: s.chain.config.ChainID.Uint64(),
TD: new(big.Int).SetBytes(randBuf(2048)),
Head: s.chain.Head().Hash(),
Genesis: s.chain.GetBlock(0).Hash(),
ForkID: s.chain.ForkID(),
}
if err := conn.statusExchange(s.chain, status); err != nil {
t.Fatalf("status exchange failed: %v", err)
}
// Wait for disconnect.
code, _, err := conn.Read()
if err != nil {
t.Fatalf("error reading from connection: %v", err)
}
switch code {
case discMsg:
break
default:
t.Fatalf("expected disconnect, got: %d", code)
}
}
func (s *Suite) TestTransaction(t *utesting.T) {
t.Log(`This test sends a valid transaction to the node and checks if the
transaction gets propagated.`)
@ -781,7 +743,7 @@ func (s *Suite) makeBlobTxs(count, blobs int, discriminator byte) (txs types.Tra
GasTipCap: uint256.NewInt(1),
GasFeeCap: uint256.MustFromBig(s.chain.Head().BaseFee()),
Gas: 100000,
BlobFeeCap: uint256.MustFromBig(eip4844.CalcBlobFee(*s.chain.Head().ExcessBlobGas())),
BlobFeeCap: uint256.MustFromBig(eip4844.CalcBlobFee(s.chain.config, s.chain.Head().Header())),
BlobHashes: makeSidecar(blobdata...).BlobHashes(),
Sidecar: makeSidecar(blobdata...),
}

9
cmd/devp2p/internal/ethtest/testdata/genesis.json vendored
View file

@ -18,7 +18,14 @@
"shanghaiTime": 780,
"cancunTime": 840,
"terminalTotalDifficulty": 9454784,
"ethash": {}
"ethash": {},
"blobSchedule": {
"cancun": {
"target": 3,
"max": 6,
"baseFeeUpdateFraction": 3338477
}
}
},
"nonce": "0x0",
"timestamp": "0x0",

18
cmd/devp2p/internal/v4test/discv4tests.go
View file

@ -194,7 +194,7 @@ func PingExtraData(t *utesting.T) {
}
}
// This test sends a PING packet with additional data and wrong 'from' field
// PingExtraDataWrongFrom sends a PING packet with additional data and wrong 'from' field
// and expects a PONG response.
func PingExtraDataWrongFrom(t *utesting.T) {
te := newTestEnv(Remote, Listen1, Listen2)
@ -215,7 +215,7 @@ func PingExtraDataWrongFrom(t *utesting.T) {
}
}
// This test sends a PING packet with an expiration in the past.
// PingPastExpiration sends a PING packet with an expiration in the past.
// The remote node should not respond.
func PingPastExpiration(t *utesting.T) {
te := newTestEnv(Remote, Listen1, Listen2)
@ -234,7 +234,7 @@ func PingPastExpiration(t *utesting.T) {
}
}
// This test sends an invalid packet. The remote node should not respond.
// WrongPacketType sends an invalid packet. The remote node should not respond.
func WrongPacketType(t *utesting.T) {
te := newTestEnv(Remote, Listen1, Listen2)
defer te.close()
@ -252,7 +252,7 @@ func WrongPacketType(t *utesting.T) {
}
}
// This test verifies that the default behaviour of ignoring 'from' fields is unaffected by
// BondThenPingWithWrongFrom verifies that the default behaviour of ignoring 'from' fields is unaffected by
// the bonding process. After bonding, it pings the target with a different from endpoint.
func BondThenPingWithWrongFrom(t *utesting.T) {
te := newTestEnv(Remote, Listen1, Listen2)
@ -289,7 +289,7 @@ waitForPong:
}
}
// This test just sends FINDNODE. The remote node should not reply
// FindnodeWithoutEndpointProof sends FINDNODE. The remote node should not reply
// because the endpoint proof has not completed.
func FindnodeWithoutEndpointProof(t *utesting.T) {
te := newTestEnv(Remote, Listen1, Listen2)
@ -332,7 +332,7 @@ func BasicFindnode(t *utesting.T) {
}
}
// This test sends an unsolicited NEIGHBORS packet after the endpoint proof, then sends
// UnsolicitedNeighbors sends an unsolicited NEIGHBORS packet after the endpoint proof, then sends
// FINDNODE to read the remote table. The remote node should not return the node contained
// in the unsolicited NEIGHBORS packet.
func UnsolicitedNeighbors(t *utesting.T) {
@ -373,7 +373,7 @@ func UnsolicitedNeighbors(t *utesting.T) {
}
}
// This test sends FINDNODE with an expiration timestamp in the past.
// FindnodePastExpiration sends FINDNODE with an expiration timestamp in the past.
// The remote node should not respond.
func FindnodePastExpiration(t *utesting.T) {
te := newTestEnv(Remote, Listen1, Listen2)
@ -426,7 +426,7 @@ func bond(t *utesting.T, te *testenv) {
}
}
// This test attempts to perform a traffic amplification attack against a
// FindnodeAmplificationInvalidPongHash attempts to perform a traffic amplification attack against a
// 'victim' endpoint using FINDNODE. In this attack scenario, the attacker
// attempts to complete the endpoint proof non-interactively by sending a PONG
// with mismatching reply token from the 'victim' endpoint. The attack works if
@ -478,7 +478,7 @@ func FindnodeAmplificationInvalidPongHash(t *utesting.T) {
}
}
// This test attempts to perform a traffic amplification attack using FINDNODE.
// FindnodeAmplificationWrongIP attempts to perform a traffic amplification attack using FINDNODE.
// The attack works if the remote node does not verify the IP address of FINDNODE
// against the endpoint verification proof done by PING/PONG.
func FindnodeAmplificationWrongIP(t *utesting.T) {

9
cmd/devp2p/nodeset.go
View file

@ -18,6 +18,7 @@ package main
import (
"bytes"
"cmp"
"encoding/json"
"fmt"
"os"
@ -104,13 +105,7 @@ func (ns nodeSet) topN(n int) nodeSet {
byscore = append(byscore, v)
}
slices.SortFunc(byscore, func(a, b nodeJSON) int {
if a.Score > b.Score {
return -1
}
if a.Score < b.Score {
return 1
}
return 0
return cmp.Compare(b.Score, a.Score)
})
result := make(nodeSet, n)
for _, v := range byscore[:n] {

2
cmd/evm/blockrunner.go
View file

@ -50,7 +50,7 @@ func blockTestCmd(ctx *cli.Context) error {
return errors.New("path argument required")
}
var (
collected = collectJSONFiles(path)
collected = collectFiles(path)
results []testResult
)
for _, fname := range collected {

78
cmd/evm/internal/t8ntool/execution.go
View file

@ -17,9 +17,7 @@
package t8ntool
import (
"encoding/json"
"fmt"
"io"
"math/big"
"github.com/ethereum/go-ethereum/common"
@ -35,7 +33,6 @@ import (
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/core/vm"
"github.com/ethereum/go-ethereum/crypto"
"github.com/ethereum/go-ethereum/eth/tracers"
"github.com/ethereum/go-ethereum/ethdb"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/params"
@ -130,9 +127,7 @@ type rejectedTx struct {
}
// Apply applies a set of transactions to a pre-state
func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
txIt txIterator, miningReward int64,
getTracerFn func(txIndex int, txHash common.Hash, chainConfig *params.ChainConfig) (*tracers.Tracer, io.WriteCloser, error)) (*state.StateDB, *ExecutionResult, []byte, error) {
func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig, txIt txIterator, miningReward int64) (*state.StateDB, *ExecutionResult, []byte, error) {
// Capture errors for BLOCKHASH operation, if we haven't been supplied the
// required blockhashes
var hashError error
@ -183,15 +178,28 @@ func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
var excessBlobGas uint64
if pre.Env.ExcessBlobGas != nil {
excessBlobGas = *pre.Env.ExcessBlobGas
vmContext.BlobBaseFee = eip4844.CalcBlobFee(excessBlobGas)
header := &types.Header{
Time: pre.Env.Timestamp,
ExcessBlobGas: pre.Env.ExcessBlobGas,
}
vmContext.BlobBaseFee = eip4844.CalcBlobFee(chainConfig, header)
} else {
// If it is not explicitly defined, but we have the parent values, we try
// to calculate it ourselves.
parentExcessBlobGas := pre.Env.ParentExcessBlobGas
parentBlobGasUsed := pre.Env.ParentBlobGasUsed
if parentExcessBlobGas != nil && parentBlobGasUsed != nil {
excessBlobGas = eip4844.CalcExcessBlobGas(*parentExcessBlobGas, *parentBlobGasUsed)
vmContext.BlobBaseFee = eip4844.CalcBlobFee(excessBlobGas)
parent := &types.Header{
Time: pre.Env.ParentTimestamp,
ExcessBlobGas: pre.Env.ParentExcessBlobGas,
BlobGasUsed: pre.Env.ParentBlobGasUsed,
}
excessBlobGas = eip4844.CalcExcessBlobGas(chainConfig, parent)
header := &types.Header{
Time: pre.Env.Timestamp,
ExcessBlobGas: &excessBlobGas,
}
vmContext.BlobBaseFee = eip4844.CalcBlobFee(chainConfig, header)
}
}
// If DAO is supported/enabled, we need to handle it here. In geth 'proper', it's
@ -234,30 +242,21 @@ func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
txBlobGas := uint64(0)
if tx.Type() == types.BlobTxType {
txBlobGas = uint64(params.BlobTxBlobGasPerBlob * len(tx.BlobHashes()))
if used, max := blobGasUsed+txBlobGas, uint64(params.MaxBlobGasPerBlock); used > max {
max := eip4844.MaxBlobGasPerBlock(chainConfig, pre.Env.Timestamp)
if used := blobGasUsed + txBlobGas; used > max {
err := fmt.Errorf("blob gas (%d) would exceed maximum allowance %d", used, max)
log.Warn("rejected tx", "index", i, "err", err)
rejectedTxs = append(rejectedTxs, &rejectedTx{i, err.Error()})
continue
}
}
tracer, traceOutput, err := getTracerFn(txIndex, tx.Hash(), chainConfig)
if err != nil {
return nil, nil, nil, err
}
// TODO (rjl493456442) it's a bit weird to reset the tracer in the
// middle of block execution, please improve it somehow.
if tracer != nil {
evm.SetTracer(tracer.Hooks)
}
statedb.SetTxContext(tx.Hash(), txIndex)
var (
snapshot = statedb.Snapshot()
prevGas = gaspool.Gas()
)
if tracer != nil && tracer.OnTxStart != nil {
tracer.OnTxStart(evm.GetVMContext(), tx, msg.From)
if evm.Config.Tracer != nil && evm.Config.Tracer.OnTxStart != nil {
evm.Config.Tracer.OnTxStart(evm.GetVMContext(), tx, msg.From)
}
// (ret []byte, usedGas uint64, failed bool, err error)
msgResult, err := core.ApplyMessage(evm, msg, gaspool)
@ -266,13 +265,8 @@ func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
log.Info("rejected tx", "index", i, "hash", tx.Hash(), "from", msg.From, "error", err)
rejectedTxs = append(rejectedTxs, &rejectedTx{i, err.Error()})
gaspool.SetGas(prevGas)
if tracer != nil {
if tracer.OnTxEnd != nil {
tracer.OnTxEnd(nil, err)
}
if err := writeTraceResult(tracer, traceOutput); err != nil {
log.Warn("Error writing tracer output", "err", err)
}
if evm.Config.Tracer != nil && evm.Config.Tracer.OnTxEnd != nil {
evm.Config.Tracer.OnTxEnd(nil, err)
}
continue
}
@ -316,13 +310,8 @@ func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
//receipt.BlockNumber
receipt.TransactionIndex = uint(txIndex)
receipts = append(receipts, receipt)
if tracer != nil {
if tracer.Hooks.OnTxEnd != nil {
tracer.Hooks.OnTxEnd(receipt, nil)
}
if err = writeTraceResult(tracer, traceOutput); err != nil {
log.Warn("Error writing tracer output", "err", err)
}
if evm.Config.Tracer != nil && evm.Config.Tracer.OnTxEnd != nil {
evm.Config.Tracer.OnTxEnd(receipt, nil)
}
}
@ -379,7 +368,7 @@ func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
}
// Commit block
root, err := statedb.Commit(vmContext.BlockNumber.Uint64(), chainConfig.IsEIP158(vmContext.BlockNumber))
root, err := statedb.Commit(vmContext.BlockNumber.Uint64(), chainConfig.IsEIP158(vmContext.BlockNumber), chainConfig.IsCancun(vmContext.BlockNumber, vmContext.Time))
if err != nil {
return nil, nil, nil, NewError(ErrorEVM, fmt.Errorf("could not commit state: %v", err))
}
@ -437,7 +426,7 @@ func MakePreState(db ethdb.Database, accounts types.GenesisAlloc) *state.StateDB
}
}
// Commit and re-open to start with a clean state.
root, _ := statedb.Commit(0, false)
root, _ := statedb.Commit(0, false, false)
statedb, _ = state.New(root, sdb)
return statedb
}
@ -468,16 +457,3 @@ func calcDifficulty(config *params.ChainConfig, number, currentTime, parentTime
}
return ethash.CalcDifficulty(config, currentTime, parent)
}
func writeTraceResult(tracer *tracers.Tracer, f io.WriteCloser) error {
defer f.Close()
result, err := tracer.GetResult()
if err != nil || result == nil {
return err
}
err = json.NewEncoder(f).Encode(result)
if err != nil {
return err
}
return nil
}

152
cmd/evm/internal/t8ntool/file_tracer.go Normal file
View file

@ -0,0 +1,152 @@
// Copyright 2024 The go-ethereum Authors
// This file is part of go-ethereum.
//
// go-ethereum is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// go-ethereum is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with go-ethereum. If not, see <http://www.gnu.org/licenses/>.
package t8ntool
import (
"encoding/json"
"fmt"
"io"
"math/big"
"os"
"path/filepath"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/core/tracing"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/eth/tracers"
"github.com/ethereum/go-ethereum/log"
)
// fileWritingTracer wraps either a tracer or a logger. On tx start,
// it instantiates a tracer/logger, creates a new file to direct output to,
// and on tx end it closes the file.
type fileWritingTracer struct {
txIndex int // transaction counter
inner *tracing.Hooks // inner hooks
destination io.WriteCloser // the currently open file (if any)
baseDir string // baseDir to write output-files to
suffix string // suffix is the suffix to use when creating files
// for custom tracing
getResult func() (json.RawMessage, error)
}
func (l *fileWritingTracer) Write(p []byte) (n int, err error) {
if l.destination != nil {
return l.destination.Write(p)
}
log.Warn("Tracer wrote to non-existing output")
// It is tempting to return an error here, however, the json encoder
// will no retry writing to an io.Writer once it has returned an error once.
// Therefore, we must squash the error.
return n, nil
}
// newFileWriter creates a set of hooks which wraps inner hooks (typically a logger),
// and writes the output to a file, one file per transaction.
func newFileWriter(baseDir string, innerFn func(out io.Writer) *tracing.Hooks) *tracing.Hooks {
t := &fileWritingTracer{
baseDir: baseDir,
suffix: "jsonl",
}
t.inner = innerFn(t) // instantiate the inner tracer
return t.hooks()
}
// newResultWriter creates a set of hooks wraps and invokes an underlying tracer,
// and writes the result (getResult-output) to file, one per transaction.
func newResultWriter(baseDir string, tracer *tracers.Tracer) *tracing.Hooks {
t := &fileWritingTracer{
baseDir: baseDir,
getResult: tracer.GetResult,
inner: tracer.Hooks,
suffix: "json",
}
return t.hooks()
}
// OnTxStart creates a new output-file specific for this transaction, and invokes
// the inner OnTxStart handler.
func (l *fileWritingTracer) OnTxStart(env *tracing.VMContext, tx *types.Transaction, from common.Address) {
// Open a new file, or print a warning log if it's failed
fname := filepath.Join(l.baseDir, fmt.Sprintf("trace-%d-%v.%v", l.txIndex, tx.Hash().String(), l.suffix))
traceFile, err := os.Create(fname)
if err != nil {
log.Warn("Failed creating trace-file", "err", err)
} else {
log.Info("Created tracing-file", "path", fname)
l.destination = traceFile
}
if l.inner != nil && l.inner.OnTxStart != nil {
l.inner.OnTxStart(env, tx, from)
}
}
// OnTxEnd writes result (if getResult exist), closes any currently open output-file,
// and invokes the inner OnTxEnd handler.
func (l *fileWritingTracer) OnTxEnd(receipt *types.Receipt, err error) {
if l.inner != nil && l.inner.OnTxEnd != nil {
l.inner.OnTxEnd(receipt, err)
}
if l.getResult != nil && l.destination != nil {
if result, err := l.getResult(); result != nil {
json.NewEncoder(l.destination).Encode(result)
} else {
log.Warn("Error obtaining tracer result", "err", err)
}
l.destination.Close()
l.destination = nil
}
l.txIndex++
}
func (l *fileWritingTracer) hooks() *tracing.Hooks {
return &tracing.Hooks{
OnTxStart: l.OnTxStart,
OnTxEnd: l.OnTxEnd,
OnEnter: func(depth int, typ byte, from common.Address, to common.Address, input []byte, gas uint64, value *big.Int) {
if l.inner != nil && l.inner.OnEnter != nil {
l.inner.OnEnter(depth, typ, from, to, input, gas, value)
}
},
OnExit: func(depth int, output []byte, gasUsed uint64, err error, reverted bool) {
if l.inner != nil && l.inner.OnExit != nil {
l.inner.OnExit(depth, output, gasUsed, err, reverted)
}
},
OnOpcode: func(pc uint64, op byte, gas, cost uint64, scope tracing.OpContext, rData []byte, depth int, err error) {
if l.inner != nil && l.inner.OnOpcode != nil {
l.inner.OnOpcode(pc, op, gas, cost, scope, rData, depth, err)
}
},
OnFault: func(pc uint64, op byte, gas, cost uint64, scope tracing.OpContext, depth int, err error) {
if l.inner != nil && l.inner.OnFault != nil {
l.inner.OnFault(pc, op, gas, cost, scope, depth, err)
}
},
OnSystemCallStart: func() {
if l.inner != nil && l.inner.OnSystemCallStart != nil {
l.inner.OnSystemCallStart()
}
},
OnSystemCallEnd: func() {
if l.inner != nil && l.inner.OnSystemCallEnd != nil {
l.inner.OnSystemCallEnd()
}
},
}
}

32
cmd/evm/internal/t8ntool/transaction.go
View file

@ -64,12 +64,10 @@ func (r *result) MarshalJSON() ([]byte, error) {
}
func Transaction(ctx *cli.Context) error {
var (
err error
)
// We need to load the transactions. May be either in stdin input or in files.
// Check if anything needs to be read from stdin
var (
err error
txStr = ctx.String(InputTxsFlag.Name)
inputData = &input{}
chainConfig *params.ChainConfig
@ -82,6 +80,7 @@ func Transaction(ctx *cli.Context) error {
}
// Set the chain id
chainConfig.ChainID = big.NewInt(ctx.Int64(ChainIDFlag.Name))
var body hexutil.Bytes
if txStr == stdinSelector {
decoder := json.NewDecoder(os.Stdin)
@ -107,6 +106,7 @@ func Transaction(ctx *cli.Context) error {
}
}
signer := types.MakeSigner(chainConfig, new(big.Int), 0)
// We now have the transactions in 'body', which is supposed to be an
// rlp list of transactions
it, err := rlp.NewListIterator([]byte(body))
@ -133,15 +133,29 @@ func Transaction(ctx *cli.Context) error {
r.Address = sender
}
// Check intrinsic gas
if gas, err := core.IntrinsicGas(tx.Data(), tx.AccessList(), tx.AuthList(), tx.To() == nil,
chainConfig.IsHomestead(new(big.Int)), chainConfig.IsIstanbul(new(big.Int)), chainConfig.IsShanghai(new(big.Int), 0)); err != nil {
rules := chainConfig.Rules(common.Big0, true, 0)
gas, err := core.IntrinsicGas(tx.Data(), tx.AccessList(), tx.SetCodeAuthorizations(), tx.To() == nil, rules.IsHomestead, rules.IsIstanbul, rules.IsShanghai)
if err != nil {
r.Error = err
results = append(results, r)
continue
} else {
r.IntrinsicGas = gas
if tx.Gas() < gas {
r.Error = fmt.Errorf("%w: have %d, want %d", core.ErrIntrinsicGas, tx.Gas(), gas)
}
r.IntrinsicGas = gas
if tx.Gas() < gas {
r.Error = fmt.Errorf("%w: have %d, want %d", core.ErrIntrinsicGas, tx.Gas(), gas)
results = append(results, r)
continue
}
// For Prague txs, validate the floor data gas.
if rules.IsPrague {
floorDataGas, err := core.FloorDataGas(tx.Data())
if err != nil {
r.Error = err
results = append(results, r)
continue
}
if tx.Gas() < floorDataGas {
r.Error = fmt.Errorf("%w: have %d, want %d", core.ErrFloorDataGas, tx.Gas(), floorDataGas)
results = append(results, r)
continue
}

77
cmd/evm/internal/t8ntool/transition.go
View file

@ -82,58 +82,10 @@ type input struct {
}
func Transition(ctx *cli.Context) error {
var getTracer = func(txIndex int, txHash common.Hash, chainConfig *params.ChainConfig) (*tracers.Tracer, io.WriteCloser, error) {
return nil, nil, nil
}
baseDir, err := createBasedir(ctx)
if err != nil {
return NewError(ErrorIO, fmt.Errorf("failed creating output basedir: %v", err))
}
if ctx.Bool(TraceFlag.Name) { // JSON opcode tracing
// Configure the EVM logger
logConfig := &logger.Config{
DisableStack: ctx.Bool(TraceDisableStackFlag.Name),
EnableMemory: ctx.Bool(TraceEnableMemoryFlag.Name),
EnableReturnData: ctx.Bool(TraceEnableReturnDataFlag.Name),
}
getTracer = func(txIndex int, txHash common.Hash, _ *params.ChainConfig) (*tracers.Tracer, io.WriteCloser, error) {
traceFile, err := os.Create(filepath.Join(baseDir, fmt.Sprintf("trace-%d-%v.jsonl", txIndex, txHash.String())))
if err != nil {
return nil, nil, NewError(ErrorIO, fmt.Errorf("failed creating trace-file: %v", err))
}
var l *tracing.Hooks
if ctx.Bool(TraceEnableCallFramesFlag.Name) {
l = logger.NewJSONLoggerWithCallFrames(logConfig, traceFile)
} else {
l = logger.NewJSONLogger(logConfig, traceFile)
}
tracer := &tracers.Tracer{
Hooks: l,
// jsonLogger streams out result to file.
GetResult: func() (json.RawMessage, error) { return nil, nil },
Stop: func(err error) {},
}
return tracer, traceFile, nil
}
} else if ctx.IsSet(TraceTracerFlag.Name) {
var config json.RawMessage
if ctx.IsSet(TraceTracerConfigFlag.Name) {
config = []byte(ctx.String(TraceTracerConfigFlag.Name))
}
getTracer = func(txIndex int, txHash common.Hash, chainConfig *params.ChainConfig) (*tracers.Tracer, io.WriteCloser, error) {
traceFile, err := os.Create(filepath.Join(baseDir, fmt.Sprintf("trace-%d-%v.json", txIndex, txHash.String())))
if err != nil {
return nil, nil, NewError(ErrorIO, fmt.Errorf("failed creating trace-file: %v", err))
}
tracer, err := tracers.DefaultDirectory.New(ctx.String(TraceTracerFlag.Name), nil, config, chainConfig)
if err != nil {
return nil, nil, NewError(ErrorConfig, fmt.Errorf("failed instantiating tracer: %w", err))
}
return tracer, traceFile, nil
}
}
// We need to load three things: alloc, env and transactions. May be either in
// stdin input or in files.
// Check if anything needs to be read from stdin
@ -179,6 +131,7 @@ func Transition(ctx *cli.Context) error {
chainConfig = cConf
vmConfig.ExtraEips = extraEips
}
// Set the chain id
chainConfig.ChainID = big.NewInt(ctx.Int64(ChainIDFlag.Name))
@ -197,8 +150,34 @@ func Transition(ctx *cli.Context) error {
if err := applyCancunChecks(&prestate.Env, chainConfig); err != nil {
return err
}
// Configure tracer
if ctx.IsSet(TraceTracerFlag.Name) { // Custom tracing
config := json.RawMessage(ctx.String(TraceTracerConfigFlag.Name))
tracer, err := tracers.DefaultDirectory.New(ctx.String(TraceTracerFlag.Name),
nil, config, chainConfig)
if err != nil {
return NewError(ErrorConfig, fmt.Errorf("failed instantiating tracer: %v", err))
}
vmConfig.Tracer = newResultWriter(baseDir, tracer)
} else if ctx.Bool(TraceFlag.Name) { // JSON opcode tracing
logConfig := &logger.Config{
DisableStack: ctx.Bool(TraceDisableStackFlag.Name),
EnableMemory: ctx.Bool(TraceEnableMemoryFlag.Name),
EnableReturnData: ctx.Bool(TraceEnableReturnDataFlag.Name),
}
if ctx.Bool(TraceEnableCallFramesFlag.Name) {
vmConfig.Tracer = newFileWriter(baseDir, func(out io.Writer) *tracing.Hooks {
return logger.NewJSONLoggerWithCallFrames(logConfig, out)
})
} else {
vmConfig.Tracer = newFileWriter(baseDir, func(out io.Writer) *tracing.Hooks {
return logger.NewJSONLogger(logConfig, out)
})
}
}
// Run the test and aggregate the result
s, result, body, err := prestate.Apply(vmConfig, chainConfig, txIt, ctx.Int64(RewardFlag.Name), getTracer)
s, result, body, err := prestate.Apply(vmConfig, chainConfig, txIt, ctx.Int64(RewardFlag.Name))
if err != nil {
return err
}

15
cmd/evm/main.go
View file

@ -83,8 +83,8 @@ var (
}
TraceFormatFlag = &cli.StringFlag{
Name: "trace.format",
Usage: "Trace output format to use (struct|json)",
Value: "struct",
Usage: "Trace output format to use (json|struct|md)",
Value: "json",
Category: traceCategory,
}
TraceDisableMemoryFlag = &cli.BoolFlag{
@ -262,10 +262,15 @@ func tracerFromFlags(ctx *cli.Context) *tracing.Hooks {
}
}
// collectJSONFiles walks the given path and accumulates all files with json
// extension.
func collectJSONFiles(path string) []string {
// collectFiles walks the given path. If the path is a directory, it will
// return a list of all accumulates all files with json extension.
// Otherwise (if path points to a file), it will return the path.
func collectFiles(path string) []string {
var out []string
if info, err := os.Stat(path); err == nil && !info.IsDir() {
// User explicitly pointed out a file, ignore extension.
return []string{path}
}
err := filepath.Walk(path, func(path string, info fs.FileInfo, err error) error {
if err != nil {
return err

2
cmd/evm/runner.go
View file

@ -336,7 +336,7 @@ func runCmd(ctx *cli.Context) error {
output, stats, err := timedExec(bench, execFunc)
if ctx.Bool(DumpFlag.Name) {
root, err := runtimeConfig.State.Commit(genesisConfig.Number, true)
root, err := runtimeConfig.State.Commit(genesisConfig.Number, true, false)
if err != nil {
fmt.Printf("Failed to commit changes %v\n", err)
return err

3
cmd/evm/staterunner.go
View file

@ -51,6 +51,7 @@ var stateTestCommand = &cli.Command{
Usage: "Executes the given state tests. Filenames can be fed via standard input (batch mode) or as an argument (one-off execution).",
ArgsUsage: "<file>",
Flags: slices.Concat([]cli.Flag{
BenchFlag,
DumpFlag,
HumanReadableFlag,
RunFlag,
@ -63,7 +64,7 @@ func stateTestCmd(ctx *cli.Context) error {
// If path is provided, run the tests at that path.
if len(path) != 0 {
var (
collected = collectJSONFiles(path)
collected = collectFiles(path)
results []testResult
)
for _, fname := range collected {

82
cmd/evm/t8n_test.go
View file

@ -24,6 +24,7 @@ import (
"os"
"path/filepath"
"reflect"
"regexp"
"strings"
"testing"
@ -606,18 +607,18 @@ func TestEvmRun(t *testing.T) {
wantStdout: "./testdata/evmrun/1.out.1.txt",
wantStderr: "./testdata/evmrun/1.out.2.txt",
},
{ // default tracing (struct)
input: []string{"run", "--trace", "0x6040"},
{ // Struct tracing
input: []string{"run", "--trace", "--trace.format=struct", "0x6040"},
wantStdout: "./testdata/evmrun/2.out.1.txt",
wantStderr: "./testdata/evmrun/2.out.2.txt",
},
{ // default tracing (struct), plus alloc-dump
input: []string{"run", "--trace", "--dump", "0x6040"},
{ // struct-tracing, plus alloc-dump
input: []string{"run", "--trace", "--trace.format=struct", "--dump", "0x6040"},
wantStdout: "./testdata/evmrun/3.out.1.txt",
//wantStderr: "./testdata/evmrun/3.out.2.txt",
},
{ // json-tracing, plus alloc-dump
input: []string{"run", "--trace", "--trace.format=json", "--dump", "0x6040"},
{ // json-tracing (default), plus alloc-dump
input: []string{"run", "--trace", "--dump", "0x6040"},
wantStdout: "./testdata/evmrun/4.out.1.txt",
//wantStderr: "./testdata/evmrun/4.out.2.txt",
},
@ -670,6 +671,61 @@ func TestEvmRun(t *testing.T) {
}
}
func TestEvmRunRegEx(t *testing.T) {
t.Parallel()
tt := cmdtest.NewTestCmd(t, nil)
for i, tc := range []struct {
input []string
wantStdout string
wantStderr string
}{
{ // json tracing
input: []string{"run", "--bench", "6040"},
wantStdout: "./testdata/evmrun/9.out.1.txt",
wantStderr: "./testdata/evmrun/9.out.2.txt",
},
{ // statetest subcommand
input: []string{"statetest", "--bench", "./testdata/statetest.json"},
wantStdout: "./testdata/evmrun/10.out.1.txt",
wantStderr: "./testdata/evmrun/10.out.2.txt",
},
} {
tt.Logf("args: go run ./cmd/evm %v\n", strings.Join(tc.input, " "))
tt.Run("evm-test", tc.input...)
haveStdOut := tt.Output()
tt.WaitExit()
haveStdErr := tt.StderrText()
if have, wantFile := haveStdOut, tc.wantStdout; wantFile != "" {
want, err := os.ReadFile(wantFile)
if err != nil {
t.Fatalf("test %d: could not read expected output: %v", i, err)
}
re, err := regexp.Compile(string(want))
if err != nil {
t.Fatalf("test %d: could not compile regular expression: %v", i, err)
}
if !re.Match(have) {
t.Fatalf("test %d, output wrong, have \n%v\nwant\n%v\n", i, string(have), re)
}
}
if have, wantFile := haveStdErr, tc.wantStderr; wantFile != "" {
want, err := os.ReadFile(wantFile)
if err != nil {
t.Fatalf("test %d: could not read expected output: %v", i, err)
}
re, err := regexp.Compile(string(want))
if err != nil {
t.Fatalf("test %d: could not compile regular expression: %v", i, err)
}
if !re.MatchString(have) {
t.Fatalf("test %d, output wrong, have \n%v\nwant\n%v\n", i, have, re)
}
}
}
}
// cmpJson compares the JSON in two byte slices.
func cmpJson(a, b []byte) (bool, error) {
var j, j2 interface{}
@ -698,7 +754,10 @@ func TestEVMTracing(t *testing.T) {
"--input.env=./testdata/31/env.json", "--state.fork=Cancun",
"--trace",
},
expectedTraces: []string{"trace-0-0x88f5fbd1524731a81e49f637aa847543268a5aaf2a6b32a69d2c6d978c45dcfb.jsonl"},
//expectedTraces: []string{"trace-0-0x88f5fbd1524731a81e49f637aa847543268a5aaf2a6b32a69d2c6d978c45dcfb.jsonl"},
expectedTraces: []string{"trace-0-0x88f5fbd1524731a81e49f637aa847543268a5aaf2a6b32a69d2c6d978c45dcfb.jsonl",
"trace-1-0x03a7b0a91e61a170d64ea94b8263641ef5a8bbdb10ac69f466083a6789c77fb8.jsonl",
"trace-2-0xd96e0ce6418ee3360e11d3c7b6886f5a9a08f7ef183da72c23bb3b2374530128.jsonl"},
},
{
base: "./testdata/31",
@ -706,14 +765,17 @@ func TestEVMTracing(t *testing.T) {
"--input.alloc=./testdata/31/alloc.json", "--input.txs=./testdata/31/txs.json",
"--input.env=./testdata/31/env.json", "--state.fork=Cancun",
"--trace.tracer", `
{
{ count: 0,
result: function(){
return "hello world"
this.count = this.count + 1;
return "hello world " + this.count
},
fault: function(){}
}`,
},
expectedTraces: []string{"trace-0-0x88f5fbd1524731a81e49f637aa847543268a5aaf2a6b32a69d2c6d978c45dcfb.json"},
expectedTraces: []string{"trace-0-0x88f5fbd1524731a81e49f637aa847543268a5aaf2a6b32a69d2c6d978c45dcfb.json",
"trace-1-0x03a7b0a91e61a170d64ea94b8263641ef5a8bbdb10ac69f466083a6789c77fb8.json",
"trace-2-0xd96e0ce6418ee3360e11d3c7b6886f5a9a08f7ef183da72c23bb3b2374530128.json"},
},
{
base: "./testdata/32",

10
cmd/evm/testdata/31/README.md vendored
View file

@ -1 +1,11 @@
This test does some EVM execution, and can be used to test the tracers and trace-outputs.
This test should yield three output-traces, in separate files
For example:
```
[user@work evm]$ go run . t8n --input.alloc ./testdata/31/alloc.json --input.txs ./testdata/31/txs.json --input.env ./testdata/31/env.json --state.fork Cancun --output.basedir /tmp --trace
INFO [12-06|09:53:32.123] Created tracing-file path=/tmp/trace-0-0x88f5fbd1524731a81e49f637aa847543268a5aaf2a6b32a69d2c6d978c45dcfb.jsonl
INFO [12-06|09:53:32.124] Created tracing-file path=/tmp/trace-1-0x03a7b0a91e61a170d64ea94b8263641ef5a8bbdb10ac69f466083a6789c77fb8.jsonl
INFO [12-06|09:53:32.125] Created tracing-file path=/tmp/trace-2-0xd96e0ce6418ee3360e11d3c7b6886f5a9a08f7ef183da72c23bb3b2374530128.jsonl
```

2
cmd/evm/testdata/31/trace-0-0x88f5fbd1524731a81e49f637aa847543268a5aaf2a6b32a69d2c6d978c45dcfb.json vendored
View file

@ -1 +1 @@
"hello world"
"hello world 1"

1
cmd/evm/testdata/31/trace-1-0x03a7b0a91e61a170d64ea94b8263641ef5a8bbdb10ac69f466083a6789c77fb8.json vendored Normal file
View file

@ -0,0 +1 @@
"hello world 2"

6
cmd/evm/testdata/31/trace-1-0x03a7b0a91e61a170d64ea94b8263641ef5a8bbdb10ac69f466083a6789c77fb8.jsonl vendored Normal file
View file

@ -0,0 +1,6 @@
{"pc":0,"op":96,"gas":"0x13498","gasCost":"0x3","memSize":0,"stack":[],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":2,"op":96,"gas":"0x13495","gasCost":"0x3","memSize":0,"stack":["0x40"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":4,"op":96,"gas":"0x13492","gasCost":"0x3","memSize":0,"stack":["0x40","0x40"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":6,"op":96,"gas":"0x1348f","gasCost":"0x3","memSize":0,"stack":["0x40","0x40","0x40"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":8,"op":0,"gas":"0x1348c","gasCost":"0x0","memSize":0,"stack":["0x40","0x40","0x40","0x40"],"depth":1,"refund":0,"opName":"STOP"}
{"output":"","gasUsed":"0xc"}

1
cmd/evm/testdata/31/trace-2-0xd96e0ce6418ee3360e11d3c7b6886f5a9a08f7ef183da72c23bb3b2374530128.json vendored Normal file
View file

@ -0,0 +1 @@
"hello world 3"

6
cmd/evm/testdata/31/trace-2-0xd96e0ce6418ee3360e11d3c7b6886f5a9a08f7ef183da72c23bb3b2374530128.jsonl vendored Normal file
View file

@ -0,0 +1,6 @@
{"pc":0,"op":96,"gas":"0x13498","gasCost":"0x3","memSize":0,"stack":[],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":2,"op":96,"gas":"0x13495","gasCost":"0x3","memSize":0,"stack":["0x40"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":4,"op":96,"gas":"0x13492","gasCost":"0x3","memSize":0,"stack":["0x40","0x40"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":6,"op":96,"gas":"0x1348f","gasCost":"0x3","memSize":0,"stack":["0x40","0x40","0x40"],"depth":1,"refund":0,"opName":"PUSH1"}
{"pc":8,"op":0,"gas":"0x1348c","gasCost":"0x0","memSize":0,"stack":["0x40","0x40","0x40","0x40"],"depth":1,"refund":0,"opName":"STOP"}
{"output":"","gasUsed":"0xc"}

24
cmd/evm/testdata/31/txs.json vendored
View file

@ -10,5 +10,29 @@
"r" : "0x0",
"s" : "0x0",
"secretKey" : "0x45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8"
},
{
"gas": "0x186a0",
"gasPrice": "0x600",
"input": "0x",
"nonce": "0x1",
"to": "0x1111111111111111111111111111111111111111",
"value": "0x1",
"v" : "0x0",
"r" : "0x0",
"s" : "0x0",
"secretKey" : "0x45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8"
},
{
"gas": "0x186a0",
"gasPrice": "0x600",
"input": "0x",
"nonce": "0x2",
"to": "0x1111111111111111111111111111111111111111",
"value": "0x1",
"v" : "0x0",
"r" : "0x0",
"s" : "0x0",
"secretKey" : "0x45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8"
}
]

4
cmd/evm/testdata/33/txs.json vendored
View file

@ -16,7 +16,7 @@
"chainId": "0x1",
"address": "0x000000000000000000000000000000000000aaaa",
"nonce": "0x1",
"v": "0x1",
"yParity": "0x1",
"r": "0xf7e3e597fc097e71ed6c26b14b25e5395bc8510d58b9136af439e12715f2d721",
"s": "0x6cf7c3d7939bfdb784373effc0ebb0bd7549691a513f395e3cdabf8602724987"
},
@ -24,7 +24,7 @@
"chainId": "0x0",
"address": "0x000000000000000000000000000000000000bbbb",
"nonce": "0x0",
"v": "0x1",
"yParity": "0x1",
"r": "0x5011890f198f0356a887b0779bde5afa1ed04e6acb1e3f37f8f18c7b6f521b98",
"s": "0x56c3fa3456b103f3ef4a0acb4b647b9cab9ec4bc68fbcdf1e10b49fb2bcbcf61"
}

15
cmd/evm/testdata/evmrun/10.out.1.txt vendored Normal file
View file

@ -0,0 +1,15 @@
\[
\{
"name": "00000006-naivefuzz-0",
"pass": false,
"stateRoot": "0xad1024c87b5548e77c937aa50f72b6cb620d278f4dd79bae7f78f71ff75af458",
"fork": "London",
"error": "post state root mismatch: got ad1024c87b5548e77c937aa50f72b6cb620d278f4dd79bae7f78f71ff75af458, want 0000000000000000000000000000000000000000000000000000000000000000",
"benchStats": \{
"time": \d+,
"allocs": \d+,
"bytesAllocated": \d+,
"gasUsed": \d+
\}
\}
\]

0
cmd/evm/testdata/evmrun/10.out.2.txt vendored Normal file
View file

0
cmd/evm/testdata/evmrun/9.out.1.txt vendored Normal file
View file

4
cmd/evm/testdata/evmrun/9.out.2.txt vendored Normal file
View file

@ -0,0 +1,4 @@
EVM gas used: \d+
execution time: \d+\.\d+.s
allocations: \d+
allocated bytes: \d+

3
cmd/geth/chaincmd.go
View file

@ -230,11 +230,10 @@ func initGenesis(ctx *cli.Context) error {
triedb := utils.MakeTrieDatabase(ctx, chaindb, ctx.Bool(utils.CachePreimagesFlag.Name), false, genesis.IsVerkle())
defer triedb.Close()
_, hash, err := core.SetupGenesisBlockWithOverride(chaindb, triedb, genesis, &overrides)
_, hash, _, err := core.SetupGenesisBlockWithOverride(chaindb, triedb, genesis, &overrides)
if err != nil {
utils.Fatalf("Failed to write genesis block: %v", err)
}
log.Info("Successfully wrote genesis state", "database", "chaindata", "hash", hash)
return nil

3
cmd/geth/dbcmd.go
View file

@ -829,8 +829,7 @@ func inspectAccount(db *triedb.Database, start uint64, end uint64, address commo
func inspectStorage(db *triedb.Database, start uint64, end uint64, address common.Address, slot common.Hash, raw bool) error {
// The hash of storage slot key is utilized in the history
// rather than the raw slot key, make the conversion.
slotHash := crypto.Keccak256Hash(slot.Bytes())
stats, err := db.StorageHistory(address, slotHash, start, end)
stats, err := db.StorageHistory(address, slot, start, end)
if err != nil {
return err
}

12
cmd/utils/cmd.go
View file

@ -25,6 +25,7 @@ import (
"errors"
"fmt"
"io"
"math/big"
"os"
"os/signal"
"path/filepath"
@ -422,6 +423,10 @@ func ExportHistory(bc *core.BlockChain, dir string, first, last, step uint64) er
buf = bytes.NewBuffer(nil)
checksums []string
)
td := new(big.Int)
for i := uint64(0); i < first; i++ {
td.Add(td, bc.GetHeaderByNumber(i).Difficulty)
}
for i := first; i <= last; i += step {
err := func() error {
filename := filepath.Join(dir, era.Filename(network, int(i/step), common.Hash{}))
@ -444,11 +449,8 @@ func ExportHistory(bc *core.BlockChain, dir string, first, last, step uint64) er
if receipts == nil {
return fmt.Errorf("export failed on #%d: receipts not found", n)
}
td := bc.GetTd(block.Hash(), block.NumberU64())
if td == nil {
return fmt.Errorf("export failed on #%d: total difficulty not found", n)
}
if err := w.Add(block, receipts, td); err != nil {
td.Add(td, block.Difficulty())
if err := w.Add(block, receipts, new(big.Int).Set(td)); err != nil {
return err
}
}

4
cmd/utils/diskusage.go
View file

@ -14,8 +14,8 @@
// You should have received a copy of the GNU General Public License
// along with go-ethereum. If not, see <http://www.gnu.org/licenses/>.
//go:build !windows && !openbsd
// +build !windows,!openbsd
//go:build !windows && !openbsd && !wasip1
// +build !windows,!openbsd,!wasip1
package utils

2
cmd/utils/flags.go
View file

@ -760,7 +760,7 @@ var (
}
NATFlag = &cli.StringFlag{
Name: "nat",
Usage: "NAT port mapping mechanism (any|none|upnp|pmp|pmp:<IP>|extip:<IP>)",
Usage: "NAT port mapping mechanism (any|none|upnp|pmp|pmp:<IP>|extip:<IP>|stun:<IP:PORT>)",
Value: "any",
Category: flags.NetworkingCategory,
}

2
cmd/utils/history_test.go
View file

@ -170,7 +170,7 @@ func TestHistoryImportAndExport(t *testing.T) {
db2.Close()
})
genesis.MustCommit(db2, triedb.NewDatabase(db, triedb.HashDefaults))
genesis.MustCommit(db2, triedb.NewDatabase(db2, triedb.HashDefaults))
imported, err := core.NewBlockChain(db2, nil, genesis, nil, ethash.NewFaker(), vm.Config{}, nil)
if err != nil {
t.Fatalf("unable to initialize chain: %v", err)

131
consensus/beacon/consensus.go
View file

@ -61,7 +61,8 @@ var (
// is only used for necessary consensus checks. The legacy consensus engine can be any
// engine implements the consensus interface (except the beacon itself).
type Beacon struct {
ethone consensus.Engine // Original consensus engine used in eth1, e.g. ethash or clique
ethone consensus.Engine // Original consensus engine used in eth1, e.g. ethash or clique
ttdblock *uint64 // Merge block-number for testchain generation without TTDs
}
// New creates a consensus engine with the given embedded eth1 engine.
@ -72,6 +73,18 @@ func New(ethone consensus.Engine) *Beacon {
return &Beacon{ethone: ethone}
}
// TestingTTDBlock is a replacement mechanism for TTD-based pre-/post-merge
// splitting. With chain history deletion, TD calculations become impossible.
// This is fine for progressing the live chain, but to be able to generate test
// chains, we do need a split point. This method supports setting an explicit
// block number to use as the splitter *for testing*, instead of having to keep
// the notion of TDs in the client just for testing.
//
// The block with supplied number is regarded as the last pre-merge block.
func (beacon *Beacon) TestingTTDBlock(number uint64) {
beacon.ttdblock = &number
}
// Author implements consensus.Engine, returning the verified author of the block.
func (beacon *Beacon) Author(header *types.Header) (common.Address, error) {
if !beacon.IsPoSHeader(header) {
@ -83,67 +96,55 @@ func (beacon *Beacon) Author(header *types.Header) (common.Address, error) {
// VerifyHeader checks whether a header conforms to the consensus rules of the
// stock Ethereum consensus engine.
func (beacon *Beacon) VerifyHeader(chain consensus.ChainHeaderReader, header *types.Header) error {
reached, err := IsTTDReached(chain, header.ParentHash, header.Number.Uint64()-1)
if err != nil {
return err
}
if !reached {
return beacon.ethone.VerifyHeader(chain, header)
}
// Short circuit if the parent is not known
// During the live merge transition, the consensus engine used the terminal
// total difficulty to detect when PoW (PoA) switched to PoS. Maintaining the
// total difficulty values however require applying all the blocks from the
// genesis to build up the TD. This stops being a possibility if the tail of
// the chain is pruned already during sync.
//
// One heuristic that can be used to distinguish pre-merge and post-merge
// blocks is whether their *difficulty* is >0 or ==0 respectively. This of
// course would mean that we cannot prove anymore for a past chain that it
// truly transitioned at the correct TTD, but if we consider that ancient
// point in time finalized a long time ago, there should be no attempt from
// the consensus client to rewrite very old history.
//
// One thing that's probably not needed but which we can add to make this
// verification even stricter is to enforce that the chain can switch from
// >0 to ==0 TD only once by forbidding an ==0 to be followed by a >0.
// Verify that we're not reverting to pre-merge from post-merge
parent := chain.GetHeader(header.ParentHash, header.Number.Uint64()-1)
if parent == nil {
return consensus.ErrUnknownAncestor
}
// Sanity checks passed, do a proper verification
if parent.Difficulty.Sign() == 0 && header.Difficulty.Sign() > 0 {
return consensus.ErrInvalidTerminalBlock
}
// Check >0 TDs with pre-merge, --0 TDs with post-merge rules
if header.Difficulty.Sign() > 0 {
return beacon.ethone.VerifyHeader(chain, header)
}
return beacon.verifyHeader(chain, header, parent)
}
// errOut constructs an error channel with prefilled errors inside.
func errOut(n int, err error) chan error {
errs := make(chan error, n)
for i := 0; i < n; i++ {
errs <- err
}
return errs
}
// splitHeaders splits the provided header batch into two parts according to
// the configured ttd. It requires the parent of header batch along with its
// td are stored correctly in chain. If ttd is not configured yet, all headers
// will be treated legacy PoW headers.
// the difficulty field.
//
// Note, this function will not verify the header validity but just split them.
func (beacon *Beacon) splitHeaders(chain consensus.ChainHeaderReader, headers []*types.Header) ([]*types.Header, []*types.Header, error) {
// TTD is not defined yet, all headers should be in legacy format.
ttd := chain.Config().TerminalTotalDifficulty
ptd := chain.GetTd(headers[0].ParentHash, headers[0].Number.Uint64()-1)
if ptd == nil {
return nil, nil, consensus.ErrUnknownAncestor
}
// The entire header batch already crosses the transition.
if ptd.Cmp(ttd) >= 0 {
return nil, headers, nil
}
func (beacon *Beacon) splitHeaders(headers []*types.Header) ([]*types.Header, []*types.Header) {
var (
preHeaders = headers
postHeaders []*types.Header
td = new(big.Int).Set(ptd)
tdPassed bool
)
for i, header := range headers {
if tdPassed {
if header.Difficulty.Sign() == 0 {
preHeaders = headers[:i]
postHeaders = headers[i:]
break
}
td = td.Add(td, header.Difficulty)
if td.Cmp(ttd) >= 0 {
// This is the last PoW header, it still belongs to
// the preHeaders, so we cannot split+break yet.
tdPassed = true
}
}
return preHeaders, postHeaders, nil
return preHeaders, postHeaders
}
// VerifyHeaders is similar to VerifyHeader, but verifies a batch of headers
@ -151,10 +152,7 @@ func (beacon *Beacon) splitHeaders(chain consensus.ChainHeaderReader, headers []
// a results channel to retrieve the async verifications.
// VerifyHeaders expect the headers to be ordered and continuous.
func (beacon *Beacon) VerifyHeaders(chain consensus.ChainHeaderReader, headers []*types.Header) (chan<- struct{}, <-chan error) {
preHeaders, postHeaders, err := beacon.splitHeaders(chain, headers)
if err != nil {
return make(chan struct{}), errOut(len(headers), err)
}
preHeaders, postHeaders := beacon.splitHeaders(headers)
if len(postHeaders) == 0 {
return beacon.ethone.VerifyHeaders(chain, headers)
}
@ -284,7 +282,7 @@ func (beacon *Beacon) verifyHeader(chain consensus.ChainHeaderReader, header, pa
if header.ParentBeaconRoot == nil {
return errors.New("header is missing beaconRoot")
}
if err := eip4844.VerifyEIP4844Header(parent, header); err != nil {
if err := eip4844.VerifyEIP4844Header(chain.Config(), parent, header); err != nil {
return err
}
}
@ -334,12 +332,15 @@ func (beacon *Beacon) verifyHeaders(chain consensus.ChainHeaderReader, headers [
// Prepare implements consensus.Engine, initializing the difficulty field of a
// header to conform to the beacon protocol. The changes are done inline.
func (beacon *Beacon) Prepare(chain consensus.ChainHeaderReader, header *types.Header) error {
// Transition isn't triggered yet, use the legacy rules for preparation.
reached, err := IsTTDReached(chain, header.ParentHash, header.Number.Uint64()-1)
if err != nil {
return err
}
if !reached {
// The beacon engine requires access to total difficulties to be able to
// seal pre-merge and post-merge blocks. With the transition to removing
// old blocks, TDs become unaccessible, thus making TTD based pre-/post-
// merge decisions impossible.
//
// We do not need to seal non-merge blocks anymore live, but we do need
// to be able to generate test chains, thus we're reverting to a testing-
// settable field to direct that.
if beacon.ttdblock != nil && *beacon.ttdblock >= header.Number.Uint64() {
return beacon.ethone.Prepare(chain, header)
}
header.Difficulty = beaconDifficulty
@ -449,8 +450,15 @@ func (beacon *Beacon) SealHash(header *types.Header) common.Hash {
// the difficulty that a new block should have when created at time
// given the parent block's time and difficulty.
func (beacon *Beacon) CalcDifficulty(chain consensus.ChainHeaderReader, time uint64, parent *types.Header) *big.Int {
// Transition isn't triggered yet, use the legacy rules for calculation
if reached, _ := IsTTDReached(chain, parent.Hash(), parent.Number.Uint64()); !reached {
// The beacon engine requires access to total difficulties to be able to
// seal pre-merge and post-merge blocks. With the transition to removing
// old blocks, TDs become unaccessible, thus making TTD based pre-/post-
// merge decisions impossible.
//
// We do not need to seal non-merge blocks anymore live, but we do need
// to be able to generate test chains, thus we're reverting to a testing-
// settable field to direct that.
if beacon.ttdblock != nil && *beacon.ttdblock > parent.Number.Uint64() {
return beacon.ethone.CalcDifficulty(chain, time, parent)
}
return beaconDifficulty
@ -491,14 +499,3 @@ func (beacon *Beacon) SetThreads(threads int) {
th.SetThreads(threads)
}
}
// IsTTDReached checks if the TotalTerminalDifficulty has been surpassed on the `parentHash` block.
// It depends on the parentHash already being stored in the database.
// If the parentHash is not stored in the database a UnknownAncestor error is returned.
func IsTTDReached(chain consensus.ChainHeaderReader, parentHash common.Hash, parentNumber uint64) (bool, error) {
td := chain.GetTd(parentHash, parentNumber)
if td == nil {
return false, consensus.ErrUnknownAncestor
}
return td.Cmp(chain.Config().TerminalTotalDifficulty) >= 0, nil
}

1
consensus/clique/snapshot_test.go
View file

@ -467,7 +467,6 @@ func (tt *cliqueTest) run(t *testing.T) {
for j := 0; j < len(batches)-1; j++ {
if k, err := chain.InsertChain(batches[j]); err != nil {
t.Fatalf("failed to import batch %d, block %d: %v", j, k, err)
break
}
}
if _, err = chain.InsertChain(batches[len(batches)-1]); err != tt.failure {

3
consensus/consensus.go
View file

@ -45,9 +45,6 @@ type ChainHeaderReader interface {
// GetHeaderByHash retrieves a block header from the database by its hash.
GetHeaderByHash(hash common.Hash) *types.Header
// GetTd retrieves the total difficulty from the database by hash and number.
GetTd(hash common.Hash, number uint64) *big.Int
}
// ChainReader defines a small collection of methods needed to access the local

110
consensus/misc/eip4844/eip4844.go
View file

@ -23,17 +23,17 @@ import (
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/params/forks"
)
var (
minBlobGasPrice = big.NewInt(params.BlobTxMinBlobGasprice)
blobGaspriceUpdateFraction = big.NewInt(params.BlobTxBlobGaspriceUpdateFraction)
minBlobGasPrice = big.NewInt(params.BlobTxMinBlobGasprice)
)
// VerifyEIP4844Header verifies the presence of the excessBlobGas field and that
// if the current block contains no transactions, the excessBlobGas is updated
// accordingly.
func VerifyEIP4844Header(parent, header *types.Header) error {
func VerifyEIP4844Header(config *params.ChainConfig, parent, header *types.Header) error {
// Verify the header is not malformed
if header.ExcessBlobGas == nil {
return errors.New("header is missing excessBlobGas")
@ -42,14 +42,26 @@ func VerifyEIP4844Header(parent, header *types.Header) error {
return errors.New("header is missing blobGasUsed")
}
// Verify that the blob gas used remains within reasonable limits.
if *header.BlobGasUsed > params.MaxBlobGasPerBlock {
return fmt.Errorf("blob gas used %d exceeds maximum allowance %d", *header.BlobGasUsed, params.MaxBlobGasPerBlock)
maxBlobGas := MaxBlobGasPerBlock(config, header.Time)
if *header.BlobGasUsed > maxBlobGas {
return fmt.Errorf("blob gas used %d exceeds maximum allowance %d", *header.BlobGasUsed, maxBlobGas)
}
if *header.BlobGasUsed%params.BlobTxBlobGasPerBlob != 0 {
return fmt.Errorf("blob gas used %d not a multiple of blob gas per blob %d", header.BlobGasUsed, params.BlobTxBlobGasPerBlob)
}
// Verify the excessBlobGas is correct based on the parent header
expectedExcessBlobGas := CalcExcessBlobGas(config, parent)
if *header.ExcessBlobGas != expectedExcessBlobGas {
return fmt.Errorf("invalid excessBlobGas: have %d, want %d", *header.ExcessBlobGas, expectedExcessBlobGas)
}
return nil
}
// CalcExcessBlobGas calculates the excess blob gas after applying the set of
// blobs on top of the excess blob gas.
func CalcExcessBlobGas(config *params.ChainConfig, parent *types.Header) uint64 {
var (
targetGas = uint64(targetBlobsPerBlock(config, parent.Time)) * params.BlobTxBlobGasPerBlob
parentExcessBlobGas uint64
parentBlobGasUsed uint64
)
@ -57,27 +69,85 @@ func VerifyEIP4844Header(parent, header *types.Header) error {
parentExcessBlobGas = *parent.ExcessBlobGas
parentBlobGasUsed = *parent.BlobGasUsed
}
expectedExcessBlobGas := CalcExcessBlobGas(parentExcessBlobGas, parentBlobGasUsed)
if *header.ExcessBlobGas != expectedExcessBlobGas {
return fmt.Errorf("invalid excessBlobGas: have %d, want %d, parent excessBlobGas %d, parent blobDataUsed %d",
*header.ExcessBlobGas, expectedExcessBlobGas, parentExcessBlobGas, parentBlobGasUsed)
}
return nil
}
// CalcExcessBlobGas calculates the excess blob gas after applying the set of
// blobs on top of the excess blob gas.
func CalcExcessBlobGas(parentExcessBlobGas uint64, parentBlobGasUsed uint64) uint64 {
excessBlobGas := parentExcessBlobGas + parentBlobGasUsed
if excessBlobGas < params.BlobTxTargetBlobGasPerBlock {
if excessBlobGas < targetGas {
return 0
}
return excessBlobGas - params.BlobTxTargetBlobGasPerBlock
return excessBlobGas - targetGas
}
// CalcBlobFee calculates the blobfee from the header's excess blob gas field.
func CalcBlobFee(excessBlobGas uint64) *big.Int {
return fakeExponential(minBlobGasPrice, new(big.Int).SetUint64(excessBlobGas), blobGaspriceUpdateFraction)
func CalcBlobFee(config *params.ChainConfig, header *types.Header) *big.Int {
var frac uint64
switch config.LatestFork(header.Time) {
case forks.Prague:
frac = config.BlobScheduleConfig.Prague.UpdateFraction
case forks.Cancun:
frac = config.BlobScheduleConfig.Cancun.UpdateFraction
default:
panic("calculating blob fee on unsupported fork")
}
return fakeExponential(minBlobGasPrice, new(big.Int).SetUint64(*header.ExcessBlobGas), new(big.Int).SetUint64(frac))
}
// MaxBlobsPerBlock returns the max blobs per block for a block at the given timestamp.
func MaxBlobsPerBlock(cfg *params.ChainConfig, time uint64) int {
if cfg.BlobScheduleConfig == nil {
return 0
}
var (
london = cfg.LondonBlock
s = cfg.BlobScheduleConfig
)
switch {
case cfg.IsPrague(london, time) && s.Prague != nil:
return s.Prague.Max
case cfg.IsCancun(london, time) && s.Cancun != nil:
return s.Cancun.Max
default:
return 0
}
}
// MaxBlobsPerBlock returns the maximum blob gas that can be spent in a block at the given timestamp.
func MaxBlobGasPerBlock(cfg *params.ChainConfig, time uint64) uint64 {
return uint64(MaxBlobsPerBlock(cfg, time)) * params.BlobTxBlobGasPerBlob
}
// LatestMaxBlobsPerBlock returns the latest max blobs per block defined by the
// configuration, regardless of the currently active fork.
func LatestMaxBlobsPerBlock(cfg *params.ChainConfig) int {
s := cfg.BlobScheduleConfig
if s == nil {
return 0
}
switch {
case s.Prague != nil:
return s.Prague.Max
case s.Cancun != nil:
return s.Cancun.Max
default:
return 0
}
}
// targetBlobsPerBlock returns the target number of blobs in a block at the given timestamp.
func targetBlobsPerBlock(cfg *params.ChainConfig, time uint64) int {
if cfg.BlobScheduleConfig == nil {
return 0
}
var (
london = cfg.LondonBlock
s = cfg.BlobScheduleConfig
)
switch {
case cfg.IsPrague(london, time) && s.Prague != nil:
return s.Prague.Target
case cfg.IsCancun(london, time) && s.Cancun != nil:
return s.Cancun.Target
default:
return 0
}
}
// fakeExponential approximates factor * e ** (numerator / denominator) using

38
consensus/misc/eip4844/eip4844_test.go
View file

@ -21,36 +21,48 @@ import (
"math/big"
"testing"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/params"
)
func TestCalcExcessBlobGas(t *testing.T) {
var (
config = params.MainnetChainConfig
targetBlobs = targetBlobsPerBlock(config, *config.CancunTime)
targetBlobGas = uint64(targetBlobs) * params.BlobTxBlobGasPerBlob
)
var tests = []struct {
excess uint64
blobs uint64
blobs int
want uint64
}{
// The excess blob gas should not increase from zero if the used blob
// slots are below - or equal - to the target.
{0, 0, 0},
{0, 1, 0},
{0, params.BlobTxTargetBlobGasPerBlock / params.BlobTxBlobGasPerBlob, 0},
{0, targetBlobs, 0},
// If the target blob gas is exceeded, the excessBlobGas should increase
// by however much it was overshot
{0, (params.BlobTxTargetBlobGasPerBlock / params.BlobTxBlobGasPerBlob) + 1, params.BlobTxBlobGasPerBlob},
{1, (params.BlobTxTargetBlobGasPerBlock / params.BlobTxBlobGasPerBlob) + 1, params.BlobTxBlobGasPerBlob + 1},
{1, (params.BlobTxTargetBlobGasPerBlock / params.BlobTxBlobGasPerBlob) + 2, 2*params.BlobTxBlobGasPerBlob + 1},
{0, targetBlobs + 1, params.BlobTxBlobGasPerBlob},
{1, targetBlobs + 1, params.BlobTxBlobGasPerBlob + 1},
{1, targetBlobs + 2, 2*params.BlobTxBlobGasPerBlob + 1},
// The excess blob gas should decrease by however much the target was
// under-shot, capped at zero.
{params.BlobTxTargetBlobGasPerBlock, params.BlobTxTargetBlobGasPerBlock / params.BlobTxBlobGasPerBlob, params.BlobTxTargetBlobGasPerBlock},
{params.BlobTxTargetBlobGasPerBlock, (params.BlobTxTargetBlobGasPerBlock / params.BlobTxBlobGasPerBlob) - 1, params.BlobTxTargetBlobGasPerBlock - params.BlobTxBlobGasPerBlob},
{params.BlobTxTargetBlobGasPerBlock, (params.BlobTxTargetBlobGasPerBlock / params.BlobTxBlobGasPerBlob) - 2, params.BlobTxTargetBlobGasPerBlock - (2 * params.BlobTxBlobGasPerBlob)},
{params.BlobTxBlobGasPerBlob - 1, (params.BlobTxTargetBlobGasPerBlock / params.BlobTxBlobGasPerBlob) - 1, 0},
{targetBlobGas, targetBlobs, targetBlobGas},
{targetBlobGas, targetBlobs - 1, targetBlobGas - params.BlobTxBlobGasPerBlob},
{targetBlobGas, targetBlobs - 2, targetBlobGas - (2 * params.BlobTxBlobGasPerBlob)},
{params.BlobTxBlobGasPerBlob - 1, targetBlobs - 1, 0},
}
for i, tt := range tests {
result := CalcExcessBlobGas(tt.excess, tt.blobs*params.BlobTxBlobGasPerBlob)
blobGasUsed := uint64(tt.blobs) * params.BlobTxBlobGasPerBlob
parent := &types.Header{
Time: *config.CancunTime,
ExcessBlobGas: &tt.excess,
BlobGasUsed: &blobGasUsed,
}
result := CalcExcessBlobGas(config, parent)
if result != tt.want {
t.Errorf("test %d: excess blob gas mismatch: have %v, want %v", i, result, tt.want)
}
@ -58,6 +70,8 @@ func TestCalcExcessBlobGas(t *testing.T) {
}
func TestCalcBlobFee(t *testing.T) {
zero := uint64(0)
tests := []struct {
excessBlobGas uint64
blobfee int64
@ -68,7 +82,9 @@ func TestCalcBlobFee(t *testing.T) {
{10 * 1024 * 1024, 23},
}
for i, tt := range tests {
have := CalcBlobFee(tt.excessBlobGas)
config := &params.ChainConfig{LondonBlock: big.NewInt(0), CancunTime: &zero, BlobScheduleConfig: params.DefaultBlobSchedule}
header := &types.Header{ExcessBlobGas: &tt.excessBlobGas}
have := CalcBlobFee(config, header)
if have.Int64() != tt.blobfee {
t.Errorf("test %d: blobfee mismatch: have %v want %v", i, have, tt.blobfee)
}

1
core/bench_test.go
View file

@ -285,7 +285,6 @@ func makeChainForBench(db ethdb.Database, genesis *Genesis, full bool, count uin
rawdb.WriteHeader(db, header)
rawdb.WriteCanonicalHash(db, hash, n)
rawdb.WriteTd(db, hash, n, big.NewInt(int64(n+1)))
if n == 0 {
rawdb.WriteChainConfig(db, hash, genesis.Config)

106
core/blockchain.go
View file

@ -113,21 +113,30 @@ const (
// * the `BlockNumber`, `TxHash`, `TxIndex`, `BlockHash` and `Index` fields of log are deleted
// * the `Bloom` field of receipt is deleted
// * the `BlockIndex` and `TxIndex` fields of txlookup are deleted
//
// - Version 5
// The following incompatible database changes were added:
// * the `TxHash`, `GasCost`, and `ContractAddress` fields are no longer stored for a receipt
// * the `TxHash`, `GasCost`, and `ContractAddress` fields are computed by looking up the
// receipts' corresponding block
//
// - Version 6
// The following incompatible database changes were added:
// * Transaction lookup information stores the corresponding block number instead of block hash
//
// - Version 7
// The following incompatible database changes were added:
// * Use freezer as the ancient database to maintain all ancient data
//
// - Version 8
// The following incompatible database changes were added:
// * New scheme for contract code in order to separate the codes and trie nodes
BlockChainVersion uint64 = 8
//
// - Version 9
// The following incompatible database changes were added:
// * Total difficulty has been removed from both the key-value store and the ancient store.
// * The metadata structure of freezer is changed by adding 'flushOffset'
BlockChainVersion uint64 = 9
)
// CacheConfig contains the configuration values for the trie database
@ -269,14 +278,19 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, genesis *Genesis
cacheConfig = defaultCacheConfig
}
// Open trie database with provided config
triedb := triedb.NewDatabase(db, cacheConfig.triedbConfig(genesis != nil && genesis.IsVerkle()))
enableVerkle, err := EnableVerkleAtGenesis(db, genesis)
if err != nil {
return nil, err
}
triedb := triedb.NewDatabase(db, cacheConfig.triedbConfig(enableVerkle))
// Setup the genesis block, commit the provided genesis specification
// to database if the genesis block is not present yet, or load the
// stored one from database.
chainConfig, genesisHash, genesisErr := SetupGenesisBlockWithOverride(db, triedb, genesis, overrides)
if _, ok := genesisErr.(*params.ConfigCompatError); genesisErr != nil && !ok {
return nil, genesisErr
// Write the supplied genesis to the database if it has not been initialized
// yet. The corresponding chain config will be returned, either from the
// provided genesis or from the locally stored configuration if the genesis
// has already been initialized.
chainConfig, genesisHash, compatErr, err := SetupGenesisBlockWithOverride(db, triedb, genesis, overrides)
if err != nil {
return nil, err
}
log.Info("")
log.Info(strings.Repeat("-", 153))
@ -303,7 +317,6 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, genesis *Genesis
vmConfig: vmConfig,
logger: vmConfig.Tracer,
}
var err error
bc.hc, err = NewHeaderChain(db, chainConfig, engine, bc.insertStopped)
if err != nil {
return nil, err
@ -453,16 +466,15 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, genesis *Genesis
}
// Rewind the chain in case of an incompatible config upgrade.
if compat, ok := genesisErr.(*params.ConfigCompatError); ok {
log.Warn("Rewinding chain to upgrade configuration", "err", compat)
if compat.RewindToTime > 0 {
bc.SetHeadWithTimestamp(compat.RewindToTime)
if compatErr != nil {
log.Warn("Rewinding chain to upgrade configuration", "err", compatErr)
if compatErr.RewindToTime > 0 {
bc.SetHeadWithTimestamp(compatErr.RewindToTime)
} else {
bc.SetHead(compat.RewindToBlock)
bc.SetHead(compatErr.RewindToBlock)
}
rawdb.WriteChainConfig(db, genesisHash, chainConfig)
}
// Start tx indexer if it's enabled.
if txLookupLimit != nil {
bc.txIndexer = newTxIndexer(*txLookupLimit, bc)
@ -540,21 +552,16 @@ func (bc *BlockChain) loadLastState() error {
var (
currentSnapBlock = bc.CurrentSnapBlock()
currentFinalBlock = bc.CurrentFinalBlock()
headerTd = bc.GetTd(headHeader.Hash(), headHeader.Number.Uint64())
blockTd = bc.GetTd(headBlock.Hash(), headBlock.NumberU64())
)
if headHeader.Hash() != headBlock.Hash() {
log.Info("Loaded most recent local header", "number", headHeader.Number, "hash", headHeader.Hash(), "td", headerTd, "age", common.PrettyAge(time.Unix(int64(headHeader.Time), 0)))
log.Info("Loaded most recent local header", "number", headHeader.Number, "hash", headHeader.Hash(), "age", common.PrettyAge(time.Unix(int64(headHeader.Time), 0)))
}
log.Info("Loaded most recent local block", "number", headBlock.Number(), "hash", headBlock.Hash(), "td", blockTd, "age", common.PrettyAge(time.Unix(int64(headBlock.Time()), 0)))
log.Info("Loaded most recent local block", "number", headBlock.Number(), "hash", headBlock.Hash(), "age", common.PrettyAge(time.Unix(int64(headBlock.Time()), 0)))
if headBlock.Hash() != currentSnapBlock.Hash() {
snapTd := bc.GetTd(currentSnapBlock.Hash(), currentSnapBlock.Number.Uint64())
log.Info("Loaded most recent local snap block", "number", currentSnapBlock.Number, "hash", currentSnapBlock.Hash(), "td", snapTd, "age", common.PrettyAge(time.Unix(int64(currentSnapBlock.Time), 0)))
log.Info("Loaded most recent local snap block", "number", currentSnapBlock.Number, "hash", currentSnapBlock.Hash(), "age", common.PrettyAge(time.Unix(int64(currentSnapBlock.Time), 0)))
}
if currentFinalBlock != nil {
finalTd := bc.GetTd(currentFinalBlock.Hash(), currentFinalBlock.Number.Uint64())
log.Info("Loaded most recent local finalized block", "number", currentFinalBlock.Number, "hash", currentFinalBlock.Hash(), "td", finalTd, "age", common.PrettyAge(time.Unix(int64(currentFinalBlock.Time), 0)))
log.Info("Loaded most recent local finalized block", "number", currentFinalBlock.Number, "hash", currentFinalBlock.Hash(), "age", common.PrettyAge(time.Unix(int64(currentFinalBlock.Time), 0)))
}
if pivot := rawdb.ReadLastPivotNumber(bc.db); pivot != nil {
log.Info("Loaded last snap-sync pivot marker", "number", *pivot)
@ -986,7 +993,6 @@ func (bc *BlockChain) ResetWithGenesisBlock(genesis *types.Block) error {
// Prepare the genesis block and reinitialise the chain
batch := bc.db.NewBatch()
rawdb.WriteTd(batch, genesis.Hash(), genesis.NumberU64(), genesis.Difficulty())
rawdb.WriteBlock(batch, genesis)
if err := batch.Write(); err != nil {
log.Crit("Failed to write genesis block", "err", err)
@ -1259,8 +1265,7 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
// Ensure genesis is in ancients.
if first.NumberU64() == 1 {
if frozen, _ := bc.db.Ancients(); frozen == 0 {
td := bc.genesisBlock.Difficulty()
writeSize, err := rawdb.WriteAncientBlocks(bc.db, []*types.Block{bc.genesisBlock}, []types.Receipts{nil}, td)
writeSize, err := rawdb.WriteAncientBlocks(bc.db, []*types.Block{bc.genesisBlock}, []types.Receipts{nil})
if err != nil {
log.Error("Error writing genesis to ancients", "err", err)
return 0, err
@ -1275,10 +1280,8 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
if !bc.HasHeader(last.Hash(), last.NumberU64()) {
return 0, fmt.Errorf("containing header #%d [%x..] unknown", last.Number(), last.Hash().Bytes()[:4])
}
// Write all chain data to ancients.
td := bc.GetTd(first.Hash(), first.NumberU64())
writeSize, err := rawdb.WriteAncientBlocks(bc.db, blockChain, receiptChain, td)
writeSize, err := rawdb.WriteAncientBlocks(bc.db, blockChain, receiptChain)
if err != nil {
log.Error("Error importing chain data to ancients", "err", err)
return 0, err
@ -1418,12 +1421,11 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
// writeBlockWithoutState writes only the block and its metadata to the database,
// but does not write any state. This is used to construct competing side forks
// up to the point where they exceed the canonical total difficulty.
func (bc *BlockChain) writeBlockWithoutState(block *types.Block, td *big.Int) (err error) {
func (bc *BlockChain) writeBlockWithoutState(block *types.Block) (err error) {
if bc.insertStopped() {
return errInsertionInterrupted
}
batch := bc.db.NewBatch()
rawdb.WriteTd(batch, block.Hash(), block.NumberU64(), td)
rawdb.WriteBlock(batch, block)
if err := batch.Write(); err != nil {
log.Crit("Failed to write block into disk", "err", err)
@ -1447,20 +1449,14 @@ func (bc *BlockChain) writeKnownBlock(block *types.Block) error {
// writeBlockWithState writes block, metadata and corresponding state data to the
// database.
func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types.Receipt, statedb *state.StateDB) error {
// Calculate the total difficulty of the block
ptd := bc.GetTd(block.ParentHash(), block.NumberU64()-1)
if ptd == nil {
if !bc.HasHeader(block.ParentHash(), block.NumberU64()-1) {
return consensus.ErrUnknownAncestor
}
// Make sure no inconsistent state is leaked during insertion
externTd := new(big.Int).Add(block.Difficulty(), ptd)
// Irrelevant of the canonical status, write the block itself to the database.
//
// Note all the components of block(td, hash->number map, header, body, receipts)
// Note all the components of block(hash->number map, header, body, receipts)
// should be written atomically. BlockBatch is used for containing all components.
blockBatch := bc.db.NewBatch()
rawdb.WriteTd(blockBatch, block.Hash(), block.NumberU64(), externTd)
rawdb.WriteBlock(blockBatch, block)
rawdb.WriteReceipts(blockBatch, block.Hash(), block.NumberU64(), receipts)
rawdb.WritePreimages(blockBatch, statedb.Preimages())
@ -1468,7 +1464,7 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types.
log.Crit("Failed to write block into disk", "err", err)
}
// Commit all cached state changes into underlying memory database.
root, err := statedb.Commit(block.NumberU64(), bc.chainConfig.IsEIP158(block.Number()))
root, err := statedb.Commit(block.NumberU64(), bc.chainConfig.IsEIP158(block.Number()), bc.chainConfig.IsCancun(block.Number(), block.Time()))
if err != nil {
return err
}
@ -1616,7 +1612,7 @@ func (bc *BlockChain) insertChain(chain types.Blocks, setHead bool, makeWitness
return nil, 0, nil
}
// Start a parallel signature recovery (signer will fluke on fork transition, minimal perf loss)
SenderCacher.RecoverFromBlocks(types.MakeSigner(bc.chainConfig, chain[0].Number(), chain[0].Time()), chain)
SenderCacher().RecoverFromBlocks(types.MakeSigner(bc.chainConfig, chain[0].Number(), chain[0].Time()), chain)
var (
stats = insertStats{startTime: mclock.Now()}
@ -1754,7 +1750,6 @@ func (bc *BlockChain) insertChain(chain types.Blocks, setHead bool, makeWitness
if bc.logger != nil && bc.logger.OnSkippedBlock != nil {
bc.logger.OnSkippedBlock(tracing.BlockEvent{
Block: block,
TD: bc.GetTd(block.ParentHash(), block.NumberU64()-1),
Finalized: bc.CurrentFinalBlock(),
Safe: bc.CurrentSafeBlock(),
})
@ -1879,10 +1874,8 @@ type blockProcessingResult struct {
// it writes the block and associated state to database.
func (bc *BlockChain) processBlock(block *types.Block, statedb *state.StateDB, start time.Time, setHead bool) (_ *blockProcessingResult, blockEndErr error) {
if bc.logger != nil && bc.logger.OnBlockStart != nil {
td := bc.GetTd(block.ParentHash(), block.NumberU64()-1)
bc.logger.OnBlockStart(tracing.BlockEvent{
Block: block,
TD: td,
Finalized: bc.CurrentFinalBlock(),
Safe: bc.CurrentSafeBlock(),
})
@ -1992,10 +1985,8 @@ func (bc *BlockChain) processBlock(block *types.Block, statedb *state.StateDB, s
// switch over to the new chain if the TD exceeded the current chain.
// insertSideChain is only used pre-merge.
func (bc *BlockChain) insertSideChain(block *types.Block, it *insertIterator, makeWitness bool) (*stateless.Witness, int, error) {
var (
externTd *big.Int
current = bc.CurrentBlock()
)
var current = bc.CurrentBlock()
// The first sidechain block error is already verified to be ErrPrunedAncestor.
// Since we don't import them here, we expect ErrUnknownAncestor for the remaining
// ones. Any other errors means that the block is invalid, and should not be written
@ -2007,11 +1998,6 @@ func (bc *BlockChain) insertSideChain(block *types.Block, it *insertIterator, ma
canonical := bc.GetBlockByNumber(number)
if canonical != nil && canonical.Hash() == block.Hash() {
// Not a sidechain block, this is a re-import of a canon block which has it's state pruned
// Collect the TD of the block. Since we know it's a canon one,
// we can get it directly, and not (like further below) use
// the parent and then add the block on top
externTd = bc.GetTd(block.Hash(), block.NumberU64())
continue
}
if canonical != nil && canonical.Root() == block.Root() {
@ -2030,14 +2016,9 @@ func (bc *BlockChain) insertSideChain(block *types.Block, it *insertIterator, ma
return nil, it.index, errors.New("sidechain ghost-state attack")
}
}
if externTd == nil {
externTd = bc.GetTd(block.ParentHash(), block.NumberU64()-1)
}
externTd = new(big.Int).Add(externTd, block.Difficulty())
if !bc.HasBlock(block.Hash(), block.NumberU64()) {
start := time.Now()
if err := bc.writeBlockWithoutState(block, externTd); err != nil {
if err := bc.writeBlockWithoutState(block); err != nil {
return nil, it.index, err
}
log.Debug("Injected sidechain block", "number", block.Number(), "hash", block.Hash(),
@ -2158,9 +2139,8 @@ func (bc *BlockChain) recoverAncestors(block *types.Block, makeWitness bool) (co
// processing of a block. These logs are later announced as deleted or reborn.
func (bc *BlockChain) collectLogs(b *types.Block, removed bool) []*types.Log {
var blobGasPrice *big.Int
excessBlobGas := b.ExcessBlobGas()
if excessBlobGas != nil {
blobGasPrice = eip4844.CalcBlobFee(*excessBlobGas)
if b.ExcessBlobGas() != nil {
blobGasPrice = eip4844.CalcBlobFee(bc.chainConfig, b.Header())
}
receipts := rawdb.ReadRawReceipts(bc.db, b.Hash(), b.NumberU64())
if err := receipts.DeriveFields(bc.chainConfig, b.Hash(), b.NumberU64(), b.Time(), b.BaseFee(), blobGasPrice, b.Transactions()); err != nil {

7
core/blockchain_reader.go
View file

@ -18,7 +18,6 @@ package core
import (
"errors"
"math/big"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/consensus"
@ -307,12 +306,6 @@ func (bc *BlockChain) GetTransactionLookup(hash common.Hash) (*rawdb.LegacyTxLoo
return lookup, tx, nil
}
// GetTd retrieves a block's total difficulty in the canonical chain from the
// database by hash and number, caching it if found.
func (bc *BlockChain) GetTd(hash common.Hash, number uint64) *big.Int {
return bc.hc.GetTd(hash, number)
}
// HasState checks if state trie is fully present in the database or not.
func (bc *BlockChain) HasState(hash common.Hash) bool {
_, err := bc.statedb.OpenTrie(hash)

125
core/blockchain_test.go
View file

@ -88,7 +88,7 @@ func newGwei(n int64) *big.Int {
}
// Test fork of length N starting from block i
func testFork(t *testing.T, blockchain *BlockChain, i, n int, full bool, comparator func(td1, td2 *big.Int), scheme string) {
func testFork(t *testing.T, blockchain *BlockChain, i, n int, full bool, scheme string) {
// Copy old chain up to #i into a new db
genDb, _, blockchain2, err := newCanonical(ethash.NewFaker(), i, full, scheme)
if err != nil {
@ -125,27 +125,15 @@ func testFork(t *testing.T, blockchain *BlockChain, i, n int, full bool, compara
}
}
// Sanity check that the forked chain can be imported into the original
var tdPre, tdPost *big.Int
if full {
cur := blockchain.CurrentBlock()
tdPre = blockchain.GetTd(cur.Hash(), cur.Number.Uint64())
if err := testBlockChainImport(blockChainB, blockchain); err != nil {
t.Fatalf("failed to import forked block chain: %v", err)
}
last := blockChainB[len(blockChainB)-1]
tdPost = blockchain.GetTd(last.Hash(), last.NumberU64())
} else {
cur := blockchain.CurrentHeader()
tdPre = blockchain.GetTd(cur.Hash(), cur.Number.Uint64())
if err := testHeaderChainImport(headerChainB, blockchain); err != nil {
t.Fatalf("failed to import forked header chain: %v", err)
}
last := headerChainB[len(headerChainB)-1]
tdPost = blockchain.GetTd(last.Hash(), last.Number.Uint64())
}
// Compare the total difficulties of the chains
comparator(tdPre, tdPost)
}
// testBlockChainImport tries to process a chain of blocks, writing them into
@ -179,9 +167,8 @@ func testBlockChainImport(chain types.Blocks, blockchain *BlockChain) error {
}
blockchain.chainmu.MustLock()
rawdb.WriteTd(blockchain.db, block.Hash(), block.NumberU64(), new(big.Int).Add(block.Difficulty(), blockchain.GetTd(block.ParentHash(), block.NumberU64()-1)))
rawdb.WriteBlock(blockchain.db, block)
statedb.Commit(block.NumberU64(), false)
statedb.Commit(block.NumberU64(), false, false)
blockchain.chainmu.Unlock()
}
return nil
@ -197,7 +184,6 @@ func testHeaderChainImport(chain []*types.Header, blockchain *BlockChain) error
}
// Manually insert the header into the database, but don't reorganise (allows subsequent testing)
blockchain.chainmu.MustLock()
rawdb.WriteTd(blockchain.db, header.Hash(), header.Number.Uint64(), new(big.Int).Add(header.Difficulty, blockchain.GetTd(header.ParentHash, header.Number.Uint64()-1)))
rawdb.WriteHeader(blockchain.db, header)
blockchain.chainmu.Unlock()
}
@ -294,17 +280,11 @@ func testExtendCanonical(t *testing.T, full bool, scheme string) {
}
defer processor.Stop()
// Define the difficulty comparator
better := func(td1, td2 *big.Int) {
if td2.Cmp(td1) <= 0 {
t.Errorf("total difficulty mismatch: have %v, expected more than %v", td2, td1)
}
}
// Start fork from current height
testFork(t, processor, length, 1, full, better, scheme)
testFork(t, processor, length, 2, full, better, scheme)
testFork(t, processor, length, 5, full, better, scheme)
testFork(t, processor, length, 10, full, better, scheme)
testFork(t, processor, length, 1, full, scheme)
testFork(t, processor, length, 2, full, scheme)
testFork(t, processor, length, 5, full, scheme)
testFork(t, processor, length, 10, full, scheme)
}
// Tests that given a starting canonical chain of a given size, it can be extended
@ -353,19 +333,13 @@ func testShorterFork(t *testing.T, full bool, scheme string) {
}
defer processor.Stop()
// Define the difficulty comparator
worse := func(td1, td2 *big.Int) {
if td2.Cmp(td1) >= 0 {
t.Errorf("total difficulty mismatch: have %v, expected less than %v", td2, td1)
}
}
// Sum of numbers must be less than `length` for this to be a shorter fork
testFork(t, processor, 0, 3, full, worse, scheme)
testFork(t, processor, 0, 7, full, worse, scheme)
testFork(t, processor, 1, 1, full, worse, scheme)
testFork(t, processor, 1, 7, full, worse, scheme)
testFork(t, processor, 5, 3, full, worse, scheme)
testFork(t, processor, 5, 4, full, worse, scheme)
testFork(t, processor, 0, 3, full, scheme)
testFork(t, processor, 0, 7, full, scheme)
testFork(t, processor, 1, 1, full, scheme)
testFork(t, processor, 1, 7, full, scheme)
testFork(t, processor, 5, 3, full, scheme)
testFork(t, processor, 5, 4, full, scheme)
}
// Tests that given a starting canonical chain of a given size, creating shorter
@ -476,19 +450,13 @@ func testEqualFork(t *testing.T, full bool, scheme string) {
}
defer processor.Stop()
// Define the difficulty comparator
equal := func(td1, td2 *big.Int) {
if td2.Cmp(td1) != 0 {
t.Errorf("total difficulty mismatch: have %v, want %v", td2, td1)
}
}
// Sum of numbers must be equal to `length` for this to be an equal fork
testFork(t, processor, 0, 10, full, equal, scheme)
testFork(t, processor, 1, 9, full, equal, scheme)
testFork(t, processor, 2, 8, full, equal, scheme)
testFork(t, processor, 5, 5, full, equal, scheme)
testFork(t, processor, 6, 4, full, equal, scheme)
testFork(t, processor, 9, 1, full, equal, scheme)
testFork(t, processor, 0, 10, full, scheme)
testFork(t, processor, 1, 9, full, scheme)
testFork(t, processor, 2, 8, full, scheme)
testFork(t, processor, 5, 5, full, scheme)
testFork(t, processor, 6, 4, full, scheme)
testFork(t, processor, 9, 1, full, scheme)
}
// Tests that given a starting canonical chain of a given size, creating equal
@ -647,19 +615,6 @@ func testReorg(t *testing.T, first, second []int64, td int64, full bool, scheme
}
}
}
// Make sure the chain total difficulty is the correct one
want := new(big.Int).Add(blockchain.genesisBlock.Difficulty(), big.NewInt(td))
if full {
cur := blockchain.CurrentBlock()
if have := blockchain.GetTd(cur.Hash(), cur.Number.Uint64()); have.Cmp(want) != 0 {
t.Errorf("total difficulty mismatch: have %v, want %v", have, want)
}
} else {
cur := blockchain.CurrentHeader()
if have := blockchain.GetTd(cur.Hash(), cur.Number.Uint64()); have.Cmp(want) != 0 {
t.Errorf("total difficulty mismatch: have %v, want %v", have, want)
}
}
}
// Tests chain insertions in the face of one entity containing an invalid nonce.
@ -809,12 +764,6 @@ func testFastVsFullChains(t *testing.T, scheme string) {
for i := 0; i < len(blocks); i++ {
num, hash, time := blocks[i].NumberU64(), blocks[i].Hash(), blocks[i].Time()
if ftd, atd := fast.GetTd(hash, num), archive.GetTd(hash, num); ftd.Cmp(atd) != 0 {
t.Errorf("block #%d [%x]: td mismatch: fastdb %v, archivedb %v", num, hash, ftd, atd)
}
if antd, artd := ancient.GetTd(hash, num), archive.GetTd(hash, num); antd.Cmp(artd) != 0 {
t.Errorf("block #%d [%x]: td mismatch: ancientdb %v, archivedb %v", num, hash, antd, artd)
}
if fheader, aheader := fast.GetHeaderByHash(hash), archive.GetHeaderByHash(hash); fheader.Hash() != aheader.Hash() {
t.Errorf("block #%d [%x]: header mismatch: fastdb %v, archivedb %v", num, hash, fheader, aheader)
}
@ -2535,7 +2484,7 @@ func testReorgToShorterRemovesCanonMappingHeaderChain(t *testing.T, scheme strin
}
// Benchmarks large blocks with value transfers to non-existing accounts
func benchmarkLargeNumberOfValueToNonexisting(b *testing.B, numTxs, numBlocks int, recipientFn func(uint64) common.Address, dataFn func(uint64) []byte) {
func benchmarkLargeNumberOfValueToNonexisting(b *testing.B, numTxs, numBlocks int, recipientFn func(uint64) common.Address) {
var (
signer = types.HomesteadSigner{}
testBankKey, _ = crypto.HexToECDSA("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291")
@ -2598,10 +2547,7 @@ func BenchmarkBlockChain_1x1000ValueTransferToNonexisting(b *testing.B) {
recipientFn := func(nonce uint64) common.Address {
return common.BigToAddress(new(big.Int).SetUint64(1337 + nonce))
}
dataFn := func(nonce uint64) []byte {
return nil
}
benchmarkLargeNumberOfValueToNonexisting(b, numTxs, numBlocks, recipientFn, dataFn)
benchmarkLargeNumberOfValueToNonexisting(b, numTxs, numBlocks, recipientFn)
}
func BenchmarkBlockChain_1x1000ValueTransferToExisting(b *testing.B) {
@ -2615,10 +2561,7 @@ func BenchmarkBlockChain_1x1000ValueTransferToExisting(b *testing.B) {
recipientFn := func(nonce uint64) common.Address {
return common.BigToAddress(new(big.Int).SetUint64(1337))
}
dataFn := func(nonce uint64) []byte {
return nil
}
benchmarkLargeNumberOfValueToNonexisting(b, numTxs, numBlocks, recipientFn, dataFn)
benchmarkLargeNumberOfValueToNonexisting(b, numTxs, numBlocks, recipientFn)
}
func BenchmarkBlockChain_1x1000Executions(b *testing.B) {
@ -2632,10 +2575,7 @@ func BenchmarkBlockChain_1x1000Executions(b *testing.B) {
recipientFn := func(nonce uint64) common.Address {
return common.BigToAddress(new(big.Int).SetUint64(0xc0de))
}
dataFn := func(nonce uint64) []byte {
return nil
}
benchmarkLargeNumberOfValueToNonexisting(b, numTxs, numBlocks, recipientFn, dataFn)
benchmarkLargeNumberOfValueToNonexisting(b, numTxs, numBlocks, recipientFn)
}
// Tests that importing a some old blocks, where all blocks are before the
@ -4047,7 +3987,7 @@ func TestEIP3651(t *testing.T) {
var (
aa = common.HexToAddress("0x000000000000000000000000000000000000aaaa")
bb = common.HexToAddress("0x000000000000000000000000000000000000bbbb")
engine = beacon.NewFaker()
engine = beacon.New(ethash.NewFaker())
// A sender who makes transactions, has some funds
key1, _ = crypto.HexToECDSA("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291")
@ -4163,7 +4103,7 @@ func TestPragueRequests(t *testing.T) {
addr1 = crypto.PubkeyToAddress(key1.PublicKey)
config = *params.MergedTestChainConfig
signer = types.LatestSigner(&config)
engine = beacon.NewFaker()
engine = beacon.New(ethash.NewFaker())
)
gspec := &Genesis{
Config: &config,
@ -4241,7 +4181,7 @@ func TestEIP7702(t *testing.T) {
var (
config = *params.MergedTestChainConfig
signer = types.LatestSigner(&config)
engine = beacon.NewFaker()
engine = beacon.New(ethash.NewFaker())
key1, _ = crypto.HexToECDSA("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291")
key2, _ = crypto.HexToECDSA("8a1f9a8f95be41cd7ccb6168179afb4504aefe388d1e14474d32c45c72ce7b7a")
addr1 = crypto.PubkeyToAddress(key1.PublicKey)
@ -4273,27 +4213,26 @@ func TestEIP7702(t *testing.T) {
// 1. tx -> addr1 which is delegated to 0xaaaa
// 2. addr1:0xaaaa calls into addr2:0xbbbb
// 3. addr2:0xbbbb writes to storage
auth1, _ := types.SignAuth(types.Authorization{
ChainID: gspec.Config.ChainID.Uint64(),
auth1, _ := types.SignSetCode(key1, types.SetCodeAuthorization{
ChainID: *uint256.MustFromBig(gspec.Config.ChainID),
Address: aa,
Nonce: 1,
}, key1)
auth2, _ := types.SignAuth(types.Authorization{
ChainID: 0,
})
auth2, _ := types.SignSetCode(key2, types.SetCodeAuthorization{
Address: bb,
Nonce: 0,
}, key2)
})
_, blocks, _ := GenerateChainWithGenesis(gspec, engine, 1, func(i int, b *BlockGen) {
b.SetCoinbase(aa)
txdata := &types.SetCodeTx{
ChainID: gspec.Config.ChainID.Uint64(),
ChainID: uint256.MustFromBig(gspec.Config.ChainID),
Nonce: 0,
To: addr1,
Gas: 500000,
GasFeeCap: uint256.MustFromBig(newGwei(5)),
GasTipCap: uint256.NewInt(2),
AuthList: []types.Authorization{auth1, auth2},
AuthList: []types.SetCodeAuthorization{auth1, auth2},
}
tx := types.MustSignNewTx(key1, signer, txdata)
b.AddTx(tx)

112
core/chain_makers.go
View file

@ -122,6 +122,11 @@ func (b *BlockGen) addTx(bc *BlockChain, vmConfig vm.Config, tx *types.Transacti
if err != nil {
panic(err)
}
// Merge the tx-local access event into the "block-local" one, in order to collect
// all values, so that the witness can be built.
if b.statedb.GetTrie().IsVerkle() {
b.statedb.AccessEvents().Merge(evm.AccessEvents)
}
b.txs = append(b.txs, tx)
b.receipts = append(b.receipts, receipt)
if b.header.BlobGasUsed != nil {
@ -138,7 +143,9 @@ func (b *BlockGen) addTx(bc *BlockChain, vmConfig vm.Config, tx *types.Transacti
// instruction will panic during execution if it attempts to access a block number outside
// of the range created by GenerateChain.
func (b *BlockGen) AddTx(tx *types.Transaction) {
b.addTx(nil, vm.Config{}, tx)
// Wrap the chain config in an empty BlockChain object to satisfy ChainContext.
bc := &BlockChain{chainConfig: b.cm.config}
b.addTx(bc, vm.Config{}, tx)
}
// AddTxWithChain adds a transaction to the generated block. If no coinbase has
@ -293,6 +300,41 @@ func (b *BlockGen) OffsetTime(seconds int64) {
b.header.Difficulty = b.engine.CalcDifficulty(b.cm, b.header.Time, b.parent.Header())
}
// ConsensusLayerRequests returns the EIP-7685 requests which have accumulated so far.
func (b *BlockGen) ConsensusLayerRequests() [][]byte {
return b.collectRequests(true)
}
func (b *BlockGen) collectRequests(readonly bool) (requests [][]byte) {
statedb := b.statedb
if readonly {
// The system contracts clear themselves on a system-initiated read.
// When reading the requests mid-block, we don't want this behavior, so fork
// off the statedb before executing the system calls.
statedb = statedb.Copy()
}
if b.cm.config.IsPrague(b.header.Number, b.header.Time) {
requests = [][]byte{}
// EIP-6110 deposits
var blockLogs []*types.Log
for _, r := range b.receipts {
blockLogs = append(blockLogs, r.Logs...)
}
if err := ParseDepositLogs(&requests, blockLogs, b.cm.config); err != nil {
panic(fmt.Sprintf("failed to parse deposit log: %v", err))
}
// create EVM for system calls
blockContext := NewEVMBlockContext(b.header, b.cm, &b.header.Coinbase)
evm := vm.NewEVM(blockContext, statedb, b.cm.config, vm.Config{})
// EIP-7002
ProcessWithdrawalQueue(&requests, evm)
// EIP-7251
ProcessConsolidationQueue(&requests, evm)
}
return requests
}
// GenerateChain creates a chain of n blocks. The first block's
// parent will be the provided parent. db is used to store
// intermediate states and should contain the parent's state trie.
@ -330,6 +372,7 @@ func GenerateChain(config *params.ChainConfig, parent *types.Block, engine conse
b.header.Difficulty = big.NewInt(0)
}
}
// Mutate the state and block according to any hard-fork specs
if daoBlock := config.DAOForkBlock; daoBlock != nil {
limit := new(big.Int).Add(daoBlock, params.DAOForkExtraRange)
@ -342,30 +385,21 @@ func GenerateChain(config *params.ChainConfig, parent *types.Block, engine conse
if config.DAOForkSupport && config.DAOForkBlock != nil && config.DAOForkBlock.Cmp(b.header.Number) == 0 {
misc.ApplyDAOHardFork(statedb)
}
if config.IsPrague(b.header.Number, b.header.Time) || config.IsVerkle(b.header.Number, b.header.Time) {
// EIP-2935
blockContext := NewEVMBlockContext(b.header, cm, &b.header.Coinbase)
blockContext.Random = &common.Hash{} // enable post-merge instruction set
evm := vm.NewEVM(blockContext, statedb, cm.config, vm.Config{})
ProcessParentBlockHash(b.header.ParentHash, evm)
}
// Execute any user modifications to the block
if gen != nil {
gen(i, b)
}
var requests [][]byte
if config.IsPrague(b.header.Number, b.header.Time) {
requests = [][]byte{}
// EIP-6110 deposits
var blockLogs []*types.Log
for _, r := range b.receipts {
blockLogs = append(blockLogs, r.Logs...)
}
if err := ParseDepositLogs(&requests, blockLogs, config); err != nil {
panic(fmt.Sprintf("failed to parse deposit log: %v", err))
}
// create EVM for system calls
blockContext := NewEVMBlockContext(b.header, cm, &b.header.Coinbase)
evm := vm.NewEVM(blockContext, statedb, cm.config, vm.Config{})
// EIP-7002
ProcessWithdrawalQueue(&requests, evm)
// EIP-7251
ProcessConsolidationQueue(&requests, evm)
}
requests := b.collectRequests(false)
if requests != nil {
reqHash := types.CalcRequestsHash(requests)
b.header.RequestsHash = &reqHash
@ -378,7 +412,7 @@ func GenerateChain(config *params.ChainConfig, parent *types.Block, engine conse
}
// Write state changes to db
root, err := statedb.Commit(b.header.Number.Uint64(), config.IsEIP158(b.header.Number))
root, err := statedb.Commit(b.header.Number.Uint64(), config.IsEIP158(b.header.Number), config.IsCancun(b.header.Number, b.header.Time))
if err != nil {
panic(fmt.Sprintf("state write error: %v", err))
}
@ -413,7 +447,7 @@ func GenerateChain(config *params.ChainConfig, parent *types.Block, engine conse
}
var blobGasPrice *big.Int
if block.ExcessBlobGas() != nil {
blobGasPrice = eip4844.CalcBlobFee(*block.ExcessBlobGas())
blobGasPrice = eip4844.CalcBlobFee(cm.config, block.Header())
}
if err := receipts.DeriveFields(config, block.Hash(), block.NumberU64(), block.Time(), block.BaseFee(), blobGasPrice, txs); err != nil {
panic(err)
@ -460,13 +494,11 @@ func GenerateVerkleChain(config *params.ChainConfig, parent *types.Block, engine
// Save pre state for proof generation
// preState := statedb.Copy()
// Pre-execution system calls.
if config.IsPrague(b.header.Number, b.header.Time) {
// EIP-2935
blockContext := NewEVMBlockContext(b.header, cm, &b.header.Coinbase)
evm := vm.NewEVM(blockContext, statedb, cm.config, vm.Config{})
ProcessParentBlockHash(b.header.ParentHash, evm)
}
// EIP-2935 / 7709
blockContext := NewEVMBlockContext(b.header, cm, &b.header.Coinbase)
blockContext.Random = &common.Hash{} // enable post-merge instruction set
evm := vm.NewEVM(blockContext, statedb, cm.config, vm.Config{})
ProcessParentBlockHash(b.header.ParentHash, evm)
// Execute any user modifications to the block.
if gen != nil {
@ -483,7 +515,7 @@ func GenerateVerkleChain(config *params.ChainConfig, parent *types.Block, engine
}
// Write state changes to DB.
root, err := statedb.Commit(b.header.Number.Uint64(), config.IsEIP158(b.header.Number))
root, err := statedb.Commit(b.header.Number.Uint64(), config.IsEIP158(b.header.Number), config.IsCancun(b.header.Number, b.header.Time))
if err != nil {
panic(fmt.Sprintf("state write error: %v", err))
}
@ -518,7 +550,7 @@ func GenerateVerkleChain(config *params.ChainConfig, parent *types.Block, engine
}
var blobGasPrice *big.Int
if block.ExcessBlobGas() != nil {
blobGasPrice = eip4844.CalcBlobFee(*block.ExcessBlobGas())
blobGasPrice = eip4844.CalcBlobFee(cm.config, block.Header())
}
if err := receipts.DeriveFields(config, block.Hash(), block.NumberU64(), block.Time(), block.BaseFee(), blobGasPrice, txs); err != nil {
panic(err)
@ -534,7 +566,7 @@ func GenerateVerkleChain(config *params.ChainConfig, parent *types.Block, engine
return cm.chain, cm.receipts, proofs, keyvals
}
func GenerateVerkleChainWithGenesis(genesis *Genesis, engine consensus.Engine, n int, gen func(int, *BlockGen)) (ethdb.Database, []*types.Block, []types.Receipts, []*verkle.VerkleProof, []verkle.StateDiff) {
func GenerateVerkleChainWithGenesis(genesis *Genesis, engine consensus.Engine, n int, gen func(int, *BlockGen)) (common.Hash, ethdb.Database, []*types.Block, []types.Receipts, []*verkle.VerkleProof, []verkle.StateDiff) {
db := rawdb.NewMemoryDatabase()
cacheConfig := DefaultCacheConfigWithScheme(rawdb.PathScheme)
cacheConfig.SnapshotLimit = 0
@ -545,7 +577,7 @@ func GenerateVerkleChainWithGenesis(genesis *Genesis, engine consensus.Engine, n
panic(err)
}
blocks, receipts, proofs, keyvals := GenerateVerkleChain(genesis.Config, genesisBlock, engine, db, triedb, n, gen)
return db, blocks, receipts, proofs, keyvals
return genesisBlock.Hash(), db, blocks, receipts, proofs, keyvals
}
func (cm *chainMaker) makeHeader(parent *types.Block, state *state.StateDB, engine consensus.Engine) *types.Header {
@ -568,15 +600,7 @@ func (cm *chainMaker) makeHeader(parent *types.Block, state *state.StateDB, engi
}
}
if cm.config.IsCancun(header.Number, header.Time) {
var (
parentExcessBlobGas uint64
parentBlobGasUsed uint64
)
if parent.ExcessBlobGas() != nil {
parentExcessBlobGas = *parent.ExcessBlobGas()
parentBlobGasUsed = *parent.BlobGasUsed()
}
excessBlobGas := eip4844.CalcExcessBlobGas(parentExcessBlobGas, parentBlobGasUsed)
excessBlobGas := eip4844.CalcExcessBlobGas(cm.config, parent.Header())
header.ExcessBlobGas = &excessBlobGas
header.BlobGasUsed = new(uint64)
header.ParentBeaconRoot = new(common.Hash)
@ -699,7 +723,3 @@ func (cm *chainMaker) GetHeader(hash common.Hash, number uint64) *types.Header {
func (cm *chainMaker) GetBlock(hash common.Hash, number uint64) *types.Block {
return cm.blockByNumber(number)
}
func (cm *chainMaker) GetTd(hash common.Hash, number uint64) *big.Int {
return nil // not supported
}

11
core/chain_makers_test.go
View file

@ -42,7 +42,7 @@ func TestGeneratePOSChain(t *testing.T) {
aa = common.Address{0xaa}
bb = common.Address{0xbb}
funds = big.NewInt(0).Mul(big.NewInt(1337), big.NewInt(params.Ether))
config = *params.AllEthashProtocolChanges
config = *params.MergedTestChainConfig
gspec = &Genesis{
Config: &config,
Alloc: types.GenesisAlloc{
@ -57,10 +57,6 @@ func TestGeneratePOSChain(t *testing.T) {
db = rawdb.NewMemoryDatabase()
)
config.TerminalTotalDifficulty = common.Big0
config.ShanghaiTime = u64(0)
config.CancunTime = u64(0)
// init 0xaa with some storage elements
storage := make(map[common.Hash]common.Hash)
storage[common.Hash{0x00}] = common.Hash{0x00}
@ -80,8 +76,9 @@ func TestGeneratePOSChain(t *testing.T) {
Code: common.Hex2Bytes("600154600354"),
}
genesis := gspec.MustCommit(gendb, triedb.NewDatabase(gendb, triedb.HashDefaults))
engine := beacon.New(ethash.NewFaker())
genchain, genreceipts := GenerateChain(gspec.Config, genesis, beacon.NewFaker(), gendb, 4, func(i int, gen *BlockGen) {
genchain, genreceipts := GenerateChain(gspec.Config, genesis, engine, gendb, 4, func(i int, gen *BlockGen) {
gen.SetParentBeaconRoot(common.Hash{byte(i + 1)})
// Add value transfer tx.
@ -122,7 +119,7 @@ func TestGeneratePOSChain(t *testing.T) {
})
// Import the chain. This runs all block validation rules.
blockchain, _ := NewBlockChain(db, nil, gspec, nil, beacon.NewFaker(), vm.Config{}, nil)
blockchain, _ := NewBlockChain(db, nil, gspec, nil, engine, vm.Config{}, nil)
defer blockchain.Stop()
if i, err := blockchain.InsertChain(genchain); err != nil {

4
core/error.go
View file

@ -80,6 +80,10 @@ var (
// than required to start the invocation.
ErrIntrinsicGas = errors.New("intrinsic gas too low")
// ErrFloorDataGas is returned if the transaction is specified to use less gas
// than required for the data floor cost.
ErrFloorDataGas = errors.New("insufficient gas for floor data gas cost")
// ErrTxTypeNotSupported is returned if a transaction is not supported in the
// current network configuration.
ErrTxTypeNotSupported = types.ErrTxTypeNotSupported

6
core/evm.go
View file

@ -25,6 +25,7 @@ import (
"github.com/ethereum/go-ethereum/core/tracing"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/core/vm"
"github.com/ethereum/go-ethereum/params"
"github.com/holiman/uint256"
)
@ -36,6 +37,9 @@ type ChainContext interface {
// GetHeader returns the header corresponding to the hash/number argument pair.
GetHeader(common.Hash, uint64) *types.Header
// Config returns the chain's configuration.
Config() *params.ChainConfig
}
// NewEVMBlockContext creates a new context for use in the EVM.
@ -57,7 +61,7 @@ func NewEVMBlockContext(header *types.Header, chain ChainContext, author *common
baseFee = new(big.Int).Set(header.BaseFee)
}
if header.ExcessBlobGas != nil {
blobBaseFee = eip4844.CalcBlobFee(*header.ExcessBlobGas)
blobBaseFee = eip4844.CalcBlobFee(chain.Config(), header)
}
if header.Difficulty.Sign() == 0 {
random = &header.MixDigest

222
core/genesis.go
View file

@ -75,6 +75,19 @@ type Genesis struct {
BlobGasUsed *uint64 `json:"blobGasUsed"` // EIP-4844
}
// copy copies the genesis.
func (g *Genesis) copy() *Genesis {
if g != nil {
cpy := *g
if g.Config != nil {
conf := *g.Config
cpy.Config = &conf
}
return &cpy
}
return nil
}
func ReadGenesis(db ethdb.Database) (*Genesis, error) {
var genesis Genesis
stored := rawdb.ReadCanonicalHash(db, 0)
@ -127,8 +140,12 @@ func hashAlloc(ga *types.GenesisAlloc, isVerkle bool) (common.Hash, error) {
}
// Create an ephemeral in-memory database for computing hash,
// all the derived states will be discarded to not pollute disk.
emptyRoot := types.EmptyRootHash
if isVerkle {
emptyRoot = types.EmptyVerkleHash
}
db := rawdb.NewMemoryDatabase()
statedb, err := state.New(types.EmptyRootHash, state.NewDatabase(triedb.NewDatabase(db, config), nil))
statedb, err := state.New(emptyRoot, state.NewDatabase(triedb.NewDatabase(db, config), nil))
if err != nil {
return common.Hash{}, err
}
@ -142,13 +159,17 @@ func hashAlloc(ga *types.GenesisAlloc, isVerkle bool) (common.Hash, error) {
statedb.SetState(addr, key, value)
}
}
return statedb.Commit(0, false)
return statedb.Commit(0, false, false)
}
// flushAlloc is very similar with hash, but the main difference is all the
// generated states will be persisted into the given database.
func flushAlloc(ga *types.GenesisAlloc, triedb *triedb.Database) (common.Hash, error) {
statedb, err := state.New(types.EmptyRootHash, state.NewDatabase(triedb, nil))
emptyRoot := types.EmptyRootHash
if triedb.IsVerkle() {
emptyRoot = types.EmptyVerkleHash
}
statedb, err := state.New(emptyRoot, state.NewDatabase(triedb, nil))
if err != nil {
return common.Hash{}, err
}
@ -164,7 +185,7 @@ func flushAlloc(ga *types.GenesisAlloc, triedb *triedb.Database) (common.Hash, e
statedb.SetState(addr, key, value)
}
}
root, err := statedb.Commit(0, false)
root, err := statedb.Commit(0, false, false)
if err != nil {
return common.Hash{}, err
}
@ -239,6 +260,20 @@ type ChainOverrides struct {
OverrideVerkle *uint64
}
// apply applies the chain overrides on the supplied chain config.
func (o *ChainOverrides) apply(cfg *params.ChainConfig) error {
if o == nil || cfg == nil {
return nil
}
if o.OverrideCancun != nil {
cfg.CancunTime = o.OverrideCancun
}
if o.OverrideVerkle != nil {
cfg.VerkleTime = o.OverrideVerkle
}
return cfg.CheckConfigForkOrder()
}
// SetupGenesisBlock writes or updates the genesis block in db.
// The block that will be used is:
//
@ -250,109 +285,98 @@ type ChainOverrides struct {
// The stored chain configuration will be updated if it is compatible (i.e. does not
// specify a fork block below the local head block). In case of a conflict, the
// error is a *params.ConfigCompatError and the new, unwritten config is returned.
//
// The returned chain configuration is never nil.
func SetupGenesisBlock(db ethdb.Database, triedb *triedb.Database, genesis *Genesis) (*params.ChainConfig, common.Hash, error) {
func SetupGenesisBlock(db ethdb.Database, triedb *triedb.Database, genesis *Genesis) (*params.ChainConfig, common.Hash, *params.ConfigCompatError, error) {
return SetupGenesisBlockWithOverride(db, triedb, genesis, nil)
}
func SetupGenesisBlockWithOverride(db ethdb.Database, triedb *triedb.Database, genesis *Genesis, overrides *ChainOverrides) (*params.ChainConfig, common.Hash, error) {
func SetupGenesisBlockWithOverride(db ethdb.Database, triedb *triedb.Database, genesis *Genesis, overrides *ChainOverrides) (*params.ChainConfig, common.Hash, *params.ConfigCompatError, error) {
// Copy the genesis, so we can operate on a copy.
genesis = genesis.copy()
// Sanitize the supplied genesis, ensuring it has the associated chain
// config attached.
if genesis != nil && genesis.Config == nil {
return params.AllEthashProtocolChanges, common.Hash{}, errGenesisNoConfig
return nil, common.Hash{}, nil, errGenesisNoConfig
}
applyOverrides := func(config *params.ChainConfig) {
if config != nil {
if overrides != nil && overrides.OverrideCancun != nil {
config.CancunTime = overrides.OverrideCancun
}
if overrides != nil && overrides.OverrideVerkle != nil {
config.VerkleTime = overrides.OverrideVerkle
}
}
}
// Just commit the new block if there is no stored genesis block.
stored := rawdb.ReadCanonicalHash(db, 0)
if (stored == common.Hash{}) {
// Commit the genesis if the database is empty
ghash := rawdb.ReadCanonicalHash(db, 0)
if (ghash == common.Hash{}) {
if genesis == nil {
log.Info("Writing default main-net genesis block")
genesis = DefaultGenesisBlock()
} else {
log.Info("Writing custom genesis block")
}
if err := overrides.apply(genesis.Config); err != nil {
return nil, common.Hash{}, nil, err
}
applyOverrides(genesis.Config)
block, err := genesis.Commit(db, triedb)
if err != nil {
return genesis.Config, common.Hash{}, err
return nil, common.Hash{}, nil, err
}
return genesis.Config, block.Hash(), nil
return genesis.Config, block.Hash(), nil, nil
}
// The genesis block is present(perhaps in ancient database) while the
// state database is not initialized yet. It can happen that the node
// is initialized with an external ancient store. Commit genesis state
// in this case.
header := rawdb.ReadHeader(db, stored, 0)
if header.Root != types.EmptyRootHash && !triedb.Initialized(header.Root) {
// Commit the genesis if the genesis block exists in the ancient database
// but the key-value database is empty without initializing the genesis
// fields. This scenario can occur when the node is created from scratch
// with an existing ancient store.
storedCfg := rawdb.ReadChainConfig(db, ghash)
if storedCfg == nil {
// Ensure the stored genesis block matches with the given genesis. Private
// networks must explicitly specify the genesis in the config file, mainnet
// genesis will be used as default and the initialization will always fail.
if genesis == nil {
log.Info("Writing default main-net genesis block")
genesis = DefaultGenesisBlock()
} else {
log.Info("Writing custom genesis block")
}
applyOverrides(genesis.Config)
// Ensure the stored genesis matches with the given one.
hash := genesis.ToBlock().Hash()
if hash != stored {
return genesis.Config, hash, &GenesisMismatchError{stored, hash}
if err := overrides.apply(genesis.Config); err != nil {
return nil, common.Hash{}, nil, err
}
if hash := genesis.ToBlock().Hash(); hash != ghash {
return nil, common.Hash{}, nil, &GenesisMismatchError{ghash, hash}
}
block, err := genesis.Commit(db, triedb)
if err != nil {
return genesis.Config, hash, err
return nil, common.Hash{}, nil, err
}
return genesis.Config, block.Hash(), nil
return genesis.Config, block.Hash(), nil, nil
}
// Check whether the genesis block is already written.
// The genesis block has already been committed previously. Verify that the
// provided genesis with chain overrides matches the existing one, and update
// the stored chain config if necessary.
if genesis != nil {
applyOverrides(genesis.Config)
hash := genesis.ToBlock().Hash()
if hash != stored {
return genesis.Config, hash, &GenesisMismatchError{stored, hash}
if err := overrides.apply(genesis.Config); err != nil {
return nil, common.Hash{}, nil, err
}
if hash := genesis.ToBlock().Hash(); hash != ghash {
return nil, common.Hash{}, nil, &GenesisMismatchError{ghash, hash}
}
}
// Get the existing chain configuration.
newcfg := genesis.configOrDefault(stored)
applyOverrides(newcfg)
if err := newcfg.CheckConfigForkOrder(); err != nil {
return newcfg, common.Hash{}, err
}
storedcfg := rawdb.ReadChainConfig(db, stored)
if storedcfg == nil {
log.Warn("Found genesis block without chain config")
rawdb.WriteChainConfig(db, stored, newcfg)
return newcfg, stored, nil
}
storedData, _ := json.Marshal(storedcfg)
// Special case: if a private network is being used (no genesis and also no
// mainnet hash in the database), we must not apply the `configOrDefault`
// chain config as that would be AllProtocolChanges (applying any new fork
// on top of an existing private network genesis block). In that case, only
// apply the overrides.
if genesis == nil && stored != params.MainnetGenesisHash {
newcfg = storedcfg
applyOverrides(newcfg)
}
// Check config compatibility and write the config. Compatibility errors
// are returned to the caller unless we're already at block zero.
head := rawdb.ReadHeadHeader(db)
if head == nil {
return newcfg, stored, errors.New("missing head header")
return nil, common.Hash{}, nil, errors.New("missing head header")
}
compatErr := storedcfg.CheckCompatible(newcfg, head.Number.Uint64(), head.Time)
newCfg := genesis.chainConfigOrDefault(ghash, storedCfg)
// TODO(rjl493456442) better to define the comparator of chain config
// and short circuit if the chain config is not changed.
compatErr := storedCfg.CheckCompatible(newCfg, head.Number.Uint64(), head.Time)
if compatErr != nil && ((head.Number.Uint64() != 0 && compatErr.RewindToBlock != 0) || (head.Time != 0 && compatErr.RewindToTime != 0)) {
return newcfg, stored, compatErr
return newCfg, ghash, compatErr, nil
}
// Don't overwrite if the old is identical to the new
if newData, _ := json.Marshal(newcfg); !bytes.Equal(storedData, newData) {
rawdb.WriteChainConfig(db, stored, newcfg)
// Don't overwrite if the old is identical to the new. It's useful
// for the scenarios that database is opened in the read-only mode.
storedData, _ := json.Marshal(storedCfg)
if newData, _ := json.Marshal(newCfg); !bytes.Equal(storedData, newData) {
rawdb.WriteChainConfig(db, ghash, newCfg)
}
return newcfg, stored, nil
return newCfg, ghash, nil, nil
}
// LoadChainConfig loads the stored chain config if it is already present in
@ -388,7 +412,10 @@ func LoadChainConfig(db ethdb.Database, genesis *Genesis) (*params.ChainConfig,
return params.MainnetChainConfig, nil
}
func (g *Genesis) configOrDefault(ghash common.Hash) *params.ChainConfig {
// chainConfigOrDefault retrieves the attached chain configuration. If the genesis
// object is null, it returns the default chain configuration based on the given
// genesis hash, or the locally stored config if it's not a pre-defined network.
func (g *Genesis) chainConfigOrDefault(ghash common.Hash, stored *params.ChainConfig) *params.ChainConfig {
switch {
case g != nil:
return g.Config
@ -399,14 +426,14 @@ func (g *Genesis) configOrDefault(ghash common.Hash) *params.ChainConfig {
case ghash == params.SepoliaGenesisHash:
return params.SepoliaChainConfig
default:
return params.AllEthashProtocolChanges
return stored
}
}
// IsVerkle indicates whether the state is already stored in a verkle
// tree at genesis time.
func (g *Genesis) IsVerkle() bool {
return g.Config.IsVerkle(new(big.Int).SetUint64(g.Number), g.Timestamp)
return g.Config.IsVerkleGenesis()
}
// ToBlock returns the genesis block according to genesis specification.
@ -486,7 +513,7 @@ func (g *Genesis) Commit(db ethdb.Database, triedb *triedb.Database) (*types.Blo
}
config := g.Config
if config == nil {
config = params.AllEthashProtocolChanges
return nil, errors.New("invalid genesis without chain config")
}
if err := config.CheckConfigForkOrder(); err != nil {
return nil, err
@ -506,16 +533,16 @@ func (g *Genesis) Commit(db ethdb.Database, triedb *triedb.Database) (*types.Blo
if err != nil {
return nil, err
}
rawdb.WriteGenesisStateSpec(db, block.Hash(), blob)
rawdb.WriteTd(db, block.Hash(), block.NumberU64(), block.Difficulty())
rawdb.WriteBlock(db, block)
rawdb.WriteReceipts(db, block.Hash(), block.NumberU64(), nil)
rawdb.WriteCanonicalHash(db, block.Hash(), block.NumberU64())
rawdb.WriteHeadBlockHash(db, block.Hash())
rawdb.WriteHeadFastBlockHash(db, block.Hash())
rawdb.WriteHeadHeaderHash(db, block.Hash())
rawdb.WriteChainConfig(db, block.Hash(), config)
return block, nil
batch := db.NewBatch()
rawdb.WriteGenesisStateSpec(batch, block.Hash(), blob)
rawdb.WriteBlock(batch, block)
rawdb.WriteReceipts(batch, block.Hash(), block.NumberU64(), nil)
rawdb.WriteCanonicalHash(batch, block.Hash(), block.NumberU64())
rawdb.WriteHeadBlockHash(batch, block.Hash())
rawdb.WriteHeadFastBlockHash(batch, block.Hash())
rawdb.WriteHeadHeaderHash(batch, block.Hash())
rawdb.WriteChainConfig(batch, block.Hash(), config)
return block, batch.Write()
}
// MustCommit writes the genesis block and state to db, panicking on error.
@ -528,6 +555,29 @@ func (g *Genesis) MustCommit(db ethdb.Database, triedb *triedb.Database) *types.
return block
}
// EnableVerkleAtGenesis indicates whether the verkle fork should be activated
// at genesis. This is a temporary solution only for verkle devnet testing, where
// verkle fork is activated at genesis, and the configured activation date has
// already passed.
//
// In production networks (mainnet and public testnets), verkle activation always
// occurs after the genesis block, making this function irrelevant in those cases.
func EnableVerkleAtGenesis(db ethdb.Database, genesis *Genesis) (bool, error) {
if genesis != nil {
if genesis.Config == nil {
return false, errGenesisNoConfig
}
return genesis.Config.EnableVerkleAtGenesis, nil
}
if ghash := rawdb.ReadCanonicalHash(db, 0); ghash != (common.Hash{}) {
chainCfg := rawdb.ReadChainConfig(db, ghash)
if chainCfg != nil {
return chainCfg.EnableVerkleAtGenesis, nil
}
}
return false, nil
}
// DefaultGenesisBlock returns the Ethereum main net genesis block.
func DefaultGenesisBlock() *Genesis {
return &Genesis{

56
core/genesis_test.go
View file

@ -54,23 +54,23 @@ func testSetupGenesis(t *testing.T, scheme string) {
oldcustomg.Config = &params.ChainConfig{HomesteadBlock: big.NewInt(2)}
tests := []struct {
name string
fn func(ethdb.Database) (*params.ChainConfig, common.Hash, error)
wantConfig *params.ChainConfig
wantHash common.Hash
wantErr error
name string
fn func(ethdb.Database) (*params.ChainConfig, common.Hash, *params.ConfigCompatError, error)
wantConfig *params.ChainConfig
wantHash common.Hash
wantErr error
wantCompactErr *params.ConfigCompatError
}{
{
name: "genesis without ChainConfig",
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, error) {
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, *params.ConfigCompatError, error) {
return SetupGenesisBlock(db, triedb.NewDatabase(db, newDbConfig(scheme)), new(Genesis))
},
wantErr: errGenesisNoConfig,
wantConfig: params.AllEthashProtocolChanges,
wantErr: errGenesisNoConfig,
},
{
name: "no block in DB, genesis == nil",
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, error) {
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, *params.ConfigCompatError, error) {
return SetupGenesisBlock(db, triedb.NewDatabase(db, newDbConfig(scheme)), nil)
},
wantHash: params.MainnetGenesisHash,
@ -78,7 +78,7 @@ func testSetupGenesis(t *testing.T, scheme string) {
},
{
name: "mainnet block in DB, genesis == nil",
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, error) {
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, *params.ConfigCompatError, error) {
DefaultGenesisBlock().MustCommit(db, triedb.NewDatabase(db, newDbConfig(scheme)))
return SetupGenesisBlock(db, triedb.NewDatabase(db, newDbConfig(scheme)), nil)
},
@ -87,7 +87,7 @@ func testSetupGenesis(t *testing.T, scheme string) {
},
{
name: "custom block in DB, genesis == nil",
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, error) {
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, *params.ConfigCompatError, error) {
tdb := triedb.NewDatabase(db, newDbConfig(scheme))
customg.Commit(db, tdb)
return SetupGenesisBlock(db, tdb, nil)
@ -97,18 +97,16 @@ func testSetupGenesis(t *testing.T, scheme string) {
},
{
name: "custom block in DB, genesis == sepolia",
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, error) {
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, *params.ConfigCompatError, error) {
tdb := triedb.NewDatabase(db, newDbConfig(scheme))
customg.Commit(db, tdb)
return SetupGenesisBlock(db, tdb, DefaultSepoliaGenesisBlock())
},
wantErr: &GenesisMismatchError{Stored: customghash, New: params.SepoliaGenesisHash},
wantHash: params.SepoliaGenesisHash,
wantConfig: params.SepoliaChainConfig,
wantErr: &GenesisMismatchError{Stored: customghash, New: params.SepoliaGenesisHash},
},
{
name: "compatible config in DB",
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, error) {
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, *params.ConfigCompatError, error) {
tdb := triedb.NewDatabase(db, newDbConfig(scheme))
oldcustomg.Commit(db, tdb)
return SetupGenesisBlock(db, tdb, &customg)
@ -118,7 +116,7 @@ func testSetupGenesis(t *testing.T, scheme string) {
},
{
name: "incompatible config in DB",
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, error) {
fn: func(db ethdb.Database) (*params.ChainConfig, common.Hash, *params.ConfigCompatError, error) {
// Commit the 'old' genesis block with Homestead transition at #2.
// Advance to block #4, past the homestead transition block of customg.
tdb := triedb.NewDatabase(db, newDbConfig(scheme))
@ -135,7 +133,7 @@ func testSetupGenesis(t *testing.T, scheme string) {
},
wantHash: customghash,
wantConfig: customg.Config,
wantErr: &params.ConfigCompatError{
wantCompactErr: &params.ConfigCompatError{
What: "Homestead fork block",
StoredBlock: big.NewInt(2),
NewBlock: big.NewInt(3),
@ -146,12 +144,16 @@ func testSetupGenesis(t *testing.T, scheme string) {
for _, test := range tests {
db := rawdb.NewMemoryDatabase()
config, hash, err := test.fn(db)
config, hash, compatErr, err := test.fn(db)
// Check the return values.
if !reflect.DeepEqual(err, test.wantErr) {
spew := spew.ConfigState{DisablePointerAddresses: true, DisableCapacities: true}
t.Errorf("%s: returned error %#v, want %#v", test.name, spew.NewFormatter(err), spew.NewFormatter(test.wantErr))
}
if !reflect.DeepEqual(compatErr, test.wantCompactErr) {
spew := spew.ConfigState{DisablePointerAddresses: true, DisableCapacities: true}
t.Errorf("%s: returned error %#v, want %#v", test.name, spew.NewFormatter(compatErr), spew.NewFormatter(test.wantCompactErr))
}
if !reflect.DeepEqual(config, test.wantConfig) {
t.Errorf("%s:\nreturned %v\nwant %v", test.name, config, test.wantConfig)
}
@ -189,7 +191,7 @@ func TestGenesisHashes(t *testing.T) {
}
}
func TestGenesis_Commit(t *testing.T) {
func TestGenesisCommit(t *testing.T) {
genesis := &Genesis{
BaseFee: big.NewInt(params.InitialBaseFee),
Config: params.TestChainConfig,
@ -207,13 +209,6 @@ func TestGenesis_Commit(t *testing.T) {
if genesisBlock.Difficulty().Cmp(params.GenesisDifficulty) != 0 {
t.Errorf("assumption wrong: want: %d, got: %v", params.GenesisDifficulty, genesisBlock.Difficulty())
}
// Expect the stored total difficulty to be the difficulty of the genesis block.
stored := rawdb.ReadTd(db, genesisBlock.Hash(), genesisBlock.NumberU64())
if stored.Cmp(genesisBlock.Difficulty()) != 0 {
t.Errorf("inequal difficulty; stored: %v, genesisBlock: %v", stored, genesisBlock.Difficulty())
}
}
func TestReadWriteGenesisAlloc(t *testing.T) {
@ -277,10 +272,17 @@ func TestVerkleGenesisCommit(t *testing.T) {
ShanghaiTime: &verkleTime,
CancunTime: &verkleTime,
PragueTime: &verkleTime,
OsakaTime: &verkleTime,
VerkleTime: &verkleTime,
TerminalTotalDifficulty: big.NewInt(0),
EnableVerkleAtGenesis: true,
Ethash: nil,
Clique: nil,
BlobScheduleConfig: &params.BlobScheduleConfig{
Cancun: params.DefaultCancunBlobConfig,
Prague: params.DefaultPragueBlobConfig,
Verkle: params.DefaultPragueBlobConfig,
},
}
genesis := &Genesis{

37
core/headerchain.go
View file

@ -19,7 +19,6 @@ package core
import (
"errors"
"fmt"
"math/big"
"sync/atomic"
"time"
@ -36,7 +35,6 @@ import (
const (
headerCacheLimit = 512
tdCacheLimit = 1024
numberCacheLimit = 2048
)
@ -65,8 +63,7 @@ type HeaderChain struct {
currentHeaderHash common.Hash // Hash of the current head of the header chain (prevent recomputing all the time)
headerCache *lru.Cache[common.Hash, *types.Header]
tdCache *lru.Cache[common.Hash, *big.Int] // most recent total difficulties
numberCache *lru.Cache[common.Hash, uint64] // most recent block numbers
numberCache *lru.Cache[common.Hash, uint64] // most recent block numbers
procInterrupt func() bool
engine consensus.Engine
@ -79,7 +76,6 @@ func NewHeaderChain(chainDb ethdb.Database, config *params.ChainConfig, engine c
config: config,
chainDb: chainDb,
headerCache: lru.NewCache[common.Hash, *types.Header](headerCacheLimit),
tdCache: lru.NewCache[common.Hash, *big.Int](tdCacheLimit),
numberCache: lru.NewCache[common.Hash, uint64](numberCacheLimit),
procInterrupt: procInterrupt,
engine: engine,
@ -197,14 +193,12 @@ func (hc *HeaderChain) WriteHeaders(headers []*types.Header) (int, error) {
if len(headers) == 0 {
return 0, nil
}
ptd := hc.GetTd(headers[0].ParentHash, headers[0].Number.Uint64()-1)
if ptd == nil {
if !hc.HasHeader(headers[0].ParentHash, headers[0].Number.Uint64()-1) {
return 0, consensus.ErrUnknownAncestor
}
var (
newTD = new(big.Int).Set(ptd) // Total difficulty of inserted chain
inserted []rawdb.NumberHash // Ephemeral lookup of number/hash for the chain
parentKnown = true // Set to true to force hc.HasHeader check the first iteration
inserted []rawdb.NumberHash // Ephemeral lookup of number/hash for the chain
parentKnown = true // Set to true to force hc.HasHeader check the first iteration
batch = hc.chainDb.NewBatch()
)
for i, header := range headers {
@ -218,16 +212,11 @@ func (hc *HeaderChain) WriteHeaders(headers []*types.Header) (int, error) {
hash = header.Hash()
}
number := header.Number.Uint64()
newTD.Add(newTD, header.Difficulty)
// If the parent was not present, store it
// If the header is already known, skip it, otherwise store
alreadyKnown := parentKnown && hc.HasHeader(hash, number)
if !alreadyKnown {
// Irrelevant of the canonical status, write the TD and header to the database.
rawdb.WriteTd(batch, hash, number, newTD)
hc.tdCache.Add(hash, new(big.Int).Set(newTD))
rawdb.WriteHeader(batch, header)
inserted = append(inserted, rawdb.NumberHash{Number: number, Hash: hash})
hc.headerCache.Add(hash, header)
@ -392,22 +381,6 @@ func (hc *HeaderChain) GetAncestor(hash common.Hash, number, ancestor uint64, ma
return hash, number
}
// GetTd retrieves a block's total difficulty in the canonical chain from the
// database by hash and number, caching it if found.
func (hc *HeaderChain) GetTd(hash common.Hash, number uint64) *big.Int {
// Short circuit if the td's already in the cache, retrieve otherwise
if cached, ok := hc.tdCache.Get(hash); ok {
return cached
}
td := rawdb.ReadTd(hc.chainDb, hash, number)
if td == nil {
return nil
}
// Cache the found body for next time and return
hc.tdCache.Add(hash, td)
return td
}
// GetHeader retrieves a block header from the database by hash and number,
// caching it if found.
func (hc *HeaderChain) GetHeader(hash common.Hash, number uint64) *types.Header {
@ -620,7 +593,6 @@ func (hc *HeaderChain) setHead(headBlock uint64, headTime uint64, updateFn Updat
delFn(batch, hash, num)
}
rawdb.DeleteHeader(batch, hash, num)
rawdb.DeleteTd(batch, hash, num)
}
rawdb.DeleteCanonicalHash(batch, num)
}
@ -631,7 +603,6 @@ func (hc *HeaderChain) setHead(headBlock uint64, headTime uint64, updateFn Updat
}
// Clear out any stale content from the caches
hc.headerCache.Purge()
hc.tdCache.Purge()
hc.numberCache.Purge()
}

4
core/headerchain_test.go
View file

@ -39,10 +39,6 @@ func verifyUnbrokenCanonchain(hc *HeaderChain) error {
return fmt.Errorf("Canon hash chain broken, block %d got %x, expected %x",
h.Number, canonHash[:8], exp[:8])
}
// Verify that we have the TD
if td := rawdb.ReadTd(hc.chainDb, canonHash, h.Number.Uint64()); td == nil {
return fmt.Errorf("Canon TD missing at block %d", h.Number)
}
if h.Number.Uint64() == 0 {
break
}

70
core/rawdb/accessors_chain.go
View file

@ -508,54 +508,6 @@ func DeleteBody(db ethdb.KeyValueWriter, hash common.Hash, number uint64) {
}
}
// ReadTdRLP retrieves a block's total difficulty corresponding to the hash in RLP encoding.
func ReadTdRLP(db ethdb.Reader, hash common.Hash, number uint64) rlp.RawValue {
var data []byte
db.ReadAncients(func(reader ethdb.AncientReaderOp) error {
// Check if the data is in ancients
if isCanon(reader, number, hash) {
data, _ = reader.Ancient(ChainFreezerDifficultyTable, number)
return nil
}
// If not, try reading from leveldb
data, _ = db.Get(headerTDKey(number, hash))
return nil
})
return data
}
// ReadTd retrieves a block's total difficulty corresponding to the hash.
func ReadTd(db ethdb.Reader, hash common.Hash, number uint64) *big.Int {
data := ReadTdRLP(db, hash, number)
if len(data) == 0 {
return nil
}
td := new(big.Int)
if err := rlp.DecodeBytes(data, td); err != nil {
log.Error("Invalid block total difficulty RLP", "hash", hash, "err", err)
return nil
}
return td
}
// WriteTd stores the total difficulty of a block into the database.
func WriteTd(db ethdb.KeyValueWriter, hash common.Hash, number uint64, td *big.Int) {
data, err := rlp.EncodeToBytes(td)
if err != nil {
log.Crit("Failed to RLP encode block total difficulty", "err", err)
}
if err := db.Put(headerTDKey(number, hash), data); err != nil {
log.Crit("Failed to store block total difficulty", "err", err)
}
}
// DeleteTd removes all block total difficulty data associated with a hash.
func DeleteTd(db ethdb.KeyValueWriter, hash common.Hash, number uint64) {
if err := db.Delete(headerTDKey(number, hash)); err != nil {
log.Crit("Failed to delete block total difficulty", "err", err)
}
}
// HasReceipts verifies the existence of all the transaction receipts belonging
// to a block.
func HasReceipts(db ethdb.Reader, hash common.Hash, number uint64) bool {
@ -635,7 +587,7 @@ func ReadReceipts(db ethdb.Reader, hash common.Hash, number uint64, time uint64,
// Compute effective blob gas price.
var blobGasPrice *big.Int
if header != nil && header.ExcessBlobGas != nil {
blobGasPrice = eip4844.CalcBlobFee(*header.ExcessBlobGas)
blobGasPrice = eip4844.CalcBlobFee(config, header)
}
if err := receipts.DeriveFields(config, hash, number, time, baseFee, blobGasPrice, body.Transactions); err != nil {
log.Error("Failed to derive block receipts fields", "hash", hash, "number", number, "err", err)
@ -741,11 +693,9 @@ func WriteBlock(db ethdb.KeyValueWriter, block *types.Block) {
}
// WriteAncientBlocks writes entire block data into ancient store and returns the total written size.
func WriteAncientBlocks(db ethdb.AncientWriter, blocks []*types.Block, receipts []types.Receipts, td *big.Int) (int64, error) {
var (
tdSum = new(big.Int).Set(td)
stReceipts []*types.ReceiptForStorage
)
func WriteAncientBlocks(db ethdb.AncientWriter, blocks []*types.Block, receipts []types.Receipts) (int64, error) {
var stReceipts []*types.ReceiptForStorage
return db.ModifyAncients(func(op ethdb.AncientWriteOp) error {
for i, block := range blocks {
// Convert receipts to storage format and sum up total difficulty.
@ -754,10 +704,7 @@ func WriteAncientBlocks(db ethdb.AncientWriter, blocks []*types.Block, receipts
stReceipts = append(stReceipts, (*types.ReceiptForStorage)(receipt))
}
header := block.Header()
if i > 0 {
tdSum.Add(tdSum, header.Difficulty)
}
if err := writeAncientBlock(op, block, header, stReceipts, tdSum); err != nil {
if err := writeAncientBlock(op, block, header, stReceipts); err != nil {
return err
}
}
@ -765,7 +712,7 @@ func WriteAncientBlocks(db ethdb.AncientWriter, blocks []*types.Block, receipts
})
}
func writeAncientBlock(op ethdb.AncientWriteOp, block *types.Block, header *types.Header, receipts []*types.ReceiptForStorage, td *big.Int) error {
func writeAncientBlock(op ethdb.AncientWriteOp, block *types.Block, header *types.Header, receipts []*types.ReceiptForStorage) error {
num := block.NumberU64()
if err := op.AppendRaw(ChainFreezerHashTable, num, block.Hash().Bytes()); err != nil {
return fmt.Errorf("can't add block %d hash: %v", num, err)
@ -779,9 +726,6 @@ func writeAncientBlock(op ethdb.AncientWriteOp, block *types.Block, header *type
if err := op.Append(ChainFreezerReceiptTable, num, receipts); err != nil {
return fmt.Errorf("can't append block %d receipts: %v", num, err)
}
if err := op.Append(ChainFreezerDifficultyTable, num, td); err != nil {
return fmt.Errorf("can't append block %d total difficulty: %v", num, err)
}
return nil
}
@ -790,7 +734,6 @@ func DeleteBlock(db ethdb.KeyValueWriter, hash common.Hash, number uint64) {
DeleteReceipts(db, hash, number)
DeleteHeader(db, hash, number)
DeleteBody(db, hash, number)
DeleteTd(db, hash, number)
}
// DeleteBlockWithoutNumber removes all block data associated with a hash, except
@ -799,7 +742,6 @@ func DeleteBlockWithoutNumber(db ethdb.KeyValueWriter, hash common.Hash, number
DeleteReceipts(db, hash, number)
deleteHeaderWithoutNumber(db, hash, number)
DeleteBody(db, hash, number)
DeleteTd(db, hash, number)
}
const badBlockToKeep = 10

43
core/rawdb/accessors_chain_test.go
View file

@ -258,29 +258,6 @@ func TestBadBlockStorage(t *testing.T) {
}
}
// Tests block total difficulty storage and retrieval operations.
func TestTdStorage(t *testing.T) {
db := NewMemoryDatabase()
// Create a test TD to move around the database and make sure it's really new
hash, td := common.Hash{}, big.NewInt(314)
if entry := ReadTd(db, hash, 0); entry != nil {
t.Fatalf("Non existent TD returned: %v", entry)
}
// Write and verify the TD in the database
WriteTd(db, hash, 0, td)
if entry := ReadTd(db, hash, 0); entry == nil {
t.Fatalf("Stored TD not found")
} else if entry.Cmp(td) != 0 {
t.Fatalf("Retrieved TD mismatch: have %v, want %v", entry, td)
}
// Delete the TD and verify the execution
DeleteTd(db, hash, 0)
if entry := ReadTd(db, hash, 0); entry != nil {
t.Fatalf("Deleted TD returned: %v", entry)
}
}
// Tests that canonical numbers can be mapped to hashes and retrieved.
func TestCanonicalMappingStorage(t *testing.T) {
db := NewMemoryDatabase()
@ -460,12 +437,9 @@ func TestAncientStorage(t *testing.T) {
if blob := ReadReceiptsRLP(db, hash, number); len(blob) > 0 {
t.Fatalf("non existent receipts returned")
}
if blob := ReadTdRLP(db, hash, number); len(blob) > 0 {
t.Fatalf("non existent td returned")
}
// Write and verify the header in the database
WriteAncientBlocks(db, []*types.Block{block}, []types.Receipts{nil}, big.NewInt(100))
WriteAncientBlocks(db, []*types.Block{block}, []types.Receipts{nil})
if blob := ReadHeaderRLP(db, hash, number); len(blob) == 0 {
t.Fatalf("no header returned")
@ -476,9 +450,6 @@ func TestAncientStorage(t *testing.T) {
if blob := ReadReceiptsRLP(db, hash, number); len(blob) == 0 {
t.Fatalf("no receipts returned")
}
if blob := ReadTdRLP(db, hash, number); len(blob) == 0 {
t.Fatalf("no td returned")
}
// Use a fake hash for data retrieval, nothing should be returned.
fakeHash := common.BytesToHash([]byte{0x01, 0x02, 0x03})
@ -491,9 +462,6 @@ func TestAncientStorage(t *testing.T) {
if blob := ReadReceiptsRLP(db, fakeHash, number); len(blob) != 0 {
t.Fatalf("invalid receipts returned")
}
if blob := ReadTdRLP(db, fakeHash, number); len(blob) != 0 {
t.Fatalf("invalid td returned")
}
}
func TestCanonicalHashIteration(t *testing.T) {
@ -518,7 +486,6 @@ func TestCanonicalHashIteration(t *testing.T) {
// Fill database with testing data.
for i := uint64(1); i <= 8; i++ {
WriteCanonicalHash(db, common.Hash{}, i)
WriteTd(db, common.Hash{}, i, big.NewInt(10)) // Write some interferential data
}
for i, c := range cases {
numbers, _ := ReadAllCanonicalHashes(db, c.from, c.to, c.limit)
@ -591,7 +558,6 @@ func BenchmarkWriteAncientBlocks(b *testing.B) {
// The benchmark loop writes batches of blocks, but note that the total block count is
// b.N. This means the resulting ns/op measurement is the time it takes to write a
// single block and its associated data.
var td = big.NewInt(55)
var totalSize int64
for i := 0; i < b.N; i += batchSize {
length := batchSize
@ -601,7 +567,7 @@ func BenchmarkWriteAncientBlocks(b *testing.B) {
blocks := allBlocks[i : i+length]
receipts := batchReceipts[:length]
writeSize, err := WriteAncientBlocks(db, blocks, receipts, td)
writeSize, err := WriteAncientBlocks(db, blocks, receipts)
if err != nil {
b.Fatal(err)
}
@ -883,6 +849,7 @@ func TestHeadersRLPStorage(t *testing.T) {
t.Fatalf("failed to create database with ancient backend")
}
defer db.Close()
// Create blocks
var chain []*types.Block
var pHash common.Hash
@ -898,9 +865,9 @@ func TestHeadersRLPStorage(t *testing.T) {
chain = append(chain, block)
pHash = block.Hash()
}
var receipts []types.Receipts = make([]types.Receipts, 100)
receipts := make([]types.Receipts, 100)
// Write first half to ancients
WriteAncientBlocks(db, chain[:50], receipts[:50], big.NewInt(100))
WriteAncientBlocks(db, chain[:50], receipts[:50])
// Write second half to db
for i := 50; i < 100; i++ {
WriteCanonicalHash(db, chain[i].Hash(), chain[i].NumberU64())

13
core/rawdb/ancient_scheme.go
View file

@ -35,19 +35,15 @@ const (
// ChainFreezerReceiptTable indicates the name of the freezer receipts table.
ChainFreezerReceiptTable = "receipts"
// ChainFreezerDifficultyTable indicates the name of the freezer total difficulty table.
ChainFreezerDifficultyTable = "diffs"
)
// chainFreezerNoSnappy configures whether compression is disabled for the ancient-tables.
// Hashes and difficulties don't compress well.
var chainFreezerNoSnappy = map[string]bool{
ChainFreezerHeaderTable: false,
ChainFreezerHashTable: true,
ChainFreezerBodiesTable: false,
ChainFreezerReceiptTable: false,
ChainFreezerDifficultyTable: true,
ChainFreezerHeaderTable: false,
ChainFreezerHashTable: true,
ChainFreezerBodiesTable: false,
ChainFreezerReceiptTable: false,
}
const (
@ -62,6 +58,7 @@ const (
stateHistoryStorageData = "storage.data"
)
// stateFreezerNoSnappy configures whether compression is disabled for the state freezer.
var stateFreezerNoSnappy = map[string]bool{
stateHistoryMeta: true,
stateHistoryAccountIndex: false,

8
core/rawdb/chain_freezer.go
View file

@ -315,11 +315,6 @@ func (f *chainFreezer) freezeRange(nfdb *nofreezedb, number, limit uint64) (hash
if len(receipts) == 0 {
return fmt.Errorf("block receipts missing, can't freeze block %d", number)
}
td := ReadTdRLP(nfdb, hash, number)
if len(td) == 0 {
return fmt.Errorf("total difficulty missing, can't freeze block %d", number)
}
// Write to the batch.
if err := op.AppendRaw(ChainFreezerHashTable, number, hash[:]); err != nil {
return fmt.Errorf("can't write hash to Freezer: %v", err)
@ -333,9 +328,6 @@ func (f *chainFreezer) freezeRange(nfdb *nofreezedb, number, limit uint64) (hash
if err := op.AppendRaw(ChainFreezerReceiptTable, number, receipts); err != nil {
return fmt.Errorf("can't write receipts to Freezer: %v", err)
}
if err := op.AppendRaw(ChainFreezerDifficultyTable, number, td); err != nil {
return fmt.Errorf("can't write td to Freezer: %v", err)
}
hashes = append(hashes, hash)
}
return nil

2
core/rawdb/database.go
View file

@ -500,7 +500,7 @@ func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
{"Key-Value store", "Headers", headers.Size(), headers.Count()},
{"Key-Value store", "Bodies", bodies.Size(), bodies.Count()},
{"Key-Value store", "Receipt lists", receipts.Size(), receipts.Count()},
{"Key-Value store", "Difficulties", tds.Size(), tds.Count()},
{"Key-Value store", "Difficulties (deprecated)", tds.Size(), tds.Count()},
{"Key-Value store", "Block number->hash", numHashPairings.Size(), numHashPairings.Count()},
{"Key-Value store", "Block hash->number", hashNumPairings.Size(), hashNumPairings.Count()},
{"Key-Value store", "Transaction index", txLookups.Size(), txLookups.Count()},

4
core/rawdb/freezer.go
View file

@ -87,6 +87,10 @@ func NewFreezer(datadir string, namespace string, readonly bool, maxTableSize ui
)
// Ensure the datadir is not a symbolic link if it exists.
if info, err := os.Lstat(datadir); !os.IsNotExist(err) {
if info == nil {
log.Warn("Could not Lstat the database", "path", datadir)
return nil, errors.New("lstat failed")
}
if info.Mode()&os.ModeSymlink != 0 {
log.Warn("Symbolic link ancient database is not supported", "path", datadir)
return nil, errSymlinkDatadir

10
core/rawdb/freezer_batch.go
View file

@ -19,6 +19,7 @@ package rawdb
import (
"fmt"
"math"
"time"
"github.com/ethereum/go-ethereum/rlp"
"github.com/golang/snappy"
@ -188,9 +189,6 @@ func (batch *freezerTableBatch) commit() error {
if err != nil {
return err
}
if err := batch.t.head.Sync(); err != nil {
return err
}
dataSize := int64(len(batch.dataBuffer))
batch.dataBuffer = batch.dataBuffer[:0]
@ -208,6 +206,12 @@ func (batch *freezerTableBatch) commit() error {
// Update metrics.
batch.t.sizeGauge.Inc(dataSize + indexSize)
batch.t.writeMeter.Mark(dataSize + indexSize)
// Periodically sync the table, todo (rjl493456442) make it configurable?
if time.Since(batch.t.lastSync) > 30*time.Second {
batch.t.lastSync = time.Now()
return batch.t.Sync()
}
return nil
}

192
core/rawdb/freezer_meta.go
View file

@ -17,93 +17,173 @@
package rawdb
import (
"errors"
"io"
"math"
"os"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/rlp"
)
const freezerVersion = 1 // The initial version tag of freezer table metadata
const (
freezerTableV1 = 1 // Initial version of metadata struct
freezerTableV2 = 2 // Add field: 'flushOffset'
freezerVersion = freezerTableV2 // The current used version
)
// freezerTableMeta wraps all the metadata of the freezer table.
// freezerTableMeta is a collection of additional properties that describe the
// freezer table. These properties are designed with error resilience, allowing
// them to be automatically corrected after an error occurs without significantly
// impacting overall correctness.
type freezerTableMeta struct {
// Version is the versioning descriptor of the freezer table.
Version uint16
file *os.File // file handler of metadata
version uint16 // version descriptor of the freezer table
// VirtualTail indicates how many items have been marked as deleted.
// Its value is equal to the number of items removed from the table
// plus the number of items hidden in the table, so it should never
// be lower than the "actual tail".
VirtualTail uint64
// virtualTail represents the number of items marked as deleted. It is
// calculated as the sum of items removed from the table and the items
// hidden within the table, and should never be less than the "actual
// tail".
//
// If lost due to a crash or other reasons, it will be reset to the number
// of items deleted from the table, causing the previously hidden items
// to become visible, which is an acceptable consequence.
virtualTail uint64
// flushOffset represents the offset in the index file up to which the index
// items along with the corresponding data items in data files has been flushed
// (fsync’d) to disk. Beyond this offset, data integrity is not guaranteed,
// the extra index items along with the associated data items should be removed
// during the startup.
//
// The principle is that all data items above the flush offset are considered
// volatile and should be recoverable if they are discarded after the unclean
// shutdown. If data integrity is required, manually force a sync of the
// freezer before proceeding with further operations (e.g. do freezer.Sync()
// first and then write data to key value store in some circumstances).
//
// The offset could be moved forward by applying sync operation, or be moved
// backward in cases of head/tail truncation, etc.
flushOffset int64
}
// newMetadata initializes the metadata object with the given virtual tail.
func newMetadata(tail uint64) *freezerTableMeta {
// decodeV1 attempts to decode the metadata structure in v1 format. If fails or
// the result is incompatible, nil is returned.
func decodeV1(file *os.File) *freezerTableMeta {
_, err := file.Seek(0, io.SeekStart)
if err != nil {
return nil
}
type obj struct {
Version uint16
Tail uint64
}
var o obj
if err := rlp.Decode(file, &o); err != nil {
return nil
}
if o.Version != freezerTableV1 {
return nil
}
return &freezerTableMeta{
Version: freezerVersion,
VirtualTail: tail,
file: file,
version: o.Version,
virtualTail: o.Tail,
}
}
// readMetadata reads the metadata of the freezer table from the
// given metadata file.
func readMetadata(file *os.File) (*freezerTableMeta, error) {
// decodeV2 attempts to decode the metadata structure in v2 format. If fails or
// the result is incompatible, nil is returned.
func decodeV2(file *os.File) *freezerTableMeta {
_, err := file.Seek(0, io.SeekStart)
if err != nil {
return nil, err
return nil
}
var meta freezerTableMeta
if err := rlp.Decode(file, &meta); err != nil {
return nil, err
type obj struct {
Version uint16
Tail uint64
Offset uint64
}
var o obj
if err := rlp.Decode(file, &o); err != nil {
return nil
}
if o.Version != freezerTableV2 {
return nil
}
if o.Offset > math.MaxInt64 {
log.Error("Invalid flushOffset %d in freezer metadata", o.Offset, "file", file.Name())
return nil
}
return &freezerTableMeta{
file: file,
version: freezerTableV2,
virtualTail: o.Tail,
flushOffset: int64(o.Offset),
}
return &meta, nil
}
// writeMetadata writes the metadata of the freezer table into the
// given metadata file.
func writeMetadata(file *os.File, meta *freezerTableMeta) error {
_, err := file.Seek(0, io.SeekStart)
if err != nil {
return err
}
return rlp.Encode(file, meta)
}
// loadMetadata loads the metadata from the given metadata file.
// Initializes the metadata file with the given "actual tail" if
// it's empty.
func loadMetadata(file *os.File, tail uint64) (*freezerTableMeta, error) {
// newMetadata initializes the metadata object, either by loading it from the file
// or by constructing a new one from scratch.
func newMetadata(file *os.File) (*freezerTableMeta, error) {
stat, err := file.Stat()
if err != nil {
return nil, err
}
// Write the metadata with the given actual tail into metadata file
// if it's non-existent. There are two possible scenarios here:
// - the freezer table is empty
// - the freezer table is legacy
// In both cases, write the meta into the file with the actual tail
// as the virtual tail.
if stat.Size() == 0 {
m := newMetadata(tail)
if err := writeMetadata(file, m); err != nil {
m := &freezerTableMeta{
file: file,
version: freezerTableV2,
virtualTail: 0,
flushOffset: 0,
}
if err := m.write(true); err != nil {
return nil, err
}
return m, nil
}
m, err := readMetadata(file)
if err != nil {
return nil, err
if m := decodeV2(file); m != nil {
return m, nil
}
// Update the virtual tail with the given actual tail if it's even
// lower than it. Theoretically it shouldn't happen at all, print
// a warning here.
if m.VirtualTail < tail {
log.Warn("Updated virtual tail", "have", m.VirtualTail, "now", tail)
m.VirtualTail = tail
if err := writeMetadata(file, m); err != nil {
return nil, err
}
if m := decodeV1(file); m != nil {
return m, nil // legacy metadata
}
return m, nil
return nil, errors.New("failed to decode metadata")
}
// setVirtualTail sets the virtual tail and flushes the metadata if sync is true.
func (m *freezerTableMeta) setVirtualTail(tail uint64, sync bool) error {
m.virtualTail = tail
return m.write(sync)
}
// setFlushOffset sets the flush offset and flushes the metadata if sync is true.
func (m *freezerTableMeta) setFlushOffset(offset int64, sync bool) error {
m.flushOffset = offset
return m.write(sync)
}
// write flushes the content of metadata into file and performs a fsync if required.
func (m *freezerTableMeta) write(sync bool) error {
type obj struct {
Version uint16
Tail uint64
Offset uint64
}
var o obj
o.Version = freezerVersion // forcibly use the current version
o.Tail = m.virtualTail
o.Offset = uint64(m.flushOffset)
_, err := m.file.Seek(0, io.SeekStart)
if err != nil {
return err
}
if err := rlp.Encode(m.file, &o); err != nil {
return err
}
if !sync {
return nil
}
return m.file.Sync()
}

88
core/rawdb/freezer_meta_test.go
View file

@ -19,6 +19,8 @@ package rawdb
import (
"os"
"testing"
"github.com/ethereum/go-ethereum/rlp"
)
func TestReadWriteFreezerTableMeta(t *testing.T) {
@ -27,36 +29,98 @@ func TestReadWriteFreezerTableMeta(t *testing.T) {
t.Fatalf("Failed to create file %v", err)
}
defer f.Close()
err = writeMetadata(f, newMetadata(100))
meta, err := newMetadata(f)
if err != nil {
t.Fatalf("Failed to write metadata %v", err)
t.Fatalf("Failed to new metadata %v", err)
}
meta, err := readMetadata(f)
meta.setVirtualTail(100, false)
meta, err = newMetadata(f)
if err != nil {
t.Fatalf("Failed to read metadata %v", err)
t.Fatalf("Failed to reload metadata %v", err)
}
if meta.Version != freezerVersion {
if meta.version != freezerTableV2 {
t.Fatalf("Unexpected version field")
}
if meta.VirtualTail != uint64(100) {
if meta.virtualTail != uint64(100) {
t.Fatalf("Unexpected virtual tail field")
}
}
func TestInitializeFreezerTableMeta(t *testing.T) {
func TestUpgradeMetadata(t *testing.T) {
f, err := os.CreateTemp(t.TempDir(), "*")
if err != nil {
t.Fatalf("Failed to create file %v", err)
}
defer f.Close()
meta, err := loadMetadata(f, uint64(100))
// Write legacy metadata into file
type obj struct {
Version uint16
Tail uint64
}
var o obj
o.Version = freezerTableV1
o.Tail = 100
if err := rlp.Encode(f, &o); err != nil {
t.Fatalf("Failed to encode %v", err)
}
// Reload the metadata, a silent upgrade is expected
meta, err := newMetadata(f)
if err != nil {
t.Fatalf("Failed to read metadata %v", err)
}
if meta.Version != freezerVersion {
t.Fatalf("Unexpected version field")
if meta.version != freezerTableV1 {
t.Fatal("Unexpected version field")
}
if meta.VirtualTail != uint64(100) {
t.Fatalf("Unexpected virtual tail field")
if meta.virtualTail != uint64(100) {
t.Fatal("Unexpected virtual tail field")
}
if meta.flushOffset != 0 {
t.Fatal("Unexpected flush offset field")
}
meta.setFlushOffset(100, true)
meta, err = newMetadata(f)
if err != nil {
t.Fatalf("Failed to read metadata %v", err)
}
if meta.version != freezerTableV2 {
t.Fatal("Unexpected version field")
}
if meta.virtualTail != uint64(100) {
t.Fatal("Unexpected virtual tail field")
}
if meta.flushOffset != 100 {
t.Fatal("Unexpected flush offset field")
}
}
func TestInvalidMetadata(t *testing.T) {
f, err := os.CreateTemp(t.TempDir(), "*")
if err != nil {
t.Fatalf("Failed to create file %v", err)
}
defer f.Close()
// Write invalid legacy metadata into file
type obj struct {
Version uint16
Tail uint64
}
var o obj
o.Version = freezerTableV2 // -> invalid version tag
o.Tail = 100
if err := rlp.Encode(f, &o); err != nil {
t.Fatalf("Failed to encode %v", err)
}
_, err = newMetadata(f)
if err == nil {
t.Fatal("Unexpected success")
}
}

264
core/rawdb/freezer_table.go
View file

@ -108,11 +108,13 @@ type freezerTable struct {
head *os.File // File descriptor for the data head of the table
index *os.File // File descriptor for the indexEntry file of the table
meta *os.File // File descriptor for metadata of the table
files map[uint32]*os.File // open files
headId uint32 // number of the currently active head file
tailId uint32 // number of the earliest file
metadata *freezerTableMeta // metadata of the table
lastSync time.Time // Timestamp when the last sync was performed
headBytes int64 // Number of bytes written to the head file
readMeter *metrics.Meter // Meter for measuring the effective amount of data read
writeMeter *metrics.Meter // Meter for measuring the effective amount of data written
@ -166,10 +168,17 @@ func newTable(path string, name string, readMeter, writeMeter *metrics.Meter, si
return nil, err
}
}
// Load metadata from the file. The tag will be true if legacy metadata
// is detected.
metadata, err := newMetadata(meta)
if err != nil {
return nil, err
}
// Create the table and repair any past inconsistency
tab := &freezerTable{
index: index,
meta: meta,
metadata: metadata,
lastSync: time.Now(),
files: make(map[uint32]*os.File),
readMeter: readMeter,
writeMeter: writeMeter,
@ -221,13 +230,11 @@ func (t *freezerTable) repair() error {
return err
} // New file can't trigger this path
}
// Validate the index file as it might contain some garbage data after the
// power failures.
if err := t.repairIndex(); err != nil {
return err
}
// Retrieve the file sizes and prepare for truncation. Note the file size
// might be changed after index validation.
// might be changed after index repair.
if stat, err = t.index.Stat(); err != nil {
return err
}
@ -253,12 +260,14 @@ func (t *freezerTable) repair() error {
t.tailId = firstIndex.filenum
t.itemOffset.Store(uint64(firstIndex.offset))
// Load metadata from the file
meta, err := loadMetadata(t.meta, t.itemOffset.Load())
if err != nil {
return err
// Adjust the number of hidden items if it is less than the number of items
// being removed.
if t.itemOffset.Load() > t.metadata.virtualTail {
if err := t.metadata.setVirtualTail(t.itemOffset.Load(), true); err != nil {
return err
}
}
t.itemHidden.Store(meta.VirtualTail)
t.itemHidden.Store(t.metadata.virtualTail)
// Read the last index, use the default value in case the freezer is empty
if offsetsSize == indexEntrySize {
@ -267,12 +276,6 @@ func (t *freezerTable) repair() error {
t.index.ReadAt(buffer, offsetsSize-indexEntrySize)
lastIndex.unmarshalBinary(buffer)
}
// Print an error log if the index is corrupted due to an incorrect
// last index item. While it is theoretically possible to have a zero offset
// by storing all zero-size items, it is highly unlikely to occur in practice.
if lastIndex.offset == 0 && offsetsSize/indexEntrySize > 1 {
log.Error("Corrupted index file detected", "lastOffset", lastIndex.offset, "indexes", offsetsSize/indexEntrySize)
}
if t.readonly {
t.head, err = t.openFile(lastIndex.filenum, openFreezerFileForReadOnly)
} else {
@ -293,6 +296,7 @@ func (t *freezerTable) repair() error {
return fmt.Errorf("freezer table(path: %s, name: %s, num: %d) is corrupted", t.path, t.name, lastIndex.filenum)
}
verbose = true
// Truncate the head file to the last offset pointer
if contentExp < contentSize {
t.logger.Warn("Truncating dangling head", "indexed", contentExp, "stored", contentSize)
@ -304,11 +308,23 @@ func (t *freezerTable) repair() error {
// Truncate the index to point within the head file
if contentExp > contentSize {
t.logger.Warn("Truncating dangling indexes", "indexes", offsetsSize/indexEntrySize, "indexed", contentExp, "stored", contentSize)
if err := truncateFreezerFile(t.index, offsetsSize-indexEntrySize); err != nil {
newOffset := offsetsSize - indexEntrySize
if err := truncateFreezerFile(t.index, newOffset); err != nil {
return err
}
offsetsSize -= indexEntrySize
// If the index file is truncated beyond the flush offset, move the flush
// offset back to the new end of the file. A crash may occur before the
// offset is updated, leaving a dangling reference that points to a position
// outside the file. If so, the offset will be reset to the new end of the
// file during the next run.
if t.metadata.flushOffset > newOffset {
if err := t.metadata.setFlushOffset(newOffset, true); err != nil {
return err
}
}
// Read the new head index, use the default value in case
// the freezer is already empty.
var newLastIndex indexEntry
@ -345,7 +361,7 @@ func (t *freezerTable) repair() error {
if err := t.head.Sync(); err != nil {
return err
}
if err := t.meta.Sync(); err != nil {
if err := t.metadata.file.Sync(); err != nil {
return err
}
}
@ -372,7 +388,65 @@ func (t *freezerTable) repair() error {
return nil
}
// repairIndex validates the integrity of the index file. According to the design,
func (t *freezerTable) repairIndex() error {
stat, err := t.index.Stat()
if err != nil {
return err
}
size := stat.Size()
// Validate the items in the index file to ensure the data integrity.
// It's possible some garbage data is retained in the index file after
// the power failures and should be truncated first.
size, err = t.checkIndex(size)
if err != nil {
return err
}
// If legacy metadata is detected, attempt to recover the offset from the
// index file to avoid clearing the entire table.
if t.metadata.version == freezerTableV1 {
t.logger.Info("Recovering freezer flushOffset for legacy table", "offset", size)
return t.metadata.setFlushOffset(size, true)
}
switch {
case size == indexEntrySize && t.metadata.flushOffset == 0:
// It's a new freezer table with no content.
// Move the flush offset to the end of the file.
return t.metadata.setFlushOffset(size, true)
case size == t.metadata.flushOffset:
// flushOffset is aligned with the index file, all is well.
return nil
case size > t.metadata.flushOffset:
// Extra index items have been detected beyond the flush offset. Since these
// entries correspond to data that has not been fully flushed to disk in the
// last run (because of unclean shutdown), their integrity cannot be guaranteed.
// To ensure consistency, these index items will be truncated, as there is no
// reliable way to validate or recover their associated data.
extraSize := size - t.metadata.flushOffset
if t.readonly {
return fmt.Errorf("index file(path: %s, name: %s) contains %d garbage data bytes", t.path, t.name, extraSize)
}
t.logger.Warn("Truncating freezer items after flushOffset", "size", extraSize)
return truncateFreezerFile(t.index, t.metadata.flushOffset)
default: // size < flushOffset
// Flush offset refers to a position larger than index file. The only
// possible scenario for this is: a power failure or system crash has occurred after
// truncating the segment in index file from head or tail, but without updating
// the flush offset. In this case, automatically reset the flush offset with
// the file size which implies the entire index file is complete.
if t.readonly {
return nil // do nothing in read only mode
}
t.logger.Warn("Rewinding freezer flushOffset", "old", t.metadata.flushOffset, "new", size)
return t.metadata.setFlushOffset(size, true)
}
}
// checkIndex validates the integrity of the index file. According to the design,
// the initial entry in the file denotes the earliest data file along with the
// count of deleted items. Following this, all subsequent entries in the file must
// be in order. This function identifies any corrupted entries and truncates items
@ -392,18 +466,11 @@ func (t *freezerTable) repair() error {
// leftover garbage or if all items in the table have zero size is impossible.
// In such instances, the file will remain unchanged to prevent potential data
// loss or misinterpretation.
func (t *freezerTable) repairIndex() error {
// Retrieve the file sizes and prepare for validation
stat, err := t.index.Stat()
if err != nil {
return err
}
size := stat.Size()
func (t *freezerTable) checkIndex(size int64) (int64, error) {
// Move the read cursor to the beginning of the file
_, err = t.index.Seek(0, io.SeekStart)
_, err := t.index.Seek(0, io.SeekStart)
if err != nil {
return err
return 0, err
}
fr := bufio.NewReader(t.index)
@ -425,21 +492,21 @@ func (t *freezerTable) repairIndex() error {
entry.unmarshalBinary(buff)
return entry, nil
}
truncate = func(offset int64) error {
truncate = func(offset int64) (int64, error) {
if t.readonly {
return fmt.Errorf("index file is corrupted at %d, size: %d", offset, size)
return 0, fmt.Errorf("index file is corrupted at %d, size: %d", offset, size)
}
if err := truncateFreezerFile(t.index, offset); err != nil {
return err
return 0, err
}
log.Warn("Truncated index file", "offset", offset, "truncated", size-offset)
return nil
return offset, nil
}
)
for offset := int64(0); offset < size; offset += indexEntrySize {
entry, err := read()
if err != nil {
return err
return 0, err
}
if offset == 0 {
head = entry
@ -468,10 +535,10 @@ func (t *freezerTable) repairIndex() error {
// the seek operation anyway as a precaution.
_, err = t.index.Seek(0, io.SeekEnd)
if err != nil {
return err
return 0, err
}
log.Debug("Verified index file", "items", size/indexEntrySize, "elapsed", common.PrettyDuration(time.Since(start)))
return nil
return size, nil
}
// checkIndexItems validates the correctness of two consecutive index items based
@ -550,12 +617,23 @@ func (t *freezerTable) truncateHead(items uint64) error {
// Truncate the index file first, the tail position is also considered
// when calculating the new freezer table length.
length := items - t.itemOffset.Load()
if err := truncateFreezerFile(t.index, int64(length+1)*indexEntrySize); err != nil {
newOffset := (length + 1) * indexEntrySize
if err := truncateFreezerFile(t.index, int64(newOffset)); err != nil {
return err
}
if err := t.index.Sync(); err != nil {
return err
}
// If the index file is truncated beyond the flush offset, move the flush
// offset back to the new end of the file. A crash may occur before the
// offset is updated, leaving a dangling reference that points to a position
// outside the file. If so, the offset will be reset to the new end of the
// file during the next run.
if t.metadata.flushOffset > int64(newOffset) {
if err := t.metadata.setFlushOffset(int64(newOffset), true); err != nil {
return err
}
}
// Calculate the new expected size of the data file and truncate it
var expected indexEntry
if length == 0 {
@ -652,7 +730,10 @@ func (t *freezerTable) truncateTail(items uint64) error {
}
// Update the virtual tail marker and hidden these entries in table.
t.itemHidden.Store(items)
if err := writeMetadata(t.meta, newMetadata(items)); err != nil {
// Update the virtual tail without fsync, otherwise it will significantly
// impact the overall performance.
if err := t.metadata.setVirtualTail(items, false); err != nil {
return err
}
// Hidden items still fall in the current tail file, no data file
@ -664,6 +745,18 @@ func (t *freezerTable) truncateTail(items uint64) error {
if t.tailId > newTailId {
return fmt.Errorf("invalid index, tail-file %d, item-file %d", t.tailId, newTailId)
}
// Sync the table before performing the index tail truncation. A crash may
// occur after truncating the index file without updating the flush offset,
// leaving a dangling offset that points to a position outside the file.
// The offset will be rewound to the end of file during the next run
// automatically and implicitly assumes all the items within the file are
// complete.
//
// Therefore, forcibly flush everything above the offset to ensure this
// assumption is satisfied!
if err := t.doSync(); err != nil {
return err
}
// Count how many items can be deleted from the file.
var (
newDeleted = items
@ -681,11 +774,6 @@ func (t *freezerTable) truncateTail(items uint64) error {
}
newDeleted = current
}
// Commit the changes of metadata file first before manipulating
// the indexes file.
if err := t.meta.Sync(); err != nil {
return err
}
// Close the index file before shorten it.
if err := t.index.Close(); err != nil {
return err
@ -716,6 +804,21 @@ func (t *freezerTable) truncateTail(items uint64) error {
t.itemOffset.Store(newDeleted)
t.releaseFilesBefore(t.tailId, true)
// Move the index flush offset backward due to the deletion of an index segment.
// A crash may occur before the offset is updated, leaving a dangling reference
// that points to a position outside the file. If so, the offset will be reset
// to the new end of the file during the next run.
//
// Note, both the index and head data file has been persisted before performing
// tail truncation and all the items in these files are regarded as complete.
shorten := indexEntrySize * int64(newDeleted-deleted)
if t.metadata.flushOffset <= shorten {
return fmt.Errorf("invalid index flush offset: %d, shorten: %d", t.metadata.flushOffset, shorten)
} else {
if err := t.metadata.setFlushOffset(t.metadata.flushOffset-shorten, true); err != nil {
return err
}
}
// Retrieve the new size and update the total size counter
newSize, err := t.sizeNolock()
if err != nil {
@ -725,40 +828,30 @@ func (t *freezerTable) truncateTail(items uint64) error {
return nil
}
// Close closes all opened files.
// Close closes all opened files and finalizes the freezer table for use.
// This operation must be completed before shutdown to prevent the loss of
// recent writes.
func (t *freezerTable) Close() error {
t.lock.Lock()
defer t.lock.Unlock()
if err := t.doSync(); err != nil {
return err
}
var errs []error
doClose := func(f *os.File, sync bool, close bool) {
if sync && !t.readonly {
if err := f.Sync(); err != nil {
errs = append(errs, err)
}
}
if close {
if err := f.Close(); err != nil {
errs = append(errs, err)
}
doClose := func(f *os.File) {
if err := f.Close(); err != nil {
errs = append(errs, err)
}
}
// Trying to fsync a file opened in rdonly causes "Access denied"
// error on Windows.
doClose(t.index, true, true)
doClose(t.meta, true, true)
// The preopened non-head data-files are all opened in readonly.
// The head is opened in rw-mode, so we sync it here - but since it's also
// part of t.files, it will be closed in the loop below.
doClose(t.head, true, false) // sync but do not close
doClose(t.index)
doClose(t.metadata.file)
for _, f := range t.files {
doClose(f, false, true) // close but do not sync
doClose(f)
}
t.index = nil
t.meta = nil
t.head = nil
t.metadata.file = nil
if errs != nil {
return fmt.Errorf("%v", errs)
@ -917,7 +1010,7 @@ func (t *freezerTable) retrieveItems(start, count, maxBytes uint64) ([]byte, []i
defer t.lock.RUnlock()
// Ensure the table and the item are accessible
if t.index == nil || t.head == nil || t.meta == nil {
if t.index == nil || t.head == nil || t.metadata.file == nil {
return nil, nil, errClosed
}
var (
@ -1042,6 +1135,9 @@ func (t *freezerTable) advanceHead() error {
t.lock.Lock()
defer t.lock.Unlock()
if err := t.doSync(); err != nil {
return err
}
// We open the next file in truncated mode -- if this file already
// exists, we need to start over from scratch on it.
nextID := t.headId + 1
@ -1069,7 +1165,18 @@ func (t *freezerTable) advanceHead() error {
func (t *freezerTable) Sync() error {
t.lock.Lock()
defer t.lock.Unlock()
if t.index == nil || t.head == nil || t.meta == nil {
return t.doSync()
}
// doSync is the internal version of Sync which assumes the lock is already held.
func (t *freezerTable) doSync() error {
// Trying to fsync a file opened in rdonly causes "Access denied"
// error on Windows.
if t.readonly {
return nil
}
if t.index == nil || t.head == nil || t.metadata.file == nil {
return errClosed
}
var err error
@ -1078,10 +1185,18 @@ func (t *freezerTable) Sync() error {
err = e
}
}
trackError(t.index.Sync())
trackError(t.meta.Sync())
trackError(t.head.Sync())
// A crash may occur before the offset is updated, leaving the offset
// points to a old position. If so, the extra items above the offset
// will be truncated during the next run.
stat, err := t.index.Stat()
if err != nil {
return err
}
offset := stat.Size()
trackError(t.metadata.setFlushOffset(offset, true))
return err
}
@ -1097,13 +1212,8 @@ func (t *freezerTable) dumpIndexString(start, stop int64) string {
}
func (t *freezerTable) dumpIndex(w io.Writer, start, stop int64) {
meta, err := readMetadata(t.meta)
if err != nil {
fmt.Fprintf(w, "Failed to decode freezer table %v\n", err)
return
}
fmt.Fprintf(w, "Version %d count %d, deleted %d, hidden %d\n", meta.Version,
t.items.Load(), t.itemOffset.Load(), t.itemHidden.Load())
fmt.Fprintf(w, "Version %d count %d, deleted %d, hidden %d\n",
t.metadata.version, t.items.Load(), t.itemOffset.Load(), t.itemHidden.Load())
buf := make([]byte, indexEntrySize)

356
core/rawdb/freezer_table_test.go
View file

@ -262,18 +262,6 @@ func TestSnappyDetection(t *testing.T) {
f.Close()
}
// Open without snappy
{
f, err := newTable(os.TempDir(), fname, rm, wm, sg, 50, false, false)
if err != nil {
t.Fatal(err)
}
if _, err = f.Retrieve(0); err == nil {
f.Close()
t.Fatalf("expected empty table")
}
}
// Open with snappy
{
f, err := newTable(os.TempDir(), fname, rm, wm, sg, 50, true, false)
@ -286,6 +274,18 @@ func TestSnappyDetection(t *testing.T) {
t.Fatalf("expected no error, got %v", err)
}
}
// Open without snappy
{
f, err := newTable(os.TempDir(), fname, rm, wm, sg, 50, false, false)
if err != nil {
t.Fatal(err)
}
if _, err = f.Retrieve(0); err == nil {
f.Close()
t.Fatalf("expected empty table")
}
}
}
func assertFileSize(f string, size int64) error {
@ -521,93 +521,53 @@ func TestFreezerOffset(t *testing.T) {
fname := fmt.Sprintf("offset-%d", rand.Uint64())
// Fill table
{
f, err := newTable(os.TempDir(), fname, rm, wm, sg, 40, true, false)
if err != nil {
t.Fatal(err)
}
// Write 6 x 20 bytes, splitting out into three files
batch := f.newBatch()
require.NoError(t, batch.AppendRaw(0, getChunk(20, 0xFF)))
require.NoError(t, batch.AppendRaw(1, getChunk(20, 0xEE)))
require.NoError(t, batch.AppendRaw(2, getChunk(20, 0xdd)))
require.NoError(t, batch.AppendRaw(3, getChunk(20, 0xcc)))
require.NoError(t, batch.AppendRaw(4, getChunk(20, 0xbb)))
require.NoError(t, batch.AppendRaw(5, getChunk(20, 0xaa)))
require.NoError(t, batch.commit())
t.Log(f.dumpIndexString(0, 100))
f.Close()
f, err := newTable(os.TempDir(), fname, rm, wm, sg, 40, true, false)
if err != nil {
t.Fatal(err)
}
// Write 6 x 20 bytes, splitting out into three files
batch := f.newBatch()
require.NoError(t, batch.AppendRaw(0, getChunk(20, 0xFF)))
require.NoError(t, batch.AppendRaw(1, getChunk(20, 0xEE)))
require.NoError(t, batch.AppendRaw(2, getChunk(20, 0xdd)))
require.NoError(t, batch.AppendRaw(3, getChunk(20, 0xcc)))
require.NoError(t, batch.AppendRaw(4, getChunk(20, 0xbb)))
require.NoError(t, batch.AppendRaw(5, getChunk(20, 0xaa)))
require.NoError(t, batch.commit())
t.Log(f.dumpIndexString(0, 100))
// Now crop it.
{
// delete files 0 and 1
for i := 0; i < 2; i++ {
p := filepath.Join(os.TempDir(), fmt.Sprintf("%v.%04d.rdat", fname, i))
if err := os.Remove(p); err != nil {
t.Fatal(err)
}
}
// Read the index file
p := filepath.Join(os.TempDir(), fmt.Sprintf("%v.ridx", fname))
indexFile, err := os.OpenFile(p, os.O_RDWR, 0644)
if err != nil {
t.Fatal(err)
}
indexBuf := make([]byte, 7*indexEntrySize)
indexFile.Read(indexBuf)
// Update the index file, so that we store
// [ file = 2, offset = 4 ] at index zero
zeroIndex := indexEntry{
filenum: uint32(2), // First file is 2
offset: uint32(4), // We have removed four items
}
buf := zeroIndex.append(nil)
// Overwrite index zero
copy(indexBuf, buf)
// Remove the four next indices by overwriting
copy(indexBuf[indexEntrySize:], indexBuf[indexEntrySize*5:])
indexFile.WriteAt(indexBuf, 0)
// Need to truncate the moved index items
indexFile.Truncate(indexEntrySize * (1 + 2))
indexFile.Close()
}
f.truncateTail(4)
f.Close()
// Now open again
{
f, err := newTable(os.TempDir(), fname, rm, wm, sg, 40, true, false)
if err != nil {
t.Fatal(err)
}
defer f.Close()
t.Log(f.dumpIndexString(0, 100))
// It should allow writing item 6.
batch := f.newBatch()
require.NoError(t, batch.AppendRaw(6, getChunk(20, 0x99)))
require.NoError(t, batch.commit())
checkRetrieveError(t, f, map[uint64]error{
0: errOutOfBounds,
1: errOutOfBounds,
2: errOutOfBounds,
3: errOutOfBounds,
})
checkRetrieve(t, f, map[uint64][]byte{
4: getChunk(20, 0xbb),
5: getChunk(20, 0xaa),
6: getChunk(20, 0x99),
})
f, err = newTable(os.TempDir(), fname, rm, wm, sg, 40, true, false)
if err != nil {
t.Fatal(err)
}
t.Log(f.dumpIndexString(0, 100))
// It should allow writing item 6.
batch = f.newBatch()
require.NoError(t, batch.AppendRaw(6, getChunk(20, 0x99)))
require.NoError(t, batch.commit())
checkRetrieveError(t, f, map[uint64]error{
0: errOutOfBounds,
1: errOutOfBounds,
2: errOutOfBounds,
3: errOutOfBounds,
})
checkRetrieve(t, f, map[uint64][]byte{
4: getChunk(20, 0xbb),
5: getChunk(20, 0xaa),
6: getChunk(20, 0x99),
})
f.Close()
// Edit the index again, with a much larger initial offset of 1M.
{
@ -1369,45 +1329,63 @@ func TestRandom(t *testing.T) {
}
func TestIndexValidation(t *testing.T) {
const (
items = 30
dataSize = 10
)
const dataSize = 10
garbage := indexEntry{
filenum: 100,
offset: 200,
}
var cases = []struct {
offset int64
data []byte
expItems int
write int
offset int64
data []byte
expItems int
hasCorruption bool
}{
// extend index file with zero bytes at the end
{
offset: (items + 1) * indexEntrySize,
write: 5,
offset: (5 + 1) * indexEntrySize,
data: make([]byte, indexEntrySize),
expItems: 30,
expItems: 5,
},
// extend index file with unaligned zero bytes at the end
{
write: 5,
offset: (5 + 1) * indexEntrySize,
data: make([]byte, indexEntrySize*1.5),
expItems: 5,
},
// write garbage in the first non-head item
{
write: 5,
offset: indexEntrySize,
data: garbage.append(nil),
expItems: 0,
},
// write garbage in the first non-head item
// write garbage in the middle
{
offset: (items/2 + 1) * indexEntrySize,
write: 5,
offset: 3 * indexEntrySize,
data: garbage.append(nil),
expItems: items / 2,
expItems: 2,
},
// fulfill the first data file (but not yet advanced), the zero bytes
// at tail should be truncated.
{
write: 10,
offset: 11 * indexEntrySize,
data: garbage.append(nil),
expItems: 10,
},
}
for _, c := range cases {
fn := fmt.Sprintf("t-%d", rand.Uint64())
f, err := newTable(os.TempDir(), fn, metrics.NewMeter(), metrics.NewMeter(), metrics.NewGauge(), 100, true, false)
f, err := newTable(os.TempDir(), fn, metrics.NewMeter(), metrics.NewMeter(), metrics.NewGauge(), 10*dataSize, true, false)
if err != nil {
t.Fatal(err)
}
writeChunks(t, f, items, dataSize)
writeChunks(t, f, c.write, dataSize)
// write corrupted data
f.index.WriteAt(c.data, c.offset)
@ -1421,10 +1399,10 @@ func TestIndexValidation(t *testing.T) {
for i := 0; i < c.expItems; i++ {
exp := getChunk(10, i)
got, err := f.Retrieve(uint64(i))
if err != nil {
if err != nil && !c.hasCorruption {
t.Fatalf("Failed to read from table, %v", err)
}
if !bytes.Equal(exp, got) {
if !bytes.Equal(exp, got) && !c.hasCorruption {
t.Fatalf("Unexpected item data, want: %v, got: %v", exp, got)
}
}
@ -1433,3 +1411,163 @@ func TestIndexValidation(t *testing.T) {
}
}
}
// TestFlushOffsetTracking tests the flush offset tracking. The offset moving
// in the test is mostly triggered by the advanceHead (new data file) and
// heda/tail truncation.
func TestFlushOffsetTracking(t *testing.T) {
const (
items = 35
dataSize = 10
fileSize = 100
)
fn := fmt.Sprintf("t-%d", rand.Uint64())
f, err := newTable(os.TempDir(), fn, metrics.NewMeter(), metrics.NewMeter(), metrics.NewGauge(), fileSize, true, false)
if err != nil {
t.Fatal(err)
}
// Data files:
// F1(10 items) -> F2(10 items) -> F3(10 items) -> F4(5 items, non-full)
writeChunks(t, f, items, dataSize)
var cases = []struct {
op func(*freezerTable)
offset int64
}{
{
// Data files:
// F1(10 items) -> F2(10 items) -> F3(10 items) -> F4(5 items, non-full)
func(f *freezerTable) {}, // no-op
31 * indexEntrySize,
},
{
// Write more items to fulfill the newest data file, but the file advance
// is not triggered.
// Data files:
// F1(10 items) -> F2(10 items) -> F3(10 items) -> F4(10 items, full)
func(f *freezerTable) {
batch := f.newBatch()
for i := 0; i < 5; i++ {
batch.AppendRaw(items+uint64(i), make([]byte, dataSize))
}
batch.commit()
},
31 * indexEntrySize,
},
{
// Write more items to trigger the data file advance
// Data files:
// F1(10 items) -> F2(10 items) -> F3(10 items) -> F4(10 items) -> F5(1 item)
func(f *freezerTable) {
batch := f.newBatch()
batch.AppendRaw(items+5, make([]byte, dataSize))
batch.commit()
},
41 * indexEntrySize,
},
{
// Head truncate
// Data files:
// F1(10 items) -> F2(10 items) -> F3(10 items) -> F4(10 items) -> F5(0 item)
func(f *freezerTable) {
f.truncateHead(items + 5)
},
41 * indexEntrySize,
},
{
// Tail truncate
// Data files:
// F1(1 hidden, 9 visible) -> F2(10 items) -> F3(10 items) -> F4(10 items) -> F5(0 item)
func(f *freezerTable) {
f.truncateTail(1)
},
41 * indexEntrySize,
},
{
// Tail truncate
// Data files:
// F2(10 items) -> F3(10 items) -> F4(10 items) -> F5(0 item)
func(f *freezerTable) {
f.truncateTail(10)
},
31 * indexEntrySize,
},
{
// Tail truncate
// Data files:
// F4(10 items) -> F5(0 item)
func(f *freezerTable) {
f.truncateTail(30)
},
11 * indexEntrySize,
},
{
// Head truncate
// Data files:
// F4(9 items)
func(f *freezerTable) {
f.truncateHead(items + 4)
},
10 * indexEntrySize,
},
}
for _, c := range cases {
c.op(f)
if f.metadata.flushOffset != c.offset {
t.Fatalf("Unexpected index flush offset, want: %d, got: %d", c.offset, f.metadata.flushOffset)
}
}
}
func TestTailTruncationCrash(t *testing.T) {
const (
items = 35
dataSize = 10
fileSize = 100
)
fn := fmt.Sprintf("t-%d", rand.Uint64())
f, err := newTable(os.TempDir(), fn, metrics.NewMeter(), metrics.NewMeter(), metrics.NewGauge(), fileSize, true, false)
if err != nil {
t.Fatal(err)
}
// Data files:
// F1(10 items) -> F2(10 items) -> F3(10 items) -> F4(5 items, non-full)
writeChunks(t, f, items, dataSize)
// The latest 5 items are not persisted yet
if f.metadata.flushOffset != 31*indexEntrySize {
t.Fatalf("Unexpected index flush offset, want: %d, got: %d", 31*indexEntrySize, f.metadata.flushOffset)
}
f.truncateTail(5)
if f.metadata.flushOffset != 31*indexEntrySize {
t.Fatalf("Unexpected index flush offset, want: %d, got: %d", 31*indexEntrySize, f.metadata.flushOffset)
}
// Truncate the first 10 items which results in the first data file
// being removed. The offset should be moved to 26*indexEntrySize.
f.truncateTail(10)
if f.metadata.flushOffset != 26*indexEntrySize {
t.Fatalf("Unexpected index flush offset, want: %d, got: %d", 26*indexEntrySize, f.metadata.flushOffset)
}
// Write the offset back to 31*indexEntrySize to simulate a crash
// which occurs after truncating the index file without updating
// the offset
f.metadata.setFlushOffset(31*indexEntrySize, true)
f, err = newTable(os.TempDir(), fn, metrics.NewMeter(), metrics.NewMeter(), metrics.NewGauge(), fileSize, true, false)
if err != nil {
t.Fatal(err)
}
if f.metadata.flushOffset != 26*indexEntrySize {
t.Fatalf("Unexpected index flush offset, want: %d, got: %d", 26*indexEntrySize, f.metadata.flushOffset)
}
}

7
core/rawdb/schema.go
View file

@ -98,7 +98,7 @@ var (
// Data item prefixes (use single byte to avoid mixing data types, avoid `i`, used for indexes).
headerPrefix = []byte("h") // headerPrefix + num (uint64 big endian) + hash -> header
headerTDSuffix = []byte("t") // headerPrefix + num (uint64 big endian) + hash + headerTDSuffix -> td
headerTDSuffix = []byte("t") // headerPrefix + num (uint64 big endian) + hash + headerTDSuffix -> td (deprecated)
headerHashSuffix = []byte("n") // headerPrefix + num (uint64 big endian) + headerHashSuffix -> hash
headerNumberPrefix = []byte("H") // headerNumberPrefix + hash -> num (uint64 big endian)
@ -174,11 +174,6 @@ func headerKey(number uint64, hash common.Hash) []byte {
return append(append(headerPrefix, encodeBlockNumber(number)...), hash.Bytes()...)
}
// headerTDKey = headerPrefix + num (uint64 big endian) + hash + headerTDSuffix
func headerTDKey(number uint64, hash common.Hash) []byte {
return append(headerKey(number, hash), headerTDSuffix...)
}
// headerHashKey = headerPrefix + num (uint64 big endian) + headerHashSuffix
func headerHashKey(number uint64) []byte {
return append(append(headerPrefix, encodeBlockNumber(number)...), headerHashSuffix...)

13
core/sender_cacher.go
View file

@ -18,12 +18,21 @@ package core
import (
"runtime"
"sync"
"github.com/ethereum/go-ethereum/core/types"
)
// SenderCacher is a concurrent transaction sender recoverer and cacher.
var SenderCacher = newTxSenderCacher(runtime.NumCPU())
// senderCacherOnce is used to ensure that the SenderCacher is initialized only once.
var senderCacherOnce = sync.OnceValue(func() *txSenderCacher {
return newTxSenderCacher(runtime.NumCPU())
})
// SenderCacher returns the singleton instance of SenderCacher, initializing it if called for the first time.
// This function is thread-safe and ensures that initialization happens only once.
func SenderCacher() *txSenderCacher {
return senderCacherOnce()
}
// txSenderCacherRequest is a request for recovering transaction senders with a
// specific signature scheme and caching it into the transactions themselves.

9
core/state/snapshot/iterator_fast.go
View file

@ -18,6 +18,7 @@ package snapshot
import (
"bytes"
"cmp"
"fmt"
"slices"
"sort"
@ -45,13 +46,7 @@ func (it *weightedIterator) Cmp(other *weightedIterator) int {
return 1
}
// Same account/storage-slot in multiple layers, split by priority
if it.priority < other.priority {
return -1
}
if it.priority > other.priority {
return 1
}
return 0
return cmp.Compare(it.priority, other.priority)
}
// fastIterator is a more optimized multi-layer iterator which maintains a

12
core/state/state_object.go
View file

@ -399,10 +399,16 @@ func (s *stateObject) commitStorage(op *accountUpdate) {
op.storages = make(map[common.Hash][]byte)
}
op.storages[hash] = encode(val)
if op.storagesOrigin == nil {
op.storagesOrigin = make(map[common.Hash][]byte)
if op.storagesOriginByKey == nil {
op.storagesOriginByKey = make(map[common.Hash][]byte)
}
op.storagesOrigin[hash] = encode(s.originStorage[key])
if op.storagesOriginByHash == nil {
op.storagesOriginByHash = make(map[common.Hash][]byte)
}
origin := encode(s.originStorage[key])
op.storagesOriginByKey[key] = origin
op.storagesOriginByHash[hash] = origin
// Overwrite the clean value of storage slots
s.originStorage[key] = val

6
core/state/state_test.go
View file

@ -56,7 +56,7 @@ func TestDump(t *testing.T) {
// write some of them to the trie
s.state.updateStateObject(obj1)
s.state.updateStateObject(obj2)
root, _ := s.state.Commit(0, false)
root, _ := s.state.Commit(0, false, false)
// check that DumpToCollector contains the state objects that are in trie
s.state, _ = New(root, tdb)
@ -116,7 +116,7 @@ func TestIterativeDump(t *testing.T) {
// write some of them to the trie
s.state.updateStateObject(obj1)
s.state.updateStateObject(obj2)
root, _ := s.state.Commit(0, false)
root, _ := s.state.Commit(0, false, false)
s.state, _ = New(root, tdb)
b := &bytes.Buffer{}
@ -142,7 +142,7 @@ func TestNull(t *testing.T) {
var value common.Hash
s.state.SetState(address, common.Hash{}, value)
s.state.Commit(0, false)
s.state.Commit(0, false, false)
if value := s.state.GetState(address, common.Hash{}); value != (common.Hash{}) {
t.Errorf("expected empty current value, got %x", value)

25
core/state/statedb.go
View file

@ -1051,7 +1051,7 @@ func (s *StateDB) deleteStorage(addr common.Address, addrHash common.Hash, root
// with their values be tracked as original value.
// In case (d), **original** account along with its storages should be deleted,
// with their values be tracked as original value.
func (s *StateDB) handleDestruction() (map[common.Hash]*accountDelete, []*trienode.NodeSet, error) {
func (s *StateDB) handleDestruction(noStorageWiping bool) (map[common.Hash]*accountDelete, []*trienode.NodeSet, error) {
var (
nodes []*trienode.NodeSet
buf = crypto.NewKeccakState()
@ -1080,6 +1080,9 @@ func (s *StateDB) handleDestruction() (map[common.Hash]*accountDelete, []*trieno
if prev.Root == types.EmptyRootHash || s.db.TrieDB().IsVerkle() {
continue
}
if noStorageWiping {
return nil, nil, fmt.Errorf("unexpected storage wiping, %x", addr)
}
// Remove storage slots belonging to the account.
storages, storagesOrigin, set, err := s.deleteStorage(addr, addrHash, prev.Root)
if err != nil {
@ -1101,7 +1104,7 @@ func (s *StateDB) GetTrie() Trie {
// commit gathers the state mutations accumulated along with the associated
// trie changes, resetting all internal flags with the new state as the base.
func (s *StateDB) commit(deleteEmptyObjects bool) (*stateUpdate, error) {
func (s *StateDB) commit(deleteEmptyObjects bool, noStorageWiping bool) (*stateUpdate, error) {
// Short circuit in case any database failure occurred earlier.
if s.dbErr != nil {
return nil, fmt.Errorf("commit aborted due to earlier error: %v", s.dbErr)
@ -1155,7 +1158,7 @@ func (s *StateDB) commit(deleteEmptyObjects bool) (*stateUpdate, error) {
// the same block, account deletions must be processed first. This ensures
// that the storage trie nodes deleted during destruction and recreated
// during subsequent resurrection can be combined correctly.
deletes, delNodes, err := s.handleDestruction()
deletes, delNodes, err := s.handleDestruction(noStorageWiping)
if err != nil {
return nil, err
}
@ -1252,13 +1255,14 @@ func (s *StateDB) commit(deleteEmptyObjects bool) (*stateUpdate, error) {
origin := s.originalRoot
s.originalRoot = root
return newStateUpdate(origin, root, deletes, updates, nodes), nil
return newStateUpdate(noStorageWiping, origin, root, deletes, updates, nodes), nil
}
// commitAndFlush is a wrapper of commit which also commits the state mutations
// to the configured data stores.
func (s *StateDB) commitAndFlush(block uint64, deleteEmptyObjects bool) (*stateUpdate, error) {
ret, err := s.commit(deleteEmptyObjects)
func (s *StateDB) commitAndFlush(block uint64, deleteEmptyObjects bool, noStorageWiping bool) (*stateUpdate, error) {
ret, err := s.commit(deleteEmptyObjects, noStorageWiping)
if err != nil {
return nil, err
}
@ -1310,8 +1314,13 @@ func (s *StateDB) commitAndFlush(block uint64, deleteEmptyObjects bool) (*stateU
//
// The associated block number of the state transition is also provided
// for more chain context.
func (s *StateDB) Commit(block uint64, deleteEmptyObjects bool) (common.Hash, error) {
ret, err := s.commitAndFlush(block, deleteEmptyObjects)
//
// noStorageWiping is a flag indicating whether storage wiping is permitted.
// Since self-destruction was deprecated with the Cancun fork and there are
// no empty accounts left that could be deleted by EIP-158, storage wiping
// should not occur.
func (s *StateDB) Commit(block uint64, deleteEmptyObjects bool, noStorageWiping bool) (common.Hash, error) {
ret, err := s.commitAndFlush(block, deleteEmptyObjects, noStorageWiping)
if err != nil {
return common.Hash{}, err
}

2
core/state/statedb_fuzz_test.go
View file

@ -228,7 +228,7 @@ func (test *stateTest) run() bool {
} else {
state.IntermediateRoot(true) // call intermediateRoot at the transaction boundary
}
ret, err := state.commitAndFlush(0, true) // call commit at the block boundary
ret, err := state.commitAndFlush(0, true, false) // call commit at the block boundary
if err != nil {
panic(err)
}

4
core/state/statedb_hooked.go
View file

@ -157,6 +157,10 @@ func (s *hookedStateDB) Witness() *stateless.Witness {
return s.inner.Witness()
}
func (s *hookedStateDB) AccessEvents() *AccessEvents {
return s.inner.AccessEvents()
}
func (s *hookedStateDB) SubBalance(addr common.Address, amount *uint256.Int, reason tracing.BalanceChangeReason) uint256.Int {
prev := s.inner.SubBalance(addr, amount, reason)
if s.hooks.OnBalanceChange != nil && !amount.IsZero() {

2
core/state/statedb_hooked_test.go
View file

@ -71,7 +71,7 @@ func TestBurn(t *testing.T) {
hooked.AddBalance(addC, uint256.NewInt(200), tracing.BalanceChangeUnspecified)
hooked.Finalise(true)
s.Commit(0, false)
s.Commit(0, false, false)
if have, want := burned, uint256.NewInt(600); !have.Eq(want) {
t.Fatalf("burn-count wrong, have %v want %v", have, want)
}

29
core/state/statedb_test.go
View file

@ -119,7 +119,7 @@ func TestIntermediateLeaks(t *testing.T) {
}
// Commit and cross check the databases.
transRoot, err := transState.Commit(0, false)
transRoot, err := transState.Commit(0, false, false)
if err != nil {
t.Fatalf("failed to commit transition state: %v", err)
}
@ -127,7 +127,7 @@ func TestIntermediateLeaks(t *testing.T) {
t.Errorf("can not commit trie %v to persistent database", transRoot.Hex())
}
finalRoot, err := finalState.Commit(0, false)
finalRoot, err := finalState.Commit(0, false, false)
if err != nil {
t.Fatalf("failed to commit final state: %v", err)
}
@ -240,7 +240,7 @@ func TestCopyWithDirtyJournal(t *testing.T) {
obj.data.Root = common.HexToHash("0xdeadbeef")
orig.updateStateObject(obj)
}
root, _ := orig.Commit(0, true)
root, _ := orig.Commit(0, true, false)
orig, _ = New(root, db)
// modify all in memory without finalizing
@ -293,7 +293,7 @@ func TestCopyObjectState(t *testing.T) {
t.Fatalf("Error in test itself, the 'done' flag should not be set before Commit, have %v want %v", have, want)
}
}
orig.Commit(0, true)
orig.Commit(0, true, false)
for _, op := range cpy.mutations {
if have, want := op.applied, false; have != want {
t.Fatalf("Error: original state affected copy, have %v want %v", have, want)
@ -696,7 +696,7 @@ func (test *snapshotTest) checkEqual(state, checkstate *StateDB) error {
func TestTouchDelete(t *testing.T) {
s := newStateEnv()
s.state.getOrNewStateObject(common.Address{})
root, _ := s.state.Commit(0, false)
root, _ := s.state.Commit(0, false, false)
s.state, _ = New(root, s.state.db)
snapshot := s.state.Snapshot()
@ -784,7 +784,7 @@ func TestCopyCommitCopy(t *testing.T) {
t.Fatalf("second copy committed storage slot mismatch: have %x, want %x", val, sval)
}
// Commit state, ensure states can be loaded from disk
root, _ := state.Commit(0, false)
root, _ := state.Commit(0, false, false)
state, _ = New(root, tdb)
if balance := state.GetBalance(addr); balance.Cmp(uint256.NewInt(42)) != 0 {
t.Fatalf("state post-commit balance mismatch: have %v, want %v", balance, 42)
@ -898,11 +898,11 @@ func TestCommitCopy(t *testing.T) {
if val := state.GetCommittedState(addr, skey1); val != (common.Hash{}) {
t.Fatalf("initial committed storage slot mismatch: have %x, want %x", val, common.Hash{})
}
root, _ := state.Commit(0, true)
root, _ := state.Commit(0, true, false)
state, _ = New(root, db)
state.SetState(addr, skey2, sval2)
state.Commit(1, true)
state.Commit(1, true, false)
// Copy the committed state database, the copied one is not fully functional.
copied := state.Copy()
@ -943,7 +943,7 @@ func TestDeleteCreateRevert(t *testing.T) {
addr := common.BytesToAddress([]byte("so"))
state.SetBalance(addr, uint256.NewInt(1), tracing.BalanceChangeUnspecified)
root, _ := state.Commit(0, false)
root, _ := state.Commit(0, false, false)
state, _ = New(root, state.db)
// Simulate self-destructing in one transaction, then create-reverting in another
@ -955,7 +955,7 @@ func TestDeleteCreateRevert(t *testing.T) {
state.RevertToSnapshot(id)
// Commit the entire state and make sure we don't crash and have the correct state
root, _ = state.Commit(0, true)
root, _ = state.Commit(0, true, false)
state, _ = New(root, state.db)
if state.getStateObject(addr) != nil {
@ -998,7 +998,7 @@ func testMissingTrieNodes(t *testing.T, scheme string) {
a2 := common.BytesToAddress([]byte("another"))
state.SetBalance(a2, uint256.NewInt(100), tracing.BalanceChangeUnspecified)
state.SetCode(a2, []byte{1, 2, 4})
root, _ = state.Commit(0, false)
root, _ = state.Commit(0, false, false)
t.Logf("root: %x", root)
// force-flush
tdb.Commit(root, false)
@ -1022,7 +1022,7 @@ func testMissingTrieNodes(t *testing.T, scheme string) {
}
// Modify the state
state.SetBalance(addr, uint256.NewInt(2), tracing.BalanceChangeUnspecified)
root, err := state.Commit(0, false)
root, err := state.Commit(0, false, false)
if err == nil {
t.Fatalf("expected error, got root :%x", root)
}
@ -1213,7 +1213,7 @@ func TestFlushOrderDataLoss(t *testing.T) {
state.SetState(common.Address{a}, common.Hash{a, s}, common.Hash{a, s})
}
}
root, err := state.Commit(0, false)
root, err := state.Commit(0, false, false)
if err != nil {
t.Fatalf("failed to commit state trie: %v", err)
}
@ -1288,8 +1288,7 @@ func TestDeleteStorage(t *testing.T) {
value := common.Hash(uint256.NewInt(uint64(10 * i)).Bytes32())
state.SetState(addr, slot, value)
}
root, _ := state.Commit(0, true)
root, _ := state.Commit(0, true, false)
// Init phase done, create two states, one with snap and one without
fastState, _ := New(root, NewDatabase(tdb, snaps))
slowState, _ := New(root, NewDatabase(tdb, nil))

91
core/state/stateupdate.go
View file

@ -20,7 +20,6 @@ import (
"maps"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/trie/trienode"
"github.com/ethereum/go-ethereum/triedb"
)
@ -33,34 +32,56 @@ type contractCode struct {
// accountDelete represents an operation for deleting an Ethereum account.
type accountDelete struct {
address common.Address // address is the unique account identifier
origin []byte // origin is the original value of account data in slim-RLP encoding.
storages map[common.Hash][]byte // storages stores mutated slots, the value should be nil.
storagesOrigin map[common.Hash][]byte // storagesOrigin stores the original values of mutated slots in prefix-zero-trimmed RLP format.
address common.Address // address is the unique account identifier
origin []byte // origin is the original value of account data in slim-RLP encoding.
// storages stores mutated slots, the value should be nil.
storages map[common.Hash][]byte
// storagesOrigin stores the original values of mutated slots in
// prefix-zero-trimmed RLP format. The map key refers to the **HASH**
// of the raw storage slot key.
storagesOrigin map[common.Hash][]byte
}
// accountUpdate represents an operation for updating an Ethereum account.
type accountUpdate struct {
address common.Address // address is the unique account identifier
data []byte // data is the slim-RLP encoded account data.
origin []byte // origin is the original value of account data in slim-RLP encoding.
code *contractCode // code represents mutated contract code; nil means it's not modified.
storages map[common.Hash][]byte // storages stores mutated slots in prefix-zero-trimmed RLP format.
storagesOrigin map[common.Hash][]byte // storagesOrigin stores the original values of mutated slots in prefix-zero-trimmed RLP format.
address common.Address // address is the unique account identifier
data []byte // data is the slim-RLP encoded account data.
origin []byte // origin is the original value of account data in slim-RLP encoding.
code *contractCode // code represents mutated contract code; nil means it's not modified.
storages map[common.Hash][]byte // storages stores mutated slots in prefix-zero-trimmed RLP format.
// storagesOriginByKey and storagesOriginByHash both store the original values
// of mutated slots in prefix-zero-trimmed RLP format. The difference is that
// storagesOriginByKey uses the **raw** storage slot key as the map ID, while
// storagesOriginByHash uses the **hash** of the storage slot key instead.
storagesOriginByKey map[common.Hash][]byte
storagesOriginByHash map[common.Hash][]byte
}
// stateUpdate represents the difference between two states resulting from state
// execution. It contains information about mutated contract codes, accounts,
// and storage slots, along with their original values.
type stateUpdate struct {
originRoot common.Hash // hash of the state before applying mutation
root common.Hash // hash of the state after applying mutation
accounts map[common.Hash][]byte // accounts stores mutated accounts in 'slim RLP' encoding
accountsOrigin map[common.Address][]byte // accountsOrigin stores the original values of mutated accounts in 'slim RLP' encoding
storages map[common.Hash]map[common.Hash][]byte // storages stores mutated slots in 'prefix-zero-trimmed' RLP format
storagesOrigin map[common.Address]map[common.Hash][]byte // storagesOrigin stores the original values of mutated slots in 'prefix-zero-trimmed' RLP format
codes map[common.Address]contractCode // codes contains the set of dirty codes
nodes *trienode.MergedNodeSet // Aggregated dirty nodes caused by state changes
originRoot common.Hash // hash of the state before applying mutation
root common.Hash // hash of the state after applying mutation
accounts map[common.Hash][]byte // accounts stores mutated accounts in 'slim RLP' encoding
accountsOrigin map[common.Address][]byte // accountsOrigin stores the original values of mutated accounts in 'slim RLP' encoding
// storages stores mutated slots in 'prefix-zero-trimmed' RLP format.
// The value is keyed by account hash and **storage slot key hash**.
storages map[common.Hash]map[common.Hash][]byte
// storagesOrigin stores the original values of mutated slots in
// 'prefix-zero-trimmed' RLP format.
// (a) the value is keyed by account hash and **storage slot key** if rawStorageKey is true;
// (b) the value is keyed by account hash and **storage slot key hash** if rawStorageKey is false;
storagesOrigin map[common.Address]map[common.Hash][]byte
rawStorageKey bool
codes map[common.Address]contractCode // codes contains the set of dirty codes
nodes *trienode.MergedNodeSet // Aggregated dirty nodes caused by state changes
}
// empty returns a flag indicating the state transition is empty or not.
@ -68,10 +89,13 @@ func (sc *stateUpdate) empty() bool {
return sc.originRoot == sc.root
}
// newStateUpdate constructs a state update object, representing the differences
// between two states by performing state execution. It aggregates the given
// account deletions and account updates to form a comprehensive state update.
func newStateUpdate(originRoot common.Hash, root common.Hash, deletes map[common.Hash]*accountDelete, updates map[common.Hash]*accountUpdate, nodes *trienode.MergedNodeSet) *stateUpdate {
// newStateUpdate constructs a state update object by identifying the differences
// between two states through state execution. It combines the specified account
// deletions and account updates to create a complete state update.
//
// rawStorageKey is a flag indicating whether to use the raw storage slot key or
// the hash of the slot key for constructing state update object.
func newStateUpdate(rawStorageKey bool, originRoot common.Hash, root common.Hash, deletes map[common.Hash]*accountDelete, updates map[common.Hash]*accountUpdate, nodes *trienode.MergedNodeSet) *stateUpdate {
var (
accounts = make(map[common.Hash][]byte)
accountsOrigin = make(map[common.Address][]byte)
@ -79,13 +103,14 @@ func newStateUpdate(originRoot common.Hash, root common.Hash, deletes map[common
storagesOrigin = make(map[common.Address]map[common.Hash][]byte)
codes = make(map[common.Address]contractCode)
)
// Due to the fact that some accounts could be destructed and resurrected
// within the same block, the deletions must be aggregated first.
// Since some accounts might be destroyed and recreated within the same
// block, deletions must be aggregated first.
for addrHash, op := range deletes {
addr := op.address
accounts[addrHash] = nil
accountsOrigin[addr] = op.origin
// If storage wiping exists, the hash of the storage slot key must be used
if len(op.storages) > 0 {
storages[addrHash] = op.storages
}
@ -119,12 +144,16 @@ func newStateUpdate(originRoot common.Hash, root common.Hash, deletes map[common
}
// Aggregate the storage original values. If the slot is already present
// in aggregated storagesOrigin set, skip it.
if len(op.storagesOrigin) > 0 {
storageOriginSet := op.storagesOriginByHash
if rawStorageKey {
storageOriginSet = op.storagesOriginByKey
}
if len(storageOriginSet) > 0 {
origin, exist := storagesOrigin[addr]
if !exist {
storagesOrigin[addr] = op.storagesOrigin
storagesOrigin[addr] = storageOriginSet
} else {
for key, slot := range op.storagesOrigin {
for key, slot := range storageOriginSet {
if _, found := origin[key]; !found {
origin[key] = slot
}
@ -133,12 +162,13 @@ func newStateUpdate(originRoot common.Hash, root common.Hash, deletes map[common
}
}
return &stateUpdate{
originRoot: types.TrieRootHash(originRoot),
root: types.TrieRootHash(root),
originRoot: originRoot,
root: root,
accounts: accounts,
accountsOrigin: accountsOrigin,
storages: storages,
storagesOrigin: storagesOrigin,
rawStorageKey: rawStorageKey,
codes: codes,
nodes: nodes,
}
@ -154,5 +184,6 @@ func (sc *stateUpdate) stateSet() *triedb.StateSet {
AccountsOrigin: sc.accountsOrigin,
Storages: sc.storages,
StoragesOrigin: sc.storagesOrigin,
RawStorageKey: sc.rawStorageKey,
}
}

2
core/state/sync_test.go
View file

@ -79,7 +79,7 @@ func makeTestState(scheme string) (ethdb.Database, Database, *triedb.Database, c
}
accounts = append(accounts, acc)
}
root, _ := state.Commit(0, false)
root, _ := state.Commit(0, false, false)
// Return the generated state
return db, sdb, nodeDb, root, accounts

2
core/state/trie_prefetcher_test.go
View file

@ -83,7 +83,7 @@ func TestVerklePrefetcher(t *testing.T) {
state.SetBalance(addr, uint256.NewInt(42), tracing.BalanceChangeUnspecified) // Change the account trie
state.SetCode(addr, []byte("hello")) // Change an external metadata
state.SetState(addr, skey, sval) // Change the storage trie
root, _ := state.Commit(0, true)
root, _ := state.Commit(0, true, false)
state, _ = New(root, sdb)
sRoot := state.GetStorageRoot(addr)

24
core/state_processor.go
View file

@ -85,7 +85,7 @@ func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg
if beaconRoot := block.BeaconRoot(); beaconRoot != nil {
ProcessBeaconBlockRoot(*beaconRoot, evm)
}
if p.config.IsPrague(block.Number(), block.Time()) {
if p.config.IsPrague(block.Number(), block.Time()) || p.config.IsVerkle(block.Number(), block.Time()) {
ProcessParentBlockHash(block.ParentHash(), evm)
}
@ -155,6 +155,12 @@ func ApplyTransactionWithEVM(msg *Message, gp *GasPool, statedb *state.StateDB,
}
*usedGas += result.UsedGas
// Merge the tx-local access event into the "block-local" one, in order to collect
// all values, so that the witness can be built.
if statedb.GetTrie().IsVerkle() {
statedb.AccessEvents().Merge(evm.AccessEvents)
}
return MakeReceipt(evm, result, statedb, blockNumber, blockHash, tx, *usedGas, root), nil
}
@ -181,12 +187,6 @@ func MakeReceipt(evm *vm.EVM, result *ExecutionResult, statedb *state.StateDB, b
receipt.ContractAddress = crypto.CreateAddress(evm.TxContext.Origin, tx.Nonce())
}
// Merge the tx-local access event into the "block-local" one, in order to collect
// all values, so that the witness can be built.
if statedb.GetTrie().IsVerkle() {
statedb.AccessEvents().Merge(evm.AccessEvents)
}
// Set the receipt logs and create the bloom filter.
receipt.Logs = statedb.GetLogs(tx.Hash(), blockNumber.Uint64(), blockHash)
receipt.Bloom = types.CreateBloom(types.Receipts{receipt})
@ -234,7 +234,7 @@ func ProcessBeaconBlockRoot(beaconRoot common.Hash, evm *vm.EVM) {
}
// ProcessParentBlockHash stores the parent block hash in the history storage contract
// as per EIP-2935.
// as per EIP-2935/7709.
func ProcessParentBlockHash(prevHash common.Hash, evm *vm.EVM) {
if tracer := evm.Config.Tracer; tracer != nil {
onSystemCallStart(tracer, evm.GetVMContext())
@ -253,7 +253,13 @@ func ProcessParentBlockHash(prevHash common.Hash, evm *vm.EVM) {
}
evm.SetTxContext(NewEVMTxContext(msg))
evm.StateDB.AddAddressToAccessList(params.HistoryStorageAddress)
_, _, _ = evm.Call(vm.AccountRef(msg.From), *msg.To, msg.Data, 30_000_000, common.U2560)
_, _, err := evm.Call(vm.AccountRef(msg.From), *msg.To, msg.Data, 30_000_000, common.U2560)
if err != nil {
panic(err)
}
if evm.StateDB.AccessEvents() != nil {
evm.StateDB.AccessEvents().Merge(evm.AccessEvents)
}
evm.StateDB.Finalise(true)
}

33
core/state_processor_test.go
View file

@ -46,25 +46,7 @@ func u64(val uint64) *uint64 { return &val }
// contain invalid transactions
func TestStateProcessorErrors(t *testing.T) {
var (
config = &params.ChainConfig{
ChainID: big.NewInt(1),
HomesteadBlock: big.NewInt(0),
EIP150Block: big.NewInt(0),
EIP155Block: big.NewInt(0),
EIP158Block: big.NewInt(0),
ByzantiumBlock: big.NewInt(0),
ConstantinopleBlock: big.NewInt(0),
PetersburgBlock: big.NewInt(0),
IstanbulBlock: big.NewInt(0),
MuirGlacierBlock: big.NewInt(0),
BerlinBlock: big.NewInt(0),
LondonBlock: big.NewInt(0),
Ethash: new(params.EthashConfig),
TerminalTotalDifficulty: big.NewInt(0),
ShanghaiTime: new(uint64),
CancunTime: new(uint64),
PragueTime: new(uint64),
}
config = params.MergedTestChainConfig
signer = types.LatestSigner(config)
key1, _ = crypto.HexToECDSA("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291")
key2, _ = crypto.HexToECDSA("0202020202020202020202020202020202020202020202020202002020202020")
@ -111,7 +93,7 @@ func TestStateProcessorErrors(t *testing.T) {
}
return tx
}
var mkSetCodeTx = func(nonce uint64, to common.Address, gasLimit uint64, gasTipCap, gasFeeCap *big.Int, authlist []types.Authorization) *types.Transaction {
var mkSetCodeTx = func(nonce uint64, to common.Address, gasLimit uint64, gasTipCap, gasFeeCap *big.Int, authlist []types.SetCodeAuthorization) *types.Transaction {
tx, err := types.SignTx(types.NewTx(&types.SetCodeTx{
Nonce: nonce,
GasTipCap: uint256.MustFromBig(gasTipCap),
@ -251,9 +233,9 @@ func TestStateProcessorErrors(t *testing.T) {
},
{ // ErrMaxInitCodeSizeExceeded
txs: []*types.Transaction{
mkDynamicCreationTx(0, 500000, common.Big0, big.NewInt(params.InitialBaseFee), tooBigInitCode[:]),
mkDynamicCreationTx(0, 520000, common.Big0, big.NewInt(params.InitialBaseFee), tooBigInitCode[:]),
},
want: "could not apply tx 0 [0xd491405f06c92d118dd3208376fcee18a57c54bc52063ee4a26b1cf296857c25]: max initcode size exceeded: code size 49153 limit 49152",
want: "could not apply tx 0 [0x3a30404d42d6ccc843d7c391fd0c87b9b9795a0c174261b46d2ac95ca17b81cd]: max initcode size exceeded: code size 49153 limit 49152",
},
{ // ErrIntrinsicGas: Not enough gas to cover init code
txs: []*types.Transaction{
@ -425,12 +407,7 @@ func GenerateBadBlock(parent *types.Block, engine consensus.Engine, txs types.Tr
}
header.Root = common.BytesToHash(hasher.Sum(nil))
if config.IsCancun(header.Number, header.Time) {
var pExcess, pUsed = uint64(0), uint64(0)
if parent.ExcessBlobGas() != nil {
pExcess = *parent.ExcessBlobGas()
pUsed = *parent.BlobGasUsed()
}
excess := eip4844.CalcExcessBlobGas(pExcess, pUsed)
excess := eip4844.CalcExcessBlobGas(config, parent.Header())
used := uint64(nBlobs * params.BlobTxBlobGasPerBlob)
header.ExcessBlobGas = &excess
header.BlobGasUsed = &used

158
core/state_transition.go
View file

@ -17,6 +17,7 @@
package core
import (
"bytes"
"fmt"
"math"
"math/big"
@ -67,7 +68,7 @@ func (result *ExecutionResult) Revert() []byte {
}
// IntrinsicGas computes the 'intrinsic gas' for a message with the given data.
func IntrinsicGas(data []byte, accessList types.AccessList, authList []types.Authorization, isContractCreation, isHomestead, isEIP2028, isEIP3860 bool) (uint64, error) {
func IntrinsicGas(data []byte, accessList types.AccessList, authList []types.SetCodeAuthorization, isContractCreation, isHomestead, isEIP2028, isEIP3860 bool) (uint64, error) {
// Set the starting gas for the raw transaction
var gas uint64
if isContractCreation && isHomestead {
@ -79,12 +80,9 @@ func IntrinsicGas(data []byte, accessList types.AccessList, authList []types.Aut
// Bump the required gas by the amount of transactional data
if dataLen > 0 {
// Zero and non-zero bytes are priced differently
var nz uint64
for _, byt := range data {
if byt != 0 {
nz++
}
}
z := uint64(bytes.Count(data, []byte{0}))
nz := dataLen - z
// Make sure we don't exceed uint64 for all data combinations
nonZeroGas := params.TxDataNonZeroGasFrontier
if isEIP2028 {
@ -95,7 +93,6 @@ func IntrinsicGas(data []byte, accessList types.AccessList, authList []types.Aut
}
gas += nz * nonZeroGas
z := dataLen - nz
if (math.MaxUint64-gas)/params.TxDataZeroGas < z {
return 0, ErrGasUintOverflow
}
@ -119,6 +116,21 @@ func IntrinsicGas(data []byte, accessList types.AccessList, authList []types.Aut
return gas, nil
}
// FloorDataGas computes the minimum gas required for a transaction based on its data tokens (EIP-7623).
func FloorDataGas(data []byte) (uint64, error) {
var (
z = uint64(bytes.Count(data, []byte{0}))
nz = uint64(len(data)) - z
tokens = nz*params.TxTokenPerNonZeroByte + z
)
// Check for overflow
if (math.MaxUint64-params.TxGas)/params.TxCostFloorPerToken < tokens {
return 0, ErrGasUintOverflow
}
// Minimum gas required for a transaction based on its data tokens (EIP-7623).
return params.TxGas + tokens*params.TxCostFloorPerToken, nil
}
// toWordSize returns the ceiled word size required for init code payment calculation.
func toWordSize(size uint64) uint64 {
if size > math.MaxUint64-31 {
@ -131,19 +143,19 @@ func toWordSize(size uint64) uint64 {
// A Message contains the data derived from a single transaction that is relevant to state
// processing.
type Message struct {
To *common.Address
From common.Address
Nonce uint64
Value *big.Int
GasLimit uint64
GasPrice *big.Int
GasFeeCap *big.Int
GasTipCap *big.Int
Data []byte
AccessList types.AccessList
BlobGasFeeCap *big.Int
BlobHashes []common.Hash
AuthList []types.Authorization
To *common.Address
From common.Address
Nonce uint64
Value *big.Int
GasLimit uint64
GasPrice *big.Int
GasFeeCap *big.Int
GasTipCap *big.Int
Data []byte
AccessList types.AccessList
BlobGasFeeCap *big.Int
BlobHashes []common.Hash
SetCodeAuthorizations []types.SetCodeAuthorization
// When SkipNonceChecks is true, the message nonce is not checked against the
// account nonce in state.
@ -157,20 +169,20 @@ type Message struct {
// TransactionToMessage converts a transaction into a Message.
func TransactionToMessage(tx *types.Transaction, s types.Signer, baseFee *big.Int) (*Message, error) {
msg := &Message{
Nonce: tx.Nonce(),
GasLimit: tx.Gas(),
GasPrice: new(big.Int).Set(tx.GasPrice()),
GasFeeCap: new(big.Int).Set(tx.GasFeeCap()),
GasTipCap: new(big.Int).Set(tx.GasTipCap()),
To: tx.To(),
Value: tx.Value(),
Data: tx.Data(),
AccessList: tx.AccessList(),
AuthList: tx.AuthList(),
SkipNonceChecks: false,
SkipFromEOACheck: false,
BlobHashes: tx.BlobHashes(),
BlobGasFeeCap: tx.BlobGasFeeCap(),
Nonce: tx.Nonce(),
GasLimit: tx.Gas(),
GasPrice: new(big.Int).Set(tx.GasPrice()),
GasFeeCap: new(big.Int).Set(tx.GasFeeCap()),
GasTipCap: new(big.Int).Set(tx.GasTipCap()),
To: tx.To(),
Value: tx.Value(),
Data: tx.Data(),
AccessList: tx.AccessList(),
SetCodeAuthorizations: tx.SetCodeAuthorizations(),
SkipNonceChecks: false,
SkipFromEOACheck: false,
BlobHashes: tx.BlobHashes(),
BlobGasFeeCap: tx.BlobGasFeeCap(),
}
// If baseFee provided, set gasPrice to effectiveGasPrice.
if baseFee != nil {
@ -372,11 +384,11 @@ func (st *stateTransition) preCheck() error {
}
}
// Check that EIP-7702 authorization list signatures are well formed.
if msg.AuthList != nil {
if msg.SetCodeAuthorizations != nil {
if msg.To == nil {
return fmt.Errorf("%w (sender %v)", ErrSetCodeTxCreate, msg.From)
}
if len(msg.AuthList) == 0 {
if len(msg.SetCodeAuthorizations) == 0 {
return fmt.Errorf("%w (sender %v)", ErrEmptyAuthList, msg.From)
}
}
@ -414,16 +426,27 @@ func (st *stateTransition) execute() (*ExecutionResult, error) {
sender = vm.AccountRef(msg.From)
rules = st.evm.ChainConfig().Rules(st.evm.Context.BlockNumber, st.evm.Context.Random != nil, st.evm.Context.Time)
contractCreation = msg.To == nil
floorDataGas uint64
)
// Check clauses 4-5, subtract intrinsic gas if everything is correct
gas, err := IntrinsicGas(msg.Data, msg.AccessList, msg.AuthList, contractCreation, rules.IsHomestead, rules.IsIstanbul, rules.IsShanghai)
gas, err := IntrinsicGas(msg.Data, msg.AccessList, msg.SetCodeAuthorizations, contractCreation, rules.IsHomestead, rules.IsIstanbul, rules.IsShanghai)
if err != nil {
return nil, err
}
if st.gasRemaining < gas {
return nil, fmt.Errorf("%w: have %d, want %d", ErrIntrinsicGas, st.gasRemaining, gas)
}
// Gas limit suffices for the floor data cost (EIP-7623)
if rules.IsPrague {
floorDataGas, err = FloorDataGas(msg.Data)
if err != nil {
return nil, err
}
if msg.GasLimit < floorDataGas {
return nil, fmt.Errorf("%w: have %d, want %d", ErrFloorDataGas, msg.GasLimit, floorDataGas)
}
}
if t := st.evm.Config.Tracer; t != nil && t.OnGasChange != nil {
t.OnGasChange(st.gasRemaining, st.gasRemaining-gas, tracing.GasChangeTxIntrinsicGas)
}
@ -467,10 +490,10 @@ func (st *stateTransition) execute() (*ExecutionResult, error) {
st.state.SetNonce(msg.From, st.state.GetNonce(msg.From)+1, tracing.NonceChangeEoACall)
// Apply EIP-7702 authorizations.
if msg.AuthList != nil {
for _, auth := range msg.AuthList {
if msg.SetCodeAuthorizations != nil {
for _, auth := range msg.SetCodeAuthorizations {
// Note errors are ignored, we simply skip invalid authorizations here.
st.applyAuthorization(msg, &auth)
st.applyAuthorization(&auth)
}
}
@ -487,14 +510,21 @@ func (st *stateTransition) execute() (*ExecutionResult, error) {
ret, st.gasRemaining, vmerr = st.evm.Call(sender, st.to(), msg.Data, st.gasRemaining, value)
}
var gasRefund uint64
if !rules.IsLondon {
// Before EIP-3529: refunds were capped to gasUsed / 2
gasRefund = st.refundGas(params.RefundQuotient)
} else {
// After EIP-3529: refunds are capped to gasUsed / 5
gasRefund = st.refundGas(params.RefundQuotientEIP3529)
// Compute refund counter, capped to a refund quotient.
gasRefund := st.calcRefund()
st.gasRemaining += gasRefund
if rules.IsPrague {
// After EIP-7623: Data-heavy transactions pay the floor gas.
if st.gasUsed() < floorDataGas {
prev := st.gasRemaining
st.gasRemaining = st.initialGas - floorDataGas
if t := st.evm.Config.Tracer; t != nil && t.OnGasChange != nil {
t.OnGasChange(prev, st.gasRemaining, tracing.GasChangeTxDataFloor)
}
}
}
st.returnGas()
effectiveTip := msg.GasPrice
if rules.IsLondon {
effectiveTip = new(big.Int).Sub(msg.GasFeeCap, st.evm.Context.BaseFee)
@ -528,9 +558,9 @@ func (st *stateTransition) execute() (*ExecutionResult, error) {
}
// validateAuthorization validates an EIP-7702 authorization against the state.
func (st *stateTransition) validateAuthorization(auth *types.Authorization) (authority common.Address, err error) {
// Verify chain ID is 0 or equal to current chain ID.
if auth.ChainID != 0 && st.evm.ChainConfig().ChainID.Uint64() != auth.ChainID {
func (st *stateTransition) validateAuthorization(auth *types.SetCodeAuthorization) (authority common.Address, err error) {
// Verify chain ID is null or equal to current chain ID.
if !auth.ChainID.IsZero() && auth.ChainID.CmpBig(st.evm.ChainConfig().ChainID) != 0 {
return authority, ErrAuthorizationWrongChainID
}
// Limit nonce to 2^64-1 per EIP-2681.
@ -559,7 +589,7 @@ func (st *stateTransition) validateAuthorization(auth *types.Authorization) (aut
}
// applyAuthorization applies an EIP-7702 code delegation to the state.
func (st *stateTransition) applyAuthorization(msg *Message, auth *types.Authorization) error {
func (st *stateTransition) applyAuthorization(auth *types.SetCodeAuthorization) error {
authority, err := st.validateAuthorization(auth)
if err != nil {
return err
@ -585,20 +615,28 @@ func (st *stateTransition) applyAuthorization(msg *Message, auth *types.Authoriz
return nil
}
func (st *stateTransition) refundGas(refundQuotient uint64) uint64 {
// Apply refund counter, capped to a refund quotient
refund := st.gasUsed() / refundQuotient
// calcRefund computes refund counter, capped to a refund quotient.
func (st *stateTransition) calcRefund() uint64 {
var refund uint64
if !st.evm.ChainConfig().IsLondon(st.evm.Context.BlockNumber) {
// Before EIP-3529: refunds were capped to gasUsed / 2
refund = st.gasUsed() / params.RefundQuotient
} else {
// After EIP-3529: refunds are capped to gasUsed / 5
refund = st.gasUsed() / params.RefundQuotientEIP3529
}
if refund > st.state.GetRefund() {
refund = st.state.GetRefund()
}
if st.evm.Config.Tracer != nil && st.evm.Config.Tracer.OnGasChange != nil && refund > 0 {
st.evm.Config.Tracer.OnGasChange(st.gasRemaining, st.gasRemaining+refund, tracing.GasChangeTxRefunds)
}
return refund
}
st.gasRemaining += refund
// Return ETH for remaining gas, exchanged at the original rate.
// returnGas returns ETH for remaining gas,
// exchanged at the original rate.
func (st *stateTransition) returnGas() {
remaining := uint256.NewInt(st.gasRemaining)
remaining.Mul(remaining, uint256.MustFromBig(st.msg.GasPrice))
st.state.AddBalance(st.msg.From, remaining, tracing.BalanceIncreaseGasReturn)
@ -610,8 +648,6 @@ func (st *stateTransition) refundGas(refundQuotient uint64) uint64 {
// Also return remaining gas to the block gas counter so it is
// available for the next transaction.
st.gp.AddGas(st.gasRemaining)
return refund
}
// gasUsed returns the amount of gas used up by the state transition.

8
core/tracing/CHANGELOG.md
View file

@ -56,6 +56,14 @@ The state changes that are covered by the journaling library are:
- `GasChangeReason` has been extended with the following reasons which will be enabled only post-Verkle. There shouldn't be any gas changes with those reasons prior to the fork.
- `GasChangeWitnessContractCollisionCheck` flags the event of adding to the witness when checking for contract address collision.
## [v1.14.12]
This release contains a change in behavior for `OnCodeChange` hook.
### `OnCodeChange` change
The `OnCodeChange` hook is now called when the code of a contract is removed due to a selfdestruct. Previously, no code change was emitted on such occasions.
## [v1.14.4]
This release contained only minor extensions to the tracing interface.

6
core/tracing/hooks.go
View file

@ -63,7 +63,6 @@ type VMContext struct {
// It contains the block as well as consensus related information.
type BlockEvent struct {
Block *types.Block
TD *big.Int
Finalized *types.Header
Safe *types.Header
}
@ -314,7 +313,7 @@ const (
GasChangeCallLeftOverRefunded GasChangeReason = 7
// GasChangeCallContractCreation is the amount of gas that will be burned for a CREATE.
GasChangeCallContractCreation GasChangeReason = 8
// GasChangeContractCreation is the amount of gas that will be burned for a CREATE2.
// GasChangeCallContractCreation2 is the amount of gas that will be burned for a CREATE2.
GasChangeCallContractCreation2 GasChangeReason = 9
// GasChangeCallCodeStorage is the amount of gas that will be charged for code storage.
GasChangeCallCodeStorage GasChangeReason = 10
@ -335,6 +334,9 @@ const (
GasChangeWitnessCodeChunk GasChangeReason = 17
// GasChangeWitnessContractCollisionCheck flags the event of adding to the witness when checking for contract address collision.
GasChangeWitnessContractCollisionCheck GasChangeReason = 18
// GasChangeTxDataFloor is the amount of extra gas the transaction has to pay to reach the minimum gas requirement for the
// transaction data. This change will always be a negative change.
GasChangeTxDataFloor GasChangeReason = 19
// GasChangeIgnored is a special value that can be used to indicate that the gas change should be ignored as
// it will be "manually" tracked by a direct emit of the gas change event.

2
core/txindexer_test.go
View file

@ -211,7 +211,7 @@ func TestTxIndexer(t *testing.T) {
}
for _, c := range cases {
db, _ := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), "", "", false)
rawdb.WriteAncientBlocks(db, append([]*types.Block{gspec.ToBlock()}, blocks...), append([]types.Receipts{{}}, receipts...), big.NewInt(0))
rawdb.WriteAncientBlocks(db, append([]*types.Block{gspec.ToBlock()}, blocks...), append([]types.Receipts{{}}, receipts...))
// Index the initial blocks from ancient store
indexer := &txIndexer{

24
core/txpool/blobpool/blobpool.go
View file

@ -51,11 +51,6 @@ const (
// transaction. There can be multiple of these embedded into a single tx.
blobSize = params.BlobTxFieldElementsPerBlob * params.BlobTxBytesPerFieldElement
// maxBlobsPerTransaction is the maximum number of blobs a single transaction
// is allowed to contain. Whilst the spec states it's unlimited, the block
// data slots are protocol bound, which implicitly also limit this.
maxBlobsPerTransaction = params.MaxBlobGasPerBlock / params.BlobTxBlobGasPerBlob
// txAvgSize is an approximate byte size of a transaction metadata to avoid
// tiny overflows causing all txs to move a shelf higher, wasting disk space.
txAvgSize = 4 * 1024
@ -223,6 +218,11 @@ func newBlobTxMeta(id uint64, size uint32, tx *types.Transaction) *blobTxMeta {
// very relaxed ones can be included even if the fees go up, when the closer
// ones could already be invalid.
//
// - Because the maximum number of blobs allowed in a block can change per
// fork, the pool is designed to handle the maximum number of blobs allowed
// in the chain's latest defined fork -- even if it isn't active. This
// avoids needing to upgrade the database around the fork boundary.
//
// When the pool eventually reaches saturation, some old transactions - that may
// never execute - will need to be evicted in favor of newer ones. The eviction
// strategy is quite complex:
@ -387,7 +387,8 @@ func (p *BlobPool) Init(gasTip uint64, head *types.Header, reserve txpool.Addres
fails = append(fails, id)
}
}
store, err := billy.Open(billy.Options{Path: queuedir, Repair: true}, newSlotter(), index)
slotter := newSlotter(eip4844.LatestMaxBlobsPerBlock(p.chain.Config()))
store, err := billy.Open(billy.Options{Path: queuedir, Repair: true}, slotter, index)
if err != nil {
return err
}
@ -414,13 +415,13 @@ func (p *BlobPool) Init(gasTip uint64, head *types.Header, reserve txpool.Addres
blobfee = uint256.NewInt(params.BlobTxMinBlobGasprice)
)
if p.head.ExcessBlobGas != nil {
blobfee = uint256.MustFromBig(eip4844.CalcBlobFee(*p.head.ExcessBlobGas))
blobfee = uint256.MustFromBig(eip4844.CalcBlobFee(p.chain.Config(), p.head))
}
p.evict = newPriceHeap(basefee, blobfee, p.index)
// Pool initialized, attach the blob limbo to it to track blobs included
// recently but not yet finalized
p.limbo, err = newLimbo(limbodir)
p.limbo, err = newLimbo(limbodir, eip4844.LatestMaxBlobsPerBlock(p.chain.Config()))
if err != nil {
p.Close()
return err
@ -834,7 +835,7 @@ func (p *BlobPool) Reset(oldHead, newHead *types.Header) {
blobfee = uint256.MustFromBig(big.NewInt(params.BlobTxMinBlobGasprice))
)
if newHead.ExcessBlobGas != nil {
blobfee = uint256.MustFromBig(eip4844.CalcBlobFee(*newHead.ExcessBlobGas))
blobfee = uint256.MustFromBig(eip4844.CalcBlobFee(p.chain.Config(), newHead))
}
p.evict.reinit(basefee, blobfee, false)
@ -1598,7 +1599,8 @@ func (p *BlobPool) updateStorageMetrics() {
metrics.GetOrRegisterGauge(fmt.Sprintf(shelfSlotusedGaugeName, shelf.SlotSize/blobSize), nil).Update(int64(shelf.FilledSlots))
metrics.GetOrRegisterGauge(fmt.Sprintf(shelfSlotgapsGaugeName, shelf.SlotSize/blobSize), nil).Update(int64(shelf.GappedSlots))
if shelf.SlotSize/blobSize > maxBlobsPerTransaction {
maxBlobs := eip4844.LatestMaxBlobsPerBlock(p.chain.Config())
if shelf.SlotSize/blobSize > uint32(maxBlobs) {
oversizedDataused += slotDataused
oversizedDatagaps += slotDatagaps
oversizedSlotused += shelf.FilledSlots
@ -1751,7 +1753,7 @@ func (p *BlobPool) Clear() {
// The transaction addition may attempt to reserve the sender addr which
// can't happen until Clear releases the reservation lock. Clear cannot
// acquire the subpool lock until the transaction addition is completed.
for acct, _ := range p.index {
for acct := range p.index {
p.reserve(acct, false)
}
p.lookup = newLookup()

170
core/txpool/blobpool/blobpool_test.go
View file

@ -51,8 +51,10 @@ var (
testBlobVHashes [][32]byte
)
const testMaxBlobsPerBlock = 6
func init() {
for i := 0; i < 10; i++ {
for i := 0; i < 24; i++ {
testBlob := &kzg4844.Blob{byte(i)}
testBlobs = append(testBlobs, testBlob)
@ -121,7 +123,12 @@ func (bc *testBlockChain) CurrentBlock() *types.Header {
mid := new(big.Int).Add(lo, hi)
mid.Div(mid, big.NewInt(2))
if eip4844.CalcBlobFee(mid.Uint64()).Cmp(bc.blobfee.ToBig()) > 0 {
tmp := mid.Uint64()
if eip4844.CalcBlobFee(bc.Config(), &types.Header{
Number: blockNumber,
Time: blockTime,
ExcessBlobGas: &tmp,
}).Cmp(bc.blobfee.ToBig()) > 0 {
hi = mid
} else {
lo = mid
@ -194,10 +201,43 @@ func makeTx(nonce uint64, gasTipCap uint64, gasFeeCap uint64, blobFeeCap uint64,
return types.MustSignNewTx(key, types.LatestSigner(params.MainnetChainConfig), blobtx)
}
// makeMultiBlobTx is a utility method to construct a ramdom blob tx with
// certain number of blobs in its sidecar.
func makeMultiBlobTx(nonce uint64, gasTipCap uint64, gasFeeCap uint64, blobFeeCap uint64, blobCount int, key *ecdsa.PrivateKey) *types.Transaction {
var (
blobs []kzg4844.Blob
blobHashes []common.Hash
commitments []kzg4844.Commitment
proofs []kzg4844.Proof
)
for i := 0; i < blobCount; i++ {
blobs = append(blobs, *testBlobs[i])
commitments = append(commitments, testBlobCommits[i])
proofs = append(proofs, testBlobProofs[i])
blobHashes = append(blobHashes, testBlobVHashes[i])
}
blobtx := &types.BlobTx{
ChainID: uint256.MustFromBig(params.MainnetChainConfig.ChainID),
Nonce: nonce,
GasTipCap: uint256.NewInt(gasTipCap),
GasFeeCap: uint256.NewInt(gasFeeCap),
Gas: 21000,
BlobFeeCap: uint256.NewInt(blobFeeCap),
BlobHashes: blobHashes,
Value: uint256.NewInt(100),
Sidecar: &types.BlobTxSidecar{
Blobs: blobs,
Commitments: commitments,
Proofs: proofs,
},
}
return types.MustSignNewTx(key, types.LatestSigner(params.MainnetChainConfig), blobtx)
}
// makeUnsignedTx is a utility method to construct a random blob transaction
// without signing it.
func makeUnsignedTx(nonce uint64, gasTipCap uint64, gasFeeCap uint64, blobFeeCap uint64) *types.BlobTx {
return makeUnsignedTxWithTestBlob(nonce, gasTipCap, gasFeeCap, blobFeeCap, rand.Intn(len(testBlobs)))
return makeUnsignedTxWithTestBlob(nonce, gasTipCap, gasFeeCap, blobFeeCap, rnd.Intn(len(testBlobs)))
}
// makeUnsignedTx is a utility method to construct a random blob transaction
@ -415,7 +455,7 @@ func TestOpenDrops(t *testing.T) {
defer os.RemoveAll(storage)
os.MkdirAll(filepath.Join(storage, pendingTransactionStore), 0700)
store, _ := billy.Open(billy.Options{Path: filepath.Join(storage, pendingTransactionStore)}, newSlotter(), nil)
store, _ := billy.Open(billy.Options{Path: filepath.Join(storage, pendingTransactionStore)}, newSlotter(testMaxBlobsPerBlock), nil)
// Insert a malformed transaction to verify that decoding errors (or format
// changes) are handled gracefully (case 1)
@ -650,7 +690,7 @@ func TestOpenDrops(t *testing.T) {
statedb.AddBalance(crypto.PubkeyToAddress(overcapper.PublicKey), uint256.NewInt(10000000), tracing.BalanceChangeUnspecified)
statedb.AddBalance(crypto.PubkeyToAddress(duplicater.PublicKey), uint256.NewInt(1000000), tracing.BalanceChangeUnspecified)
statedb.AddBalance(crypto.PubkeyToAddress(repeater.PublicKey), uint256.NewInt(1000000), tracing.BalanceChangeUnspecified)
statedb.Commit(0, true)
statedb.Commit(0, true, false)
chain := &testBlockChain{
config: params.MainnetChainConfig,
@ -738,7 +778,7 @@ func TestOpenIndex(t *testing.T) {
defer os.RemoveAll(storage)
os.MkdirAll(filepath.Join(storage, pendingTransactionStore), 0700)
store, _ := billy.Open(billy.Options{Path: filepath.Join(storage, pendingTransactionStore)}, newSlotter(), nil)
store, _ := billy.Open(billy.Options{Path: filepath.Join(storage, pendingTransactionStore)}, newSlotter(testMaxBlobsPerBlock), nil)
// Insert a sequence of transactions with varying price points to check that
// the cumulative minimum will be maintained.
@ -769,7 +809,7 @@ func TestOpenIndex(t *testing.T) {
// Create a blob pool out of the pre-seeded data
statedb, _ := state.New(types.EmptyRootHash, state.NewDatabaseForTesting())
statedb.AddBalance(addr, uint256.NewInt(1_000_000_000), tracing.BalanceChangeUnspecified)
statedb.Commit(0, true)
statedb.Commit(0, true, false)
chain := &testBlockChain{
config: params.MainnetChainConfig,
@ -827,7 +867,7 @@ func TestOpenHeap(t *testing.T) {
defer os.RemoveAll(storage)
os.MkdirAll(filepath.Join(storage, pendingTransactionStore), 0700)
store, _ := billy.Open(billy.Options{Path: filepath.Join(storage, pendingTransactionStore)}, newSlotter(), nil)
store, _ := billy.Open(billy.Options{Path: filepath.Join(storage, pendingTransactionStore)}, newSlotter(testMaxBlobsPerBlock), nil)
// Insert a few transactions from a few accounts. To remove randomness from
// the heap initialization, use a deterministic account/tx/priority ordering.
@ -871,7 +911,7 @@ func TestOpenHeap(t *testing.T) {
statedb.AddBalance(addr1, uint256.NewInt(1_000_000_000), tracing.BalanceChangeUnspecified)
statedb.AddBalance(addr2, uint256.NewInt(1_000_000_000), tracing.BalanceChangeUnspecified)
statedb.AddBalance(addr3, uint256.NewInt(1_000_000_000), tracing.BalanceChangeUnspecified)
statedb.Commit(0, true)
statedb.Commit(0, true, false)
chain := &testBlockChain{
config: params.MainnetChainConfig,
@ -914,7 +954,7 @@ func TestOpenCap(t *testing.T) {
defer os.RemoveAll(storage)
os.MkdirAll(filepath.Join(storage, pendingTransactionStore), 0700)
store, _ := billy.Open(billy.Options{Path: filepath.Join(storage, pendingTransactionStore)}, newSlotter(), nil)
store, _ := billy.Open(billy.Options{Path: filepath.Join(storage, pendingTransactionStore)}, newSlotter(testMaxBlobsPerBlock), nil)
// Insert a few transactions from a few accounts
var (
@ -951,7 +991,7 @@ func TestOpenCap(t *testing.T) {
statedb.AddBalance(addr1, uint256.NewInt(1_000_000_000), tracing.BalanceChangeUnspecified)
statedb.AddBalance(addr2, uint256.NewInt(1_000_000_000), tracing.BalanceChangeUnspecified)
statedb.AddBalance(addr3, uint256.NewInt(1_000_000_000), tracing.BalanceChangeUnspecified)
statedb.Commit(0, true)
statedb.Commit(0, true, false)
chain := &testBlockChain{
config: params.MainnetChainConfig,
@ -992,6 +1032,108 @@ func TestOpenCap(t *testing.T) {
}
}
// TestChangingSlotterSize attempts to mimic a scenario where the max blob count
// of the pool is increased. This would happen during a client release where a
// new fork is added with a max blob count higher than the previous fork. We
// want to make sure transactions a persisted between those runs.
func TestChangingSlotterSize(t *testing.T) {
//log.SetDefault(log.NewLogger(log.NewTerminalHandlerWithLevel(os.Stderr, log.LevelTrace, true)))
// Create a temporary folder for the persistent backend
storage, _ := os.MkdirTemp("", "blobpool-")
defer os.RemoveAll(storage)
os.MkdirAll(filepath.Join(storage, pendingTransactionStore), 0700)
store, _ := billy.Open(billy.Options{Path: filepath.Join(storage, pendingTransactionStore)}, newSlotter(6), nil)
// Create transactions from a few accounts.
var (
key1, _ = crypto.GenerateKey()
key2, _ = crypto.GenerateKey()
key3, _ = crypto.GenerateKey()
addr1 = crypto.PubkeyToAddress(key1.PublicKey)
addr2 = crypto.PubkeyToAddress(key2.PublicKey)
addr3 = crypto.PubkeyToAddress(key3.PublicKey)
tx1 = makeMultiBlobTx(0, 1, 1000, 100, 6, key1)
tx2 = makeMultiBlobTx(0, 1, 800, 70, 6, key2)
tx3 = makeMultiBlobTx(0, 1, 800, 110, 24, key3)
blob1, _ = rlp.EncodeToBytes(tx1)
blob2, _ = rlp.EncodeToBytes(tx2)
)
// Write the two safely sized txs to store. note: although the store is
// configured for a blob count of 6, it can also support around ~1mb of call
// data - all this to say that we aren't using the the absolute largest shelf
// available.
store.Put(blob1)
store.Put(blob2)
store.Close()
// Mimic a blobpool with max blob count of 6 upgrading to a max blob count of 24.
for _, maxBlobs := range []int{6, 24} {
statedb, _ := state.New(types.EmptyRootHash, state.NewDatabaseForTesting())
statedb.AddBalance(addr1, uint256.NewInt(1_000_000_000), tracing.BalanceChangeUnspecified)
statedb.AddBalance(addr2, uint256.NewInt(1_000_000_000), tracing.BalanceChangeUnspecified)
statedb.AddBalance(addr3, uint256.NewInt(1_000_000_000), tracing.BalanceChangeUnspecified)
statedb.Commit(0, true, false)
// Make custom chain config where the max blob count changes based on the loop variable.
cancunTime := uint64(0)
config := &params.ChainConfig{
ChainID: big.NewInt(1),
LondonBlock: big.NewInt(0),
BerlinBlock: big.NewInt(0),
CancunTime: &cancunTime,
BlobScheduleConfig: &params.BlobScheduleConfig{
Cancun: &params.BlobConfig{
Target: maxBlobs / 2,
Max: maxBlobs,
UpdateFraction: params.DefaultCancunBlobConfig.UpdateFraction,
},
},
}
chain := &testBlockChain{
config: config,
basefee: uint256.NewInt(1050),
blobfee: uint256.NewInt(105),
statedb: statedb,
}
pool := New(Config{Datadir: storage}, chain)
if err := pool.Init(1, chain.CurrentBlock(), makeAddressReserver()); err != nil {
t.Fatalf("failed to create blob pool: %v", err)
}
// Try to add the big blob tx. In the initial iteration it should overflow
// the pool. On the subsequent iteration it should be accepted.
errs := pool.Add([]*types.Transaction{tx3}, false, true)
if _, ok := pool.index[addr3]; ok && maxBlobs == 6 {
t.Errorf("expected insert of oversized blob tx to fail: blobs=24, maxBlobs=%d, err=%v", maxBlobs, errs[0])
} else if !ok && maxBlobs == 10 {
t.Errorf("expected insert of oversized blob tx to succeed: blobs=24, maxBlobs=%d, err=%v", maxBlobs, errs[0])
}
// Verify the regular two txs are always available.
if got := pool.Get(tx1.Hash()); got == nil {
t.Errorf("expected tx %s from %s in pool", tx1.Hash(), addr1)
}
if got := pool.Get(tx2.Hash()); got == nil {
t.Errorf("expected tx %s from %s in pool", tx2.Hash(), addr2)
}
// Verify all the calculated pool internals. Interestingly, this is **not**
// a duplication of the above checks, this actually validates the verifier
// using the above already hard coded checks.
//
// Do not remove this, nor alter the above to be generic.
verifyPoolInternals(t, pool)
pool.Close()
}
}
// Tests that adding transaction will correctly store it in the persistent store
// and update all the indices.
//
@ -1369,7 +1511,7 @@ func TestAdd(t *testing.T) {
defer os.RemoveAll(storage) // late defer, still ok
os.MkdirAll(filepath.Join(storage, pendingTransactionStore), 0700)
store, _ := billy.Open(billy.Options{Path: filepath.Join(storage, pendingTransactionStore)}, newSlotter(), nil)
store, _ := billy.Open(billy.Options{Path: filepath.Join(storage, pendingTransactionStore)}, newSlotter(testMaxBlobsPerBlock), nil)
// Insert the seed transactions for the pool startup
var (
@ -1393,7 +1535,7 @@ func TestAdd(t *testing.T) {
store.Put(blob)
}
}
statedb.Commit(0, true)
statedb.Commit(0, true, false)
store.Close()
// Create a blob pool out of the pre-seeded dats
@ -1519,7 +1661,7 @@ func benchmarkPoolPending(b *testing.B, datacap uint64) {
statedb.AddBalance(addr, uint256.NewInt(1_000_000_000), tracing.BalanceChangeUnspecified)
pool.add(tx)
}
statedb.Commit(0, true)
statedb.Commit(0, true, false)
defer pool.Close()
// Benchmark assembling the pending

34
core/txpool/blobpool/evictheap_test.go
View file

@ -26,7 +26,7 @@ import (
"github.com/holiman/uint256"
)
var rand = mrand.New(mrand.NewSource(1))
var rnd = mrand.New(mrand.NewSource(1))
// verifyHeapInternals verifies that all accounts present in the index are also
// present in the heap and internals are consistent across various indices.
@ -193,12 +193,12 @@ func benchmarkPriceHeapReinit(b *testing.B, datacap uint64) {
index := make(map[common.Address][]*blobTxMeta)
for i := 0; i < int(blobs); i++ {
var addr common.Address
rand.Read(addr[:])
rnd.Read(addr[:])
var (
execTip = uint256.NewInt(rand.Uint64())
execFee = uint256.NewInt(rand.Uint64())
blobFee = uint256.NewInt(rand.Uint64())
execTip = uint256.NewInt(rnd.Uint64())
execFee = uint256.NewInt(rnd.Uint64())
blobFee = uint256.NewInt(rnd.Uint64())
basefeeJumps = dynamicFeeJumps(execFee)
blobfeeJumps = dynamicFeeJumps(blobFee)
@ -218,13 +218,13 @@ func benchmarkPriceHeapReinit(b *testing.B, datacap uint64) {
}}
}
// Create a price heap and reinit it over and over
heap := newPriceHeap(uint256.NewInt(rand.Uint64()), uint256.NewInt(rand.Uint64()), index)
heap := newPriceHeap(uint256.NewInt(rnd.Uint64()), uint256.NewInt(rnd.Uint64()), index)
basefees := make([]*uint256.Int, b.N)
blobfees := make([]*uint256.Int, b.N)
for i := 0; i < b.N; i++ {
basefees[i] = uint256.NewInt(rand.Uint64())
blobfees[i] = uint256.NewInt(rand.Uint64())
basefees[i] = uint256.NewInt(rnd.Uint64())
blobfees[i] = uint256.NewInt(rnd.Uint64())
}
b.ResetTimer()
b.ReportAllocs()
@ -269,12 +269,12 @@ func benchmarkPriceHeapOverflow(b *testing.B, datacap uint64) {
index := make(map[common.Address][]*blobTxMeta)
for i := 0; i < int(blobs); i++ {
var addr common.Address
rand.Read(addr[:])
rnd.Read(addr[:])
var (
execTip = uint256.NewInt(rand.Uint64())
execFee = uint256.NewInt(rand.Uint64())
blobFee = uint256.NewInt(rand.Uint64())
execTip = uint256.NewInt(rnd.Uint64())
execFee = uint256.NewInt(rnd.Uint64())
blobFee = uint256.NewInt(rnd.Uint64())
basefeeJumps = dynamicFeeJumps(execFee)
blobfeeJumps = dynamicFeeJumps(blobFee)
@ -294,18 +294,18 @@ func benchmarkPriceHeapOverflow(b *testing.B, datacap uint64) {
}}
}
// Create a price heap and overflow it over and over
evict := newPriceHeap(uint256.NewInt(rand.Uint64()), uint256.NewInt(rand.Uint64()), index)
evict := newPriceHeap(uint256.NewInt(rnd.Uint64()), uint256.NewInt(rnd.Uint64()), index)
var (
addrs = make([]common.Address, b.N)
metas = make([]*blobTxMeta, b.N)
)
for i := 0; i < b.N; i++ {
rand.Read(addrs[i][:])
rnd.Read(addrs[i][:])
var (
execTip = uint256.NewInt(rand.Uint64())
execFee = uint256.NewInt(rand.Uint64())
blobFee = uint256.NewInt(rand.Uint64())
execTip = uint256.NewInt(rnd.Uint64())
execFee = uint256.NewInt(rnd.Uint64())
blobFee = uint256.NewInt(rnd.Uint64())
basefeeJumps = dynamicFeeJumps(execFee)
blobfeeJumps = dynamicFeeJumps(blobFee)

4
core/txpool/blobpool/limbo.go
View file

@ -48,7 +48,7 @@ type limbo struct {
}
// newLimbo opens and indexes a set of limboed blob transactions.
func newLimbo(datadir string) (*limbo, error) {
func newLimbo(datadir string, maxBlobsPerTransaction int) (*limbo, error) {
l := &limbo{
index: make(map[common.Hash]uint64),
groups: make(map[uint64]map[uint64]common.Hash),
@ -60,7 +60,7 @@ func newLimbo(datadir string) (*limbo, error) {
fails = append(fails, id)
}
}
store, err := billy.Open(billy.Options{Path: datadir, Repair: true}, newSlotter(), index)
store, err := billy.Open(billy.Options{Path: datadir, Repair: true}, newSlotter(maxBlobsPerTransaction), index)
if err != nil {
return nil, err
}

6
core/txpool/blobpool/priority_test.go
View file

@ -52,7 +52,7 @@ func TestPriorityCalculation(t *testing.T) {
func BenchmarkDynamicFeeJumpCalculation(b *testing.B) {
fees := make([]*uint256.Int, b.N)
for i := 0; i < b.N; i++ {
fees[i] = uint256.NewInt(rand.Uint64())
fees[i] = uint256.NewInt(rnd.Uint64())
}
b.ResetTimer()
b.ReportAllocs()
@ -76,8 +76,8 @@ func BenchmarkPriorityCalculation(b *testing.B) {
txBasefeeJumps := make([]float64, b.N)
txBlobfeeJumps := make([]float64, b.N)
for i := 0; i < b.N; i++ {
txBasefeeJumps[i] = dynamicFeeJumps(uint256.NewInt(rand.Uint64()))
txBlobfeeJumps[i] = dynamicFeeJumps(uint256.NewInt(rand.Uint64()))
txBasefeeJumps[i] = dynamicFeeJumps(uint256.NewInt(rnd.Uint64()))
txBlobfeeJumps[i] = dynamicFeeJumps(uint256.NewInt(rnd.Uint64()))
}
b.ResetTimer()
b.ReportAllocs()

4
core/txpool/blobpool/slotter.go
View file

@ -25,13 +25,13 @@ package blobpool
// The slotter also creates a shelf for 0-blob transactions. Whilst those are not
// allowed in the current protocol, having an empty shelf is not a relevant use
// of resources, but it makes stress testing with junk transactions simpler.
func newSlotter() func() (uint32, bool) {
func newSlotter(maxBlobsPerTransaction int) func() (uint32, bool) {
slotsize := uint32(txAvgSize)
slotsize -= uint32(blobSize) // underflows, it's ok, will overflow back in the first return
return func() (size uint32, done bool) {
slotsize += blobSize
finished := slotsize > maxBlobsPerTransaction*blobSize+txMaxSize
finished := slotsize > uint32(maxBlobsPerTransaction)*blobSize+txMaxSize
return slotsize, finished
}

2
core/txpool/blobpool/slotter_test.go
View file

@ -21,7 +21,7 @@ import "testing"
// Tests that the slotter creates the expected database shelves.
func TestNewSlotter(t *testing.T) {
// Generate the database shelve sizes
slotter := newSlotter()
slotter := newSlotter(6)
var shelves []uint32
for {

5
core/txpool/legacypool/legacypool.go
View file

@ -1440,7 +1440,7 @@ func (pool *LegacyPool) reset(oldHead, newHead *types.Header) {
// Inject any transactions discarded due to reorgs
log.Debug("Reinjecting stale transactions", "count", len(reinject))
core.SenderCacher.Recover(pool.signer, reinject)
core.SenderCacher().Recover(pool.signer, reinject)
pool.addTxsLocked(reinject, false)
}
@ -1986,7 +1986,7 @@ func (pool *LegacyPool) Clear() {
senderAddr, _ := types.Sender(pool.signer, tx)
pool.reserve(senderAddr, false)
}
for localSender, _ := range pool.locals.accounts {
for localSender := range pool.locals.accounts {
pool.reserve(localSender, false)
}
@ -1994,6 +1994,7 @@ func (pool *LegacyPool) Clear() {
pool.priced = newPricedList(pool.all)
pool.pending = make(map[common.Address]*list)
pool.queue = make(map[common.Address]*list)
pool.pendingNonces = newNoncer(pool.currentState)
if !pool.config.NoLocals && pool.config.Journal != "" {
pool.journal = newTxJournal(pool.config.Journal)

Some files were not shown because too many files have changed in this diff Show more