forks/go-ethereum
1
0
Fork 1
mirror of https://github.com/ethereum/go-ethereum.git synced 2026-07-27 15:16:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Merge pull request #647 from cffls/qa

Browse source 
Merge qa to master for v0.3.2
This commit is contained in:
Arpit Temani 2023-01-05 12:03:50 +05:30 committed by GitHub
commit 3eb234cd9a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
214 changed files with 8953 additions and 1753 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

23
.github/CODEOWNERS vendored
View file

@ -1,23 +0,0 @@
# Lines starting with '#' are comments.
# Each line is a file pattern followed by one or more owners.
accounts/usbwallet @karalabe
accounts/scwallet @gballet
accounts/abi @gballet @MariusVanDerWijden
cmd/clef @holiman
cmd/puppeth @karalabe
consensus @karalabe
core/ @karalabe @holiman @rjl493456442
eth/ @karalabe @holiman @rjl493456442
eth/catalyst/ @gballet
graphql/ @gballet
les/ @zsfelfoldi @rjl493456442
light/ @zsfelfoldi @rjl493456442
mobile/ @karalabe @ligi
node/ @fjl @renaynay
p2p/ @fjl @zsfelfoldi
rpc/ @fjl @holiman
p2p/simulations @fjl
p2p/protocols @fjl
p2p/testing @fjl
signer/ @holiman

67
.github/ISSUE_TEMPLATE/bug.md vendored
View file

@ -1,30 +1,71 @@
---
name: Report a bug
about: Something with go-ethereum is not working as expected
about: Something with bor client is not working as expected
title: ''
labels: 'type:bug'
assignees: ''
---
#### System information
Our support team has aggregated some common issues and their solutions from past which are faced while running or interacting with a bor client. In order to prevent redundant efforts, we would encourage you to have a look at the [FAQ's section](https://docs.polygon.technology/docs/faq/technical-faqs) of our documentation mentioning the same, before filing an issue here. In case of additional support, you can also join our [discord](https://discord.com/invite/zdwkdvMNY2) server
Geth version: `geth version`
OS & Version: Windows/Linux/OSX
Commit hash : (if `develop`)
<!--
NOTE: Please make sure to check of any addresses / private keys / rpc url's / IP's before sharing the logs or anything from the additional information section (start.sh or heimdall config).
-->
#### Expected behaviour
#### **System information**
Bor client version: [e.g. v0.2.16] <!--Can be found by running the command `bor version`-->
#### Actual behaviour
Heimdall client version: [e.g. v0.2.10] <!--Can be found by running the command `heimdalld version`-->
OS & Version: Windows / Linux / OSX
#### Steps to reproduce the behaviour
Environment: Polygon Mainnet / Polygon Mumbai / Devnet
Type of node: Validator / Sentry / Archive
#### Backtrace
Additional Information: <!--Modifications in the client (if any)-->
````
[backtrace]
````
#### **Overview of the problem**
When submitting logs: please submit them as text and not screenshots.
Please describe the issue you experiencing.
<!--
Mention in detail about the issue. Also mention the actual and expected behaviour.
-->
#### **Reproduction Steps**
Please mention the steps required to reproduce this issue.
<!--
E.g.
1. Start bor using these flags.
2. Node is unable to connect with other peers in the network and keeps disconnecting.
-->
#### **Logs / Traces / Output / Error Messages**
Please post any logs/traces/output/error messages (as text and not screenshots) which you believe may have caused the issue. If the log is longer than a few dozen lines, please include the URL to the [gist](https://gist.github.com/) of the log instead of posting it in the issue.
**Additional Information**
In order to debug the issue faster, we would stongly encourage if you can provide some of the details mentioned below (whichever seems relevant to your issue)
1. Your `start.sh` file or `bor.service`, if you're facing some peering issue or unable to use some service (like `http` endpoint) as expected. Moreover, if possible mention the chain configuration printed while starting the node which looks something like `Initialised chain configuration config="{ChainID: 137, ..., Engine: bor}"`
<!--
It should be start.sh if you're using bor v0.2.x and bor.service (ideally located under `/lib/systemd/system/`) if it's bor v0.3.x. Mention this file if you're facing any issues like unable to use some flag/s according to their expected behaviour.
-->
2. The result of `eth.syncing`, `admin.peers.length`, `admin.nodeInfo`, value of the `maxpeers` flag in start.sh, and bootnodes/static nodes (if any) is you're facing some syncing issue.
<!--
You can get the above results by attaching to the IPC using the command `bor attach $BORDIR/bor.ipc` or `bor attach $DATADIR/bor.ipc` and running the mentioned commands.
Mention this if you're facing issues where bor keeps stalling and is not importing new blocks or making any progress. Adding chain configuration mentioned in the previous step would also be really helpful here as it might also be a genesis mismatch issue.
-->
3. Your `heimdall-config.toml` parameters for checking the ETH and BOR RPC url's, incase of issue with bor heimdall communication.
<!--
The location should be `~/.heimdalld/config/` if running heimdall v0.2.x and `/var/lib/heimdalld/config` if running heimdall v0.3.x.
As a sub-set of syncing issues, if your node keeps printing logs like `Retrying again in 5 seconds to fetch data from Heimdall`, it might be an issue with the communication between your bor node and heimdall node. In this case, also check if all the heimdall services (heimdalld, bridge, rest-server) are running correctly.
-->
4. The CURL request (for that specific error) if you're facing any issues or identify a bug while making RPC request.
<!--
Make sure you hide the IP of your machine if you're doing the request externally.
-->

7
.github/ISSUE_TEMPLATE/feature.md vendored
View file

@ -8,10 +8,11 @@ assignees: ''
# Rationale
Why should this feature exist?
The motivation behind the feature and why should this feature exist?
What are the use-cases?
# Implementation
Do you have ideas regarding the implementation of this feature?
Are you willing to implement this feature?
Do you have ideas regarding the implementation of this feature? (Mention reference links if any)
Any alternative solutions or features you've considered?
Are you willing to implement this feature?

8
.github/ISSUE_TEMPLATE/question.md vendored
View file

@ -1,9 +1,11 @@
---
name: Ask a question
about: Something is unclear
name: Question/Support
about: Ask a question or request support
title: ''
labels: 'type:docs'
assignees: ''
---
This should only be used in very rare cases e.g. if you are not 100% sure if something is a bug or asking a question that leads to improving the documentation. For general questions please use [discord](https://discord.gg/nthXNEv) or the Ethereum stack exchange at https://ethereum.stackexchange.com.
This should only be used in very rare cases e.g. if you are not 100% sure if something is a bug or asking a question that leads to improving the documentation.
For general questions please join our [discord](https://discord.com/invite/zdwkdvMNY2) server.

6
.github/matic-cli-config.yml vendored
View file

@ -3,9 +3,13 @@ defaultFee: 2000
borChainId: "15001"
heimdallChainId: heimdall-15001
contractsBranch: jc/v0.3.1-backport
sprintSize: 64
blockNumber: '0'
blockTime: '2'
numOfValidators: 3
numOfNonValidators: 0
ethURL: http://ganache:9545
ethHostUser: ubuntu
devnetType: docker
borDockerBuildContext: "../../bor"
heimdallDockerBuildContext: "https://github.com/maticnetwork/heimdall.git#develop"
heimdallDockerBuildContext: "https://github.com/maticnetwork/heimdall.git#develop"

44
.github/pull_request_template.md vendored Normal file
View file

@ -0,0 +1,44 @@
# Description
Please provide a detailed description of what was done in this PR
# Changes
- [ ] Bugfix (non-breaking change that solves an issue)
- [ ] Hotfix (change that solves an urgent issue, and requires immediate attention)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (change that is not backwards-compatible and/or changes current functionality)
# Breaking changes
Please complete this section if any breaking changes have been made, otherwise delete it
# Checklist
- [ ] I have added at least 2 reviewer or the whole pos-v1 team
- [ ] I have added sufficient documentation in code
- [ ] I will be resolving comments - if any - by pushing each fix in a separate commit and linking the commit hash in the comment reply
# Cross repository changes
- [ ] This PR requires changes to heimdall
- In case link the PR here:
- [ ] This PR requires changes to matic-cli
- In case link the PR here:
## Testing
- [ ] I have added unit tests
- [ ] I have added tests to CI
- [ ] I have tested this code manually on local environment
- [ ] I have tested this code manually on remote devnet using express-cli
- [ ] I have tested this code manually on mumbai
- [ ] I have created new e2e tests into express-cli
### Manual tests
Please complete this section with the steps you performed if you ran manual tests for this functionality, otherwise delete it
# Additional comments
Please post additional comments in this section if you have them, otherwise delete it

12
.github/workflows/ci.yml vendored
View file

@ -8,7 +8,7 @@ on:
pull_request:
branches:
- "**"
types: [opened, synchronize, edited]
types: [opened, synchronize]
concurrency:
group: build-${{ github.event.pull_request.number || github.ref }}
@ -19,7 +19,7 @@ jobs:
if: (github.event.action != 'closed' || github.event.pull_request.merged == true)
strategy:
matrix:
os: [ ubuntu-20.04, macos-11 ] # list of os: https://github.com/actions/virtual-environments
os: [ ubuntu-20.04 ] # list of os: https://github.com/actions/virtual-environments
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
@ -29,7 +29,7 @@ jobs:
- uses: actions/setup-go@v3
with:
go-version: 1.18.x
go-version: 1.19.x
- name: Install dependencies on Linux
if: runner.os == 'Linux'
@ -63,7 +63,7 @@ jobs:
- name: test-integration
run: make test-integration
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
@ -104,7 +104,7 @@ jobs:
uses: actions/checkout@v3
with:
repository: maticnetwork/matic-cli
ref: v0.3.0-dev
ref: arpit/pos-655-2
path: matic-cli
- name: Install dependencies on Linux
@ -154,7 +154,7 @@ jobs:
cd matic-cli/devnet/code/contracts
npm run truffle exec scripts/deposit.js -- --network development $(jq -r .root.tokens.MaticToken contractAddresses.json) 100000000000000000000
cd -
bash bor/integration-tests/smoke_test.sh
timeout 20m bash bor/integration-tests/smoke_test.sh
- name: Upload logs
if: always()

5
.github/workflows/packager.yml vendored
View file

@ -377,6 +377,9 @@ jobs:
- name: Removing systemd file
run: rm -rf packaging/deb/bor/lib/systemd/system/bor.service
- name: Updating the apt-get
run: sudo apt-get update -y
- name: Adding requirements for cross compile
run: sudo apt-get install g++-aarch64-linux-gnu gcc-aarch64-linux-gnu
@ -730,4 +733,4 @@ jobs:
prerelease: true
files: |
packaging/deb/bor**.deb
binary/bo**
binary/bo**

10
.github/workflows/release.yml vendored
View file

@ -21,7 +21,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@master
with:
go-version: 1.17.x
go-version: 1.19.x
- name: Prepare
id: prepare
@ -29,6 +29,12 @@ jobs:
TAG=${GITHUB_REF#refs/tags/}
echo ::set-output name=tag_name::${TAG}
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB }}
password: ${{ secrets.DOCKERHUB_KEY }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
@ -39,5 +45,3 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ steps.prepare.outputs.tag_name }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
DOCKER_USERNAME: ${{ secrets.DOCKERHUB }}
DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_KEY }}

2
.gitignore vendored
View file

@ -53,3 +53,5 @@ profile.cov
./bor-debug-*
dist
*.csv

3
.gitmodules vendored
View file

@ -6,3 +6,6 @@
path = tests/evm-benchmarks
url = https://github.com/ipsilon/evm-benchmarks
shallow = true
[submodule "tests/testdata"]
path = tests/testdata
url = https://github.com/ethereum/tests.git

1
.golangci.yml
View file

@ -50,6 +50,7 @@ linters:
- unconvert
- unparam
- wsl
- asasalint
#- errorlint causes stack overflow. TODO: recheck after each golangci update
linters-settings:

13
Makefile
View file

@ -34,6 +34,10 @@ bor:
protoc:
protoc --go_out=. --go-grpc_out=. ./internal/cli/server/proto/*.proto
generate-mocks:
go generate mockgen -destination=./tests/bor/mocks/IHeimdallClient.go -package=mocks ./consensus/bor IHeimdallClient
go generate mockgen -destination=./eth/filters/IBackend.go -package=filters ./eth/filters Backend
geth:
$(GORUN) build/ci.go install ./cmd/geth
@echo "Done building."
@ -61,7 +65,7 @@ test-race:
$(GOTEST) --timeout 15m -race -shuffle=on $(TESTALL)
test-integration:
$(GOTEST) --timeout 30m -tags integration $(TESTE2E)
$(GOTEST) --timeout 60m -tags integration $(TESTE2E)
escape:
cd $(path) && go test -gcflags "-m -m" -run none -bench=BenchmarkJumpdest* -benchmem -memprofile mem.out
@ -71,7 +75,7 @@ lint:
lintci-deps:
rm -f ./build/bin/golangci-lint
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b ./build/bin v1.46.0
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b ./build/bin v1.48.0
goimports:
goimports -local "$(PACKAGE)" -w .
@ -93,7 +97,7 @@ devtools:
$(GOBUILD) -o $(GOBIN)/codecgen github.com/ugorji/go/codec/codecgen
$(GOBUILD) -o $(GOBIN)/abigen ./cmd/abigen
$(GOBUILD) -o $(GOBIN)/mockgen github.com/golang/mock/mockgen
$(GOBUILD) -o $(GOBIN)/protoc-gen-go github.com/golang/protobuf/protoc-gen-go
$(GOBUILD) -o $(GOBIN)/protoc-gen-go google.golang.org/protobuf/cmd/protoc-gen-go
PATH=$(GOBIN):$(PATH) go generate ./common
PATH=$(GOBIN):$(PATH) go generate ./core/types
PATH=$(GOBIN):$(PATH) go generate ./consensus/bor
@ -192,7 +196,7 @@ geth-windows-amd64:
@ls -ld $(GOBIN)/geth-windows-* | grep amd64
PACKAGE_NAME := github.com/maticnetwork/bor
GOLANG_CROSS_VERSION ?= v1.18.1
GOLANG_CROSS_VERSION ?= v1.19.1
.PHONY: release-dry-run
release-dry-run:
@ -220,6 +224,7 @@ release:
-e DOCKER_PASSWORD \
-e SLACK_WEBHOOK \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $(HOME)/.docker/config.json:/root/.docker/config.json \
-v `pwd`:/go/src/$(PACKAGE_NAME) \
-w /go/src/$(PACKAGE_NAME) \
goreleaser/goreleaser-cross:${GOLANG_CROSS_VERSION} \

2
README.md
View file

@ -1,5 +1,5 @@
# Bor Overview
Bor is the Official Golang implementation of the Matic protocol. It is a fork of Go Ethereum - https://github.com/ethereum/go-ethereum and EVM compabile.
Bor is the Official Golang implementation of the Matic protocol. It is a fork of Go Ethereum - https://github.com/ethereum/go-ethereum and EVM compatible.
![Forks](https://img.shields.io/github/forks/maticnetwork/bor?style=social)
![Stars](https://img.shields.io/github/stars/maticnetwork/bor?style=social)

103
RETESTBOR.md Normal file
View file

@ -0,0 +1,103 @@
# Retesteth - bor
These integration tests are included in the bor repo via using the git submodule
```
[submodule "tests/testdata"]
path = tests/testdata
url = https://github.com/ethereum/tests.git
```
The version used is the last stable release, tagged as v10.4 from branch develop in ethereum/tests
Details on release code can be found here https://github.com/ethereum/tests/commit/a380655e5ffab1a5ea0f4d860224bdb19013f06a
To run the tests, we hava a `make` command:
```
make test-integration
```
which is also integrated into the CI pipeline on GitHub
## Retesteth - bor on remote machine
To explore and test the `retesteth` package, the following steps were executed.
This is only for educational purposes.
For future usage, there is no need to go through this section, the only thing needed is to have 'green' integration tests.
- `ssh` into a VM running bor
- Change configs by replacing geth with bor inside the docker container
```
mkdir ~/retestethBuild
cd ~/retestethBuild
wget https://raw.githubusercontent.com/ethereum/retesteth/develop/dretesteth.sh
chmod +x dretesteth.sh
wget https://raw.githubusercontent.com/ethereum/retesteth/develop/Dockerfile
```
Modify the RUN git clone line in the Dockerfile for repo “retesteth” to change branch -b from master to develop. Do not modify repo branches for “winsvega/solidity” [LLLC opcode support] and “go-ethereum”.
Modify the Dockerfile so that the eth client points to bor
e.g. : `https://github.com/ethereum/retesteth/blob/master/Dockerfile#L41`
from `RUN git clone --depth 1 -b master https://github.com/ethereum/go-ethereum.git /geth`
to: `RUN git clone --depth 1 -b master https://github.com/maticnetwork/bor.git /geth`
- build docker image
`sudo ./dretesteth.sh build`
- clone repo
```
git clone --branch develop https://github.com/ethereum/tests.git
```
this step is eventually replaced by adding the git submodule directly into bor repo with
```
git submodule add --depth 1 https://github.com/ethereum/tests.git tests/testdata
```
- Let's move to the restestethBuild folder
```
cd /home/ubuntu/retestethBuild
```
Now we have the tests repo here
```
ls
> Dockerfile dretesteth.sh tests
```
- Run test example
```
./dretesteth.sh -t GeneralStateTests/stExample -- --testpath /home/ubuntu/retestethBuild/tests --datadir /tests/config
```
This will create the config files for the different clients in ~/tests/config
Eventually. these config needs to be adapted according to the following doc
https://ethereum-tests.readthedocs.io/en/latest/retesteth-tutorial.html
Specifically:
```
f you look inside ~/tests/config, you’ll see a directory for each configured client. Typically this directory has these files:
config, which contains the configuration for the client:
The communication protocol to use with the client (typically TCP)
The address(es) to use with that protocol
The forks the client supports
The exceptions the client can throw, and how retesteth should interpret them. This is particularly important when testing the client’s behavior when given invalid blocks.
start.sh, which starts the client inside the docker image
stop.sh, which stops the client instance(s)
genesis, a directory which includes the genesis blocks for various forks the client supports. If this directory does not exist for a client, it uses the genesis blocks for the default client.
```
We replaced geth inside docker by using https://ethereum-tests.readthedocs.io/en/latest/retesteth-tutorial.html#replace-geth-inside-the-docker
Theoretically, we would not need any additional config change
- Run test suites
```
./dretesteth.sh -t <TestSuiteName> -- --testpath /home/ubuntu/retestethBuild/tests --datadir /tests/config
```
Where `TestSuiteName` is one of the maintained test suites, reported here https://github.com/ethereum/tests
```
BasicTests
BlockchainTests
GeneralStateTests
TransactionTests
RLPTest
src
```
If you want to run retestheth against a bor client on localhost:8545 (using 8 threads), instead of isolating it into a docker image, run
`sudo ./dretesteth.sh -t GeneralStateTests -- --testpath ~/tests --datadir /tests/config --clients t8ntool --nodes 127.0.0.1:8545 -j 8`

8
accounts/abi/bind/auth.go
View file

@ -21,7 +21,6 @@ import (
"crypto/ecdsa"
"errors"
"io"
"io/ioutil"
"math/big"
"github.com/ethereum/go-ethereum/accounts"
@ -45,14 +44,17 @@ var ErrNotAuthorized = errors.New("not authorized to sign this account")
// Deprecated: Use NewTransactorWithChainID instead.
func NewTransactor(keyin io.Reader, passphrase string) (*TransactOpts, error) {
log.Warn("WARNING: NewTransactor has been deprecated in favour of NewTransactorWithChainID")
json, err := ioutil.ReadAll(keyin)
json, err := io.ReadAll(keyin)
if err != nil {
return nil, err
}
key, err := keystore.DecryptKey(json, passphrase)
if err != nil {
return nil, err
}
return NewKeyedTransactor(key.PrivateKey), nil
}
@ -106,7 +108,7 @@ func NewKeyedTransactor(key *ecdsa.PrivateKey) *TransactOpts {
// NewTransactorWithChainID is a utility method to easily create a transaction signer from
// an encrypted json key stream and the associated passphrase.
func NewTransactorWithChainID(keyin io.Reader, passphrase string, chainID *big.Int) (*TransactOpts, error) {
json, err := ioutil.ReadAll(keyin)
json, err := io.ReadAll(keyin)
if err != nil {
return nil, err
}

2
accounts/abi/bind/backends/simulated.go
View file

@ -78,7 +78,7 @@ type SimulatedBackend struct {
func NewSimulatedBackendWithDatabase(database ethdb.Database, alloc core.GenesisAlloc, gasLimit uint64) *SimulatedBackend {
genesis := core.Genesis{Config: params.AllEthashProtocolChanges, GasLimit: gasLimit, Alloc: alloc}
genesis.MustCommit(database)
blockchain, _ := core.NewBlockChain(database, nil, genesis.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ := core.NewBlockChain(database, nil, genesis.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
backend := &SimulatedBackend{
database: database,

7
accounts/abi/bind/bind_test.go
View file

@ -18,7 +18,6 @@ package bind
import (
"fmt"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
@ -1966,7 +1965,7 @@ func TestGolangBindings(t *testing.T) {
t.Skip("go sdk not found for testing")
}
// Create a temporary workspace for the test suite
ws, err := ioutil.TempDir("", "binding-test")
ws, err := os.MkdirTemp("", "binding-test")
if err != nil {
t.Fatalf("failed to create temporary workspace: %v", err)
}
@ -1990,7 +1989,7 @@ func TestGolangBindings(t *testing.T) {
if err != nil {
t.Fatalf("test %d: failed to generate binding: %v", i, err)
}
if err = ioutil.WriteFile(filepath.Join(pkg, strings.ToLower(tt.name)+".go"), []byte(bind), 0600); err != nil {
if err = os.WriteFile(filepath.Join(pkg, strings.ToLower(tt.name)+".go"), []byte(bind), 0600); err != nil {
t.Fatalf("test %d: failed to write binding: %v", i, err)
}
// Generate the test file with the injected test code
@ -2006,7 +2005,7 @@ func TestGolangBindings(t *testing.T) {
%s
}
`, tt.imports, tt.name, tt.tester)
if err := ioutil.WriteFile(filepath.Join(pkg, strings.ToLower(tt.name)+"_test.go"), []byte(code), 0600); err != nil {
if err := os.WriteFile(filepath.Join(pkg, strings.ToLower(tt.name)+"_test.go"), []byte(code), 0600); err != nil {
t.Fatalf("test %d: failed to write tests: %v", i, err)
}
})

1
builder/files/config.toml
View file

@ -126,6 +126,7 @@ syncmode = "full"
# noprefetch = false
# preimages = false
# txlookuplimit = 2350000
# triesinmemory = 128
# timeout = "1h0m0s"
[accounts]

11
builder/files/genesis-mainnet-v1.json
View file

@ -15,11 +15,18 @@
"londonBlock": 23850000,
"bor": {
"jaipurBlock": 23850000,
"delhiBlock": 38189056,
"period": {
"0": 2
},
"producerDelay": 6,
"sprint": 64,
"producerDelay": {
"0": 6,
"38189056": 4
},
"sprint": {
"0": 64,
"38189056": 16
},
"backupMultiplier": {
"0": 2
},

17
builder/files/genesis-testnet-v4.json
View file

@ -15,15 +15,24 @@
"londonBlock": 22640000,
"bor": {
"jaipurBlock": 22770000,
"delhiBlock": 29638656,
"period": {
"0": 2,
"25275000": 5
"25275000": 5,
"29638656": 2
},
"producerDelay": {
"0": 6,
"29638656": 4
},
"sprint": {
"0": 64,
"29638656": 16
},
"producerDelay": 6,
"sprint": 64,
"backupMultiplier": {
"0": 2,
"25275000": 5
"25275000": 5,
"29638656": 2
},
"validatorContract": "0x0000000000000000000000000000000000001000",
"stateReceiverContract": "0x0000000000000000000000000000000000001001",

3
cmd/evm/internal/t8ntool/block.go
View file

@ -17,6 +17,7 @@
package t8ntool
import (
"context"
"crypto/ecdsa"
"encoding/json"
"errors"
@ -188,7 +189,7 @@ func (i *bbInput) sealEthash(block *types.Block) (*types.Block, error) {
// If the testmode is used, the sealer will return quickly, and complain
// "Sealing result is not read by miner" if it cannot write the result.
results := make(chan *types.Block, 1)
if err := engine.Seal(nil, block, results, nil); err != nil {
if err := engine.Seal(context.Background(), nil, block, results, nil); err != nil {
panic(fmt.Sprintf("failed to seal block: %v", err))
}
found := <-results

1
cmd/evm/internal/t8ntool/execution.go
View file

@ -223,6 +223,7 @@ func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
txIndex++
}
statedb.IntermediateRoot(chainConfig.IsEIP158(vmContext.BlockNumber))
// Add mining reward?
if miningReward > 0 {

1
cmd/geth/chaincmd.go
View file

@ -108,6 +108,7 @@ The dumpgenesis command dumps the genesis block configuration in JSON format to
// bor related flags
utils.HeimdallURLFlag,
utils.WithoutHeimdallFlag,
utils.HeimdallgRPCAddressFlag,
},
Category: "BLOCKCHAIN COMMANDS",
Description: `

6
cmd/geth/config.go
View file

@ -18,14 +18,12 @@ package main
import (
"fmt"
"io/ioutil"
"math/big"
"os"
"time"
"gopkg.in/urfave/cli.v1"
"github.com/BurntSushi/toml"
"gopkg.in/urfave/cli.v1"
"github.com/ethereum/go-ethereum/accounts/external"
"github.com/ethereum/go-ethereum/accounts/keystore"
@ -71,7 +69,7 @@ type gethConfig struct {
}
func loadConfig(file string, cfg *gethConfig) error {
data, err := ioutil.ReadFile(file)
data, err := os.ReadFile(file)
if err != nil {
return err
}

12
cmd/utils/bor_flags.go
View file

@ -5,11 +5,12 @@ import (
"io/ioutil"
"os"
"gopkg.in/urfave/cli.v1"
"github.com/ethereum/go-ethereum/core"
"github.com/ethereum/go-ethereum/eth"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/node"
"gopkg.in/urfave/cli.v1"
)
var (
@ -30,10 +31,18 @@ var (
Usage: "Run without Heimdall service (for testing purpose)",
}
// HeimdallgRPCAddressFlag flag for heimdall gRPC address
HeimdallgRPCAddressFlag = cli.StringFlag{
Name: "bor.heimdallgRPC",
Usage: "Address of Heimdall gRPC service",
Value: "",
}
// BorFlags all bor related flags
BorFlags = []cli.Flag{
HeimdallURLFlag,
WithoutHeimdallFlag,
HeimdallgRPCAddressFlag,
}
)
@ -56,6 +65,7 @@ func getGenesis(genesisPath string) (*core.Genesis, error) {
func SetBorConfig(ctx *cli.Context, cfg *eth.Config) {
cfg.HeimdallURL = ctx.GlobalString(HeimdallURLFlag.Name)
cfg.WithoutHeimdall = ctx.GlobalBool(WithoutHeimdallFlag.Name)
cfg.HeimdallgRPCAddress = ctx.GlobalString(HeimdallgRPCAddressFlag.Name)
}
// CreateBorEthereum Creates bor ethereum object from eth.Config

18
cmd/utils/flags.go
View file

@ -33,6 +33,10 @@ import (
"text/template"
"time"
pcsclite "github.com/gballet/go-libpcsclite"
gopsutil "github.com/shirou/gopsutil/mem"
"gopkg.in/urfave/cli.v1"
"github.com/ethereum/go-ethereum/accounts"
"github.com/ethereum/go-ethereum/accounts/keystore"
"github.com/ethereum/go-ethereum/common"
@ -68,9 +72,6 @@ import (
"github.com/ethereum/go-ethereum/p2p/nat"
"github.com/ethereum/go-ethereum/p2p/netutil"
"github.com/ethereum/go-ethereum/params"
pcsclite "github.com/gballet/go-libpcsclite"
gopsutil "github.com/shirou/gopsutil/mem"
"gopkg.in/urfave/cli.v1"
)
func init() {
@ -1503,7 +1504,7 @@ func setPeerRequiredBlocks(ctx *cli.Context, cfg *ethconfig.Config) {
if peerRequiredBlocks == "" {
if ctx.GlobalIsSet(LegacyWhitelistFlag.Name) {
log.Warn("The flag --rpc is deprecated and will be removed, please use --peer.requiredblocks")
log.Warn("The flag --whitelist is deprecated and will be removed, please use --eth.requiredblocks")
peerRequiredBlocks = ctx.GlobalString(LegacyWhitelistFlag.Name)
} else {
return
@ -2030,9 +2031,10 @@ func MakeChain(ctx *cli.Context, stack *node.Node) (chain *core.BlockChain, chai
engine = clique.New(config.Clique, chainDb)
} else if config.Bor != nil {
ethereum = CreateBorEthereum(&eth.Config{
Genesis: genesis,
HeimdallURL: ctx.GlobalString(HeimdallURLFlag.Name),
WithoutHeimdall: ctx.GlobalBool(WithoutHeimdallFlag.Name),
Genesis: genesis,
HeimdallURL: ctx.GlobalString(HeimdallURLFlag.Name),
WithoutHeimdall: ctx.GlobalBool(WithoutHeimdallFlag.Name),
HeimdallgRPCAddress: ctx.GlobalString(HeimdallgRPCAddressFlag.Name),
})
engine = ethereum.Engine()
} else {
@ -2079,7 +2081,7 @@ func MakeChain(ctx *cli.Context, stack *node.Node) (chain *core.BlockChain, chai
// TODO(rjl493456442) disable snapshot generation/wiping if the chain is read only.
// Disable transaction indexing/unindexing by default.
chain, err = core.NewBlockChain(chainDb, cache, config, engine, vmcfg, nil, nil)
chain, err = core.NewBlockChain(chainDb, cache, config, engine, vmcfg, nil, nil, nil)
if err != nil {
Fatalf("Can't create BlockChain: %v", err)
}

36
common/network/port.go Normal file
View file

@ -0,0 +1,36 @@
package network
import (
"errors"
"fmt"
"net"
)
const (
maxPortCheck = 100
emptyPort = "127.0.0.1:0"
)
var (
ErrCantFindAPort = errors.New("no available port found")
)
// FindAvailablePort returns the an available port
func FindAvailablePort() (int, net.Listener, error) {
var (
listener net.Listener
err error
)
for i := uint(0); i < maxPortCheck; i++ {
listener, err = net.Listen("tcp", emptyPort)
if err != nil {
continue
}
return listener.Addr().(*net.TCPAddr).Port, listener, nil
}
return 0, nil, fmt.Errorf("%w: %s", ErrCantFindAPort, err)
}

96
common/tracing/context.go Normal file
View file

@ -0,0 +1,96 @@
package tracing
import (
"context"
"time"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/trace"
)
type tracerKey struct{}
type Option func(context.Context, trace.Span)
func WithTracer(ctx context.Context, tr trace.Tracer) context.Context {
return context.WithValue(ctx, tracerKey{}, tr)
}
func FromContext(ctx context.Context) trace.Tracer {
tr, _ := ctx.Value(tracerKey{}).(trace.Tracer)
return tr
}
func StartSpan(ctx context.Context, snapName string) (context.Context, trace.Span) {
tr := FromContext(ctx)
if tr == nil {
return ctx, nil
}
ctx, span := tr.Start(ctx, snapName)
ctx = WithTracer(ctx, tr)
return ctx, span
}
func EndSpan(span trace.Span) {
if span != nil {
span.End()
}
}
func Trace(ctx context.Context, spanName string) (context.Context, trace.Span) {
tr := FromContext(ctx)
if tr == nil {
return ctx, nil
}
return tr.Start(ctx, spanName)
}
func Exec(ctx context.Context, spanName string, opts ...Option) {
var span trace.Span
tr := FromContext(ctx)
if tr != nil {
ctx, span = tr.Start(ctx, spanName)
}
for _, optFn := range opts {
optFn(ctx, span)
}
if tr != nil {
span.End()
}
}
func WithTime(fn func(context.Context, trace.Span)) Option {
return func(ctx context.Context, span trace.Span) {
ElapsedTime(ctx, span, "elapsed", fn)
}
}
func ElapsedTime(ctx context.Context, span trace.Span, msg string, fn func(context.Context, trace.Span)) {
var now time.Time
if span != nil {
now = time.Now()
}
fn(ctx, span)
if span != nil {
span.SetAttributes(attribute.Int(msg, int(time.Since(now).Milliseconds())))
}
}
func SetAttributes(span trace.Span, kvs ...attribute.KeyValue) {
if span != nil {
span.SetAttributes(kvs...)
}
}

16
consensus/beacon/consensus.go
View file

@ -17,6 +17,7 @@
package beacon
import (
"context"
"errors"
"fmt"
"math/big"
@ -170,10 +171,11 @@ func (beacon *Beacon) VerifyUncles(chain consensus.ChainReader, block *types.Blo
// verifyHeader checks whether a header conforms to the consensus rules of the
// stock Ethereum consensus engine. The difference between the beacon and classic is
// (a) The following fields are expected to be constants:
// - difficulty is expected to be 0
// - nonce is expected to be 0
// - unclehash is expected to be Hash(emptyHeader)
// - difficulty is expected to be 0
// - nonce is expected to be 0
// - unclehash is expected to be Hash(emptyHeader)
// to be the desired constants
//
// (b) the timestamp is not verified anymore
// (c) the extradata is limited to 32 bytes
func (beacon *Beacon) verifyHeader(chain consensus.ChainHeaderReader, header, parent *types.Header) error {
@ -278,11 +280,11 @@ func (beacon *Beacon) Finalize(chain consensus.ChainHeaderReader, header *types.
// FinalizeAndAssemble implements consensus.Engine, setting the final state and
// assembling the block.
func (beacon *Beacon) FinalizeAndAssemble(chain consensus.ChainHeaderReader, header *types.Header, state *state.StateDB, txs []*types.Transaction, uncles []*types.Header, receipts []*types.Receipt) (*types.Block, error) {
func (beacon *Beacon) FinalizeAndAssemble(ctx context.Context, chain consensus.ChainHeaderReader, header *types.Header, state *state.StateDB, txs []*types.Transaction, uncles []*types.Header, receipts []*types.Receipt) (*types.Block, error) {
// FinalizeAndAssemble is different with Prepare, it can be used in both block
// generation and verification. So determine the consensus rules by header type.
if !beacon.IsPoSHeader(header) {
return beacon.ethone.FinalizeAndAssemble(chain, header, state, txs, uncles, receipts)
return beacon.ethone.FinalizeAndAssemble(ctx, chain, header, state, txs, uncles, receipts)
}
// Finalize and assemble the block
beacon.Finalize(chain, header, state, txs, uncles)
@ -294,9 +296,9 @@ func (beacon *Beacon) FinalizeAndAssemble(chain consensus.ChainHeaderReader, hea
//
// Note, the method returns immediately and will send the result async. More
// than one result may also be returned depending on the consensus algorithm.
func (beacon *Beacon) Seal(chain consensus.ChainHeaderReader, block *types.Block, results chan<- *types.Block, stop <-chan struct{}) error {
func (beacon *Beacon) Seal(ctx context.Context, chain consensus.ChainHeaderReader, block *types.Block, results chan<- *types.Block, stop <-chan struct{}) error {
if !beacon.IsPoSHeader(block.Header()) {
return beacon.ethone.Seal(chain, block, results, stop)
return beacon.ethone.Seal(ctx, chain, block, results, stop)
}
// The seal verification is done by the external consensus engine,
// return directly without pushing any block back. In another word

255
consensus/bor/bor.go
View file

@ -2,6 +2,7 @@ package bor
import (
"bytes"
"context"
"encoding/hex"
"encoding/json"
"errors"
@ -11,13 +12,17 @@ import (
"sort"
"strconv"
"sync"
"sync/atomic"
"time"
lru "github.com/hashicorp/golang-lru"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/trace"
"golang.org/x/crypto/sha3"
"github.com/ethereum/go-ethereum/accounts"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/tracing"
"github.com/ethereum/go-ethereum/consensus"
"github.com/ethereum/go-ethereum/consensus/bor/api"
"github.com/ethereum/go-ethereum/consensus/bor/clerk"
@ -45,7 +50,9 @@ const (
// Bor protocol constants.
var (
defaultSprintLength = uint64(64) // Default number of blocks after which to checkpoint and reset the pending votes
defaultSprintLength = map[string]uint64{
"0": 64,
} // Default number of blocks after which to checkpoint and reset the pending votes
extraVanity = 32 // Fixed number of extra-data prefix bytes reserved for signer vanity
extraSeal = 65 // Fixed number of extra-data suffix bytes reserved for signer seal
@ -96,6 +103,9 @@ var (
// errOutOfRangeChain is returned if an authorization list is attempted to
// be modified via out-of-range or non-contiguous headers.
errOutOfRangeChain = errors.New("out of range or non-contiguous chain")
errUncleDetected = errors.New("uncles not allowed")
errUnknownValidators = errors.New("unknown validators")
)
// SignerFn is a signer callback function to request a header to be signed by a
@ -159,7 +169,7 @@ func encodeSigHeader(w io.Writer, header *types.Header, c *params.BorConfig) {
header.Nonce,
}
if c.IsJaipur(header.Number.Uint64()) {
if c.IsJaipur(header.Number) {
if header.BaseFee != nil {
enc = append(enc, header.BaseFee)
}
@ -175,8 +185,8 @@ func CalcProducerDelay(number uint64, succession int, c *params.BorConfig) uint6
// When the block is the first block of the sprint, it is expected to be delayed by `producerDelay`.
// That is to allow time for block propagation in the last sprint
delay := c.CalculatePeriod(number)
if number%c.Sprint == 0 {
delay = c.ProducerDelay
if number%c.CalculateSprint(number) == 0 {
delay = c.CalculateProducerDelay(number)
}
if succession > 0 {
@ -209,9 +219,7 @@ type Bor struct {
recents *lru.ARCCache // Snapshots for recent block to speed up reorgs
signatures *lru.ARCCache // Signatures of recent blocks to speed up mining
signer common.Address // Ethereum address of the signing key
signFn SignerFn // Signer function to authorize hashes with
lock sync.RWMutex // Protects the signer fields
authorizedSigner atomic.Pointer[signer] // Ethereum address and sign function of the signing key
ethAPI api.Caller
spanner Spanner
@ -224,6 +232,11 @@ type Bor struct {
closeOnce sync.Once
}
type signer struct {
signer common.Address // Ethereum address of the signing key
signFn SignerFn // Signer function to authorize hashes with
}
// New creates a Matic Bor consensus engine.
func New(
chainConfig *params.ChainConfig,
@ -237,7 +250,7 @@ func New(
borConfig := chainConfig.Bor
// Set any missing consensus parameters to their defaults
if borConfig != nil && borConfig.Sprint == 0 {
if borConfig != nil && borConfig.CalculateSprint(0) == 0 {
borConfig.Sprint = defaultSprintLength
}
// Allocate the snapshot caches and create the engine
@ -256,6 +269,14 @@ func New(
HeimdallClient: heimdallClient,
}
c.authorizedSigner.Store(&signer{
common.Address{},
func(_ accounts.Account, _ string, i []byte) ([]byte, error) {
// return an error to prevent panics
return nil, &UnauthorizedSignerError{0, common.Address{}.Bytes()}
},
})
// make sure we can decode all the GenesisAlloc in the BorConfig.
for key, genesisAlloc := range c.config.BlockAlloc {
if _, err := decodeGenesisAlloc(genesisAlloc); err != nil {
@ -273,14 +294,14 @@ func (c *Bor) Author(header *types.Header) (common.Address, error) {
}
// VerifyHeader checks whether a header conforms to the consensus rules.
func (c *Bor) VerifyHeader(chain consensus.ChainHeaderReader, header *types.Header, seal bool) error {
func (c *Bor) VerifyHeader(chain consensus.ChainHeaderReader, header *types.Header, _ bool) error {
return c.verifyHeader(chain, header, nil)
}
// VerifyHeaders is similar to VerifyHeader, but verifies a batch of headers. The
// method returns a quit channel to abort the operations and a results channel to
// retrieve the async verifications (the order is that of the input slice).
func (c *Bor) VerifyHeaders(chain consensus.ChainHeaderReader, headers []*types.Header, seals []bool) (chan<- struct{}, <-chan error) {
func (c *Bor) VerifyHeaders(chain consensus.ChainHeaderReader, headers []*types.Header, _ []bool) (chan<- struct{}, <-chan error) {
abort := make(chan struct{})
results := make(chan error, len(headers))
@ -320,7 +341,7 @@ func (c *Bor) verifyHeader(chain consensus.ChainHeaderReader, header *types.Head
}
// check extr adata
isSprintEnd := IsSprintStart(number+1, c.config.Sprint)
isSprintEnd := IsSprintStart(number+1, c.config.CalculateSprint(number))
// Ensure that the extra-data contains a signer list on checkpoint, but none otherwise
signersBytes := len(header.Extra) - extraVanity - extraSeal
@ -434,7 +455,7 @@ func (c *Bor) verifyCascadingFields(chain consensus.ChainHeaderReader, header *t
}
// verify the validator list in the last sprint block
if IsSprintStart(number, c.config.Sprint) {
if IsSprintStart(number, c.config.CalculateSprint(number)) {
parentValidatorBytes := parent.Extra[extraVanity : len(parent.Extra)-extraSeal]
validatorsBytes := make([]byte, len(snap.ValidatorSet.Validators)*validatorHeaderBytesLength)
@ -487,6 +508,7 @@ func (c *Bor) snapshot(chain consensus.ChainHeaderReader, number uint64, hash co
// at a checkpoint block without a parent (light client CHT), or we have piled
// up more headers than allowed to be reorged (chain reinit from a freezer),
// consider the checkpoint trusted and snapshot it.
// TODO fix this
// nolint:nestif
if number == 0 {
@ -496,7 +518,7 @@ func (c *Bor) snapshot(chain consensus.ChainHeaderReader, number uint64, hash co
hash := checkpoint.Hash()
// get validators and current span
validators, err := c.spanner.GetCurrentValidators(hash, number+1)
validators, err := c.spanner.GetCurrentValidators(context.Background(), hash, number+1)
if err != nil {
return nil, err
}
@ -537,7 +559,7 @@ func (c *Bor) snapshot(chain consensus.ChainHeaderReader, number uint64, hash co
// check if snapshot is nil
if snap == nil {
return nil, fmt.Errorf("Unknown error while retrieving snapshot at block number %v", number)
return nil, fmt.Errorf("unknown error while retrieving snapshot at block number %v", number)
}
// Previous snapshot found, apply any pending headers on top of it
@ -566,9 +588,9 @@ func (c *Bor) snapshot(chain consensus.ChainHeaderReader, number uint64, hash co
// VerifyUncles implements consensus.Engine, always returning an error for any
// uncles as this consensus mechanism doesn't permit uncles.
func (c *Bor) VerifyUncles(chain consensus.ChainReader, block *types.Block) error {
func (c *Bor) VerifyUncles(_ consensus.ChainReader, block *types.Block) error {
if len(block.Uncles()) > 0 {
return errors.New("uncles not allowed")
return errUncleDetected
}
return nil
@ -602,7 +624,7 @@ func (c *Bor) verifySeal(chain consensus.ChainHeaderReader, header *types.Header
return err
}
if !snap.ValidatorSet.HasAddress(signer.Bytes()) {
if !snap.ValidatorSet.HasAddress(signer) {
// Check the UnauthorizedSignerError.Error() msg to see why we pass number-1
return &UnauthorizedSignerError{number - 1, signer.Bytes()}
}
@ -619,13 +641,13 @@ func (c *Bor) verifySeal(chain consensus.ChainHeaderReader, header *types.Header
parent = chain.GetHeader(header.ParentHash, number-1)
}
if parent != nil && header.Time < parent.Time+CalcProducerDelay(number, succession, c.config) {
if IsBlockOnTime(parent, header, number, succession, c.config) {
return &BlockTooSoonError{number, succession}
}
// Ensure that the difficulty corresponds to the turn-ness of the signer
if !c.fakeDiff {
difficulty := snap.Difficulty(signer)
difficulty := Difficulty(snap.ValidatorSet, signer)
if header.Difficulty.Uint64() != difficulty {
return &WrongDifficultyError{number, difficulty, header.Difficulty.Uint64(), signer.Bytes()}
}
@ -634,6 +656,10 @@ func (c *Bor) verifySeal(chain consensus.ChainHeaderReader, header *types.Header
return nil
}
func IsBlockOnTime(parent *types.Header, header *types.Header, number uint64, succession int, cfg *params.BorConfig) bool {
return parent != nil && header.Time < parent.Time+CalcProducerDelay(number, succession, cfg)
}
// Prepare implements consensus.Engine, preparing all the consensus fields of the
// header for running the transactions on top.
func (c *Bor) Prepare(chain consensus.ChainHeaderReader, header *types.Header) error {
@ -648,8 +674,10 @@ func (c *Bor) Prepare(chain consensus.ChainHeaderReader, header *types.Header) e
return err
}
currentSigner := *c.authorizedSigner.Load()
// Set the correct difficulty
header.Difficulty = new(big.Int).SetUint64(snap.Difficulty(c.signer))
header.Difficulty = new(big.Int).SetUint64(Difficulty(snap.ValidatorSet, currentSigner.signer))
// Ensure the extra data has all it's components
if len(header.Extra) < extraVanity {
@ -659,10 +687,10 @@ func (c *Bor) Prepare(chain consensus.ChainHeaderReader, header *types.Header) e
header.Extra = header.Extra[:extraVanity]
// get validator set if number
if IsSprintStart(number+1, c.config.Sprint) {
newValidators, err := c.spanner.GetCurrentValidators(header.ParentHash, number+1)
if IsSprintStart(number+1, c.config.CalculateSprint(number)) {
newValidators, err := c.spanner.GetCurrentValidators(context.Background(), header.ParentHash, number+1)
if err != nil {
return errors.New("unknown validators")
return errUnknownValidators
}
// sort validator by address
@ -687,8 +715,8 @@ func (c *Bor) Prepare(chain consensus.ChainHeaderReader, header *types.Header) e
var succession int
// if signer is not empty
if c.signer != (common.Address{}) {
succession, err = snap.GetSignerSuccessionNumber(c.signer)
if currentSigner.signer != (common.Address{}) {
succession, err = snap.GetSignerSuccessionNumber(currentSigner.signer)
if err != nil {
return err
}
@ -704,24 +732,26 @@ func (c *Bor) Prepare(chain consensus.ChainHeaderReader, header *types.Header) e
// Finalize implements consensus.Engine, ensuring no uncles are set, nor block
// rewards given.
func (c *Bor) Finalize(chain consensus.ChainHeaderReader, header *types.Header, state *state.StateDB, txs []*types.Transaction, uncles []*types.Header) {
stateSyncData := []*types.StateSyncData{}
var err error
func (c *Bor) Finalize(chain consensus.ChainHeaderReader, header *types.Header, state *state.StateDB, _ []*types.Transaction, _ []*types.Header) {
var (
stateSyncData []*types.StateSyncData
err error
)
headerNumber := header.Number.Uint64()
if headerNumber%c.config.Sprint == 0 {
if IsSprintStart(headerNumber, c.config.CalculateSprint(headerNumber)) {
ctx := context.Background()
cx := statefull.ChainContext{Chain: chain, Bor: c}
// check and commit span
if err := c.checkAndCommitSpan(state, header, cx); err != nil {
if err := c.checkAndCommitSpan(ctx, state, header, cx); err != nil {
log.Error("Error while committing span", "error", err)
return
}
if c.HeimdallClient != nil {
// commit statees
stateSyncData, err = c.CommitStates(state, header, cx)
// commit states
stateSyncData, err = c.CommitStates(ctx, state, header, cx)
if err != nil {
log.Error("Error while committing states", "error", err)
return
@ -763,7 +793,7 @@ func (c *Bor) changeContractCodeIfNeeded(headerNumber uint64, state *state.State
if blockNumber == strconv.FormatUint(headerNumber, 10) {
allocs, err := decodeGenesisAlloc(genesisAlloc)
if err != nil {
return fmt.Errorf("failed to decode genesis alloc: %v", err)
return fmt.Errorf("failed to decode genesis alloc: %w", err)
}
for addr, account := range allocs {
@ -778,24 +808,35 @@ func (c *Bor) changeContractCodeIfNeeded(headerNumber uint64, state *state.State
// FinalizeAndAssemble implements consensus.Engine, ensuring no uncles are set,
// nor block rewards given, and returns the final block.
func (c *Bor) FinalizeAndAssemble(chain consensus.ChainHeaderReader, header *types.Header, state *state.StateDB, txs []*types.Transaction, uncles []*types.Header, receipts []*types.Receipt) (*types.Block, error) {
func (c *Bor) FinalizeAndAssemble(ctx context.Context, chain consensus.ChainHeaderReader, header *types.Header, state *state.StateDB, txs []*types.Transaction, uncles []*types.Header, receipts []*types.Receipt) (*types.Block, error) {
finalizeCtx, finalizeSpan := tracing.StartSpan(ctx, "bor.FinalizeAndAssemble")
defer tracing.EndSpan(finalizeSpan)
stateSyncData := []*types.StateSyncData{}
headerNumber := header.Number.Uint64()
if headerNumber%c.config.Sprint == 0 {
var err error
if IsSprintStart(headerNumber, c.config.CalculateSprint(headerNumber)) {
cx := statefull.ChainContext{Chain: chain, Bor: c}
// check and commit span
err := c.checkAndCommitSpan(state, header, cx)
tracing.Exec(finalizeCtx, "bor.checkAndCommitSpan", func(ctx context.Context, span trace.Span) {
// check and commit span
err = c.checkAndCommitSpan(finalizeCtx, state, header, cx)
})
if err != nil {
log.Error("Error while committing span", "error", err)
return nil, err
}
if c.HeimdallClient != nil {
// commit states
stateSyncData, err = c.CommitStates(state, header, cx)
tracing.Exec(finalizeCtx, "bor.checkAndCommitSpan", func(ctx context.Context, span trace.Span) {
// commit states
stateSyncData, err = c.CommitStates(finalizeCtx, state, header, cx)
})
if err != nil {
log.Error("Error while committing states", "error", err)
return nil, err
@ -803,13 +844,21 @@ func (c *Bor) FinalizeAndAssemble(chain consensus.ChainHeaderReader, header *typ
}
}
if err := c.changeContractCodeIfNeeded(headerNumber, state); err != nil {
tracing.Exec(finalizeCtx, "bor.changeContractCodeIfNeeded", func(ctx context.Context, span trace.Span) {
err = c.changeContractCodeIfNeeded(headerNumber, state)
})
if err != nil {
log.Error("Error changing contract code", "error", err)
return nil, err
}
// No block rewards in PoA, so the state remains as is and uncles are dropped
header.Root = state.IntermediateRoot(chain.Config().IsEIP158(header.Number))
// No block rewards in PoA, so the state remains as it is
tracing.Exec(finalizeCtx, "bor.IntermediateRoot", func(ctx context.Context, span trace.Span) {
header.Root = state.IntermediateRoot(chain.Config().IsEIP158(header.Number))
})
// Uncles are dropped
header.UncleHash = types.CalcUncleHash(nil)
// Assemble block
@ -819,23 +868,41 @@ func (c *Bor) FinalizeAndAssemble(chain consensus.ChainHeaderReader, header *typ
bc := chain.(core.BorStateSyncer)
bc.SetStateSync(stateSyncData)
tracing.SetAttributes(
finalizeSpan,
attribute.Int("number", int(header.Number.Int64())),
attribute.String("hash", header.Hash().String()),
attribute.Int("number of txs", len(txs)),
attribute.Int("gas used", int(block.GasUsed())),
)
// return the final block for sealing
return block, nil
}
// Authorize injects a private key into the consensus engine to mint new blocks
// with.
func (c *Bor) Authorize(signer common.Address, signFn SignerFn) {
c.lock.Lock()
defer c.lock.Unlock()
c.signer = signer
c.signFn = signFn
func (c *Bor) Authorize(currentSigner common.Address, signFn SignerFn) {
c.authorizedSigner.Store(&signer{
signer: currentSigner,
signFn: signFn,
})
}
// Seal implements consensus.Engine, attempting to create a sealed block using
// the local signing credentials.
func (c *Bor) Seal(chain consensus.ChainHeaderReader, block *types.Block, results chan<- *types.Block, stop <-chan struct{}) error {
func (c *Bor) Seal(ctx context.Context, chain consensus.ChainHeaderReader, block *types.Block, results chan<- *types.Block, stop <-chan struct{}) error {
_, sealSpan := tracing.StartSpan(ctx, "bor.Seal")
var endSpan bool = true
defer func() {
// Only end span in case of early-returns/errors
if endSpan {
tracing.EndSpan(sealSpan)
}
}()
header := block.Header()
// Sealing the genesis block is not supported
number := header.Number.Uint64()
@ -847,10 +914,9 @@ func (c *Bor) Seal(chain consensus.ChainHeaderReader, block *types.Block, result
log.Info("Sealing paused, waiting for transactions")
return nil
}
// Don't hold the signer fields for the entire sealing procedure
c.lock.RLock()
signer, signFn := c.signer, c.signFn
c.lock.RUnlock()
currentSigner := *c.authorizedSigner.Load()
snap, err := c.snapshot(chain, number-1, header.ParentHash, nil)
if err != nil {
@ -858,12 +924,12 @@ func (c *Bor) Seal(chain consensus.ChainHeaderReader, block *types.Block, result
}
// Bail out if we're unauthorized to sign a block
if !snap.ValidatorSet.HasAddress(signer.Bytes()) {
if !snap.ValidatorSet.HasAddress(currentSigner.signer) {
// Check the UnauthorizedSignerError.Error() msg to see why we pass number-1
return &UnauthorizedSignerError{number - 1, signer.Bytes()}
return &UnauthorizedSignerError{number - 1, currentSigner.signer.Bytes()}
}
successionNumber, err := snap.GetSignerSuccessionNumber(signer)
successionNumber, err := snap.GetSignerSuccessionNumber(currentSigner.signer)
if err != nil {
return err
}
@ -874,7 +940,7 @@ func (c *Bor) Seal(chain consensus.ChainHeaderReader, block *types.Block, result
wiggle := time.Duration(successionNumber) * time.Duration(c.config.CalculateBackupMultiplier(number)) * time.Second
// Sign all the things!
err = Sign(signFn, signer, header, c.config)
err = Sign(currentSigner.signFn, currentSigner.signer, header, c.config)
if err != nil {
return err
}
@ -882,7 +948,7 @@ func (c *Bor) Seal(chain consensus.ChainHeaderReader, block *types.Block, result
// Wait until sealing is terminated or delay timeout.
log.Info("Waiting for slot to sign and propagate", "number", number, "hash", header.Hash, "delay-in-sec", uint(delay), "delay", common.PrettyDuration(delay))
go func() {
go func(sealSpan trace.Span) {
select {
case <-stop:
log.Debug("Discarding sealing operation for block", "number", number)
@ -905,13 +971,27 @@ func (c *Bor) Seal(chain consensus.ChainHeaderReader, block *types.Block, result
"delay", delay,
"headerDifficulty", header.Difficulty,
)
tracing.SetAttributes(
sealSpan,
attribute.Int("number", int(number)),
attribute.String("hash", header.Hash().String()),
attribute.Int("delay", int(delay.Milliseconds())),
attribute.Int("wiggle", int(wiggle.Milliseconds())),
attribute.Bool("out-of-turn", wiggle > 0),
)
tracing.EndSpan(sealSpan)
}
select {
case results <- block.WithSeal(header):
default:
log.Warn("Sealing result was not read by miner", "number", number, "sealhash", SealHash(header, c.config))
}
}()
}(sealSpan)
// Set the endSpan flag to false, as the go routine will handle it
endSpan = false
return nil
}
@ -930,13 +1010,13 @@ func Sign(signFn SignerFn, signer common.Address, header *types.Header, c *param
// CalcDifficulty is the difficulty adjustment algorithm. It returns the difficulty
// that a new block should have based on the previous blocks in the chain and the
// current signer.
func (c *Bor) CalcDifficulty(chain consensus.ChainHeaderReader, time uint64, parent *types.Header) *big.Int {
func (c *Bor) CalcDifficulty(chain consensus.ChainHeaderReader, _ uint64, parent *types.Header) *big.Int {
snap, err := c.snapshot(chain, parent.Number.Uint64(), parent.Hash(), nil)
if err != nil {
return nil
}
return new(big.Int).SetUint64(snap.Difficulty(c.signer))
return new(big.Int).SetUint64(Difficulty(snap.ValidatorSet, c.authorizedSigner.Load().signer))
}
// SealHash returns the hash of a block prior to it being sealed.
@ -967,37 +1047,38 @@ func (c *Bor) Close() error {
}
func (c *Bor) checkAndCommitSpan(
ctx context.Context,
state *state.StateDB,
header *types.Header,
chain core.ChainContext,
) error {
headerNumber := header.Number.Uint64()
span, err := c.spanner.GetCurrentSpan(header.ParentHash)
span, err := c.spanner.GetCurrentSpan(ctx, header.ParentHash)
if err != nil {
return err
}
if c.needToCommitSpan(span, headerNumber) {
return c.FetchAndCommitSpan(span.ID+1, state, header, chain)
return c.FetchAndCommitSpan(ctx, span.ID+1, state, header, chain)
}
return nil
}
func (c *Bor) needToCommitSpan(span *span.Span, headerNumber uint64) bool {
func (c *Bor) needToCommitSpan(currentSpan *span.Span, headerNumber uint64) bool {
// if span is nil
if span == nil {
if currentSpan == nil {
return false
}
// check span is not set initially
if span.EndBlock == 0 {
if currentSpan.EndBlock == 0 {
return true
}
// if current block is first block of last sprint in current span
if span.EndBlock > c.config.Sprint && span.EndBlock-c.config.Sprint+1 == headerNumber {
if currentSpan.EndBlock > c.config.CalculateSprint(headerNumber) && currentSpan.EndBlock-c.config.CalculateSprint(headerNumber)+1 == headerNumber {
return true
}
@ -1005,6 +1086,7 @@ func (c *Bor) needToCommitSpan(span *span.Span, headerNumber uint64) bool {
}
func (c *Bor) FetchAndCommitSpan(
ctx context.Context,
newSpanID uint64,
state *state.StateDB,
header *types.Header,
@ -1014,14 +1096,14 @@ func (c *Bor) FetchAndCommitSpan(
if c.HeimdallClient == nil {
// fixme: move to a new mock or fake and remove c.HeimdallClient completely
s, err := c.getNextHeimdallSpanForTest(newSpanID, header, chain)
s, err := c.getNextHeimdallSpanForTest(ctx, newSpanID, header, chain)
if err != nil {
return err
}
heimdallSpan = *s
} else {
response, err := c.HeimdallClient.Span(newSpanID)
response, err := c.HeimdallClient.Span(ctx, newSpanID)
if err != nil {
return err
}
@ -1038,16 +1120,17 @@ func (c *Bor) FetchAndCommitSpan(
)
}
return c.spanner.CommitSpan(heimdallSpan, state, header, chain)
return c.spanner.CommitSpan(ctx, heimdallSpan, state, header, chain)
}
// CommitStates commit states
func (c *Bor) CommitStates(
ctx context.Context,
state *state.StateDB,
header *types.Header,
chain statefull.ChainContext,
) ([]*types.StateSyncData, error) {
stateSyncs := make([]*types.StateSyncData, 0)
fetchStart := time.Now()
number := header.Number.Uint64()
_lastStateID, err := c.GenesisContractsClient.LastStateId(number - 1)
@ -1055,7 +1138,7 @@ func (c *Bor) CommitStates(
return nil, err
}
to := time.Unix(int64(chain.Chain.GetHeaderByNumber(number-c.config.Sprint).Time), 0)
to := time.Unix(int64(chain.Chain.GetHeaderByNumber(number-c.config.CalculateSprint(number)).Time), 0)
lastStateID := _lastStateID.Uint64()
log.Info(
@ -1063,7 +1146,7 @@ func (c *Bor) CommitStates(
"fromID", lastStateID+1,
"to", to.Format(time.RFC3339))
eventRecords, err := c.HeimdallClient.StateSyncEvents(lastStateID+1, to.Unix())
eventRecords, err := c.HeimdallClient.StateSyncEvents(ctx, lastStateID+1, to.Unix())
if err != nil {
log.Error("Error occurred when fetching state sync events", "stateID", lastStateID+1, "error", err)
}
@ -1074,16 +1157,20 @@ func (c *Bor) CommitStates(
}
}
fetchTime := time.Since(fetchStart)
processStart := time.Now()
totalGas := 0 /// limit on gas for state sync per block
chainID := c.chainConfig.ChainID.String()
stateSyncs := make([]*types.StateSyncData, len(eventRecords))
var gasUsed uint64
for _, eventRecord := range eventRecords {
if eventRecord.ID <= lastStateID {
continue
}
if err := validateEventRecord(eventRecord, number, to, lastStateID, chainID); err != nil {
if err = validateEventRecord(eventRecord, number, to, lastStateID, chainID); err != nil {
log.Error("while validating event record", "block", number, "to", to, "stateID", lastStateID, "error", err.Error())
break
}
@ -1097,7 +1184,10 @@ func (c *Bor) CommitStates(
stateSyncs = append(stateSyncs, &stateData)
gasUsed, err := c.GenesisContractsClient.CommitState(eventRecord, state, header, chain)
// we expect that this call MUST emit an event, otherwise we wouldn't make a receipt
// if the receiver address is not a contract then we'll skip the most of the execution and emitting an event as well
// https://github.com/maticnetwork/genesis-contracts/blob/master/contracts/StateReceiver.sol#L27
gasUsed, err = c.GenesisContractsClient.CommitState(eventRecord, state, header, chain)
if err != nil {
return nil, err
}
@ -1107,7 +1197,9 @@ func (c *Bor) CommitStates(
lastStateID++
}
log.Info("StateSyncData", "Gas", totalGas, "Block-number", number, "LastStateID", lastStateID, "TotalRecords", len(eventRecords))
processTime := time.Since(processStart)
log.Info("StateSyncData", "gas", totalGas, "number", number, "lastStateID", lastStateID, "total records", len(eventRecords), "fetch time", int(fetchTime.Milliseconds()), "process time", int(processTime.Milliseconds()))
return stateSyncs, nil
}
@ -1125,8 +1217,8 @@ func (c *Bor) SetHeimdallClient(h IHeimdallClient) {
c.HeimdallClient = h
}
func (c *Bor) GetCurrentValidators(headerHash common.Hash, blockNumber uint64) ([]*valset.Validator, error) {
return c.spanner.GetCurrentValidators(headerHash, blockNumber)
func (c *Bor) GetCurrentValidators(ctx context.Context, headerHash common.Hash, blockNumber uint64) ([]*valset.Validator, error) {
return c.spanner.GetCurrentValidators(ctx, headerHash, blockNumber)
}
//
@ -1134,13 +1226,14 @@ func (c *Bor) GetCurrentValidators(headerHash common.Hash, blockNumber uint64) (
//
func (c *Bor) getNextHeimdallSpanForTest(
ctx context.Context,
newSpanID uint64,
header *types.Header,
chain core.ChainContext,
) (*span.HeimdallSpan, error) {
headerNumber := header.Number.Uint64()
spanBor, err := c.spanner.GetCurrentSpan(header.ParentHash)
spanBor, err := c.spanner.GetCurrentSpan(ctx, header.ParentHash)
if err != nil {
return nil, err
}
@ -1161,7 +1254,7 @@ func (c *Bor) getNextHeimdallSpanForTest(
spanBor.StartBlock = spanBor.EndBlock + 1
}
spanBor.EndBlock = spanBor.StartBlock + (100 * c.config.Sprint) - 1
spanBor.EndBlock = spanBor.StartBlock + (100 * c.config.CalculateSprint(headerNumber)) - 1
selectedProducers := make([]valset.Validator, len(snap.ValidatorSet.Validators))
for i, v := range snap.ValidatorSet.Validators {

48
consensus/bor/bor_test.go
View file

@ -4,10 +4,10 @@ import (
"math/big"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/ethereum/go-ethereum/common/hexutil" //nolint:typecheck
"github.com/ethereum/go-ethereum/core"
"github.com/ethereum/go-ethereum/core/rawdb"
"github.com/ethereum/go-ethereum/core/state"
@ -23,7 +23,9 @@ func TestGenesisContractChange(t *testing.T) {
b := &Bor{
config: &params.BorConfig{
Sprint: 10, // skip sprint transactions in sprint
Sprint: map[string]uint64{
"0": 10,
}, // skip sprint transactions in sprint
BlockAlloc: map[string]interface{}{
// write as interface since that is how it is decoded in genesis
"2": map[string]interface{}{
@ -55,11 +57,11 @@ func TestGenesisContractChange(t *testing.T) {
genesis := genspec.MustCommit(db)
statedb, err := state.New(genesis.Root(), state.NewDatabase(db), nil)
assert.NoError(t, err)
require.NoError(t, err)
config := params.ChainConfig{}
chain, err := core.NewBlockChain(db, nil, &config, b, vm.Config{}, nil, nil)
assert.NoError(t, err)
chain, err := core.NewBlockChain(db, nil, &config, b, vm.Config{}, nil, nil, nil)
require.NoError(t, err)
addBlock := func(root common.Hash, num int64) (common.Hash, *state.StateDB) {
h := &types.Header{
@ -70,37 +72,37 @@ func TestGenesisContractChange(t *testing.T) {
// write state to database
root, err := statedb.Commit(false)
assert.NoError(t, err)
assert.NoError(t, statedb.Database().TrieDB().Commit(root, true, nil))
require.NoError(t, err)
require.NoError(t, statedb.Database().TrieDB().Commit(root, true, nil))
statedb, err := state.New(h.Root, state.NewDatabase(db), nil)
assert.NoError(t, err)
require.NoError(t, err)
return root, statedb
}
assert.Equal(t, statedb.GetCode(addr0), []byte{0x1, 0x1})
require.Equal(t, statedb.GetCode(addr0), []byte{0x1, 0x1})
root := genesis.Root()
// code does not change
root, statedb = addBlock(root, 1)
assert.Equal(t, statedb.GetCode(addr0), []byte{0x1, 0x1})
require.Equal(t, statedb.GetCode(addr0), []byte{0x1, 0x1})
// code changes 1st time
root, statedb = addBlock(root, 2)
assert.Equal(t, statedb.GetCode(addr0), []byte{0x1, 0x2})
require.Equal(t, statedb.GetCode(addr0), []byte{0x1, 0x2})
// code same as 1st change
root, statedb = addBlock(root, 3)
assert.Equal(t, statedb.GetCode(addr0), []byte{0x1, 0x2})
require.Equal(t, statedb.GetCode(addr0), []byte{0x1, 0x2})
// code changes 2nd time
_, statedb = addBlock(root, 4)
assert.Equal(t, statedb.GetCode(addr0), []byte{0x1, 0x3})
require.Equal(t, statedb.GetCode(addr0), []byte{0x1, 0x3})
// make sure balance change DOES NOT take effect
assert.Equal(t, statedb.GetBalance(addr0), big.NewInt(0))
require.Equal(t, statedb.GetBalance(addr0), big.NewInt(0))
}
func TestEncodeSigHeaderJaipur(t *testing.T) {
@ -123,20 +125,20 @@ func TestEncodeSigHeaderJaipur(t *testing.T) {
)
// Jaipur NOT enabled and BaseFee not set
hash := SealHash(h, &params.BorConfig{JaipurBlock: 10})
assert.Equal(t, hash, hashWithoutBaseFee)
hash := SealHash(h, &params.BorConfig{JaipurBlock: big.NewInt(10)})
require.Equal(t, hash, hashWithoutBaseFee)
// Jaipur enabled (Jaipur=0) and BaseFee not set
hash = SealHash(h, &params.BorConfig{JaipurBlock: 0})
assert.Equal(t, hash, hashWithoutBaseFee)
hash = SealHash(h, &params.BorConfig{JaipurBlock: common.Big0})
require.Equal(t, hash, hashWithoutBaseFee)
h.BaseFee = big.NewInt(2)
// Jaipur enabled (Jaipur=Header block) and BaseFee set
hash = SealHash(h, &params.BorConfig{JaipurBlock: 1})
assert.Equal(t, hash, hashWithBaseFee)
hash = SealHash(h, &params.BorConfig{JaipurBlock: common.Big1})
require.Equal(t, hash, hashWithBaseFee)
// Jaipur NOT enabled and BaseFee set
hash = SealHash(h, &params.BorConfig{JaipurBlock: 10})
assert.Equal(t, hash, hashWithoutBaseFee)
hash = SealHash(h, &params.BorConfig{JaipurBlock: big.NewInt(10)})
require.Equal(t, hash, hashWithoutBaseFee)
}

5
consensus/bor/contract/client.go
View file

@ -88,7 +88,7 @@ func (gc *GenesisContractsClient) CommitState(
}
msg := statefull.GetSystemMessage(common.HexToAddress(gc.StateReceiverContract), data)
gasUsed, err := statefull.ApplyMessage(msg, state, header, gc.chainConfig, chCtx)
gasUsed, err := statefull.ApplyMessage(context.Background(), msg, state, header, gc.chainConfig, chCtx)
// Logging event log with time and individual gasUsed
log.Info("→ committing new state", "eventRecord", event.String(gasUsed))
@ -102,7 +102,8 @@ func (gc *GenesisContractsClient) CommitState(
func (gc *GenesisContractsClient) LastStateId(snapshotNumber uint64) (*big.Int, error) {
blockNr := rpc.BlockNumber(snapshotNumber)
method := "lastStateId"
const method = "lastStateId"
data, err := gc.stateReceiverABI.Pack(method)
if err != nil {

9
consensus/bor/heimdall.go
View file

@ -1,6 +1,8 @@
package bor
import (
"context"
"github.com/ethereum/go-ethereum/consensus/bor/clerk"
"github.com/ethereum/go-ethereum/consensus/bor/heimdall/checkpoint"
"github.com/ethereum/go-ethereum/consensus/bor/heimdall/span"
@ -8,8 +10,9 @@ import (
//go:generate mockgen -destination=../../tests/bor/mocks/IHeimdallClient.go -package=mocks . IHeimdallClient
type IHeimdallClient interface {
StateSyncEvents(fromID uint64, to int64) ([]*clerk.EventRecordWithTime, error)
Span(spanID uint64) (*span.HeimdallSpan, error)
FetchLatestCheckpoint() (*checkpoint.Checkpoint, error)
StateSyncEvents(ctx context.Context, fromID uint64, to int64) ([]*clerk.EventRecordWithTime, error)
Span(ctx context.Context, spanID uint64) (*span.HeimdallSpan, error)
FetchCheckpoint(ctx context.Context, number int64) (*checkpoint.Checkpoint, error)
FetchCheckpointCount(ctx context.Context) (int64, error)
Close()
}

9
consensus/bor/heimdall/checkpoint/checkpoint.go
View file

@ -20,3 +20,12 @@ type CheckpointResponse struct {
Height string `json:"height"`
Result Checkpoint `json:"result"`
}
type CheckpointCount struct {
Result int64 `json:"result"`
}
type CheckpointCountResponse struct {
Height string `json:"height"`
Result CheckpointCount `json:"result"`
}

177
consensus/bor/heimdall/client.go
View file

@ -5,7 +5,7 @@ import (
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"io"
"net/http"
"net/url"
"sort"
@ -15,14 +15,20 @@ import (
"github.com/ethereum/go-ethereum/consensus/bor/heimdall/checkpoint"
"github.com/ethereum/go-ethereum/consensus/bor/heimdall/span"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/metrics"
)
// errShutdownDetected is returned if a shutdown was detected
var errShutdownDetected = errors.New("shutdown detected")
var (
// ErrShutdownDetected is returned if a shutdown was detected
ErrShutdownDetected = errors.New("shutdown detected")
ErrNoResponse = errors.New("got a nil response")
ErrNotSuccessfulResponse = errors.New("error while fetching data from Heimdall")
)
const (
stateFetchLimit = 50
apiHeimdallTimeout = 5 * time.Second
retryCall = 5 * time.Second
)
type StateSyncEventsResponse struct {
@ -41,6 +47,12 @@ type HeimdallClient struct {
closeCh chan struct{}
}
type Request struct {
client http.Client
url *url.URL
start time.Time
}
func NewHeimdallClient(urlString string) *HeimdallClient {
return &HeimdallClient{
urlString: urlString,
@ -54,12 +66,13 @@ func NewHeimdallClient(urlString string) *HeimdallClient {
const (
fetchStateSyncEventsFormat = "from-id=%d&to-time=%d&limit=%d"
fetchStateSyncEventsPath = "clerk/event-record/list"
fetchLatestCheckpoint = "/checkpoints/latest"
fetchCheckpoint = "/checkpoints/%s"
fetchCheckpointCount = "/checkpoints/count"
fetchSpanFormat = "bor/span/%d"
)
func (h *HeimdallClient) StateSyncEvents(fromID uint64, to int64) ([]*clerk.EventRecordWithTime, error) {
func (h *HeimdallClient) StateSyncEvents(ctx context.Context, fromID uint64, to int64) ([]*clerk.EventRecordWithTime, error) {
eventRecords := make([]*clerk.EventRecordWithTime, 0)
for {
@ -70,7 +83,9 @@ func (h *HeimdallClient) StateSyncEvents(fromID uint64, to int64) ([]*clerk.Even
log.Info("Fetching state sync events", "queryParams", url.RawQuery)
response, err := FetchWithRetry[StateSyncEventsResponse](h.client, url, h.closeCh)
ctx = withRequestType(ctx, stateSyncRequest)
response, err := FetchWithRetry[StateSyncEventsResponse](ctx, h.client, url, h.closeCh)
if err != nil {
return nil, err
}
@ -96,13 +111,15 @@ func (h *HeimdallClient) StateSyncEvents(fromID uint64, to int64) ([]*clerk.Even
return eventRecords, nil
}
func (h *HeimdallClient) Span(spanID uint64) (*span.HeimdallSpan, error) {
func (h *HeimdallClient) Span(ctx context.Context, spanID uint64) (*span.HeimdallSpan, error) {
url, err := spanURL(h.urlString, spanID)
if err != nil {
return nil, err
}
response, err := FetchWithRetry[SpanResponse](h.client, url, h.closeCh)
ctx = withRequestType(ctx, spanRequest)
response, err := FetchWithRetry[SpanResponse](ctx, h.client, url, h.closeCh)
if err != nil {
return nil, err
}
@ -110,14 +127,16 @@ func (h *HeimdallClient) Span(spanID uint64) (*span.HeimdallSpan, error) {
return &response.Result, nil
}
// FetchLatestCheckpoint fetches the latest bor submitted checkpoint from heimdall
func (h *HeimdallClient) FetchLatestCheckpoint() (*checkpoint.Checkpoint, error) {
url, err := latestCheckpointURL(h.urlString)
// FetchCheckpoint fetches the checkpoint from heimdall
func (h *HeimdallClient) FetchCheckpoint(ctx context.Context, number int64) (*checkpoint.Checkpoint, error) {
url, err := checkpointURL(h.urlString, number)
if err != nil {
return nil, err
}
response, err := FetchWithRetry[checkpoint.CheckpointResponse](h.client, url, h.closeCh)
ctx = withRequestType(ctx, checkpointRequest)
response, err := FetchWithRetry[checkpoint.CheckpointResponse](ctx, h.client, url, h.closeCh)
if err != nil {
return nil, err
}
@ -125,59 +144,107 @@ func (h *HeimdallClient) FetchLatestCheckpoint() (*checkpoint.Checkpoint, error)
return &response.Result, nil
}
// FetchCheckpointCount fetches the checkpoint count from heimdall
func (h *HeimdallClient) FetchCheckpointCount(ctx context.Context) (int64, error) {
url, err := checkpointCountURL(h.urlString)
if err != nil {
return 0, err
}
ctx = withRequestType(ctx, checkpointCountRequest)
response, err := FetchWithRetry[checkpoint.CheckpointCountResponse](ctx, h.client, url, h.closeCh)
if err != nil {
return 0, err
}
return response.Result.Result, nil
}
// FetchWithRetry returns data from heimdall with retry
func FetchWithRetry[T any](client http.Client, url *url.URL, closeCh chan struct{}) (*T, error) {
// attempt counter
attempt := 1
result := new(T)
ctx, cancel := context.WithTimeout(context.Background(), apiHeimdallTimeout)
func FetchWithRetry[T any](ctx context.Context, client http.Client, url *url.URL, closeCh chan struct{}) (*T, error) {
// request data once
body, err := internalFetch(ctx, client, url)
cancel()
if err == nil && body != nil {
err = json.Unmarshal(body, result)
if err != nil {
return nil, err
}
request := &Request{client: client, url: url, start: time.Now()}
result, err := Fetch[T](ctx, request)
if err == nil {
return result, nil
}
// attempt counter
attempt := 1
log.Warn("an error while trying fetching from Heimdall", "attempt", attempt, "error", err)
// create a new ticker for retrying the request
ticker := time.NewTicker(5 * time.Second)
ticker := time.NewTicker(retryCall)
defer ticker.Stop()
const logEach = 5
retryLoop:
for {
log.Info("Retrying again in 5 seconds to fetch data from Heimdall", "path", url.Path, "attempt", attempt)
attempt++
select {
case <-ctx.Done():
log.Debug("Shutdown detected, terminating request by context.Done")
return nil, ctx.Err()
case <-closeCh:
log.Debug("Shutdown detected, terminating request")
log.Debug("Shutdown detected, terminating request by closing")
return nil, errShutdownDetected
return nil, ErrShutdownDetected
case <-ticker.C:
ctx, cancel = context.WithTimeout(context.Background(), apiHeimdallTimeout)
request = &Request{client: client, url: url, start: time.Now()}
result, err = Fetch[T](ctx, request)
body, err = internalFetch(ctx, client, url)
cancel()
if err == nil && body != nil {
err = json.Unmarshal(body, result)
if err != nil {
return nil, err
if err != nil {
if attempt%logEach == 0 {
log.Warn("an error while trying fetching from Heimdall", "attempt", attempt, "error", err)
}
return result, nil
continue retryLoop
}
return result, nil
}
}
}
// Fetch returns data from heimdall
func Fetch[T any](ctx context.Context, request *Request) (*T, error) {
isSuccessful := false
defer func() {
if metrics.EnabledExpensive {
sendMetrics(ctx, request.start, isSuccessful)
}
}()
result := new(T)
body, err := internalFetchWithTimeout(ctx, request.client, request.url)
if err != nil {
return nil, err
}
if body == nil {
return nil, ErrNoResponse
}
err = json.Unmarshal(body, result)
if err != nil {
return nil, err
}
isSuccessful = true
return result, nil
}
func spanURL(urlString string, spanID uint64) (*url.URL, error) {
return makeURL(urlString, fmt.Sprintf(fetchSpanFormat, spanID), "")
}
@ -188,8 +255,19 @@ func stateSyncURL(urlString string, fromID uint64, to int64) (*url.URL, error) {
return makeURL(urlString, fetchStateSyncEventsPath, queryParams)
}
func latestCheckpointURL(urlString string) (*url.URL, error) {
return makeURL(urlString, fetchLatestCheckpoint, "")
func checkpointURL(urlString string, number int64) (*url.URL, error) {
url := ""
if number == -1 {
url = fmt.Sprintf(fetchCheckpoint, "latest")
} else {
url = fmt.Sprintf(fetchCheckpoint, fmt.Sprint(number))
}
return makeURL(urlString, url, "")
}
func checkpointCountURL(urlString string) (*url.URL, error) {
return makeURL(urlString, fetchCheckpointCount, "")
}
func makeURL(urlString, rawPath, rawQuery string) (*url.URL, error) {
@ -215,11 +293,12 @@ func internalFetch(ctx context.Context, client http.Client, u *url.URL) ([]byte,
if err != nil {
return nil, err
}
defer res.Body.Close()
// check status code
if res.StatusCode != 200 && res.StatusCode != 204 {
return nil, fmt.Errorf("Error while fetching data from Heimdall")
return nil, fmt.Errorf("%w: response code %d", ErrNotSuccessfulResponse, res.StatusCode)
}
// unmarshall data from buffer
@ -228,7 +307,7 @@ func internalFetch(ctx context.Context, client http.Client, u *url.URL) ([]byte,
}
// get response
body, err := ioutil.ReadAll(res.Body)
body, err := io.ReadAll(res.Body)
if err != nil {
return nil, err
}
@ -236,6 +315,14 @@ func internalFetch(ctx context.Context, client http.Client, u *url.URL) ([]byte,
return body, nil
}
func internalFetchWithTimeout(ctx context.Context, client http.Client, url *url.URL) ([]byte, error) {
ctx, cancel := context.WithTimeout(ctx, apiHeimdallTimeout)
defer cancel()
// request data once
return internalFetch(ctx, client, url)
}
// Close sends a signal to stop the running process
func (h *HeimdallClient) Close() {
close(h.closeCh)

297
consensus/bor/heimdall/client_test.go
View file

@ -1,9 +1,306 @@
package heimdall
import (
"context"
"encoding/json"
"errors"
"fmt"
"math/big"
"net"
"net/http"
"sync"
"testing"
"time"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/network"
"github.com/ethereum/go-ethereum/consensus/bor/heimdall/checkpoint"
"github.com/stretchr/testify/require"
)
// HttpHandlerFake defines the handler functions required to serve
// requests to the mock heimdal server for specific functions. Add more handlers
// according to requirements.
type HttpHandlerFake struct {
handleFetchCheckpoint http.HandlerFunc
}
func (h *HttpHandlerFake) GetCheckpointHandler() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
h.handleFetchCheckpoint.ServeHTTP(w, r)
}
}
func CreateMockHeimdallServer(wg *sync.WaitGroup, port int, listener net.Listener, handler *HttpHandlerFake) (*http.Server, error) {
// Create a new server mux
mux := http.NewServeMux()
// Create a route for fetching latest checkpoint
mux.HandleFunc("/checkpoints/latest", func(w http.ResponseWriter, r *http.Request) {
handler.GetCheckpointHandler()(w, r)
})
// Add other routes as per requirement
// Create the server with given port and mux
srv := &http.Server{
Addr: fmt.Sprintf("localhost:%d", port),
Handler: mux,
}
// Close the listener using the port and immediately consume it below
err := listener.Close()
if err != nil {
return nil, err
}
go func() {
defer wg.Done()
// always returns error. ErrServerClosed on graceful close
if err := srv.ListenAndServe(); err != http.ErrServerClosed {
fmt.Printf("error in server.ListenAndServe(): %v", err)
}
}()
return srv, nil
}
// TestFetchCheckpointFromMockHeimdall tests the heimdall client side logic
// to fetch checkpoints (latest for the scope of test) from a mock heimdall server.
// It can be used for debugging purpose (like response fields, marshalling/unmarshalling, etc).
func TestFetchCheckpointFromMockHeimdall(t *testing.T) {
t.Parallel()
// Create a wait group for sending across the mock server
wg := &sync.WaitGroup{}
wg.Add(1)
// Initialize the fake handler and add a fake checkpoint handler function
handler := &HttpHandlerFake{}
handler.handleFetchCheckpoint = func(w http.ResponseWriter, _ *http.Request) {
err := json.NewEncoder(w).Encode(checkpoint.CheckpointResponse{
Height: "0",
Result: checkpoint.Checkpoint{
Proposer: common.Address{},
StartBlock: big.NewInt(0),
EndBlock: big.NewInt(512),
RootHash: common.Hash{},
BorChainID: "15001",
Timestamp: 0,
},
})
if err != nil {
w.WriteHeader(500) // Return 500 Internal Server Error.
}
}
// Fetch available port
port, listener, err := network.FindAvailablePort()
require.NoError(t, err, "expect no error in finding available port")
// Create mock heimdall server and pass handler instance for setting up the routes
srv, err := CreateMockHeimdallServer(wg, port, listener, handler)
require.NoError(t, err, "expect no error in starting mock heimdall server")
// Create a new heimdall client and use same port for connection
client := NewHeimdallClient(fmt.Sprintf("http://localhost:%d", port))
_, err = client.FetchCheckpoint(context.Background(), -1)
require.NoError(t, err, "expect no error in fetching checkpoint")
// Shutdown the server
err = srv.Shutdown(context.TODO())
require.NoError(t, err, "expect no error in shutting down mock heimdall server")
// Wait for `wg.Done()` to be called in the mock server's routine.
wg.Wait()
}
// TestFetchShutdown tests the heimdall client side logic for context timeout and
// interrupt handling while fetching checkpoints (latest for the scope of test)
// from a mock heimdall server.
func TestFetchShutdown(t *testing.T) {
t.Parallel()
// Create a wait group for sending across the mock server
wg := &sync.WaitGroup{}
wg.Add(1)
// Initialize the fake handler and add a fake checkpoint handler function
handler := &HttpHandlerFake{}
// Case1 - Testing context timeout: Create delay in serving requests for simulating timeout. Add delay slightly
// greater than `retryDelay`. This should cause the request to timeout and trigger shutdown
// due to `ctx.Done()`. Expect context timeout error.
handler.handleFetchCheckpoint = func(w http.ResponseWriter, _ *http.Request) {
time.Sleep(6 * time.Second)
err := json.NewEncoder(w).Encode(checkpoint.CheckpointResponse{
Height: "0",
Result: checkpoint.Checkpoint{
Proposer: common.Address{},
StartBlock: big.NewInt(0),
EndBlock: big.NewInt(512),
RootHash: common.Hash{},
BorChainID: "15001",
Timestamp: 0,
},
})
if err != nil {
w.WriteHeader(500) // Return 500 Internal Server Error.
}
}
// Fetch available port
port, listener, err := network.FindAvailablePort()
require.NoError(t, err, "expect no error in finding available port")
// Create mock heimdall server and pass handler instance for setting up the routes
srv, err := CreateMockHeimdallServer(wg, port, listener, handler)
require.NoError(t, err, "expect no error in starting mock heimdall server")
// Create a new heimdall client and use same port for connection
client := NewHeimdallClient(fmt.Sprintf("http://localhost:%d", port))
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
// Expect this to fail due to timeout
_, err = client.FetchCheckpoint(ctx, -1)
require.Equal(t, "context deadline exceeded", err.Error(), "expect the function error to be a context deadline exeeded error")
require.Equal(t, "context deadline exceeded", ctx.Err().Error(), "expect the ctx error to be a context deadline exeeded error")
cancel()
// Case2 - Testing context cancellation. Pass a context with timeout to the request and
// cancel it before timeout. This should cause the request to timeout and trigger shutdown
// due to `ctx.Done()`. Expect context cancellation error.
handler.handleFetchCheckpoint = func(w http.ResponseWriter, _ *http.Request) {
time.Sleep(10 * time.Millisecond)
w.WriteHeader(500) // Return 500 Internal Server Error.
}
ctx, cancel = context.WithTimeout(context.Background(), 10*time.Second) // Use some high value for timeout
// Cancel the context after a delay until we make request
go func(cancel context.CancelFunc) {
time.Sleep(10 * time.Millisecond)
cancel()
}(cancel)
// Expect this to fail due to cancellation
_, err = client.FetchCheckpoint(ctx, -1)
require.Equal(t, "context canceled", err.Error(), "expect the function error to be a context cancelled error")
require.Equal(t, "context canceled", ctx.Err().Error(), "expect the ctx error to be a context cancelled error")
// Case3 - Testing interrupt: Closing the `closeCh` in heimdall client simulating interrupt. This
// should cause the request to fail and throw an error due to `<-closeCh` in fetchWithRetry.
// Expect shutdown detected error.
handler.handleFetchCheckpoint = func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(500) // Return 500 Internal Server Error.
}
// Close the channel after a delay until we make request
go func() {
time.Sleep(1 * time.Second)
close(client.closeCh)
}()
// Expect this to fail due to shutdown
_, err = client.FetchCheckpoint(context.Background(), -1)
require.Equal(t, ErrShutdownDetected.Error(), err.Error(), "expect the function error to be a shutdown detected error")
// Shutdown the server
err = srv.Shutdown(context.TODO())
require.NoError(t, err, "expect no error in shutting down mock heimdall server")
// Wait for `wg.Done()` to be called in the mock server's routine.
wg.Wait()
}
// TestContext includes bunch of simple tests to verify the working of timeout
// based context and cancellation.
func TestContext(t *testing.T) {
t.Parallel()
ctx, cancel1 := context.WithTimeout(context.Background(), 1*time.Second)
// Case1: Done is not yet closed, so Err returns nil.
require.NoError(t, ctx.Err(), "expect nil error")
wg := &sync.WaitGroup{}
// Case2: Check if timeout is being handled
wg.Add(1)
go func(ctx context.Context, wg *sync.WaitGroup) {
defer wg.Done()
select {
case <-ctx.Done():
// Expect context deadline exceeded error
require.Equal(t, "context deadline exceeded", ctx.Err().Error(), "expect the ctx error to be a context deadline exceeded error")
case <-time.After(2 * time.Second):
// Case for safely exiting the tests
return
}
}(ctx, wg)
// Case3: Check normal case
ctx, cancel2 := context.WithTimeout(context.Background(), 3*time.Second)
wg.Add(1)
errCh := make(chan error, 1)
go func(ctx context.Context, wg *sync.WaitGroup) {
defer wg.Done()
select {
case <-ctx.Done():
// Expect this to never occur, throw explicit error
errCh <- errors.New("unexpectecd call to `ctx.Done()`")
case <-time.After(2 * time.Second):
// Case for safely exiting the tests
errCh <- nil
return
}
}(ctx, wg)
if err := <-errCh; err != nil {
t.Fatalf("err: %v", err)
}
// Case4: Check if cancellation is being handled
ctx, cancel3 := context.WithTimeout(context.Background(), 1*time.Second)
wg.Add(1)
go func(cancel context.CancelFunc) {
time.Sleep(500 * time.Millisecond)
cancel()
}(cancel3)
go func(ctx context.Context, wg *sync.WaitGroup) {
defer wg.Done()
select {
case <-ctx.Done():
// Expect context canceled error
require.Equal(t, "context canceled", ctx.Err().Error(), "expect the ctx error to be a context canceled error")
case <-time.After(2 * time.Second):
// Case for safely exiting the tests
return
}
}(ctx, wg)
// Wait for all tests to pass
wg.Wait()
// Cancel all remaining contexts
cancel1()
cancel2()
}
func TestSpanURL(t *testing.T) {
t.Parallel()

82
consensus/bor/heimdall/metrics.go Normal file
View file

@ -0,0 +1,82 @@
package heimdall
import (
"context"
"time"
"github.com/ethereum/go-ethereum/metrics"
)
type (
requestTypeKey struct{}
requestType string
meter struct {
request map[bool]metrics.Meter // map[isSuccessful]metrics.Meter
timer metrics.Timer
}
)
const (
stateSyncRequest requestType = "state-sync"
spanRequest requestType = "span"
checkpointRequest requestType = "checkpoint"
checkpointCountRequest requestType = "checkpoint-count"
)
func withRequestType(ctx context.Context, reqType requestType) context.Context {
return context.WithValue(ctx, requestTypeKey{}, reqType)
}
func getRequestType(ctx context.Context) (requestType, bool) {
reqType, ok := ctx.Value(requestTypeKey{}).(requestType)
return reqType, ok
}
var (
requestMeters = map[requestType]meter{
stateSyncRequest: {
request: map[bool]metrics.Meter{
true: metrics.NewRegisteredMeter("client/requests/statesync/valid", nil),
false: metrics.NewRegisteredMeter("client/requests/statesync/invalid", nil),
},
timer: metrics.NewRegisteredTimer("client/requests/statesync/duration", nil),
},
spanRequest: {
request: map[bool]metrics.Meter{
true: metrics.NewRegisteredMeter("client/requests/span/valid", nil),
false: metrics.NewRegisteredMeter("client/requests/span/invalid", nil),
},
timer: metrics.NewRegisteredTimer("client/requests/span/duration", nil),
},
checkpointRequest: {
request: map[bool]metrics.Meter{
true: metrics.NewRegisteredMeter("client/requests/checkpoint/valid", nil),
false: metrics.NewRegisteredMeter("client/requests/checkpoint/invalid", nil),
},
timer: metrics.NewRegisteredTimer("client/requests/checkpoint/duration", nil),
},
checkpointCountRequest: {
request: map[bool]metrics.Meter{
true: metrics.NewRegisteredMeter("client/requests/checkpointcount/valid", nil),
false: metrics.NewRegisteredMeter("client/requests/checkpointcount/invalid", nil),
},
timer: metrics.NewRegisteredTimer("client/requests/checkpointcount/duration", nil),
},
}
)
func sendMetrics(ctx context.Context, start time.Time, isSuccessful bool) {
reqType, ok := getRequestType(ctx)
if !ok {
return
}
meters, ok := requestMeters[reqType]
if !ok {
return
}
meters.request[isSuccessful].Mark(1)
meters.timer.Update(time.Since(start))
}

14
consensus/bor/heimdall/span/spanner.go
View file

@ -39,12 +39,12 @@ func NewChainSpanner(ethAPI api.Caller, validatorSet abi.ABI, chainConfig *param
}
// GetCurrentSpan get current span from contract
func (c *ChainSpanner) GetCurrentSpan(headerHash common.Hash) (*Span, error) {
func (c *ChainSpanner) GetCurrentSpan(ctx context.Context, headerHash common.Hash) (*Span, error) {
// block
blockNr := rpc.BlockNumberOrHashWithHash(headerHash, false)
// method
method := "getCurrentSpan"
const method = "getCurrentSpan"
data, err := c.validatorSet.Pack(method)
if err != nil {
@ -58,7 +58,7 @@ func (c *ChainSpanner) GetCurrentSpan(headerHash common.Hash) (*Span, error) {
gas := (hexutil.Uint64)(uint64(math.MaxUint64 / 2))
// todo: would we like to have a timeout here?
result, err := c.ethAPI.Call(context.Background(), ethapi.TransactionArgs{
result, err := c.ethAPI.Call(ctx, ethapi.TransactionArgs{
Gas: &gas,
To: &toAddress,
Data: &msgData,
@ -89,8 +89,8 @@ func (c *ChainSpanner) GetCurrentSpan(headerHash common.Hash) (*Span, error) {
}
// GetCurrentValidators get current validators
func (c *ChainSpanner) GetCurrentValidators(headerHash common.Hash, blockNumber uint64) ([]*valset.Validator, error) {
ctx, cancel := context.WithCancel(context.Background())
func (c *ChainSpanner) GetCurrentValidators(ctx context.Context, headerHash common.Hash, blockNumber uint64) ([]*valset.Validator, error) {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
// method
@ -146,7 +146,7 @@ func (c *ChainSpanner) GetCurrentValidators(headerHash common.Hash, blockNumber
const method = "commitSpan"
func (c *ChainSpanner) CommitSpan(heimdallSpan HeimdallSpan, state *state.StateDB, header *types.Header, chainContext core.ChainContext) error {
func (c *ChainSpanner) CommitSpan(ctx context.Context, heimdallSpan HeimdallSpan, state *state.StateDB, header *types.Header, chainContext core.ChainContext) error {
// get validators bytes
validators := make([]valset.MinimalVal, 0, len(heimdallSpan.ValidatorSet.Validators))
for _, val := range heimdallSpan.ValidatorSet.Validators {
@ -194,7 +194,7 @@ func (c *ChainSpanner) CommitSpan(heimdallSpan HeimdallSpan, state *state.StateD
msg := statefull.GetSystemMessage(c.validatorContractAddress, data)
// apply message
_, err = statefull.ApplyMessage(msg, state, header, c.chainConfig, chainContext)
_, err = statefull.ApplyMessage(ctx, msg, state, header, c.chainConfig, chainContext)
return err
}

51
consensus/bor/heimdallgrpc/checkpoint.go Normal file
View file

@ -0,0 +1,51 @@
package heimdallgrpc
import (
"context"
"math/big"
"github.com/ethereum/go-ethereum/consensus/bor/heimdall/checkpoint"
"github.com/ethereum/go-ethereum/log"
proto "github.com/maticnetwork/polyproto/heimdall"
protoutils "github.com/maticnetwork/polyproto/utils"
)
func (h *HeimdallGRPCClient) FetchCheckpointCount(ctx context.Context) (int64, error) {
log.Info("Fetching checkpoint count")
res, err := h.client.FetchCheckpointCount(ctx, nil)
if err != nil {
return 0, err
}
log.Info("Fetched checkpoint count")
return res.Result.Result, nil
}
func (h *HeimdallGRPCClient) FetchCheckpoint(ctx context.Context, number int64) (*checkpoint.Checkpoint, error) {
req := &proto.FetchCheckpointRequest{
ID: number,
}
log.Info("Fetching checkpoint", "number", number)
res, err := h.client.FetchCheckpoint(ctx, req)
if err != nil {
return nil, err
}
log.Info("Fetched checkpoint", "number", number)
checkpoint := &checkpoint.Checkpoint{
StartBlock: new(big.Int).SetUint64(res.Result.StartBlock),
EndBlock: new(big.Int).SetUint64(res.Result.EndBlock),
RootHash: protoutils.ConvertH256ToHash(res.Result.RootHash),
Proposer: protoutils.ConvertH160toAddress(res.Result.Proposer),
BorChainID: res.Result.BorChainID,
Timestamp: uint64(res.Result.Timestamp.GetSeconds()),
}
return checkpoint, nil
}

52
consensus/bor/heimdallgrpc/client.go Normal file
View file

@ -0,0 +1,52 @@
package heimdallgrpc
import (
"time"
"github.com/ethereum/go-ethereum/log"
grpc_retry "github.com/grpc-ecosystem/go-grpc-middleware/retry"
proto "github.com/maticnetwork/polyproto/heimdall"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/credentials/insecure"
)
const (
stateFetchLimit = 50
)
type HeimdallGRPCClient struct {
conn *grpc.ClientConn
client proto.HeimdallClient
}
func NewHeimdallGRPCClient(address string) *HeimdallGRPCClient {
opts := []grpc_retry.CallOption{
grpc_retry.WithMax(10000),
grpc_retry.WithBackoff(grpc_retry.BackoffLinear(5 * time.Second)),
grpc_retry.WithCodes(codes.Internal, codes.Unavailable, codes.Aborted, codes.NotFound),
}
conn, err := grpc.Dial(address,
grpc.WithStreamInterceptor(grpc_retry.StreamClientInterceptor(opts...)),
grpc.WithUnaryInterceptor(grpc_retry.UnaryClientInterceptor(opts...)),
grpc.WithTransportCredentials(insecure.NewCredentials()),
)
if err != nil {
log.Crit("Failed to connect to Heimdall gRPC", "error", err)
}
log.Info("Connected to Heimdall gRPC server", "address", address)
return &HeimdallGRPCClient{
conn: conn,
client: proto.NewHeimdallClient(conn),
}
}
func (h *HeimdallGRPCClient) Close() {
log.Debug("Shutdown detected, Closing Heimdall gRPC client")
h.conn.Close()
}

63
consensus/bor/heimdallgrpc/span.go Normal file
View file

@ -0,0 +1,63 @@
package heimdallgrpc
import (
"context"
"github.com/ethereum/go-ethereum/consensus/bor/heimdall/span"
"github.com/ethereum/go-ethereum/consensus/bor/valset"
"github.com/ethereum/go-ethereum/log"
proto "github.com/maticnetwork/polyproto/heimdall"
protoutils "github.com/maticnetwork/polyproto/utils"
)
func (h *HeimdallGRPCClient) Span(ctx context.Context, spanID uint64) (*span.HeimdallSpan, error) {
req := &proto.SpanRequest{
ID: spanID,
}
log.Info("Fetching span", "spanID", spanID)
res, err := h.client.Span(ctx, req)
if err != nil {
return nil, err
}
log.Info("Fetched span", "spanID", spanID)
return parseSpan(res.Result), nil
}
func parseSpan(protoSpan *proto.Span) *span.HeimdallSpan {
resp := &span.HeimdallSpan{
Span: span.Span{
ID: protoSpan.ID,
StartBlock: protoSpan.StartBlock,
EndBlock: protoSpan.EndBlock,
},
ValidatorSet: valset.ValidatorSet{},
SelectedProducers: []valset.Validator{},
ChainID: protoSpan.ChainID,
}
for _, validator := range protoSpan.ValidatorSet.Validators {
resp.ValidatorSet.Validators = append(resp.ValidatorSet.Validators, parseValidator(validator))
}
resp.ValidatorSet.Proposer = parseValidator(protoSpan.ValidatorSet.Proposer)
for _, validator := range protoSpan.SelectedProducers {
resp.SelectedProducers = append(resp.SelectedProducers, *parseValidator(validator))
}
return resp
}
func parseValidator(validator *proto.Validator) *valset.Validator {
return &valset.Validator{
ID: validator.ID,
Address: protoutils.ConvertH160toAddress(validator.Address),
VotingPower: validator.VotingPower,
ProposerPriority: validator.ProposerPriority,
}
}

59
consensus/bor/heimdallgrpc/state_sync.go Normal file
View file

@ -0,0 +1,59 @@
package heimdallgrpc
import (
"context"
"errors"
"io"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/consensus/bor/clerk"
proto "github.com/maticnetwork/polyproto/heimdall"
)
func (h *HeimdallGRPCClient) StateSyncEvents(ctx context.Context, fromID uint64, to int64) ([]*clerk.EventRecordWithTime, error) {
eventRecords := make([]*clerk.EventRecordWithTime, 0)
req := &proto.StateSyncEventsRequest{
FromID: fromID,
ToTime: uint64(to),
Limit: uint64(stateFetchLimit),
}
var (
res proto.Heimdall_StateSyncEventsClient
events *proto.StateSyncEventsResponse
err error
)
res, err = h.client.StateSyncEvents(ctx, req)
if err != nil {
return nil, err
}
for {
events, err = res.Recv()
if errors.Is(err, io.EOF) {
return eventRecords, nil
}
if err != nil {
return nil, err
}
for _, event := range events.Result {
eventRecord := &clerk.EventRecordWithTime{
EventRecord: clerk.EventRecord{
ID: event.ID,
Contract: common.HexToAddress(event.Contract),
Data: common.Hex2Bytes(event.Data[2:]),
TxHash: common.HexToHash(event.TxHash),
LogIndex: event.LogIndex,
ChainID: event.ChainID,
},
Time: event.Time.AsTime(),
}
eventRecords = append(eventRecords, eventRecord)
}
}
}

2
consensus/bor/merkle.go
View file

@ -39,7 +39,7 @@ func convertTo32(input []byte) (output [32]byte) {
return
}
func convert(input []([32]byte)) [][]byte {
func convert(input [][32]byte) [][]byte {
output := make([][]byte, 0, len(input))
for _, in := range input {

20
consensus/bor/snapshot.go
View file

@ -59,6 +59,8 @@ func loadSnapshot(config *params.BorConfig, sigcache *lru.ARCCache, db ethdb.Dat
return nil, err
}
snap.ValidatorSet.UpdateValidatorMap()
snap.config = config
snap.sigcache = sigcache
@ -120,8 +122,8 @@ func (s *Snapshot) apply(headers []*types.Header) (*Snapshot, error) {
number := header.Number.Uint64()
// Delete the oldest signer from the recent list to allow it signing again
if number >= s.config.Sprint {
delete(snap.Recents, number-s.config.Sprint)
if number >= s.config.CalculateSprint(number) {
delete(snap.Recents, number-s.config.CalculateSprint(number))
}
// Resolve the authorization key and check against signers
@ -131,7 +133,7 @@ func (s *Snapshot) apply(headers []*types.Header) (*Snapshot, error) {
}
// check if signer is in validator set
if !snap.ValidatorSet.HasAddress(signer.Bytes()) {
if !snap.ValidatorSet.HasAddress(signer) {
return nil, &UnauthorizedSignerError{number, signer.Bytes()}
}
@ -143,7 +145,7 @@ func (s *Snapshot) apply(headers []*types.Header) (*Snapshot, error) {
snap.Recents[number] = signer
// change validator set and change proposer
if number > 0 && (number+1)%s.config.Sprint == 0 {
if number > 0 && (number+1)%s.config.CalculateSprint(number) == 0 {
if err := validateHeaderExtraField(header.Extra); err != nil {
return nil, err
}
@ -201,18 +203,18 @@ func (s *Snapshot) signers() []common.Address {
}
// Difficulty returns the difficulty for a particular signer at the current snapshot number
func (s *Snapshot) Difficulty(signer common.Address) uint64 {
func Difficulty(validatorSet *valset.ValidatorSet, signer common.Address) uint64 {
// if signer is empty
if signer == (common.Address{}) {
return 1
}
validators := s.ValidatorSet.Validators
proposer := s.ValidatorSet.GetProposer().Address
validators := validatorSet.Validators
proposer := validatorSet.GetProposer().Address
totalValidators := len(validators)
proposerIndex, _ := s.ValidatorSet.GetByAddress(proposer)
signerIndex, _ := s.ValidatorSet.GetByAddress(signer)
proposerIndex, _ := validatorSet.GetByAddress(proposer)
signerIndex, _ := validatorSet.GetByAddress(signer)
// temp index
tempIndex := signerIndex

99
consensus/bor/snapshot_test.go
View file

@ -1,12 +1,12 @@
package bor
import (
"math/rand"
"math/big"
"sort"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/maticnetwork/crand"
"github.com/stretchr/testify/require"
"pgregory.net/rapid"
"github.com/ethereum/go-ethereum/common"
@ -28,13 +28,13 @@ func TestGetSignerSuccessionNumber_ProposerIsSigner(t *testing.T) {
}
// proposer is signer
signer := validatorSet.Proposer.Address
successionNumber, err := snap.GetSignerSuccessionNumber(signer)
signerTest := validatorSet.Proposer.Address
successionNumber, err := snap.GetSignerSuccessionNumber(signerTest)
if err != nil {
t.Fatalf("%s", err)
}
assert.Equal(t, 0, successionNumber)
require.Equal(t, 0, successionNumber)
}
func TestGetSignerSuccessionNumber_SignerIndexIsLarger(t *testing.T) {
@ -54,13 +54,13 @@ func TestGetSignerSuccessionNumber_SignerIndexIsLarger(t *testing.T) {
}
// choose a signer at an index greater than proposer index
signer := snap.ValidatorSet.Validators[signerIndex].Address
successionNumber, err := snap.GetSignerSuccessionNumber(signer)
signerTest := snap.ValidatorSet.Validators[signerIndex].Address
successionNumber, err := snap.GetSignerSuccessionNumber(signerTest)
if err != nil {
t.Fatalf("%s", err)
}
assert.Equal(t, signerIndex-proposerIndex, successionNumber)
require.Equal(t, signerIndex-proposerIndex, successionNumber)
}
func TestGetSignerSuccessionNumber_SignerIndexIsSmaller(t *testing.T) {
@ -76,13 +76,13 @@ func TestGetSignerSuccessionNumber_SignerIndexIsSmaller(t *testing.T) {
}
// choose a signer at an index greater than proposer index
signer := snap.ValidatorSet.Validators[signerIndex].Address
successionNumber, err := snap.GetSignerSuccessionNumber(signer)
signerTest := snap.ValidatorSet.Validators[signerIndex].Address
successionNumber, err := snap.GetSignerSuccessionNumber(signerTest)
if err != nil {
t.Fatalf("%s", err)
}
assert.Equal(t, signerIndex+numVals-proposerIndex, successionNumber)
require.Equal(t, signerIndex+numVals-proposerIndex, successionNumber)
}
func TestGetSignerSuccessionNumber_ProposerNotFound(t *testing.T) {
@ -93,18 +93,20 @@ func TestGetSignerSuccessionNumber_ProposerNotFound(t *testing.T) {
ValidatorSet: valset.NewValidatorSet(validators),
}
dummyProposerAddress := randomAddress()
require.Len(t, snap.ValidatorSet.Validators, numVals)
dummyProposerAddress := randomAddress(toAddresses(validators)...)
snap.ValidatorSet.Proposer = &valset.Validator{Address: dummyProposerAddress}
// choose any signer
signer := snap.ValidatorSet.Validators[3].Address
signerTest := snap.ValidatorSet.Validators[3].Address
_, err := snap.GetSignerSuccessionNumber(signer)
assert.NotNil(t, err)
_, err := snap.GetSignerSuccessionNumber(signerTest)
require.NotNil(t, err)
e, ok := err.(*UnauthorizedProposerError)
assert.True(t, ok)
assert.Equal(t, dummyProposerAddress.Bytes(), e.Proposer)
require.True(t, ok)
require.Equal(t, dummyProposerAddress.Bytes(), e.Proposer)
}
func TestGetSignerSuccessionNumber_SignerNotFound(t *testing.T) {
@ -114,26 +116,30 @@ func TestGetSignerSuccessionNumber_SignerNotFound(t *testing.T) {
snap := Snapshot{
ValidatorSet: valset.NewValidatorSet(validators),
}
dummySignerAddress := randomAddress()
dummySignerAddress := randomAddress(toAddresses(validators)...)
_, err := snap.GetSignerSuccessionNumber(dummySignerAddress)
assert.NotNil(t, err)
require.NotNil(t, err)
e, ok := err.(*UnauthorizedSignerError)
assert.True(t, ok)
assert.Equal(t, dummySignerAddress.Bytes(), e.Signer)
require.True(t, ok)
require.Equal(t, dummySignerAddress.Bytes(), e.Signer)
}
// nolint: unparam
func buildRandomValidatorSet(numVals int) []*valset.Validator {
rand.Seed(time.Now().Unix())
validators := make([]*valset.Validator, numVals)
valAddrs := randomAddresses(numVals)
for i := 0; i < numVals; i++ {
power := crand.BigInt(big.NewInt(99))
powerN := power.Int64() + 1
validators[i] = &valset.Validator{
Address: valAddrs[i],
// cannot process validators with voting power 0, hence +1
VotingPower: int64(rand.Intn(99) + 1),
VotingPower: powerN,
}
}
@ -143,11 +149,23 @@ func buildRandomValidatorSet(numVals int) []*valset.Validator {
return validators
}
func randomAddress() common.Address {
bytes := make([]byte, 32)
rand.Read(bytes)
func randomAddress(exclude ...common.Address) common.Address {
excl := make(map[common.Address]struct{}, len(exclude))
return common.BytesToAddress(bytes)
for _, addr := range exclude {
excl[addr] = struct{}{}
}
r := crand.NewRand()
for {
addr := r.Address()
if _, ok := excl[addr]; ok {
continue
}
return addr
}
}
func randomAddresses(n int) []common.Address {
@ -158,17 +176,12 @@ func randomAddresses(n int) []common.Address {
addrs := make([]common.Address, 0, n)
addrsSet := make(map[common.Address]struct{}, n)
var (
addr common.Address
exist bool
)
var exist bool
bytes := make([]byte, 32)
r := crand.NewRand()
for {
rand.Read(bytes)
addr = common.BytesToAddress(bytes)
addr := r.Address()
_, exist = addrsSet[addr]
if !exist {
@ -187,7 +200,7 @@ func TestRandomAddresses(t *testing.T) {
t.Parallel()
rapid.Check(t, func(t *rapid.T) {
length := rapid.IntMax(100).Draw(t, "length").(int)
length := rapid.IntMax(300).Draw(t, "length").(int)
addrs := randomAddresses(length)
addressSet := unique.New(addrs)
@ -197,3 +210,13 @@ func TestRandomAddresses(t *testing.T) {
}
})
}
func toAddresses(vals []*valset.Validator) []common.Address {
addrs := make([]common.Address, len(vals))
for i, val := range vals {
addrs[i] = val.Address
}
return addrs
}

8
consensus/bor/span.go
View file

@ -1,6 +1,8 @@
package bor
import (
"context"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/consensus/bor/heimdall/span"
"github.com/ethereum/go-ethereum/consensus/bor/valset"
@ -11,7 +13,7 @@ import (
//go:generate mockgen -destination=./span_mock.go -package=bor . Spanner
type Spanner interface {
GetCurrentSpan(headerHash common.Hash) (*span.Span, error)
GetCurrentValidators(headerHash common.Hash, blockNumber uint64) ([]*valset.Validator, error)
CommitSpan(heimdallSpan span.HeimdallSpan, state *state.StateDB, header *types.Header, chainContext core.ChainContext) error
GetCurrentSpan(ctx context.Context, headerHash common.Hash) (*span.Span, error)
GetCurrentValidators(ctx context.Context, headerHash common.Hash, blockNumber uint64) ([]*valset.Validator, error)
CommitSpan(ctx context.Context, heimdallSpan span.HeimdallSpan, state *state.StateDB, header *types.Header, chainContext core.ChainContext) error
}

25
consensus/bor/span_mock.go
View file

@ -5,6 +5,7 @@
package bor
import (
context "context"
reflect "reflect"
common "github.com/ethereum/go-ethereum/common"
@ -40,45 +41,45 @@ func (m *MockSpanner) EXPECT() *MockSpannerMockRecorder {
}
// CommitSpan mocks base method.
func (m *MockSpanner) CommitSpan(arg0 span.HeimdallSpan, arg1 *state.StateDB, arg2 *types.Header, arg3 core.ChainContext) error {
func (m *MockSpanner) CommitSpan(arg0 context.Context, arg1 span.HeimdallSpan, arg2 *state.StateDB, arg3 *types.Header, arg4 core.ChainContext) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "CommitSpan", arg0, arg1, arg2, arg3)
ret := m.ctrl.Call(m, "CommitSpan", arg0, arg1, arg2, arg3, arg4)
ret0, _ := ret[0].(error)
return ret0
}
// CommitSpan indicates an expected call of CommitSpan.
func (mr *MockSpannerMockRecorder) CommitSpan(arg0, arg1, arg2, arg3 interface{}) *gomock.Call {
func (mr *MockSpannerMockRecorder) CommitSpan(arg0, arg1, arg2, arg3, arg4 interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CommitSpan", reflect.TypeOf((*MockSpanner)(nil).CommitSpan), arg0, arg1, arg2, arg3)
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CommitSpan", reflect.TypeOf((*MockSpanner)(nil).CommitSpan), arg0, arg1, arg2, arg3, arg4)
}
// GetCurrentSpan mocks base method.
func (m *MockSpanner) GetCurrentSpan(arg0 common.Hash) (*span.Span, error) {
func (m *MockSpanner) GetCurrentSpan(arg0 context.Context, arg1 common.Hash) (*span.Span, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetCurrentSpan", arg0)
ret := m.ctrl.Call(m, "GetCurrentSpan", arg0, arg1)
ret0, _ := ret[0].(*span.Span)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetCurrentSpan indicates an expected call of GetCurrentSpan.
func (mr *MockSpannerMockRecorder) GetCurrentSpan(arg0 interface{}) *gomock.Call {
func (mr *MockSpannerMockRecorder) GetCurrentSpan(arg0, arg1 interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetCurrentSpan", reflect.TypeOf((*MockSpanner)(nil).GetCurrentSpan), arg0)
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetCurrentSpan", reflect.TypeOf((*MockSpanner)(nil).GetCurrentSpan), arg0, arg1)
}
// GetCurrentValidators mocks base method.
func (m *MockSpanner) GetCurrentValidators(arg0 common.Hash, arg1 uint64) ([]*valset.Validator, error) {
func (m *MockSpanner) GetCurrentValidators(arg0 context.Context, arg1 common.Hash, arg2 uint64) ([]*valset.Validator, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetCurrentValidators", arg0, arg1)
ret := m.ctrl.Call(m, "GetCurrentValidators", arg0, arg1, arg2)
ret0, _ := ret[0].([]*valset.Validator)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetCurrentValidators indicates an expected call of GetCurrentValidators.
func (mr *MockSpannerMockRecorder) GetCurrentValidators(arg0, arg1 interface{}) *gomock.Call {
func (mr *MockSpannerMockRecorder) GetCurrentValidators(arg0, arg1, arg2 interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetCurrentValidators", reflect.TypeOf((*MockSpanner)(nil).GetCurrentValidators), arg0, arg1)
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetCurrentValidators", reflect.TypeOf((*MockSpanner)(nil).GetCurrentValidators), arg0, arg1, arg2)
}

2
consensus/bor/statefull/processor.go
View file

@ -1,6 +1,7 @@
package statefull
import (
"context"
"math"
"math/big"
@ -59,6 +60,7 @@ func GetSystemMessage(toAddress common.Address, data []byte) Callmsg {
// apply message
func ApplyMessage(
_ context.Context,
msg Callmsg,
state *state.StateDB,
header *types.Header,

11
consensus/bor/validators_getter.go
View file

@ -1,11 +0,0 @@
package bor
import (
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/consensus/bor/valset"
)
//go:generate mockgen -destination=./validators_getter_mock.go -package=bor . ValidatorsGetter
type ValidatorsGetter interface {
GetCurrentValidators(headerHash common.Hash, blockNumber uint64) ([]*valset.Validator, error)
}

51
consensus/bor/validators_getter_mock.go
View file

@ -1,51 +0,0 @@
// Code generated by MockGen. DO NOT EDIT.
// Source: github.com/ethereum/go-ethereum/consensus/bor (interfaces: ValidatorsGetter)
// Package bor is a generated GoMock package.
package bor
import (
reflect "reflect"
common "github.com/ethereum/go-ethereum/common"
valset "github.com/ethereum/go-ethereum/consensus/bor/valset"
gomock "github.com/golang/mock/gomock"
)
// MockValidatorsGetter is a mock of ValidatorsGetter interface.
type MockValidatorsGetter struct {
ctrl *gomock.Controller
recorder *MockValidatorsGetterMockRecorder
}
// MockValidatorsGetterMockRecorder is the mock recorder for MockValidatorsGetter.
type MockValidatorsGetterMockRecorder struct {
mock *MockValidatorsGetter
}
// NewMockValidatorsGetter creates a new mock instance.
func NewMockValidatorsGetter(ctrl *gomock.Controller) *MockValidatorsGetter {
mock := &MockValidatorsGetter{ctrl: ctrl}
mock.recorder = &MockValidatorsGetterMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use.
func (m *MockValidatorsGetter) EXPECT() *MockValidatorsGetterMockRecorder {
return m.recorder
}
// GetCurrentValidators mocks base method.
func (m *MockValidatorsGetter) GetCurrentValidators(arg0 common.Hash, arg1 uint64) ([]*valset.Validator, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetCurrentValidators", arg0, arg1)
ret0, _ := ret[0].([]*valset.Validator)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetCurrentValidators indicates an expected call of GetCurrentValidators.
func (mr *MockValidatorsGetterMockRecorder) GetCurrentValidators(arg0, arg1 interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetCurrentValidators", reflect.TypeOf((*MockValidatorsGetter)(nil).GetCurrentValidators), arg0, arg1)
}

29
consensus/bor/valset/validator.go
View file

@ -47,21 +47,26 @@ func (v *Validator) Cmp(other *Validator) *Validator {
return v
}
// nolint:nestif
if v.ProposerPriority > other.ProposerPriority {
return v
} else if v.ProposerPriority < other.ProposerPriority {
return other
} else {
result := bytes.Compare(v.Address.Bytes(), other.Address.Bytes())
if result < 0 {
return v
} else if result > 0 {
return other
} else {
panic("Cannot compare identical validators")
}
}
if v.ProposerPriority < other.ProposerPriority {
return other
}
result := bytes.Compare(v.Address.Bytes(), other.Address.Bytes())
if result == 0 {
panic("Cannot compare identical validators")
}
if result < 0 {
return v
}
// result > 0
return other
}
func (v *Validator) String() string {

70
consensus/bor/valset/validator_set.go
View file

@ -46,6 +46,7 @@ type ValidatorSet struct {
// cached (unexported)
totalVotingPower int64
validatorsMap map[common.Address]int // address -> index
}
// NewValidatorSet initializes a ValidatorSet by copying over the
@ -54,7 +55,7 @@ type ValidatorSet struct {
// The addresses of validators in `valz` must be unique otherwise the
// function panics.
func NewValidatorSet(valz []*Validator) *ValidatorSet {
vals := &ValidatorSet{}
vals := &ValidatorSet{validatorsMap: make(map[common.Address]int)}
err := vals.updateWithChangeSet(valz, false)
if err != nil {
@ -232,30 +233,34 @@ func validatorListCopy(valsList []*Validator) []*Validator {
// Copy each validator into a new ValidatorSet.
func (vals *ValidatorSet) Copy() *ValidatorSet {
valCopy := validatorListCopy(vals.Validators)
validatorsMap := make(map[common.Address]int, len(vals.Validators))
for i, val := range valCopy {
validatorsMap[val.Address] = i
}
return &ValidatorSet{
Validators: validatorListCopy(vals.Validators),
Proposer: vals.Proposer,
totalVotingPower: vals.totalVotingPower,
validatorsMap: validatorsMap,
}
}
// HasAddress returns true if address given is in the validator set, false -
// otherwise.
func (vals *ValidatorSet) HasAddress(address []byte) bool {
idx := sort.Search(len(vals.Validators), func(i int) bool {
return bytes.Compare(address, vals.Validators[i].Address.Bytes()) <= 0
})
func (vals *ValidatorSet) HasAddress(address common.Address) bool {
_, ok := vals.validatorsMap[address]
return idx < len(vals.Validators) && bytes.Equal(vals.Validators[idx].Address.Bytes(), address)
return ok
}
// GetByAddress returns an index of the validator with address and validator
// itself if found. Otherwise, -1 and nil are returned.
func (vals *ValidatorSet) GetByAddress(address common.Address) (index int, val *Validator) {
idx := sort.Search(len(vals.Validators), func(i int) bool {
return bytes.Compare(address.Bytes(), vals.Validators[i].Address.Bytes()) <= 0
})
if idx < len(vals.Validators) && vals.Validators[idx].Address == address {
idx, ok := vals.validatorsMap[address]
if ok {
return idx, vals.Validators[idx].Copy()
}
@ -265,14 +270,14 @@ func (vals *ValidatorSet) GetByAddress(address common.Address) (index int, val *
// GetByIndex returns the validator's address and validator itself by index.
// It returns nil values if index is less than 0 or greater or equal to
// len(ValidatorSet.Validators).
func (vals *ValidatorSet) GetByIndex(index int) (address []byte, val *Validator) {
func (vals *ValidatorSet) GetByIndex(index int) (address common.Address, val *Validator) {
if index < 0 || index >= len(vals.Validators) {
return nil, nil
return common.Address{}, nil
}
val = vals.Validators[index]
return val.Address.Bytes(), val.Copy()
return val.Address, val.Copy()
}
// Size returns the length of the validator set.
@ -328,7 +333,7 @@ func (vals *ValidatorSet) GetProposer() (proposer *Validator) {
func (vals *ValidatorSet) findProposer() *Validator {
var proposer *Validator
for _, val := range vals.Validators {
if proposer == nil || !bytes.Equal(val.Address.Bytes(), proposer.Address.Bytes()) {
if proposer == nil || val.Address != proposer.Address {
proposer = proposer.Cmp(val)
}
}
@ -371,14 +376,19 @@ func processChanges(origChanges []*Validator) (updates, removals []*Validator, e
changes := validatorListCopy(origChanges)
sort.Sort(ValidatorsByAddress(changes))
removals = make([]*Validator, 0, len(changes))
updates = make([]*Validator, 0, len(changes))
sliceCap := len(changes) / 2
if sliceCap == 0 {
sliceCap = 1
}
removals = make([]*Validator, 0, sliceCap)
updates = make([]*Validator, 0, sliceCap)
var prevAddr common.Address
// Scan changes by address and append valid validators to updates or removals lists.
for _, valUpdate := range changes {
if bytes.Equal(valUpdate.Address.Bytes(), prevAddr.Bytes()) {
if valUpdate.Address == prevAddr {
err = fmt.Errorf("duplicate entry %v in %v", valUpdate, changes)
return nil, nil, err
}
@ -489,10 +499,11 @@ func (vals *ValidatorSet) applyUpdates(updates []*Validator) {
} else {
// Apply add or update.
merged[i] = updates[0]
if bytes.Equal(existing[0].Address.Bytes(), updates[0].Address.Bytes()) {
if existing[0].Address == updates[0].Address {
// Validator is present in both, advance existing.
existing = existing[1:]
}
updates = updates[1:]
}
i++
@ -503,6 +514,7 @@ func (vals *ValidatorSet) applyUpdates(updates []*Validator) {
merged[i] = existing[j]
i++
}
// OR add updates which are left.
for j := 0; j < len(updates); j++ {
merged[i] = updates[j]
@ -541,7 +553,7 @@ func (vals *ValidatorSet) applyRemovals(deletes []*Validator) {
// Loop over deletes until we removed all of them.
for len(deletes) > 0 {
if bytes.Equal(existing[0].Address.Bytes(), deletes[0].Address.Bytes()) {
if existing[0].Address == deletes[0].Address {
deletes = deletes[1:]
} else { // Leave it in the resulting slice.
merged[i] = existing[0]
@ -599,8 +611,7 @@ func (vals *ValidatorSet) updateWithChangeSet(changes []*Validator, allowDeletes
computeNewPriorities(updates, vals, updatedTotalVotingPower)
// Apply updates and removals.
vals.applyUpdates(updates)
vals.applyRemovals(deletes)
vals.updateValidators(updates, deletes)
if err := vals.UpdateTotalVotingPower(); err != nil {
return err
@ -613,6 +624,21 @@ func (vals *ValidatorSet) updateWithChangeSet(changes []*Validator, allowDeletes
return nil
}
func (vals *ValidatorSet) updateValidators(updates []*Validator, deletes []*Validator) {
vals.applyUpdates(updates)
vals.applyRemovals(deletes)
vals.UpdateValidatorMap()
}
func (vals *ValidatorSet) UpdateValidatorMap() {
vals.validatorsMap = make(map[common.Address]int, len(vals.Validators))
for i, val := range vals.Validators {
vals.validatorsMap[val.Address] = i
}
}
// UpdateWithChangeSet attempts to update the validator set with 'changes'.
// It performs the following steps:
// - validates the changes making sure there are no duplicates and splits them in updates and deletes
@ -661,7 +687,7 @@ func (vals *ValidatorSet) StringIndented(indent string) string {
return "nil-ValidatorSet"
}
var valStrings []string
valStrings := make([]string, 0, len(vals.Validators))
vals.Iterate(func(index int, val *Validator) bool {
valStrings = append(valStrings, val.String())

199
consensus/bor/valset/validator_set_test.go Normal file
View file

@ -0,0 +1,199 @@
package valset
import (
"testing"
"github.com/stretchr/testify/require"
"gotest.tools/assert"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/crypto"
)
func NewValidatorFromKey(key string, votingPower int64) *Validator {
privKey, _ := crypto.HexToECDSA(key)
return NewValidator(crypto.PubkeyToAddress(privKey.PublicKey), votingPower)
}
func GetValidators() [4]*Validator {
const (
// addr0 = 0x96C42C56fdb78294F96B0cFa33c92bed7D75F96a
signer0 = "c8deb0bea5c41afe8e37b4d1bd84e31adff11b09c8c96ff4b605003cce067cd9"
// addr1 = 0x98925BE497f6dFF6A5a33dDA8B5933cA35262d69
signer1 = "c8deb0bea5c41afe8e37b4d1bd84e31adff11b09c8c96ff4b605003cce067cd8"
//addr2 = 0x648Cf2A5b119E2c04061021834F8f75735B1D36b
signer2 = "c8deb0bea5c41afe8e37b4d1bd84e31adff11b09c8c96ff4b605003cce067cd7"
//addr3 = 0x168f220B3b313D456eD4797520eFdFA9c57E6C45
signer3 = "c8deb0bea5c41afe8e37b4d1bd84e31adff11b09c8c96ff4b605003cce067cd6"
)
return [4]*Validator{
NewValidatorFromKey(signer0, 100),
NewValidatorFromKey(signer1, 200),
NewValidatorFromKey(signer2, 300),
NewValidatorFromKey(signer3, 400),
}
}
func TestIncrementProposerPriority(t *testing.T) {
t.Parallel()
vals := GetValidators()
// Validator set length = 1
valSet := NewValidatorSet(vals[:1])
expectedPropsers := []*Validator{vals[0], vals[0], vals[0], vals[0], vals[0], vals[0], vals[0], vals[0], vals[0], vals[0]}
for i := 0; i < 10; i++ {
valSet.IncrementProposerPriority(1)
require.Equal(t, expectedPropsers[i].Address, valSet.GetProposer().Address)
}
// Validator set length = 2
valSet = NewValidatorSet(vals[:2])
expectedPropsers = []*Validator{vals[0], vals[1], vals[1], vals[0], vals[1], vals[1], vals[0], vals[1], vals[1], vals[0]}
for i := 0; i < 10; i++ {
valSet.IncrementProposerPriority(1)
require.Equal(t, expectedPropsers[i].Address, valSet.GetProposer().Address)
}
// Validator set length = 3
valSet = NewValidatorSet(vals[:3])
expectedPropsers = []*Validator{vals[1], vals[2], vals[0], vals[1], vals[2], vals[2], vals[1], vals[2], vals[0], vals[1]}
for i := 0; i < 10; i++ {
valSet.IncrementProposerPriority(1)
require.Equal(t, expectedPropsers[i].Address, valSet.GetProposer().Address)
}
// Validator set length = 4
valSet = NewValidatorSet(vals[:4])
expectedPropsers = []*Validator{vals[2], vals[1], vals[3], vals[2], vals[0], vals[3], vals[1], vals[2], vals[3], vals[3]}
for i := 0; i < 10; i++ {
valSet.IncrementProposerPriority(1)
require.Equal(t, expectedPropsers[i].Address, valSet.GetProposer().Address)
}
}
func TestRescalePriorities(t *testing.T) {
t.Parallel()
vals := GetValidators()
// Validator set length = 1
valSet := NewValidatorSet(vals[:1])
valSet.RescalePriorities(10)
expectedPriorities := []int64{0}
for i, val := range valSet.Validators {
require.Equal(t, expectedPriorities[i], val.ProposerPriority)
}
// Validator set length = 2
valSet = NewValidatorSet(vals[:2])
valSet.RescalePriorities(100)
expectedPriorities = []int64{50, -50}
for i, val := range valSet.Validators {
require.Equal(t, expectedPriorities[i], val.ProposerPriority)
}
// Validator set length = 3
valSet = NewValidatorSet(vals[:3])
valSet.RescalePriorities(30)
expectedPriorities = []int64{-17, 5, 11}
for i, val := range valSet.Validators {
require.Equal(t, expectedPriorities[i], val.ProposerPriority)
}
// Validator set length = 4
valSet = NewValidatorSet(vals[:4])
valSet.RescalePriorities(10)
expectedPriorities = []int64{-6, 3, 1, 2}
for i, val := range valSet.Validators {
require.Equal(t, expectedPriorities[i], val.ProposerPriority)
}
}
func TestGetValidatorByAddressAndIndex(t *testing.T) {
t.Parallel()
vals := GetValidators()
valSet := NewValidatorSet(vals[:4])
for _, val := range valSet.Validators {
idx, valByAddress := valSet.GetByAddress(val.Address)
addr, valByIndex := valSet.GetByIndex(idx)
assert.DeepEqual(t, val, valByIndex)
assert.DeepEqual(t, val, valByAddress)
assert.DeepEqual(t, val.Address, addr)
}
tempAddress := common.HexToAddress("0x12345")
// Negative Testcase
idx, _ := valSet.GetByAddress(tempAddress)
require.Equal(t, idx, -1)
// checking for validator index out of range
addr, _ := valSet.GetByIndex(100)
require.Equal(t, addr, common.Address{})
}
func TestUpdateWithChangeSet(t *testing.T) {
t.Parallel()
vals := GetValidators()
valSet := NewValidatorSet(vals[:4])
// halved the power of vals[2] and doubled the power of vals[3]
vals[2].VotingPower = 150
vals[3].VotingPower = 800
// Adding new temp validator in the set
const tempSigner = "c8deb0bea5c41afe8e37b4d1bd84e31adff11b09c8c96ff4b605003cce067cd5"
tempVal := NewValidatorFromKey(tempSigner, 250)
// check totalVotingPower before updating validator set
require.Equal(t, int64(1000), valSet.TotalVotingPower())
err := valSet.UpdateWithChangeSet([]*Validator{vals[2], vals[3], tempVal})
require.NoError(t, err)
// check totalVotingPower after updating validator set
require.Equal(t, int64(1500), valSet.TotalVotingPower())
_, updatedVal2 := valSet.GetByAddress(vals[2].Address)
require.Equal(t, int64(150), updatedVal2.VotingPower)
_, updatedVal3 := valSet.GetByAddress(vals[3].Address)
require.Equal(t, int64(800), updatedVal3.VotingPower)
_, updatedTempVal := valSet.GetByAddress(tempVal.Address)
require.Equal(t, int64(250), updatedTempVal.VotingPower)
}

5
consensus/clique/clique.go
View file

@ -19,6 +19,7 @@ package clique
import (
"bytes"
"context"
"errors"
"fmt"
"io"
@ -569,7 +570,7 @@ func (c *Clique) Finalize(chain consensus.ChainHeaderReader, header *types.Heade
// FinalizeAndAssemble implements consensus.Engine, ensuring no uncles are set,
// nor block rewards given, and returns the final block.
func (c *Clique) FinalizeAndAssemble(chain consensus.ChainHeaderReader, header *types.Header, state *state.StateDB, txs []*types.Transaction, uncles []*types.Header, receipts []*types.Receipt) (*types.Block, error) {
func (c *Clique) FinalizeAndAssemble(ctx context.Context, chain consensus.ChainHeaderReader, header *types.Header, state *state.StateDB, txs []*types.Transaction, uncles []*types.Header, receipts []*types.Receipt) (*types.Block, error) {
// Finalize block
c.Finalize(chain, header, state, txs, uncles)
@ -589,7 +590,7 @@ func (c *Clique) Authorize(signer common.Address, signFn SignerFn) {
// Seal implements consensus.Engine, attempting to create a sealed block using
// the local signing credentials.
func (c *Clique) Seal(chain consensus.ChainHeaderReader, block *types.Block, results chan<- *types.Block, stop <-chan struct{}) error {
func (c *Clique) Seal(ctx context.Context, chain consensus.ChainHeaderReader, block *types.Block, results chan<- *types.Block, stop <-chan struct{}) error {
header := block.Header()
// Sealing the genesis block is not supported

6
consensus/clique/clique_test.go
View file

@ -55,7 +55,7 @@ func TestReimportMirroredState(t *testing.T) {
genesis := genspec.MustCommit(db)
// Generate a batch of blocks, each properly signed
chain, _ := core.NewBlockChain(db, nil, params.AllCliqueProtocolChanges, engine, vm.Config{}, nil, nil)
chain, _ := core.NewBlockChain(db, nil, params.AllCliqueProtocolChanges, engine, vm.Config{}, nil, nil, nil)
defer chain.Stop()
blocks, _ := core.GenerateChain(params.AllCliqueProtocolChanges, genesis, engine, db, 3, func(i int, block *core.BlockGen) {
@ -89,7 +89,7 @@ func TestReimportMirroredState(t *testing.T) {
db = rawdb.NewMemoryDatabase()
genspec.MustCommit(db)
chain, _ = core.NewBlockChain(db, nil, params.AllCliqueProtocolChanges, engine, vm.Config{}, nil, nil)
chain, _ = core.NewBlockChain(db, nil, params.AllCliqueProtocolChanges, engine, vm.Config{}, nil, nil, nil)
defer chain.Stop()
if _, err := chain.InsertChain(blocks[:2]); err != nil {
@ -102,7 +102,7 @@ func TestReimportMirroredState(t *testing.T) {
// Simulate a crash by creating a new chain on top of the database, without
// flushing the dirty states out. Insert the last block, triggering a sidechain
// reimport.
chain, _ = core.NewBlockChain(db, nil, params.AllCliqueProtocolChanges, engine, vm.Config{}, nil, nil)
chain, _ = core.NewBlockChain(db, nil, params.AllCliqueProtocolChanges, engine, vm.Config{}, nil, nil, nil)
defer chain.Stop()
if _, err := chain.InsertChain(blocks[2:]); err != nil {

2
consensus/clique/snapshot_test.go
View file

@ -450,7 +450,7 @@ func TestClique(t *testing.T) {
batches[len(batches)-1] = append(batches[len(batches)-1], block)
}
// Pass all the headers through clique and ensure tallying succeeds
chain, err := core.NewBlockChain(db, nil, &config, engine, vm.Config{}, nil, nil)
chain, err := core.NewBlockChain(db, nil, &config, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Errorf("test %d: failed to create test chain: %v", i, err)
continue

5
consensus/consensus.go
View file

@ -18,6 +18,7 @@
package consensus
import (
"context"
"math/big"
"github.com/ethereum/go-ethereum/common"
@ -97,7 +98,7 @@ type Engine interface {
//
// Note: The block header and state database might be updated to reflect any
// consensus rules that happen at finalization (e.g. block rewards).
FinalizeAndAssemble(chain ChainHeaderReader, header *types.Header, state *state.StateDB, txs []*types.Transaction,
FinalizeAndAssemble(ctx context.Context, chain ChainHeaderReader, header *types.Header, state *state.StateDB, txs []*types.Transaction,
uncles []*types.Header, receipts []*types.Receipt) (*types.Block, error)
// Seal generates a new sealing request for the given input block and pushes
@ -105,7 +106,7 @@ type Engine interface {
//
// Note, the method returns immediately and will send the result async. More
// than one result may also be returned depending on the consensus algorithm.
Seal(chain ChainHeaderReader, block *types.Block, results chan<- *types.Block, stop <-chan struct{}) error
Seal(ctx context.Context, chain ChainHeaderReader, block *types.Block, results chan<- *types.Block, stop <-chan struct{}) error
// SealHash returns the hash of a block prior to it being sealed.
SealHash(header *types.Header) common.Hash

3
consensus/ethash/consensus.go
View file

@ -18,6 +18,7 @@ package ethash
import (
"bytes"
"context"
"errors"
"fmt"
"math/big"
@ -598,7 +599,7 @@ func (ethash *Ethash) Finalize(chain consensus.ChainHeaderReader, header *types.
// FinalizeAndAssemble implements consensus.Engine, accumulating the block and
// uncle rewards, setting the final state and assembling the block.
func (ethash *Ethash) FinalizeAndAssemble(chain consensus.ChainHeaderReader, header *types.Header, state *state.StateDB, txs []*types.Transaction, uncles []*types.Header, receipts []*types.Receipt) (*types.Block, error) {
func (ethash *Ethash) FinalizeAndAssemble(ctx context.Context, chain consensus.ChainHeaderReader, header *types.Header, state *state.StateDB, txs []*types.Transaction, uncles []*types.Header, receipts []*types.Receipt) (*types.Block, error) {
// Finalize block
ethash.Finalize(chain, header, state, txs, uncles)

20
consensus/ethash/ethash_test.go
View file

@ -17,6 +17,7 @@
package ethash
import (
"context"
"io/ioutil"
"math/big"
"math/rand"
@ -38,7 +39,7 @@ func TestTestMode(t *testing.T) {
defer ethash.Close()
results := make(chan *types.Block)
err := ethash.Seal(nil, types.NewBlockWithHeader(header), results, nil)
err := ethash.Seal(context.Background(), nil, types.NewBlockWithHeader(header), results, nil)
if err != nil {
t.Fatalf("failed to seal block: %v", err)
}
@ -111,12 +112,13 @@ func TestRemoteSealer(t *testing.T) {
// Push new work.
results := make(chan *types.Block)
ethash.Seal(nil, block, results, nil)
err := ethash.Seal(context.Background(), nil, block, results, nil)
var (
work [4]string
err error
)
if err != nil {
t.Error("error in sealing block")
}
var work [4]string
if work, err = api.GetWork(); err != nil || work[0] != sealhash.Hex() {
t.Error("expect to return a mining work has same hash")
}
@ -128,7 +130,11 @@ func TestRemoteSealer(t *testing.T) {
header = &types.Header{Number: big.NewInt(1), Difficulty: big.NewInt(1000)}
block = types.NewBlockWithHeader(header)
sealhash = ethash.SealHash(header)
ethash.Seal(nil, block, results, nil)
err = ethash.Seal(context.Background(), nil, block, results, nil)
if err != nil {
t.Error("error in sealing block")
}
if work, err = api.GetWork(); err != nil || work[0] != sealhash.Hex() {
t.Error("expect to return the latest pushed work")

7
consensus/ethash/sealer.go
View file

@ -48,7 +48,7 @@ var (
// Seal implements consensus.Engine, attempting to find a nonce that satisfies
// the block's difficulty requirements.
func (ethash *Ethash) Seal(chain consensus.ChainHeaderReader, block *types.Block, results chan<- *types.Block, stop <-chan struct{}) error {
func (ethash *Ethash) Seal(ctx context.Context, chain consensus.ChainHeaderReader, block *types.Block, results chan<- *types.Block, stop <-chan struct{}) error {
// If we're running a fake PoW, simply return a 0 nonce immediately
if ethash.config.PowMode == ModeFake || ethash.config.PowMode == ModeFullFake {
header := block.Header()
@ -62,7 +62,7 @@ func (ethash *Ethash) Seal(chain consensus.ChainHeaderReader, block *types.Block
}
// If we're running a shared PoW, delegate sealing to it
if ethash.shared != nil {
return ethash.shared.Seal(chain, block, results, stop)
return ethash.shared.Seal(ctx, chain, block, results, stop)
}
// Create a runner and the multiple search threads it directs
abort := make(chan struct{})
@ -117,7 +117,8 @@ func (ethash *Ethash) Seal(chain consensus.ChainHeaderReader, block *types.Block
case <-ethash.update:
// Thread count was changed on user request, restart
close(abort)
if err := ethash.Seal(chain, block, results, stop); err != nil {
if err := ethash.Seal(ctx, chain, block, results, stop); err != nil {
ethash.config.Log.Error("Failed to restart sealing after update", "err", err)
}
}

34
consensus/ethash/sealer_test.go
View file

@ -17,6 +17,7 @@
package ethash
import (
"context"
"encoding/json"
"io/ioutil"
"math/big"
@ -57,7 +58,11 @@ func TestRemoteNotify(t *testing.T) {
header := &types.Header{Number: big.NewInt(1), Difficulty: big.NewInt(100)}
block := types.NewBlockWithHeader(header)
ethash.Seal(nil, block, nil, nil)
err := ethash.Seal(context.Background(), nil, block, nil, nil)
if err != nil {
t.Error("error in sealing block")
}
select {
case work := <-sink:
if want := ethash.SealHash(header).Hex(); work[0] != want {
@ -105,7 +110,11 @@ func TestRemoteNotifyFull(t *testing.T) {
header := &types.Header{Number: big.NewInt(1), Difficulty: big.NewInt(100)}
block := types.NewBlockWithHeader(header)
ethash.Seal(nil, block, nil, nil)
err := ethash.Seal(context.Background(), nil, block, nil, nil)
if err != nil {
t.Error("error in sealing block")
}
select {
case work := <-sink:
if want := "0x" + strconv.FormatUint(header.Number.Uint64(), 16); work["number"] != want {
@ -151,7 +160,11 @@ func TestRemoteMultiNotify(t *testing.T) {
for i := 0; i < cap(sink); i++ {
header := &types.Header{Number: big.NewInt(int64(i)), Difficulty: big.NewInt(100)}
block := types.NewBlockWithHeader(header)
ethash.Seal(nil, block, results, nil)
err := ethash.Seal(context.Background(), nil, block, results, nil)
if err != nil {
t.Error("error in sealing block")
}
}
for i := 0; i < cap(sink); i++ {
@ -180,6 +193,9 @@ func TestRemoteMultiNotifyFull(t *testing.T) {
}
sink <- work
}))
// Allowing the server to start listening.
time.Sleep(2 * time.Second)
defer server.Close()
// Create the custom ethash engine.
@ -200,7 +216,11 @@ func TestRemoteMultiNotifyFull(t *testing.T) {
for i := 0; i < cap(sink); i++ {
header := &types.Header{Number: big.NewInt(int64(i)), Difficulty: big.NewInt(100)}
block := types.NewBlockWithHeader(header)
ethash.Seal(nil, block, results, nil)
err := ethash.Seal(context.Background(), nil, block, results, nil)
if err != nil {
t.Error("error in sealing block")
}
}
for i := 0; i < cap(sink); i++ {
@ -266,7 +286,11 @@ func TestStaleSubmission(t *testing.T) {
for id, c := range testcases {
for _, h := range c.headers {
ethash.Seal(nil, types.NewBlockWithHeader(h), results, nil)
err := ethash.Seal(context.Background(), nil, types.NewBlockWithHeader(h), results, nil)
if err != nil {
t.Error("error in sealing block")
}
}
if res := api.SubmitWork(fakeNonce, ethash.SealHash(c.headers[c.submitIndex]), fakeDigest); res != c.submitRes {
t.Errorf("case %d submit result mismatch, want %t, get %t", id+1, c.submitRes, res)

7
consensus/misc/eip1559.go
View file

@ -59,9 +59,10 @@ func CalcBaseFee(config *params.ChainConfig, parent *types.Header) *big.Int {
}
var (
parentGasTarget = parent.GasLimit / params.ElasticityMultiplier
parentGasTargetBig = new(big.Int).SetUint64(parentGasTarget)
baseFeeChangeDenominator = new(big.Int).SetUint64(params.BaseFeeChangeDenominator)
parentGasTarget = parent.GasLimit / params.ElasticityMultiplier
parentGasTargetBig = new(big.Int).SetUint64(parentGasTarget)
baseFeeChangeDenominatorUint64 = params.BaseFeeChangeDenominator(config.Bor, parent.Number)
baseFeeChangeDenominator = new(big.Int).SetUint64(baseFeeChangeDenominatorUint64)
)
// If the parent gasUsed is the same as the target, the baseFee remains unchanged.
if parent.GasUsed == parentGasTarget {

44
consensus/misc/eip1559_test.go
View file

@ -20,7 +20,6 @@ import (
"math/big"
"testing"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/params"
)
@ -47,12 +46,14 @@ func copyConfig(original *params.ChainConfig) *params.ChainConfig {
TerminalTotalDifficulty: original.TerminalTotalDifficulty,
Ethash: original.Ethash,
Clique: original.Clique,
Bor: original.Bor,
}
}
func config() *params.ChainConfig {
config := copyConfig(params.TestChainConfig)
config.LondonBlock = big.NewInt(5)
config.Bor.DelhiBlock = big.NewInt(8)
return config
}
@ -108,6 +109,8 @@ func TestBlockGasLimits(t *testing.T) {
// TestCalcBaseFee assumes all blocks are 1559-blocks
func TestCalcBaseFee(t *testing.T) {
t.Parallel()
tests := []struct {
parentBaseFee int64
parentGasLimit uint64
@ -117,10 +120,12 @@ func TestCalcBaseFee(t *testing.T) {
{params.InitialBaseFee, 20000000, 10000000, params.InitialBaseFee}, // usage == target
{params.InitialBaseFee, 20000000, 9000000, 987500000}, // usage below target
{params.InitialBaseFee, 20000000, 11000000, 1012500000}, // usage above target
{params.InitialBaseFee, 20000000, 20000000, 1125000000}, // usage full
{params.InitialBaseFee, 20000000, 0, 875000000}, // usage 0
}
for i, test := range tests {
parent := &types.Header{
Number: common.Big32,
Number: big.NewInt(6),
GasLimit: test.parentGasLimit,
GasUsed: test.parentGasUsed,
BaseFee: big.NewInt(test.parentBaseFee),
@ -130,3 +135,38 @@ func TestCalcBaseFee(t *testing.T) {
}
}
}
// TestCalcBaseFee assumes all blocks are 1559-blocks post Delhi Hard Fork
func TestCalcBaseFeeDelhi(t *testing.T) {
t.Parallel()
testConfig := copyConfig(config())
// Test Delhi Hard Fork
// Hard fork kicks in at block 8
tests := []struct {
parentBaseFee int64
parentGasLimit uint64
parentGasUsed uint64
expectedBaseFee int64
}{
{params.InitialBaseFee, 20000000, 10000000, params.InitialBaseFee}, // usage == target
{params.InitialBaseFee, 20000000, 9000000, 993750000}, // usage below target
{params.InitialBaseFee, 20000000, 11000000, 1006250000}, // usage above target
{params.InitialBaseFee, 20000000, 20000000, 1062500000}, // usage full
{params.InitialBaseFee, 20000000, 0, 937500000}, // usage 0
}
for i, test := range tests {
parent := &types.Header{
Number: big.NewInt(8),
GasLimit: test.parentGasLimit,
GasUsed: test.parentGasUsed,
BaseFee: big.NewInt(test.parentBaseFee),
}
if have, want := CalcBaseFee(testConfig, parent), big.NewInt(test.expectedBaseFee); have.Cmp(want) != 0 {
t.Errorf("test %d: have %d want %d, ", i, have, want)
}
}
}

7
core/bench_test.go
View file

@ -162,7 +162,7 @@ func genTxRing(naccounts int) func(int, *BlockGen) {
// genUncles generates blocks with two uncle headers.
func genUncles(i int, gen *BlockGen) {
if i >= 6 {
if i >= 7 {
b2 := gen.PrevBlock(i - 6).Header()
b2.Extra = []byte("foo")
gen.AddUncle(b2)
@ -201,7 +201,7 @@ func benchInsertChain(b *testing.B, disk bool, gen func(int, *BlockGen)) {
// Time the insertion of the new chain.
// State and blocks are stored in the same DB.
chainman, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
chainman, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer chainman.Stop()
b.ReportAllocs()
b.ResetTimer()
@ -317,7 +317,8 @@ func benchReadChain(b *testing.B, full bool, count uint64) {
if err != nil {
b.Fatalf("error opening database at %v: %v", dir, err)
}
chain, err := NewBlockChain(db, &cacheConfig, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, nil)
chain, err := NewBlockChain(db, &cacheConfig, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
if err != nil {
b.Fatalf("error creating chain: %v", err)
}

10
core/block_validator_test.go
View file

@ -49,7 +49,7 @@ func TestHeaderVerification(t *testing.T) {
headers[i] = block.Header()
}
// Run the header checker for blocks one-by-one, checking for both valid and invalid nonces
chain, _ := NewBlockChain(testdb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, nil)
chain, _ := NewBlockChain(testdb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer chain.Stop()
for i := 0; i < len(blocks); i++ {
@ -168,7 +168,7 @@ func testHeaderVerificationForMerging(t *testing.T, isClique bool) {
t.Logf("Log header after the merging %d: %v", block.NumberU64(), string(blob))
}
// Run the header checker for blocks one-by-one, checking for both valid and invalid nonces
chain, _ := NewBlockChain(testdb, nil, chainConfig, runEngine, vm.Config{}, nil, nil)
chain, _ := NewBlockChain(testdb, nil, chainConfig, runEngine, vm.Config{}, nil, nil, nil)
defer chain.Stop()
// Verify the blocks before the merging
@ -279,11 +279,11 @@ func testHeaderConcurrentVerification(t *testing.T, threads int) {
var results <-chan error
if valid {
chain, _ := NewBlockChain(testdb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, nil)
chain, _ := NewBlockChain(testdb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
_, results = chain.engine.VerifyHeaders(chain, headers, seals)
chain.Stop()
} else {
chain, _ := NewBlockChain(testdb, nil, params.TestChainConfig, ethash.NewFakeFailer(uint64(len(headers)-1)), vm.Config{}, nil, nil)
chain, _ := NewBlockChain(testdb, nil, params.TestChainConfig, ethash.NewFakeFailer(uint64(len(headers)-1)), vm.Config{}, nil, nil, nil)
_, results = chain.engine.VerifyHeaders(chain, headers, seals)
chain.Stop()
}
@ -346,7 +346,7 @@ func testHeaderConcurrentAbortion(t *testing.T, threads int) {
defer runtime.GOMAXPROCS(old)
// Start the verifications and immediately abort
chain, _ := NewBlockChain(testdb, nil, params.TestChainConfig, ethash.NewFakeDelayer(time.Millisecond), vm.Config{}, nil, nil)
chain, _ := NewBlockChain(testdb, nil, params.TestChainConfig, ethash.NewFakeDelayer(time.Millisecond), vm.Config{}, nil, nil, nil)
defer chain.Stop()
abort, results := chain.engine.VerifyHeaders(chain, headers, seals)

73
core/blockchain.go
View file

@ -29,6 +29,7 @@ import (
lru "github.com/hashicorp/golang-lru"
"github.com/ethereum/go-ethereum"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/mclock"
"github.com/ethereum/go-ethereum/common/prque"
@ -38,6 +39,7 @@ import (
"github.com/ethereum/go-ethereum/core/state/snapshot"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/core/vm"
"github.com/ethereum/go-ethereum/eth/downloader/whitelist"
"github.com/ethereum/go-ethereum/ethdb"
"github.com/ethereum/go-ethereum/event"
"github.com/ethereum/go-ethereum/internal/syncx"
@ -90,7 +92,6 @@ const (
txLookupCacheLimit = 1024
maxFutureBlocks = 256
maxTimeFutureBlocks = 30
TriesInMemory = 128
// BlockChainVersion ensures that an incompatible database forces a resync from scratch.
//
@ -130,6 +131,7 @@ type CacheConfig struct {
TrieTimeLimit time.Duration // Time limit after which to flush the current in-memory trie to disk
SnapshotLimit int // Memory allowance (MB) to use for caching snapshot entries in memory
Preimages bool // Whether to store preimage of trie key to the disk
TriesInMemory uint64 // Number of recent tries to keep in memory
SnapshotWait bool // Wait for snapshot construction on startup. TODO(karalabe): This is a dirty hack for testing, nuke it
}
@ -142,6 +144,7 @@ var DefaultCacheConfig = &CacheConfig{
TrieTimeLimit: 5 * time.Minute,
SnapshotLimit: 256,
SnapshotWait: true,
TriesInMemory: 128,
}
// BlockChain represents the canonical chain given a database with a genesis
@ -221,10 +224,16 @@ type BlockChain struct {
// NewBlockChain returns a fully initialised block chain using information
// available in the database. It initialises the default Ethereum Validator
// and Processor.
func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, chainConfig *params.ChainConfig, engine consensus.Engine, vmConfig vm.Config, shouldPreserve func(header *types.Header) bool, txLookupLimit *uint64) (*BlockChain, error) {
//
//nolint:gocognit
func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, chainConfig *params.ChainConfig, engine consensus.Engine, vmConfig vm.Config, shouldPreserve func(header *types.Header) bool, txLookupLimit *uint64, checker ethereum.ChainValidator) (*BlockChain, error) {
if cacheConfig == nil {
cacheConfig = DefaultCacheConfig
}
if cacheConfig.TriesInMemory <= 0 {
cacheConfig.TriesInMemory = DefaultCacheConfig.TriesInMemory
}
bodyCache, _ := lru.New(bodyCacheLimit)
bodyRLPCache, _ := lru.New(bodyCacheLimit)
receiptsCache, _ := lru.New(receiptsCacheLimit)
@ -257,7 +266,7 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, chainConfig *par
borReceiptsCache: borReceiptsCache,
}
bc.forker = NewForkChoice(bc, shouldPreserve)
bc.forker = NewForkChoice(bc, shouldPreserve, checker)
bc.validator = NewBlockValidator(chainConfig, bc, engine)
bc.prefetcher = newStatePrefetcher(chainConfig, bc, engine)
bc.processor = NewStateProcessor(chainConfig, bc, engine)
@ -825,7 +834,7 @@ func (bc *BlockChain) Stop() {
if !bc.cacheConfig.TrieDirtyDisabled {
triedb := bc.stateCache.TrieDB()
for _, offset := range []uint64{0, 1, TriesInMemory - 1} {
for _, offset := range []uint64{0, 1, bc.cacheConfig.TriesInMemory - 1} {
if number := bc.CurrentBlock().NumberU64(); number > offset {
recent := bc.GetBlockByNumber(number - offset)
@ -908,6 +917,7 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
ancientBlocks, liveBlocks types.Blocks
ancientReceipts, liveReceipts []types.Receipts
)
// Do a sanity check that the provided chain is actually ordered and linked
for i := 0; i < len(blockChain); i++ {
if i != 0 {
@ -933,7 +943,7 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
// updateHead updates the head fast sync block if the inserted blocks are better
// and returns an indicator whether the inserted blocks are canonical.
updateHead := func(head *types.Block) bool {
updateHead := func(head *types.Block, headers []*types.Header) bool {
if !bc.chainmu.TryLock() {
return false
}
@ -948,6 +958,14 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
} else if !reorg {
return false
}
isValid, err := bc.forker.ValidateReorg(bc.CurrentFastBlock().Header(), headers)
if err != nil {
log.Warn("Reorg failed", "err", err)
return false
} else if !isValid {
return false
}
rawdb.WriteHeadFastBlockHash(bc.db, head.Hash())
bc.currentFastBlock.Store(head)
headFastBlockGauge.Update(int64(head.NumberU64()))
@ -985,10 +1003,14 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
return 0, fmt.Errorf("containing header #%d [%x..] unknown", last.Number(), last.Hash().Bytes()[:4])
}
// BOR: Retrieve all the bor receipts.
// BOR: Retrieve all the bor receipts and also maintain the array of headers
// for bor specific reorg check.
borReceipts := []types.Receipts{}
var headers []*types.Header
for _, block := range blockChain {
borReceipts = append(borReceipts, []*types.Receipt{bc.GetBorReceiptByHash(block.Hash())})
headers = append(headers, block.Header())
}
// Write all chain data to ancients.
@ -1039,7 +1061,7 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
}
// Update the current fast block because all block data is now present in DB.
previousFastBlock := bc.CurrentFastBlock().NumberU64()
if !updateHead(blockChain[len(blockChain)-1]) {
if !updateHead(blockChain[len(blockChain)-1], headers) {
// We end up here if the header chain has reorg'ed, and the blocks/receipts
// don't match the canonical chain.
if err := bc.db.TruncateHead(previousFastBlock + 1); err != nil {
@ -1075,7 +1097,11 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
writeLive := func(blockChain types.Blocks, receiptChain []types.Receipts) (int, error) {
skipPresenceCheck := false
batch := bc.db.NewBatch()
headers := make([]*types.Header, 0, len(blockChain))
for i, block := range blockChain {
// Update the headers for bor specific reorg check
headers = append(headers, block.Header())
// Short circuit insertion if shutting down or processing failed
if bc.insertStopped() {
return 0, errInsertionInterrupted
@ -1122,7 +1148,8 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
return 0, err
}
}
updateHead(blockChain[len(blockChain)-1])
updateHead(blockChain[len(blockChain)-1], headers)
return 0, nil
}
@ -1275,7 +1302,7 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types.
triedb.Reference(root, common.Hash{}) // metadata reference to keep trie alive
bc.triegc.Push(root, -int64(block.NumberU64()))
if current := block.NumberU64(); current > TriesInMemory {
if current := block.NumberU64(); current > bc.cacheConfig.TriesInMemory {
// If we exceeded our memory allowance, flush matured singleton nodes to disk
var (
nodes, imgs = triedb.Size()
@ -1285,7 +1312,7 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types.
triedb.Cap(limit - ethdb.IdealBatchSize)
}
// Find the next state trie we need to commit
chosen := current - TriesInMemory
chosen := current - bc.cacheConfig.TriesInMemory
// If we exceeded out time allowance, flush an entire trie to disk
if bc.gcproc > bc.cacheConfig.TrieTimeLimit {
@ -1297,8 +1324,8 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types.
} else {
// If we're exceeding limits but haven't reached a large enough memory gap,
// warn the user that the system is becoming unstable.
if chosen < lastWrite+TriesInMemory && bc.gcproc >= 2*bc.cacheConfig.TrieTimeLimit {
log.Info("State in memory for too long, committing", "time", bc.gcproc, "allowance", bc.cacheConfig.TrieTimeLimit, "optimum", float64(chosen-lastWrite)/TriesInMemory)
if chosen < lastWrite+bc.cacheConfig.TriesInMemory && bc.gcproc >= 2*bc.cacheConfig.TrieTimeLimit {
log.Info("State in memory for too long, committing", "time", bc.gcproc, "allowance", bc.cacheConfig.TriesInMemory, "optimum", float64(chosen-lastWrite)/float64((bc.cacheConfig.TriesInMemory)))
}
// Flush an entire trie and restart the counters
triedb.Commit(header.Root, true, nil)
@ -1491,6 +1518,18 @@ func (bc *BlockChain) insertChain(chain types.Blocks, verifySeals, setHead bool)
it := newInsertIterator(chain, results, bc.validator)
block, err := it.next()
// Check the validity of incoming chain
isValid, err1 := bc.forker.ValidateReorg(bc.CurrentBlock().Header(), headers)
if err1 != nil {
return it.index, err1
}
if !isValid {
// The chain to be imported is invalid as the blocks doesn't match with
// the whitelisted checkpoints.
return it.index, whitelist.ErrCheckpointMismatch
}
// Left-trim all the known blocks that don't need to build snapshot
if bc.skipBlock(err, it) {
// First block (and state) is known
@ -1833,6 +1872,7 @@ func (bc *BlockChain) insertSideChain(block *types.Block, it *insertIterator) (i
externTd *big.Int
lastBlock = block
current = bc.CurrentBlock()
headers []*types.Header
)
// The first sidechain block error is already verified to be ErrPrunedAncestor.
// Since we don't import them here, we expect ErrUnknownAncestor for the remaining
@ -1840,6 +1880,7 @@ func (bc *BlockChain) insertSideChain(block *types.Block, it *insertIterator) (i
// to disk.
err := consensus.ErrPrunedAncestor
for ; block != nil && errors.Is(err, consensus.ErrPrunedAncestor); block, err = it.next() {
headers = append(headers, block.Header())
// Check the canonical state root for that number
if number := block.NumberU64(); current.NumberU64() >= number {
canonical := bc.GetBlockByNumber(number)
@ -1895,7 +1936,13 @@ func (bc *BlockChain) insertSideChain(block *types.Block, it *insertIterator) (i
if err != nil {
return it.index, err
}
if !reorg {
isValid, err := bc.forker.ValidateReorg(current.Header(), headers)
if err != nil {
return it.index, err
}
if !reorg || !isValid {
localTd := bc.GetTd(current.Hash(), current.NumberU64())
log.Info("Sidechain written to disk", "start", it.first().NumberU64(), "end", it.previous().Number, "sidetd", externTd, "localtd", localTd)
return it.index, err

2
core/blockchain_bor_test.go
View file

@ -27,7 +27,7 @@ func TestChain2HeadEvent(t *testing.T) {
signer = types.LatestSigner(gspec.Config)
)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer blockchain.Stop()
chain2HeadCh := make(chan Chain2HeadEvent, 64)

159
core/blockchain_test.go
View file

@ -59,7 +59,7 @@ func newCanonical(engine consensus.Engine, n int, full bool) (ethdb.Database, *B
)
// Initialize a fresh chain with only a genesis block
blockchain, _ := NewBlockChain(db, nil, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil)
blockchain, _ := NewBlockChain(db, nil, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil, nil)
// Create and inject the requested chain
if n == 0 {
return db, blockchain, nil
@ -655,7 +655,7 @@ func testReorgBadHashes(t *testing.T, full bool) {
blockchain.Stop()
// Create a new BlockChain and check that it rolled back the state.
ncm, err := NewBlockChain(blockchain.db, nil, blockchain.chainConfig, ethash.NewFaker(), vm.Config{}, nil, nil)
ncm, err := NewBlockChain(blockchain.db, nil, blockchain.chainConfig, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create new chain manager: %v", err)
}
@ -768,7 +768,7 @@ func TestFastVsFullChains(t *testing.T) {
// Import the chain as an archive node for the comparison baseline
archiveDb := rawdb.NewMemoryDatabase()
gspec.MustCommit(archiveDb)
archive, _ := NewBlockChain(archiveDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
archive, _ := NewBlockChain(archiveDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer archive.Stop()
if n, err := archive.InsertChain(blocks); err != nil {
@ -777,7 +777,7 @@ func TestFastVsFullChains(t *testing.T) {
// Fast import the chain as a non-archive node to test
fastDb := rawdb.NewMemoryDatabase()
gspec.MustCommit(fastDb)
fast, _ := NewBlockChain(fastDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
fast, _ := NewBlockChain(fastDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer fast.Stop()
headers := make([]*types.Header, len(blocks))
@ -801,7 +801,7 @@ func TestFastVsFullChains(t *testing.T) {
t.Fatalf("failed to create temp freezer db: %v", err)
}
gspec.MustCommit(ancientDb)
ancient, _ := NewBlockChain(ancientDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
ancient, _ := NewBlockChain(ancientDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer ancient.Stop()
if n, err := ancient.InsertHeaderChain(headers, 1); err != nil {
@ -923,7 +923,7 @@ func TestLightVsFastVsFullChainHeads(t *testing.T) {
archiveCaching := *DefaultCacheConfig
archiveCaching.TrieDirtyDisabled = true
archive, _ := NewBlockChain(archiveDb, &archiveCaching, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
archive, _ := NewBlockChain(archiveDb, &archiveCaching, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
if n, err := archive.InsertChain(blocks); err != nil {
t.Fatalf("failed to process block %d: %v", n, err)
}
@ -936,7 +936,8 @@ func TestLightVsFastVsFullChainHeads(t *testing.T) {
// Import the chain as a non-archive node and ensure all pointers are updated
fastDb, delfn := makeDb()
defer delfn()
fast, _ := NewBlockChain(fastDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
fast, _ := NewBlockChain(fastDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer fast.Stop()
headers := make([]*types.Header, len(blocks))
@ -956,7 +957,8 @@ func TestLightVsFastVsFullChainHeads(t *testing.T) {
// Import the chain as a ancient-first node and ensure all pointers are updated
ancientDb, delfn := makeDb()
defer delfn()
ancient, _ := NewBlockChain(ancientDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
ancient, _ := NewBlockChain(ancientDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer ancient.Stop()
if n, err := ancient.InsertHeaderChain(headers, 1); err != nil {
@ -975,7 +977,8 @@ func TestLightVsFastVsFullChainHeads(t *testing.T) {
// Import the chain as a light node and ensure all pointers are updated
lightDb, delfn := makeDb()
defer delfn()
light, _ := NewBlockChain(lightDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
light, _ := NewBlockChain(lightDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
if n, err := light.InsertHeaderChain(headers, 1); err != nil {
t.Fatalf("failed to insert header %d: %v", n, err)
}
@ -1044,7 +1047,7 @@ func TestChainTxReorgs(t *testing.T) {
}
})
// Import the chain. This runs all block validation rules.
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
if i, err := blockchain.InsertChain(chain); err != nil {
t.Fatalf("failed to insert original chain[%d]: %v", i, err)
}
@ -1114,7 +1117,7 @@ func TestLogReorgs(t *testing.T) {
signer = types.LatestSigner(gspec.Config)
)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer blockchain.Stop()
rmLogsCh := make(chan RemovedLogsEvent)
@ -1167,7 +1170,7 @@ func TestLogRebirth(t *testing.T) {
genesis = gspec.MustCommit(db)
signer = types.LatestSigner(gspec.Config)
engine = ethash.NewFaker()
blockchain, _ = NewBlockChain(db, nil, gspec.Config, engine, vm.Config{}, nil, nil)
blockchain, _ = NewBlockChain(db, nil, gspec.Config, engine, vm.Config{}, nil, nil, nil)
)
defer blockchain.Stop()
@ -1230,7 +1233,7 @@ func TestSideLogRebirth(t *testing.T) {
gspec = &Genesis{Config: params.TestChainConfig, Alloc: GenesisAlloc{addr1: {Balance: big.NewInt(10000000000000000)}}}
genesis = gspec.MustCommit(db)
signer = types.LatestSigner(gspec.Config)
blockchain, _ = NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ = NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
)
defer blockchain.Stop()
@ -1305,7 +1308,7 @@ func TestReorgSideEvent(t *testing.T) {
signer = types.LatestSigner(gspec.Config)
)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer blockchain.Stop()
chain, _ := GenerateChain(gspec.Config, genesis, ethash.NewFaker(), db, 3, func(i int, gen *BlockGen) {})
@ -1437,7 +1440,7 @@ func TestEIP155Transition(t *testing.T) {
genesis = gspec.MustCommit(db)
)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer blockchain.Stop()
blocks, _ := GenerateChain(gspec.Config, genesis, ethash.NewFaker(), db, 4, func(i int, block *BlockGen) {
@ -1545,7 +1548,8 @@ func TestEIP161AccountRemoval(t *testing.T) {
}
genesis = gspec.MustCommit(db)
)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer blockchain.Stop()
blocks, _ := GenerateChain(gspec.Config, genesis, ethash.NewFaker(), db, 3, func(i int, block *BlockGen) {
@ -1620,7 +1624,7 @@ func TestBlockchainHeaderchainReorgConsistency(t *testing.T) {
diskdb := rawdb.NewMemoryDatabase()
(&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(diskdb)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -1648,7 +1652,7 @@ func TestTrieForkGC(t *testing.T) {
db := rawdb.NewMemoryDatabase()
genesis := (&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(db)
blocks, _ := GenerateChain(params.TestChainConfig, genesis, engine, db, 2*TriesInMemory, func(i int, b *BlockGen) { b.SetCoinbase(common.Address{1}) })
blocks, _ := GenerateChain(params.TestChainConfig, genesis, engine, db, 2*int(DefaultCacheConfig.TriesInMemory), func(i int, b *BlockGen) { b.SetCoinbase(common.Address{1}) })
// Generate a bunch of fork blocks, each side forking from the canonical chain
forks := make([]*types.Block, len(blocks))
@ -1664,7 +1668,7 @@ func TestTrieForkGC(t *testing.T) {
diskdb := rawdb.NewMemoryDatabase()
(&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(diskdb)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -1677,7 +1681,7 @@ func TestTrieForkGC(t *testing.T) {
}
}
// Dereference all the recent tries and ensure no past trie is left in
for i := 0; i < TriesInMemory; i++ {
for i := 0; i < int(chain.cacheConfig.TriesInMemory); i++ {
chain.stateCache.TrieDB().Dereference(blocks[len(blocks)-1-i].Root())
chain.stateCache.TrieDB().Dereference(forks[len(blocks)-1-i].Root())
}
@ -1696,14 +1700,14 @@ func TestLargeReorgTrieGC(t *testing.T) {
genesis := (&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(db)
shared, _ := GenerateChain(params.TestChainConfig, genesis, engine, db, 64, func(i int, b *BlockGen) { b.SetCoinbase(common.Address{1}) })
original, _ := GenerateChain(params.TestChainConfig, shared[len(shared)-1], engine, db, 2*TriesInMemory, func(i int, b *BlockGen) { b.SetCoinbase(common.Address{2}) })
competitor, _ := GenerateChain(params.TestChainConfig, shared[len(shared)-1], engine, db, 2*TriesInMemory+1, func(i int, b *BlockGen) { b.SetCoinbase(common.Address{3}) })
original, _ := GenerateChain(params.TestChainConfig, shared[len(shared)-1], engine, db, 2*int(DefaultCacheConfig.TriesInMemory), func(i int, b *BlockGen) { b.SetCoinbase(common.Address{2}) })
competitor, _ := GenerateChain(params.TestChainConfig, shared[len(shared)-1], engine, db, 2*int(DefaultCacheConfig.TriesInMemory)+1, func(i int, b *BlockGen) { b.SetCoinbase(common.Address{3}) })
// Import the shared chain and the original canonical one
diskdb := rawdb.NewMemoryDatabase()
(&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(diskdb)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -1732,7 +1736,8 @@ func TestLargeReorgTrieGC(t *testing.T) {
if _, err := chain.InsertChain(competitor[len(competitor)-2:]); err != nil {
t.Fatalf("failed to finalize competitor chain: %v", err)
}
for i, block := range competitor[:len(competitor)-TriesInMemory] {
for i, block := range competitor[:len(competitor)-int(chain.cacheConfig.TriesInMemory)] {
if node, _ := chain.stateCache.TrieDB().Node(block.Root()); node != nil {
t.Fatalf("competitor %d: competing chain state missing", i)
}
@ -1764,7 +1769,7 @@ func TestBlockchainRecovery(t *testing.T) {
t.Fatalf("failed to create temp freezer db: %v", err)
}
gspec.MustCommit(ancientDb)
ancient, _ := NewBlockChain(ancientDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
ancient, _ := NewBlockChain(ancientDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
headers := make([]*types.Header, len(blocks))
for i, block := range blocks {
@ -1784,7 +1789,7 @@ func TestBlockchainRecovery(t *testing.T) {
rawdb.WriteHeadFastBlockHash(ancientDb, midBlock.Hash())
// Reopen broken blockchain again
ancient, _ = NewBlockChain(ancientDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
ancient, _ = NewBlockChain(ancientDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer ancient.Stop()
if num := ancient.CurrentBlock().NumberU64(); num != 0 {
t.Errorf("head block mismatch: have #%v, want #%v", num, 0)
@ -1836,7 +1841,7 @@ func TestInsertReceiptChainRollback(t *testing.T) {
}
gspec := Genesis{Config: params.AllEthashProtocolChanges}
gspec.MustCommit(ancientDb)
ancientChain, _ := NewBlockChain(ancientDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
ancientChain, _ := NewBlockChain(ancientDb, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer ancientChain.Stop()
// Import the canonical header chain.
@ -1878,8 +1883,8 @@ func TestInsertReceiptChainRollback(t *testing.T) {
// overtake the 'canon' chain until after it's passed canon by about 200 blocks.
//
// Details at:
// - https://github.com/ethereum/go-ethereum/issues/18977
// - https://github.com/ethereum/go-ethereum/pull/18988
// - https://github.com/ethereum/go-ethereum/issues/18977
// - https://github.com/ethereum/go-ethereum/pull/18988
func TestLowDiffLongChain(t *testing.T) {
// Generate a canonical chain to act as the main dataset
engine := ethash.NewFaker()
@ -1888,7 +1893,7 @@ func TestLowDiffLongChain(t *testing.T) {
// We must use a pretty long chain to ensure that the fork doesn't overtake us
// until after at least 128 blocks post tip
blocks, _ := GenerateChain(params.TestChainConfig, genesis, engine, db, 6*TriesInMemory, func(i int, b *BlockGen) {
blocks, _ := GenerateChain(params.TestChainConfig, genesis, engine, db, 6*int(DefaultCacheConfig.TriesInMemory), func(i int, b *BlockGen) {
b.SetCoinbase(common.Address{1})
b.OffsetTime(-9)
})
@ -1897,7 +1902,7 @@ func TestLowDiffLongChain(t *testing.T) {
diskdb := rawdb.NewMemoryDatabase()
(&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(diskdb)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -1906,7 +1911,7 @@ func TestLowDiffLongChain(t *testing.T) {
}
// Generate fork chain, starting from an early block
parent := blocks[10]
fork, _ := GenerateChain(params.TestChainConfig, parent, engine, db, 8*TriesInMemory, func(i int, b *BlockGen) {
fork, _ := GenerateChain(params.TestChainConfig, parent, engine, db, 8*int(DefaultCacheConfig.TriesInMemory), func(i int, b *BlockGen) {
b.SetCoinbase(common.Address{2})
})
@ -1963,7 +1968,7 @@ func testSideImport(t *testing.T, numCanonBlocksInSidechain, blocksBetweenCommon
// Generate and import the canonical chain
diskdb := rawdb.NewMemoryDatabase()
gspec.MustCommit(diskdb)
chain, err := NewBlockChain(diskdb, nil, &chainConfig, runEngine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(diskdb, nil, &chainConfig, runEngine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -1975,7 +1980,8 @@ func testSideImport(t *testing.T, numCanonBlocksInSidechain, blocksBetweenCommon
// Set the terminal total difficulty in the config
gspec.Config.TerminalTotalDifficulty = big.NewInt(0)
}
blocks, _ := GenerateChain(&chainConfig, genesis, genEngine, db, 2*TriesInMemory, func(i int, gen *BlockGen) {
blocks, _ := GenerateChain(&chainConfig, genesis, genEngine, db, 2*int(DefaultCacheConfig.TriesInMemory), func(i int, gen *BlockGen) {
tx, err := types.SignTx(types.NewTransaction(nonce, common.HexToAddress("deadbeef"), big.NewInt(100), 21000, big.NewInt(int64(i+1)*params.GWei), nil), signer, key)
if err != nil {
t.Fatalf("failed to create tx: %v", err)
@ -1987,9 +1993,9 @@ func testSideImport(t *testing.T, numCanonBlocksInSidechain, blocksBetweenCommon
t.Fatalf("block %d: failed to insert into chain: %v", n, err)
}
lastPrunedIndex := len(blocks) - TriesInMemory - 1
lastPrunedIndex := len(blocks) - int(chain.cacheConfig.TriesInMemory) - 1
lastPrunedBlock := blocks[lastPrunedIndex]
firstNonPrunedBlock := blocks[len(blocks)-TriesInMemory]
firstNonPrunedBlock := blocks[len(blocks)-int(chain.cacheConfig.TriesInMemory)]
// Verify pruning of lastPrunedBlock
if chain.HasBlockAndState(lastPrunedBlock.Hash(), lastPrunedBlock.NumberU64()) {
@ -2015,7 +2021,7 @@ func testSideImport(t *testing.T, numCanonBlocksInSidechain, blocksBetweenCommon
// Generate fork chain, make it longer than canon
parentIndex := lastPrunedIndex + blocksBetweenCommonAncestorAndPruneblock
parent := blocks[parentIndex]
fork, _ := GenerateChain(&chainConfig, parent, genEngine, db, 2*TriesInMemory, func(i int, b *BlockGen) {
fork, _ := GenerateChain(&chainConfig, parent, genEngine, db, 2*int(DefaultCacheConfig.TriesInMemory), func(i int, b *BlockGen) {
b.SetCoinbase(common.Address{2})
})
// Prepend the parent(s)
@ -2042,7 +2048,8 @@ func testSideImport(t *testing.T, numCanonBlocksInSidechain, blocksBetweenCommon
// That is: the sidechain for import contains some blocks already present in canon chain.
// So the blocks are
// [ Cn, Cn+1, Cc, Sn+3 ... Sm]
// ^ ^ ^ pruned
//
// ^ ^ ^ pruned
func TestPrunedImportSide(t *testing.T) {
//glogger := log.NewGlogHandler(log.StreamHandler(os.Stdout, log.TerminalFormat(false)))
//glogger.Verbosity(3)
@ -2102,7 +2109,7 @@ func testInsertKnownChainData(t *testing.T, typ string) {
(&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(chaindb)
defer os.RemoveAll(dir)
chain, err := NewBlockChain(chaindb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(chaindb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -2266,7 +2273,7 @@ func testInsertKnownChainDataWithMerging(t *testing.T, typ string, mergeHeight i
(&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(chaindb)
defer os.RemoveAll(dir)
chain, err := NewBlockChain(chaindb, nil, &chainConfig, runEngine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(chaindb, nil, &chainConfig, runEngine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -2384,7 +2391,7 @@ func getLongAndShortChains() (bc *BlockChain, longChain []*types.Block, heavyCha
diskdb := rawdb.NewMemoryDatabase()
(&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(diskdb)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil, nil)
if err != nil {
return nil, nil, nil, fmt.Errorf("failed to create tester chain: %v", err)
}
@ -2577,7 +2584,7 @@ func TestTransactionIndices(t *testing.T) {
// Import all blocks into ancient db
l := uint64(0)
chain, err := NewBlockChain(ancientDb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, &l)
chain, err := NewBlockChain(ancientDb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, &l, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -2602,7 +2609,7 @@ func TestTransactionIndices(t *testing.T) {
t.Fatalf("failed to create temp freezer db: %v", err)
}
gspec.MustCommit(ancientDb)
chain, err = NewBlockChain(ancientDb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, &l)
chain, err = NewBlockChain(ancientDb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, &l, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -2626,7 +2633,7 @@ func TestTransactionIndices(t *testing.T) {
limit = []uint64{0, 64 /* drop stale */, 32 /* shorten history */, 64 /* extend history */, 0 /* restore all */}
tails := []uint64{0, 67 /* 130 - 64 + 1 */, 100 /* 131 - 32 + 1 */, 69 /* 132 - 64 + 1 */, 0}
for i, l := range limit {
chain, err = NewBlockChain(ancientDb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, &l)
chain, err = NewBlockChain(ancientDb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, &l, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -2704,7 +2711,7 @@ func TestSkipStaleTxIndicesInSnapSync(t *testing.T) {
// Import all blocks into ancient db, only HEAD-32 indices are kept.
l := uint64(32)
chain, err := NewBlockChain(ancientDb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, &l)
chain, err := NewBlockChain(ancientDb, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, &l, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -2768,7 +2775,7 @@ func benchmarkLargeNumberOfValueToNonexisting(b *testing.B, numTxs, numBlocks in
diskdb := rawdb.NewMemoryDatabase()
gspec.MustCommit(diskdb)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil, nil)
if err != nil {
b.Fatalf("failed to create tester chain: %v", err)
}
@ -2837,9 +2844,9 @@ func BenchmarkBlockChain_1x1000Executions(b *testing.B) {
// This internally leads to a sidechain import, since the blocks trigger an
// ErrPrunedAncestor error.
// This may e.g. happen if
// 1. Downloader rollbacks a batch of inserted blocks and exits
// 2. Downloader starts to sync again
// 3. The blocks fetched are all known and canonical blocks
// 1. Downloader rollbacks a batch of inserted blocks and exits
// 2. Downloader starts to sync again
// 3. The blocks fetched are all known and canonical blocks
func TestSideImportPrunedBlocks(t *testing.T) {
// Generate a canonical chain to act as the main dataset
engine := ethash.NewFaker()
@ -2847,11 +2854,11 @@ func TestSideImportPrunedBlocks(t *testing.T) {
genesis := (&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(db)
// Generate and import the canonical chain
blocks, _ := GenerateChain(params.TestChainConfig, genesis, engine, db, 2*TriesInMemory, nil)
blocks, _ := GenerateChain(params.TestChainConfig, genesis, engine, db, 2*int(DefaultCacheConfig.TriesInMemory), nil)
diskdb := rawdb.NewMemoryDatabase()
(&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(diskdb)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -2859,14 +2866,15 @@ func TestSideImportPrunedBlocks(t *testing.T) {
t.Fatalf("block %d: failed to insert into chain: %v", n, err)
}
lastPrunedIndex := len(blocks) - TriesInMemory - 1
lastPrunedIndex := len(blocks) - int(chain.cacheConfig.TriesInMemory) - 1
lastPrunedBlock := blocks[lastPrunedIndex]
// Verify pruning of lastPrunedBlock
if chain.HasBlockAndState(lastPrunedBlock.Hash(), lastPrunedBlock.NumberU64()) {
t.Errorf("Block %d not pruned", lastPrunedBlock.NumberU64())
}
firstNonPrunedBlock := blocks[len(blocks)-TriesInMemory]
firstNonPrunedBlock := blocks[len(blocks)-int(chain.cacheConfig.TriesInMemory)]
// Verify firstNonPrunedBlock is not pruned
if !chain.HasBlockAndState(firstNonPrunedBlock.Hash(), firstNonPrunedBlock.NumberU64()) {
t.Errorf("Block %d pruned", firstNonPrunedBlock.NumberU64())
@ -2945,7 +2953,7 @@ func TestDeleteCreateRevert(t *testing.T) {
diskdb := rawdb.NewMemoryDatabase()
gspec.MustCommit(diskdb)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -3059,7 +3067,7 @@ func TestDeleteRecreateSlots(t *testing.T) {
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{
Debug: true,
Tracer: logger.NewJSONLogger(nil, os.Stdout),
}, nil, nil)
}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -3139,7 +3147,7 @@ func TestDeleteRecreateAccount(t *testing.T) {
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{
Debug: true,
Tracer: logger.NewJSONLogger(nil, os.Stdout),
}, nil, nil)
}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -3312,7 +3320,7 @@ func TestDeleteRecreateSlotsAcrossManyBlocks(t *testing.T) {
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{
//Debug: true,
//Tracer: vm.NewJSONLogger(nil, os.Stdout),
}, nil, nil)
}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -3352,20 +3360,19 @@ func TestDeleteRecreateSlotsAcrossManyBlocks(t *testing.T) {
// TestInitThenFailCreateContract tests a pretty notorious case that happened
// on mainnet over blocks 7338108, 7338110 and 7338115.
// - Block 7338108: address e771789f5cccac282f23bb7add5690e1f6ca467c is initiated
// with 0.001 ether (thus created but no code)
// - Block 7338110: a CREATE2 is attempted. The CREATE2 would deploy code on
// the same address e771789f5cccac282f23bb7add5690e1f6ca467c. However, the
// deployment fails due to OOG during initcode execution
// - Block 7338115: another tx checks the balance of
// e771789f5cccac282f23bb7add5690e1f6ca467c, and the snapshotter returned it as
// zero.
// - Block 7338108: address e771789f5cccac282f23bb7add5690e1f6ca467c is initiated
// with 0.001 ether (thus created but no code)
// - Block 7338110: a CREATE2 is attempted. The CREATE2 would deploy code on
// the same address e771789f5cccac282f23bb7add5690e1f6ca467c. However, the
// deployment fails due to OOG during initcode execution
// - Block 7338115: another tx checks the balance of
// e771789f5cccac282f23bb7add5690e1f6ca467c, and the snapshotter returned it as
// zero.
//
// The problem being that the snapshotter maintains a destructset, and adds items
// to the destructset in case something is created "onto" an existing item.
// We need to either roll back the snapDestructs, or not place it into snapDestructs
// in the first place.
//
func TestInitThenFailCreateContract(t *testing.T) {
var (
// Generate a canonical chain to act as the main dataset
@ -3446,7 +3453,7 @@ func TestInitThenFailCreateContract(t *testing.T) {
chain, err := NewBlockChain(diskdb, nil, params.TestChainConfig, engine, vm.Config{
//Debug: true,
//Tracer: vm.NewJSONLogger(nil, os.Stdout),
}, nil, nil)
}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -3533,7 +3540,7 @@ func TestEIP2718Transition(t *testing.T) {
diskdb := rawdb.NewMemoryDatabase()
gspec.MustCommit(diskdb)
chain, err := NewBlockChain(diskdb, nil, gspec.Config, engine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(diskdb, nil, gspec.Config, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}
@ -3554,13 +3561,13 @@ func TestEIP2718Transition(t *testing.T) {
// TestEIP1559Transition tests the following:
//
// 1. A transaction whose gasFeeCap is greater than the baseFee is valid.
// 2. Gas accounting for access lists on EIP-1559 transactions is correct.
// 3. Only the transaction's tip will be received by the coinbase.
// 4. The transaction sender pays for both the tip and baseFee.
// 5. The coinbase receives only the partially realized tip when
// gasFeeCap - gasTipCap < baseFee.
// 6. Legacy transaction behave as expected (e.g. gasPrice = gasFeeCap = gasTipCap).
// 1. A transaction whose gasFeeCap is greater than the baseFee is valid.
// 2. Gas accounting for access lists on EIP-1559 transactions is correct.
// 3. Only the transaction's tip will be received by the coinbase.
// 4. The transaction sender pays for both the tip and baseFee.
// 5. The coinbase receives only the partially realized tip when
// gasFeeCap - gasTipCap < baseFee.
// 6. Legacy transaction behave as expected (e.g. gasPrice = gasFeeCap = gasTipCap).
func TestEIP1559Transition(t *testing.T) {
var (
aa = common.HexToAddress("0x000000000000000000000000000000000000aaaa")
@ -3628,7 +3635,7 @@ func TestEIP1559Transition(t *testing.T) {
diskdb := rawdb.NewMemoryDatabase()
gspec.MustCommit(diskdb)
chain, err := NewBlockChain(diskdb, nil, gspec.Config, engine, vm.Config{}, nil, nil)
chain, err := NewBlockChain(diskdb, nil, gspec.Config, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("failed to create tester chain: %v", err)
}

3
core/chain_makers.go
View file

@ -17,6 +17,7 @@
package core
import (
"context"
"fmt"
"math/big"
@ -258,7 +259,7 @@ func GenerateChain(config *params.ChainConfig, parent *types.Block, engine conse
}
if b.engine != nil {
// Finalize and seal the block
block, _ := b.engine.FinalizeAndAssemble(chainreader, b.header, statedb, b.txs, b.uncles, b.receipts)
block, _ := b.engine.FinalizeAndAssemble(context.Background(), chainreader, b.header, statedb, b.txs, b.uncles, b.receipts)
// Write state changes to db
root, err := statedb.Commit(config.IsEIP158(b.header.Number))

2
core/chain_makers_test.go
View file

@ -79,7 +79,7 @@ func ExampleGenerateChain() {
})
// Import the chain. This runs all block validation rules.
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ := NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer blockchain.Stop()
if i, err := blockchain.InsertChain(chain); err != nil {

12
core/dao_test.go
View file

@ -45,7 +45,7 @@ func TestDAOForkRangeExtradata(t *testing.T) {
proConf.DAOForkBlock = forkBlock
proConf.DAOForkSupport = true
proBc, _ := NewBlockChain(proDb, nil, &proConf, ethash.NewFaker(), vm.Config{}, nil, nil)
proBc, _ := NewBlockChain(proDb, nil, &proConf, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer proBc.Stop()
conDb := rawdb.NewMemoryDatabase()
@ -55,7 +55,7 @@ func TestDAOForkRangeExtradata(t *testing.T) {
conConf.DAOForkBlock = forkBlock
conConf.DAOForkSupport = false
conBc, _ := NewBlockChain(conDb, nil, &conConf, ethash.NewFaker(), vm.Config{}, nil, nil)
conBc, _ := NewBlockChain(conDb, nil, &conConf, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer conBc.Stop()
if _, err := proBc.InsertChain(prefix); err != nil {
@ -69,7 +69,7 @@ func TestDAOForkRangeExtradata(t *testing.T) {
// Create a pro-fork block, and try to feed into the no-fork chain
db = rawdb.NewMemoryDatabase()
gspec.MustCommit(db)
bc, _ := NewBlockChain(db, nil, &conConf, ethash.NewFaker(), vm.Config{}, nil, nil)
bc, _ := NewBlockChain(db, nil, &conConf, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer bc.Stop()
blocks := conBc.GetBlocksFromHash(conBc.CurrentBlock().Hash(), int(conBc.CurrentBlock().NumberU64()))
@ -94,7 +94,7 @@ func TestDAOForkRangeExtradata(t *testing.T) {
// Create a no-fork block, and try to feed into the pro-fork chain
db = rawdb.NewMemoryDatabase()
gspec.MustCommit(db)
bc, _ = NewBlockChain(db, nil, &proConf, ethash.NewFaker(), vm.Config{}, nil, nil)
bc, _ = NewBlockChain(db, nil, &proConf, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer bc.Stop()
blocks = proBc.GetBlocksFromHash(proBc.CurrentBlock().Hash(), int(proBc.CurrentBlock().NumberU64()))
@ -120,7 +120,7 @@ func TestDAOForkRangeExtradata(t *testing.T) {
// Verify that contra-forkers accept pro-fork extra-datas after forking finishes
db = rawdb.NewMemoryDatabase()
gspec.MustCommit(db)
bc, _ := NewBlockChain(db, nil, &conConf, ethash.NewFaker(), vm.Config{}, nil, nil)
bc, _ := NewBlockChain(db, nil, &conConf, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer bc.Stop()
blocks := conBc.GetBlocksFromHash(conBc.CurrentBlock().Hash(), int(conBc.CurrentBlock().NumberU64()))
@ -140,7 +140,7 @@ func TestDAOForkRangeExtradata(t *testing.T) {
// Verify that pro-forkers accept contra-fork extra-datas after forking finishes
db = rawdb.NewMemoryDatabase()
gspec.MustCommit(db)
bc, _ = NewBlockChain(db, nil, &proConf, ethash.NewFaker(), vm.Config{}, nil, nil)
bc, _ = NewBlockChain(db, nil, &proConf, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
defer bc.Stop()
blocks = proBc.GetBlocksFromHash(proBc.CurrentBlock().Hash(), int(proBc.CurrentBlock().NumberU64()))

41
core/forkchoice.go
View file

@ -17,15 +17,14 @@
package core
import (
crand "crypto/rand"
"errors"
"math/big"
mrand "math/rand"
"github.com/maticnetwork/crand"
"github.com/ethereum/go-ethereum"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/math"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/params"
)
@ -47,25 +46,29 @@ type ChainReader interface {
// for all other proof-of-work networks.
type ForkChoice struct {
chain ChainReader
rand *mrand.Rand
rand Floater
// preserve is a helper function used in td fork choice.
// Miners will prefer to choose the local mined block if the
// local td is equal to the extern one. It can be nil for light
// client
preserve func(header *types.Header) bool
validator ethereum.ChainValidator
}
func NewForkChoice(chainReader ChainReader, preserve func(header *types.Header) bool) *ForkChoice {
type Floater interface {
Float64() float64
}
func NewForkChoice(chainReader ChainReader, preserve func(header *types.Header) bool, validator ethereum.ChainValidator) *ForkChoice {
// Seed a fast but crypto originating random generator
seed, err := crand.Int(crand.Reader, big.NewInt(math.MaxInt64))
if err != nil {
log.Crit("Failed to initialize random seed", "err", err)
}
r := crand.NewRand()
return &ForkChoice{
chain: chainReader,
rand: mrand.New(mrand.NewSource(seed.Int64())),
preserve: preserve,
chain: chainReader,
rand: r,
preserve: preserve,
validator: validator,
}
}
@ -106,3 +109,15 @@ func (f *ForkChoice) ReorgNeeded(current *types.Header, header *types.Header) (b
}
return reorg, nil
}
// ValidateReorg calls the chain validator service to check if the reorg is valid or not
func (f *ForkChoice) ValidateReorg(current *types.Header, chain []*types.Header) (bool, error) {
// Call the bor chain validator service
if f.validator != nil {
if isValid := f.validator.IsValidChain(current, chain); !isValid {
return false, nil
}
}
return true, nil
}

240
core/forkchoice_test.go Normal file
View file

@ -0,0 +1,240 @@
package core
import (
"math/big"
"testing"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/consensus/ethash"
"github.com/ethereum/go-ethereum/core/rawdb"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/params"
)
// chainValidatorFake is a mock for the chain validator service
type chainValidatorFake struct {
validate func(currentHeader *types.Header, chain []*types.Header) bool
}
// chainReaderFake is a mock for the chain reader service
type chainReaderFake struct {
getTd func(hash common.Hash, number uint64) *big.Int
}
func newChainValidatorFake(validate func(currentHeader *types.Header, chain []*types.Header) bool) *chainValidatorFake {
return &chainValidatorFake{validate: validate}
}
func newChainReaderFake(getTd func(hash common.Hash, number uint64) *big.Int) *chainReaderFake {
return &chainReaderFake{getTd: getTd}
}
func TestPastChainInsert(t *testing.T) {
t.Parallel()
var (
db = rawdb.NewMemoryDatabase()
genesis = (&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(db)
)
hc, err := NewHeaderChain(db, params.AllEthashProtocolChanges, ethash.NewFaker(), func() bool { return false })
if err != nil {
t.Fatal(err)
}
// Create mocks for forker
getTd := func(hash common.Hash, number uint64) *big.Int {
return big.NewInt(int64(number))
}
validate := func(currentHeader *types.Header, chain []*types.Header) bool {
// Put all explicit conditions here
// If canonical chain is empty and we're importing a chain of 64 blocks
if currentHeader.Number.Uint64() == uint64(0) && len(chain) == 64 {
return true
}
// If canonical chain is of len 64 and we're importing a past chain from 54-64, then accept it
if currentHeader.Number.Uint64() == uint64(64) && chain[0].Number.Uint64() == 55 && len(chain) == 10 {
return true
}
return false
}
mockChainReader := newChainReaderFake(getTd)
mockChainValidator := newChainValidatorFake(validate)
mockForker := NewForkChoice(mockChainReader, nil, mockChainValidator)
// chain A: G->A1->A2...A64
chainA := makeHeaderChain(genesis.Header(), 64, ethash.NewFaker(), db, 10)
// Inserting 64 headers on an empty chain
// expecting 1 write status with no error
testInsert(t, hc, chainA, CanonStatTy, nil, mockForker)
// The current chain is: G->A1->A2...A64
// chain B: G->A1->A2...A44->B45->B46...B64
chainB := makeHeaderChain(chainA[43], 20, ethash.NewFaker(), db, 10)
// The current chain is: G->A1->A2...A64
// chain C: G->A1->A2...A54->C55->C56...C64
chainC := makeHeaderChain(chainA[53], 10, ethash.NewFaker(), db, 10)
// Update the function to consider chainC with higher difficulty
getTd = func(hash common.Hash, number uint64) *big.Int {
td := big.NewInt(int64(number))
if hash == chainB[len(chainB)-1].Hash() || hash == chainC[len(chainC)-1].Hash() {
td = big.NewInt(65)
}
return td
}
mockChainReader = newChainReaderFake(getTd)
mockForker = NewForkChoice(mockChainReader, nil, mockChainValidator)
// Inserting 20 blocks from chainC on canonical chain
// expecting 2 write status with no error
testInsert(t, hc, chainB, SideStatTy, nil, mockForker)
// Inserting 10 blocks from chainB on canonical chain
// expecting 1 write status with no error
testInsert(t, hc, chainC, CanonStatTy, nil, mockForker)
}
func TestFutureChainInsert(t *testing.T) {
t.Parallel()
var (
db = rawdb.NewMemoryDatabase()
genesis = (&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(db)
)
hc, err := NewHeaderChain(db, params.AllEthashProtocolChanges, ethash.NewFaker(), func() bool { return false })
if err != nil {
t.Fatal(err)
}
// Create mocks for forker
getTd := func(hash common.Hash, number uint64) *big.Int {
return big.NewInt(int64(number))
}
validate := func(currentHeader *types.Header, chain []*types.Header) bool {
// Put all explicit conditions here
// If canonical chain is empty and we're importing a chain of 64 blocks
if currentHeader.Number.Uint64() == uint64(0) && len(chain) == 64 {
return true
}
// If length of future chains > some value, they should not be accepted
if currentHeader.Number.Uint64() == uint64(64) && len(chain) <= 10 {
return true
}
return false
}
mockChainReader := newChainReaderFake(getTd)
mockChainValidator := newChainValidatorFake(validate)
mockForker := NewForkChoice(mockChainReader, nil, mockChainValidator)
// chain A: G->A1->A2...A64
chainA := makeHeaderChain(genesis.Header(), 64, ethash.NewFaker(), db, 10)
// Inserting 64 headers on an empty chain
// expecting 1 write status with no error
testInsert(t, hc, chainA, CanonStatTy, nil, mockForker)
// The current chain is: G->A1->A2...A64
// chain B: G->A1->A2...A64->B65->B66...B84
chainB := makeHeaderChain(chainA[63], 20, ethash.NewFaker(), db, 10)
// Inserting 20 headers on the canonical chain
// expecting 0 write status with no error
testInsert(t, hc, chainB, SideStatTy, nil, mockForker)
// The current chain is: G->A1->A2...A64
// chain C: G->A1->A2...A64->C65->C66...C74
chainC := makeHeaderChain(chainA[63], 10, ethash.NewFaker(), db, 10)
// Inserting 10 headers on the canonical chain
// expecting 0 write status with no error
testInsert(t, hc, chainC, CanonStatTy, nil, mockForker)
}
func TestOverlappingChainInsert(t *testing.T) {
t.Parallel()
var (
db = rawdb.NewMemoryDatabase()
genesis = (&Genesis{BaseFee: big.NewInt(params.InitialBaseFee)}).MustCommit(db)
)
hc, err := NewHeaderChain(db, params.AllEthashProtocolChanges, ethash.NewFaker(), func() bool { return false })
if err != nil {
t.Fatal(err)
}
// Create mocks for forker
getTd := func(hash common.Hash, number uint64) *big.Int {
return big.NewInt(int64(number))
}
validate := func(currentHeader *types.Header, chain []*types.Header) bool {
// Put all explicit conditions here
// If canonical chain is empty and we're importing a chain of 64 blocks
if currentHeader.Number.Uint64() == uint64(0) && len(chain) == 64 {
return true
}
// If length of chain is > some fixed value then don't accept it
if currentHeader.Number.Uint64() == uint64(64) && len(chain) <= 20 {
return true
}
return false
}
mockChainReader := newChainReaderFake(getTd)
mockChainValidator := newChainValidatorFake(validate)
mockForker := NewForkChoice(mockChainReader, nil, mockChainValidator)
// chain A: G->A1->A2...A64
chainA := makeHeaderChain(genesis.Header(), 64, ethash.NewFaker(), db, 10)
// Inserting 64 headers on an empty chain
// expecting 1 write status with no error
testInsert(t, hc, chainA, CanonStatTy, nil, mockForker)
// The current chain is: G->A1->A2...A64
// chain B: G->A1->A2...A54->B55->B56...B84
chainB := makeHeaderChain(chainA[53], 30, ethash.NewFaker(), db, 10)
// Inserting 20 blocks on canonical chain
// expecting 2 write status with no error
testInsert(t, hc, chainB, SideStatTy, nil, mockForker)
// The current chain is: G->A1->A2...A64
// chain C: G->A1->A2...A54->C55->C56...C74
chainC := makeHeaderChain(chainA[53], 20, ethash.NewFaker(), db, 10)
// Inserting 10 blocks on canonical chain
// expecting 1 write status with no error
testInsert(t, hc, chainC, CanonStatTy, nil, mockForker)
}
// Mock chain reader functions
func (c *chainReaderFake) Config() *params.ChainConfig {
return &params.ChainConfig{TerminalTotalDifficulty: nil}
}
func (c *chainReaderFake) GetTd(hash common.Hash, number uint64) *big.Int {
return c.getTd(hash, number)
}
// Mock chain validator functions
func (w *chainValidatorFake) IsValidPeer(remoteHeader *types.Header, fetchHeadersByNumber func(number uint64, amount int, skip int, reverse bool) ([]*types.Header, []common.Hash, error)) (bool, error) {
return true, nil
}
func (w *chainValidatorFake) IsValidChain(current *types.Header, headers []*types.Header) bool {
return w.validate(current, headers)
}
func (w *chainValidatorFake) ProcessCheckpoint(endBlockNum uint64, endBlockHash common.Hash) {}
func (w *chainValidatorFake) GetCheckpointWhitelist() map[uint64]common.Hash {
return nil
}
func (w *chainValidatorFake) PurgeCheckpointWhitelist() {}
func (w *chainValidatorFake) GetCheckpoints(current, sidechainHeader *types.Header, sidechainCheckpoints []*types.Header) (map[uint64]*types.Header, error) {
return map[uint64]*types.Header{}, nil
}

2
core/genesis_test.go
View file

@ -117,7 +117,7 @@ func TestSetupGenesis(t *testing.T) {
// Advance to block #4, past the homestead transition block of customg.
genesis := oldcustomg.MustCommit(db)
bc, _ := NewBlockChain(db, nil, oldcustomg.Config, ethash.NewFullFaker(), vm.Config{}, nil, nil)
bc, _ := NewBlockChain(db, nil, oldcustomg.Config, ethash.NewFullFaker(), vm.Config{}, nil, nil, nil)
defer bc.Stop()
blocks, _ := GenerateChain(oldcustomg.Config, genesis, ethash.NewFaker(), db, 4, nil)

14
core/headerchain.go
View file

@ -283,8 +283,10 @@ func (hc *HeaderChain) writeHeadersAndSetHead(headers []*types.Header, forker *F
lastHeader: lastHeader,
}
)
// Ask the fork choicer if the reorg is necessary
if reorg, err := forker.ReorgNeeded(hc.CurrentHeader(), lastHeader); err != nil {
reorg, err := forker.ReorgNeeded(hc.CurrentHeader(), lastHeader)
if err != nil {
return nil, err
} else if !reorg {
if inserted != 0 {
@ -292,6 +294,16 @@ func (hc *HeaderChain) writeHeadersAndSetHead(headers []*types.Header, forker *F
}
return result, nil
}
isValid, err := forker.ValidateReorg(hc.CurrentHeader(), headers)
if err != nil {
return nil, err
} else if !isValid {
if inserted != 0 {
result.status = SideStatTy
}
return result, nil
}
// Special case, all the inserted headers are already on the canonical
// header chain, skip the reorg operation.
if hc.GetCanonicalHash(lastHeader.Number.Uint64()) == lastHash && lastHeader.Number.Uint64() <= hc.CurrentHeader().Number.Uint64() {

2
core/headerchain_test.go
View file

@ -84,7 +84,7 @@ func TestHeaderInsertion(t *testing.T) {
chainB := makeHeaderChain(chainA[0], 128, ethash.NewFaker(), db, 10)
log.Root().SetHandler(log.StdoutHandler)
forker := NewForkChoice(hc, nil)
forker := NewForkChoice(hc, nil, nil)
// Inserting 64 headers on an empty chain, expecting
// 1 callbacks, 1 canon-status, 0 sidestatus,
testInsert(t, hc, chainA[:64], CanonStatTy, nil, forker)

6
core/rawdb/bor_receipt.go
View file

@ -18,7 +18,11 @@ var (
getDerivedBorTxHash = types.GetDerivedBorTxHash
// borTxLookupPrefix + hash -> transaction/receipt lookup metadata
borTxLookupPrefix = []byte("matic-bor-tx-lookup-")
borTxLookupPrefix = []byte(borTxLookupPrefixStr)
)
const (
borTxLookupPrefixStr = "matic-bor-tx-lookup-"
// freezerBorReceiptTable indicates the name of the freezer bor receipts table.
freezerBorReceiptTable = "matic-bor-receipts"

4
core/state/statedb.go
View file

@ -601,8 +601,8 @@ func (s *StateDB) createObject(addr common.Address) (newobj, prev *stateObject)
// CreateAccount is called during the EVM CREATE operation. The situation might arise that
// a contract does the following:
//
// 1. sends funds to sha(account ++ (nonce + 1))
// 2. tx_create(sha(account ++ nonce)) (note that this gets the address of 1)
// 1. sends funds to sha(account ++ (nonce + 1))
// 2. tx_create(sha(account ++ nonce)) (note that this gets the address of 1)
//
// Carrying over the balance ensures that Ether doesn't disappear.
func (s *StateDB) CreateAccount(addr common.Address) {

6
core/state_processor_test.go
View file

@ -94,7 +94,7 @@ func TestStateProcessorErrors(t *testing.T) {
},
}
genesis = gspec.MustCommit(db)
blockchain, _ = NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ = NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
)
defer blockchain.Stop()
bigNumber := new(big.Int).SetBytes(common.FromHex("0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"))
@ -235,7 +235,7 @@ func TestStateProcessorErrors(t *testing.T) {
},
}
genesis = gspec.MustCommit(db)
blockchain, _ = NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ = NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
)
defer blockchain.Stop()
for i, tt := range []struct {
@ -275,7 +275,7 @@ func TestStateProcessorErrors(t *testing.T) {
},
}
genesis = gspec.MustCommit(db)
blockchain, _ = NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil)
blockchain, _ = NewBlockChain(db, nil, gspec.Config, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
)
defer blockchain.Stop()
for i, tt := range []struct {

20
core/state_transition.go
View file

@ -42,8 +42,10 @@ The state transitioning model does all the necessary work to work out a valid ne
3) Create a new state object if the recipient is \0*32
4) Value transfer
== If contract creation ==
4a) Attempt to run transaction data
4b) If valid, use result as code for the new state object
4a) Attempt to run transaction data
4b) If valid, use result as code for the new state object
== end ==
5) Run Script section
6) Derive new state root
@ -262,13 +264,13 @@ func (st *StateTransition) preCheck() error {
// TransitionDb will transition the state by applying the current message and
// returning the evm execution result with following fields.
//
// - used gas:
// total gas used (including gas being refunded)
// - returndata:
// the returned data from evm
// - concrete execution error:
// various **EVM** error which aborts the execution,
// e.g. ErrOutOfGas, ErrExecutionReverted
// - used gas:
// total gas used (including gas being refunded)
// - returndata:
// the returned data from evm
// - concrete execution error:
// various **EVM** error which aborts the execution,
// e.g. ErrOutOfGas, ErrExecutionReverted
//
// However if any consensus issue encountered, return the error directly with
// nil evm execution result.

81
core/tests/blockchain_repair_test.go
View file

@ -29,7 +29,6 @@ import (
"github.com/golang/mock/gomock"
"github.com/ethereum/go-ethereum/accounts"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/consensus/bor"
"github.com/ethereum/go-ethereum/consensus/bor/api"
@ -1788,6 +1787,8 @@ func testRepair(t *testing.T, tt *rewindTest, snapshots bool) {
}
defer db.Close() // Might double close, should be fine
chainConfig := params.BorUnittestChainConfig
ctrl := gomock.NewController(t)
defer ctrl.Finish()
@ -1795,10 +1796,10 @@ func testRepair(t *testing.T, tt *rewindTest, snapshots bool) {
ethAPIMock.EXPECT().Call(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any()).AnyTimes()
spanner := bor.NewMockSpanner(ctrl)
spanner.EXPECT().GetCurrentValidators(gomock.Any(), gomock.Any()).Return([]*valset.Validator{
spanner.EXPECT().GetCurrentValidators(gomock.Any(), gomock.Any(), gomock.Any()).Return([]*valset.Validator{
{
ID: 0,
Address: common.Address{0x1},
Address: miner.TestBankAddress,
VotingPower: 100,
ProposerPriority: 0,
},
@ -1809,44 +1810,15 @@ func testRepair(t *testing.T, tt *rewindTest, snapshots bool) {
contractMock := bor.NewMockGenesisContract(ctrl)
// Initialize a fresh chain
var (
gspec = &core.Genesis{
Config: params.BorUnittestChainConfig,
BaseFee: big.NewInt(params.InitialBaseFee),
}
config = &core.CacheConfig{
TrieCleanLimit: 256,
TrieDirtyLimit: 256,
TrieTimeLimit: 5 * time.Minute,
SnapshotLimit: 0, // Disable snapshot by default
}
)
engine := miner.NewFakeBor(t, db, params.BorUnittestChainConfig, ethAPIMock, spanner, heimdallClientMock, contractMock)
engine := miner.NewFakeBor(t, db, chainConfig, ethAPIMock, spanner, heimdallClientMock, contractMock)
defer engine.Close()
engineBorInternal, ok := engine.(*bor.Bor)
if ok {
gspec.ExtraData = make([]byte, 32+common.AddressLength+crypto.SignatureLength)
copy(gspec.ExtraData[32:32+common.AddressLength], testAddress1.Bytes())
chainConfig.LondonBlock = big.NewInt(0)
engineBorInternal.Authorize(testAddress1, func(account accounts.Account, s string, data []byte) ([]byte, error) {
return crypto.Sign(crypto.Keccak256(data), testKey1)
})
}
_, back, closeFn := miner.NewTestWorker(t, chainConfig, engine, db, 0)
defer closeFn()
genesis := gspec.MustCommit(db)
if snapshots {
config.SnapshotLimit = 256
config.SnapshotWait = true
}
chain, err := core.NewBlockChain(db, config, params.BorUnittestChainConfig, engine, vm.Config{}, nil, nil)
if err != nil {
t.Fatalf("Failed to create chain: %v", err)
}
genesis := back.BlockChain().Genesis()
// If sidechain blocks are needed, make a light chain and import it
var sideblocks types.Blocks
@ -1854,56 +1826,60 @@ func testRepair(t *testing.T, tt *rewindTest, snapshots bool) {
sideblocks, _ = core.GenerateChain(params.BorUnittestChainConfig, genesis, engine, rawdb.NewMemoryDatabase(), tt.sidechainBlocks, func(i int, b *core.BlockGen) {
b.SetCoinbase(testAddress1)
if bor.IsSprintStart(b.Number().Uint64(), params.BorUnittestChainConfig.Bor.Sprint) {
b.SetExtra(gspec.ExtraData)
if bor.IsSprintStart(b.Number().Uint64(), params.BorUnittestChainConfig.Bor.CalculateSprint(b.Number().Uint64())) {
b.SetExtra(back.Genesis.ExtraData)
} else {
b.SetExtra(make([]byte, 32+crypto.SignatureLength))
}
})
if _, err := chain.InsertChain(sideblocks); err != nil {
if _, err := back.BlockChain().InsertChain(sideblocks); err != nil {
t.Fatalf("Failed to import side chain: %v", err)
}
}
canonblocks, _ := core.GenerateChain(params.BorUnittestChainConfig, genesis, engine, rawdb.NewMemoryDatabase(), tt.canonicalBlocks, func(i int, b *core.BlockGen) {
b.SetCoinbase(common.Address{0x02})
b.SetCoinbase(miner.TestBankAddress)
b.SetDifficulty(big.NewInt(1000000))
if bor.IsSprintStart(b.Number().Uint64(), params.BorUnittestChainConfig.Bor.Sprint) {
b.SetExtra(gspec.ExtraData)
if bor.IsSprintStart(b.Number().Uint64(), params.BorUnittestChainConfig.Bor.CalculateSprint(b.Number().Uint64())) {
b.SetExtra(back.Genesis.ExtraData)
} else {
b.SetExtra(make([]byte, 32+crypto.SignatureLength))
}
})
if _, err := chain.InsertChain(canonblocks[:tt.commitBlock]); err != nil {
if _, err := back.BlockChain().InsertChain(canonblocks[:tt.commitBlock]); err != nil {
t.Fatalf("Failed to import canonical chain start: %v", err)
}
if tt.commitBlock > 0 {
err = chain.StateCache().TrieDB().Commit(canonblocks[tt.commitBlock-1].Root(), true, nil)
err = back.BlockChain().StateCache().TrieDB().Commit(canonblocks[tt.commitBlock-1].Root(), true, nil)
if err != nil {
t.Fatal("on trieDB.Commit", err)
}
if snapshots {
if err := chain.Snaps().Cap(canonblocks[tt.commitBlock-1].Root(), 0); err != nil {
if err := back.BlockChain().Snaps().Cap(canonblocks[tt.commitBlock-1].Root(), 0); err != nil {
t.Fatalf("Failed to flatten snapshots: %v", err)
}
}
}
if _, err := chain.InsertChain(canonblocks[tt.commitBlock:]); err != nil {
if _, err := back.BlockChain().InsertChain(canonblocks[tt.commitBlock:]); err != nil {
t.Fatalf("Failed to import canonical chain tail: %v", err)
}
// Force run a freeze cycle
type freezer interface {
Freeze(threshold uint64) error
Ancients() (uint64, error)
}
db.(freezer).Freeze(tt.freezeThreshold)
// Set the simulated pivot block
if tt.pivotBlock != nil {
rawdb.WriteLastPivotNumber(db, *tt.pivotBlock)
}
// Pull the plug on the database, simulating a hard crash
db.Close()
@ -1912,12 +1888,14 @@ func testRepair(t *testing.T, tt *rewindTest, snapshots bool) {
if err != nil {
t.Fatalf("Failed to reopen persistent database: %v", err)
}
defer db.Close()
newChain, err := core.NewBlockChain(db, nil, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil)
newChain, err := core.NewBlockChain(db, nil, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}
defer newChain.Stop()
// Iterate over all the remaining blocks and ensure there are no gaps
@ -1929,12 +1907,15 @@ func testRepair(t *testing.T, tt *rewindTest, snapshots bool) {
if head := newChain.CurrentHeader(); head.Number.Uint64() != tt.expHeadHeader {
t.Errorf("Head header mismatch: have %d, want %d", head.Number, tt.expHeadHeader)
}
if head := newChain.CurrentFastBlock(); head.NumberU64() != tt.expHeadFastBlock {
t.Errorf("Head fast block mismatch: have %d, want %d", head.NumberU64(), tt.expHeadFastBlock)
}
if head := newChain.CurrentBlock(); head.NumberU64() != tt.expHeadBlock {
t.Errorf("Head block mismatch: have %d, want %d", head.NumberU64(), tt.expHeadBlock)
}
if frozen, err := db.(freezer).Ancients(); err != nil {
t.Errorf("Failed to retrieve ancient count: %v\n", err)
} else if int(frozen) != tt.expFrozen {
@ -1988,7 +1969,7 @@ func TestIssue23496(t *testing.T) {
}
)
chain, err := core.NewBlockChain(db, config, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil)
chain, err := core.NewBlockChain(db, config, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to create chain: %v", err)
}
@ -2043,7 +2024,7 @@ func TestIssue23496(t *testing.T) {
defer db.Close()
chain, err = core.NewBlockChain(db, nil, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil)
chain, err = core.NewBlockChain(db, nil, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}

4
core/tests/blockchain_sethead_test.go
View file

@ -1987,7 +1987,7 @@ func testSetHead(t *testing.T, tt *rewindTest, snapshots bool) {
config.SnapshotWait = true
}
chain, err := core.NewBlockChain(db, config, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil)
chain, err := core.NewBlockChain(db, config, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to create chain: %v", err)
}
@ -2082,6 +2082,7 @@ func testSetHead(t *testing.T, tt *rewindTest, snapshots bool) {
// verifyNoGaps checks that there are no gaps after the initial set of blocks in
// the database and errors if found.
//
//nolint:gocognit
func verifyNoGaps(t *testing.T, chain *core.BlockChain, canonical bool, inserted types.Blocks) {
t.Helper()
@ -2135,6 +2136,7 @@ func verifyNoGaps(t *testing.T, chain *core.BlockChain, canonical bool, inserted
// verifyCutoff checks that there are no chain data available in the chain after
// the specified limit, but that it is available before.
//
//nolint:gocognit
func verifyCutoff(t *testing.T, chain *core.BlockChain, canonical bool, inserted types.Blocks, head int) {
t.Helper()

26
core/tests/blockchain_snapshot_test.go
View file

@ -84,7 +84,7 @@ func (basic *snapshotTestBasic) prepare(t *testing.T) (*core.BlockChain, []*type
cacheConfig = core.DefaultCacheConfig
)
chain, err := core.NewBlockChain(db, cacheConfig, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil)
chain, err := core.NewBlockChain(db, cacheConfig, params.AllEthashProtocolChanges, engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to create chain: %v", err)
}
@ -246,7 +246,7 @@ func (snaptest *snapshotTest) test(t *testing.T) {
// Restart the chain normally
chain.Stop()
newchain, err := core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil)
newchain, err := core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}
@ -283,13 +283,13 @@ func (snaptest *crashSnapshotTest) test(t *testing.T) {
// the crash, we do restart twice here: one after the crash and one
// after the normal stop. It's used to ensure the broken snapshot
// can be detected all the time.
newchain, err := core.NewBlockChain(newdb, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil)
newchain, err := core.NewBlockChain(newdb, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}
newchain.Stop()
newchain, err = core.NewBlockChain(newdb, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil)
newchain, err = core.NewBlockChain(newdb, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}
@ -327,7 +327,7 @@ func (snaptest *gappedSnapshotTest) test(t *testing.T) {
SnapshotLimit: 0,
}
newchain, err := core.NewBlockChain(snaptest.db, cacheConfig, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil)
newchain, err := core.NewBlockChain(snaptest.db, cacheConfig, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}
@ -336,7 +336,7 @@ func (snaptest *gappedSnapshotTest) test(t *testing.T) {
newchain.Stop()
// Restart the chain with enabling the snapshot
newchain, err = core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil)
newchain, err = core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}
@ -365,7 +365,7 @@ func (snaptest *setHeadSnapshotTest) test(t *testing.T) {
chain.SetHead(snaptest.setHead)
chain.Stop()
newchain, err := core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil)
newchain, err := core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}
@ -396,7 +396,7 @@ func (snaptest *restartCrashSnapshotTest) test(t *testing.T) {
// and state committed.
chain.Stop()
newchain, err := core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil)
newchain, err := core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}
@ -414,7 +414,7 @@ func (snaptest *restartCrashSnapshotTest) test(t *testing.T) {
// journal and latest state will be committed
// Restart the chain after the crash
newchain, err = core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil)
newchain, err = core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}
@ -449,8 +449,7 @@ func (snaptest *wipeCrashSnapshotTest) test(t *testing.T) {
TrieTimeLimit: 5 * time.Minute,
SnapshotLimit: 0,
}
newchain, err := core.NewBlockChain(snaptest.db, config, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil)
newchain, err := core.NewBlockChain(snaptest.db, config, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}
@ -467,14 +466,13 @@ func (snaptest *wipeCrashSnapshotTest) test(t *testing.T) {
SnapshotLimit: 256,
SnapshotWait: false, // Don't wait rebuild
}
_, err = core.NewBlockChain(snaptest.db, config, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil)
_, err = core.NewBlockChain(snaptest.db, config, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}
// Simulate the blockchain crash.
newchain, err = core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil)
newchain, err = core.NewBlockChain(snaptest.db, nil, params.AllEthashProtocolChanges, snaptest.engine, vm.Config{}, nil, nil, nil)
if err != nil {
t.Fatalf("Failed to recreate chain: %v", err)
}

22
core/tx_pool.go
View file

@ -268,6 +268,8 @@ type TxPool struct {
initDoneCh chan struct{} // is closed once the pool is initialized (for tests)
changesSinceReorg int // A counter for how many drops we've performed in-between reorg.
promoteTxCh chan struct{} // should be used only for tests
}
type txpoolResetRequest struct {
@ -276,7 +278,7 @@ type txpoolResetRequest struct {
// NewTxPool creates a new transaction pool to gather, sort and filter inbound
// transactions from the network.
func NewTxPool(config TxPoolConfig, chainconfig *params.ChainConfig, chain blockChain) *TxPool {
func NewTxPool(config TxPoolConfig, chainconfig *params.ChainConfig, chain blockChain, options ...func(pool *TxPool)) *TxPool {
// Sanitize the input to ensure no vulnerable gas prices are set
config = (&config).sanitize()
@ -299,6 +301,7 @@ func NewTxPool(config TxPoolConfig, chainconfig *params.ChainConfig, chain block
initDoneCh: make(chan struct{}),
gasPrice: new(big.Int).SetUint64(config.PriceLimit),
}
pool.locals = newAccountSet(pool.signer)
for _, addr := range config.Locals {
log.Info("Setting new local account", "address", addr)
@ -307,6 +310,11 @@ func NewTxPool(config TxPoolConfig, chainconfig *params.ChainConfig, chain block
pool.priced = newTxPricedList(pool.all)
pool.reset(nil, chain.CurrentBlock().Header())
// apply options
for _, fn := range options {
fn(pool)
}
// Start the reorg loop early so it can handle requests generated during journal loading.
pool.wg.Add(1)
go pool.scheduleReorgLoop()
@ -809,6 +817,17 @@ func (pool *TxPool) journalTx(from common.Address, tx *types.Transaction) {
//
// Note, this method assumes the pool lock is held!
func (pool *TxPool) promoteTx(addr common.Address, hash common.Hash, tx *types.Transaction) bool {
defer func() {
if pool.promoteTxCh == nil {
return
}
select {
case pool.promoteTxCh <- struct{}{}:
default:
}
}()
// Try to insert the transaction into the pending queue
if pool.pending[addr] == nil {
pool.pending[addr] = newTxList(true)
@ -1080,6 +1099,7 @@ func (pool *TxPool) scheduleReorgLoop() {
dirtyAccounts *accountSet
queuedEvents = make(map[common.Address]*txSortedMap)
)
for {
// Launch next background reorg if needed
if curDone == nil && launchNextRun {

557
core/tx_pool_test.go
View file

@ -17,6 +17,7 @@
package core
import (
"context"
"crypto/ecdsa"
"errors"
"fmt"
@ -24,10 +25,17 @@ import (
"math/big"
"math/rand"
"os"
"runtime"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
"gonum.org/v1/gonum/floats"
"gonum.org/v1/gonum/stat"
"pgregory.net/rapid"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/core/rawdb"
"github.com/ethereum/go-ethereum/core/state"
@ -47,6 +55,10 @@ var (
eip1559Config *params.ChainConfig
)
const (
txPoolGasLimit = 10_000_000
)
func init() {
testTxPoolConfig = DefaultTxPoolConfig
testTxPoolConfig.Journal = ""
@ -114,15 +126,17 @@ func dynamicFeeTx(nonce uint64, gaslimit uint64, gasFee *big.Int, tip *big.Int,
}
func setupTxPool() (*TxPool, *ecdsa.PrivateKey) {
return setupTxPoolWithConfig(params.TestChainConfig)
return setupTxPoolWithConfig(params.TestChainConfig, testTxPoolConfig, txPoolGasLimit)
}
func setupTxPoolWithConfig(config *params.ChainConfig) (*TxPool, *ecdsa.PrivateKey) {
func setupTxPoolWithConfig(config *params.ChainConfig, txPoolConfig TxPoolConfig, gasLimit uint64, options ...func(pool *TxPool)) (*TxPool, *ecdsa.PrivateKey) {
statedb, _ := state.New(common.Hash{}, state.NewDatabase(rawdb.NewMemoryDatabase()), nil)
blockchain := &testBlockChain{10000000, statedb, new(event.Feed)}
blockchain := &testBlockChain{gasLimit, statedb, new(event.Feed)}
key, _ := crypto.GenerateKey()
pool := NewTxPool(testTxPoolConfig, config, blockchain)
pool := NewTxPool(txPoolConfig, config, blockchain, options...)
// wait for the pool to initialize
<-pool.initDoneCh
@ -273,6 +287,16 @@ func testSetNonce(pool *TxPool, addr common.Address, nonce uint64) {
pool.mu.Unlock()
}
func getBalance(pool *TxPool, addr common.Address) *big.Int {
bal := big.NewInt(0)
pool.mu.Lock()
bal.Set(pool.currentState.GetBalance(addr))
pool.mu.Unlock()
return bal
}
func TestInvalidTransactions(t *testing.T) {
t.Parallel()
@ -384,7 +408,7 @@ func TestTransactionNegativeValue(t *testing.T) {
func TestTransactionTipAboveFeeCap(t *testing.T) {
t.Parallel()
pool, key := setupTxPoolWithConfig(eip1559Config)
pool, key := setupTxPoolWithConfig(eip1559Config, testTxPoolConfig, txPoolGasLimit)
defer pool.Stop()
tx := dynamicFeeTx(0, 100, big.NewInt(1), big.NewInt(2), key)
@ -397,7 +421,7 @@ func TestTransactionTipAboveFeeCap(t *testing.T) {
func TestTransactionVeryHighValues(t *testing.T) {
t.Parallel()
pool, key := setupTxPoolWithConfig(eip1559Config)
pool, key := setupTxPoolWithConfig(eip1559Config, testTxPoolConfig, txPoolGasLimit)
defer pool.Stop()
veryBigNumber := big.NewInt(1)
@ -1449,7 +1473,7 @@ func TestTransactionPoolRepricingDynamicFee(t *testing.T) {
t.Parallel()
// Create the pool to test the pricing enforcement with
pool, _ := setupTxPoolWithConfig(eip1559Config)
pool, _ := setupTxPoolWithConfig(eip1559Config, testTxPoolConfig, txPoolGasLimit)
defer pool.Stop()
// Keep track of transaction events to ensure all executables get announced
@ -1820,7 +1844,7 @@ func TestTransactionPoolStableUnderpricing(t *testing.T) {
func TestTransactionPoolUnderpricingDynamicFee(t *testing.T) {
t.Parallel()
pool, _ := setupTxPoolWithConfig(eip1559Config)
pool, _ := setupTxPoolWithConfig(eip1559Config, testTxPoolConfig, txPoolGasLimit)
defer pool.Stop()
pool.config.GlobalSlots = 2
@ -1927,7 +1951,7 @@ func TestTransactionPoolUnderpricingDynamicFee(t *testing.T) {
func TestDualHeapEviction(t *testing.T) {
t.Parallel()
pool, _ := setupTxPoolWithConfig(eip1559Config)
pool, _ := setupTxPoolWithConfig(eip1559Config, testTxPoolConfig, txPoolGasLimit)
defer pool.Stop()
pool.config.GlobalSlots = 10
@ -2130,7 +2154,7 @@ func TestTransactionReplacementDynamicFee(t *testing.T) {
t.Parallel()
// Create the pool to test the pricing enforcement with
pool, key := setupTxPoolWithConfig(eip1559Config)
pool, key := setupTxPoolWithConfig(eip1559Config, testTxPoolConfig, txPoolGasLimit)
defer pool.Stop()
testAddBalance(pool, crypto.PubkeyToAddress(key.PublicKey), big.NewInt(1000000000))
@ -2561,3 +2585,516 @@ func BenchmarkPoolMultiAccountBatchInsert(b *testing.B) {
pool.AddRemotesSync([]*types.Transaction{tx})
}
}
type acc struct {
nonce uint64
key *ecdsa.PrivateKey
account common.Address
}
type testTx struct {
tx *types.Transaction
idx int
isLocal bool
}
const localIdx = 0
func getTransactionGen(t *rapid.T, keys []*acc, nonces []uint64, localKey *acc, gasPriceMin, gasPriceMax, gasLimitMin, gasLimitMax uint64) *testTx {
idx := rapid.IntRange(0, len(keys)-1).Draw(t, "accIdx").(int)
var (
isLocal bool
key *ecdsa.PrivateKey
)
if idx == localIdx {
isLocal = true
key = localKey.key
} else {
key = keys[idx].key
}
nonces[idx]++
gasPriceUint := rapid.Uint64Range(gasPriceMin, gasPriceMax).Draw(t, "gasPrice").(uint64)
gasPrice := big.NewInt(0).SetUint64(gasPriceUint)
gasLimit := rapid.Uint64Range(gasLimitMin, gasLimitMax).Draw(t, "gasLimit").(uint64)
return &testTx{
tx: pricedTransaction(nonces[idx]-1, gasLimit, gasPrice, key),
idx: idx,
isLocal: isLocal,
}
}
type transactionBatches struct {
txs []*testTx
totalTxs int
}
func transactionsGen(keys []*acc, nonces []uint64, localKey *acc, minTxs int, maxTxs int, gasPriceMin, gasPriceMax, gasLimitMin, gasLimitMax uint64, caseParams *strings.Builder) func(t *rapid.T) *transactionBatches {
return func(t *rapid.T) *transactionBatches {
totalTxs := rapid.IntRange(minTxs, maxTxs).Draw(t, "totalTxs").(int)
txs := make([]*testTx, totalTxs)
gasValues := make([]float64, totalTxs)
fmt.Fprintf(caseParams, " totalTxs = %d;", totalTxs)
keys = keys[:len(nonces)]
for i := 0; i < totalTxs; i++ {
txs[i] = getTransactionGen(t, keys, nonces, localKey, gasPriceMin, gasPriceMax, gasLimitMin, gasLimitMax)
gasValues[i] = float64(txs[i].tx.Gas())
}
mean, stddev := stat.MeanStdDev(gasValues, nil)
fmt.Fprintf(caseParams, " gasValues mean %d, stdev %d, %d-%d);", int64(mean), int64(stddev), int64(floats.Min(gasValues)), int64(floats.Max(gasValues)))
return &transactionBatches{txs, totalTxs}
}
}
type txPoolRapidConfig struct {
gasLimit uint64
avgBlockTxs uint64
minTxs int
maxTxs int
minAccs int
maxAccs int
// less tweakable, more like constants
gasPriceMin uint64
gasPriceMax uint64
gasLimitMin uint64
gasLimitMax uint64
balance int64
blockTime time.Duration
maxEmptyBlocks int
maxStuckBlocks int
}
func defaultTxPoolRapidConfig() txPoolRapidConfig {
gasLimit := uint64(30_000_000)
avgBlockTxs := gasLimit/params.TxGas + 1
maxTxs := int(25 * avgBlockTxs)
return txPoolRapidConfig{
gasLimit: gasLimit,
avgBlockTxs: avgBlockTxs,
minTxs: 1,
maxTxs: maxTxs,
minAccs: 1,
maxAccs: maxTxs,
// less tweakable, more like constants
gasPriceMin: 1,
gasPriceMax: 1_000,
gasLimitMin: params.TxGas,
gasLimitMax: gasLimit / 2,
balance: 0xffffffffffffff,
blockTime: 2 * time.Second,
maxEmptyBlocks: 10,
maxStuckBlocks: 10,
}
}
// TestSmallTxPool is not something to run in parallel as far it uses all CPUs
// nolint:paralleltest
func TestSmallTxPool(t *testing.T) {
t.Parallel()
t.Skip("a red test to be fixed")
cfg := defaultTxPoolRapidConfig()
cfg.maxEmptyBlocks = 10
cfg.maxStuckBlocks = 10
cfg.minTxs = 1
cfg.maxTxs = 2
cfg.minAccs = 1
cfg.maxAccs = 2
testPoolBatchInsert(t, cfg)
}
// This test is not something to run in parallel as far it uses all CPUs
// nolint:paralleltest
func TestBigTxPool(t *testing.T) {
t.Parallel()
t.Skip("a red test to be fixed")
cfg := defaultTxPoolRapidConfig()
testPoolBatchInsert(t, cfg)
}
//nolint:gocognit,thelper
func testPoolBatchInsert(t *testing.T, cfg txPoolRapidConfig) {
t.Helper()
t.Parallel()
const debug = false
initialBalance := big.NewInt(cfg.balance)
keys := make([]*acc, cfg.maxAccs)
var key *ecdsa.PrivateKey
// prealloc keys
for idx := 0; idx < cfg.maxAccs; idx++ {
key, _ = crypto.GenerateKey()
keys[idx] = &acc{
key: key,
nonce: 0,
account: crypto.PubkeyToAddress(key.PublicKey),
}
}
var threads = runtime.NumCPU()
if debug {
// 1 is set only for debug
threads = 1
}
testsDone := new(uint64)
for i := 0; i < threads; i++ {
t.Run(fmt.Sprintf("thread %d", i), func(t *testing.T) {
t.Parallel()
rapid.Check(t, func(rt *rapid.T) {
caseParams := new(strings.Builder)
defer func() {
res := atomic.AddUint64(testsDone, 1)
if res%100 == 0 {
fmt.Println("case-done", res)
}
}()
// Generate a batch of transactions to enqueue into the pool
testTxPoolConfig := testTxPoolConfig
// from sentry config
testTxPoolConfig.AccountQueue = 16
testTxPoolConfig.AccountSlots = 16
testTxPoolConfig.GlobalQueue = 32768
testTxPoolConfig.GlobalSlots = 32768
testTxPoolConfig.Lifetime = time.Hour + 30*time.Minute //"1h30m0s"
testTxPoolConfig.PriceLimit = 1
now := time.Now()
pendingAddedCh := make(chan struct{}, 1024)
pool, key := setupTxPoolWithConfig(params.TestChainConfig, testTxPoolConfig, cfg.gasLimit, MakeWithPromoteTxCh(pendingAddedCh))
defer pool.Stop()
totalAccs := rapid.IntRange(cfg.minAccs, cfg.maxAccs).Draw(rt, "totalAccs").(int)
fmt.Fprintf(caseParams, "Case params: totalAccs = %d;", totalAccs)
defer func() {
pending, queued := pool.Content()
if len(pending) != 0 {
pendingGas := make([]float64, 0, len(pending))
for _, txs := range pending {
for _, tx := range txs {
pendingGas = append(pendingGas, float64(tx.Gas()))
}
}
mean, stddev := stat.MeanStdDev(pendingGas, nil)
fmt.Fprintf(caseParams, "\tpending mean %d, stdev %d, %d-%d;\n", int64(mean), int64(stddev), int64(floats.Min(pendingGas)), int64(floats.Max(pendingGas)))
}
if len(queued) != 0 {
queuedGas := make([]float64, 0, len(queued))
for _, txs := range queued {
for _, tx := range txs {
queuedGas = append(queuedGas, float64(tx.Gas()))
}
}
mean, stddev := stat.MeanStdDev(queuedGas, nil)
fmt.Fprintf(caseParams, "\tqueued mean %d, stdev %d, %d-%d);\n\n", int64(mean), int64(stddev), int64(floats.Min(queuedGas)), int64(floats.Max(queuedGas)))
}
rt.Log(caseParams)
}()
// regenerate only local key
localKey := &acc{
key: key,
account: crypto.PubkeyToAddress(key.PublicKey),
}
if err := validateTxPoolInternals(pool); err != nil {
rt.Fatalf("pool internal state corrupted: %v", err)
}
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
now = time.Now()
testAddBalance(pool, localKey.account, initialBalance)
for idx := 0; idx < totalAccs; idx++ {
testAddBalance(pool, keys[idx].account, initialBalance)
}
}()
nonces := make([]uint64, totalAccs)
gen := rapid.Custom(transactionsGen(keys, nonces, localKey, cfg.minTxs, cfg.maxTxs, cfg.gasPriceMin, cfg.gasPriceMax, cfg.gasLimitMin, cfg.gasLimitMax, caseParams))
txs := gen.Draw(rt, "batches").(*transactionBatches)
wg.Wait()
var (
addIntoTxPool func(tx []*types.Transaction) []error
totalInBatch int
)
for _, tx := range txs.txs {
addIntoTxPool = pool.AddRemotesSync
if tx.isLocal {
addIntoTxPool = pool.AddLocals
}
err := addIntoTxPool([]*types.Transaction{tx.tx})
if len(err) != 0 && err[0] != nil {
rt.Log("on adding a transaction to the tx pool", err[0], tx.tx.Gas(), tx.tx.GasPrice(), pool.GasPrice(), getBalance(pool, keys[tx.idx].account))
}
}
var (
block int
emptyBlocks int
stuckBlocks int
lastTxPoolStats int
currentTxPoolStats int
)
for {
// we'd expect fulfilling block take comparable, but less than blockTime
ctx, cancel := context.WithTimeout(context.Background(), time.Duration(cfg.maxStuckBlocks)*cfg.blockTime)
select {
case <-pendingAddedCh:
case <-ctx.Done():
pendingStat, queuedStat := pool.Stats()
if pendingStat+queuedStat == 0 {
cancel()
break
}
rt.Fatalf("got %ds block timeout (expected less then %s): total accounts %d. Pending %d, queued %d)",
block, 5*cfg.blockTime, txs.totalTxs, pendingStat, queuedStat)
}
pendingStat, queuedStat := pool.Stats()
currentTxPoolStats = pendingStat + queuedStat
if currentTxPoolStats == 0 {
cancel()
break
}
// check if txPool got stuck
if currentTxPoolStats == lastTxPoolStats {
stuckBlocks++ //todo: переписать
} else {
stuckBlocks = 0
lastTxPoolStats = currentTxPoolStats
}
// copy-paste
start := time.Now()
pending := pool.Pending(true)
locals := pool.Locals()
// from fillTransactions
removedFromPool, blockGasLeft, err := fillTransactions(ctx, pool, locals, pending, cfg.gasLimit)
done := time.Since(start)
if removedFromPool > 0 {
emptyBlocks = 0
} else {
emptyBlocks++
}
if emptyBlocks >= cfg.maxEmptyBlocks || stuckBlocks >= cfg.maxStuckBlocks {
// check for nonce gaps
var lastNonce, currentNonce int
pending = pool.Pending(true)
for txAcc, pendingTxs := range pending {
lastNonce = int(pool.Nonce(txAcc)) - len(pendingTxs) - 1
isFirst := true
for _, tx := range pendingTxs {
currentNonce = int(tx.Nonce())
if currentNonce-lastNonce != 1 {
rt.Fatalf("got a nonce gap for account %q. Current pending nonce %d, previous %d %v; emptyBlocks - %v; stuckBlocks - %v",
txAcc, currentNonce, lastNonce, isFirst, emptyBlocks >= cfg.maxEmptyBlocks, stuckBlocks >= cfg.maxStuckBlocks)
}
lastNonce = currentNonce
}
}
}
if emptyBlocks >= cfg.maxEmptyBlocks {
rt.Fatalf("got %d empty blocks in a row(expected less then %d): total time %s, total accounts %d. Pending %d, locals %d)",
emptyBlocks, cfg.maxEmptyBlocks, done, txs.totalTxs, len(pending), len(locals))
}
if stuckBlocks >= cfg.maxStuckBlocks {
rt.Fatalf("got %d empty blocks in a row(expected less then %d): total time %s, total accounts %d. Pending %d, locals %d)",
emptyBlocks, cfg.maxEmptyBlocks, done, txs.totalTxs, len(pending), len(locals))
}
if err != nil {
rt.Fatalf("took too long: total time %s(expected %s), total accounts %d. Pending %d, locals %d)",
done, cfg.blockTime, txs.totalTxs, len(pending), len(locals))
}
rt.Log("current_total", txs.totalTxs, "in_batch", totalInBatch, "removed", removedFromPool, "emptyBlocks", emptyBlocks, "blockGasLeft", blockGasLeft, "pending", len(pending), "locals", len(locals),
"locals+pending", done)
rt.Log("block", block, "pending", pendingStat, "queued", queuedStat, "elapsed", done)
block++
cancel()
//time.Sleep(time.Second)
}
rt.Logf("case completed totalTxs %d %v\n\n", txs.totalTxs, time.Since(now))
})
})
}
t.Log("done test cases", atomic.LoadUint64(testsDone))
}
func fillTransactions(ctx context.Context, pool *TxPool, locals []common.Address, pending map[common.Address]types.Transactions, gasLimit uint64) (int, uint64, error) {
localTxs := make(map[common.Address]types.Transactions)
remoteTxs := pending
for _, txAcc := range locals {
if txs := remoteTxs[txAcc]; len(txs) > 0 {
delete(remoteTxs, txAcc)
localTxs[txAcc] = txs
}
}
// fake signer
signer := types.NewLondonSigner(big.NewInt(1))
// fake baseFee
baseFee := big.NewInt(1)
blockGasLimit := gasLimit
var (
txLocalCount int
txRemoteCount int
)
if len(localTxs) > 0 {
txs := types.NewTransactionsByPriceAndNonce(signer, localTxs, baseFee)
select {
case <-ctx.Done():
return txLocalCount + txRemoteCount, blockGasLimit, ctx.Err()
default:
}
blockGasLimit, txLocalCount = commitTransactions(pool, txs, blockGasLimit)
}
select {
case <-ctx.Done():
return txLocalCount + txRemoteCount, blockGasLimit, ctx.Err()
default:
}
if len(remoteTxs) > 0 {
txs := types.NewTransactionsByPriceAndNonce(signer, remoteTxs, baseFee)
select {
case <-ctx.Done():
return txLocalCount + txRemoteCount, blockGasLimit, ctx.Err()
default:
}
blockGasLimit, txRemoteCount = commitTransactions(pool, txs, blockGasLimit)
}
return txLocalCount + txRemoteCount, blockGasLimit, nil
}
func commitTransactions(pool *TxPool, txs *types.TransactionsByPriceAndNonce, blockGasLimit uint64) (uint64, int) {
var (
tx *types.Transaction
txCount int
)
for {
tx = txs.Peek()
if tx == nil {
return blockGasLimit, txCount
}
if tx.Gas() <= blockGasLimit {
blockGasLimit -= tx.Gas()
pool.removeTx(tx.Hash(), false)
txCount++
} else {
// we don't maximize fulfilment of the block. just fill somehow
return blockGasLimit, txCount
}
}
}
func MakeWithPromoteTxCh(ch chan struct{}) func(*TxPool) {
return func(pool *TxPool) {
pool.promoteTxCh = ch
}
}

8
core/types/transaction.go
View file

@ -508,6 +508,10 @@ func NewTransactionsByPriceAndNonce(signer Signer, txs map[common.Address]Transa
// Initialize a price and received time based heap with the head transactions
heads := make(TxByPriceAndTime, 0, len(txs))
for from, accTxs := range txs {
if len(accTxs) == 0 {
continue
}
acc, _ := Sender(signer, accTxs[0])
wrapped, err := NewTxWithMinerFee(accTxs[0], baseFee)
// Remove transaction if sender doesn't match from, or if wrapping fails.
@ -550,6 +554,10 @@ func (t *TransactionsByPriceAndNonce) Shift() {
heap.Pop(&t.heads)
}
func (t *TransactionsByPriceAndNonce) GetTxs() int {
return len(t.txs)
}
// Pop removes the best transaction, *not* replacing it with the next one from
// the same account. This should be used when a transaction cannot be executed
// and hence all subsequent ones should be discarded from the same account.

4
docs/cli/README.md
View file

@ -44,6 +44,10 @@
- [```server```](./server.md)
- [```snapshot```](./snapshot.md)
- [```snapshot prune-state```](./snapshot_prune-state.md)
- [```status```](./status.md)
- [```version```](./version.md)

2
docs/cli/account_import.md
View file

@ -6,4 +6,4 @@ The ```account import``` command imports an account in Json format to the Bor da
- ```datadir```: Path of the data directory to store information
- ```keystore```: Path of the data directory to store information
- ```keystore```: Path of the data directory to store keys

2
docs/cli/account_list.md
View file

@ -6,4 +6,4 @@ The `account list` command lists all the accounts in the Bor data directory.
- ```datadir```: Path of the data directory to store information
- ```keystore```: Path of the data directory to store information
- ```keystore```: Path of the data directory to store keys

2
docs/cli/account_new.md
View file

@ -6,4 +6,4 @@ The `account new` command creates a new local account file on the Bor data direc
- ```datadir```: Path of the data directory to store information
- ```keystore```: Path of the data directory to store information
- ```keystore```: Path of the data directory to store keys

4
docs/cli/server.md
View file

@ -32,6 +32,8 @@ The ```bor server``` command runs the Bor client.
- ```bor.withoutheimdall```: Run without Heimdall service (for testing purpose) (default: false)
- ```bor.heimdallgRPC```: Address of Heimdall gRPC service
- ```ethstats```: Reporting URL of a ethstats service (nodename:secret@host:port)
- ```gpo.blocks```: Number of recent blocks to check for gas prices (default: 20)
@ -80,6 +82,8 @@ The ```bor server``` command runs the Bor client.
- ```cache.preimages```: Enable recording the SHA3/keccak preimages of trie keys (default: false)
- ```cache.triesinmemory```: Number of block states (tries) to keep in memory (default = 128) (default: 128)
- ```txlookuplimit```: Number of recent blocks to maintain transactions index for (default: 2350000)
### JsonRPC Options

5
docs/cli/snapshot.md Normal file
View file

@ -0,0 +1,5 @@
# snapshot
The ```snapshot``` command groups snapshot related actions:
- [```snapshot prune-state```](./snapshot_prune-state.md): Prune state databases at the given datadir location.

21
docs/cli/snapshot_prune-state.md Normal file
View file

@ -0,0 +1,21 @@
# Prune state
The ```bor snapshot prune-state``` command will prune historical state data with the help of the state snapshot. All trie nodes and contract codes that do not belong to the specified version state will be deleted from the database. After pruning, only two version states are available: genesis and the specific one.
## Options
- ```datadir```: Path of the data directory to store information
- ```keystore```: Path of the data directory to store keys
- ```datadir.ancient```: Path of the ancient data directory to store information
- ```bloomfilter.size```: Size of the bloom filter (default: 2048)
### Cache Options
- ```cache```: Megabytes of memory allocated to internal caching (default: 1024)
- ```cache.trie```: Percentage of cache memory allowance to use for trie caching (default: 25)
- ```cache.trie.journal```: Path of the trie journal directory to store information (default: triecache)

3
eth/api.go
View file

@ -343,7 +343,8 @@ func (api *PrivateDebugAPI) GetBadBlocks(ctx context.Context) ([]*BadBlockArgs,
} else {
blockRlp = fmt.Sprintf("0x%x", rlpBytes)
}
if blockJSON, err = ethapi.RPCMarshalBlock(block, true, true, api.eth.APIBackend.ChainConfig()); err != nil {
if blockJSON, err = ethapi.RPCMarshalBlock(block, true, true, api.eth.APIBackend.ChainConfig(), api.eth.chainDb); err != nil {
blockJSON = map[string]interface{}{"error": err.Error()}
}
results = append(results, &BadBlockArgs{

52
eth/backend.go
View file

@ -18,6 +18,7 @@
package eth
import (
"context"
"errors"
"fmt"
"math/big"
@ -40,6 +41,7 @@ import (
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/core/vm"
"github.com/ethereum/go-ethereum/eth/downloader"
"github.com/ethereum/go-ethereum/eth/downloader/whitelist"
"github.com/ethereum/go-ethereum/eth/ethconfig"
"github.com/ethereum/go-ethereum/eth/filters"
"github.com/ethereum/go-ethereum/eth/gasprice"
@ -216,9 +218,13 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) {
TrieTimeLimit: config.TrieTimeout,
SnapshotLimit: config.SnapshotCache,
Preimages: config.Preimages,
TriesInMemory: config.TriesInMemory,
}
)
eth.blockchain, err = core.NewBlockChain(chainDb, cacheConfig, chainConfig, eth.engine, vmConfig, eth.shouldPreserve, &config.TxLookupLimit)
checker := whitelist.NewService(10)
eth.blockchain, err = core.NewBlockChain(chainDb, cacheConfig, chainConfig, eth.engine, vmConfig, eth.shouldPreserve, &config.TxLookupLimit, checker)
if err != nil {
return nil, err
}
@ -259,6 +265,7 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) {
Checkpoint: checkpoint,
EthAPI: ethAPI,
PeerRequiredBlocks: config.PeerRequiredBlocks,
checker: checker,
}); err != nil {
return nil, err
}
@ -376,6 +383,10 @@ func (s *Ethereum) ResetWithGenesisBlock(gb *types.Block) {
s.blockchain.ResetWithGenesisBlock(gb)
}
func (s *Ethereum) PublicBlockChainAPI() *ethapi.PublicBlockChainAPI {
return s.handler.ethAPI
}
func (s *Ethereum) Etherbase() (eb common.Address, err error) {
s.lock.RLock()
etherbase := s.etherbase
@ -621,6 +632,13 @@ func (s *Ethereum) Start() error {
return nil
}
var (
ErrNotBorConsensus = errors.New("not bor consensus was given")
ErrBorConsensusWithoutHeimdall = errors.New("bor consensus without heimdall")
whitelistTimeout = 30 * time.Second
)
// StartCheckpointWhitelistService starts the goroutine to fetch checkpoints and update the
// checkpoint whitelist map.
func (s *Ethereum) startCheckpointWhitelistService() {
@ -632,7 +650,11 @@ func (s *Ethereum) startCheckpointWhitelistService() {
}
// first run the checkpoint whitelist
err := s.handleWhitelistCheckpoint()
firstCtx, cancel := context.WithTimeout(context.Background(), whitelistTimeout)
err := s.handleWhitelistCheckpoint(firstCtx, true)
cancel()
if err != nil {
if errors.Is(err, ErrBorConsensusWithoutHeimdall) || errors.Is(err, ErrNotBorConsensus) {
return
@ -647,7 +669,11 @@ func (s *Ethereum) startCheckpointWhitelistService() {
for {
select {
case <-ticker.C:
err := s.handleWhitelistCheckpoint()
ctx, cancel := context.WithTimeout(context.Background(), whitelistTimeout)
err := s.handleWhitelistCheckpoint(ctx, false)
cancel()
if err != nil {
log.Warn("unable to whitelist checkpoint", "err", err)
}
@ -657,13 +683,8 @@ func (s *Ethereum) startCheckpointWhitelistService() {
}
}
var (
ErrNotBorConsensus = errors.New("not bor consensus was given")
ErrBorConsensusWithoutHeimdall = errors.New("bor consensus without heimdall")
)
// handleWhitelistCheckpoint handles the checkpoint whitelist mechanism.
func (s *Ethereum) handleWhitelistCheckpoint() error {
func (s *Ethereum) handleWhitelistCheckpoint(ctx context.Context, first bool) error {
ethHandler := (*ethHandler)(s.handler)
bor, ok := ethHandler.chain.Engine().(*bor.Bor)
@ -675,13 +696,20 @@ func (s *Ethereum) handleWhitelistCheckpoint() error {
return ErrBorConsensusWithoutHeimdall
}
endBlockNum, endBlockHash, err := ethHandler.fetchWhitelistCheckpoint(bor)
if err != nil {
// Create a new checkpoint verifier
verifier := newCheckpointVerifier(nil)
blockNums, blockHashes, err := ethHandler.fetchWhitelistCheckpoints(ctx, bor, verifier, first)
// If the array is empty, we're bound to receive an error. Non-nill error and non-empty array
// means that array has partial elements and it failed for some block. We'll add those partial
// elements anyway.
if len(blockNums) == 0 {
return err
}
// Update the checkpoint whitelist map.
ethHandler.downloader.ProcessCheckpoint(endBlockNum, endBlockHash)
for i := 0; i < len(blockNums); i++ {
ethHandler.downloader.ProcessCheckpoint(blockNums[i], blockHashes[i])
}
return nil
}

60
eth/bor_checkpoint_verifier.go Normal file
View file

@ -0,0 +1,60 @@
package eth
import (
"context"
"fmt"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/ethereum/go-ethereum/consensus/bor/heimdall/checkpoint"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/rpc"
)
type checkpointVerifier struct {
verify func(ctx context.Context, handler *ethHandler, checkpoint *checkpoint.Checkpoint) (string, error)
}
func newCheckpointVerifier(verifyFn func(ctx context.Context, handler *ethHandler, checkpoint *checkpoint.Checkpoint) (string, error)) *checkpointVerifier {
if verifyFn != nil {
return &checkpointVerifier{verifyFn}
}
verifyFn = func(ctx context.Context, handler *ethHandler, checkpoint *checkpoint.Checkpoint) (string, error) {
var (
startBlock = checkpoint.StartBlock.Uint64()
endBlock = checkpoint.EndBlock.Uint64()
)
// check if we have the checkpoint blocks
head := handler.ethAPI.BlockNumber()
if head < hexutil.Uint64(endBlock) {
log.Debug("Head block behind checkpoint block", "head", head, "checkpoint end block", endBlock)
return "", errMissingCheckpoint
}
// verify the root hash of checkpoint
roothash, err := handler.ethAPI.GetRootHash(ctx, startBlock, endBlock)
if err != nil {
log.Debug("Failed to get root hash of checkpoint while whitelisting", "err", err)
return "", errRootHash
}
if roothash != checkpoint.RootHash.String()[2:] {
log.Warn("Checkpoint root hash mismatch while whitelisting", "expected", checkpoint.RootHash.String()[2:], "got", roothash)
return "", errCheckpointRootHashMismatch
}
// fetch the end checkpoint block hash
block, err := handler.ethAPI.GetBlockByNumber(ctx, rpc.BlockNumber(endBlock), false)
if err != nil {
log.Debug("Failed to get end block hash of checkpoint while whitelisting", "err", err)
return "", errEndBlock
}
hash := fmt.Sprintf("%v", block["hash"])
return hash, nil
}
return &checkpointVerifier{verifyFn}
}

30
eth/downloader/downloader.go
View file

@ -144,7 +144,7 @@ type Downloader struct {
quitCh chan struct{} // Quit channel to signal termination
quitLock sync.Mutex // Lock to prevent double closes
ChainValidator
ethereum.ChainValidator
// Testing hooks
syncInitHook func(uint64, uint64) // Method to call upon initiating a new sync run
@ -153,14 +153,6 @@ type Downloader struct {
chainInsertHook func([]*fetchResult) // Method to call upon inserting a chain of blocks (possibly in multiple invocations)
}
// interface for whitelist service
type ChainValidator interface {
IsValidChain(remoteHeader *types.Header, fetchHeadersByNumber func(number uint64, amount int, skip int, reverse bool) ([]*types.Header, []common.Hash, error)) (bool, error)
ProcessCheckpoint(endBlockNum uint64, endBlockHash common.Hash)
GetCheckpointWhitelist() map[uint64]common.Hash
PurgeCheckpointWhitelist()
}
// LightChain encapsulates functions required to synchronise a light chain.
type LightChain interface {
// HasHeader verifies a header's presence in the local chain.
@ -215,8 +207,8 @@ type BlockChain interface {
}
// New creates a new downloader to fetch hashes and blocks from remote peers.
//nolint: staticcheck
func New(checkpoint uint64, stateDb ethdb.Database, mux *event.TypeMux, chain BlockChain, lightchain LightChain, dropPeer peerDropFn, success func(), whitelistService ChainValidator) *Downloader {
// nolint: staticcheck
func New(checkpoint uint64, stateDb ethdb.Database, mux *event.TypeMux, chain BlockChain, lightchain LightChain, dropPeer peerDropFn, success func(), whitelistService ethereum.ChainValidator) *Downloader {
if lightchain == nil {
lightchain = chain
}
@ -737,9 +729,11 @@ func (d *Downloader) fetchHead(p *peerConnection) (head *types.Header, pivot *ty
// calculateRequestSpan calculates what headers to request from a peer when trying to determine the
// common ancestor.
// It returns parameters to be used for peer.RequestHeadersByNumber:
// from - starting block number
// count - number of headers to request
// skip - number of headers to skip
//
// from - starting block number
// count - number of headers to request
// skip - number of headers to skip
//
// and also returns 'max', the last block which is expected to be returned by the remote peers,
// given the (from,count,skip)
func calculateRequestSpan(remoteHeight, localHeight uint64) (int64, int, int, uint64) {
@ -799,9 +793,11 @@ func (d *Downloader) getFetchHeadersByNumber(p *peerConnection) func(number uint
// In the rare scenario when we ended up on a long reorganisation (i.e. none of
// the head links match), we do a binary search to find the common ancestor.
func (d *Downloader) findAncestor(p *peerConnection, remoteHeader *types.Header) (uint64, error) {
// Check the validity of chain to be downloaded
if _, err := d.IsValidChain(remoteHeader, d.getFetchHeadersByNumber(p)); err != nil {
return 0, err
// Check the validity of peer from which the chain is to be downloaded
if d.ChainValidator != nil {
if _, err := d.IsValidPeer(remoteHeader, d.getFetchHeadersByNumber(p)); err != nil {
return 0, err
}
}
// Figure out the valid ancestor range to prevent rewrite attacks

15
eth/downloader/downloader_test.go
View file

@ -71,7 +71,7 @@ func newTester() *downloadTester {
core.GenesisBlockForTesting(db, testAddress, big.NewInt(1000000000000000))
chain, err := core.NewBlockChain(db, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, nil)
chain, err := core.NewBlockChain(db, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
if err != nil {
panic(err)
}
@ -88,7 +88,7 @@ func newTester() *downloadTester {
return tester
}
func (dl *downloadTester) setWhitelist(w ChainValidator) {
func (dl *downloadTester) setWhitelist(w ethereum.ChainValidator) {
dl.downloader.ChainValidator = w
}
@ -1416,9 +1416,9 @@ func newWhitelistFake(validate func(count int) (bool, error)) *whitelistFake {
return &whitelistFake{0, validate}
}
// IsValidChain is the mock function which the downloader will use to validate the chain
// IsValidPeer is the mock function which the downloader will use to validate the chain
// to be received from a peer.
func (w *whitelistFake) IsValidChain(_ *types.Header, _ func(number uint64, amount int, skip int, reverse bool) ([]*types.Header, []common.Hash, error)) (bool, error) {
func (w *whitelistFake) IsValidPeer(_ *types.Header, _ func(number uint64, amount int, skip int, reverse bool) ([]*types.Header, []common.Hash, error)) (bool, error) {
defer func() {
w.count++
}()
@ -1426,13 +1426,18 @@ func (w *whitelistFake) IsValidChain(_ *types.Header, _ func(number uint64, amou
return w.validate(w.count)
}
func (w *whitelistFake) IsValidChain(current *types.Header, headers []*types.Header) bool {
return true
}
func (w *whitelistFake) ProcessCheckpoint(_ uint64, _ common.Hash) {}
func (w *whitelistFake) GetCheckpointWhitelist() map[uint64]common.Hash {
return nil
}
func (w *whitelistFake) PurgeCheckpointWhitelist() {}
func (w *whitelistFake) GetCheckpoints(current, sidechainHeader *types.Header, sidechainCheckpoints []*types.Header) (map[uint64]*types.Header, error) {
return map[uint64]*types.Header{}, nil
}
// TestFakedSyncProgress66WhitelistMismatch tests if in case of whitelisted
// checkpoint mismatch with opposite peer, the sync should fail.

2
eth/downloader/testchain_test.go
View file

@ -214,7 +214,7 @@ func newTestBlockchain(blocks []*types.Block) *core.BlockChain {
db := rawdb.NewMemoryDatabase()
core.GenesisBlockForTesting(db, testAddress, big.NewInt(1000000000000000))
chain, err := core.NewBlockChain(db, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, nil)
chain, err := core.NewBlockChain(db, nil, params.TestChainConfig, ethash.NewFaker(), vm.Config{}, nil, nil, nil)
if err != nil {
panic(err)
}

Some files were not shown because too many files have changed in this diff Show more