go-ethereum/internal/cli/server/config.go
marcello33 3f71e3b676
mardizzone/merge-qa-to-master (#856)
* Added checks to RPC requests and introduced new flags to customise the parameters (#657)

* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)

* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)

* updated the config files and docs

* chg : trieTimeout from 60 to 10 mins (#692)

* chg : trieTimeout from 60 to 10 mins

* chg : cache.timout to 10m from 1h in configs

* internal/cli/server : fix : added triesInMemory in config (#691)

* changed version from 0.3.0 to 0.3.4-beta (#693)

* fix nil state-sync issue, increase grpc limit (#695)

* Increase grpc message size limit in pprof

* consensus/bor/bor.go : stateSyncs init fixed [Fix #686]

* eth/filters: handle nil state-sync before notify

* eth/filters: update check

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>

* core, tests/bor: add more tests for state-sync validation (#710)

* core: add get state sync function for tests

* tests/bor: add validation for state sync events post consensus

* Arpit/temp bor sync (#701)

* Increase grpc message size limit in pprof

* ReadBorReceipts improvements

* use internal function

* fix tests

* fetch geth upstread for ReadBorReceiptRLP

* Only query bor receipt when the query index is equal to # tx in block body

This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.

* Revert "fetch geth upstread for ReadBorReceiptRLP"

This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.

* Restore ReadBorReceiptRLP

* fix bor receipts

* remove unused

* fix lints

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)

This reverts commit 241843c7e7.

* Arpit/add execution pool 2 (#719)

* initial

* linters

* linters

* remove timeout

* update pool

* change pool size function

* check nil

* check nil

* fix tests

* Use execution pool from server in all handlers

* simplify things

* test fix

* add support for cli, config

* add to cli and config

* merge base branch

* debug statements

* fix bug

* atomic pointer timeout

* add apis

* update workerpool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* fix tests

* mutex

* refactor flag and value names

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* debug statements

* fix bug

* update workerpool

* atomic pointer timeout

* add apis

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* merge base branch

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* mutex

* fix tests

* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* Change default size of execution pool to 40

* refactor flag and value names

* fix merge conflicts

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* fix linters

* fix go.mod

* change sec to ms

* change default value for ep timeout

* fix node api calls

* comment setter for ep timeout

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* version change (#721)

* Event based pprof (#732)

* feature

* Save pprof to /tmp

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>

* Cherry-pick changes from develop (#738)

* Check if block is nil to prevent panic (#736)

* miner: use env for tracing instead of block object (#728)

---------

Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* add max code init size check in txpool (#739)

* Revert "Event based pprof" and update version (#742)

* Revert "Event based pprof (#732)"

This reverts commit 22fa4033e8.

* params: update version to 0.3.4-beta3

* packaging/templates: update bor version

* internal/ethapi :: Fix : newRPCTransactionFromBlockIndex

* Merge master to qa (#813)

* Merge qa to master (#750)

* Added checks to RPC requests and introduced new flags to customise the parameters (#657)

* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)

* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)

* updated the config files and docs

* chg : trieTimeout from 60 to 10 mins (#692)

* chg : trieTimeout from 60 to 10 mins

* chg : cache.timout to 10m from 1h in configs

* internal/cli/server : fix : added triesInMemory in config (#691)

* changed version from 0.3.0 to 0.3.4-beta (#693)

* fix nil state-sync issue, increase grpc limit (#695)

* Increase grpc message size limit in pprof

* consensus/bor/bor.go : stateSyncs init fixed [Fix #686]

* eth/filters: handle nil state-sync before notify

* eth/filters: update check

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>

* core, tests/bor: add more tests for state-sync validation (#710)

* core: add get state sync function for tests

* tests/bor: add validation for state sync events post consensus

* Arpit/temp bor sync (#701)

* Increase grpc message size limit in pprof

* ReadBorReceipts improvements

* use internal function

* fix tests

* fetch geth upstread for ReadBorReceiptRLP

* Only query bor receipt when the query index is equal to # tx in block body

This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.

* Revert "fetch geth upstread for ReadBorReceiptRLP"

This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.

* Restore ReadBorReceiptRLP

* fix bor receipts

* remove unused

* fix lints

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)

This reverts commit 241843c7e7.

* Arpit/add execution pool 2 (#719)

* initial

* linters

* linters

* remove timeout

* update pool

* change pool size function

* check nil

* check nil

* fix tests

* Use execution pool from server in all handlers

* simplify things

* test fix

* add support for cli, config

* add to cli and config

* merge base branch

* debug statements

* fix bug

* atomic pointer timeout

* add apis

* update workerpool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* fix tests

* mutex

* refactor flag and value names

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* debug statements

* fix bug

* update workerpool

* atomic pointer timeout

* add apis

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* merge base branch

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* mutex

* fix tests

* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* Change default size of execution pool to 40

* refactor flag and value names

* fix merge conflicts

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* fix linters

* fix go.mod

* change sec to ms

* change default value for ep timeout

* fix node api calls

* comment setter for ep timeout

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* version change (#721)

* Event based pprof (#732)

* feature

* Save pprof to /tmp

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>

* Cherry-pick changes from develop (#738)

* Check if block is nil to prevent panic (#736)

* miner: use env for tracing instead of block object (#728)

---------

Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* add max code init size check in txpool (#739)

* Revert "Event based pprof" and update version (#742)

* Revert "Event based pprof (#732)"

This reverts commit 22fa4033e8.

* params: update version to 0.3.4-beta3

* packaging/templates: update bor version

* params, packaging/templates: update bor version

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* core, miner: add sub-spans for tracing (#753)

* core, miner: add sub-spans for tracing

* fix linters

* core: add logs for debugging

* core: add more logs to print tdd while reorg

* fix linters

* core: minor fix

* core: remove debug logs

* core: use different span for write block and set head

* core: use internal context for sending traces (#755)

* core: add : impossible reorg block dump (#754)

* add : impossible reorg block dump

* chg : 3 seperate files for impossoble reorg dump

* add : use exportBlocks method and RLP blocks before writing

* chg : small changes

* bump : go version from 1.19 to 1.20.1 (#761)

* Revert "bump : go version from 1.19 to 1.20.1 (#761)"

This reverts commit 4561012af9.

* core/vm: use optimized bigint (#26021)

* Add holiman/big

* Fix linter

* Bump version to v0.3.5

* fix lints from develop (few lints decided to appear from code that was untouched, weird)

* upgrade crypto lib version (#770)

* bump dep : github.com/Masterminds/goutils to v1.1.1 (#769)

* mardizzone/pos-1313: bump crypto dependency (#772)

* dev: chg: bumd net dependency

* dev: chg: bump crypto dependency

* dev: chg: bump crypto dependency

* bump dep : golang.org/x/net to v0.8.0 (#771)

* Verify validator set against local contract on receiving an end-of-sprint block (#768)

* Verify validator set against local contract on receiving an end-of-sprint block

* Fix tests

* Respect error returned by ParseValidators

* Keep going back until a parent block presents

* core/txpool: implement DoS defenses from geth (#778)

* Hotfixes and deps bump (#776)

* dev: chg: bump deps

* internal/cli/server, rpc: lower down http readtimeout to 10s

* dev: chg: get p2p adapter

* dev: chg: lower down jsonrpc readtimeout to 10s

* cherry-pick txpool optimisation changes

* add check for empty lists in txpool (#704)

* add check

* linters

* core, miner: add empty instrumentation name for tracing

---------

Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Evgeny Danilenko <6655321@bk.ru>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* packaging,params: bump to v0.3.6 (#782)

* v0.3.6 fix (#787)

* Fix get validator set in header verifier

* chg : commit tx logs from info to debug (#673)

* chg : commit tx logs from info to debug

* fix : minor changes

* chg : miner : commitTransactions-stats moved from info to debug

* lint : fix linters

* refactor logging

* miner : chg : UnauthorizedSignerError to debug

* lint : fix lint

* fix : log.Logger interface compatibility

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Remove unnecessary sorting of valset from header in verification

* dev: chg: version bump

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>

* core: improve locks in txpool (#807)

* added a write lock to the txs.filter method and a read lock to the txs.reheap method - both of which are called by Filter during reorg adjustments to txpool

* txpool reorg locks

* more locks

* locks

* linters

* params, packaging: update version for v0.3.8-beta release

* core: add logs in reheap

---------

Co-authored-by: Alex <dalexwatts@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Merge qa to master (#808)

* Added checks to RPC requests and introduced new flags to customise the parameters (#657)

* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)

* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)

* updated the config files and docs

* chg : trieTimeout from 60 to 10 mins (#692)

* chg : trieTimeout from 60 to 10 mins

* chg : cache.timout to 10m from 1h in configs

* internal/cli/server : fix : added triesInMemory in config (#691)

* changed version from 0.3.0 to 0.3.4-beta (#693)

* fix nil state-sync issue, increase grpc limit (#695)

* Increase grpc message size limit in pprof

* consensus/bor/bor.go : stateSyncs init fixed [Fix #686]

* eth/filters: handle nil state-sync before notify

* eth/filters: update check

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>

* core, tests/bor: add more tests for state-sync validation (#710)

* core: add get state sync function for tests

* tests/bor: add validation for state sync events post consensus

* Arpit/temp bor sync (#701)

* Increase grpc message size limit in pprof

* ReadBorReceipts improvements

* use internal function

* fix tests

* fetch geth upstread for ReadBorReceiptRLP

* Only query bor receipt when the query index is equal to # tx in block body

This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.

* Revert "fetch geth upstread for ReadBorReceiptRLP"

This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.

* Restore ReadBorReceiptRLP

* fix bor receipts

* remove unused

* fix lints

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)

This reverts commit 241843c7e7.

* Arpit/add execution pool 2 (#719)

* initial

* linters

* linters

* remove timeout

* update pool

* change pool size function

* check nil

* check nil

* fix tests

* Use execution pool from server in all handlers

* simplify things

* test fix

* add support for cli, config

* add to cli and config

* merge base branch

* debug statements

* fix bug

* atomic pointer timeout

* add apis

* update workerpool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* fix tests

* mutex

* refactor flag and value names

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* debug statements

* fix bug

* update workerpool

* atomic pointer timeout

* add apis

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* merge base branch

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* mutex

* fix tests

* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* Change default size of execution pool to 40

* refactor flag and value names

* fix merge conflicts

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* fix linters

* fix go.mod

* change sec to ms

* change default value for ep timeout

* fix node api calls

* comment setter for ep timeout

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* version change (#721)

* Event based pprof (#732)

* feature

* Save pprof to /tmp

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>

* Cherry-pick changes from develop (#738)

* Check if block is nil to prevent panic (#736)

* miner: use env for tracing instead of block object (#728)

---------

Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* add max code init size check in txpool (#739)

* Revert "Event based pprof" and update version (#742)

* Revert "Event based pprof (#732)"

This reverts commit 22fa4033e8.

* params: update version to 0.3.4-beta3

* packaging/templates: update bor version

* internal/ethapi :: Fix : newRPCTransactionFromBlockIndex

* fix: remove assignment for bor receipt

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* Setting up bor to use hosted 18.04 runner as ubuntu provided 18.04 runner is end of life

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>
Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com>
Co-authored-by: Alex <dalexwatts@gmail.com>
Co-authored-by: Daniel Jones <djones@polygon.technology>

* merge v0.3.9-alpha to qa (#824)

* commit logs

* CI: test launch devnet without hardcoded sleep time (pos-534)

* CI: try using checked out bor path

* CI: fix missing ;

* CI: fix assignment operator

* CI: echo peers and block no.

* CI: cleanup

* minor chg: add new line

* dev: add: pos-944: snyk and govuln integration (#578)

* dev: add: pos-944 security ci and readme

* dev: add: pos-944 remove linters as this is included already in build ci

* dev: chg: pos-947 dependencies upgrade to solve snyk security issues

* dev: chg: update security-ci

* dev: chg: remove linter to allow replacements for security issues

* dev: add: pos-944 verify path when updating metrics from config

* dev: add: pos-944 fix linter

* dev: add: pos-944 add .snyk policy file / fix snyk code vulnerabilities

* dev: fix: pos-944 import common package / gitignore snyk dccache file

* dev: fix: pos-944 verify canonical path for crashers

* dev: fix: pos-944 linter

* dev: add: pos-976 add govuln check

* dev: add: pos-976 test upload with permissions

* dev: add: pos-976 remove duplicated upload

* dev: add: pos-976 report upload

* dev: add: pos-976 remove upload

* dev: fix: pos-944 fix govuln action

* dev: fix: pos-944 move govulncheck to security-ci

* dev: fix: pos-944 bump golvun action and golang versions

* dev: fix: pos-944 remove persmissions and fix conflicts

* dev: chg: restore err msg

* dev: chg: remove duplicated function

* dev: chg: sort import

* dev: chg: fix linter

* dev: add: use common VerifyCrasher function to avoid duplications / replace deprecated ioutils.ReadFile

* dev: fix: typo

* fix linters

* upgrade grpc version

* add ignore rule for net/http2

* Shivam/txpool tracing (#604)

* lock, unlock to rlock, runlock

* add : tracing Pending() and Locals()

* Log time spent in committing a tx during mining

* Remove data from logging

* Move log into case where a tx completes without error

* profile fillTransactions

* fix conflict

* bug fixes

* add logs

* txpool: add tracing in Pending()

* rearrange tracing

* add attributes

* fix

* fix

* log error in profiling

* update file mode and file path for profiling

* full profiling

* fix log

* fix log

* less wait

* fix

* fix

* logs

* worker: use block number for prof files

* initial

* txList add

* fix gas calculation

* fix

* green tests

* linters

* prettify

* allocate less

* no locks between pending and reorg

* no locks

* no locks on locals

* more tests

* linters

* less allocs

* comment

* optimize errors

* linters

* fix

* fix

* Linters

* linters

* linters

* simplify errors

* atomics for transactions

* fix

* optimize workers

* fix copy

* linters

* txpool tracing

* linters

* fix tracing

* duration in mcs

* locks

* metrics

* fix

* cache hit/miss

* less locks on evict

* remove once

* remove once

* switch off pprof

* fix data race

* fix data race

* add : sealed total/empty blocks metric gauge

* add : RPC debug_getTraceStack

* fix : RPC debug_getTraceStack

* fix : RPC debug_getTraceStack for all go-routines

* linters

* add data race test on txpool

* fix concurrency

* noleak

* increase batch size

* prettify

* tests

* baseFee mutex

* panic fix

* linters

* fix gas fee data race

* linters

* more transactions

* debug

* debug

* fix ticker

* fix test

* add cacheMu

* more tests

* fix test panic

* linters

* add statistics

* add statistics

* txitems data race

* fix tx list Has

* fix : lint

Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Reduce txArriveTimeout to 100ms

* init : remove exit on keystore err

* add : multiple keystore tolerance

* lint : fix linters

* chg : use standard logging

* chg : logging strings

* Added flags to run heimdall as a child process (#597)

* Added flags to run heimdall as a child process

* Fix: Lint

* Fix btcd package dependency for CI

* Update btcd package version

* Try removing ambigious importts

* dev: fix: lint fix for parallel tests

* remove delete for ambigious import

* Remove unwanted space

* go mod tidy

* try replace

* try replace

* use vendor

* rename vendor

* tidy

* vendor btcec

* clean up

* remove submodule

* remove submodule

* remove submodule

* remove submodule

* remove vendor & added replacer in test

* go mod tidy

* added replacer

* Update replacer

* Update replacer

* Merge branch 'develop' into run-heimdall-flags

* Skip TestGolangBindings

* Typo fix

* Remove unwanted changes

Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* dev: chg: update PR template to include nodes audience check (#641)

* dev: chg: update PR template to include nodes audience check

* dev: chg: better description

* dev: chg: add entry to changes too

* sonarqube integration (#658)

* dev: add: sonarqube integration into security-ci

* dev: add: exclude java files from sonarqube analysis

* Merge branch 'qa' and 'master' into develop (#663)

* Adding in Mumbai/Mainnet precursor deb packaging for tests to use during upgrade(iterations to come)

* Added changes per discussion in PR, more changes may be necessary

* Adding prerelease true

* Disabling goreleaser

* Removing README swap file

* change bor_dir and add bor user for v0.3.0 release

* rollback bor user and use root

* metrics: handle equal to separated config flag (#596)

* metrics: handle  based config path

* internal/cli/server: add more context to logs

* use space separated flag and value in bor.service

* fixed static-nodes related buf (os independent) (#598)

* fixed static-nodes related buf (os independent)

* taking static-nodes as input if default not present

* Update default flags (#600)

* internal/cli/server: use geth's default for txpool.pricelimit and add comments

* builder/files: update config.toml for mainnet

* packaging/templates: update defaults for mainnet and mumbai

* internal/cli/server: skip overriding cache

* packaging/templates: update cache value for mainnet

* packaging/templates: update gcmode for archive mumbai node

* metrics: handle nil telemetry config (#601)

* resolve merge conflicts

* update go version in release.yml

* update goversion in makefile

* update Docker login for goreleaser-cross v1.19

* Cleanup for the packager to use git tag in the package profile naming. Added conditional check for directory structure, this is in prep for v0.3.1, as this will create a failure on upgrade path in package due to file exist

* added a toml configuration file with comments describing each flag (#607)

* added a toml configuration file with comments describing each flag

* internal/cli/server: update flag description

* docs/cli: update example config and description of flags

* docs: update new-cli docs

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* Adding of 0.3.0 package changes, control file updates, postinst changes, and packager update

* added ancient datadir flag and toml field, need to decide on default value and update the conversion script

* updated toml files with ancient field

* Add support for new flags in new config.toml, which were present in old config.toml (#612)

* added HTTPTimeouts, and TrieTimeout flag in new tol, from old toml

* added RAW fields for these time.Duration flags

* updated the conversion script to support these extra 4 flags

* removed hcl and json config tests as we are only supporting toml config files

* updated toml files with cache.timeout field

* updated toml files with jsonrpc.timeouts field

* tests/bor: expect a call for latest checkpoint

* tests/bor: expect a call for latest checkpoint

* packaging/templates: update cache values for archive nodes

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* remove unwanted code

* Fix docker publish authentication issue

In gorelease-cross 1.19+, dockerhub authentication will require docker
logion action followed by mounting docker config file. See
https://github.com/goreleaser/goreleaser-cross#github-actions.

* Revert "update Docker login for goreleaser-cross v1.19"

This reverts commit 4d19cf5342.

* Bump version to stable

* Revert "Merge pull request #435 from maticnetwork/POS-553"

This reverts commit 657d262def, reversing
changes made to 88dbfa1c13.

* revert change for release for go1.19

* Add default values to CLI helper and docs

This commit adds default values to CLI helper and docs. When the default value of a string flag, slice string flag, or map string flag is empty, its helper message won't show any default value.

* Add a summary of new CLI in docs

* Updating packager as binutils changed version so that apt-get installs current versions

* Add state pruning to new CLI

* Minor wording fix in prune state description

* Bumping control file versions

* Mainnet Delhi fork

* Set version to stable

* change delhi hardfork block number

* handle future chain import and skip peer drop (#650)

* handle future chain import and skip peer drop

* add block import metric

* params: bump version to v0.3.3-stable

* Bump bor version in control files for v0.3.3 mainnet release

Co-authored-by: Daniel Jones <djones@polygon.technology>
Co-authored-by: Will Button <wbutton@polygon.technology>
Co-authored-by: Will Button <will@willbutton.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>

* CI: use matic-cli master branch

* trigger ci

* internal/cli/server : fix : added triesInMemory in config (#677)

* update requirements in README (#681)

* consensus/bor : add : devFakeAuthor flag

* core,eth,internal/cli,internal/ethapi: add --rpc.allow-unprotected-txs flag to allow txs to get replayed (for shadow node)

* internal/cli: add `skiptrace` flag for profiling (#715)

* internal/cli: add skiptrace flag

* docs: update docs for skiptrace flag

* Added flag in Status to wait for backend, and fixed panic issue. (#708)

* checking if backend is available during status call, and added a flag to wait if backend is not available

* added test for status command (does not cover the whole code)

* Revert "Reduce txArriveTimeout to 100ms" (#707)

This reverts commit 243d231fe4.

* consensus/bor : add : devFakeAuthor flag (#697)

* add check for empty lists in txpool (#704)

* add check

* linters

* dev: chg: POS-215 move sonarqube to own ci (#733)

* Added verbosity flag, supports log-level as well, but will remove that in future. (#722)

* changed log-level flag back to verbosity, and updated the conversion script

* supporting both verbosity and log-level, with a message to deprecat log-level

* converted verbosity to a int value

* Check if block is nil to prevent panic (#736)

* miner: use env for tracing instead of block object (#728)

* Add : mutex pprof profile (#731)

* add : mutex pprof profile

* rm : remove trace from default pprof profiles

* chg : commit tx logs from info to debug (#673)

* chg : commit tx logs from info to debug

* fix : minor changes

* chg : miner : commitTransactions-stats moved from info to debug

* lint : fix linters

* refactor logging

* miner : chg : UnauthorizedSignerError to debug

* lint : fix lint

* fix : log.Logger interface compatibility

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Add : commit details to bor version (#730)

* add : commit details to bor version

* fix : MAKEFILE

* undo : rm test-txpool-race

* rm : params/build_date

* rm : params/gitbranch and params/gitdate

* core,docs/cli,internal/cli/server: make docs

* builder,docs/cli,packaging: update toml files

* mardizzone/hotfix-snyk: remove vcs build when running snyk (#745)

* Feat : SetMaxPeers (#726)

* init : add admin.setMaxPeers and getMaxPeers

* lint : fix lint

* add : some comments

* Update wiki link (#762)

* Heimdall App implementation (#646)

* added support for miner.recommit flag (#743)

* interrupting commit experiment (#556)

* initial

* delete

* linters

* big benchmark

* benchmark big ints

* delay

* fix generate

* remove debug

* miner : chg : remove noempty check

* fix lints

* consensus/bor: handle unauthorized signer in consensus.Prepare (#651)

* fix : break loop fix

* lint : fix

* lint : more lint fix

* fix : skip TestEmptyWorkEthash and TestEmptyWorkClique

* add : metrics commitInterruptCounter

---------

Co-authored-by: Shivam Sharma <shivam691999@gmail.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* internal/cli: added missing flags (#744)

* minor comment update

* added support for rpc.evmtimeout flag

* added support for vmdebug (EnablePreimageRecording) flag

* added support for jsonrpc.auth.(jwtsecret, addr, port, vhosts) flags

* added support for miner.recommit flag

* added support for gpo.maxheaderhistory and gpo.maxblockhistory flag

* Revert "added support for miner.recommit flag"

This reverts commit fcb722e4bf5bdda23d4c91da40ab95c9023a80c3.

* added pprof related flags (expect --pprof.cpuprofile - Write CPU profile to the given file)

* added support for --dev.gaslimit flag

* added support for --fdlimit flag

* added support for --netrestrict flag

* added support for --nodekey and --nodekeyhex flag

* added support for --vmodule, --log.json, --log.backtrace, and --log.debug flags

* fixed related lint errors

* fix lints from develop (few lints decided to appear from code that was untouched, weird)

* more weird lints from develop

* small precautionary fix

* small bug ;) fix in NetRestrict

* weird lints

* change vmdebug = true to vmdebug = false

* internal/ethapi: set rpc gas cap as default gas limit when creating access list

* add : testcase for CommitInterruptExperiment (#792)

* add : TestCommitInterruptExperimentBor

* lint : fix lint

* chg : add waitgroup

* dev: chg: continue on error when uploading snyk results to GH (#795)

* consensus/bor: revert handle unauthorized signer in consensus.Prepare

* Revert "consensus/bor: handle unauthorized signer in consensus.Prepare (#651)"

This reverts commit 9ce8c7de75a1b5021bb6f2b8086e813c8244b22c.

* eth/downloader/whitelist: skip future chain validation (#796)

* eth/downloader/whitelist: skip future chain validation

* eth/downloader/whitelist: fix tests for future chain import

* Setting up bor to use hosted 18.04 runner as ubuntu provided 18.04 runner is end of life

* Merge qa to develop (#814)

* Adding in Mumbai/Mainnet precursor deb packaging for tests to use during upgrade(iterations to come)

* Added changes per discussion in PR, more changes may be necessary

* Adding prerelease true

* Disabling goreleaser

* Removing README swap file

* change bor_dir and add bor user for v0.3.0 release

* rollback bor user and use root

* metrics: handle equal to separated config flag (#596)

* metrics: handle  based config path

* internal/cli/server: add more context to logs

* use space separated flag and value in bor.service

* fixed static-nodes related buf (os independent) (#598)

* fixed static-nodes related buf (os independent)

* taking static-nodes as input if default not present

* Update default flags (#600)

* internal/cli/server: use geth's default for txpool.pricelimit and add comments

* builder/files: update config.toml for mainnet

* packaging/templates: update defaults for mainnet and mumbai

* internal/cli/server: skip overriding cache

* packaging/templates: update cache value for mainnet

* packaging/templates: update gcmode for archive mumbai node

* metrics: handle nil telemetry config (#601)

* resolve merge conflicts

* update go version in release.yml

* update goversion in makefile

* update Docker login for goreleaser-cross v1.19

* Cleanup for the packager to use git tag in the package profile naming. Added conditional check for directory structure, this is in prep for v0.3.1, as this will create a failure on upgrade path in package due to file exist

* added a toml configuration file with comments describing each flag (#607)

* added a toml configuration file with comments describing each flag

* internal/cli/server: update flag description

* docs/cli: update example config and description of flags

* docs: update new-cli docs

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* Adding of 0.3.0 package changes, control file updates, postinst changes, and packager update

* added ancient datadir flag and toml field, need to decide on default value and update the conversion script

* updated toml files with ancient field

* Add support for new flags in new config.toml, which were present in old config.toml (#612)

* added HTTPTimeouts, and TrieTimeout flag in new tol, from old toml

* added RAW fields for these time.Duration flags

* updated the conversion script to support these extra 4 flags

* removed hcl and json config tests as we are only supporting toml config files

* updated toml files with cache.timeout field

* updated toml files with jsonrpc.timeouts field

* tests/bor: expect a call for latest checkpoint

* tests/bor: expect a call for latest checkpoint

* packaging/templates: update cache values for archive nodes

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* remove unwanted code

* Fix docker publish authentication issue

In gorelease-cross 1.19+, dockerhub authentication will require docker
logion action followed by mounting docker config file. See
https://github.com/goreleaser/goreleaser-cross#github-actions.

* Revert "update Docker login for goreleaser-cross v1.19"

This reverts commit 4d19cf5342.

* Bump version to stable

* Revert "Merge pull request #435 from maticnetwork/POS-553"

This reverts commit 657d262def, reversing
changes made to 88dbfa1c13.

* revert change for release for go1.19

* Add default values to CLI helper and docs

This commit adds default values to CLI helper and docs. When the default value of a string flag, slice string flag, or map string flag is empty, its helper message won't show any default value.

* Add a summary of new CLI in docs

* Updating packager as binutils changed version so that apt-get installs current versions

* Add state pruning to new CLI

* Minor wording fix in prune state description

* Bumping control file versions

* Mainnet Delhi fork

* Set version to stable

* change delhi hardfork block number

* handle future chain import and skip peer drop (#650)

* handle future chain import and skip peer drop

* add block import metric

* params: bump version to v0.3.3-stable

* Bump bor version in control files for v0.3.3 mainnet release

* Added checks to RPC requests and introduced new flags to customise the parameters (#657)

* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)

* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)

* updated the config files and docs

* chg : trieTimeout from 60 to 10 mins (#692)

* chg : trieTimeout from 60 to 10 mins

* chg : cache.timout to 10m from 1h in configs

* internal/cli/server : fix : added triesInMemory in config (#691)

* changed version from 0.3.0 to 0.3.4-beta (#693)

* fix nil state-sync issue, increase grpc limit (#695)

* Increase grpc message size limit in pprof

* consensus/bor/bor.go : stateSyncs init fixed [Fix #686]

* eth/filters: handle nil state-sync before notify

* eth/filters: update check

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>

* core, tests/bor: add more tests for state-sync validation (#710)

* core: add get state sync function for tests

* tests/bor: add validation for state sync events post consensus

* Arpit/temp bor sync (#701)

* Increase grpc message size limit in pprof

* ReadBorReceipts improvements

* use internal function

* fix tests

* fetch geth upstread for ReadBorReceiptRLP

* Only query bor receipt when the query index is equal to # tx in block body

This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.

* Revert "fetch geth upstread for ReadBorReceiptRLP"

This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.

* Restore ReadBorReceiptRLP

* fix bor receipts

* remove unused

* fix lints

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)

This reverts commit 241843c7e7.

* Arpit/add execution pool 2 (#719)

* initial

* linters

* linters

* remove timeout

* update pool

* change pool size function

* check nil

* check nil

* fix tests

* Use execution pool from server in all handlers

* simplify things

* test fix

* add support for cli, config

* add to cli and config

* merge base branch

* debug statements

* fix bug

* atomic pointer timeout

* add apis

* update workerpool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* fix tests

* mutex

* refactor flag and value names

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* debug statements

* fix bug

* update workerpool

* atomic pointer timeout

* add apis

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* merge base branch

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* mutex

* fix tests

* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* Change default size of execution pool to 40

* refactor flag and value names

* fix merge conflicts

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* fix linters

* fix go.mod

* change sec to ms

* change default value for ep timeout

* fix node api calls

* comment setter for ep timeout

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* version change (#721)

* Event based pprof (#732)

* feature

* Save pprof to /tmp

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>

* Cherry-pick changes from develop (#738)

* Check if block is nil to prevent panic (#736)

* miner: use env for tracing instead of block object (#728)

---------

Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* add max code init size check in txpool (#739)

* Revert "Event based pprof" and update version (#742)

* Revert "Event based pprof (#732)"

This reverts commit 22fa4033e8.

* params: update version to 0.3.4-beta3

* packaging/templates: update bor version

* internal/ethapi :: Fix : newRPCTransactionFromBlockIndex

* Merge master to qa (#813)

* Merge qa to master (#750)

* Added checks to RPC requests and introduced new flags to customise the parameters (#657)

* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)

* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)

* updated the config files and docs

* chg : trieTimeout from 60 to 10 mins (#692)

* chg : trieTimeout from 60 to 10 mins

* chg : cache.timout to 10m from 1h in configs

* internal/cli/server : fix : added triesInMemory in config (#691)

* changed version from 0.3.0 to 0.3.4-beta (#693)

* fix nil state-sync issue, increase grpc limit (#695)

* Increase grpc message size limit in pprof

* consensus/bor/bor.go : stateSyncs init fixed [Fix #686]

* eth/filters: handle nil state-sync before notify

* eth/filters: update check

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>

* core, tests/bor: add more tests for state-sync validation (#710)

* core: add get state sync function for tests

* tests/bor: add validation for state sync events post consensus

* Arpit/temp bor sync (#701)

* Increase grpc message size limit in pprof

* ReadBorReceipts improvements

* use internal function

* fix tests

* fetch geth upstread for ReadBorReceiptRLP

* Only query bor receipt when the query index is equal to # tx in block body

This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.

* Revert "fetch geth upstread for ReadBorReceiptRLP"

This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.

* Restore ReadBorReceiptRLP

* fix bor receipts

* remove unused

* fix lints

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)

This reverts commit 241843c7e7.

* Arpit/add execution pool 2 (#719)

* initial

* linters

* linters

* remove timeout

* update pool

* change pool size function

* check nil

* check nil

* fix tests

* Use execution pool from server in all handlers

* simplify things

* test fix

* add support for cli, config

* add to cli and config

* merge base branch

* debug statements

* fix bug

* atomic pointer timeout

* add apis

* update workerpool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* fix tests

* mutex

* refactor flag and value names

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* debug statements

* fix bug

* update workerpool

* atomic pointer timeout

* add apis

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* merge base branch

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* mutex

* fix tests

* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* Change default size of execution pool to 40

* refactor flag and value names

* fix merge conflicts

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* fix linters

* fix go.mod

* change sec to ms

* change default value for ep timeout

* fix node api calls

* comment setter for ep timeout

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* version change (#721)

* Event based pprof (#732)

* feature

* Save pprof to /tmp

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>

* Cherry-pick changes from develop (#738)

* Check if block is nil to prevent panic (#736)

* miner: use env for tracing instead of block object (#728)

---------

Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* add max code init size check in txpool (#739)

* Revert "Event based pprof" and update version (#742)

* Revert "Event based pprof (#732)"

This reverts commit 22fa4033e8.

* params: update version to 0.3.4-beta3

* packaging/templates: update bor version

* params, packaging/templates: update bor version

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* core, miner: add sub-spans for tracing (#753)

* core, miner: add sub-spans for tracing

* fix linters

* core: add logs for debugging

* core: add more logs to print tdd while reorg

* fix linters

* core: minor fix

* core: remove debug logs

* core: use different span for write block and set head

* core: use internal context for sending traces (#755)

* core: add : impossible reorg block dump (#754)

* add : impossible reorg block dump

* chg : 3 seperate files for impossoble reorg dump

* add : use exportBlocks method and RLP blocks before writing

* chg : small changes

* bump : go version from 1.19 to 1.20.1 (#761)

* Revert "bump : go version from 1.19 to 1.20.1 (#761)"

This reverts commit 4561012af9.

* core/vm: use optimized bigint (#26021)

* Add holiman/big

* Fix linter

* Bump version to v0.3.5

* fix lints from develop (few lints decided to appear from code that was untouched, weird)

* upgrade crypto lib version (#770)

* bump dep : github.com/Masterminds/goutils to v1.1.1 (#769)

* mardizzone/pos-1313: bump crypto dependency (#772)

* dev: chg: bumd net dependency

* dev: chg: bump crypto dependency

* dev: chg: bump crypto dependency

* bump dep : golang.org/x/net to v0.8.0 (#771)

* Verify validator set against local contract on receiving an end-of-sprint block (#768)

* Verify validator set against local contract on receiving an end-of-sprint block

* Fix tests

* Respect error returned by ParseValidators

* Keep going back until a parent block presents

* core/txpool: implement DoS defenses from geth (#778)

* Hotfixes and deps bump (#776)

* dev: chg: bump deps

* internal/cli/server, rpc: lower down http readtimeout to 10s

* dev: chg: get p2p adapter

* dev: chg: lower down jsonrpc readtimeout to 10s

* cherry-pick txpool optimisation changes

* add check for empty lists in txpool (#704)

* add check

* linters

* core, miner: add empty instrumentation name for tracing

---------

Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Evgeny Danilenko <6655321@bk.ru>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* packaging,params: bump to v0.3.6 (#782)

* v0.3.6 fix (#787)

* Fix get validator set in header verifier

* chg : commit tx logs from info to debug (#673)

* chg : commit tx logs from info to debug

* fix : minor changes

* chg : miner : commitTransactions-stats moved from info to debug

* lint : fix linters

* refactor logging

* miner : chg : UnauthorizedSignerError to debug

* lint : fix lint

* fix : log.Logger interface compatibility

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Remove unnecessary sorting of valset from header in verification

* dev: chg: version bump

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>

* core: improve locks in txpool (#807)

* added a write lock to the txs.filter method and a read lock to the txs.reheap method - both of which are called by Filter during reorg adjustments to txpool

* txpool reorg locks

* more locks

* locks

* linters

* params, packaging: update version for v0.3.8-beta release

* core: add logs in reheap

---------

Co-authored-by: Alex <dalexwatts@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Merge qa to master (#808)

* Added checks to RPC requests and introduced new flags to customise the parameters (#657)

* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)

* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)

* updated the config files and docs

* chg : trieTimeout from 60 to 10 mins (#692)

* chg : trieTimeout from 60 to 10 mins

* chg : cache.timout to 10m from 1h in configs

* internal/cli/server : fix : added triesInMemory in config (#691)

* changed version from 0.3.0 to 0.3.4-beta (#693)

* fix nil state-sync issue, increase grpc limit (#695)

* Increase grpc message size limit in pprof

* consensus/bor/bor.go : stateSyncs init fixed [Fix #686]

* eth/filters: handle nil state-sync before notify

* eth/filters: update check

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>

* core, tests/bor: add more tests for state-sync validation (#710)

* core: add get state sync function for tests

* tests/bor: add validation for state sync events post consensus

* Arpit/temp bor sync (#701)

* Increase grpc message size limit in pprof

* ReadBorReceipts improvements

* use internal function

* fix tests

* fetch geth upstread for ReadBorReceiptRLP

* Only query bor receipt when the query index is equal to # tx in block body

This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.

* Revert "fetch geth upstread for ReadBorReceiptRLP"

This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.

* Restore ReadBorReceiptRLP

* fix bor receipts

* remove unused

* fix lints

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)

This reverts commit 241843c7e7.

* Arpit/add execution pool 2 (#719)

* initial

* linters

* linters

* remove timeout

* update pool

* change pool size function

* check nil

* check nil

* fix tests

* Use execution pool from server in all handlers

* simplify things

* test fix

* add support for cli, config

* add to cli and config

* merge base branch

* debug statements

* fix bug

* atomic pointer timeout

* add apis

* update workerpool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* fix tests

* mutex

* refactor flag and value names

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* debug statements

* fix bug

* update workerpool

* atomic pointer timeout

* add apis

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* merge base branch

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* mutex

* fix tests

* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* Change default size of execution pool to 40

* refactor flag and value names

* fix merge conflicts

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* fix linters

* fix go.mod

* change sec to ms

* change default value for ep timeout

* fix node api calls

* comment setter for ep timeout

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* version change (#721)

* Event based pprof (#732)

* feature

* Save pprof to /tmp

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>

* Cherry-pick changes from develop (#738)

* Check if block is nil to prevent panic (#736)

* miner: use env for tracing instead of block object (#728)

---------

Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* add max code init size check in txpool (#739)

* Revert "Event based pprof" and update version (#742)

* Revert "Event based pprof (#732)"

This reverts commit 22fa4033e8.

* params: update version to 0.3.4-beta3

* packaging/templates: update bor version

* internal/ethapi :: Fix : newRPCTransactionFromBlockIndex

* fix: remove assignment for bor receipt

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* Setting up bor to use hosted 18.04 runner as ubuntu provided 18.04 runner is end of life

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>
Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com>
Co-authored-by: Alex <dalexwatts@gmail.com>
Co-authored-by: Daniel Jones <djones@polygon.technology>

* core: remove duplicate tests

* miner: use get validators by hash in tests

---------

Co-authored-by: Daniel Jones <djones@polygon.technology>
Co-authored-by: Will Button <wbutton@polygon.technology>
Co-authored-by: Will Button <will@willbutton.com>
Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>
Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com>
Co-authored-by: Alex <dalexwatts@gmail.com>

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: builder90210 <anonbuilder90210@gmail.com>
Co-authored-by: Krishna Upadhyaya <KRISHNAU1604@GMAIL.COM>
Co-authored-by: Daniel Jones <djones@polygon.technology>
Co-authored-by: Will Button <wbutton@polygon.technology>
Co-authored-by: Will Button <will@willbutton.com>
Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com>
Co-authored-by: Didi <github@d10r.net>
Co-authored-by: ephess <e+github@ephess.nz>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Alex <dalexwatts@gmail.com>

* dev: chg: version bump for v0.3.9-beta (#825)

* eth/fetcher: if peers never respond, drop them (#837) (#844)

* dev: chg: bump version to beta-2 for v0.3.9 (#845)

* rm : disable interruptCommit on txsCh select case (#848)

* dev: chg: version bump (#849)

* fix : go-routine leak in commitInterrupt channel (#851)

* fix : go-routine leak in commitInterrupt channel

* fix : fix lint

* rm : remove t.Parallel() from TestGenerateBlockAndImport tests

* fix : minor optimisations in test

* fix : BenchmarkBorMining

* dev: chg: beta version bump (#853)

* dev: chg: bump version to stable

* dev: chg: contracts branch

* dev: fix: linter

* dev: chg: update templum/govulncheck-action to fix execution

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com>
Co-authored-by: Alex <dalexwatts@gmail.com>
Co-authored-by: Daniel Jones <djones@polygon.technology>
Co-authored-by: builder90210 <anonbuilder90210@gmail.com>
Co-authored-by: Krishna Upadhyaya <KRISHNAU1604@GMAIL.COM>
Co-authored-by: Will Button <wbutton@polygon.technology>
Co-authored-by: Will Button <will@willbutton.com>
Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com>
Co-authored-by: Didi <github@d10r.net>
Co-authored-by: ephess <e+github@ephess.nz>
2023-05-10 22:52:14 +02:00

1477 lines
46 KiB
Go

package server
import (
"crypto/ecdsa"
"fmt"
"io/ioutil"
"math"
"math/big"
"os"
"path/filepath"
"runtime"
"strconv"
"strings"
"time"
godebug "runtime/debug"
"github.com/hashicorp/hcl/v2/hclsimple"
"github.com/imdario/mergo"
"github.com/mitchellh/go-homedir"
gopsutil "github.com/shirou/gopsutil/mem"
"github.com/ethereum/go-ethereum/accounts"
"github.com/ethereum/go-ethereum/accounts/keystore"
"github.com/ethereum/go-ethereum/cmd/utils"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/fdlimit"
"github.com/ethereum/go-ethereum/crypto"
"github.com/ethereum/go-ethereum/eth/downloader"
"github.com/ethereum/go-ethereum/eth/ethconfig"
"github.com/ethereum/go-ethereum/eth/gasprice"
"github.com/ethereum/go-ethereum/internal/cli/server/chains"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/node"
"github.com/ethereum/go-ethereum/p2p"
"github.com/ethereum/go-ethereum/p2p/enode"
"github.com/ethereum/go-ethereum/p2p/nat"
"github.com/ethereum/go-ethereum/p2p/netutil"
"github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/rpc"
)
type Config struct {
chain *chains.Chain
// Chain is the chain to sync with
Chain string `hcl:"chain,optional" toml:"chain,optional"`
// Identity of the node
Identity string `hcl:"identity,optional" toml:"identity,optional"`
// RequiredBlocks is a list of required (block number, hash) pairs to accept
RequiredBlocks map[string]string `hcl:"eth.requiredblocks,optional" toml:"eth.requiredblocks,optional"`
// Verbosity is the level of the logs to put out
Verbosity int `hcl:"verbosity,optional" toml:"verbosity,optional"`
// LogLevel is the level of the logs to put out
LogLevel string `hcl:"log-level,optional" toml:"log-level,optional"`
// Record information useful for VM and contract debugging
EnablePreimageRecording bool `hcl:"vmdebug,optional" toml:"vmdebug,optional"`
// DataDir is the directory to store the state in
DataDir string `hcl:"datadir,optional" toml:"datadir,optional"`
// Ancient is the directory to store the state in
Ancient string `hcl:"ancient,optional" toml:"ancient,optional"`
// KeyStoreDir is the directory to store keystores
KeyStoreDir string `hcl:"keystore,optional" toml:"keystore,optional"`
// Maximum number of messages in a batch (default=100, use 0 for no limits)
RPCBatchLimit uint64 `hcl:"rpc.batchlimit,optional" toml:"rpc.batchlimit,optional"`
// Maximum size (in bytes) a result of an rpc request could have (default=100000, use 0 for no limits)
RPCReturnDataLimit uint64 `hcl:"rpc.returndatalimit,optional" toml:"rpc.returndatalimit,optional"`
// SyncMode selects the sync protocol
SyncMode string `hcl:"syncmode,optional" toml:"syncmode,optional"`
// GcMode selects the garbage collection mode for the trie
GcMode string `hcl:"gcmode,optional" toml:"gcmode,optional"`
// Snapshot enables the snapshot database mode
Snapshot bool `hcl:"snapshot,optional" toml:"snapshot,optional"`
// BorLogs enables bor log retrieval
BorLogs bool `hcl:"bor.logs,optional" toml:"bor.logs,optional"`
// Ethstats is the address of the ethstats server to send telemetry
Ethstats string `hcl:"ethstats,optional" toml:"ethstats,optional"`
// Logging has the logging related settings
Logging *LoggingConfig `hcl:"log,block" toml:"log,block"`
// P2P has the p2p network related settings
P2P *P2PConfig `hcl:"p2p,block" toml:"p2p,block"`
// Heimdall has the heimdall connection related settings
Heimdall *HeimdallConfig `hcl:"heimdall,block" toml:"heimdall,block"`
// TxPool has the transaction pool related settings
TxPool *TxPoolConfig `hcl:"txpool,block" toml:"txpool,block"`
// Sealer has the validator related settings
Sealer *SealerConfig `hcl:"miner,block" toml:"miner,block"`
// JsonRPC has the json-rpc related settings
JsonRPC *JsonRPCConfig `hcl:"jsonrpc,block" toml:"jsonrpc,block"`
// Gpo has the gas price oracle related settings
Gpo *GpoConfig `hcl:"gpo,block" toml:"gpo,block"`
// Telemetry has the telemetry related settings
Telemetry *TelemetryConfig `hcl:"telemetry,block" toml:"telemetry,block"`
// Cache has the cache related settings
Cache *CacheConfig `hcl:"cache,block" toml:"cache,block"`
// Account has the validator account related settings
Accounts *AccountsConfig `hcl:"accounts,block" toml:"accounts,block"`
// GRPC has the grpc server related settings
GRPC *GRPCConfig `hcl:"grpc,block" toml:"grpc,block"`
// Developer has the developer mode related settings
Developer *DeveloperConfig `hcl:"developer,block" toml:"developer,block"`
// Develop Fake Author mode to produce blocks without authorisation
DevFakeAuthor bool `hcl:"devfakeauthor,optional" toml:"devfakeauthor,optional"`
// Pprof has the pprof related settings
Pprof *PprofConfig `hcl:"pprof,block" toml:"pprof,block"`
}
type LoggingConfig struct {
// Per-module verbosity: comma-separated list of <pattern>=<level> (e.g. eth/*=5,p2p=4)
Vmodule string `hcl:"vmodule,optional" toml:"vmodule,optional"`
// Format logs with JSON
Json bool `hcl:"json,optional" toml:"json,optional"`
// Request a stack trace at a specific logging statement (e.g. "block.go:271")
Backtrace string `hcl:"backtrace,optional" toml:"backtrace,optional"`
// Prepends log messages with call-site location (file and line number)
Debug bool `hcl:"debug,optional" toml:"debug,optional"`
// TODO - implement this
// // Write execution trace to the given file
// Trace string `hcl:"trace,optional" toml:"trace,optional"`
}
type PprofConfig struct {
// Enableed enable the pprof HTTP server
Enabled bool `hcl:"pprof,optional" toml:"pprof,optional"`
// pprof HTTP server listening port
Port int `hcl:"port,optional" toml:"port,optional"`
// pprof HTTP server listening interface
Addr string `hcl:"addr,optional" toml:"addr,optional"`
// Turn on memory profiling with the given rate
MemProfileRate int `hcl:"memprofilerate,optional" toml:"memprofilerate,optional"`
// Turn on block profiling with the given rate
BlockProfileRate int `hcl:"blockprofilerate,optional" toml:"blockprofilerate,optional"`
// // Write CPU profile to the given file
// CPUProfile string `hcl:"cpuprofile,optional" toml:"cpuprofile,optional"`
}
type P2PConfig struct {
// MaxPeers sets the maximum number of connected peers
MaxPeers uint64 `hcl:"maxpeers,optional" toml:"maxpeers,optional"`
// MaxPendPeers sets the maximum number of pending connected peers
MaxPendPeers uint64 `hcl:"maxpendpeers,optional" toml:"maxpendpeers,optional"`
// Bind is the bind address
Bind string `hcl:"bind,optional" toml:"bind,optional"`
// Port is the port number
Port uint64 `hcl:"port,optional" toml:"port,optional"`
// NoDiscover is used to disable discovery
NoDiscover bool `hcl:"nodiscover,optional" toml:"nodiscover,optional"`
// NAT it used to set NAT options
NAT string `hcl:"nat,optional" toml:"nat,optional"`
// Connectivity can be restricted to certain IP networks.
// If this option is set to a non-nil value, only hosts which match one of the
// IP networks contained in the list are considered.
NetRestrict string `hcl:"netrestrict,optional" toml:"netrestrict,optional"`
// P2P node key file
NodeKey string `hcl:"nodekey,optional" toml:"nodekey,optional"`
// P2P node key as hex
NodeKeyHex string `hcl:"nodekeyhex,optional" toml:"nodekeyhex,optional"`
// Discovery has the p2p discovery related settings
Discovery *P2PDiscovery `hcl:"discovery,block" toml:"discovery,block"`
}
type P2PDiscovery struct {
// V5Enabled is used to enable disc v5 discovery mode
V5Enabled bool `hcl:"v5disc,optional" toml:"v5disc,optional"`
// Bootnodes is the list of initial bootnodes
Bootnodes []string `hcl:"bootnodes,optional" toml:"bootnodes,optional"`
// BootnodesV4 is the list of initial v4 bootnodes
BootnodesV4 []string `hcl:"bootnodesv4,optional" toml:"bootnodesv4,optional"`
// BootnodesV5 is the list of initial v5 bootnodes
BootnodesV5 []string `hcl:"bootnodesv5,optional" toml:"bootnodesv5,optional"`
// StaticNodes is the list of static nodes
StaticNodes []string `hcl:"static-nodes,optional" toml:"static-nodes,optional"`
// TrustedNodes is the list of trusted nodes
TrustedNodes []string `hcl:"trusted-nodes,optional" toml:"trusted-nodes,optional"`
// DNS is the list of enrtree:// URLs which will be queried for nodes to connect to
DNS []string `hcl:"dns,optional" toml:"dns,optional"`
}
type HeimdallConfig struct {
// URL is the url of the heimdall server
URL string `hcl:"url,optional" toml:"url,optional"`
// Without is used to disable remote heimdall during testing
Without bool `hcl:"bor.without,optional" toml:"bor.without,optional"`
// GRPCAddress is the address of the heimdall grpc server
GRPCAddress string `hcl:"grpc-address,optional" toml:"grpc-address,optional"`
// RunHeimdall is used to run heimdall as a child process
RunHeimdall bool `hcl:"bor.runheimdall,optional" toml:"bor.runheimdall,optional"`
// RunHeimdal args are the arguments to run heimdall with
RunHeimdallArgs string `hcl:"bor.runheimdallargs,optional" toml:"bor.runheimdallargs,optional"`
// UseHeimdallApp is used to fetch data from heimdall app when running heimdall as a child process
UseHeimdallApp bool `hcl:"bor.useheimdallapp,optional" toml:"bor.useheimdallapp,optional"`
}
type TxPoolConfig struct {
// Locals are the addresses that should be treated by default as local
Locals []string `hcl:"locals,optional" toml:"locals,optional"`
// NoLocals enables whether local transaction handling should be disabled
NoLocals bool `hcl:"nolocals,optional" toml:"nolocals,optional"`
// Journal is the path to store local transactions to survive node restarts
Journal string `hcl:"journal,optional" toml:"journal,optional"`
// Rejournal is the time interval to regenerate the local transaction journal
Rejournal time.Duration `hcl:"-,optional" toml:"-"`
RejournalRaw string `hcl:"rejournal,optional" toml:"rejournal,optional"`
// PriceLimit is the minimum gas price to enforce for acceptance into the pool
PriceLimit uint64 `hcl:"pricelimit,optional" toml:"pricelimit,optional"`
// PriceBump is the minimum price bump percentage to replace an already existing transaction (nonce)
PriceBump uint64 `hcl:"pricebump,optional" toml:"pricebump,optional"`
// AccountSlots is the number of executable transaction slots guaranteed per account
AccountSlots uint64 `hcl:"accountslots,optional" toml:"accountslots,optional"`
// GlobalSlots is the maximum number of executable transaction slots for all accounts
GlobalSlots uint64 `hcl:"globalslots,optional" toml:"globalslots,optional"`
// AccountQueue is the maximum number of non-executable transaction slots permitted per account
AccountQueue uint64 `hcl:"accountqueue,optional" toml:"accountqueue,optional"`
// GlobalQueueis the maximum number of non-executable transaction slots for all accounts
GlobalQueue uint64 `hcl:"globalqueue,optional" toml:"globalqueue,optional"`
// lifetime is the maximum amount of time non-executable transaction are queued
LifeTime time.Duration `hcl:"-,optional" toml:"-"`
LifeTimeRaw string `hcl:"lifetime,optional" toml:"lifetime,optional"`
}
type SealerConfig struct {
// Enabled is used to enable validator mode
Enabled bool `hcl:"mine,optional" toml:"mine,optional"`
// Etherbase is the address of the validator
Etherbase string `hcl:"etherbase,optional" toml:"etherbase,optional"`
// ExtraData is the block extra data set by the miner
ExtraData string `hcl:"extradata,optional" toml:"extradata,optional"`
// GasCeil is the target gas ceiling for mined blocks.
GasCeil uint64 `hcl:"gaslimit,optional" toml:"gaslimit,optional"`
// GasPrice is the minimum gas price for mining a transaction
GasPrice *big.Int `hcl:"-,optional" toml:"-"`
GasPriceRaw string `hcl:"gasprice,optional" toml:"gasprice,optional"`
// The time interval for miner to re-create mining work.
Recommit time.Duration `hcl:"-,optional" toml:"-"`
RecommitRaw string `hcl:"recommit,optional" toml:"recommit,optional"`
}
type JsonRPCConfig struct {
// IPCDisable enables whether ipc is enabled or not
IPCDisable bool `hcl:"ipcdisable,optional" toml:"ipcdisable,optional"`
// IPCPath is the path of the ipc endpoint
IPCPath string `hcl:"ipcpath,optional" toml:"ipcpath,optional"`
// GasCap is the global gas cap for eth-call variants.
GasCap uint64 `hcl:"gascap,optional" toml:"gascap,optional"`
// Sets a timeout used for eth_call (0=infinite)
RPCEVMTimeout time.Duration `hcl:"-,optional" toml:"-"`
RPCEVMTimeoutRaw string `hcl:"evmtimeout,optional" toml:"evmtimeout,optional"`
// TxFeeCap is the global transaction fee cap for send-transaction variants
TxFeeCap float64 `hcl:"txfeecap,optional" toml:"txfeecap,optional"`
// Http has the json-rpc http related settings
Http *APIConfig `hcl:"http,block" toml:"http,block"`
// Ws has the json-rpc websocket related settings
Ws *APIConfig `hcl:"ws,block" toml:"ws,block"`
// Graphql has the json-rpc graphql related settings
Graphql *APIConfig `hcl:"graphql,block" toml:"graphql,block"`
// AUTH RPC related settings
Auth *AUTHConfig `hcl:"auth,block" toml:"auth,block"`
HttpTimeout *HttpTimeouts `hcl:"timeouts,block" toml:"timeouts,block"`
AllowUnprotectedTxs bool `hcl:"allow-unprotected-txs,optional" toml:"allow-unprotected-txs,optional"`
}
type AUTHConfig struct {
// JWTSecret is the hex-encoded jwt secret.
JWTSecret string `hcl:"jwtsecret,optional" toml:"jwtsecret,optional"`
// Addr is the listening address on which authenticated APIs are provided.
Addr string `hcl:"addr,optional" toml:"addr,optional"`
// Port is the port number on which authenticated APIs are provided.
Port uint64 `hcl:"port,optional" toml:"port,optional"`
// VHosts is the list of virtual hostnames which are allowed on incoming requests
// for the authenticated api. This is by default {'localhost'}.
VHosts []string `hcl:"vhosts,optional" toml:"vhosts,optional"`
}
type GRPCConfig struct {
// Addr is the bind address for the grpc rpc server
Addr string `hcl:"addr,optional" toml:"addr,optional"`
}
type APIConfig struct {
// Enabled selects whether the api is enabled
Enabled bool `hcl:"enabled,optional" toml:"enabled,optional"`
// Port is the port number for this api
Port uint64 `hcl:"port,optional" toml:"port,optional"`
// Prefix is the http prefix to expose this api
Prefix string `hcl:"prefix,optional" toml:"prefix,optional"`
// Host is the address to bind the api
Host string `hcl:"host,optional" toml:"host,optional"`
// API is the list of enabled api modules
API []string `hcl:"api,optional" toml:"api,optional"`
// VHost is the list of valid virtual hosts
VHost []string `hcl:"vhosts,optional" toml:"vhosts,optional"`
// Cors is the list of Cors endpoints
Cors []string `hcl:"corsdomain,optional" toml:"corsdomain,optional"`
// Origins is the list of endpoints to accept requests from (only consumed for websockets)
Origins []string `hcl:"origins,optional" toml:"origins,optional"`
// ExecutionPoolSize is max size of workers to be used for rpc execution
ExecutionPoolSize uint64 `hcl:"ep-size,optional" toml:"ep-size,optional"`
// ExecutionPoolRequestTimeout is timeout used by execution pool for rpc execution
ExecutionPoolRequestTimeout time.Duration `hcl:"-,optional" toml:"-"`
ExecutionPoolRequestTimeoutRaw string `hcl:"ep-requesttimeout,optional" toml:"ep-requesttimeout,optional"`
}
// Used from rpc.HTTPTimeouts
type HttpTimeouts struct {
// ReadTimeout is the maximum duration for reading the entire
// request, including the body.
//
// Because ReadTimeout does not let Handlers make per-request
// decisions on each request body's acceptable deadline or
// upload rate, most users will prefer to use
// ReadHeaderTimeout. It is valid to use them both.
ReadTimeout time.Duration `hcl:"-,optional" toml:"-"`
ReadTimeoutRaw string `hcl:"read,optional" toml:"read,optional"`
// WriteTimeout is the maximum duration before timing out
// writes of the response. It is reset whenever a new
// request's header is read. Like ReadTimeout, it does not
// let Handlers make decisions on a per-request basis.
WriteTimeout time.Duration `hcl:"-,optional" toml:"-"`
WriteTimeoutRaw string `hcl:"write,optional" toml:"write,optional"`
// IdleTimeout is the maximum amount of time to wait for the
// next request when keep-alives are enabled. If IdleTimeout
// is zero, the value of ReadTimeout is used. If both are
// zero, ReadHeaderTimeout is used.
IdleTimeout time.Duration `hcl:"-,optional" toml:"-"`
IdleTimeoutRaw string `hcl:"idle,optional" toml:"idle,optional"`
}
type GpoConfig struct {
// Blocks is the number of blocks to track to compute the price oracle
Blocks uint64 `hcl:"blocks,optional" toml:"blocks,optional"`
// Percentile sets the weights to new blocks
Percentile uint64 `hcl:"percentile,optional" toml:"percentile,optional"`
// Maximum header history of gasprice oracle
MaxHeaderHistory int `hcl:"maxheaderhistory,optional" toml:"maxheaderhistory,optional"`
// Maximum block history of gasprice oracle
MaxBlockHistory int `hcl:"maxblockhistory,optional" toml:"maxblockhistory,optional"`
// MaxPrice is an upper bound gas price
MaxPrice *big.Int `hcl:"-,optional" toml:"-"`
MaxPriceRaw string `hcl:"maxprice,optional" toml:"maxprice,optional"`
// IgnorePrice is a lower bound gas price
IgnorePrice *big.Int `hcl:"-,optional" toml:"-"`
IgnorePriceRaw string `hcl:"ignoreprice,optional" toml:"ignoreprice,optional"`
}
type TelemetryConfig struct {
// Enabled enables metrics
Enabled bool `hcl:"metrics,optional" toml:"metrics,optional"`
// Expensive enables expensive metrics
Expensive bool `hcl:"expensive,optional" toml:"expensive,optional"`
// InfluxDB has the influxdb related settings
InfluxDB *InfluxDBConfig `hcl:"influx,block" toml:"influx,block"`
// Prometheus Address
PrometheusAddr string `hcl:"prometheus-addr,optional" toml:"prometheus-addr,optional"`
// Open collector endpoint
OpenCollectorEndpoint string `hcl:"opencollector-endpoint,optional" toml:"opencollector-endpoint,optional"`
}
type InfluxDBConfig struct {
// V1Enabled enables influx v1 mode
V1Enabled bool `hcl:"influxdb,optional" toml:"influxdb,optional"`
// Endpoint is the url endpoint of the influxdb service
Endpoint string `hcl:"endpoint,optional" toml:"endpoint,optional"`
// Database is the name of the database in Influxdb to store the metrics.
Database string `hcl:"database,optional" toml:"database,optional"`
// Enabled is the username to authorize access to Influxdb
Username string `hcl:"username,optional" toml:"username,optional"`
// Password is the password to authorize access to Influxdb
Password string `hcl:"password,optional" toml:"password,optional"`
// Tags are tags attaches to all generated metrics
Tags map[string]string `hcl:"tags,optional" toml:"tags,optional"`
// Enabled enables influx v2 mode
V2Enabled bool `hcl:"influxdbv2,optional" toml:"influxdbv2,optional"`
// Token is the token to authorize access to Influxdb V2.
Token string `hcl:"token,optional" toml:"token,optional"`
// Bucket is the bucket to store metrics in Influxdb V2.
Bucket string `hcl:"bucket,optional" toml:"bucket,optional"`
// Organization is the name of the organization for Influxdb V2.
Organization string `hcl:"organization,optional" toml:"organization,optional"`
}
type CacheConfig struct {
// Cache is the amount of cache of the node
Cache uint64 `hcl:"cache,optional" toml:"cache,optional"`
// PercGc is percentage of cache used for garbage collection
PercGc uint64 `hcl:"gc,optional" toml:"gc,optional"`
// PercSnapshot is percentage of cache used for snapshots
PercSnapshot uint64 `hcl:"snapshot,optional" toml:"snapshot,optional"`
// PercDatabase is percentage of cache used for the database
PercDatabase uint64 `hcl:"database,optional" toml:"database,optional"`
// PercTrie is percentage of cache used for the trie
PercTrie uint64 `hcl:"trie,optional" toml:"trie,optional"`
// Journal is the disk journal directory for trie cache to survive node restarts
Journal string `hcl:"journal,optional" toml:"journal,optional"`
// Rejournal is the time interval to regenerate the journal for clean cache
Rejournal time.Duration `hcl:"-,optional" toml:"-"`
RejournalRaw string `hcl:"rejournal,optional" toml:"rejournal,optional"`
// NoPrefetch is used to disable prefetch of tries
NoPrefetch bool `hcl:"noprefetch,optional" toml:"noprefetch,optional"`
// Preimages is used to enable the track of hash preimages
Preimages bool `hcl:"preimages,optional" toml:"preimages,optional"`
// TxLookupLimit sets the maximum number of blocks from head whose tx indices are reserved.
TxLookupLimit uint64 `hcl:"txlookuplimit,optional" toml:"txlookuplimit,optional"`
// Number of block states to keep in memory (default = 128)
TriesInMemory uint64 `hcl:"triesinmemory,optional" toml:"triesinmemory,optional"`
// Time after which the Merkle Patricia Trie is stored to disc from memory
TrieTimeout time.Duration `hcl:"-,optional" toml:"-"`
TrieTimeoutRaw string `hcl:"timeout,optional" toml:"timeout,optional"`
// Raise the open file descriptor resource limit (default = system fd limit)
FDLimit int `hcl:"fdlimit,optional" toml:"fdlimit,optional"`
}
type AccountsConfig struct {
// Unlock is the list of addresses to unlock in the node
Unlock []string `hcl:"unlock,optional" toml:"unlock,optional"`
// PasswordFile is the file where the account passwords are stored
PasswordFile string `hcl:"password,optional" toml:"password,optional"`
// AllowInsecureUnlock allows user to unlock accounts in unsafe http environment.
AllowInsecureUnlock bool `hcl:"allow-insecure-unlock,optional" toml:"allow-insecure-unlock,optional"`
// UseLightweightKDF enables a faster but less secure encryption of accounts
UseLightweightKDF bool `hcl:"lightkdf,optional" toml:"lightkdf,optional"`
// DisableBorWallet disables the personal wallet endpoints
DisableBorWallet bool `hcl:"disable-bor-wallet,optional" toml:"disable-bor-wallet,optional"`
}
type DeveloperConfig struct {
// Enabled enables the developer mode
Enabled bool `hcl:"dev,optional" toml:"dev,optional"`
// Period is the block period to use in developer mode
Period uint64 `hcl:"period,optional" toml:"period,optional"`
// Initial block gas limit
GasLimit uint64 `hcl:"gaslimit,optional" toml:"gaslimit,optional"`
}
func DefaultConfig() *Config {
return &Config{
Chain: "mainnet",
Identity: Hostname(),
RequiredBlocks: map[string]string{},
Verbosity: 3,
LogLevel: "",
EnablePreimageRecording: false,
DataDir: DefaultDataDir(),
Ancient: "",
Logging: &LoggingConfig{
Vmodule: "",
Json: false,
Backtrace: "",
Debug: false,
},
RPCBatchLimit: 100,
RPCReturnDataLimit: 100000,
P2P: &P2PConfig{
MaxPeers: 50,
MaxPendPeers: 50,
Bind: "0.0.0.0",
Port: 30303,
NoDiscover: false,
NAT: "any",
NetRestrict: "",
Discovery: &P2PDiscovery{
V5Enabled: false,
Bootnodes: []string{},
BootnodesV4: []string{},
BootnodesV5: []string{},
StaticNodes: []string{},
TrustedNodes: []string{},
DNS: []string{},
},
},
Heimdall: &HeimdallConfig{
URL: "http://localhost:1317",
Without: false,
GRPCAddress: "",
},
SyncMode: "full",
GcMode: "full",
Snapshot: true,
BorLogs: false,
TxPool: &TxPoolConfig{
Locals: []string{},
NoLocals: false,
Journal: "transactions.rlp",
Rejournal: 1 * time.Hour,
PriceLimit: 1, // geth's default
PriceBump: 10,
AccountSlots: 16,
GlobalSlots: 32768,
AccountQueue: 16,
GlobalQueue: 32768,
LifeTime: 3 * time.Hour,
},
Sealer: &SealerConfig{
Enabled: false,
Etherbase: "",
GasCeil: 30_000_000, // geth's default
GasPrice: big.NewInt(1 * params.GWei), // geth's default
ExtraData: "",
Recommit: 125 * time.Second,
},
Gpo: &GpoConfig{
Blocks: 20,
Percentile: 60,
MaxHeaderHistory: 1024,
MaxBlockHistory: 1024,
MaxPrice: gasprice.DefaultMaxPrice,
IgnorePrice: gasprice.DefaultIgnorePrice,
},
JsonRPC: &JsonRPCConfig{
IPCDisable: false,
IPCPath: "",
GasCap: ethconfig.Defaults.RPCGasCap,
TxFeeCap: ethconfig.Defaults.RPCTxFeeCap,
RPCEVMTimeout: ethconfig.Defaults.RPCEVMTimeout,
AllowUnprotectedTxs: false,
Http: &APIConfig{
Enabled: false,
Port: 8545,
Prefix: "",
Host: "localhost",
API: []string{"eth", "net", "web3", "txpool", "bor"},
Cors: []string{"localhost"},
VHost: []string{"localhost"},
ExecutionPoolSize: 40,
ExecutionPoolRequestTimeout: 0,
},
Ws: &APIConfig{
Enabled: false,
Port: 8546,
Prefix: "",
Host: "localhost",
API: []string{"net", "web3"},
Origins: []string{"localhost"},
ExecutionPoolSize: 40,
ExecutionPoolRequestTimeout: 0,
},
Graphql: &APIConfig{
Enabled: false,
Cors: []string{"localhost"},
VHost: []string{"localhost"},
},
HttpTimeout: &HttpTimeouts{
ReadTimeout: 10 * time.Second,
WriteTimeout: 30 * time.Second,
IdleTimeout: 120 * time.Second,
},
Auth: &AUTHConfig{
JWTSecret: "",
Port: node.DefaultAuthPort,
Addr: node.DefaultAuthHost,
VHosts: node.DefaultAuthVhosts,
},
},
Ethstats: "",
Telemetry: &TelemetryConfig{
Enabled: false,
Expensive: false,
PrometheusAddr: "127.0.0.1:7071",
OpenCollectorEndpoint: "127.0.0.1:4317",
InfluxDB: &InfluxDBConfig{
V1Enabled: false,
Endpoint: "",
Database: "",
Username: "",
Password: "",
Tags: map[string]string{},
V2Enabled: false,
Token: "",
Bucket: "",
Organization: "",
},
},
Cache: &CacheConfig{
Cache: 1024, // geth's default (suitable for mumbai)
PercDatabase: 50,
PercTrie: 15,
PercGc: 25,
PercSnapshot: 10,
Journal: "triecache",
Rejournal: 60 * time.Minute,
NoPrefetch: false,
Preimages: false,
TxLookupLimit: 2350000,
TriesInMemory: 128,
TrieTimeout: 60 * time.Minute,
FDLimit: 0,
},
Accounts: &AccountsConfig{
Unlock: []string{},
PasswordFile: "",
AllowInsecureUnlock: false,
UseLightweightKDF: false,
DisableBorWallet: true,
},
GRPC: &GRPCConfig{
Addr: ":3131",
},
Developer: &DeveloperConfig{
Enabled: false,
Period: 0,
GasLimit: 11500000,
},
DevFakeAuthor: false,
Pprof: &PprofConfig{
Enabled: false,
Port: 6060,
Addr: "127.0.0.1",
MemProfileRate: 512 * 1024,
BlockProfileRate: 0,
// CPUProfile: "",
},
}
}
func (c *Config) fillBigInt() error {
tds := []struct {
path string
td **big.Int
str *string
}{
{"gpo.maxprice", &c.Gpo.MaxPrice, &c.Gpo.MaxPriceRaw},
{"gpo.ignoreprice", &c.Gpo.IgnorePrice, &c.Gpo.IgnorePriceRaw},
{"miner.gasprice", &c.Sealer.GasPrice, &c.Sealer.GasPriceRaw},
}
for _, x := range tds {
if *x.str != "" {
b := new(big.Int)
var ok bool
if strings.HasPrefix(*x.str, "0x") {
b, ok = b.SetString((*x.str)[2:], 16)
} else {
b, ok = b.SetString(*x.str, 10)
}
if !ok {
return fmt.Errorf("%s can't parse big int %s", x.path, *x.str)
}
*x.str = ""
*x.td = b
}
}
return nil
}
func (c *Config) fillTimeDurations() error {
tds := []struct {
path string
td *time.Duration
str *string
}{
{"jsonrpc.evmtimeout", &c.JsonRPC.RPCEVMTimeout, &c.JsonRPC.RPCEVMTimeoutRaw},
{"miner.recommit", &c.Sealer.Recommit, &c.Sealer.RecommitRaw},
{"jsonrpc.timeouts.read", &c.JsonRPC.HttpTimeout.ReadTimeout, &c.JsonRPC.HttpTimeout.ReadTimeoutRaw},
{"jsonrpc.timeouts.write", &c.JsonRPC.HttpTimeout.WriteTimeout, &c.JsonRPC.HttpTimeout.WriteTimeoutRaw},
{"jsonrpc.timeouts.idle", &c.JsonRPC.HttpTimeout.IdleTimeout, &c.JsonRPC.HttpTimeout.IdleTimeoutRaw},
{"jsonrpc.ws.ep-requesttimeout", &c.JsonRPC.Ws.ExecutionPoolRequestTimeout, &c.JsonRPC.Ws.ExecutionPoolRequestTimeoutRaw},
{"jsonrpc.http.ep-requesttimeout", &c.JsonRPC.Http.ExecutionPoolRequestTimeout, &c.JsonRPC.Http.ExecutionPoolRequestTimeoutRaw},
{"txpool.lifetime", &c.TxPool.LifeTime, &c.TxPool.LifeTimeRaw},
{"txpool.rejournal", &c.TxPool.Rejournal, &c.TxPool.RejournalRaw},
{"cache.rejournal", &c.Cache.Rejournal, &c.Cache.RejournalRaw},
{"cache.timeout", &c.Cache.TrieTimeout, &c.Cache.TrieTimeoutRaw},
}
for _, x := range tds {
if x.td != nil && x.str != nil && *x.str != "" {
d, err := time.ParseDuration(*x.str)
if err != nil {
return fmt.Errorf("%s can't parse time duration %s", x.path, *x.str)
}
*x.str = ""
*x.td = d
}
}
return nil
}
func readConfigFile(path string) (*Config, error) {
ext := filepath.Ext(path)
if ext == ".toml" {
return readLegacyConfig(path)
}
config := &Config{
TxPool: &TxPoolConfig{},
Cache: &CacheConfig{},
Sealer: &SealerConfig{},
}
if err := hclsimple.DecodeFile(path, nil, config); err != nil {
return nil, fmt.Errorf("failed to decode config file '%s': %v", path, err)
}
if err := config.fillBigInt(); err != nil {
return nil, err
}
if err := config.fillTimeDurations(); err != nil {
return nil, err
}
return config, nil
}
func (c *Config) loadChain() error {
chain, err := chains.GetChain(c.Chain)
if err != nil {
return err
}
c.chain = chain
// preload some default values that depend on the chain file
if c.P2P.Discovery.DNS == nil {
c.P2P.Discovery.DNS = c.chain.DNS
}
return nil
}
//nolint:gocognit
func (c *Config) buildEth(stack *node.Node, accountManager *accounts.Manager) (*ethconfig.Config, error) {
dbHandles, err := MakeDatabaseHandles(c.Cache.FDLimit)
if err != nil {
return nil, err
}
n := ethconfig.Defaults
// only update for non-developer mode as we don't yet
// have the chain object for it.
if !c.Developer.Enabled {
n.NetworkId = c.chain.NetworkId
n.Genesis = c.chain.Genesis
}
n.HeimdallURL = c.Heimdall.URL
n.WithoutHeimdall = c.Heimdall.Without
n.HeimdallgRPCAddress = c.Heimdall.GRPCAddress
n.RunHeimdall = c.Heimdall.RunHeimdall
n.RunHeimdallArgs = c.Heimdall.RunHeimdallArgs
n.UseHeimdallApp = c.Heimdall.UseHeimdallApp
// Developer Fake Author for producing blocks without authorisation on bor consensus
n.DevFakeAuthor = c.DevFakeAuthor
// Developer Fake Author for producing blocks without authorisation on bor consensus
n.DevFakeAuthor = c.DevFakeAuthor
// gas price oracle
{
n.GPO.Blocks = int(c.Gpo.Blocks)
n.GPO.Percentile = int(c.Gpo.Percentile)
n.GPO.MaxHeaderHistory = c.Gpo.MaxHeaderHistory
n.GPO.MaxBlockHistory = c.Gpo.MaxBlockHistory
n.GPO.MaxPrice = c.Gpo.MaxPrice
n.GPO.IgnorePrice = c.Gpo.IgnorePrice
}
n.EnablePreimageRecording = c.EnablePreimageRecording
// txpool options
{
n.TxPool.NoLocals = c.TxPool.NoLocals
n.TxPool.Journal = c.TxPool.Journal
n.TxPool.Rejournal = c.TxPool.Rejournal
n.TxPool.PriceLimit = c.TxPool.PriceLimit
n.TxPool.PriceBump = c.TxPool.PriceBump
n.TxPool.AccountSlots = c.TxPool.AccountSlots
n.TxPool.GlobalSlots = c.TxPool.GlobalSlots
n.TxPool.AccountQueue = c.TxPool.AccountQueue
n.TxPool.GlobalQueue = c.TxPool.GlobalQueue
n.TxPool.Lifetime = c.TxPool.LifeTime
}
// miner options
{
n.Miner.Recommit = c.Sealer.Recommit
n.Miner.GasPrice = c.Sealer.GasPrice
n.Miner.GasCeil = c.Sealer.GasCeil
n.Miner.ExtraData = []byte(c.Sealer.ExtraData)
if etherbase := c.Sealer.Etherbase; etherbase != "" {
if !common.IsHexAddress(etherbase) {
return nil, fmt.Errorf("etherbase is not an address: %s", etherbase)
}
n.Miner.Etherbase = common.HexToAddress(etherbase)
}
}
// unlock accounts
if len(c.Accounts.Unlock) > 0 {
if !stack.Config().InsecureUnlockAllowed && stack.Config().ExtRPCEnabled() {
return nil, fmt.Errorf("account unlock with HTTP access is forbidden")
}
ks := accountManager.Backends(keystore.KeyStoreType)[0].(*keystore.KeyStore)
passwords, err := MakePasswordListFromFile(c.Accounts.PasswordFile)
if err != nil {
return nil, err
}
if len(passwords) < len(c.Accounts.Unlock) {
return nil, fmt.Errorf("number of passwords provided (%v) is less than number of accounts (%v) to unlock",
len(passwords), len(c.Accounts.Unlock))
}
for i, account := range c.Accounts.Unlock {
unlockAccount(ks, account, i, passwords)
}
}
// update for developer mode
if c.Developer.Enabled {
// Get a keystore
var ks *keystore.KeyStore
if keystores := accountManager.Backends(keystore.KeyStoreType); len(keystores) > 0 {
ks = keystores[0].(*keystore.KeyStore)
}
// Create new developer account or reuse existing one
var (
developer accounts.Account
passphrase string
err error
)
// etherbase has been set above, configuring the miner address from command line flags.
if n.Miner.Etherbase != (common.Address{}) {
developer = accounts.Account{Address: n.Miner.Etherbase}
} else if accs := ks.Accounts(); len(accs) > 0 {
developer = ks.Accounts()[0]
} else {
developer, err = ks.NewAccount(passphrase)
if err != nil {
return nil, fmt.Errorf("failed to create developer account: %v", err)
}
}
if err := ks.Unlock(developer, passphrase); err != nil {
return nil, fmt.Errorf("failed to unlock developer account: %v", err)
}
log.Info("Using developer account", "address", developer.Address)
// Set the Etherbase
c.Sealer.Etherbase = developer.Address.Hex()
n.Miner.Etherbase = developer.Address
// get developer mode chain config
c.chain = chains.GetDeveloperChain(c.Developer.Period, c.Developer.GasLimit, developer.Address)
// update the parameters
n.NetworkId = c.chain.NetworkId
n.Genesis = c.chain.Genesis
// Update cache
c.Cache.Cache = 1024
// Update sync mode
c.SyncMode = "full"
// update miner gas price
if n.Miner.GasPrice == nil {
n.Miner.GasPrice = big.NewInt(1)
}
}
// discovery (this params should be in node.Config)
{
n.EthDiscoveryURLs = c.P2P.Discovery.DNS
n.SnapDiscoveryURLs = c.P2P.Discovery.DNS
}
// RequiredBlocks
{
n.PeerRequiredBlocks = map[uint64]common.Hash{}
for k, v := range c.RequiredBlocks {
number, err := strconv.ParseUint(k, 0, 64)
if err != nil {
return nil, fmt.Errorf("invalid required block number %s: %v", k, err)
}
var hash common.Hash
if err = hash.UnmarshalText([]byte(v)); err != nil {
return nil, fmt.Errorf("invalid required block hash %s: %v", v, err)
}
n.PeerRequiredBlocks[number] = hash
}
}
// cache
{
cache := c.Cache.Cache
calcPerc := func(val uint64) int {
return int(cache * (val) / 100)
}
// Cap the cache allowance
mem, err := gopsutil.VirtualMemory()
if err == nil {
if 32<<(^uintptr(0)>>63) == 32 && mem.Total > 2*1024*1024*1024 {
log.Warn("Lowering memory allowance on 32bit arch", "available", mem.Total/1024/1024, "addressable", 2*1024)
mem.Total = 2 * 1024 * 1024 * 1024
}
allowance := uint64(mem.Total / 1024 / 1024 / 3)
if cache > allowance {
log.Warn("Sanitizing cache to Go's GC limits", "provided", cache, "updated", allowance)
cache = allowance
}
}
// Tune the garbage collector
gogc := math.Max(20, math.Min(100, 100/(float64(cache)/1024)))
log.Debug("Sanitizing Go's GC trigger", "percent", int(gogc))
godebug.SetGCPercent(int(gogc))
n.TrieCleanCacheJournal = c.Cache.Journal
n.TrieCleanCacheRejournal = c.Cache.Rejournal
n.DatabaseCache = calcPerc(c.Cache.PercDatabase)
n.SnapshotCache = calcPerc(c.Cache.PercSnapshot)
n.TrieCleanCache = calcPerc(c.Cache.PercTrie)
n.TrieDirtyCache = calcPerc(c.Cache.PercGc)
n.NoPrefetch = c.Cache.NoPrefetch
n.Preimages = c.Cache.Preimages
n.TxLookupLimit = c.Cache.TxLookupLimit
n.TrieTimeout = c.Cache.TrieTimeout
n.TriesInMemory = c.Cache.TriesInMemory
}
n.RPCGasCap = c.JsonRPC.GasCap
if n.RPCGasCap != 0 {
log.Info("Set global gas cap", "cap", n.RPCGasCap)
} else {
log.Info("Global gas cap disabled")
}
n.RPCEVMTimeout = c.JsonRPC.RPCEVMTimeout
n.RPCTxFeeCap = c.JsonRPC.TxFeeCap
// sync mode. It can either be "fast", "full" or "snap". We disable
// for now the "light" mode.
switch c.SyncMode {
case "full":
n.SyncMode = downloader.FullSync
case "snap":
// n.SyncMode = downloader.SnapSync // TODO(snap): Uncomment when we have snap sync working
n.SyncMode = downloader.FullSync
log.Warn("Bor doesn't support Snap Sync yet, switching to Full Sync mode")
default:
return nil, fmt.Errorf("sync mode '%s' not found", c.SyncMode)
}
// archive mode. It can either be "archive" or "full".
switch c.GcMode {
case "full":
n.NoPruning = false
case "archive":
n.NoPruning = true
if !n.Preimages {
n.Preimages = true
log.Info("Enabling recording of key preimages since archive mode is used")
}
default:
return nil, fmt.Errorf("gcmode '%s' not found", c.GcMode)
}
// snapshot disable check
if !c.Snapshot {
if n.SyncMode == downloader.SnapSync {
log.Info("Snap sync requested, enabling --snapshot")
} else {
// disable snapshot
n.TrieCleanCache += n.SnapshotCache
n.SnapshotCache = 0
}
}
n.BorLogs = c.BorLogs
n.DatabaseHandles = dbHandles
n.RPCReturnDataLimit = c.RPCReturnDataLimit
if c.Ancient != "" {
n.DatabaseFreezer = c.Ancient
}
return &n, nil
}
var (
clientIdentifier = "bor"
gitCommit = "" // Git SHA1 commit hash of the release (set via linker flags)
gitDate = "" // Git commit date YYYYMMDD of the release (set via linker flags)
)
// tries unlocking the specified account a few times.
func unlockAccount(ks *keystore.KeyStore, address string, i int, passwords []string) (accounts.Account, string) {
account, err := utils.MakeAddress(ks, address)
if err != nil {
utils.Fatalf("Could not list accounts: %v", err)
}
for trials := 0; trials < 3; trials++ {
prompt := fmt.Sprintf("Unlocking account %s | Attempt %d/%d", address, trials+1, 3)
password := utils.GetPassPhraseWithList(prompt, false, i, passwords)
err = ks.Unlock(account, password)
if err == nil {
log.Info("Unlocked account", "address", account.Address.Hex())
return account, password
}
if err, ok := err.(*keystore.AmbiguousAddrError); ok {
log.Info("Unlocked account", "address", account.Address.Hex())
return ambiguousAddrRecovery(ks, err, password), password
}
if err != keystore.ErrDecrypt {
// No need to prompt again if the error is not decryption-related.
break
}
}
// All trials expended to unlock account, bail out
utils.Fatalf("Failed to unlock account %s (%v)", address, err)
return accounts.Account{}, ""
}
func ambiguousAddrRecovery(ks *keystore.KeyStore, err *keystore.AmbiguousAddrError, auth string) accounts.Account {
log.Warn("Multiple key files exist for", "address", err.Addr)
for _, a := range err.Matches {
log.Info("Multiple keys", "file", a.URL.String())
}
log.Info("Testing your password against all of them...")
var match *accounts.Account
for _, a := range err.Matches {
if err := ks.Unlock(a, auth); err == nil {
// nolint: gosec, exportloopref
match = &a
break
}
}
if match == nil {
utils.Fatalf("None of the listed files could be unlocked.")
}
log.Info("Your password unlocked", "key", match.URL.String())
log.Warn("In order to avoid this warning, you need to remove the following duplicate key files:")
for _, a := range err.Matches {
if a != *match {
log.Warn("Duplicate", "key", a.URL.String())
}
}
return *match
}
// setNodeKey creates a node key from set command line flags, either loading it
// from a file or as a specified hex value. If neither flags were provided, this
// method returns nil and an emphemeral key is to be generated.
func getNodeKey(hex string, file string) *ecdsa.PrivateKey {
var (
key *ecdsa.PrivateKey
err error
)
switch {
case file != "" && hex != "":
utils.Fatalf("Options %q and %q are mutually exclusive", file, hex)
case file != "":
if key, err = crypto.LoadECDSA(file); err != nil {
utils.Fatalf("Option %q: %v", file, err)
}
return key
case hex != "":
if key, err = crypto.HexToECDSA(hex); err != nil {
utils.Fatalf("Option %q: %v", hex, err)
}
return key
}
return nil
}
func (c *Config) buildNode() (*node.Config, error) {
ipcPath := ""
if !c.JsonRPC.IPCDisable {
ipcPath = clientIdentifier + ".ipc"
if c.JsonRPC.IPCPath != "" {
ipcPath = c.JsonRPC.IPCPath
}
}
cfg := &node.Config{
Name: clientIdentifier,
DataDir: c.DataDir,
KeyStoreDir: c.KeyStoreDir,
UseLightweightKDF: c.Accounts.UseLightweightKDF,
InsecureUnlockAllowed: c.Accounts.AllowInsecureUnlock,
Version: params.VersionWithCommit(gitCommit, gitDate),
IPCPath: ipcPath,
AllowUnprotectedTxs: c.JsonRPC.AllowUnprotectedTxs,
P2P: p2p.Config{
MaxPeers: int(c.P2P.MaxPeers),
MaxPendingPeers: int(c.P2P.MaxPendPeers),
ListenAddr: c.P2P.Bind + ":" + strconv.Itoa(int(c.P2P.Port)),
DiscoveryV5: c.P2P.Discovery.V5Enabled,
},
HTTPModules: c.JsonRPC.Http.API,
HTTPCors: c.JsonRPC.Http.Cors,
HTTPVirtualHosts: c.JsonRPC.Http.VHost,
HTTPPathPrefix: c.JsonRPC.Http.Prefix,
WSModules: c.JsonRPC.Ws.API,
WSOrigins: c.JsonRPC.Ws.Origins,
WSPathPrefix: c.JsonRPC.Ws.Prefix,
GraphQLCors: c.JsonRPC.Graphql.Cors,
GraphQLVirtualHosts: c.JsonRPC.Graphql.VHost,
HTTPTimeouts: rpc.HTTPTimeouts{
ReadTimeout: c.JsonRPC.HttpTimeout.ReadTimeout,
WriteTimeout: c.JsonRPC.HttpTimeout.WriteTimeout,
IdleTimeout: c.JsonRPC.HttpTimeout.IdleTimeout,
},
JWTSecret: c.JsonRPC.Auth.JWTSecret,
AuthPort: int(c.JsonRPC.Auth.Port),
AuthAddr: c.JsonRPC.Auth.Addr,
AuthVirtualHosts: c.JsonRPC.Auth.VHosts,
RPCBatchLimit: c.RPCBatchLimit,
WSJsonRPCExecutionPoolSize: c.JsonRPC.Ws.ExecutionPoolSize,
WSJsonRPCExecutionPoolRequestTimeout: c.JsonRPC.Ws.ExecutionPoolRequestTimeout,
HTTPJsonRPCExecutionPoolSize: c.JsonRPC.Http.ExecutionPoolSize,
HTTPJsonRPCExecutionPoolRequestTimeout: c.JsonRPC.Http.ExecutionPoolRequestTimeout,
}
if c.P2P.NetRestrict != "" {
list, err := netutil.ParseNetlist(c.P2P.NetRestrict)
if err != nil {
utils.Fatalf("Option %q: %v", c.P2P.NetRestrict, err)
}
cfg.P2P.NetRestrict = list
}
key := getNodeKey(c.P2P.NodeKeyHex, c.P2P.NodeKey)
if key != nil {
cfg.P2P.PrivateKey = key
}
// dev mode
if c.Developer.Enabled {
cfg.UseLightweightKDF = true
// disable p2p networking
c.P2P.NoDiscover = true
cfg.P2P.ListenAddr = ""
cfg.P2P.NoDial = true
cfg.P2P.DiscoveryV5 = false
// enable JsonRPC HTTP API
c.JsonRPC.Http.Enabled = true
cfg.HTTPModules = []string{"admin", "debug", "eth", "miner", "net", "personal", "txpool", "web3", "bor"}
}
// enable jsonrpc endpoints
{
if c.JsonRPC.Http.Enabled {
cfg.HTTPHost = c.JsonRPC.Http.Host
cfg.HTTPPort = int(c.JsonRPC.Http.Port)
}
if c.JsonRPC.Ws.Enabled {
cfg.WSHost = c.JsonRPC.Ws.Host
cfg.WSPort = int(c.JsonRPC.Ws.Port)
}
}
natif, err := nat.Parse(c.P2P.NAT)
if err != nil {
return nil, fmt.Errorf("wrong 'nat' flag: %v", err)
}
cfg.P2P.NAT = natif
// only check for non-developer modes
if !c.Developer.Enabled {
// Discovery
// if no bootnodes are defined, use the ones from the chain file.
bootnodes := c.P2P.Discovery.Bootnodes
if len(bootnodes) == 0 {
bootnodes = c.chain.Bootnodes
}
if cfg.P2P.BootstrapNodes, err = parseBootnodes(bootnodes); err != nil {
return nil, err
}
if cfg.P2P.BootstrapNodesV5, err = parseBootnodes(c.P2P.Discovery.BootnodesV5); err != nil {
return nil, err
}
if cfg.P2P.StaticNodes, err = parseBootnodes(c.P2P.Discovery.StaticNodes); err != nil {
return nil, err
}
if len(cfg.P2P.StaticNodes) == 0 {
cfg.P2P.StaticNodes = cfg.StaticNodes()
}
if cfg.P2P.TrustedNodes, err = parseBootnodes(c.P2P.Discovery.TrustedNodes); err != nil {
return nil, err
}
if len(cfg.P2P.TrustedNodes) == 0 {
cfg.P2P.TrustedNodes = cfg.TrustedNodes()
}
}
if c.P2P.NoDiscover {
// Disable peer discovery
cfg.P2P.NoDiscovery = true
}
return cfg, nil
}
func (c *Config) Merge(cc ...*Config) error {
for _, elem := range cc {
if err := mergo.Merge(c, elem, mergo.WithOverwriteWithEmptyValue); err != nil {
return fmt.Errorf("failed to merge configurations: %v", err)
}
}
return nil
}
func MakeDatabaseHandles(max int) (int, error) {
limit, err := fdlimit.Maximum()
if err != nil {
return -1, err
}
switch {
case max == 0:
// User didn't specify a meaningful value, use system limits
case max < 128:
// User specified something unhealthy, just use system defaults
log.Error("File descriptor limit invalid (<128)", "had", max, "updated", limit)
case max > limit:
// User requested more than the OS allows, notify that we can't allocate it
log.Warn("Requested file descriptors denied by OS", "req", max, "limit", limit)
default:
// User limit is meaningful and within allowed range, use that
limit = max
}
raised, err := fdlimit.Raise(uint64(limit))
if err != nil {
return -1, err
}
return int(raised / 2), nil // Leave half for networking and other stuff
}
func parseBootnodes(urls []string) ([]*enode.Node, error) {
dst := []*enode.Node{}
for _, url := range urls {
if url != "" {
node, err := enode.Parse(enode.ValidSchemes, url)
if err != nil {
return nil, fmt.Errorf("invalid bootstrap url '%s': %v", url, err)
}
dst = append(dst, node)
}
}
return dst, nil
}
func DefaultDataDir() string {
// Try to place the data folder in the user's home dir
home, _ := homedir.Dir()
if home == "" {
// we cannot guess a stable location
return ""
}
switch runtime.GOOS {
case "darwin":
return filepath.Join(home, "Library", "Bor")
case "windows":
appdata := os.Getenv("LOCALAPPDATA")
if appdata == "" {
// Windows XP and below don't have LocalAppData.
panic("environment variable LocalAppData is undefined")
}
return filepath.Join(appdata, "Bor")
default:
return filepath.Join(home, ".bor")
}
}
func Hostname() string {
hostname, err := os.Hostname()
if err != nil {
return "bor"
}
return hostname
}
func MakePasswordListFromFile(path string) ([]string, error) {
text, err := ioutil.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("failed to read password file: %v", err)
}
lines := strings.Split(string(text), "\n")
// Sanitise DOS line endings.
for i := range lines {
lines[i] = strings.TrimRight(lines[i], "\r")
}
return lines, nil
}