Commit graph

58 commits

Author SHA1 Message Date
Shivam Sharma
184b2d628d Merge branch 'master' into shivam/temp-master-develop 2024-03-20 14:29:04 +05:30
Manav Darji
cb23b9baa3
eth, core: track block propagation (#1078)
* eth, core: track block announcement and import timings

* eth: track total delay

* eth: add more context and prefix in logs

* eth/fetcher: fix invalid tracking of received at time

* put logs behind log.enable-block-tracking flag

* fix lint

* improve logging

* add comments

* add comments
2024-03-05 22:30:51 +05:30
Shivam Sharma
fcd5002b6b fix : flags, FLOCK lock 2024-01-19 17:55:39 +05:30
SHIVAM SHARMA
2a35bf8340
add : amoy flags and genesis (#1131)
* add : amoy flags and genesis

* minor fix

* add : amoy.json in internal/cli/server

* disable : peer check in mining for Amoy

* add : amoy genesis hash

* add amoy testnet deployment entries

* update : amoy genesis

* fix typos

* add : post london forks + burn contract

* add : amoy genesis hash

* bump : version meta to amoy (#1092)

* add : amoy bootnodes

* chg : maxpeers of amoy val to 3 from 1

* bump : version to v1.1.0-amoy-2

* add : amoychain worker restriction on peercount

---------

Co-authored-by: sreevalsanmr <sreevalsan@polygon.technology>
2024-01-19 02:01:36 +05:30
Pratik Patil
923c192c7d
Added cache.blocklogs (FilterLogCacheSize) flag (#1083)
* added cache.blocklogs (FilterLogCacheSize) flag

* updated docs

* minor fix

Co-authored-by: Mael Regnery <mael@mqli.fr>

---------

Co-authored-by: Mael Regnery <mael@mqli.fr>
2023-11-20 10:18:11 +05:30
Arpit Temani
384383c1eb
Merge pull request #968 from maticnetwork/arpit/pos-1731
Geth Merge v1.12.2
2023-11-10 15:11:26 +05:30
Arpit Temani
3c97a5b7a0 Revert "aded blobpool related flags"
This reverts commit 15f0c41d2c.
2023-11-10 14:37:26 +05:30
Pratik Patil
15f0c41d2c
aded blobpool related flags 2023-11-09 15:04:28 +05:30
Pratik Patil
368ae246fd
added p2p v4 discovery flag wrt geth v1.12.2 merge 2023-11-08 17:44:35 +05:30
Pratik Patil
6aed36d9d1
removed --cache.trie.journal and --cache.trie.rejournal wrt geth v1.12.2 merge 2023-11-08 17:11:54 +05:30
Léo Vincent
22a2b63df6
chore: improve bor server --help (#1063)
* chore: update `txarrivalwait` flag description

* chore: update docs

* chore: update other flags

* fix: doc generation is now deterministic

* chore: remove debug

* feat: sort options in `bor server --help`

* fix: prealloc array

* fix: lint issues

* internal/cli: remove nolint

---------

Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
2023-11-07 10:57:18 +05:30
Manav Darji
adfd428bc0
internal/cli: set default value for keystore (#1051)
* internal/cli: set default valuee for keystore

* internal/cli: add tests

* internal/cli: fix tests
2023-10-18 12:16:34 +05:30
Manav Darji
8613ff1c06
internal/cli: add support to overwrite config.toml values via cli flags (#1008)
* internal/cli: add support to overwrite config.toml via cli flags

* fix lint and refactor

* add extensive tests for flagset

* fix type conversion for big.Int

* add more tests for coverage

* add t.parallel

* internal/cli/flagset: handle flag conversion using interface

* internal/cli/flagset: fix test
2023-09-21 12:52:59 +05:30
rroblak
ebc7dc231a
Add CLI flags to config LevelDB table/total sizes (#981)
* Add CLI flags to config LevelDB table/total sizes

I wired up CLI flags to allow configuring LevelDB table and total sizes:
  - `--leveldb.compaction.table.size`, LevelDB SSTable file size factor in MiB (default: 2)
  - `--leveldb.compaction.table.multiplier`, multiplier on LevelDB SSTable file size (default: 1)
  - `--leveldb.compaction.total.size`, total size factor in MiB of LevelDB levels (default: 10)
  - `--leveldb.compaction.total.multiplier`, multiplier on LevelDB total level size (default: 10)

N.B. that the default values for these configs are exactly the same as
before this changset and so Bor behavior should not change unless these
flags are deliberately overridden. Bor/Geth inherited the default values
from [the `goleveldb`
defaults](126854af5e/leveldb/opt/options.go).

We (Alchemy) found it necessary to override these configs as follows to
keep Bor archive nodes tracking the canonical chain:
  - `--leveldb.compaction.table.size=4`
  - `--leveldb.compaction.total.size=20`

These overrides double the size of LevelDB SSTable files (2 MiB -> 4
MiB) and also the total amount of data in each level (100 MiB -> 200
MiB, 1,000 MiB -> 2,000 MiB, etc.). The idea is to have LevelDB read and
write data in larger chunks while keeping the proportional frequency of
compaction operations the same as in the original defaults defined by
Dean and Ghemawat.

Without these overrides we found that our archive nodes would tend to
fall into a "LevelDB compaction loop of death" where the incoming stream
of blockchain data could not be flowed into LevelDB's structure quickly
enough, resulting in the node blocking writes for long periods of time
while LevelDB's single-threaded compaction organized the data.  Over
time the nodes would fall farther and farther behind the canonical chain
head, metaphorically dying a slow node's death.

These configs can be changed on existing node databases (resyncing is
not necessary). LevelDB appears to work correctly with SSTable files of
different sizes. Note that the database does not undergo any sort of
migration when changing these configs. Only newly-written files (due to
new data or compaction) are affected by these configs.

* Update docs

* Adjust line spacing for linter

* Replace map with `ExtraDBConfig`

* Rename `LevelDbConfig` to `ExtraDBConfig`

* Regenerate docs
2023-09-12 12:09:26 +02:00
Anshal Shukla
18dd5dc643 Update docs and add default config 2023-08-17 11:38:21 +05:30
Anshal Shukla
d342e654f3 add db.engine flag 2023-08-16 10:46:57 +05:30
Manav Darji
cfef444909 add rpc.enabledeprecatedpersonal flag 2023-08-01 15:21:09 +05:30
Jerry
eeb08055bd
Merge pull request #840 from maticnetwork/block-stm
Merge block-stm to develop
2023-05-12 09:33:31 -07:00
Arpit Temani
5fba09885d
Merge pull request #734 from thogard785/fetch-timeout-config
Make txArriveTimeout Configurable w/ CLI Flag
2023-05-02 08:35:07 +05:30
Jerry
a093c4e7df
Merge remote-tracking branch 'upstream/develop' into block-stm 2023-05-01 11:16:05 -07:00
SHIVAM SHARMA
acff69ac3e
core/vm, miner : Make commitInterrupt OPCODE level (#794)
* add : make commitInterrupt OPCODE level

* fix : build fix + lint

* fix : small fix

* chg : use context and add check on commitTransactions

* add : testCommitInterruptExperimentBorContract

* fix : minor fixes

* add : TestCommitInterruptExperimentBorContract working

* fix : lint and context.TODO() to context.Background()

* fix : lint

* fix : context cancel check

* add : metrics for tx/opcode level interrupts + comments

* add : interruptcommit flag

* fix : minor flag fix

* add : caching of interrupted tx

* add : minor improvements on cache

* fix : lint & added comments

* add : GetCurrentTxFromContext and SetCurrentTxOnContext

* fix : unstable testcases giving false negatives

* rm : remove parallelTest from CommitInterrupt Tests

* fix : small changes

* rm : remove t.Parallel() from TestGenerateBlockAndImport tests

* fix : commitInterrupt tests assertion changes
2023-05-01 16:38:35 +05:30
Alex
e62cf51c45 merged in updates to develop branch 2023-04-28 16:37:18 -07:00
Jerry
362db7ad10
Merge remote-tracking branch 'upstream/develop' into block-stm 2023-04-11 13:40:29 -07:00
Manav Darji
ded7767ccd
Merge qa to develop (#814)
* Adding in Mumbai/Mainnet precursor deb packaging for tests to use during upgrade(iterations to come)

* Added changes per discussion in PR, more changes may be necessary

* Adding prerelease true

* Disabling goreleaser

* Removing README swap file

* change bor_dir and add bor user for v0.3.0 release

* rollback bor user and use root

* metrics: handle equal to separated config flag (#596)

* metrics: handle  based config path

* internal/cli/server: add more context to logs

* use space separated flag and value in bor.service

* fixed static-nodes related buf (os independent) (#598)

* fixed static-nodes related buf (os independent)

* taking static-nodes as input if default not present

* Update default flags (#600)

* internal/cli/server: use geth's default for txpool.pricelimit and add comments

* builder/files: update config.toml for mainnet

* packaging/templates: update defaults for mainnet and mumbai

* internal/cli/server: skip overriding cache

* packaging/templates: update cache value for mainnet

* packaging/templates: update gcmode for archive mumbai node

* metrics: handle nil telemetry config (#601)

* resolve merge conflicts

* update go version in release.yml

* update goversion in makefile

* update Docker login for goreleaser-cross v1.19

* Cleanup for the packager to use git tag in the package profile naming. Added conditional check for directory structure, this is in prep for v0.3.1, as this will create a failure on upgrade path in package due to file exist

* added a toml configuration file with comments describing each flag (#607)

* added a toml configuration file with comments describing each flag

* internal/cli/server: update flag description

* docs/cli: update example config and description of flags

* docs: update new-cli docs

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* Adding of 0.3.0 package changes, control file updates, postinst changes, and packager update

* added ancient datadir flag and toml field, need to decide on default value and update the conversion script

* updated toml files with ancient field

* Add support for new flags in new config.toml, which were present in old config.toml (#612)

* added HTTPTimeouts, and TrieTimeout flag in new tol, from old toml

* added RAW fields for these time.Duration flags

* updated the conversion script to support these extra 4 flags

* removed hcl and json config tests as we are only supporting toml config files

* updated toml files with cache.timeout field

* updated toml files with jsonrpc.timeouts field

* tests/bor: expect a call for latest checkpoint

* tests/bor: expect a call for latest checkpoint

* packaging/templates: update cache values for archive nodes

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* remove unwanted code

* Fix docker publish authentication issue

In gorelease-cross 1.19+, dockerhub authentication will require docker
logion action followed by mounting docker config file. See
https://github.com/goreleaser/goreleaser-cross#github-actions.

* Revert "update Docker login for goreleaser-cross v1.19"

This reverts commit 4d19cf5342.

* Bump version to stable

* Revert "Merge pull request #435 from maticnetwork/POS-553"

This reverts commit 657d262def, reversing
changes made to 88dbfa1c13.

* revert change for release for go1.19

* Add default values to CLI helper and docs

This commit adds default values to CLI helper and docs. When the default value of a string flag, slice string flag, or map string flag is empty, its helper message won't show any default value.

* Add a summary of new CLI in docs

* Updating packager as binutils changed version so that apt-get installs current versions

* Add state pruning to new CLI

* Minor wording fix in prune state description

* Bumping control file versions

* Mainnet Delhi fork

* Set version to stable

* change delhi hardfork block number

* handle future chain import and skip peer drop (#650)

* handle future chain import and skip peer drop

* add block import metric

* params: bump version to v0.3.3-stable

* Bump bor version in control files for v0.3.3 mainnet release

* Added checks to RPC requests and introduced new flags to customise the parameters (#657)

* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)

* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)

* updated the config files and docs

* chg : trieTimeout from 60 to 10 mins (#692)

* chg : trieTimeout from 60 to 10 mins

* chg : cache.timout to 10m from 1h in configs

* internal/cli/server : fix : added triesInMemory in config (#691)

* changed version from 0.3.0 to 0.3.4-beta (#693)

* fix nil state-sync issue, increase grpc limit (#695)

* Increase grpc message size limit in pprof

* consensus/bor/bor.go : stateSyncs init fixed [Fix #686]

* eth/filters: handle nil state-sync before notify

* eth/filters: update check

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>

* core, tests/bor: add more tests for state-sync validation (#710)

* core: add get state sync function for tests

* tests/bor: add validation for state sync events post consensus

* Arpit/temp bor sync (#701)

* Increase grpc message size limit in pprof

* ReadBorReceipts improvements

* use internal function

* fix tests

* fetch geth upstread for ReadBorReceiptRLP

* Only query bor receipt when the query index is equal to # tx in block body

This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.

* Revert "fetch geth upstread for ReadBorReceiptRLP"

This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.

* Restore ReadBorReceiptRLP

* fix bor receipts

* remove unused

* fix lints

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)

This reverts commit 241843c7e7.

* Arpit/add execution pool 2 (#719)

* initial

* linters

* linters

* remove timeout

* update pool

* change pool size function

* check nil

* check nil

* fix tests

* Use execution pool from server in all handlers

* simplify things

* test fix

* add support for cli, config

* add to cli and config

* merge base branch

* debug statements

* fix bug

* atomic pointer timeout

* add apis

* update workerpool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* fix tests

* mutex

* refactor flag and value names

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* debug statements

* fix bug

* update workerpool

* atomic pointer timeout

* add apis

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* merge base branch

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* mutex

* fix tests

* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* Change default size of execution pool to 40

* refactor flag and value names

* fix merge conflicts

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* fix linters

* fix go.mod

* change sec to ms

* change default value for ep timeout

* fix node api calls

* comment setter for ep timeout

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* version change (#721)

* Event based pprof (#732)

* feature

* Save pprof to /tmp

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>

* Cherry-pick changes from develop (#738)

* Check if block is nil to prevent panic (#736)

* miner: use env for tracing instead of block object (#728)

---------

Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* add max code init size check in txpool (#739)

* Revert "Event based pprof" and update version (#742)

* Revert "Event based pprof (#732)"

This reverts commit 22fa4033e8.

* params: update version to 0.3.4-beta3

* packaging/templates: update bor version

* internal/ethapi :: Fix : newRPCTransactionFromBlockIndex

* Merge master to qa (#813)

* Merge qa to master (#750)

* Added checks to RPC requests and introduced new flags to customise the parameters (#657)

* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)

* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)

* updated the config files and docs

* chg : trieTimeout from 60 to 10 mins (#692)

* chg : trieTimeout from 60 to 10 mins

* chg : cache.timout to 10m from 1h in configs

* internal/cli/server : fix : added triesInMemory in config (#691)

* changed version from 0.3.0 to 0.3.4-beta (#693)

* fix nil state-sync issue, increase grpc limit (#695)

* Increase grpc message size limit in pprof

* consensus/bor/bor.go : stateSyncs init fixed [Fix #686]

* eth/filters: handle nil state-sync before notify

* eth/filters: update check

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>

* core, tests/bor: add more tests for state-sync validation (#710)

* core: add get state sync function for tests

* tests/bor: add validation for state sync events post consensus

* Arpit/temp bor sync (#701)

* Increase grpc message size limit in pprof

* ReadBorReceipts improvements

* use internal function

* fix tests

* fetch geth upstread for ReadBorReceiptRLP

* Only query bor receipt when the query index is equal to # tx in block body

This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.

* Revert "fetch geth upstread for ReadBorReceiptRLP"

This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.

* Restore ReadBorReceiptRLP

* fix bor receipts

* remove unused

* fix lints

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)

This reverts commit 241843c7e7.

* Arpit/add execution pool 2 (#719)

* initial

* linters

* linters

* remove timeout

* update pool

* change pool size function

* check nil

* check nil

* fix tests

* Use execution pool from server in all handlers

* simplify things

* test fix

* add support for cli, config

* add to cli and config

* merge base branch

* debug statements

* fix bug

* atomic pointer timeout

* add apis

* update workerpool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* fix tests

* mutex

* refactor flag and value names

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* debug statements

* fix bug

* update workerpool

* atomic pointer timeout

* add apis

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* merge base branch

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* mutex

* fix tests

* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* Change default size of execution pool to 40

* refactor flag and value names

* fix merge conflicts

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* fix linters

* fix go.mod

* change sec to ms

* change default value for ep timeout

* fix node api calls

* comment setter for ep timeout

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* version change (#721)

* Event based pprof (#732)

* feature

* Save pprof to /tmp

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>

* Cherry-pick changes from develop (#738)

* Check if block is nil to prevent panic (#736)

* miner: use env for tracing instead of block object (#728)

---------

Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* add max code init size check in txpool (#739)

* Revert "Event based pprof" and update version (#742)

* Revert "Event based pprof (#732)"

This reverts commit 22fa4033e8.

* params: update version to 0.3.4-beta3

* packaging/templates: update bor version

* params, packaging/templates: update bor version

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* core, miner: add sub-spans for tracing (#753)

* core, miner: add sub-spans for tracing

* fix linters

* core: add logs for debugging

* core: add more logs to print tdd while reorg

* fix linters

* core: minor fix

* core: remove debug logs

* core: use different span for write block and set head

* core: use internal context for sending traces (#755)

* core: add : impossible reorg block dump (#754)

* add : impossible reorg block dump

* chg : 3 seperate files for impossoble reorg dump

* add : use exportBlocks method and RLP blocks before writing

* chg : small changes

* bump : go version from 1.19 to 1.20.1 (#761)

* Revert "bump : go version from 1.19 to 1.20.1 (#761)"

This reverts commit 4561012af9.

* core/vm: use optimized bigint (#26021)

* Add holiman/big

* Fix linter

* Bump version to v0.3.5

* fix lints from develop (few lints decided to appear from code that was untouched, weird)

* upgrade crypto lib version (#770)

* bump dep : github.com/Masterminds/goutils to v1.1.1 (#769)

* mardizzone/pos-1313: bump crypto dependency (#772)

* dev: chg: bumd net dependency

* dev: chg: bump crypto dependency

* dev: chg: bump crypto dependency

* bump dep : golang.org/x/net to v0.8.0 (#771)

* Verify validator set against local contract on receiving an end-of-sprint block (#768)

* Verify validator set against local contract on receiving an end-of-sprint block

* Fix tests

* Respect error returned by ParseValidators

* Keep going back until a parent block presents

* core/txpool: implement DoS defenses from geth (#778)

* Hotfixes and deps bump (#776)

* dev: chg: bump deps

* internal/cli/server, rpc: lower down http readtimeout to 10s

* dev: chg: get p2p adapter

* dev: chg: lower down jsonrpc readtimeout to 10s

* cherry-pick txpool optimisation changes

* add check for empty lists in txpool (#704)

* add check

* linters

* core, miner: add empty instrumentation name for tracing

---------

Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Evgeny Danilenko <6655321@bk.ru>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* packaging,params: bump to v0.3.6 (#782)

* v0.3.6 fix (#787)

* Fix get validator set in header verifier

* chg : commit tx logs from info to debug (#673)

* chg : commit tx logs from info to debug

* fix : minor changes

* chg : miner : commitTransactions-stats moved from info to debug

* lint : fix linters

* refactor logging

* miner : chg : UnauthorizedSignerError to debug

* lint : fix lint

* fix : log.Logger interface compatibility

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Remove unnecessary sorting of valset from header in verification

* dev: chg: version bump

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>

* core: improve locks in txpool (#807)

* added a write lock to the txs.filter method and a read lock to the txs.reheap method - both of which are called by Filter during reorg adjustments to txpool

* txpool reorg locks

* more locks

* locks

* linters

* params, packaging: update version for v0.3.8-beta release

* core: add logs in reheap

---------

Co-authored-by: Alex <dalexwatts@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Merge qa to master (#808)

* Added checks to RPC requests and introduced new flags to customise the parameters (#657)

* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)

* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)

* updated the config files and docs

* chg : trieTimeout from 60 to 10 mins (#692)

* chg : trieTimeout from 60 to 10 mins

* chg : cache.timout to 10m from 1h in configs

* internal/cli/server : fix : added triesInMemory in config (#691)

* changed version from 0.3.0 to 0.3.4-beta (#693)

* fix nil state-sync issue, increase grpc limit (#695)

* Increase grpc message size limit in pprof

* consensus/bor/bor.go : stateSyncs init fixed [Fix #686]

* eth/filters: handle nil state-sync before notify

* eth/filters: update check

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>

* core, tests/bor: add more tests for state-sync validation (#710)

* core: add get state sync function for tests

* tests/bor: add validation for state sync events post consensus

* Arpit/temp bor sync (#701)

* Increase grpc message size limit in pprof

* ReadBorReceipts improvements

* use internal function

* fix tests

* fetch geth upstread for ReadBorReceiptRLP

* Only query bor receipt when the query index is equal to # tx in block body

This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.

* Revert "fetch geth upstread for ReadBorReceiptRLP"

This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.

* Restore ReadBorReceiptRLP

* fix bor receipts

* remove unused

* fix lints

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>

* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)

This reverts commit 241843c7e7.

* Arpit/add execution pool 2 (#719)

* initial

* linters

* linters

* remove timeout

* update pool

* change pool size function

* check nil

* check nil

* fix tests

* Use execution pool from server in all handlers

* simplify things

* test fix

* add support for cli, config

* add to cli and config

* merge base branch

* debug statements

* fix bug

* atomic pointer timeout

* add apis

* update workerpool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* fix tests

* mutex

* refactor flag and value names

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* debug statements

* fix bug

* update workerpool

* atomic pointer timeout

* add apis

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* fix issues

* change params

* fix issues

* fix ipc issue

* remove execution pool from IPC

* revert

* merge base branch

* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* mutex

* fix tests

* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool

* Change default size of execution pool to 40

* refactor flag and value names

* fix merge conflicts

* ordering fix

* refactor flag and value names

* update default ep size to 40

* fix bor start issues

* revert file changes

* fix linters

* fix go.mod

* change sec to ms

* change default value for ep timeout

* fix node api calls

* comment setter for ep timeout

---------

Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* version change (#721)

* Event based pprof (#732)

* feature

* Save pprof to /tmp

---------

Co-authored-by: Jerry <jerrycgh@gmail.com>

* Cherry-pick changes from develop (#738)

* Check if block is nil to prevent panic (#736)

* miner: use env for tracing instead of block object (#728)

---------

Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* add max code init size check in txpool (#739)

* Revert "Event based pprof" and update version (#742)

* Revert "Event based pprof (#732)"

This reverts commit 22fa4033e8.

* params: update version to 0.3.4-beta3

* packaging/templates: update bor version

* internal/ethapi :: Fix : newRPCTransactionFromBlockIndex

* fix: remove assignment for bor receipt

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>

* Setting up bor to use hosted 18.04 runner as ubuntu provided 18.04 runner is end of life

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>
Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com>
Co-authored-by: Alex <dalexwatts@gmail.com>
Co-authored-by: Daniel Jones <djones@polygon.technology>

* core: remove duplicate tests

* miner: use get validators by hash in tests

---------

Co-authored-by: Daniel Jones <djones@polygon.technology>
Co-authored-by: Will Button <wbutton@polygon.technology>
Co-authored-by: Will Button <will@willbutton.com>
Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>
Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com>
Co-authored-by: Alex <dalexwatts@gmail.com>
2023-04-10 16:21:18 +05:30
Alex
1e926cfad9 merged in the current develop branch 2023-04-04 23:55:30 -07:00
Pratik Patil
584856b52d
internal/cli: added missing flags (#744)
* minor comment update

* added support for rpc.evmtimeout flag

* added support for vmdebug (EnablePreimageRecording) flag

* added support for jsonrpc.auth.(jwtsecret, addr, port, vhosts) flags

* added support for miner.recommit flag

* added support for gpo.maxheaderhistory and gpo.maxblockhistory flag

* Revert "added support for miner.recommit flag"

This reverts commit fcb722e4bf5bdda23d4c91da40ab95c9023a80c3.

* added pprof related flags (expect --pprof.cpuprofile - Write CPU profile to the given file)

* added support for --dev.gaslimit flag

* added support for --fdlimit flag

* added support for --netrestrict flag

* added support for --nodekey and --nodekeyhex flag

* added support for --vmodule, --log.json, --log.backtrace, and --log.debug flags

* fixed related lint errors

* fix lints from develop (few lints decided to appear from code that was untouched, weird)

* more weird lints from develop

* small precautionary fix

* small bug ;) fix in NetRestrict

* weird lints

* change vmdebug = true to vmdebug = false
2023-03-20 17:28:56 +05:30
Pratik Patil
238b4496d8
added support for miner.recommit flag (#743) 2023-03-17 16:13:23 +05:30
Raneet Debnath
c917e6fe68
Merge pull request #705 from maticnetwork/raneet10/pos-1126
core,eth,internal/cli,internal/ethapi: add --rpc.allow-unprotected-txs flag to allow txs to get replayed (for shadow node)
2023-03-16 15:00:28 +05:30
Krishna Upadhyaya
39b37b6f5c
Heimdall App implementation (#646) 2023-03-06 16:57:28 +05:30
Pratik Patil
c4f33329ea
Added verbosity flag, supports log-level as well, but will remove that in future. (#722)
* changed log-level flag back to verbosity, and updated the conversion script

* supporting both verbosity and log-level, with a message to deprecat log-level

* converted verbosity to a int value
2023-02-10 08:49:32 +05:30
Alex
3c23de2486 changed txArrivalWait config type from int to time.Duration, changed flags to duration type. Tested on live both w/o flag set (default) and w/ flag set 2023-02-09 01:52:19 -08:00
Alex
b5ff754b70 updated default txArrivalWait value to 500ms. updated commented lines to indicate the new default value. updated tx_fetcher_test to test using the new default value 2023-02-09 00:18:39 -08:00
Pratik Patil
9795a287d0
added 2 flags to enable parallel EVM and set the number of speculative processes (#727) 2023-02-09 13:34:06 +05:30
SHIVAM SHARMA
a4f1ac1500
consensus/bor : add : devFakeAuthor flag (#697) 2023-02-09 12:10:32 +05:30
Alex
f6903b4f57 added flags, carried flag var through p2p config to backend start func to handler to fetcher. TODO verify default, expose var to api 2023-02-04 18:15:54 -08:00
Raneet Debnath
cb973283bc core,eth,internal/cli,internal/ethapi: add --rpc.allow-unprotected-txs flag to allow txs to get replayed (for shadow node) 2023-01-27 16:05:37 +05:30
Shivam Sharma
2ad6dcf53b consensus/bor : add : devFakeAuthor flag 2023-01-20 17:13:36 +05:30
SHIVAM SHARMA
9aeaf03f05
Merge branch 'qa' and 'master' into develop (#663)
* Adding in Mumbai/Mainnet precursor deb packaging for tests to use during upgrade(iterations to come)

* Added changes per discussion in PR, more changes may be necessary

* Adding prerelease true

* Disabling goreleaser

* Removing README swap file

* change bor_dir and add bor user for v0.3.0 release

* rollback bor user and use root

* metrics: handle equal to separated config flag (#596)

* metrics: handle  based config path

* internal/cli/server: add more context to logs

* use space separated flag and value in bor.service

* fixed static-nodes related buf (os independent) (#598)

* fixed static-nodes related buf (os independent)

* taking static-nodes as input if default not present

* Update default flags (#600)

* internal/cli/server: use geth's default for txpool.pricelimit and add comments

* builder/files: update config.toml for mainnet

* packaging/templates: update defaults for mainnet and mumbai

* internal/cli/server: skip overriding cache

* packaging/templates: update cache value for mainnet

* packaging/templates: update gcmode for archive mumbai node

* metrics: handle nil telemetry config (#601)

* resolve merge conflicts

* update go version in release.yml

* update goversion in makefile

* update Docker login for goreleaser-cross v1.19

* Cleanup for the packager to use git tag in the package profile naming. Added conditional check for directory structure, this is in prep for v0.3.1, as this will create a failure on upgrade path in package due to file exist

* added a toml configuration file with comments describing each flag (#607)

* added a toml configuration file with comments describing each flag

* internal/cli/server: update flag description

* docs/cli: update example config and description of flags

* docs: update new-cli docs

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* Adding of 0.3.0 package changes, control file updates, postinst changes, and packager update

* added ancient datadir flag and toml field, need to decide on default value and update the conversion script

* updated toml files with ancient field

* Add support for new flags in new config.toml, which were present in old config.toml (#612)

* added HTTPTimeouts, and TrieTimeout flag in new tol, from old toml

* added RAW fields for these time.Duration flags

* updated the conversion script to support these extra 4 flags

* removed hcl and json config tests as we are only supporting toml config files

* updated toml files with cache.timeout field

* updated toml files with jsonrpc.timeouts field

* tests/bor: expect a call for latest checkpoint

* tests/bor: expect a call for latest checkpoint

* packaging/templates: update cache values for archive nodes

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* remove unwanted code

* Fix docker publish authentication issue

In gorelease-cross 1.19+, dockerhub authentication will require docker
logion action followed by mounting docker config file. See
https://github.com/goreleaser/goreleaser-cross#github-actions.

* Revert "update Docker login for goreleaser-cross v1.19"

This reverts commit 4d19cf5342.

* Bump version to stable

* Revert "Merge pull request #435 from maticnetwork/POS-553"

This reverts commit 657d262def, reversing
changes made to 88dbfa1c13.

* revert change for release for go1.19

* Add default values to CLI helper and docs

This commit adds default values to CLI helper and docs. When the default value of a string flag, slice string flag, or map string flag is empty, its helper message won't show any default value.

* Add a summary of new CLI in docs

* Updating packager as binutils changed version so that apt-get installs current versions

* Add state pruning to new CLI

* Minor wording fix in prune state description

* Bumping control file versions

* Mainnet Delhi fork

* Set version to stable

* change delhi hardfork block number

* handle future chain import and skip peer drop (#650)

* handle future chain import and skip peer drop

* add block import metric

* params: bump version to v0.3.3-stable

* Bump bor version in control files for v0.3.3 mainnet release

Co-authored-by: Daniel Jones <djones@polygon.technology>
Co-authored-by: Will Button <wbutton@polygon.technology>
Co-authored-by: Will Button <will@willbutton.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
2023-01-13 14:47:14 +05:30
Krishna Upadhyaya
ad69ccd0ba
Added flags to run heimdall as a child process (#597)
* Added flags to run heimdall as a child process

* Fix: Lint

* Fix btcd package dependency for CI

* Update btcd package version

* Try removing ambigious importts

* dev: fix: lint fix for parallel tests

* remove delete for ambigious import

* Remove unwanted space

* go mod tidy

* try replace

* try replace

* use vendor

* rename vendor

* tidy

* vendor btcec

* clean up

* remove submodule

* remove submodule

* remove submodule

* remove submodule

* remove vendor & added replacer in test

* go mod tidy

* added replacer

* Update replacer

* Update replacer

* Merge branch 'develop' into run-heimdall-flags

* Skip TestGolangBindings

* Typo fix

* Remove unwanted changes

Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
2022-12-23 23:15:46 +05:30
Pratik Patil
1ab225c0e4
Removed vhosts for ws and replaced cors with origins (#574)
* removed vhosts for ws and added replaced cors with origins

* updated script
2022-11-03 12:54:57 -07:00
Shivam Sharma
225a826e0e add : TriesInMemory flag 2022-10-17 18:00:30 +05:30
Manav Darji
54f97b8598
internal/cli: add support for bor.logs flag in new-cli (#541)
* add support for bor.logs flag in new-cli

* handle bor.withoutheimdall flag commenting in conversion script
2022-10-05 23:38:19 +05:30
Pratik Patil
77db80cc91
Added script to generate config.toml fromstart.sh (#518)
* added go and bash script to get config out of start.sh and updated flagset.go

* changed 'requiredblocks' flag back to 'eth.requiredblocks'

* updated script

* changed 'requiredblocks' flag back to 'eth.requiredblocks'

* updated tests, and removed requiredblocks from json and hcl

* addressed comments
2022-09-23 13:37:04 +05:30
Krishna Upadhyaya
75bb3bdeb1 Merge branch 'develop' into POS-648 2022-07-27 19:09:04 +05:30
Pratik Patil
71980bf5b1
internal/cli: use config file for flags in new-cli, builder/files: update defaults (#457)
* updated simple.json and simple.hcl

* added annotations for developer and grpc block

* added toml tags and simple.toml file

* added support for toml config files

* updated simple files toml, hcl, json

* added config.toml in builder/files and updated bor.service

* add dumpconfig command in cli for exporting configs

* update docs

* updated .goreleaser.yml (POS-651)

* changed --config to -config

* added test config for tests and fixed lint errors

* made fields of type big.int and time.Duration private, removed merge from dumpconfig, setting up default values to the Raw fields in dumpconfig, and fixed one lint error

* fixed lint errors (strange)

* lint fix

* private no-more, using '-' in name tags to ignore

* updated name tags, made c.configFile as a stringFlag (only one config file supported) and updated the merge logic in command.go -> Run()

* fix: set method for big.Int flags, added a TODO

* handeled bigInt and timeDuration type, bug fix in config_legacy, lint fix

* updated flags, consistent across flags.go and config.go

* fixed config test and updated test hcl, json config files

* updated config legacy test

* added test to check values of cmd flags, restructured Run in command.go, linter fix

* fix linters

* lint again

* changed 2 flags and assert -> require

* changed the 2 flags back

* updated correct congig.toml path and made mainnet default

* updated config.toml with new flags

* added sample config (toml) file

* removed sample-config.toml and added it in docs/config.md

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
2022-07-27 12:28:26 +05:30
Krishna Upadhyaya
385aff85f0 Implemented heimdall gRPC client 2022-07-26 02:18:37 +05:30
Pratik Patil
3df1d6f017
Change diverged flags back to Geth's convention (POS-602) (#451)
* changed name -> identity

* changed no-snapshot -> snapchot=true/false

* changed jsonrpc.corsdomain -> http.corsdomain

* changed jsonrpc.vhosts -> http.vhosts

* changed http/ws.modules to http/ws.api

* updated readme

* updated config_test

* make docs

* handelling string array flag, overwrite insted of append

* added 'Default' to SliceStringFlag

* added separate flags for corsdomain and vhosts for http, ws, graphql

* modified tests
2022-07-14 22:36:18 +05:30
Jerry
33e646eea1 Add keystore option to cli 2022-06-15 11:30:47 -07:00
Jerry
96ac8d16d4 Overwrite default config with CLI input even when the input value is considered as empty
This change will fix two issues:
1. When CLI sets a flag to an empty value in Go, e.g. 0 as to Uint64, the flag will be skipped and never set correctly. This problem could be solved by using option "WithOverwriteWithEmptyValue" when merge two configs.
2. The default non-empty value in server config will be overwritten to an empty value after flag initialization. This problem is solved by explicitly providing default value to all flags that have a default value option.
2022-05-31 20:30:40 -07:00
Jerry
63bedf374d
Merge pull request #405 from cffls/whitelist
Rename whitelist to requiredblocks
2022-05-23 09:18:48 -07:00