Verkle trees store the code inside the trie. This PR changes the interface to pass the code, as well as the dirty flag to tell the trie package if the code is dirty and needs to be updated. This is a no-op for the MPT and the odr trie.
The state availability is checked during the creation of a state reader.
- In hash-based database, if the specified root node does not exist on disk disk, then
the state reader won't be created and an error will be returned.
- In path-based database, if the specified state layer is not available, then the
state reader won't be created and an error will be returned.
This change also contains a stricter semantics regarding the `Commit` operation: once it has been performed, the trie is no longer usable, and certain operations will return an error.
This removes the feature where top nodes of the proof can be elided.
It was intended to be used by the LES server, to save bandwidth
when the client had already fetched parts of the state and only needed
some extra nodes to complete the proof. Alas, it never got implemented
in the client.
* go.mod: update kzg libraries to use big-endian
* go.sum: ran go mod tidy
* core/testdata/precompiles: fix blob verification test
* core/testdata/precompiles: fix blob verification test
This change ensures Reheap will be called even before the London fork activates.
Since Reheap would otherwise only be called through `SetBaseFee` after London,
the list would just keep growing if the fork was not enabled or not reached yet.
* all: move main transaction pool into a subpool
* go.mod: remove superfluous updates
* core/txpool: review fixes, handle txs rejected by all subpools
* core/txpool: typos
The logs in this function are pulled straight from disk in rawdb.ReadRawReceipts and
also modified in receipts.DeriveFields, so removing the copy should be fine.
A reverted transaction, X, can incorrectly write to a slot, which might be read by another transaction, Y. If we skip checking states written by X, then Y might end up with reading the wrong value and we won't be able to detect this error.
* core/txpool: abstraction prep work for secondary pools (blob pool)
* core/txpool: leave subpool concepts to a followup pr
* les: fix tests using hard coded errors
* core/txpool: use bitmaps instead of maps for tx type filtering
An address's nonce and balance will get updated even if its transaction is reverted. Therefore, we need to keep balance and nonce in write list in order to check the transaction's validity in parallel execution.
This changes the journal logic to mark the state object dirty immediately when it
is reset.
We're mostly adding this change to appease the fuzzer. Marking it dirty immediately
makes no difference in practice because accounts will always be modified by EVM
right after creation.
Continuing with a series of PRs to make the Trie interface more generic, this PR moves
the RLP encoding of storage slots inside the StateTrie and light.Trie implementations,
as other types of tries don't use RLP.
This PR adds a staleness-check to AccountRLP, before checking the bloom-filter and potentially going directly into the disklayer.
---------
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Regenerate receipt json code to remove omit empty. Previously, there was a discrepancy between the generated code and the source.
---------
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
* cmd/utils, node: switch to Pebble as the default db if none exists
* node: fall back to LevelDB on platforms not supporting Pebble
* core/rawdb, node: default to Pebble at the node level
* cmd/geth: fix some tests explicitly using leveldb
* ethdb/pebble: allow double closes, makes tests simpler
In this PR, all TryXXX(e.g. TryGet) APIs of trie are renamed to XXX(e.g. Get) with an error returned.
The original XXX(e.g. Get) APIs are renamed to MustXXX(e.g. MustGet) and does not return any error -- they print a log output. A future PR will change the behaviour to panic on errorrs.
The EIP150Hash was an idea where, after the fork, we hardcoded the forked hash as an extra defensive mechanism. It wasn't really used, since forks weren't contentious and for all the various testnets and private networks it's been a hassle to have around.
This change removes that config field.
---------
Signed-off-by: jsvisa <delweng@gmail.com>
* Adding in Mumbai/Mainnet precursor deb packaging for tests to use during upgrade(iterations to come)
* Added changes per discussion in PR, more changes may be necessary
* Adding prerelease true
* Disabling goreleaser
* Removing README swap file
* change bor_dir and add bor user for v0.3.0 release
* rollback bor user and use root
* metrics: handle equal to separated config flag (#596)
* metrics: handle based config path
* internal/cli/server: add more context to logs
* use space separated flag and value in bor.service
* fixed static-nodes related buf (os independent) (#598)
* fixed static-nodes related buf (os independent)
* taking static-nodes as input if default not present
* Update default flags (#600)
* internal/cli/server: use geth's default for txpool.pricelimit and add comments
* builder/files: update config.toml for mainnet
* packaging/templates: update defaults for mainnet and mumbai
* internal/cli/server: skip overriding cache
* packaging/templates: update cache value for mainnet
* packaging/templates: update gcmode for archive mumbai node
* metrics: handle nil telemetry config (#601)
* resolve merge conflicts
* update go version in release.yml
* update goversion in makefile
* update Docker login for goreleaser-cross v1.19
* Cleanup for the packager to use git tag in the package profile naming. Added conditional check for directory structure, this is in prep for v0.3.1, as this will create a failure on upgrade path in package due to file exist
* added a toml configuration file with comments describing each flag (#607)
* added a toml configuration file with comments describing each flag
* internal/cli/server: update flag description
* docs/cli: update example config and description of flags
* docs: update new-cli docs
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
* Adding of 0.3.0 package changes, control file updates, postinst changes, and packager update
* added ancient datadir flag and toml field, need to decide on default value and update the conversion script
* updated toml files with ancient field
* Add support for new flags in new config.toml, which were present in old config.toml (#612)
* added HTTPTimeouts, and TrieTimeout flag in new tol, from old toml
* added RAW fields for these time.Duration flags
* updated the conversion script to support these extra 4 flags
* removed hcl and json config tests as we are only supporting toml config files
* updated toml files with cache.timeout field
* updated toml files with jsonrpc.timeouts field
* tests/bor: expect a call for latest checkpoint
* tests/bor: expect a call for latest checkpoint
* packaging/templates: update cache values for archive nodes
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
* remove unwanted code
* Fix docker publish authentication issue
In gorelease-cross 1.19+, dockerhub authentication will require docker
logion action followed by mounting docker config file. See
https://github.com/goreleaser/goreleaser-cross#github-actions.
* Revert "update Docker login for goreleaser-cross v1.19"
This reverts commit 4d19cf5342.
* Bump version to stable
* Revert "Merge pull request #435 from maticnetwork/POS-553"
This reverts commit 657d262def, reversing
changes made to 88dbfa1c13.
* revert change for release for go1.19
* Add default values to CLI helper and docs
This commit adds default values to CLI helper and docs. When the default value of a string flag, slice string flag, or map string flag is empty, its helper message won't show any default value.
* Add a summary of new CLI in docs
* Updating packager as binutils changed version so that apt-get installs current versions
* Add state pruning to new CLI
* Minor wording fix in prune state description
* Bumping control file versions
* Mainnet Delhi fork
* Set version to stable
* change delhi hardfork block number
* handle future chain import and skip peer drop (#650)
* handle future chain import and skip peer drop
* add block import metric
* params: bump version to v0.3.3-stable
* Bump bor version in control files for v0.3.3 mainnet release
* Added checks to RPC requests and introduced new flags to customise the parameters (#657)
* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)
* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)
* updated the config files and docs
* chg : trieTimeout from 60 to 10 mins (#692)
* chg : trieTimeout from 60 to 10 mins
* chg : cache.timout to 10m from 1h in configs
* internal/cli/server : fix : added triesInMemory in config (#691)
* changed version from 0.3.0 to 0.3.4-beta (#693)
* fix nil state-sync issue, increase grpc limit (#695)
* Increase grpc message size limit in pprof
* consensus/bor/bor.go : stateSyncs init fixed [Fix#686]
* eth/filters: handle nil state-sync before notify
* eth/filters: update check
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
* core, tests/bor: add more tests for state-sync validation (#710)
* core: add get state sync function for tests
* tests/bor: add validation for state sync events post consensus
* Arpit/temp bor sync (#701)
* Increase grpc message size limit in pprof
* ReadBorReceipts improvements
* use internal function
* fix tests
* fetch geth upstread for ReadBorReceiptRLP
* Only query bor receipt when the query index is equal to # tx in block body
This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.
* Revert "fetch geth upstread for ReadBorReceiptRLP"
This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.
* Restore ReadBorReceiptRLP
* fix bor receipts
* remove unused
* fix lints
---------
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)
This reverts commit 241843c7e7.
* Arpit/add execution pool 2 (#719)
* initial
* linters
* linters
* remove timeout
* update pool
* change pool size function
* check nil
* check nil
* fix tests
* Use execution pool from server in all handlers
* simplify things
* test fix
* add support for cli, config
* add to cli and config
* merge base branch
* debug statements
* fix bug
* atomic pointer timeout
* add apis
* update workerpool
* fix issues
* change params
* fix issues
* fix ipc issue
* remove execution pool from IPC
* revert
* fix tests
* mutex
* refactor flag and value names
* ordering fix
* refactor flag and value names
* update default ep size to 40
* fix bor start issues
* revert file changes
* debug statements
* fix bug
* update workerpool
* atomic pointer timeout
* add apis
* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool
* fix issues
* change params
* fix issues
* fix ipc issue
* remove execution pool from IPC
* revert
* merge base branch
* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool
* mutex
* fix tests
* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool
* Change default size of execution pool to 40
* refactor flag and value names
* fix merge conflicts
* ordering fix
* refactor flag and value names
* update default ep size to 40
* fix bor start issues
* revert file changes
* fix linters
* fix go.mod
* change sec to ms
* change default value for ep timeout
* fix node api calls
* comment setter for ep timeout
---------
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
* version change (#721)
* Event based pprof (#732)
* feature
* Save pprof to /tmp
---------
Co-authored-by: Jerry <jerrycgh@gmail.com>
* Cherry-pick changes from develop (#738)
* Check if block is nil to prevent panic (#736)
* miner: use env for tracing instead of block object (#728)
---------
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
* add max code init size check in txpool (#739)
* Revert "Event based pprof" and update version (#742)
* Revert "Event based pprof (#732)"
This reverts commit 22fa4033e8.
* params: update version to 0.3.4-beta3
* packaging/templates: update bor version
* internal/ethapi :: Fix : newRPCTransactionFromBlockIndex
* Merge master to qa (#813)
* Merge qa to master (#750)
* Added checks to RPC requests and introduced new flags to customise the parameters (#657)
* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)
* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)
* updated the config files and docs
* chg : trieTimeout from 60 to 10 mins (#692)
* chg : trieTimeout from 60 to 10 mins
* chg : cache.timout to 10m from 1h in configs
* internal/cli/server : fix : added triesInMemory in config (#691)
* changed version from 0.3.0 to 0.3.4-beta (#693)
* fix nil state-sync issue, increase grpc limit (#695)
* Increase grpc message size limit in pprof
* consensus/bor/bor.go : stateSyncs init fixed [Fix#686]
* eth/filters: handle nil state-sync before notify
* eth/filters: update check
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
* core, tests/bor: add more tests for state-sync validation (#710)
* core: add get state sync function for tests
* tests/bor: add validation for state sync events post consensus
* Arpit/temp bor sync (#701)
* Increase grpc message size limit in pprof
* ReadBorReceipts improvements
* use internal function
* fix tests
* fetch geth upstread for ReadBorReceiptRLP
* Only query bor receipt when the query index is equal to # tx in block body
This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.
* Revert "fetch geth upstread for ReadBorReceiptRLP"
This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.
* Restore ReadBorReceiptRLP
* fix bor receipts
* remove unused
* fix lints
---------
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)
This reverts commit 241843c7e7.
* Arpit/add execution pool 2 (#719)
* initial
* linters
* linters
* remove timeout
* update pool
* change pool size function
* check nil
* check nil
* fix tests
* Use execution pool from server in all handlers
* simplify things
* test fix
* add support for cli, config
* add to cli and config
* merge base branch
* debug statements
* fix bug
* atomic pointer timeout
* add apis
* update workerpool
* fix issues
* change params
* fix issues
* fix ipc issue
* remove execution pool from IPC
* revert
* fix tests
* mutex
* refactor flag and value names
* ordering fix
* refactor flag and value names
* update default ep size to 40
* fix bor start issues
* revert file changes
* debug statements
* fix bug
* update workerpool
* atomic pointer timeout
* add apis
* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool
* fix issues
* change params
* fix issues
* fix ipc issue
* remove execution pool from IPC
* revert
* merge base branch
* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool
* mutex
* fix tests
* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool
* Change default size of execution pool to 40
* refactor flag and value names
* fix merge conflicts
* ordering fix
* refactor flag and value names
* update default ep size to 40
* fix bor start issues
* revert file changes
* fix linters
* fix go.mod
* change sec to ms
* change default value for ep timeout
* fix node api calls
* comment setter for ep timeout
---------
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
* version change (#721)
* Event based pprof (#732)
* feature
* Save pprof to /tmp
---------
Co-authored-by: Jerry <jerrycgh@gmail.com>
* Cherry-pick changes from develop (#738)
* Check if block is nil to prevent panic (#736)
* miner: use env for tracing instead of block object (#728)
---------
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
* add max code init size check in txpool (#739)
* Revert "Event based pprof" and update version (#742)
* Revert "Event based pprof (#732)"
This reverts commit 22fa4033e8.
* params: update version to 0.3.4-beta3
* packaging/templates: update bor version
* params, packaging/templates: update bor version
---------
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
* core, miner: add sub-spans for tracing (#753)
* core, miner: add sub-spans for tracing
* fix linters
* core: add logs for debugging
* core: add more logs to print tdd while reorg
* fix linters
* core: minor fix
* core: remove debug logs
* core: use different span for write block and set head
* core: use internal context for sending traces (#755)
* core: add : impossible reorg block dump (#754)
* add : impossible reorg block dump
* chg : 3 seperate files for impossoble reorg dump
* add : use exportBlocks method and RLP blocks before writing
* chg : small changes
* bump : go version from 1.19 to 1.20.1 (#761)
* Revert "bump : go version from 1.19 to 1.20.1 (#761)"
This reverts commit 4561012af9.
* core/vm: use optimized bigint (#26021)
* Add holiman/big
* Fix linter
* Bump version to v0.3.5
* fix lints from develop (few lints decided to appear from code that was untouched, weird)
* upgrade crypto lib version (#770)
* bump dep : github.com/Masterminds/goutils to v1.1.1 (#769)
* mardizzone/pos-1313: bump crypto dependency (#772)
* dev: chg: bumd net dependency
* dev: chg: bump crypto dependency
* dev: chg: bump crypto dependency
* bump dep : golang.org/x/net to v0.8.0 (#771)
* Verify validator set against local contract on receiving an end-of-sprint block (#768)
* Verify validator set against local contract on receiving an end-of-sprint block
* Fix tests
* Respect error returned by ParseValidators
* Keep going back until a parent block presents
* core/txpool: implement DoS defenses from geth (#778)
* Hotfixes and deps bump (#776)
* dev: chg: bump deps
* internal/cli/server, rpc: lower down http readtimeout to 10s
* dev: chg: get p2p adapter
* dev: chg: lower down jsonrpc readtimeout to 10s
* cherry-pick txpool optimisation changes
* add check for empty lists in txpool (#704)
* add check
* linters
* core, miner: add empty instrumentation name for tracing
---------
Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Evgeny Danilenko <6655321@bk.ru>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
* packaging,params: bump to v0.3.6 (#782)
* v0.3.6 fix (#787)
* Fix get validator set in header verifier
* chg : commit tx logs from info to debug (#673)
* chg : commit tx logs from info to debug
* fix : minor changes
* chg : miner : commitTransactions-stats moved from info to debug
* lint : fix linters
* refactor logging
* miner : chg : UnauthorizedSignerError to debug
* lint : fix lint
* fix : log.Logger interface compatibility
---------
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
* Remove unnecessary sorting of valset from header in verification
* dev: chg: version bump
---------
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>
* core: improve locks in txpool (#807)
* added a write lock to the txs.filter method and a read lock to the txs.reheap method - both of which are called by Filter during reorg adjustments to txpool
* txpool reorg locks
* more locks
* locks
* linters
* params, packaging: update version for v0.3.8-beta release
* core: add logs in reheap
---------
Co-authored-by: Alex <dalexwatts@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
* Merge qa to master (#808)
* Added checks to RPC requests and introduced new flags to customise the parameters (#657)
* added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit)
* added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit)
* updated the config files and docs
* chg : trieTimeout from 60 to 10 mins (#692)
* chg : trieTimeout from 60 to 10 mins
* chg : cache.timout to 10m from 1h in configs
* internal/cli/server : fix : added triesInMemory in config (#691)
* changed version from 0.3.0 to 0.3.4-beta (#693)
* fix nil state-sync issue, increase grpc limit (#695)
* Increase grpc message size limit in pprof
* consensus/bor/bor.go : stateSyncs init fixed [Fix#686]
* eth/filters: handle nil state-sync before notify
* eth/filters: update check
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
* core, tests/bor: add more tests for state-sync validation (#710)
* core: add get state sync function for tests
* tests/bor: add validation for state sync events post consensus
* Arpit/temp bor sync (#701)
* Increase grpc message size limit in pprof
* ReadBorReceipts improvements
* use internal function
* fix tests
* fetch geth upstread for ReadBorReceiptRLP
* Only query bor receipt when the query index is equal to # tx in block body
This change reduces the frequency of calling ReadBorReceipt and
ReadBorTransaction, which are CPU and db intensive.
* Revert "fetch geth upstread for ReadBorReceiptRLP"
This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0.
* Restore ReadBorReceiptRLP
* fix bor receipts
* remove unused
* fix lints
---------
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
* Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720)
This reverts commit 241843c7e7.
* Arpit/add execution pool 2 (#719)
* initial
* linters
* linters
* remove timeout
* update pool
* change pool size function
* check nil
* check nil
* fix tests
* Use execution pool from server in all handlers
* simplify things
* test fix
* add support for cli, config
* add to cli and config
* merge base branch
* debug statements
* fix bug
* atomic pointer timeout
* add apis
* update workerpool
* fix issues
* change params
* fix issues
* fix ipc issue
* remove execution pool from IPC
* revert
* fix tests
* mutex
* refactor flag and value names
* ordering fix
* refactor flag and value names
* update default ep size to 40
* fix bor start issues
* revert file changes
* debug statements
* fix bug
* update workerpool
* atomic pointer timeout
* add apis
* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool
* fix issues
* change params
* fix issues
* fix ipc issue
* remove execution pool from IPC
* revert
* merge base branch
* Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool
* mutex
* fix tests
* Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool
* Change default size of execution pool to 40
* refactor flag and value names
* fix merge conflicts
* ordering fix
* refactor flag and value names
* update default ep size to 40
* fix bor start issues
* revert file changes
* fix linters
* fix go.mod
* change sec to ms
* change default value for ep timeout
* fix node api calls
* comment setter for ep timeout
---------
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
* version change (#721)
* Event based pprof (#732)
* feature
* Save pprof to /tmp
---------
Co-authored-by: Jerry <jerrycgh@gmail.com>
* Cherry-pick changes from develop (#738)
* Check if block is nil to prevent panic (#736)
* miner: use env for tracing instead of block object (#728)
---------
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
* add max code init size check in txpool (#739)
* Revert "Event based pprof" and update version (#742)
* Revert "Event based pprof (#732)"
This reverts commit 22fa4033e8.
* params: update version to 0.3.4-beta3
* packaging/templates: update bor version
* internal/ethapi :: Fix : newRPCTransactionFromBlockIndex
* fix: remove assignment for bor receipt
---------
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
* Setting up bor to use hosted 18.04 runner as ubuntu provided 18.04 runner is end of life
---------
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>
Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com>
Co-authored-by: Alex <dalexwatts@gmail.com>
Co-authored-by: Daniel Jones <djones@polygon.technology>
* core: remove duplicate tests
* miner: use get validators by hash in tests
---------
Co-authored-by: Daniel Jones <djones@polygon.technology>
Co-authored-by: Will Button <wbutton@polygon.technology>
Co-authored-by: Will Button <will@willbutton.com>
Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
Co-authored-by: Daniil <daniil.melnik@chainstack.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: marcello33 <marcelloardizzone@hotmail.it>
Co-authored-by: Raneet Debnath <raneetdebnath10@gmail.com>
Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com>
Co-authored-by: Alex <dalexwatts@gmail.com>
This PR unifies the error handling in miner.
Whenever an error occur while applying a transaction, the transaction should be regarded as invalid and all following transactions from the same sender not executable because of the nonce restriction. The only exception is the `nonceTooLow` error which is handled separately.
Prior to this change, it was possible that transactions are erroneously deemed as 'future' although they are in fact 'pending', causing them to be dropped due to 'future' not being allowed to replace 'pending'.
This change fixes that, by doing a more in-depth inspection of the queue.
This PR removes the Debug field from vmconfig, making it so that if a tracer is set, debug=true is implied.
---------
Co-authored-by: 0xTylerHolmes <tyler@ethereum.org>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
Currently, most of transaction validation while holding the txpool mutex: one exception being an early-on signature check.
This PR changes that, so that we do all non-stateful checks before we entering the mutex area. This means they can be performed in parallel, and to enable that, certain fields have been made atomic bools and uint64.
This change renames StateTrie methods to remove the Try* prefix.
We added the Trie methods with prefix 'Try' a long time ago, working
around the problem that most existing methods of Trie did not return the
database error. This weird naming convention has persisted until now.
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This changes the Trie interface to add the plain account address as a
parameter to all storage-related methods.
After the introduction of the TryAccount* functions, TryGet, TryUpdate and
TryDelete are now only meant to read an account's storage. In their current
form, they assume that an account storage is stored in a separate trie, and
that the hashing of the slot is independent of its account's address.
The proposed structure for a stateless storage breaks these two
assumptions: the hashing of a slot key requires the address and all slots
and accounts are stored in a single trie.
This PR therefore adds an address parameter to the interface. It is ignored
in the MPT version, so this change has no functional impact, however it
will reduce the diff size when merging verkle trees.
The meter for "for measuring the effective amount of data read" within the freezertable was never updated. This change remedies that.
---------
Signed-off-by: jsvisa <delweng@gmail.com>
When interacting with geth as a library to e.g. produce state tests, it is desirable to obtain the consensus-correct jumptable definition for a given fork. This changes adds accessors so the instructionset can be obtained and characteristics about opcodes can be inspected.
* minor comment update
* added support for rpc.evmtimeout flag
* added support for vmdebug (EnablePreimageRecording) flag
* added support for jsonrpc.auth.(jwtsecret, addr, port, vhosts) flags
* added support for miner.recommit flag
* added support for gpo.maxheaderhistory and gpo.maxblockhistory flag
* Revert "added support for miner.recommit flag"
This reverts commit fcb722e4bf5bdda23d4c91da40ab95c9023a80c3.
* added pprof related flags (expect --pprof.cpuprofile - Write CPU profile to the given file)
* added support for --dev.gaslimit flag
* added support for --fdlimit flag
* added support for --netrestrict flag
* added support for --nodekey and --nodekeyhex flag
* added support for --vmodule, --log.json, --log.backtrace, and --log.debug flags
* fixed related lint errors
* fix lints from develop (few lints decided to appear from code that was untouched, weird)
* more weird lints from develop
* small precautionary fix
* small bug ;) fix in NetRestrict
* weird lints
* change vmdebug = true to vmdebug = false
Makes clear the distinction between Finalize and FinalizedAndAssemble:
- In Finalize function, a series of state operations are applied according to consensus rules. The statedb is mutated and the root hash can be checked and compared afterwards.
This function should be used in block processing(receive afrom network and apply it locally) but not block generation.
- In FinalizeAndAssemble function, after applying state mutations, the block is also to be assembled with the latest
state root computed, updating the header.
This function should be used in block generation only.
This adds two new rules to the transaction pool:
- A future transaction can not evict a pending transaction.
- A transaction can not overspend available funds of a sender.
---
Co-authored-by: dwn1998 <42262393+dwn1998@users.noreply.github.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Here, the core.Message interface turns into a plain struct and
types.Message gets removed.
This is a breaking change to packages core and core/types. While we do
not promise API stability for package core, we do for core/types. An
exception can be made for types.Message, since it doesn't have any
purpose apart from invoking the state transition in package core.
types.Message was also marked deprecated by the same commit it
got added in, 4dca5d4db7 (November 2016).
The core.Message interface was added in December 2014, in commit
db494170dc, for the purpose of 'testing' state transitions. It's the
same change that made transaction struct fields private. Before that,
the state transition used *types.Transaction directly.
Over time, multiple implementations of the interface accrued across
different packages, since constructing a Message is required whenever
one wants to invoke the state transition. These implementations all
looked very similar, a struct with private fields exposing the fields
as accessor methods.
By changing Message into a struct with public fields we can remove all
these useless interface implementations. It will also hopefully
simplify future changes to the type with less updates to apply across
all of go-ethereum when a field is added to Message.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This changes the test to match the comment description. Using timestampedConfig in this test case is incorrect, the comment says 'local is at Gray Glacier' and isn't aware of more forks.
This change prints out more information about the problem, in the case where geth detects a gap between leveldb and ancients, so we can determine more exactly where the gap is (what the first missing is). Also prints out more metadata.
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
This change fixes a flaw where, in certain scenarios, the block sealer did not accurately reset the remaining gas after failing to include an invalid transaction. Fixes#26791
Fixes a race in TestNewPayloadOnInvalidTerminalBlock where setting the TTD raced with
the miner. Solution: set the TTD on the blockchain config not the genesis config.
Also fixes a race in CopyHeader which resulted in race reports all over the place.
This change adds a struct field EffectiveGasPrice in types.Receipt. The field is present
in RPC responses, but not in the Go struct, and thus can't easily be accessed via ethclient.
Co-authored-by: PulsarAI <dev@pulsar-systems.fi>
This fixes an issue where the withdrawal index was not calculated correctly
for multiple withdrawals in a single block.
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
The EmptyRootHash and EmptyCodeHash are defined everywhere in the codebase, this PR replaces all of them with unified one defined in core/types package, and also defines constants for TxRoot, WithdrawalsRoot and UncleRoot
This PR contains a small portion of the full pbss PR, namely
Remove the tracer from trie (and comitter), and instead using an accessList.
Related changes to the Nodeset.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR is a (superior) alternative to https://github.com/ethereum/go-ethereum/pull/26708, it handles deprecation, primarily two specific cases.
`rand.Seed` is typically used in two ways
- `rand.Seed(time.Now().UnixNano())` -- we seed it, just to be sure to get some random, and not always get the same thing on every run. This is not needed, with global seeding, so those are just removed.
- `rand.Seed(1)` this is typically done to ensure we have a stable test. If we rely on this, we need to fix up the tests to use a deterministic prng-source. A few occurrences like this has been replaced with a proper custom source.
`rand.Read` has been replaced by `crypto/rand`.`Read` in this PR.
This PR relaxes the block body ingress handling a bit: if block body withdrawals are missing (but expected to be empty), the body withdrawals are set to 'empty list' before being passed to upper layers.
This fixes an issue where a block passed from EthereumJS to geth was deemed invalid.
Logs stored on disk have minimal information. Contextual information such as block
number, index of log in block, index of transaction in block are filled in upon request.
We can fill in all these fields only having the block header and list of receipts.
But determining the transaction hash of a log requires the block body.
The goal of this PR is postponing this retrieval until we are sure we the transaction hash.
It happens often that the header bloom filter signals there might be matches in a block,
but after actually checking them reveals the logs do not match. We want to avoid fetching
the body in this case.
Note that this changes the semantics of Backend.GetLogs. Downstream callers of
GetLogs now assume log context fields have not been derived, and need to call
DeriveFields on the logs if necessary.
This is a breaking change in the tracing hooks API as well as semantics of the callTracer:
- CaptureEnter hook provided a nil value argument in case of DELEGATECALL. However to stay consistent with how delegate calls behave in EVM this hook is changed to pass in the value of the parent call.
- callTracer will return parent call's value for DELEGATECALL frames.
---------
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
* common, core, eth, les, trie: make prque generic
* les/vflux/server: fixed issues in priorityPool
* common, core, eth, les, trie: make priority also generic in prque
* les/flowcontrol: add test case for priority accumulator overflow
* les/flowcontrol: avoid priority value overflow
* common/prque: use int priority in some tests
No need to convert to int64 when we can just change the type used by the
queue.
* common/prque: remove comment about int64 range
---------
Co-authored-by: Zsolt Felfoldi <zsfelfoldi@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
This change ports some changes from the main PBSS PR:
- get rid of callback function in `trie.Database.Commit` which is not required anymore
- rework the `nodeResolver` in `trie.Iterator` to make it compatible with multiple state scheme
- some other shallow changes in tests and typo-fixes
This PR moves core/beacon to beacon/engine so that beacon-chain related code has its own top level package which also can house the the beacon lightclient-code.
This PR moves some trie-related db accessor methods to a different file, and also removes the schema type. Instead of the schema type, a string is used to distinguish between hashbased/pathbased db accessors.
This also moves some code from trie package to rawdb package.
This PR is intended to be a no-functionality-change prep PR for #25963 .
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This change improves reusability of the EVM struct. Two methods are added:
- SetBlockContext(...)
- SetTracer(...)
Other attributes like the TransactionContext and the StateDB can already be updated.
BlockContext and Tracer are partially not updateable right now. This change fixes it and
opens the potential to reuse an EVM struct in more ways.
Co-authored-by: Felix Lange <fjl@twurst.com>
This change implements withdrawals as specified in EIP-4895.
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: marioevz <marioevz@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR changes the API so that uint64 is used for fork timestamps.
It's a good choice because types.Header also uses uint64 for time.
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR does a few things.
It fixes a shutdown-order flaw in the chainfreezer. Previously, the chain-freezer would shutdown the freezer backend first, and then signal for the loop to exit. This can lead to a scenario where the freezer tries to fsync closed files, which is an error-conditon that could lead to exit via log.Crit.
It also makes the printout more detailed when truncating 'dangling' items, by showing the exact number instead of approximate MB.
This PR also adds calls to fsync files before closing them, and also makes the `db inspect` command slightly more robust.
This PR fixes an issue which might result in data lost in freezer.
Whenever mutation happens in freezer, all data will be written into head data file
and it will be rotated with a new one in case the size of file reaches the threshold.
Theoretically, the rotated old data file should be fsync'd to prevent data loss.
In freezer.Sync function, we only fsync: (1) index file (2) meta file and (3) head
data file. So this PR forcibly fsync the head data file if mutation happens in the
boundary of data file.
* Adding in Mumbai/Mainnet precursor deb packaging for tests to use during upgrade(iterations to come)
* Added changes per discussion in PR, more changes may be necessary
* Adding prerelease true
* Disabling goreleaser
* Removing README swap file
* change bor_dir and add bor user for v0.3.0 release
* rollback bor user and use root
* metrics: handle equal to separated config flag (#596)
* metrics: handle based config path
* internal/cli/server: add more context to logs
* use space separated flag and value in bor.service
* fixed static-nodes related buf (os independent) (#598)
* fixed static-nodes related buf (os independent)
* taking static-nodes as input if default not present
* Update default flags (#600)
* internal/cli/server: use geth's default for txpool.pricelimit and add comments
* builder/files: update config.toml for mainnet
* packaging/templates: update defaults for mainnet and mumbai
* internal/cli/server: skip overriding cache
* packaging/templates: update cache value for mainnet
* packaging/templates: update gcmode for archive mumbai node
* metrics: handle nil telemetry config (#601)
* resolve merge conflicts
* update go version in release.yml
* update goversion in makefile
* update Docker login for goreleaser-cross v1.19
* Cleanup for the packager to use git tag in the package profile naming. Added conditional check for directory structure, this is in prep for v0.3.1, as this will create a failure on upgrade path in package due to file exist
* added a toml configuration file with comments describing each flag (#607)
* added a toml configuration file with comments describing each flag
* internal/cli/server: update flag description
* docs/cli: update example config and description of flags
* docs: update new-cli docs
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
* Adding of 0.3.0 package changes, control file updates, postinst changes, and packager update
* added ancient datadir flag and toml field, need to decide on default value and update the conversion script
* updated toml files with ancient field
* Add support for new flags in new config.toml, which were present in old config.toml (#612)
* added HTTPTimeouts, and TrieTimeout flag in new tol, from old toml
* added RAW fields for these time.Duration flags
* updated the conversion script to support these extra 4 flags
* removed hcl and json config tests as we are only supporting toml config files
* updated toml files with cache.timeout field
* updated toml files with jsonrpc.timeouts field
* tests/bor: expect a call for latest checkpoint
* tests/bor: expect a call for latest checkpoint
* packaging/templates: update cache values for archive nodes
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
* remove unwanted code
* Fix docker publish authentication issue
In gorelease-cross 1.19+, dockerhub authentication will require docker
logion action followed by mounting docker config file. See
https://github.com/goreleaser/goreleaser-cross#github-actions.
* Revert "update Docker login for goreleaser-cross v1.19"
This reverts commit 4d19cf5342.
* Bump version to stable
* Revert "Merge pull request #435 from maticnetwork/POS-553"
This reverts commit 657d262def, reversing
changes made to 88dbfa1c13.
* revert change for release for go1.19
* Add default values to CLI helper and docs
This commit adds default values to CLI helper and docs. When the default value of a string flag, slice string flag, or map string flag is empty, its helper message won't show any default value.
* Add a summary of new CLI in docs
* Updating packager as binutils changed version so that apt-get installs current versions
* Add state pruning to new CLI
* Minor wording fix in prune state description
* Bumping control file versions
* Mainnet Delhi fork
* Set version to stable
* change delhi hardfork block number
* handle future chain import and skip peer drop (#650)
* handle future chain import and skip peer drop
* add block import metric
* params: bump version to v0.3.3-stable
* Bump bor version in control files for v0.3.3 mainnet release
Co-authored-by: Daniel Jones <djones@polygon.technology>
Co-authored-by: Will Button <wbutton@polygon.technology>
Co-authored-by: Will Button <will@willbutton.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Implementation of https://eips.ethereum.org/EIPS/eip-3860, limit and meter initcode. This PR enables EIP-3860 as part of the Shanghai fork.
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
This makes non-JS tracers execute all block txs on a single goroutine.
In the previous implementation, we used to prepare every tx pre-state
on one goroutine, and then run the transactions again with tracing enabled.
Native tracers are usually faster, so it is faster overall to use their output as
the pre-state for tracing the next transaction.
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
This PR removes the notion of fakeStorage from the state objects, and instead, for any state modifications that are needed, it simply makes the changes.
This changes moves the tracking of "deleted in this block" out from snap-only domain, so that it happens regardless of whether the execution is snapshot-backed or trie-backed.
* added support for dependencies (executor_tests)
* added a function to get dependency map
* getting all dependencies in the GetDep function
* updated GetDep function
* changed the type of AllDeps
* added a function to get dependency map
* updated GetDep function
* generate and get dependencies from block producer
* optimized getDep function
* bug fix regarding txn index and dep structure
* fixed gas bug
* optimized getDep function
* tests updated/added
* few updates regarding dependencies
* added channel to calculate the dependencies in a separate go routine
* minor changes in the executor which uses latest changes of dependecies + removed metadata flag/argument
* Use channel when metadata is available
* small bug fix
* getting reads and writes only when the transaction succeeds
* fixed bug in adding dependencies
* updated logic for delay/not delay
* bug fix (shouldDelayFeeCal) in parallel state processor
* lint fix
* using EnableMVHashMap flag
* fixed worker and stateProcessor and removed SetMVHashMapNil fumction from stateDB
* addredded few comments and fixed bug in executor tests
* commented executor tests with metadata (panic: test timed out after 5m0s)
* added a check to check len(mvReadMapList) > 0 in miner
* addressed comments, minor refactoring in dag.go
* moved blockContext out of Execute and adding it in execution task
* removed Author() from Settle() and added in execution task
* not calling block.Header() again and again, using instead
* removed EnableMVHashMap flag, and updated applyTransaction function
* addressed comments
* added unit test to check dependencies in the block header
Co-authored-by: Jerry <jerrycgh@gmail.com>
This changes the StorageTrie method to return an error when the trie
is not available. It used to return an 'empty trie' in this case, but that's
not possible anymore under PBSS.
This PR builds on #26299, but also updates the tests to the most recent version, which includes tests regarding TheMerge.
This change adds checks to the beacon consensus engine, making it more strict in validating the pre- and post-headers, and not relying on the caller to have already correctly sanitized the headers/blocks.
This PR implements resettable freezer by adding a ResettableFreezer wrapper.
The resettable freezer wraps the original freezer in a way that makes it possible to ensure atomic resets. Implementation wise, it relies on the os.Rename and os.RemoveAll to atomically delete the original freezer data and re-create a new one from scratch.
A comment suggests that contract creation happens if the recipient of a call is 0x00..00 ("zero address") but in fact the sender must be nil. The zero address is a regular valid address that is commonly used as a "burn" address.
While investigating another issue, I found that all callers of collectLogs have the
complete block available. rawdb.ReadReceipts loads the block from the database,
so it is better to use ReadRawReceipts here, and derive the receipt information using
the block which is already in memory.
This PR makes it possible to modify the flush interval time via RPC. On one extreme, `0s`, it would act as an archive node. If set to `1h`, means that after one hour of effective block processing time, the trie would be flushed. If one block takes 200ms, this means that a flush would occur every `5*3600=18000` blocks -- however, if the memory size of the cached states grows too large, it will flush sooner.
Essentially, this makes it possible to configure the node to be more or less "archive:ish", and without restarting the node while reconfiguring it.
The gcproc field tracks the amount of time spent processing blocks,
and is used to trigger a state flush to disk when a certain threshold is
reached. After the merge, single block insertion by CL is the most
common source of block processing time, but this time was not added
into gcproc.
This removes the 'time' field from logs, as well as from the tracer interface. This change makes the trace output deterministic. If a tracer needs the time they can measure it themselves. No need for evm to do this.
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
This changes the Pop method to assign the zero value before
reducing slice size. Doing so ensures the backing array does not
reference removed item values.
This PR drops the legacy receipt types, the freezer-migrate command and the startup check. The previous attempt #22852 at this failed because there were users who still had legacy receipts in their db, so it had to be reverted #23247. Since then we added a command to migrate legacy dbs #24028.
As of the last hardforks all users either must have done the migration, or used the --ignore-legacy-receipts flag which will stop working now.
This PR introduces a node scheme abstraction. The interface is only implemented by `hashScheme` at the moment, but will be extended by `pathScheme` very soon.
Apart from that, a few changes are also included which is worth mentioning:
- port the changes in the stacktrie, tracking the path prefix of nodes during commit
- use ethdb.Database for constructing trie.Database. This is not necessary right now, but it is required for path-based used to open reverse diff freezer
While investigating #22374, I noticed that the Sync operation of the
freezer does not take the table lock. It also doesn't call sync for all files
if there is an error with one of them. I doubt this will fix anything, but
didn't want to drop the fix on the floor either.
There seems to be an issue when hasher is used concurrently in parallel
execution. This change will ensure no hasher is used by multiple
executors at the same time.
It seems there is no fully typed library implementation of an LRU cache.
So I wrote one. Method names are the same as github.com/hashicorp/golang-lru,
and the new type can be used as a drop-in replacement.
Two reasons to do this:
- It's much easier to understand what a cache is for when the types are right there.
- Performance: the new implementation is slightly faster and performs zero memory
allocations in Add when the cache is at capacity. Overall, memory usage of the cache
is much reduced because keys are values are no longer wrapped in interface.
When the interpreter is configured to use extra-eips, this change makes it so that all the opcodes are deep-copied, to prevent accidental modification of the 'base' jumptable.
Closes: #26136
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR changes geth to read the eip1559 params from the chain config instead of the globals.
This way the parameters may be changed by forking the chain config code, without creating a large diff throughout the past and future usages of the parameters.
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR ports a few changes from PBSS:
- Fix the snapshot generator waiter in case the generation is not even initialized
- Refactor db inspector for ancient store
This PR fixes a regression causing snapshots not to be generated in "geth --import" mode. It also fixes the geth export command to be truly readonly, and adds a new test for geth export.
This adds a
* core/vm, tests: optimized modexp + fuzzer
* common/math: modexp optimizations
* core/vm: special case base 1 in big modexp
* core/vm: disable fastexp
This changes the error message for mismatching chain ID to show
the given and expected value. Callers expecting this error must be
changed to use errors.Is.
* Create MVHashMap and use it StateDB
* Parallel state processor
* Move fee burning and tipping out of state transition to reduce read/write dependencies between transactions
* Re-execute parallel tasks when there is a read in coinbase or burn address
* Block-stm optimization
Added tests for executor and two major improvements:
1. Add a dependency map during execution. This will prevent aborted tasks from being sent for execution immedaitely after failure.
2. Change the key of MVHashMap from string to a byte array. This will reduce time to convert byte slices to strings.
* Remove cache from executor test
* added mvhashmap unit tests (with as key)
* Shard mvhashmap to reduce the time spent in global mutex
* Skip applying intermediate states
* Dependency improvement
* added test for status
* Create MVHashMap and use it StateDB
* Parallel state processor
* Move fee burning and tipping out of state transition to reduce read/write dependencies between transactions
* Re-execute parallel tasks when there is a read in coinbase or burn address
* Txn prioritizer implemented using mutex map (#487)
* basic txn prioritizer implemented using mutex map
* Re-execute parallel tasks when there is a read in coinbase or burn address
* Re-execute parallel tasks when there is a read in coinbase or burn address
* using *sync.RWMutex{} in mutexMap
Co-authored-by: Jerry <jerrycgh@gmail.com>
* added getReadMap and getWriteMap (#473)
* Block-stm optimization
Added tests for executor and some improvements:
1. Add a dependency map during execution. This will prevent aborted tasks from being sent for execution immedaitely after failure.
2. Change the key of MVHashMap from string to a byte array. This will reduce time to convert byte slices to strings.
3. Use sync.Map to reduce the time spent in global mutex.
4. Skip applying intermediate states.
5. Estimate dependency when an execution fails without dependency information.
6. Divide execution task queue into two separate queues. One for relatively certain transactions, and the other for speculative future transactions.
7. Setting dependencies of Txs coming from the same sender before starting parallel execution.
8. Process results in their semantic order (transaction index) instead of the order when they arrive. Replace result channel with a priority queue.
* Do not write entire objects directly when applying write set in blockstm
* fixed a small bug in the Report function (#530)
* linters
Co-authored-by: Jerry <jerrycgh@gmail.com>
* ethclient/gethclient: improve time-sensitive flaky test
* eth/catalyst: fix (?) flaky test
* core: stop blockchains in tests after use
* core: fix dangling blockchain instances
* core: rm whitespace
* eth/gasprice, eth/tracers, consensus/clique: stop dangling blockchains in tests
* all: address review concerns
* core: goimports
* eth/catalyst: fix another time-sensitive test
* consensus/clique: add snapshot test run function
* core: rename stop() to stopWithoutSaving()
Co-authored-by: Felix Lange <fjl@twurst.com>
* base setup for miner opentel
* version change
* modify ctx passing
* add attributes
* update fill txs span attributes
* fix: use common attributes for remote and local txs
* pass context in seal
* fix
* fix
* add traces to finalize and assemble
* fix
* use task ctx in result loop
* only start parent span if no error
* send nil tracer from Finalize
* clean up
* add sub function timings in span attribute
* modify span attributes
* set time attribute to milliseconds
* linters fix
* fix linters for consensus
* add nolint to worker
* fix testcase
* Added fillTransactions subTraces
* add traces in intermediate root hash function
* add traces in WriteBlockAndSetHead function, fix linters
* fix: linting errors
* fix: test cases
* fix: go.mod
* fix: testcase
* extract tracing package
* linters fix
* debug
* Revert "debug"
This reverts commit 2d68b7c7b1105080563a4e1a6949dabc10acaff8.
* fix: panic in NewTransactionsByPriceAndNonce iteration
* change heimdall version to develop
* miner/worker: fix duplicate call to tx ordering
* miner/worker: refactor tracing
* consensus/bor: use tracing package
* tracing: add more abstraction for spans
* tracing: set all attributes at once
* remove nested tracing from blockchain.WriteBlockAndSetHead function
* remove nested tracing from statedb.IntermediateRoot function
* handle end span in bor.Seal function
* fix: typo
* minor fixes
* fix: linters
* fix: remove nolint
* go mod tidy
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Shivam Sharma <shivam691999@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
This PR reworks tx indexer a bit. Compared to the original version, one scenario is no longer handled - upgrading from legacy geth without indexer support.
The tx indexer was introduced in 2020 and have been present through hardforks, so it can be assumed that all Geth nodes have tx indexer already. So we can simplify the tx indexer logic a bit:
- If the tail flag is not present, it means node is just initialized may or may not with an ancient store attached. In this case all blocks are regarded as unindexed
- If the tail flag is present, it means blocks below tail are unindexed, blocks above tail are indexed
This change also address some weird cornercases that could make the indexer not work after a crash.
Added tests for executor and some improvements:
1. Add a dependency map during execution. This will prevent aborted tasks from being sent for execution immedaitely after failure.
2. Change the key of MVHashMap from string to a byte array. This will reduce time to convert byte slices to strings.
3. Use sync.Map to reduce the time spent in global mutex.
4. Skip applying intermediate states.
5. Estimate dependency when an execution fails without dependency information.
6. Divide execution task queue into two separate queues. One for relatively certain transactions, and the other for speculative future transactions.
7. Setting dependencies of Txs coming from the same sender before starting parallel execution.
8. Process results in their semantic order (transaction index) instead of the order when they arrive. Replace result channel with a priority queue.
* basic txn prioritizer implemented using mutex map
* Re-execute parallel tasks when there is a read in coinbase or burn address
* Re-execute parallel tasks when there is a read in coinbase or burn address
* using *sync.RWMutex{} in mutexMap
Co-authored-by: Jerry <jerrycgh@gmail.com>
`geth dumpgenesis` currently does not respect the content of the data directory. Instead, it outputs the genesis block created by command-line flags. This PR fixes it to read the genesis from the database, if the database already exists.
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR cleans up the configurations for pruner and snapshotter by passing a config struct.
And also, this PR disables the snapshot background generation if the chain is opened in "read-only" mode. The read-only mode is necessary in some cases. For example, we have a list of commands to open the etheruem node in "read-only" mode, like export-chain. In these cases, the snapshot background generation is non expected and should be banned explicitly.
core/blockchain: downgrade tx indexing and unindexing logs from info to debug
If a user has a finite tx lookup limit, they will see an "unindexing" info level log each time a block is imported. This information might help a user understand that they are removing the index each block and some txs may not be retrievable by hash, but overall it is generally more of a nuisance than a benefit. This change downgrades the log to a debug log.
This shortens the chain config summary in bad block reports,
and adds go-ethereum version information as well.
Co-authored-by: Felix Lange <fjl@twurst.com>
This changes the CI / release builds to use the latest Go version. It also
upgrades golangci-lint to a newer version compatible with Go 1.19.
In Go 1.19, godoc has gained official support for links and lists. The
syntax for code blocks in doc comments has changed and now requires a
leading tab character. gofmt adapts comments to the new syntax
automatically, so there are a lot of comment re-formatting changes in this
PR. We need to apply the new format in order to pass the CI lint stage with
Go 1.19.
With the linter upgrade, I have decided to disable 'gosec' - it produces
too many false-positive warnings. The 'deadcode' and 'varcheck' linters
have also been removed because golangci-lint warns about them being
unmaintained. 'unused' provides similar coverage and we already have it
enabled, so we don't lose much with this change.
This PR makes the event-sending for deleted and new logs happen in batches, to prevent OOM situation due to large reorgs.
Co-authored-by: Felix Lange <fjl@twurst.com>
