Commit graph

455 commits

Author SHA1 Message Date
Manav Darji
8a1e0ec125
Enforce parallel evm via flag (#1425)
* (feat): implement --parallelevm.enforce to force using block-stm

* core: debug log

* docs: update docs with new flag

* update all config files

* core: modify chain import test to enforce parallel processing
2025-01-30 23:01:27 +05:30
Manav Darji
9da0432ec2
miner, consensus, core: remove tracing in mining workflow (#1422)
This PR removes open telemetry based tracing from the whole mining workflow. The tracing extended into consensus and core blockchain functions from miner is also removed in this PR. We weren't checking the tracing from a long time and the way it was implemented increased the code complexity a lot.
2025-01-30 22:27:03 +05:30
Jerry
99be105427
Make block execution timer more accurate
Block execution contains the time it takes to fetch state sync data from heimdall, which sometimes returns data with long latency (> 1s). When the latency is significant, the chain execution time will be affected and become inaccurate. This PR isolate the time of heimdall requests from block execution.
2025-01-15 14:39:44 -08:00
Jerry
d68987696a
Fallback to serial execution if parallel execution fails (#1392) 2024-12-24 09:03:30 -08:00
Manav Darji
ee5318b07b
core: validate chain before writing block (#1319) 2024-11-18 12:48:04 +05:30
Jerry
48bdfa7ffb Fix tx index limit not being used correctly 2024-10-09 07:59:21 -07:00
Jerry
352fb8adfa
Fix txIndexer 2024-09-20 19:49:26 -07:00
Jerry
d9daacad17
Address goroutine leak 2024-09-12 15:56:33 -07:00
anshalshukla
a66305dae3
fix: lint, add: milestone related tests in downloader 2024-09-05 14:03:10 +05:30
anshalshukla
cec3ee18d8
fix: testcases 2024-09-03 16:21:20 +05:30
anshalshukla
43043ed7ec
fix: few linting errors 2024-09-02 11:00:33 +05:30
anshalshukla
c0ff6dbf81
fix: testcases, add: block broadcasting mechanism 2024-08-30 14:43:02 +05:30
anshalshukla
687cd76405
fix: testcases 2024-08-19 18:07:05 +05:30
anshalshukla
b1419eb41d
fix: testcases 2024-08-13 16:46:46 +05:30
anshalshukla
5cda3fe0f0
fix: testcases 2024-08-13 11:39:52 +05:30
anshalshukla
e6132fdf1e
fix: testcases 2024-08-09 11:59:34 +05:30
Anshal Shukla
03f253c1e0
add block fetcher back, fix handler and blockchain errors 2024-08-06 13:44:27 +05:30
Anshal Shukla
dbf55ce6cc
Merge tag 'v1.14.6' into upstream_merge_1.14.6 2024-08-05 16:29:10 +05:30
Anshal Shukla
f4c41e2e70
Merge branch 'release/1.13' into upstream_merge_1.13 2024-07-10 11:19:34 +05:30
jwasinger
ed8fd0ac09
all: stateless witness builder and (self-)cross validator (#29719)
* all: add stateless verifications

* all: simplify witness and integrate it into live geth

---------

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2024-06-25 14:48:08 +03:00
jwasinger
85587d5ef2
cmd, core: prefetch reads too from tries if requested (#29807)
* cmd/utils, consensus/beacon, core/state: when configured via stub  flag: prefetch all reads from account/storage tries, terminate prefetcher synchronously.

* cmd, core/state: fix nil panic, fix error handling, prefetch nosnap too

* core/state: expand prefetcher metrics for reads and writes separately

* cmd/utils, eth: fix noop collect witness flag

---------

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2024-06-11 11:10:07 +03:00
Péter Szilágyi
2ac83e197b
core/state: blocking prefetcher on term signal, parallel updates (#29519)
* core/state: trie prefetcher change: calling trie() doesn't stop the associated subfetcher

Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>

* core/state: improve prefetcher

* core/state: restore async prefetcher stask scheduling

* core/state: finish prefetching async and process storage updates async

* core/state: don't use the prefetcher for missing snapshot items

* core/state: remove update concurrency for Verkle tries

* core/state: add some termination checks to prefetcher async shutdowns

* core/state: differentiate db tries and prefetched tries

* core/state: teh teh teh

---------

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2024-05-13 15:47:45 +03:00
Manav Darji
d95c05b98a
Enable ancient block pruning (#1216)
* core/state: typo

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: backport from https://github.com/bnb-chain/bsc/pull/543

Signed-off-by: Delweng <delweng@gmail.com>

* eth,ethdb,node,core/state: backport from https://github.com/bnb-chain/bsc/pull/543

Signed-off-by: Delweng <delweng@gmail.com>

* eth,core: backport from https://github.com/bnb-chain/bsc/pull/543

Signed-off-by: Delweng <delweng@gmail.com>

* cmd: open db with freeze disabled

Signed-off-by: Delweng <delweng@gmail.com>

* cli: snapshot prune-block

Signed-off-by: Delweng <delweng@gmail.com>

* fix typo

Signed-off-by: Delweng <delweng@gmail.com>

* cli/snapshot: fix the issue of dup open db error

Signed-off-by: Delweng <delweng@gmail.com>

* cli/snapshot: resolve datadir and ancient before backup

Signed-off-by: Delweng <delweng@gmail.com>

* core: more prune-block log

Signed-off-by: Delweng <delweng@gmail.com>

* core: truncatetail missing f.offset

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: indextx adjust offset of pruned block

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: freezer batch should implement the offset commit, ref https://github.com/bnb-chain/bsc/pull/1005

Signed-off-by: Delweng <delweng@gmail.com>

* core: check of ancientdb, backport https://github.com/bnb-chain/bsc/pull/817

Signed-off-by: Delweng <delweng@gmail.com>

* core/state: read raw borReceipt to backup

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: bor receipt maybe in []Receipt or Receipt RLP format

Signed-off-by: Delweng <delweng@gmail.com>

* core/state: typo and error msg

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: offSet -> offset

Signed-off-by: Delweng <delweng@gmail.com>

* cli/snapshot: comment

Signed-off-by: Delweng <delweng@gmail.com>

* cli/snapshot: add prune-block doc

Signed-off-by: Delweng <delweng@gmail.com>

* docs: add prune-block document

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: print wrong bor-receipt length

Signed-off-by: Delweng <delweng@gmail.com>

* internal/cli: add snapshot prune block tests (referenced from bsc's PR)

* improve errors

* cmd, core, eth, internal: fix lint

* internal/cli: refactor snapshot prune block test

* fix linters in tests

* internal/cli: add inspect-ancient-db command, update docs

* pruner: use a generic function for simplification

* internal/cli: fixes for inspect-db command

* internal/cli: improve pruning tests

* core/rawdb: update end block calculation logic in inspect command

* core/rawdb: improve checks db initialisation

* core/rawdb: remove offset check

* update mocks for span, ethdb and add command in makefile

* docs/cli: update docs with inspect command

* go mod tidy

* refactor and resolve conflicts

* resolve more conflicts

* refactor

* explicitly read node for hash scheme

* add check for hash scheme, fix tests

* fix typo

* update docs and add warning

* raise error if pbss is enabled

* revert read raw bor receipt change

* consensus/bor: handle nil header case in get root hash

* address comments

* core/rawdb: check chain continuity by matching parent hash

* core/rawdb: account for pruned ancient blocks

* go mod tidy

* fix tests

* fix tests

---------

Signed-off-by: Delweng <delweng@gmail.com>
Co-authored-by: Delweng <delweng@gmail.com>
2024-05-09 11:19:34 +05:30
Guillaume Ballet
43cbcd78ea
core, core/state: move TriesInMemory to state package (#29701) 2024-05-06 13:28:53 +02:00
Péter Szilágyi
682ee820fa
core/state: parallelise parts of state commit (#29681)
* core/state, internal/workerpool: parallelize parts of state commit

* core, internal: move workerpool into syncx

* core/state: use errgroups, commit accounts concurrently

* core: resurrect detailed commit timers to almost-accuracy
2024-05-02 11:18:27 +03:00
maskpp
fecc8a0f4a
cmd/evm/internal/t8ntool, core: prealloc map sizes where possible (#29620)
set cap for map in a certain scenario
2024-04-30 11:19:59 +02:00
Péter Szilágyi
4253030ef6
core/state: move metrics out of state objects (#29665) 2024-04-26 18:35:52 +03:00
Chris Ziogas
fb08fd334a
core/tracing: Add OnClose Trace Hook (#29629)
The OnClose trace hook is being triggered on blockchain Stop, so as tracers can release any resources.
2024-04-24 07:54:59 +02:00
rjl493456442
acd1eaae2c
core: remove bad block checks (#29609) 2024-04-22 20:00:42 +03:00
rjl493456442
1126c6d8a5
core: add txlookup lock (#29343)
This change adds a lock to the transaction lookup cache, to avoid the case where reorgs make the lookup return inconsistent results.
2024-04-09 08:37:18 +02:00
Aaron Chen
ed4bc7f27b
all: replace fmt.Errorf() with errors.New() if no param required (#29472) 2024-04-08 12:59:17 +02:00
rjl493456442
3b77e0ff4b
core: remove unused code (#29381) 2024-03-28 12:06:57 +01:00
Guillaume Ballet
da7469e5c4
core: add an end-to-end verkle test (#29262)
core: add a simple verkle test

triedb, core: skip hash comparison in verkle

core: remove legacy daoFork logic in verkle chain maker

fix: nil pointer in tests

triedb/pathdb: add blob hex

core: less defensive

Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2024-03-26 21:25:41 +01:00
Aaron Chen
723b1e36ad
all: fix mismatched names in comments (#29348)
* all: fix mismatched names in comments

* metrics: fix mismatched name in UpdateIfGt
2024-03-26 21:01:28 +01:00
Sina M
064f37d6f6
eth/tracers: live chain tracing with hooks (#29189)
Here we add a Go API for running tracing plugins within the main block import process. 

As an advanced user of geth, you can now create a Go file in eth/tracers/live/, and within
that file register your custom tracer implementation. Then recompile geth and select your tracer
on the command line. Hooks defined in the tracer will run whenever a block is processed.

The hook system is defined in package core/tracing. It uses a struct with callbacks, instead of 
requiring an interface, for several reasons:

- We plan to keep this API stable long-term. The core/tracing hook API does not depend on
  on deep geth internals.
- There are a lot of hooks, and tracers will only need some of them. Using a struct allows you
   to implement only the hooks you want to actually use.

All existing tracers in eth/tracers/native have been rewritten to use the new hook system.

This change breaks compatibility with the vm.EVMLogger interface that we used to have.
If you are a user of vm.EVMLogger, please migrate to core/tracing, and sorry for breaking
your stuff. But we just couldn't have both the old and new tracing APIs coexist in the EVM.

---------

Co-authored-by: Matthieu Vachon <matthieu.o.vachon@gmail.com>
Co-authored-by: Delweng <delweng@gmail.com>
Co-authored-by: Martin HS <martin@swende.se>
2024-03-22 18:53:53 +01:00
rjl493456442
78c102dec5
core: skip the check the statefulness of head block in repair (#29245) 2024-03-20 13:11:30 +01:00
Shivam Sharma
feeca914ef Merge branch 'v1.2.9-beta-candidate' into shivam/temp-master-develop 2024-03-20 14:55:19 +05:30
Manav Darji
deb34da04d
core: remove tracing and handle err in reorg call (#1193) 2024-03-19 16:27:33 +05:30
rjl493456442
c170fa277c
core: improve chain rewinding mechanism (#29196)
* core: improve chain rewinding mechanism

* core: address comment

* core: periodically print progress log

* core: address comments

* core: fix comment

* core: fix rewinding in path

* core: fix beyondRoot condition

* core: polish code

* core: polish code

* core: extend code comment

* core: stop rewinding if chain is gapped or genesis is reached

* core: fix broken tests
2024-03-13 13:39:30 +02:00
Péter Szilágyi
b393ad8d29
cmd, core, metrics: always report expensive metrics (#29191)
* cmd, core, metrics: always report expensive metrics

* core, metrics: report block processing metrics as resetting timer

* metrics: update reporter tests
2024-03-11 10:06:57 +02:00
Péter Szilágyi
f4d53133f6
consensus, cmd, core, eth: remove support for non-merge mode of operation (#29169)
* eth: drop support for forward sync triggers and head block packets

* consensus, eth: enforce always merged network

* eth: fix tx looper startup and shutdown

* cmd, core: fix some tests

* core: remove notion of future blocks

* core, eth: drop unused methods and types
2024-03-05 16:13:28 +02:00
Sina Mahmoodi
95741b1844
core: move genesis alloc types to core/types (#29003)
We want to use these types in public user-facing APIs, so they shouldn't be in core.

Co-authored-by: Felix Lange <fjl@twurst.com>
2024-02-16 19:05:33 +01:00
rjl493456442
fe91d476ba
all: remove the dependency from trie to triedb (#28824)
This change removes the dependency from trie package to triedb package.
2024-02-13 14:49:53 +01:00
Shivam Sharma
4008d5efe8 Merge branch 'develop' into shivam/upstream-geth-1.13.5 2024-02-08 18:00:59 +05:30
Pratik Patil
36f2ae5b37
core: reset tx lookup cache if necessary (#28865) (#1157)
This pull request resets the txlookup cache if chain reorg happens, 
preventing them from remaining reachable. It addresses failures in
the hive tests.

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2024-02-07 15:00:16 +05:30
Dimitris Apostolou
8fd43c8013
all: fix typos in comments (#28881) 2024-02-05 22:16:32 +01:00
rjl493456442
eaac53ec38
core: reset tx lookup cache if necessary (#28865)
This pull request resets the txlookup cache if chain reorg happens, 
preventing them from remaining reachable. It addresses failures in
the hive tests.
2024-01-30 09:34:14 +08:00
Pratik Patil
3f58da7279 hot fix: block stm ParallelSpeculativeProcesses flag 2024-01-29 16:52:40 +05:30
Dimitris Apostolou
ce68cb17ac
fix typos 2024-01-29 09:21:47 +02:00
rjl493456442
6b0de79935
core: move tx indexer to its own file (#28857)
This change moves all the transaction indexing functions to a separate txindexer.go file and defines a txIndexer structure as a refactoring.
2024-01-23 21:00:50 +01:00