Commit graph

450 commits

Author SHA1 Message Date
Jerry
48bdfa7ffb Fix tx index limit not being used correctly 2024-10-09 07:59:21 -07:00
Jerry
352fb8adfa
Fix txIndexer 2024-09-20 19:49:26 -07:00
Jerry
d9daacad17
Address goroutine leak 2024-09-12 15:56:33 -07:00
anshalshukla
a66305dae3
fix: lint, add: milestone related tests in downloader 2024-09-05 14:03:10 +05:30
anshalshukla
cec3ee18d8
fix: testcases 2024-09-03 16:21:20 +05:30
anshalshukla
43043ed7ec
fix: few linting errors 2024-09-02 11:00:33 +05:30
anshalshukla
c0ff6dbf81
fix: testcases, add: block broadcasting mechanism 2024-08-30 14:43:02 +05:30
anshalshukla
687cd76405
fix: testcases 2024-08-19 18:07:05 +05:30
anshalshukla
b1419eb41d
fix: testcases 2024-08-13 16:46:46 +05:30
anshalshukla
5cda3fe0f0
fix: testcases 2024-08-13 11:39:52 +05:30
anshalshukla
e6132fdf1e
fix: testcases 2024-08-09 11:59:34 +05:30
Anshal Shukla
03f253c1e0
add block fetcher back, fix handler and blockchain errors 2024-08-06 13:44:27 +05:30
Anshal Shukla
dbf55ce6cc
Merge tag 'v1.14.6' into upstream_merge_1.14.6 2024-08-05 16:29:10 +05:30
Anshal Shukla
f4c41e2e70
Merge branch 'release/1.13' into upstream_merge_1.13 2024-07-10 11:19:34 +05:30
jwasinger
ed8fd0ac09
all: stateless witness builder and (self-)cross validator (#29719)
* all: add stateless verifications

* all: simplify witness and integrate it into live geth

---------

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2024-06-25 14:48:08 +03:00
jwasinger
85587d5ef2
cmd, core: prefetch reads too from tries if requested (#29807)
* cmd/utils, consensus/beacon, core/state: when configured via stub  flag: prefetch all reads from account/storage tries, terminate prefetcher synchronously.

* cmd, core/state: fix nil panic, fix error handling, prefetch nosnap too

* core/state: expand prefetcher metrics for reads and writes separately

* cmd/utils, eth: fix noop collect witness flag

---------

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2024-06-11 11:10:07 +03:00
Péter Szilágyi
2ac83e197b
core/state: blocking prefetcher on term signal, parallel updates (#29519)
* core/state: trie prefetcher change: calling trie() doesn't stop the associated subfetcher

Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>

* core/state: improve prefetcher

* core/state: restore async prefetcher stask scheduling

* core/state: finish prefetching async and process storage updates async

* core/state: don't use the prefetcher for missing snapshot items

* core/state: remove update concurrency for Verkle tries

* core/state: add some termination checks to prefetcher async shutdowns

* core/state: differentiate db tries and prefetched tries

* core/state: teh teh teh

---------

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2024-05-13 15:47:45 +03:00
Manav Darji
d95c05b98a
Enable ancient block pruning (#1216)
* core/state: typo

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: backport from https://github.com/bnb-chain/bsc/pull/543

Signed-off-by: Delweng <delweng@gmail.com>

* eth,ethdb,node,core/state: backport from https://github.com/bnb-chain/bsc/pull/543

Signed-off-by: Delweng <delweng@gmail.com>

* eth,core: backport from https://github.com/bnb-chain/bsc/pull/543

Signed-off-by: Delweng <delweng@gmail.com>

* cmd: open db with freeze disabled

Signed-off-by: Delweng <delweng@gmail.com>

* cli: snapshot prune-block

Signed-off-by: Delweng <delweng@gmail.com>

* fix typo

Signed-off-by: Delweng <delweng@gmail.com>

* cli/snapshot: fix the issue of dup open db error

Signed-off-by: Delweng <delweng@gmail.com>

* cli/snapshot: resolve datadir and ancient before backup

Signed-off-by: Delweng <delweng@gmail.com>

* core: more prune-block log

Signed-off-by: Delweng <delweng@gmail.com>

* core: truncatetail missing f.offset

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: indextx adjust offset of pruned block

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: freezer batch should implement the offset commit, ref https://github.com/bnb-chain/bsc/pull/1005

Signed-off-by: Delweng <delweng@gmail.com>

* core: check of ancientdb, backport https://github.com/bnb-chain/bsc/pull/817

Signed-off-by: Delweng <delweng@gmail.com>

* core/state: read raw borReceipt to backup

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: bor receipt maybe in []Receipt or Receipt RLP format

Signed-off-by: Delweng <delweng@gmail.com>

* core/state: typo and error msg

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: offSet -> offset

Signed-off-by: Delweng <delweng@gmail.com>

* cli/snapshot: comment

Signed-off-by: Delweng <delweng@gmail.com>

* cli/snapshot: add prune-block doc

Signed-off-by: Delweng <delweng@gmail.com>

* docs: add prune-block document

Signed-off-by: Delweng <delweng@gmail.com>

* core/rawdb: print wrong bor-receipt length

Signed-off-by: Delweng <delweng@gmail.com>

* internal/cli: add snapshot prune block tests (referenced from bsc's PR)

* improve errors

* cmd, core, eth, internal: fix lint

* internal/cli: refactor snapshot prune block test

* fix linters in tests

* internal/cli: add inspect-ancient-db command, update docs

* pruner: use a generic function for simplification

* internal/cli: fixes for inspect-db command

* internal/cli: improve pruning tests

* core/rawdb: update end block calculation logic in inspect command

* core/rawdb: improve checks db initialisation

* core/rawdb: remove offset check

* update mocks for span, ethdb and add command in makefile

* docs/cli: update docs with inspect command

* go mod tidy

* refactor and resolve conflicts

* resolve more conflicts

* refactor

* explicitly read node for hash scheme

* add check for hash scheme, fix tests

* fix typo

* update docs and add warning

* raise error if pbss is enabled

* revert read raw bor receipt change

* consensus/bor: handle nil header case in get root hash

* address comments

* core/rawdb: check chain continuity by matching parent hash

* core/rawdb: account for pruned ancient blocks

* go mod tidy

* fix tests

* fix tests

---------

Signed-off-by: Delweng <delweng@gmail.com>
Co-authored-by: Delweng <delweng@gmail.com>
2024-05-09 11:19:34 +05:30
Guillaume Ballet
43cbcd78ea
core, core/state: move TriesInMemory to state package (#29701) 2024-05-06 13:28:53 +02:00
Péter Szilágyi
682ee820fa
core/state: parallelise parts of state commit (#29681)
* core/state, internal/workerpool: parallelize parts of state commit

* core, internal: move workerpool into syncx

* core/state: use errgroups, commit accounts concurrently

* core: resurrect detailed commit timers to almost-accuracy
2024-05-02 11:18:27 +03:00
maskpp
fecc8a0f4a
cmd/evm/internal/t8ntool, core: prealloc map sizes where possible (#29620)
set cap for map in a certain scenario
2024-04-30 11:19:59 +02:00
Péter Szilágyi
4253030ef6
core/state: move metrics out of state objects (#29665) 2024-04-26 18:35:52 +03:00
Chris Ziogas
fb08fd334a
core/tracing: Add OnClose Trace Hook (#29629)
The OnClose trace hook is being triggered on blockchain Stop, so as tracers can release any resources.
2024-04-24 07:54:59 +02:00
rjl493456442
acd1eaae2c
core: remove bad block checks (#29609) 2024-04-22 20:00:42 +03:00
rjl493456442
1126c6d8a5
core: add txlookup lock (#29343)
This change adds a lock to the transaction lookup cache, to avoid the case where reorgs make the lookup return inconsistent results.
2024-04-09 08:37:18 +02:00
Aaron Chen
ed4bc7f27b
all: replace fmt.Errorf() with errors.New() if no param required (#29472) 2024-04-08 12:59:17 +02:00
rjl493456442
3b77e0ff4b
core: remove unused code (#29381) 2024-03-28 12:06:57 +01:00
Guillaume Ballet
da7469e5c4
core: add an end-to-end verkle test (#29262)
core: add a simple verkle test

triedb, core: skip hash comparison in verkle

core: remove legacy daoFork logic in verkle chain maker

fix: nil pointer in tests

triedb/pathdb: add blob hex

core: less defensive

Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2024-03-26 21:25:41 +01:00
Aaron Chen
723b1e36ad
all: fix mismatched names in comments (#29348)
* all: fix mismatched names in comments

* metrics: fix mismatched name in UpdateIfGt
2024-03-26 21:01:28 +01:00
Sina M
064f37d6f6
eth/tracers: live chain tracing with hooks (#29189)
Here we add a Go API for running tracing plugins within the main block import process. 

As an advanced user of geth, you can now create a Go file in eth/tracers/live/, and within
that file register your custom tracer implementation. Then recompile geth and select your tracer
on the command line. Hooks defined in the tracer will run whenever a block is processed.

The hook system is defined in package core/tracing. It uses a struct with callbacks, instead of 
requiring an interface, for several reasons:

- We plan to keep this API stable long-term. The core/tracing hook API does not depend on
  on deep geth internals.
- There are a lot of hooks, and tracers will only need some of them. Using a struct allows you
   to implement only the hooks you want to actually use.

All existing tracers in eth/tracers/native have been rewritten to use the new hook system.

This change breaks compatibility with the vm.EVMLogger interface that we used to have.
If you are a user of vm.EVMLogger, please migrate to core/tracing, and sorry for breaking
your stuff. But we just couldn't have both the old and new tracing APIs coexist in the EVM.

---------

Co-authored-by: Matthieu Vachon <matthieu.o.vachon@gmail.com>
Co-authored-by: Delweng <delweng@gmail.com>
Co-authored-by: Martin HS <martin@swende.se>
2024-03-22 18:53:53 +01:00
rjl493456442
78c102dec5
core: skip the check the statefulness of head block in repair (#29245) 2024-03-20 13:11:30 +01:00
Shivam Sharma
feeca914ef Merge branch 'v1.2.9-beta-candidate' into shivam/temp-master-develop 2024-03-20 14:55:19 +05:30
Manav Darji
deb34da04d
core: remove tracing and handle err in reorg call (#1193) 2024-03-19 16:27:33 +05:30
rjl493456442
c170fa277c
core: improve chain rewinding mechanism (#29196)
* core: improve chain rewinding mechanism

* core: address comment

* core: periodically print progress log

* core: address comments

* core: fix comment

* core: fix rewinding in path

* core: fix beyondRoot condition

* core: polish code

* core: polish code

* core: extend code comment

* core: stop rewinding if chain is gapped or genesis is reached

* core: fix broken tests
2024-03-13 13:39:30 +02:00
Péter Szilágyi
b393ad8d29
cmd, core, metrics: always report expensive metrics (#29191)
* cmd, core, metrics: always report expensive metrics

* core, metrics: report block processing metrics as resetting timer

* metrics: update reporter tests
2024-03-11 10:06:57 +02:00
Péter Szilágyi
f4d53133f6
consensus, cmd, core, eth: remove support for non-merge mode of operation (#29169)
* eth: drop support for forward sync triggers and head block packets

* consensus, eth: enforce always merged network

* eth: fix tx looper startup and shutdown

* cmd, core: fix some tests

* core: remove notion of future blocks

* core, eth: drop unused methods and types
2024-03-05 16:13:28 +02:00
Sina Mahmoodi
95741b1844
core: move genesis alloc types to core/types (#29003)
We want to use these types in public user-facing APIs, so they shouldn't be in core.

Co-authored-by: Felix Lange <fjl@twurst.com>
2024-02-16 19:05:33 +01:00
rjl493456442
fe91d476ba
all: remove the dependency from trie to triedb (#28824)
This change removes the dependency from trie package to triedb package.
2024-02-13 14:49:53 +01:00
Shivam Sharma
4008d5efe8 Merge branch 'develop' into shivam/upstream-geth-1.13.5 2024-02-08 18:00:59 +05:30
Pratik Patil
36f2ae5b37
core: reset tx lookup cache if necessary (#28865) (#1157)
This pull request resets the txlookup cache if chain reorg happens, 
preventing them from remaining reachable. It addresses failures in
the hive tests.

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2024-02-07 15:00:16 +05:30
Dimitris Apostolou
8fd43c8013
all: fix typos in comments (#28881) 2024-02-05 22:16:32 +01:00
rjl493456442
eaac53ec38
core: reset tx lookup cache if necessary (#28865)
This pull request resets the txlookup cache if chain reorg happens, 
preventing them from remaining reachable. It addresses failures in
the hive tests.
2024-01-30 09:34:14 +08:00
Pratik Patil
3f58da7279 hot fix: block stm ParallelSpeculativeProcesses flag 2024-01-29 16:52:40 +05:30
Dimitris Apostolou
ce68cb17ac
fix typos 2024-01-29 09:21:47 +02:00
rjl493456442
6b0de79935
core: move tx indexer to its own file (#28857)
This change moves all the transaction indexing functions to a separate txindexer.go file and defines a txIndexer structure as a refactoring.
2024-01-23 21:00:50 +01:00
rjl493456442
78a3c32ef4
core, core/rawdb, eth/sync: no tx indexing during snap sync (#28703)
This change simplifies the logic for indexing transactions and enhances the UX when transaction is not found by returning more information to users.

Transaction indexing is now considered as a part of the initial sync, and `eth.syncing` will thus be `true` if transaction indexing is not yet finished. API consumers can use the syncing status to determine if the node is ready to serve users.
2024-01-22 21:05:18 +01:00
Shivam Sharma
fcd5002b6b fix : flags, FLOCK lock 2024-01-19 17:55:39 +05:30
Shivam Sharma
c0bf289a84 fix : testcases 2024-01-10 17:12:37 +05:30
Shivam Sharma
8c088c7295 fix : resolve static problems 2023-12-15 13:57:19 +05:30
Shivam Sharma
a9fa325b7e Merge tag 'v1.13.5' of https://github.com/ethereum/go-ethereum into shivam/upstream-geth-1.13.5 2023-12-15 13:55:57 +05:30