Commit graph

234 commits

Author SHA1 Message Date
Matthieu Vachon
7b77261b7c Added back OnNewAccount which is Firehose concept so I left it out of the merge commit just before this one 2024-03-13 16:24:14 -04:00
Matthieu Vachon
f47d2fc315 Merge branch 'extended-tracer-struct' into firehose-fh3.0
# Conflicts:
#	core/blockchain.go
#	core/state/statedb.go
#	eth/backend.go
#	eth/tracers/live/printer.go
#	go.mod
#	go.sum
2024-03-13 16:23:13 -04:00
Sina Mahmoodi
73ca96b97e resolve merge conflict 2024-03-12 17:55:17 +01:00
Péter Szilágyi
b393ad8d29
cmd, core, metrics: always report expensive metrics (#29191)
* cmd, core, metrics: always report expensive metrics

* core, metrics: report block processing metrics as resetting timer

* metrics: update reporter tests
2024-03-11 10:06:57 +02:00
rjl493456442
7b81cf6362
core/state, trie/triedb/pathdb: remove storage incomplete flag (#28940)
As SELF-DESTRUCT opcode is disabled in the cancun fork(unless the
account is created within the same transaction, nothing to delete
in this case). The account will only be deleted in the following
cases:

- The account is created within the same transaction. In this case
the original storage was empty.

- The account is empty(zero nonce, zero balance, zero code) and
is touched within the transaction. Fortunately this kind of accounts
are not-existent on ethereum-mainnet.

All in all, after cancun, we are pretty sure there is no large contract
deletion and we don't need this mechanism for oom protection.
2024-03-05 14:31:55 +01:00
Sina Mahmoodi
a2829c6f49 rename LiveLogger to hooks 2024-02-28 19:38:04 +01:00
Sina Mahmoodi
7d8db9b503 move logger interface to core/tracing 2024-02-28 19:34:49 +01:00
Sina Mahmoodi
fc35780124 full struct-based tracing infra 2024-02-21 18:47:59 +01:00
Sina Mahmoodi
50095906e2 use struct for tracing events 2024-02-16 11:20:56 +01:00
Sina Mahmoodi
57cd0c30df rm OnNewAccount 2024-02-08 16:30:14 +01:00
Sina Mahmoodi
188cd4182e remove precompile check for newAccount 2024-02-05 18:37:57 +01:00
Sina Mahmoodi
6c44a594f4 fix self destruct burn condition 2024-02-05 18:29:53 +01:00
Matthieu Vachon
a46903cf0c Reduce amount of changes to Geth Live Tracer to keep Firehose backward compatibility
We removed the `precompile` check on `OnNewAccount` and moved it to the Firehose tracer directly. This way, we can remove our custom logic that toggle the check at the tracing code level and we can instead move the logic of backward compatibility back `FirehoseTracer`.

Refactored a bit also how precompiles are check. We moved the precompiles checker up to the block level now that `chainConfig` is available on `OnBlockStart` meaning we can extract the list of active precompiles. However, to make it work, we had to modify `OnGenesisBlock` to also receive `chainConfig` parameter, I've ask to add it to the geth codebase directly.
2024-02-05 12:01:21 -05:00
Matthieu Vachon
8767058368 Merge branch 'extended-tracer' into firehose-fh3.0
# Conflicts:
#	cmd/evm/internal/t8ntool/execution.go
#	consensus/beacon/consensus.go
#	consensus/ethash/consensus.go
#	consensus/misc/dao.go
#	core/genesis.go
#	core/state/statedb.go
#	core/state/statedb_test.go
#	core/state_transition.go
#	core/txpool/blobpool/blobpool_test.go
#	core/txpool/legacypool/legacypool2_test.go
#	core/txpool/legacypool/legacypool_test.go
#	core/vm/evm.go
#	core/vm/interface.go
#	tests/state_test_util.go
2024-02-02 15:08:47 -05:00
Sina Mahmoodi
56862cf07f emit onNewAccount when reseting account 2024-01-30 16:36:14 +01:00
Sina Mahmoodi
4c1d675041 resolve merge conflict 2024-01-29 20:00:43 +01:00
Martin HS
a5a4fa7032
all: use uint256 in state (#28598)
This change makes use of uin256 to represent balance in state. It touches primarily upon statedb, stateobject and state processing, trying to avoid changes in transaction pools, core types, rpc and tracers.
2024-01-23 14:51:58 +01:00
Arun Dhyani
eae09aabda create account deleted corner case fix 2024-01-17 18:43:33 +08:00
Arun Dhyani
0b70fbd76a account creation fixes 2024-01-17 10:46:52 +08:00
Sina Mahmoodi
d07264afe0 no balance change reason on 0 balance selfdestruct 2024-01-16 21:03:16 +03:30
Matthieu Vachon
8545daf9f9 Fixed some small stuff while merging extended-tracer in 2024-01-15 14:03:07 -05:00
Darioush Jalali
29b73555ae
core/state: unexport GetOrNewStateObject (#28804) 2024-01-14 12:32:23 +01:00
Sina Mahmoodi
aa0db68d3a resolve merge conflict 2024-01-12 17:58:24 +03:30
cygaar
d2e3cb894b
core/state: logic equivalence for GetCodeHash (#28733) 2023-12-26 16:38:11 +08:00
Sina Mahmoodi
2d14bb573c rename and document balance change reasons 2023-12-13 17:52:56 +03:30
Sina Mahmoodi
5e82cbec8d nil check 2023-12-11 15:59:23 +03:30
Sina Mahmoodi
0967291487 Track burnt and selfdestruct withdraw bal changes 2023-12-07 18:07:14 +03:30
Sina Mahmoodi
c5e407ba66 resolve merge conflicts 2023-12-05 18:13:41 +03:30
rjl493456442
ab0eb46a84
core/state: make stateobject.create selfcontain (#28459) 2023-11-29 16:07:51 +08:00
Sina Mahmoodi
00ee953593 rm comment 2023-11-28 14:40:20 +03:30
Sina Mahmoodi
9c00b1290b fix merge conflict 2023-11-24 16:09:47 +03:30
Sina Mahmoodi
b80aa1efb8 fix double-allocation 2023-11-24 12:35:20 +03:30
Guillaume Ballet
fa8d39807d
cmd, core, trie: verkle-capable geth init (#28270)
This change allows the creation of a genesis block for verkle testnets. This makes for a chunk of code that is easier to review and still touches many discussion points.
2023-11-14 13:09:40 +01:00
Sina Mahmoodi
ee58cc79d9 statedb: precompile check before onNewAccount 2023-11-07 15:41:58 +03:00
Sina Mahmoodi
14d603b009 add comment re newAccount precompile 2023-11-03 17:20:15 +01:00
Sina Mahmoodi
429dcc99ca fix merge conflict 2023-11-02 15:58:17 +01:00
rjl493456442
1b1611b8d0
core, trie, eth: refactor stacktrie constructor (#28350)
This change enhances the stacktrie constructor by introducing an option struct. It also simplifies the `Hash` and `Commit` operations, getting rid of the special handling round root node.
2023-10-17 14:09:25 +02:00
Martin Holst Swende
8976a0c97a
trie: remove owner and binary marshaling from stacktrie (#28291)
This change
  - Removes the owner-notion from a stacktrie; the owner is only ever needed for comitting to the database, but the commit-function, the `writeFn` is provided by the caller, so the caller can just set the owner into the `writeFn` instead of having it passed through the stacktrie.
  - Removes the `encoding.BinaryMarshaler`/`encoding.BinaryUnmarshaler` interface from stacktrie. We're not using it, and it is doubtful whether anyone downstream is either.
2023-10-11 06:12:45 +02:00
Darioush Jalali
48fdb79de5
core/state: check err for iter.Error in fastDeleteStorage (#28122)
core/state: check err for iter.Error
2023-09-15 14:09:07 +08:00
Martin Holst Swende
8b6cf128af
metrics: refactor metrics (#28035)
This change includes a lot of things, listed below. 

### Split up interfaces, write vs read

The interfaces have been split up into one write-interface and one read-interface, with `Snapshot` being the gateway from write to read. This simplifies the semantics _a lot_. 

Example of splitting up an interface into one readonly 'snapshot' part, and one updatable writeonly part: 

```golang
type MeterSnapshot interface {
	Count() int64
	Rate1() float64
	Rate5() float64
	Rate15() float64
	RateMean() float64
}

// Meters count events to produce exponentially-weighted moving average rates
// at one-, five-, and fifteen-minutes and a mean rate.
type Meter interface {
	Mark(int64)
	Snapshot() MeterSnapshot
	Stop()
}
```

### A note about concurrency

This PR makes the concurrency model clearer. We have actual meters and snapshot of meters. The `meter` is the thing which can be accessed from the registry, and updates can be made to it. 

- For all `meters`, (`Gauge`, `Timer` etc), it is assumed that they are accessed by different threads, making updates. Therefore, all `meters` update-methods (`Inc`, `Add`, `Update`, `Clear` etc) need to be concurrency-safe. 
- All `meters` have a `Snapshot()` method. This method is _usually_ called from one thread, a backend-exporter. But it's fully possible to have several exporters simultaneously: therefore this method should also be concurrency-safe. 

TLDR: `meter`s are accessible via registry, all their methods must be concurrency-safe. 

For all `Snapshot`s, it is assumed that an individual exporter-thread has obtained a `meter` from the registry, and called the `Snapshot` method to obtain a readonly snapshot. This snapshot is _not_ guaranteed to be concurrency-safe. There's no need for a snapshot to be concurrency-safe, since exporters should not share snapshots. 

Note, though: that by happenstance a lot of the snapshots _are_ concurrency-safe, being unmutable minimal representations of a value. Only the more complex ones are _not_ threadsafe, those that lazily calculate things like `Variance()`, `Mean()`.

Example of how a background exporter typically works, obtaining the snapshot and sequentially accessing the non-threadsafe methods in it: 
```golang
		ms := metric.Snapshot()
                ...
		fields := map[string]interface{}{
			"count":    ms.Count(),
			"max":      ms.Max(),
			"mean":     ms.Mean(),
			"min":      ms.Min(),
			"stddev":   ms.StdDev(),
			"variance": ms.Variance(),
```

TLDR: `snapshots` are not guaranteed to be concurrency-safe (but often are).

### Sample changes

I also changed the `Sample` type: previously, it iterated the samples fully every time `Mean()`,`Sum()`, `Min()` or `Max()` was invoked. Since we now have readonly base data, we can just iterate it once, in the constructor, and set all four values at once. 

The same thing has been done for runtimehistogram. 

### ResettingTimer API

Back when ResettingTImer was implemented, as part of https://github.com/ethereum/go-ethereum/pull/15910, Anton implemented a `Percentiles` on the new type. However, the method did not conform to the other existing types which also had a `Percentiles`. 

1. The existing ones, on input, took `0.5` to mean `50%`. Anton used `50` to mean `50%`. 
2. The existing ones returned `float64` outputs, thus interpolating between values. A value-set of `0, 10`, at `50%` would return `5`, whereas Anton's would return either `0` or `10`. 

This PR removes the 'new' version, and uses only the 'legacy' percentiles, also for the ResettingTimer type. 

The resetting timer snapshot was also defined so that it would expose the internal values. This has been removed, and getters for `Max, Min, Mean` have been added instead. 

### Unexport types

A lot of types were exported, but do not need to be. This PR unexports quite a lot of them.
2023-09-13 13:13:47 -04:00
Sina Mahmoodi
6ee4fb8eac resolve merge conflicts 2023-09-13 16:17:36 +02:00
rjl493456442
a7842c9cae
core, trie: cleanup trie database (#28062) 2023-09-07 21:17:14 +08:00
rjl493456442
0acc0a1f86
core/state: simplify storage trie update and commit (#28030)
This change improves function description and simplifies logic in statedb update and commit operations.
2023-08-31 14:33:18 -04:00
Sina Mahmoodi
0be6e22eaf add comment to statedb logger 2023-08-31 16:35:43 +02:00
Martin Holst Swende
d4e345c7d4
core/state: fix missing import (#28010) 2023-08-26 04:43:36 -04:00
rjl493456442
3ff6b3c31e
core/state: implement fast storage deletion (#27955)
This changes implements faster post-selfdestruct iteration of storage slots for deletion, by using snapshot-storage+stacktrie to recover the trienodes to be deleted. This mechanism is only implemented for path-based schema. 

For hash-based schema, the entire post-selfdestruct storage iteration is skipped, with this change, since hash-based does not actually perform deletion anyway. 

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-26 04:13:22 -04:00
Martin Holst Swende
eb6cbe37e1
core/state: remove public method ForEachStorage (#27986)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-23 21:53:31 +02:00
Guillaume Ballet
67979022aa
core/state: move UpdateContractCode before the trie hash is computed (#27853)
Context: The UpdateContractCode method was introduced for the state storage commitment
schemes that include the whole code for their commitment computation. It must therefore be called
before the root hash is computed at the end of IntermediateRoot.

This should have no impact on the MPT since, in this context, the method is a no-op.
2023-08-09 18:02:45 +02:00
rjl493456442
7de748d3f6
all: implement path-based state scheme (#25963)
* all: implement path-based state scheme

* all: edits from review

* core/rawdb, trie/triedb/pathdb: review changes

* core, light, trie, eth, tests: reimplement pbss history

* core, trie/triedb/pathdb: track block number in state history

* trie/triedb/pathdb: add history documentation

* core, trie/triedb/pathdb: address comments from Peter's review

Important changes to list:

- Cache trie nodes by path in clean cache
- Remove root->id mappings when history is truncated

* trie/triedb/pathdb: fallback to disk if unexpect node in clean cache

* core/rawdb: fix tests

* trie/triedb/pathdb: rename metrics, change clean cache key

* trie/triedb: manage the clean cache inside of disk layer

* trie/triedb/pathdb: move journal function

* trie/triedb/path: fix tests

* trie/triedb/pathdb: fix journal

* trie/triedb/pathdb: fix history

* trie/triedb/pathdb: try to fix tests on windows

* core, trie: address comments

* trie/triedb/pathdb: fix test issues

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-01 15:17:32 +03:00
rjl493456442
817553cc28
core, trie: track state change set with account address (#27815) 2023-07-31 15:07:51 +03:00