Commit graph

2398 commits

Author SHA1 Message Date
Martin Holst Swende
49a09d7e1d cmd/geth: more testcases for logging (#28501)
* cmd/geth: more testcases for logging

This adds more edgecases around logging, particularly around handling of different types of nil-values
as concrete types and within interfaces.

Also adds tests with 'reserved' values which breaks json/logfmt formats. The json output is checked in,
but not actively used by any testcase at the moment.

* cmd/geth/testdata: remove timestamps
2023-11-10 11:09:46 -06:00
Mario Vega
36d01e79d5 cmd/evm: add --run option to blocktest command (#28421)
Co-authored-by: lightclient <lightclient@protonmail.com>
2023-11-10 11:09:46 -06:00
Martin Holst Swende
5f793d2339 all: make vendored copy of reexec (#28382)
a little copying is better than a little dependency

-- go proverb

We have this dependency on docker, a.k.a moby: a gigantic library, and we only need ~70 LOC,
so here I tried moving it inline instead.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-11-10 11:09:46 -06:00
Guillaume Ballet
1183299ce5 core, cmd/geth: add --override.* flags to geth init (#28407)
* core, cmd/geth: add --override.* flags to geth init

* also apply overrides before genesis commit with new block

* review feedback
2023-11-10 11:09:46 -06:00
Martin Holst Swende
6a1deaea39 cmd/geth: test for logging-output (#28373)
This PR is a bit in preparation for the slog work in #28187 .

Our current test re logging mostly test the internals, but we have no real end-to-end test of the logging output. This PR introduces a simple reexec-based log tester. This also relies upon a special mode in geth, which can be made to eject a set of predefined log messages (only available if the build-tag `integrationtests` is used

e.g. go run --tags=integrationtests ./cmd/geth --log.format terminal logtest

While working on this, I also noticed a quirk in the setup: when geth was configured to use a file output, then two separate handlers were used (one handler for the file, one handler for the console). Using two separate handlers means that two formatters are used, thus the formatting of any/all records happened twice. This PR changes the mechanism to use two separate io.Writers instead, which is both more optimal and fixes a bug which occurs due to a global statefulness in the formatter.
2023-11-10 11:09:46 -06:00
Martin Holst Swende
b5c13fa95c cmd/evm: make t8ntool handle transaction decoding errors better (#28397)
This change closes https://github.com/ethereum/go-ethereum/issues/27730 . By using an iterator instead of a slice of transactions, we can better handle the case when an individual transaction (within an otherwise well-formed RLP-list) cannot be decoded.
2023-11-10 11:09:46 -06:00
xiaolou86
e95e022a8a cmd: fix typos (#28323) 2023-11-10 11:09:46 -06:00
Martin Holst Swende
077f13f68a tests: update execution-spec-tests to 1.0.5 (#28337)
Updates execution-spec-tests to 1.0.5: https://github.com/ethereum/execution-spec-tests/releases/tag/v1.0.5, switching to develop which contains Cancun tests (which are also enabled in this change).
2023-11-10 11:09:46 -06:00
Péter Szilágyi
4d2914cf13 cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD too (#28335)
* cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD too

* ethdb/pebble: use Pebble's internal constant calculation
2023-11-10 11:09:46 -06:00
Péter Szilágyi
c2a35a6961 go.mod: pull in the latest cloudflare API libs (#28336) 2023-11-10 11:09:46 -06:00
Martin Holst Swende
51d0547794 trie: make rhs-proof align with last key in range proofs (#28311)
During snap-sync, we request ranges of values: either a range of accounts or a range of storage values. For any large trie, e.g. the main account trie or a large storage trie, we cannot fetch everything at once.

Short version; we split it up and request in multiple stages. To do so, we use an origin field, to say "Give me all storage key/values where key > 0x20000000000000000". When the server fulfils this, the server provides the first key after origin, let's say 0x2e030000000000000 -- never providing the exact origin. However, the client-side needs to be able to verify that the 0x2e03.. indeed is the first one after 0x2000.., and therefore the attached proof concerns the origin, not the first key.

So, short-short version: the left-hand side of the proof relates to the origin, and is free-standing from the first leaf.

On the other hand, (pun intended), the right-hand side, there's no such 'gap' between "along what path does the proof walk" and the last provided leaf. The proof must prove the last element (unless there are no elements).

Therefore, we can simplify the semantics for trie.VerifyRangeProof by removing an argument. This doesn't make much difference in practice, but makes it so that we can remove some tests. The reason I am raising this is that the upcoming stacktrie-based verifier does not support such fancy features as standalone right-hand borders.
2023-11-10 11:09:46 -06:00
rjl493456442
631db5c5ae eth/protocols/snap: fix snap sync failure on empty storage range (#28306)
This change addresses an issue in snap sync, specifically when the entire sync process can be halted due to an encountered empty storage range.

Currently, on the snap sync client side, the response to an empty (partial) storage range is discarded as a non-delivery. However, this response can be a valid response, when the particular range requested does not contain any slots.

For instance, consider a large contract where the entire key space is divided into 16 chunks, and there are no available slots in the last chunk [0xf] -> [end]. When the node receives a request for this particular range, the response includes:

    The proof with origin [0xf]
    A nil storage slot set

If we simply discard this response, the finalization of the last range will be skipped, halting the entire sync process indefinitely. The test case TestSyncWithUnevenStorage can reproduce the scenario described above.

In addition, this change also defines the common variables MaxAddress and MaxHash.
2023-11-10 11:09:46 -06:00
Martin Holst Swende
7f451ef421 cmd/geth: fix failing test (#28322) 2023-11-10 11:09:46 -06:00
vuittont60
a067689e26 accounts, cmd: fix typos (#28300) 2023-11-10 11:09:46 -06:00
rjl493456442
9a372b6518 cmd, core: resolve scheme from a read-write database (#28313)
* cmd, core: resolve scheme from a read-write database

* cmd, core, eth: move the scheme check in the ethereum constructor

* cmd/geth: dump should in ro mode

* cmd: reverts
2023-11-10 11:09:46 -06:00
Martin Holst Swende
6ffa8094b3 all: move light.NodeSet to trienode.ProofSet (#28287)
This is a minor refactor in preparation of changes to range verifier. This PR contains no intentional functional changes but moves (and renames) the light.NodeSet
2023-11-10 11:09:46 -06:00
tactical_retreat
f22ae6a9e3 cmd/evm/internal/t8ntoo: tiny bugfix for difficulty field (#28245) 2023-11-10 11:09:46 -06:00
Péter Szilágyi
293fc408d9 cmd/devp2p, eth: drop eth/66 (#28239)
* cmd/devp2p, eth: drop eth/66

* eth/protocols/eth: yes sir, linter
2023-11-10 11:09:46 -06:00
lightclient
e457ceff01 cmd/evm: cancun-updates for b11r and t8n -tools (#28195)
This change updates `evm b11r` (blockbuilder) and `evm t8n` (transition) tools to contain cancun updates (e.g. new header fields)
---------

Co-authored-by: Mario Vega <marioevz@gmail.com>
2023-11-10 11:09:46 -06:00
Marius van der Wijden
5585e79348 core: implement BLOBBASEFEE opcode (0x4a) (#28098)
Implements "EIP-7516: BLOBBASEFEE opcode" for cancun, as per spec: https://eips.ethereum.org/EIPS/eip-7516
2023-11-10 11:09:46 -06:00
Péter Szilágyi
f7547536c8 cmd, eth: switch the dev synctarget to hash from block (#28209)
* cmd, eth: switch the dev synctarget to hash from block

* cmd/utils, eth/catalyst: terminate node wyen synctarget reached
2023-11-10 11:09:46 -06:00
lightclient
97e54bda96 params: update 4788 beacon roots contract addr (#28205)
This change contains the final (?) address for 4788 beacon root contract. The update to the EIP is being tracked here: https://github.com/ethereum/EIPs/pull/7672

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-11-10 11:09:46 -06:00
rjl493456442
bcc2e34f7e core, accounts, eth, trie: handle genesis state missing (#28171)
* core, accounts, eth, trie: handle genesis state missing

* core, eth, trie: polish

* core: manage txpool subscription in mainpool

* eth/backend: fix test

* cmd, eth: fix test

* core/rawdb, trie/triedb/pathdb: address comments

* eth, trie: address comments

* eth: inline the function

* eth: use synced flag

* core/txpool: revert changes in txpool

* core, eth, trie: rename functions
2023-11-10 11:09:46 -06:00
buddho
877c5be6fe cmd/utils: fix bootnodes config priority (#28095)
This fixes an issue where the --bootnodes flag was overridden by the config file.

---------

Co-authored-by: NathanBSC <Nathan.l@nodereal.io>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-11-10 11:09:46 -06:00
Delweng
a7871054a9 cmd/geth: ensure db is closed before exit (#28150) 2023-11-10 11:09:46 -06:00
Péter Szilágyi
fa255843f6 cmd: add state.scheme to the database flag group for local flag handling (#28107) 2023-11-10 11:09:46 -06:00
Péter Szilágyi
d19fe8e4fe cmd/geth: print progress logs when iterating large contracts too (#28179) 2023-11-10 11:09:46 -06:00
phenix3443
552517f260 cmd/utils: fix typo in comment (#28159) 2023-11-10 11:09:46 -06:00
Delweng
a7205347fb cmd/devp2p: use bootnodes as crawl input (#28139)
This PR makes the tool use the --bootnodes list as the input to devp2p crawl.
The flag will take effect if the input/output.json file is missing or empty.
2023-11-10 11:09:46 -06:00
Felix Lange
0264a9ad74 cmd/evm: fix some issues with the evm run command (#28109)
* cmd/evm: improve flags handling

This fixes some issues with flags in cmd/evm. The supported flags did not
actually show up in help output because they weren't categorized. I'm also
adding the VM-related flags to the run command here so they can be given
after the subcommand name. So it can be run like this now:

   ./evm run --code 6001 --debug

* cmd/evm: enable all forks by default in run command

The default genesis was just empty with no forks at all, which is annoying because
contracts will be relying on opcodes introduced in a fork. So this changes the default to
have all forks enabled.

* core/asm: fix some issues in the assembler

This fixes minor bugs in the old assembler:

- It is now possible to have comments on the same line as an instruction.
- Errors for invalid numbers in the jump instruction are reported better
- Line numbers in errors were off by one
2023-11-10 11:09:46 -06:00
Péter Szilágyi
c59e4a47dc cmd/geth, internal/flags: print envvar config source and bad names (#28119) 2023-11-10 11:09:46 -06:00
Péter Szilágyi
953f7964db cmd/geth, internal/flags, go.mod: colorize cli help, support env vars (#28103)
* cmd/geth, internal/flags, go.mod: colorize cli help, support env vars

* internal/flags: use stdout, not stderr for terminal detection
2023-11-10 11:09:46 -06:00
Péter Szilágyi
c28d7e1b8e cmd/geth: rename the protocols field in the metrics gague (#28102) 2023-11-10 11:09:46 -06:00
Martin Holst Swende
ebad10b3af deps: update minisign (#28066)
This updates minisign to the latest version. One new thing is that minisign (not go-minisign) has started to prehash the file, and in order to make geth pass the version-check, we need to sign the file in legacy-mode.
2023-11-10 11:09:46 -06:00
lightclient
cfdfeea6ad core/forkid: skip genesis forks by time (#28034)
* core/forkid: skip genesis forks by time

* core/forkid: add comment about skipping non-zero fork times

* core/forkid: skip all time based forks in genesis using loop

* core/forkid: simplify logic for dropping time-based forks
2023-11-10 11:09:46 -06:00
Jorge
c10e9371e2 metrics, cmd/geth: informational metrics (prometheus, influxdb, opentsb) (#24877)
This chang creates a GaugeInfo metrics type for registering informational (textual) metrics, e.g. geth version number. It also improves the testing for backend-exporters, and uses a shared subpackage in 'internal' to provide sample datasets and ordered registry. 

Implements #21783

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-11-10 11:09:46 -06:00
Péter Szilágyi
8f5d0635ce cmd, core, params: add support for the Holesky testnet (#28007)
* cmd, core, params: add support for the Holesky testnet

* cmd/devp2p: add support for holesky for the dns crawler
2023-11-10 11:09:46 -06:00
Péter Szilágyi
22b083a163 cmd/utils: fix a startup issue on deleted chaindata but dangling ancients (#27989) 2023-11-10 11:09:46 -06:00
Martin Holst Swende
f28ec27bcd cmd/evm: add back stateroot to jsonl-output (#27968)
The PR #26274 broke the evm statetest command a bit, in that it stopped spitting out the stateroot following a non-successful statetest-execution. 

This PR changes it back, so the stateroot is unconditionally output on stderr, and makes it so fuzzing works again.
2023-11-10 11:09:46 -06:00
Martin Holst Swende
2e9477f46b core, params, beacon/engine: implement EIP 4788 BeaconRoot (#27849)
This change implements "EIP 4788 : Beacon block root in the EVM". It implements version-2 of EPI-4788, main difference being that the contract is an actual contract rather than a precompile, as in #27289.
2023-11-10 11:09:46 -06:00
lightclient
db5684444a core/forkid: correctly compute forkid when timestamp fork is activated in genesis (#27895)
This changes the forkID calculation to ignore time-based forks that occurred before the
genesis block. It's supposed to be done this way because the spec says:

> If a chain is configured to start with a non-Frontier ruleset already in its genesis, that is NOT considered a fork.
2023-11-10 11:09:46 -06:00
Delweng
a6e1f48a51 cmd/utils: restore support for txlookuplimit flag (#27917)
This fixes a regression where -txlookuplimit was not applied anymore.
2023-11-10 11:09:46 -06:00
Péter Szilágyi
1c411fcea2 all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-11-10 11:09:46 -06:00
rjl493456442
1053526246 all: activate pbss as experimental feature (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-11-10 11:09:46 -06:00
Delweng
45e662000f cmd/geth: disable automaxprocs log (#27814)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-11-10 11:09:46 -06:00
ucwong
9ad7158974 cmd/bootnode: fix timer leak (#27754) 2023-11-10 11:09:46 -06:00
Delweng
1680bf1b71 cmd/evm: set ExcessBlobGas from env (#27796)
Sets the `currentExcessBlobGas` from env, alternatively calculates it based on `parentExcessBlobGas` and `parentBlobGasUsed`. It then emits the `currentExcessBlobGas` and `currentBlobGasUsed` into the output, to be used as parent-values for a future iteration. 

Closes #27785
Closes #27783

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-11-10 11:09:46 -06:00
Delweng
b2f7349a6b consensus/misc: move eip1559 into a package (#27828)
* consensus/misc: move eip1559 as a sub directory

Signed-off-by: jsvisa <delweng@gmail.com>

* consensus/misc: package name

Signed-off-by: jsvisa <delweng@gmail.com>

* all: eip1559

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-11-10 11:09:46 -06:00
Zig Blathazar
6d08d1239a cmd/bootnode: defer udp connection close (#27787) 2023-11-10 11:09:46 -06:00
Shude Li
13645bb8d5 cmd/geth: import package catalyst once (#27803) 2023-11-10 11:09:46 -06:00