This changes the p2p protocol handlers to delay message decoding. It's
the first part of a larger change that will delay decoding all the way
through message processing. For responses, we delay the decoding until
it is confirmed that the response matches an active request and does not
exceed its limits.
In order to make this work, all messages have been changed to use
rlp.RawList instead of a slice of the decoded item type. For block
bodies specifically, the decoding has been delayed all the way until
after verification of the response hash.
The role of p2p/tracker.Tracker changes significantly in this PR. The
Tracker's original purpose was to maintain metrics about requests and
responses in the peer-to-peer protocols. Each protocol maintained a
single global Tracker instance. As of this change, the Tracker is now
always active (regardless of metrics collection), and there is a
separate instance of it for each peer. Whenever a response arrives, it
is first verified that a request exists for it in the tracker. The
tracker is also the place where limits are kept.
This PR adds OpenTelemetry tracing configuration to geth via
command-line flags. When enabled, geth initializes the global
OpenTelemetry TracerProvider and installs standard trace context
propagation. When disabled (the default), tracing remains a no-op and
behavior is unchanged.
Co-authored-by: Felix Lange <fjl@twurst.com>
The endSpan closure accepted error by value, meaning deferred calls like
defer spanEnd(err) captured the error at defer-time (always nil), not at
function-return time. This meant errors were never recorded on spans.
- Changed endSpan to accept *error
- Updated all call sites in rpc/handler.go to pass error pointers, and
adjusted handleCall to avoid propagating child-span errors to the parent
- Added TestTracingHTTPErrorRecording to verify that errors from RPC
methods are properly recorded on the rpc.runMethod span
I removed `Iterator.Count` in #33840, because it appeared to be unused
and did not provide the documented invariant: the returned count should
always be an upper bound on the number of iterations allowed by `Next`.
In order to make `Count` work, the semantics of `CountValues` has to
change to return the number of items up and including the invalid one. I
have reviewed all callsites of `CountValues` to assess if changing this
is safe. There aren't that many, and the only call that doesn't check
the error and return is in the trie node parser,
`trie.decodeNodeUnsafe`. There, we distinguish the node type based on
the number of items, and it previously returned an error for item count
zero. In order to avoid any potential issue that could result from this
change, I'm adding an error check in that function, though it isn't
necessary.
This changes `RawList` to ensure the count of items is always valid.
Lists with invalid structure, i.e. ones where an element exceeds the
size of the container, are now detected during decoding of the `RawList`
and thus cannot exist.
Also remove `RawList.Empty` since it is now fully redundant, and
`Iterator.Count` since it returns incorrect results in the presence of
invalid input. There are no callers of these methods (yet).
The reasoning for using the cleartext format here is that the JSON-RPC
API is internal only. Providers which expose it publicly already put it
behind a proxy which handles also the encryption.
This fixes two cases where `Iterator.Err()` was misused. The method will
only return an error after `Next()` has returned false, so it makes no
sense to check for the error within the loop itself.
Most uses of the iterator are like this:
it, _ := rlp.NewListIterator(data)
for it.Next() {
do(it.Value())
}
This doesn't require the iterator to be a pointer and it's better to
have it stack-allocated. AFAIK the compiler cannot prove it is OK to
stack-allocate when it is returned as a pointer because the methods of
`Iterator` use pointer receiver and also mutate the object.
The iterator type was not exported until very recently, so I think it is
still OK to change this API.
GetStorage and DeleteStorage used GetBinaryTreeKey to compute the tree
key, while UpdateStorage used GetBinaryTreeKeyStorageSlot. The latter
applies storage slot remapping (header offset for slots <64, main
storage prefix for the rest), so reads and deletes were targeting
different tree locations than writes.
Replace GetBinaryTreeKey with GetBinaryTreeKeyStorageSlot in both
GetStorage and DeleteStorage to match UpdateStorage. Add a regression
test that verifies the write→read→delete→read round-trip for main
storage slots.
The `decodeRef` function used `size > hashLen` to reject oversized
embedded nodes, but this incorrectly allowed nodes of exactly 32 bytes
through. The encoding side (hasher.go, stacktrie.go) consistently uses
`len(enc) < 32` to decide whether to embed a node inline, meaning nodes
of 32+ bytes are always hash-referenced. The error message itself
already stated `want size < 32`, confirming the intended threshold.
Changed `size > hashLen` to `size >= hashLen` in `decodeRef` to align
the decoding validation with the encoding logic, the Yellow Paper spec,
and the surrounding comments.
This PR fixes a panic in a corner case situation when a `ChainEvent` is
received by `eth.Ethereum.updateFilterMapsHeads()` but the given chain
section does not exist in `BlockChain` any more. This can happen during
chain rewind because chain events are processed asynchronously. Ignoring
the event in this case is ok, the final event will point to the final
rewound head and the indexer will be updated.
Note that similar issues will not happen once we transition to
https://github.com/ethereum/go-ethereum/pull/32292 and the new indexer
built on top of this. Until then, the current fix should be fine.
This PR makes `TestEIP8024_Execution` verify explicit error types (e.g.,
`ErrStackUnderflow` vs `ErrInvalidOpCode`) rather than accepting any
error. It also fails fast on unexpected opcodes in the mini-interpreter
to avoid false positives from missing opcode handling.
Here is a draft for the New EraE implementation. The code follows along
with the spec listed at https://hackmd.io/pIZlxnitSciV5wUgW6W20w.
---------
Co-authored-by: shantichanal <158101918+shantichanal@users.noreply.github.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
### Problem
`HasBody` and `HasReceipts` returned `true` for pruned blocks because
they only checked `isCanon()` which verifies the hash table — but
hash/header tables have `prunable: false` while body/receipt tables have
`prunable: true`.
After `TruncateTail()`, hashes still exist but bodies/receipts are gone.
This caused inconsistency: `HasBody()` returns `true`, but `ReadBody()`
returns `nil`.
### Changes
Both functions now check `db.Tail()` when the block is in ancient store.
If `number < tail`, the data has been pruned and the function correctly
returns `false`.
This aligns `HasBody`/`HasReceipts` behavior with
`ReadBody`/`ReadReceipts` and fixes potential issues in
`skeleton.linked()` which relies on these checks during sync.
Follow-up to https://github.com/ethereum/go-ethereum/pull/33748
Same issue - ResettingTimer can be registered via loadOrRegister() but
GetAll() silently drops it during JSON export. The prometheus exporter
handles it fine (collector.go:70), so this is just an oversight in the
JSON path.
Note: ResettingTimer.Snapshot() resets the timer by design, which is
consistent with how the prometheus exporter uses it.
This adds a new type wrapper that decodes as a list, but does not
actually decode the contents of the list. The type parameter exists as a
marker, and enables decoding the elements lazily. RawList can also be
used for building a list incrementally.
The upstream libray has removed the assembly-based implementation of
keccak. We need to maintain our own library to avoid a peformance
regression.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
kzg4844.Blob is 131072 bytes. Using `for _, blob := range` copies the
entire blob on each iteration. With up to 6 blobs per transaction, this
wastes ~768KB of memory copies.
Switch to index-based iteration and pass pointers directly.
The `Witness` method was not implemented for the binary tree, which
caused `debug_excutionWitness` to panic. This PR fixes that.
Note that the `TransitionTrie` version isn't implemented, and that's on
purpose: more thought must be given to what should go in the global
witness.
I recently went on a longer flight and started profiling the geth block
production pipeline.
This PR contains a bunch of individual fixes split into separate
commits.
I can drop some if necessary.
Benchmarking is not super easy, the benchmark I wrote is a bit
non-deterministic.
I will try to write a better benchmark later
```
goos: linux
goarch: amd64
pkg: github.com/ethereum/go-ethereum/miner
cpu: Intel(R) Core(TM) Ultra 7 155U
│ /tmp/old.txt │ /tmp/new.txt │
│ sec/op │ sec/op vs base │
BuildPayload-14 141.5µ ± 3% 146.0µ ± 6% ~ (p=0.346 n=200)
│ /tmp/old.txt │ /tmp/new.txt │
│ B/op │ B/op vs base │
BuildPayload-14 188.2Ki ± 4% 177.4Ki ± 4% -5.71% (p=0.018 n=200)
│ /tmp/old.txt │ /tmp/new.txt │
│ allocs/op │ allocs/op vs base │
BuildPayload-14 2.703k ± 4% 2.453k ± 5% -9.25% (p=0.000 n=200)
```
Preallocates slices with known capacity in `stateSet.encode()` and
`StateSetWithOrigin.encode()` methods to eliminate redundant
reallocations during serialization.
Preallocate capacity for `keyOffsets` and `valOffsets` slices in
`decodeRestartTrailer` since the exact size (`nRestarts`) is known
upfront.
---------
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
core/state: add bounds check in heap eviction loop
Add len(h) > 0 check before accessing h[0] to prevent potential panic
and align with existing heap access patterns in txpool, p2p, and mclock
packages.
