d.partialSyncComplete is consulted by beaconBackfiller.resume() to skip
redundant downloader cycles after the initial partial-state sync has
finished. It was an in-memory atomic.Bool, so every process restart
reset it to false, and the next forkchoiceUpdated from the CL would
re-enter the sync loop.
Persist the flag in leveldb via a new PartialSyncComplete marker:
- Add ReadPartialSyncComplete / WritePartialSyncComplete /
DeletePartialSyncComplete accessors in core/rawdb/accessors_chain.go
backed by a single-byte value under the PartialSyncComplete key.
- Write the marker in the downloader right after AdvancePartialHead
succeeds (same spot we flip the in-memory flag).
- Rehydrate the in-memory flag from leveldb in Downloader.New() so a
freshly-started process with a completed partial-state sync keeps
the resume short-circuit active from the first beacon forkchoice.
Without this, the restart invariant relied on HasState(header.Root)
accidentally returning false to reroute the downloader back to
SnapSync; with this the resume guard is the primary protection
regardless of how header-root convergence evolves.
AdvancePartialHead's backfill loop used a strictly-greater condition, so it
wrote canonical-hash keys only for blocks above the pivot. Combined with
the Engine API path persisting the pivot via WriteBlockWithoutState (which
writes header+body but not the canonical-hash key) and InsertReceiptChain.writeLive
skipping the pivot because HasBlock already returned true, the pivot block
ended up without an H<num>n entry in leveldb. After the freezer advanced
past finalized, startup's gap check at rawdb/database.go:279 rejected the
datadir with "gap in the chain between ancients [0 - #N-1] and leveldb
[#N+1 - #head]".
Fix: explicitly write the canonical hash for currentHead at the start of
AdvancePartialHead's backfill, covering the pivot inclusively.
Also add a defensive guard in the chain-retention freezer path so that
TruncateTail never prunes past lastPivotNumber. Partial-state mode relies
on the pivot block as the anchor for state reconstruction; pruning its
body from ancients would make a future reorg spanning the pivot
unrecoverable.
Ship with a regression test that asserts AdvancePartialHead writes the
currentHead's canonical hash (covers the bug precondition directly), plus
an idempotency check and a small post-advance sanity test.
Verified end-to-end on bal-devnet-3:
- Before fix: Fatal on restart
- After fix: restart succeeds, BAL processing resumes within seconds,
verify_partial_sync_devnet3.sh passes 16/16 checks.
Apply review fixes: BAL iterator start (Fix 2), fatal root mismatch when
all storage resolved (Fix 3), WriteBlockWithoutState error handling (Fix 4),
contract filter construction order (Fix 5), canonical hash backfill (Fix 6),
underflow guard in gap processing (Fix 8), O(n²) prepend fix (Fix 9),
ReadBALHistory corruption detection (Fix 11), incomplete resolution error
(Fix 13), RLP encode panic (Fix 14), gap processing log level (Fix 16),
TriggerPartialResync message (Fix 18), and comment accuracy fixes.
Remove the stateRoot field and sync.RWMutex from PartialState entirely.
Since partial state maintains the full account trie, the computed root
always matches the header root (assuming storage root resolution succeeds).
ProcessBlockWithBAL now derives parent root from parent.Root() directly,
matching how full nodes derive state root from currentBlock headers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add chain retention for partial state mode: only the most recent N blocks
(default 1024) retain bodies and receipts. During sync, older blocks are
skipped entirely. After sync, the freezer enforces a rolling window.
Add engine API support for Block Access Lists (EIP-7928): NewPayloadV5
accepts BAL data alongside execution payloads, enabling partial state
nodes to receive per-block storage access information from the CL.
Fix beacon backfilling failure caused by dynamic chain cutoff not
clearing the cutoff hash (which remained at the genesis hash).
Add partial state awareness to eth_call/eth_estimateGas to return clear
errors when accessing untracked contract storage.
Implements EIP-7928 BAL-based partial statefulness infrastructure:
- Add PartialStateConfig to eth/ethconfig with CLI flags
- Add ContractFilter interface in core/state/partial/
- Add BAL history database accessors in core/rawdb/
- Add PartialState and BALHistory managers
This enables nodes to track only configured contracts' storage
while maintaining full account trie integrity.
Add persistent storage for Block Access Lists (BALs) in `core/rawdb/`.
This provides read/write/delete accessors for BALs in the active
key-value store.
---------
Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Fix incorrect key length calculation for `numHashPairings` in
`InspectDatabase`, introduced in #34000.
The `headerHashKey` format is `headerPrefix + num + headerHashSuffix`
(10 bytes), but the check incorrectly included `common.HashLength`,
expecting 42 bytes.
This caused all number -- hash entries to be misclassified as
unaccounted data.
This PR improves `db inspect` classification accuracy in
`core/rawdb/database.go` by tightening key-shape checks for:
- `Block number->hash`
- `Difficulties (deprecated)`
Previously, both categories used prefix/suffix heuristics and could
mis-bucket unrelated entries.
Fixes https://github.com/ethereum/go-ethereum/issues/33907
Notably there is a behavioral change:
- Previously Geth will refuse to restart if the existing trienode
history is gapped with the state data
- With this PR, the gapped trienode history will be entirely reset and
being constructed from scratch
Pebble maintains a batch pool to recycle the batch object. Unfortunately
batch object must be
explicitly returned via `batch.Close` function. This PR extends the
batch interface by adding
the close function and also invoke batch.Close in some critical code
paths.
Memory allocation must be measured before merging this change. What's
more, it's an open
question that whether we should apply batch.Close as much as possible in
every invocation.
This fixes two cases where `Iterator.Err()` was misused. The method will
only return an error after `Next()` has returned false, so it makes no
sense to check for the error within the loop itself.
Here is a draft for the New EraE implementation. The code follows along
with the spec listed at https://hackmd.io/pIZlxnitSciV5wUgW6W20w.
---------
Co-authored-by: shantichanal <158101918+shantichanal@users.noreply.github.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
### Problem
`HasBody` and `HasReceipts` returned `true` for pruned blocks because
they only checked `isCanon()` which verifies the hash table — but
hash/header tables have `prunable: false` while body/receipt tables have
`prunable: true`.
After `TruncateTail()`, hashes still exist but bodies/receipts are gone.
This caused inconsistency: `HasBody()` returns `true`, but `ReadBody()`
returns `nil`.
### Changes
Both functions now check `db.Tail()` when the block is in ancient store.
If `number < tail`, the data has been pruned and the function correctly
returns `false`.
This aligns `HasBody`/`HasReceipts` behavior with
`ReadBody`/`ReadReceipts` and fixes potential issues in
`skeleton.linked()` which relies on these checks during sync.
The upstream libray has removed the assembly-based implementation of
keccak. We need to maintain our own library to avoid a peformance
regression.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
This PR fixes an issue where the tx indexer would repeatedly try to
“unindex” a block with a missing body, causing a spike in CPU usage.
This change skips these blocks and advances the index tail. The fix was
verified both manually on a local development chain and with a new test.
resolves#33371
Adds missing trienode freezer case to InspectFreezerTable, making it
consistent with InspectFreezer which already supports it.
Co-authored-by: m6xwzzz <maskk.weller@gmail.com>
Fix#33212.
This PR remove `github.com/olekukonko/tablewriter` from dependencies and
use a naive stub implementation.
`github.com/olekukonko/tablewriter` is used to format database inspection
output neatly. However, it requires custom adjustments for TinyGo and is
incompatible with the latest version.
---------
Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
The iterator loop in findTxInBlockBody returned the outer-scoped err
when iter.Err() was non-nil, which could incorrectly propagate a nil or
stale error and hide actual RLP decoding issues. This patch returns
iter.Err() as intended by the rlp list iterator API, matching
established patterns elsewhere in the codebase and improving diagnostics
when encountering malformed transaction entries.
In this PR, several changes have been made:
(a) restructure the trienode history header section
Previously, the offsets of the key and value sections were recorded before
encoding data into these sections. As a result, these offsets referred to the
start position of each chunk rather than the end position.
This caused an issue where the end position of the last chunk was
unknown, making it incompatible with the freezer partial-read APIs.
With this update, all offsets now refer to the end position, and the
start position of the first chunk is always 0.
(b) Enable partial freezer read for trienode data retrieval
The partial freezer read feature is now utilized in trienode data
retrieval, improving efficiency.
This PR implements the partial read functionalities in the freezer, optimizing
the state history reader by resolving less data from freezer.
---------
Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
- Replace outdated NewFreezer doc that referenced map[string]bool/snappy
toggle with accurate description of -map[string]freezerTableConfig
(noSnappy, prunable).
- Fix misleading field comment on freezerTable.config that spoke as if
it were a boolean (“if true”), clarifying it’s a struct and noting
compression is non-retroactive.
Fix the t.Fatalf format arguments in TestBadBlockStorage to match the
intended #index output. Previously, the left number used i+1 and the
right index used the block number, producing misleading diagnostics.
Correct mapping improves test failure clarity and debuggability.
`db inspect` on the full database currently takes **30min+**, because
the db iterate was run in one thread, propose to split the key-space to
256 sub range, and assign them to the worker pool to speed up.
After the change, the time of running `db inspect --workers 16` reduced
to **10min**(the keyspace is not evenly distributed).
---------
Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This pull request preserves the root->ID mappings in the path database
even after the associated state histories are truncated, regardless of
whether the truncation occurs at the head or the tail.
The motivation is to support an additional history type, trienode history.
Since the root->ID mappings are shared between two history instances,
they must not be removed by either one.
As a consequence, the root->ID mappings remain in the database even
after the corresponding histories are pruned. While these mappings may
become dangling, it is safe and cheap to keep them.
Additionally, this pull request enhances validation during historical
reader construction, ensuring that only canonical historical state will be
served.
