go-ethereum/triedb/pathdb
CPerezz 437a53bbe0
triedb/pathdb: implement bintrieFlatCodec + stem blob helpers
Introduce the codec and on-disk blob format for the bintrie flat-state
layer. This commit only defines the types; the codec is NOT wired into
pathdb.Database.New yet (that happens in a later commit once the
leaf-production hook in binaryHasher and the stateUpdate wiring are in
place).

Three pieces:

1. trie/bintrie/pack.go

   Canonical PackBasicData / UnpackBasicData helpers that encode an
   account's (codeSize, nonce, balance) into the 32-byte BasicData leaf
   defined by EIP-7864. Preserves the existing BinaryTrie.UpdateAccount
   layout byte-for-byte (4-byte code_size at offset 4 rather than the
   spec's 3-byte field at offset 5 — any realistic code size has byte 4
   always zero and the two encodings are bit-equivalent in practice).

   BinaryTrie.UpdateAccount is refactored to delegate to PackBasicData
   so the flat-state codec can produce a bit-identical BasicData
   encoding without duplicating the layout logic.

2. triedb/pathdb/stem_blob.go

   Packed encoding of the populated (offset, value) pairs at a bintrie
   stem. A stem can hold up to 256 offsets per EIP-7864 but in practice
   only a handful are set; the layout is a 32-byte bitmap followed by
   N 32-byte values in ascending offset order, where N = popcount.
   Empty stems encode to nil so the caller knows to delete the on-disk
   key rather than write a zero-length value.

   Provides encodeStemBlob / decodeStemBlob / extractStemOffset /
   mergeStemBlob and a stemBuilder type for accumulating writes. The
   tombstone convention (32 zero bytes = "present with zero" as used
   by DeleteStorage) is preserved.

   11 unit tests cover: empty blob, BasicData+CodeHash roundtrip, all
   256 offsets populated, sparse high offsets, set/clear roundtrip,
   load-from-existing-blob RMW, merge helper, merge-to-empty, tombstone
   zero bytes, malformed input detection, bitmap rank sanity.

3. triedb/pathdb/flat_codec_bintrie.go

   bintrieFlatCodec implements flatStateCodec over the stem-blob layout.
   Unlike merkleFlatCodec it is stateful: it holds a ethdb.KeyValueReader
   reference used by applyWrites to read the existing stem blob before
   merging in new writes. ethdb.Batch is write-only so the batch passed
   to Write* cannot be used to fetch current state.

   Pre-aggregation requirement is documented explicitly: within a single
   flush, the caller must NOT issue two Write* calls targeting the same
   stem, because the RMW read comes from the store (not the in-flight
   batch). Commit 8 of the bintrie flat-state plan restructures
   writeStates to pre-aggregate per-stem writes so callers don't have
   to handle this manually.

   Cache keys are prefix-disambiguated with a one-byte 0x01 to keep
   bintrie stem lookups disjoint from merkle 32-byte account keys and
   64-byte storage keys in the shared clean-state fastcache.

   SplitMarker is a single-tier (stem-only) format, not the merkle
   two-tier (account, account+storage) format.

   7 unit tests cover: account roundtrip, storage roundtrip, multiple
   writes to the same stem, DeleteAccount preserving unrelated offsets,
   DeleteStorage removing the final offset collapsing the key, cache
   key disjointness from merkle, SplitMarker semantics.

The codec is not dispatched by anything yet; MPT continues through the
merkle codec and bintrie mode still runs on the (soon-to-be-replaced)
keccak-shaped path until Commit 10 wires things up.
2026-04-15 15:00:40 +02:00
..
buffer.go triedb/pathdb: thread flatStateCodec through internals 2026-04-15 15:00:39 +02:00
config.go triedb/pathdb: enhance history index initer (#33640) 2026-03-17 15:29:30 +01:00
context.go triedb/pathdb: thread flatStateCodec through internals 2026-04-15 15:00:39 +02:00
database.go triedb/pathdb: thread flatStateCodec through internals 2026-04-15 15:00:39 +02:00
database_test.go core, eth, ethclient, triedb: report trienode index progress (#34633) 2026-04-04 21:00:07 +08:00
difflayer.go triedb/pathdb: return nodeLoc by value to avoid heap allocation (#33819) 2026-02-11 22:14:43 +08:00
difflayer_test.go cmd, core, eth, triedb/pathdb: track node origins in the path database (#32418) 2025-09-05 10:37:05 +08:00
disklayer.go triedb/pathdb: thread flatStateCodec through internals 2026-04-15 15:00:39 +02:00
errors.go core, triedb/pathdb: final integration (snapshot integration pt 5) (#30661) 2025-05-16 18:29:38 +08:00
execute.go cmd, core, eth, triedb/pathdb: track node origins in the path database (#32418) 2025-09-05 10:37:05 +08:00
fileutils_unix.go triedb/pathdb: introduce file-based state journal (#32060) 2025-07-15 11:45:20 +08:00
fileutils_windows.go triedb/pathdb: introduce file-based state journal (#32060) 2025-07-15 11:45:20 +08:00
flat_codec.go triedb/pathdb: introduce flatStateCodec abstraction 2026-04-15 15:00:39 +02:00
flat_codec_bintrie.go triedb/pathdb: implement bintrieFlatCodec + stem blob helpers 2026-04-15 15:00:40 +02:00
flat_codec_bintrie_test.go triedb/pathdb: implement bintrieFlatCodec + stem blob helpers 2026-04-15 15:00:40 +02:00
flush.go triedb/pathdb: thread flatStateCodec through internals 2026-04-15 15:00:39 +02:00
generate.go triedb/pathdb: thread flatStateCodec through internals 2026-04-15 15:00:39 +02:00
generate_test.go triedb/pathdb, eth: use double-buffer mechanism in pathdb (#30464) 2025-06-22 20:40:54 +08:00
history.go core/rawdb, triedb/pathdb: enable trienode history alongside existing data (#33934) 2026-03-12 09:21:54 +08:00
history_index.go triedb/pathdb: improve trienode reader for searching (#33681) 2026-01-27 20:05:35 +08:00
history_index_block.go triedb/pathdb: improve trienode reader for searching (#33681) 2026-01-27 20:05:35 +08:00
history_index_block_test.go triedb/pathdb: introduce extension to history index structure (#33399) 2026-01-08 09:57:35 +01:00
history_index_iterator.go core, eth, internal, triedb/pathdb: enable eth_getProofs for history (#32727) 2026-01-22 09:19:27 +08:00
history_index_iterator_test.go core, eth, internal, triedb/pathdb: enable eth_getProofs for history (#32727) 2026-01-22 09:19:27 +08:00
history_index_pruner.go triedb/pathdb: implement history index pruner (#33999) 2026-04-02 00:21:58 +02:00
history_index_pruner_test.go triedb/pathdb: implement history index pruner (#33999) 2026-04-02 00:21:58 +02:00
history_index_test.go triedb/pathdb: introduce extension to history index structure (#33399) 2026-01-08 09:57:35 +01:00
history_indexer.go triedb/pathdb: implement history index pruner (#33999) 2026-04-02 00:21:58 +02:00
history_indexer_state.go triedb/pathdb: enhance history index initer (#33640) 2026-03-17 15:29:30 +01:00
history_indexer_test.go triedb/pathdb: enhance history index initer (#33640) 2026-03-17 15:29:30 +01:00
history_inspect.go triedb/pathdb: allow single-element history ranges (#33329) 2025-12-01 10:19:21 +08:00
history_reader.go triedb/pathdb: improve trienode reader for searching (#33681) 2026-01-27 20:05:35 +08:00
history_reader_test.go triedb/pathdb: implement trienode history indexing scheme (#33551) 2026-01-17 20:28:37 +08:00
history_state.go triedb/pathdb: implement trienode history indexing scheme (#33551) 2026-01-17 20:28:37 +08:00
history_state_test.go core/rawdb, triedb/pathdb: introduce trienode history (#32596) 2025-10-10 14:51:27 +08:00
history_trienode.go triedb/pathdb: preallocate slices in decodeRestartTrailer (#33715) 2026-01-30 21:14:15 +08:00
history_trienode_test.go triedb/pathdb: improve trienode reader for searching (#33681) 2026-01-27 20:05:35 +08:00
history_trienode_utils.go triedb/pathdb: implement trienode history indexing scheme (#33551) 2026-01-17 20:28:37 +08:00
history_trienode_utils_test.go triedb/pathdb: implement trienode history indexing scheme (#33551) 2026-01-17 20:28:37 +08:00
holdable_iterator.go trie/pathdb: state iterator (snapshot integration pt 4) (#30654) 2024-12-16 21:10:08 +08:00
holdable_iterator_test.go trie/pathdb: state iterator (snapshot integration pt 4) (#30654) 2024-12-16 21:10:08 +08:00
iterator.go triedb, triedb/internal, triedb/pathdb: add GenerateTrie + extract shared pipeline into triedb/internal (#34654) 2026-04-07 14:36:53 +08:00
iterator_binary.go triedb/pathdb, eth: use double-buffer mechanism in pathdb (#30464) 2025-06-22 20:40:54 +08:00
iterator_fast.go triedb/pathdb, eth: use double-buffer mechanism in pathdb (#30464) 2025-06-22 20:40:54 +08:00
iterator_test.go triedb/pathdb: thread flatStateCodec through internals 2026-04-15 15:00:39 +02:00
journal.go triedb/pathdb: bump journal version to 4 2026-04-15 15:00:40 +02:00
layertree.go triedb/pathdb: fix lookup sentinel collision with zero disk layer root (#34680) 2026-04-09 13:39:38 +08:00
layertree_test.go triedb/pathdb: fix lookup sentinel collision with zero disk layer root (#34680) 2026-04-09 13:39:38 +08:00
lookup.go triedb/pathdb: fix lookup sentinel collision with zero disk layer root (#34680) 2026-04-09 13:39:38 +08:00
metrics.go triedb/pathdb: implement history index pruner (#33999) 2026-04-02 00:21:58 +02:00
nodes.go core, eth, internal, triedb/pathdb: enable eth_getProofs for history (#32727) 2026-01-22 09:19:27 +08:00
nodes_test.go rlp, trie, triedb/pathdb: compress trienode history (#32913) 2026-01-08 21:58:02 +08:00
reader.go triedb/pathdb: thread flatStateCodec through internals 2026-04-15 15:00:39 +02:00
states.go triedb/pathdb: thread flatStateCodec through internals 2026-04-15 15:00:39 +02:00
states_test.go all: update license comments and AUTHORS (#31133) 2025-02-05 23:01:17 +01:00
stem_blob.go triedb/pathdb: implement bintrieFlatCodec + stem blob helpers 2026-04-15 15:00:40 +02:00
stem_blob_test.go triedb/pathdb: implement bintrieFlatCodec + stem blob helpers 2026-04-15 15:00:40 +02:00
verifier.go triedb, triedb/internal, triedb/pathdb: add GenerateTrie + extract shared pipeline into triedb/internal (#34654) 2026-04-07 14:36:53 +08:00