Preallocate capacity for `keyOffsets` and `valOffsets` slices in
`decodeRestartTrailer` since the exact size (`nRestarts`) is known
upfront.
---------
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
This PR optimizes the historical trie node reader by reworking how data
is accessed and memory is managed, reducing allocation overhead
significantly.
Specifically:
- Instead of decoding an entire history object to locate a specific trie node,
the reader now searches directly within the history.
- Besides, slice pre-allocation can avoid unnecessary deep-copy significantly.
It's a PR based on #33303 and introduces an approach for trienode
history indexing.
---
In the current archive node design, resolving a historical trie node at
a specific block
involves the following steps:
- Look up the corresponding trie node index and locate the first entry
whose state ID
is greater than the target state ID.
- Resolve the trie node from the associated trienode history object.
A naive approach would be to store mutation records for every trie node,
similar to
how flat state mutations are recorded. However, the total number of trie
nodes is
extremely large (approximately 2.4 billion), and the vast majority of
them are rarely
modified. Creating an index entry for each individual trie node would be
very wasteful
in both storage and indexing overhead. To address this, we aggregate
multiple trie
nodes into chunks and index mutations at the chunk level instead.
---
For a storage trie, the trie is vertically partitioned into multiple sub
tries, each spanning
three consecutive levels. The top three levels (1 + 16 + 256 nodes) form
the first chunk,
and every subsequent three-level segment forms another chunk.
```
Original trie structure
Level 0 [ ROOT ] 1 node
Level 1 [0] [1] [2] ... [f] 16 nodes
Level 2 [00] [01] ... [0f] [10] ... [ff] 256 nodes
Level 3 [000] [001] ... [00f] [010] ... [fff] 4096 nodes
Level 4 [0000] ... [000f] [0010] ... [001f] ... [ffff] 65536 nodes
Vertical split into chunks (3 levels per chunk)
Level0 [ ROOT ] 1 chunk
Level3 [000] ... [fff] 4096 chunks
Level6 [000000] ... [fffffff] 16777216 chunks
```
Within each chunk, there are 273 nodes in total, regardless of the
chunk's depth in the trie.
```
Level 0 [ 0 ] 1 node
Level 1 [ 1 ] … [ 16 ] 16 nodes
Level 2 [ 17 ] … … [ 272 ] 256 nodes
```
Each chunk is uniquely identified by the path prefix of the root node of
its corresponding
sub-trie. Within a chunk, nodes are identified by a numeric index
ranging from 0 to 272.
For example, suppose that at block 100, the nodes with paths `[]`,
`[0]`, `[f]`, `[00]`, and `[ff]`
are modified. The mutation record for chunk 0 is then appended with the
following entry:
`[100 → [0, 1, 16, 17, 272]]`, `272` is the numeric ID of path `[ff]`.
Furthermore, due to the structural properties of the Merkle Patricia
Trie, if a child node
is modified, all of its ancestors along the same path must also be
updated. As a result,
in the above example, recording mutations for nodes `00` and `ff` alone
is sufficient,
as this implicitly indicates that their ancestor nodes `[]`, `[0]` and
`[f]` were also
modified at block 100.
---
Query processing is slightly more complicated. Since trie nodes are
indexed at the chunk
level, each individual trie node lookup requires an additional filtering
step to ensure that
a given mutation record actually corresponds to the target trie node.
As mentioned earlier, mutation records store only the numeric
identifiers of leaf nodes,
while ancestor nodes are omitted for storage efficiency. Consequently,
when querying
an ancestor node, additional checks are required to determine whether
the mutation
record implicitly represents a modification to that ancestor.
Moreover, since trie nodes are indexed at the chunk level, some trie
nodes may be
updated frequently, causing their mutation records to dominate the
index. Queries
targeting rarely modified trie nodes would then scan a large amount of
irrelevant
index data, significantly degrading performance.
To address this issue, a bitmap is introduced for each index block and
stored in the
chunk's metadata. Before loading a specific index block, the bitmap is
checked to
determine whether the block contains mutation records relevant to the
target trie node.
If the bitmap indicates that the block does not contain such records,
the block is skipped entirely.
In this PR, several changes have been made:
(a) restructure the trienode history header section
Previously, the offsets of the key and value sections were recorded before
encoding data into these sections. As a result, these offsets referred to the
start position of each chunk rather than the end position.
This caused an issue where the end position of the last chunk was
unknown, making it incompatible with the freezer partial-read APIs.
With this update, all offsets now refer to the end position, and the
start position of the first chunk is always 0.
(b) Enable partial freezer read for trienode data retrieval
The partial freezer read feature is now utilized in trienode data
retrieval, improving efficiency.
The limit check for `MaxUint32` is done after the cast to `int`. On 64
bits machines, that will work without a problem. On 32 bits machines,
that will always fail. The compiler catches it and refuses to build.
Note that this only fixes the compiler build. ~~If the limit is above
`MaxInt32` but strictly below `MaxUint32` then this will fail at runtime
and we have another issue.~~ I checked and this should not happen during
regular execution, although it might happen in tests.
