Commit graph

7 commits

Author SHA1 Message Date
weiihann
cb3e13d93d nomt/merkle: add Phase 7 parallel workers for trie updates
Parallelize the PageWalker trie update across multiple goroutines by
partitioning sorted operations by the root page's 64 child subtrees
(first 6 bits of each key path).

Each worker runs an independent PageWalker constrained to child pages
below the root (using parentPage mechanism), producing ChildPageRoots.
After all workers complete, a root walker places the child roots using
AdvanceAndPlaceNode and concludes with the final trie root.

Workers operate on disjoint page subtrees so no synchronization is
needed during computation — only sync.WaitGroup for goroutine join.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 18:37:42 +08:00
weiihann
cf10f3d997 nomt/db: add Phase 5 unified NOMT trie database
Implement the orchestration layer combining Bitbox storage with the
PageWalker merkle engine:
- DB.Open: creates/opens Bitbox HT file, runs WAL recovery
- DB.Update: sorts ops, runs PageWalker, persists via Bitbox sync
- DB.LoadPage: reads pages from Bitbox for NodeReader compatibility
- BitboxPageSet: PageSet implementation backed by on-disk Bitbox storage

The DB handles only trie structure (merkle pages). Flat key-value
storage stays on geth's PebbleDB via existing infrastructure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 17:18:31 +08:00
weiihann
859312d1f5 nomt/bitbox: add Phase 4 WAL, sync controller, and crash recovery
Implement crash-safe persistence for the Bitbox hash table:
- wal.go: WAL format with START/CLEAR/UPDATE/END entries, builder/reader
- sync.go: 3-phase sync protocol (BeginSync → WriteWAL → CommitSync)
- recover.go: WAL replay for crash recovery

The WAL records page diffs (not full pages) for compact logging. The
3-phase protocol ensures: WAL fsynced before HT modification, HT fsynced
before WAL truncation, providing at-least-once delivery of page updates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 17:16:29 +08:00
weiihann
fef1ed4c4f nomt/bitbox: add Phase 3 Bitbox on-disk hash table storage
Implement the on-disk open-addressing hash table for storing trie pages:
- htfile.go: HT file layout with header, meta pages, and data pages
- metamap.go: in-memory meta byte map with dirty page tracking
- probe.go: triangular probing with xxhash64 page ID hashing
- db.go: Bitbox DB with StorePage, LoadPage, DeletePage, FlushMeta, Sync

The hash table uses 1-byte meta tags (top 7 bits of hash) for fast
filtering before reading full 4096-byte data pages. Triangular probing
with power-of-2 capacity guarantees all buckets are visited.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 17:14:20 +08:00
weiihann
88fd10529f nomt/merkle: add Phase 2 merkle engine (PageWalker, PageSet, ElidedChildren)
Implement the in-memory batch update engine for the NOMT binary merkle trie:
- elided.go: ElidedChildren 64-bit bitfield for tracking elided child pages
- pageset.go: PageSet interface + MemoryPageSet in-memory implementation
- pagewalker.go: PageWalker left-to-right walker with partial compaction
  - AdvanceAndReplace: replace terminal nodes with sub-tries
  - AdvanceAndPlaceNode: place pre-computed child page roots
  - Conclude: finalize walk and return new root + updated pages
  - compactUp/compactStep: hash upward with leaf/terminator compaction
- core/triepos.go: add SharedDepth method needed by PageWalker

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 17:10:58 +08:00
weiihann
7aebfb3c71 nomt/core: add Phase 1 core primitives for NOMT binary merkle trie
Implement foundational types and algorithms for the NOMT storage engine:
- node.go: Node/KeyPath/ValueHash types with MSB-based kind discrimination
- hasher.go: Keccak256 hashing with leaf/internal MSB labeling
- page.go: 4096-byte RawPage layout (126 nodes + elided children + pageID)
- pageid.go: PageID encode/decode with shift-then-add encoding
- triepos.go: TriePosition navigation (Down/Up/Sibling/PageID/NodeIndex)
- pagediff.go: 128-bit PageDiff bitfield for tracking changed nodes
- update.go: BuildTrie 3-pointer left-frontier algorithm, LeafOpsSpliced

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 17:05:00 +08:00
weiihann
f75573751a phase 1
phase 2

reset
2026-02-12 16:25:06 +08:00