Commit graph

262 commits

Author SHA1 Message Date
Ömer Faruk Irmak
316138099f
refactor: remove unused storage traces (#819) 2024-06-14 15:02:00 +03:00
HAOYUatHZ
760f234a1a
trie: fix two issues in trie iterator (#24539) (#778)
* trie: fix two issues in trie iterator (#24539)

* trie: fix memory leak in trie iterator

In the trie iterator, live nodes are tracked in a stack while iterating.
Popped node states should be explictly set to nil in order to get
garbage-collected.

* trie: fix empty trie iterator

* fix lint

* chore: auto version bump [bot]

---------

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: HAOYUatHZ <HAOYUatHZ@users.noreply.github.com>
2024-05-28 22:07:38 +08:00
colin
e5b4f3ee9d
docs(zktrie): remove an outdated comment (#765)
fix(zktrie): remove an outdated comment

Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
2024-05-23 15:59:35 +02:00
Ömer Faruk Irmak
6038ac8753
fix(zktrie): copy trie properly for concurrent access (#747) 2024-05-13 15:57:14 +03:00
Ömer Faruk Irmak
79c356002c
feat(zktrie): upgrade to 0.8.2 (#715)
* feat(zktrie): upgrade to v0.8.2

* chore: auto version bump [bot]

* chore: auto version bump [bot]

---------

Co-authored-by: omerfirmak <omerfirmak@users.noreply.github.com>
Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
Co-authored-by: HAOYUatHZ <HAOYUatHZ@users.noreply.github.com>
2024-04-24 11:31:56 +08:00
HAOYUatHZ
3cf590b270
fix(zktrie): fix NewZktrieDatabase (#638)
* feat(zktrie): fix `NewZktrieDatabase`

* bump version
2024-03-27 14:05:30 +08:00
Ho
11b35cd014
feat: upgrade zktrie to v0.6.0 (#403)
* add domain-spec poseidon hash

* add warning message for deprecated method

* update goimports

* update zktrie to v0.6

* fixes for upgrade

* resume deletion proof

* update alpha genesis since new hash sch eme

* dump zktrie to 0.6

* induce genesis block for scroll sepolia network

* Update version.go

---------

Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
2023-08-02 10:54:41 +08:00
Ho
134e1b8f92
feat&perf: reversed bit of key when persisting zktrie (#410)
* bit reverse when persisting zktrie

* fix according to review

* Update version.go

---------

Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
2023-07-31 16:13:12 +08:00
Ho
9b943f3e0b
fix: add missing term when merging two deletion proofs (#353)
* fix: merge emptyTermPaths

* bump version

---------

Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
2023-06-01 11:00:37 +02:00
colin
74725fbe2c
test(zkTrie): add deletion test in account update unit test (#344)
feat(zkTrie): extend account update unit tests with account deletion
2023-05-25 16:30:50 +08:00
Ho
149419cf09
fix(trace): deletion proof missed path terminated by empty node (#330)
* fix deletion proof issue on empty node

* refine for better implement and fix unittest

* Update version.go

---------

Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
2023-05-22 15:24:49 +08:00
Ho
c913b3e2f1
fix(zktrie): fix deletion proofs and collect them in commiting phase (#263)
* new deletion proof

* complete tracer and test

* extend proof for parallel tracing

* integrating into blocktrace

* lint

* deduplication of deletion proofs

* fix an issue on marking deletion

* fixs since last review

* Update version.go

---------

Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-10-2-9-63.us-west-2.compute.internal>
2023-05-08 15:40:54 +08:00
HAOYUatHZ
32aacc3fd6
style: revert some formatting (#294)
Revert "style: remove blank line & white space formatting"

This reverts commit 349def8d5d78f83c19e899d5d80b1d9f6a5d18c6.
2023-04-20 17:49:35 +08:00
Ho
8665d3e011
feat(trace): add deletion proof into storage trace (#245)
* update zktrie, inducde proof with deletion

* Add deletion proof for stateDb

* fill deletionproofs

* lint

* proving more comment for parts prone to be confused

* simplify array appending

Co-authored-by: maskpp <maskpp266@gmail.com>

* bump zktrie version and fix issues

* add delproof into witgen

---------

Co-authored-by: maskpp <maskpp266@gmail.com>
2023-03-20 13:44:09 +08:00
maskpp
d1f9523b08
feat(build): Upgrade go version to 1.18 (#241)
* update go version

* update go version

* format code by goimports

* format code by goimports

* go mod tidy

* update go version

* update go version

* upgrade linter

* add exclude rule

* fix golangci exclude

---------

Co-authored-by: vincent <419436363@qq.com>
Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
2023-03-06 13:32:39 +08:00
Ho
29e450264c
fix: error propagation in db layer of zktrie (#238)
* fix error propagation in db layer of zktrie

* goimports fixing
2023-03-06 13:17:41 +08:00
Haichen Shen
e3e0078d1b
feat: upgrade zktrie version to v0.5.0 (#214)
* upgrade zktrie version to v0.5.0

* update go.sum
2023-02-13 13:16:43 +08:00
HAOYUatHZ
0854d4bbb1
core, trie, eth, cmd: rework preimage store (#205)
* core, trie, eth, cmd: rework preimage store

* fix ci

* fix ci

* Update zk_trie_database.go

---------

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: maskpp <maskpp266@gmail.com>
2023-02-10 15:26:57 +08:00
Ho
3a080e5cb4
test(zktrie): add TestZkTrieDelete (#201)
update zktrie for deletion fix

Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
2023-02-09 21:14:10 +08:00
Péter Garamvölgyi
81e7775aa8
feat: dual code hash (#188)
* add KeccakCodeHash and CodeSize to StateAccount

* update StateAccount marshalling logic

* change emptyCodeHash to poseidon(nil)

* purge StateAccount.hash

* fix/disable failing tests

* change keccak and poseidon hash order in StateAccount

* fix lint

* update l2trace account wrapper

* update eth_getProof response type

* fix eth_getProof response type

* goimports

* update the codehash computation

* update the codehash test cases

* go mod tidy

* fix tests

* use keccak instead of poseidon

* update trace codehash field name

* upgrade zktrie to 4.2

* trigger ci

* update state account marshalling according to spec

* improve generatorStats estimation

* add comment

* upgrade zktrie to 4.3

* go mod tidy

* misc fixes

* fix TestDump

* fix snap sync tests

* handle err in the codehash

* fix tests in snapshot/generate_test.go

* remove prevhash from state journal

* add state_account_marshalling_test.go

* goimports

* add more tests

---------

Co-authored-by: Ho Vei <noelwei@gmail.com>
Co-authored-by: Haichen Shen <shenhaichen@gmail.com>
2023-02-08 16:12:41 -08:00
Ho
81aafb09f3
[FIX]: zkproof writer handle reading non-existed address (#197)
zkproof writer: fix issue in reading unexist address
2023-01-10 12:48:53 +01:00
maskpp
22963a115d
feat(trace): Add header and upgrade TransactionTrace (#181)
* Add header and upgrade TransactionTrace

* Add header

* change blockResult struct

* rm chainID from TransactionData

* Rename blockResult to blockTrace in everywhere

* fix comments
2022-11-18 17:11:04 +08:00
shu-unifra
f21acfee4a
perf: use fastcache for zktrie node (#172)
* * benchmark add benchmark for zktrie

* * improve: use fastcache for zktrie node
2022-11-14 10:49:36 +08:00
Siyuan Han
ae2ab39570
chore: fix function name typos santiyCheckByte32Key -> sanityCheckByte32Key (#176)
Fix function name typos: santiyCheckByte32Key -> sanityCheckByte32Key
2022-11-11 13:30:59 +08:00
Ho
e6500918a0
refactor: adapt to standalone zktrie module (#165)
* adapt zktrie: zktrie part

* adapt unittests

* pass tests

* refactor for new zktrie module

* run go imports
2022-10-17 14:32:42 +08:00
Ho
4ee7885525
Fix witness generator for failed call/create op and tx (#146)
* update test examples

* update tests

* solidify account deleting

* update zkwriter for reverting

* solid handling for call, and update test examples

* fix status capture for CREATE issue, update test examples

* add fail case for deploy

* update test example (call non-exist address)

* move test and stuffs to internal path, make required exports

* fix issue in selfdestruct

* update destruct test

* lint

* rename traces
2022-10-10 17:13:24 +08:00
Péter Garamvölgyi
304af60f50
feat: change bytecode hash function from Keccak to Poseidon (#149)
* feat: change bytecode hash function from Keccak to Poseidon

* fix TestStateProcessorErrors

* rename files

* copy iden3's poseidon package to local

* update poseidon hash

* apply new hash in codehash

* extend hash method, use actual code size as cap

* prune keccak(nil) for empty code

* update empty code hash references to Poseidon

* add TestPoseidonCodeHash

* use keccak for empty code hash

* go mod tidy

* goimports

* use Poseidon code hash in state and sync tests

* run goimports

* run goimports

* remove unnecessary reallocation from CodeHash

* goimports

* move codehash API to separate module

* add missing package

* fix review comments

* improve comments

Co-authored-by: Ho Vei <noelwei@gmail.com>
2022-09-08 06:16:17 +08:00
Ho
f9952a396f
unify byte order in zktrie node's fields being stored (#133) 2022-07-28 21:32:42 +08:00
Ho
37dbb86aa6
fix: add the mutation of coinbase address under clique mode (#126)
fix clique coinbase issue
2022-07-21 16:00:50 +08:00
Ho
3682e05f3f
zktrie part3: include zktrie witness in block trace; add demo for generating witness data for mpt circuit (#123)
* demo for witness generator

* add simple binary for witness generating

* revert the witgen utility

* purge some code

* induce mptwitness argument

* handle case for failed tx, and special block in POA

* mptwitness: refactor for enabling output ops with different order

* induce witness gen with rwtable order

* self-documenting options for mptwitness

* fix trace flag according to review

* fix issue in handling failed tx

* refactor

* update according to reviews

Co-authored-by: HAOYUatHZ <haoyu@protonmail.com>
2022-07-16 07:00:07 +08:00
Ho
c516a9e477
zktrie part2: add zktrie; allow switch trie type by config; (#113)
* induce zktrie

* refactoring zktrie

* fix crash issue in logger

* renaming JSON field

* unify hash scheme

* goimport and mod lint

* backward compatible with go 1.17

* lints

* add option on genesis file

* corrections according to the reviews

* trivial fixes: ValueKey entry, key in prove nodes

* fixing for the proof fix ...

* avoiding panic before loading stateDb in genesis setup

* revert ExtraData.StateList json annotation for compatibility

* fix goimports lint

* fix goimports lint

* better encoding for leaf node

* fix proof's printing issue, add handling on coinbase

* update genesis, and rule out snapshot in zktrie mode

* update readme and lint

* fix an issue

Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
Co-authored-by: HAOYUatHZ <haoyu@protonmail.com>
2022-06-27 11:17:02 +08:00
Steven
b6cc89fabe
Add goimports lint (#54)
* Add `goimports` lint.

* Fix `goimports` issue.

* Fix group to `github.com/scroll-tech/go-thereum`.
2022-03-24 16:53:59 +08:00
maskpp
b4d60884d2
Fix import fmt and run tests for given architecture (#18)
* fix gofmt

* Run tests for given architecture

* Speed up CI execution.
2022-01-12 19:45:06 -08:00
Jules
de7ed6af40
Update mod file and all imports to use scroll-tech URL (#15)
* Update mod file and all imports to use scroll-tech URL

Fixes #14

* Update Discord link
2021-12-24 10:59:22 +01:00
Péter Szilágyi
743769f48e
trie: reject deletions when verifying range proofs 2021-11-23 19:28:17 +02:00
Paweł Bylica
5d4bcbc14f
trie: more tests for stacktrie (#23936) 2021-11-22 08:49:18 +01:00
Ha ĐANG
c36f8fefc3
trie: fix typos (#23768) 2021-10-18 20:53:56 +02:00
Martin Holst Swende
3a6fe69f23
eth/protocols/snap, trie: better error-handling (#23657) 2021-09-29 15:19:40 +02:00
Guillaume Ballet
443afc975c
core/state: move state account to core/types + abstracted "write account to trie" (#23567)
* core/state: abstracted "write account to trie" method

* fix appveyor build

* Apply suggestions from code review

Co-authored-by: Martin Holst Swende <martin@swende.se>

* review feedback

* core/state/accounts: move Account to core/types

* core/types: rename Account -> StateAccount

* core/state: restore EncodeRLP for stateObject

* core/types: add the missing file

* more review feedback

* more review feedback

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-09-28 10:48:07 +02:00
aaronbuchwald
de2c44ab5c
trie: add missing copyright header (#23530)
* Add missing copyright header

* Update stacktrie_test.go

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-09-07 07:25:54 +03:00
gary rong
a5a5237178
core, light, tests, trie: add state metrics (#23433) 2021-08-24 22:00:42 +03:00
chuwt
79bb9300c1
trie, les: add missing calls to Ticker.Stop (#23415) 2021-08-24 10:51:28 +02:00
Martin Holst Swende
c131e812ae
eth/fetcher, trie: unit test reliability fixes (#23020)
Some tests take quite some time during exit, which I think causes
some appveyor fails like this:

    https://ci.appveyor.com/project/ethereum/go-ethereum/builds/39511210/job/xhom84eg2e4uulq3

One of the things that seem to take time during exit is waiting
(up to 100ms) for the syncbloom to close. This PR changes it to use
a channel, instead of looping with a 100ms wait.

This also includes some unrelated changes improving the reliability of
eth/fetcher tests, which fail a lot because they are time-dependent.
2021-06-30 22:24:17 +02:00
Evolution404
732a6a3666
trie: small optimization of delete in fullNode case (#22979)
When deleting in fullNode, and the new child node nn is not nil, there is no need
to check the number of non-nil entries in the node. This is because the fullNode 
must've contained at least two children before deletion, so there must be another
child node other than nn.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-06-20 15:59:00 +02:00
Evolution404
08379b5533
trie: remove the duplicate batch-write for 'preimage' (#23001) 2021-06-07 09:11:07 +02:00
Péter Szilágyi
fae165a5de
core, eth, ethdb, trie: simplify range proofs 2021-04-29 10:59:08 +03:00
Martin Holst Swende
a81cf0d2b3
trie: remove redundant returns + use stacktrie where applicable (#22760)
* trie: add benchmark for proofless range

* trie: remove unused returns + use stacktrie
2021-04-28 22:47:48 +03:00
Péter Szilágyi
caea6c4661
eth/protocols/snap: generate storage trie from full dirty snap data (#22668)
* eth/protocols/snap: generate storage trie from full dirty snap data

* eth/protocols/snap: get rid of some more dead code

* eth/protocols/snap: less frequent logs, also log during trie generation

* eth/protocols/snap: implement dirty account range stack-hashing

* eth/protocols/snap: don't loop on account trie generation

* eth/protocols/snap: fix account format in trie

* core, eth, ethdb: glue snap packets together, but not chunks

* eth/protocols/snap: print completion log for snap phase

* eth/protocols/snap: extended tests

* eth/protocols/snap: make testcase pass

* eth/protocols/snap: fix account stacktrie commit without defer

* ethdb: fix key counts on reset

* eth/protocols: fix typos

* eth/protocols/snap: make better use of delivered data (#44)

* eth/protocols/snap: make better use of delivered data

* squashme

* eth/protocols/snap: reduce chunking

* squashme

* eth/protocols/snap: reduce chunking further

* eth/protocols/snap: break out hash range calculations

* eth/protocols/snap: use sort.Search instead of looping

* eth/protocols/snap: prevent crash on storage response with no keys

* eth/protocols/snap: nitpicks all around

* eth/protocols/snap: clear heal need on 1-chunk storage completion

* eth/protocols/snap: fix range chunker, add tests

Co-authored-by: Péter Szilágyi <peterke@gmail.com>

* trie: fix test API error

* eth/protocols/snap: fix some further liter issues

* eth/protocols/snap: fix accidental batch reuse

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-04-27 17:19:59 +03:00
Martin Holst Swende
49281ab84f
core/state/snapshot, true: reuse dirty data instead of hitting disk when generating (#22667)
* core/state/snapshot: reuse memory data instead of hitting disk when generating

* trie: minor nitpicks wrt the resolver optimization

* core/state/snapshot, trie: use key/value store for resolver

* trie: fix linter

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-04-23 14:39:18 +03:00
Martin Holst Swende
4b783c0064
trie: improve the node iterator seek operation (#22470)
This change improves the efficiency of the nodeIterator seek
operation. Previously, seek essentially ran the iterator forward
until it found the matching node. With this change, it skips
over fullnode children and avoids resolving them from the database.
2021-04-21 12:25:26 +02:00