forked from forks/go-ethereum
14d576c002
This PR adds `rawdb.SafeDeleteRange` and uses it for range deletion in `core/filtermaps`. This includes deleting the old bloombits database, resetting the log index database and removing index data for unindexed tail epochs (which previously weren't properly implemented for the fallback case). `SafeDeleteRange` either calls `ethdb.DeleteRange` if the node uses the new path based state scheme or uses an iterator based fallback method that safely skips trie nodes in the range if the old hash based state scheme is used. Note that `ethdb.DeleteRange` also has its own iterator based fallback implementation in `ethdb/leveldb`. If a path based state scheme is used and the backing db is pebble (as it is on the majority of new nodes) then `rawdb.SafeDeleteRange` uses the fast native range delete. Also note that `rawdb.SafeDeleteRange` has different semantics from `ethdb.DeleteRange`, it does not automatically return if the operation takes a long time. Instead it receives a `stopCallback` that can interrupt the process if necessary. This is because in the safe mode potentially a lot of entries are iterated without being deleted (this is definitely the case when deleting the old bloombits database which has a single byte prefix) and therefore restarting the process every time a fixed number of entries have been iterated would result in a quadratic run time in the number of skipped entries. When running in safe mode, unindexing an epoch takes about a second, removing bloombits takes around 10s while resetting a full log index might take a few minutes. If a range delete operation takes a significant amount of time then log messages are printed. Also, any range delete operation can be interrupted by shutdown (tail uinindexing can also be interrupted by head indexing, similarly to how tail indexing works). If the last unindexed epoch might have "dirty" index data left then the indexed map range points to the first valid epoch and `cleanedEpochsBefore` points to the previous, potentially dirty one. At startup it is always assumed that the epoch before the first fully indexed one might be dirty. New tail maps are never rendered and also no further maps are unindexed before the previous unindexing is properly cleaned up. --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com> Co-authored-by: Felix Lange <fjl@twurst.com> |
||
|---|---|---|
| .. | ||
| filtermaps | ||
| forkid | ||
| rawdb | ||
| state | ||
| stateless | ||
| tracing | ||
| txpool | ||
| types | ||
| vm | ||
| .gitignore | ||
| bench_test.go | ||
| block_validator.go | ||
| block_validator_test.go | ||
| blockchain.go | ||
| blockchain_insert.go | ||
| blockchain_reader.go | ||
| blockchain_repair_test.go | ||
| blockchain_sethead_test.go | ||
| blockchain_snapshot_test.go | ||
| blockchain_test.go | ||
| chain_makers.go | ||
| chain_makers_test.go | ||
| dao_test.go | ||
| error.go | ||
| events.go | ||
| evm.go | ||
| gaspool.go | ||
| gen_genesis.go | ||
| genesis.go | ||
| genesis_alloc.go | ||
| genesis_test.go | ||
| headerchain.go | ||
| headerchain_test.go | ||
| mkalloc.go | ||
| rlp_test.go | ||
| sender_cacher.go | ||
| state_prefetcher.go | ||
| state_processor.go | ||
| state_processor_test.go | ||
| state_transition.go | ||
| stateless.go | ||
| txindexer.go | ||
| txindexer_test.go | ||
| types.go | ||
| verkle_witness_test.go | ||