Commit graph

1 commit

Author SHA1 Message Date
Csaba Kiraly
6c0d848d9c
p2p/discover: add CrawlIterator for breadth-first FINDNODE walks
Add an enode.Iterator that drives discovery by issuing a single
FINDNODE per discovered peer, rotating the target through Drange
sub-regions of the keyspace. Compared to RandomNodes (which wraps an
alpha=3 Kademlia lookup that converges on a single target), this
shape is geared for breadth: each peer is asked about a different
slice of the keyspace, so aggregate coverage grows quickly without
per-peer overlap.

The two protocols expose different FINDNODE primitives, so the
iterator threads a per-protocol queryFn:

 * discv5 takes a list of distances natively, so we just pass
   [256-d] for d in 0..Drange-1.
 * discv4 takes a target NodeID and replies with the K closest. To
   get an equivalent rotation, we pick a random pubkey whose
   Keccak256 starts with the desired prefix nibble. With Drange=16
   that's ~16 random draws per call -- negligible compared to the
   network round trip.

Concurrency is bounded by Workers (default 16). There is intentionally
no rate limit: pacing is RTT-driven, ~Workers/RTT on the wire.

Termination is implicit: when the work queue is empty AND no FINDNODE
is in flight, the iterator closes its output and Next returns false.
Close() short-circuits this for callers that want to bail early.

Adapts the algorithm from github.com/cskiraly/fast-ethereum-crawler
(dcrawl.nim) -- the prefix-rotation idea -- but drops its 1000 req/s
rate limit in favour of the bounded worker pool.
2026-05-07 14:41:58 +02:00