go-ethereum/p2p/discover
Csaba Kiraly 6c0d848d9c
p2p/discover: add CrawlIterator for breadth-first FINDNODE walks
Add an enode.Iterator that drives discovery by issuing a single
FINDNODE per discovered peer, rotating the target through Drange
sub-regions of the keyspace. Compared to RandomNodes (which wraps an
alpha=3 Kademlia lookup that converges on a single target), this
shape is geared for breadth: each peer is asked about a different
slice of the keyspace, so aggregate coverage grows quickly without
per-peer overlap.

The two protocols expose different FINDNODE primitives, so the
iterator threads a per-protocol queryFn:

 * discv5 takes a list of distances natively, so we just pass
   [256-d] for d in 0..Drange-1.
 * discv4 takes a target NodeID and replies with the K closest. To
   get an equivalent rotation, we pick a random pubkey whose
   Keccak256 starts with the desired prefix nibble. With Drange=16
   that's ~16 random draws per call -- negligible compared to the
   network round trip.

Concurrency is bounded by Workers (default 16). There is intentionally
no rate limit: pacing is RTT-driven, ~Workers/RTT on the wire.

Termination is implicit: when the work queue is empty AND no FINDNODE
is in flight, the iterator closes its output and Next returns false.
Close() short-circuits this for callers that want to bail early.

Adapts the algorithm from github.com/cskiraly/fast-ethereum-crawler
(dcrawl.nim) -- the prefix-rotation idea -- but drops its 1000 req/s
rate limit in favour of the bounded worker pool.
2026-05-07 14:41:58 +02:00
..
v4wire p2p/discover: refactor node and endpoint representation (#29844) 2024-05-29 15:02:26 +02:00
v5wire p2p/discover/v5wire: use Whoareyou.ChallengeData instead of storing encoded packet (#31547) 2026-02-22 21:58:47 +01:00
common.go p2p/discover: fix timeout loop early exit when removing expired matchers (#34743) 2026-04-28 10:57:58 +02:00
crawliter.go p2p/discover: add CrawlIterator for breadth-first FINDNODE walks 2026-05-07 14:41:58 +02:00
crawliter_test.go p2p/discover: add CrawlIterator for breadth-first FINDNODE walks 2026-05-07 14:41:58 +02:00
lookup.go p2p/discover: remove hot-spin in table refresh trigger (#32912) 2025-10-15 11:51:33 +02:00
metrics.go metrics, cmd/geth: change init-process of metrics (#30814) 2024-12-10 13:27:29 +01:00
node.go p2p/discover: remove type encPubkey (#30172) 2024-07-18 11:09:02 +02:00
ntp.go all: remove dependency on golang.org/exp (#29314) 2024-03-25 07:50:18 +01:00
table.go p2p/discover: resolve DNS hostnames for bootstrap nodes (#34101) 2026-03-28 11:37:39 +01:00
table_reval.go p2p/discover: remove unused parameter in revalidationList.get (#31155) 2025-02-11 13:45:44 +01:00
table_reval_test.go p2p/discover: refactor node and endpoint representation (#29844) 2024-05-29 15:02:26 +02:00
table_test.go p2p/discover: resolve DNS hostnames for bootstrap nodes (#34101) 2026-03-28 11:37:39 +01:00
table_util_test.go p2p/discover: remove type encPubkey (#30172) 2024-07-18 11:09:02 +02:00
v4_lookup_test.go Revert "p2p/discover: add test for lookup returning immediately" 2025-09-12 11:29:43 +02:00
v4_udp.go p2p/discover: restore nextTimeout update in UDPv4 resetTimeout loop (#34878) 2026-05-05 15:28:28 +02:00
v4_udp_test.go p2p/discover: wait for bootstrap to be done (#32881) 2025-10-13 19:58:50 +02:00
v5_talk.go p2p/discover: pass node instead of node ID to TALKREQ handler (#31075) 2025-04-02 14:56:21 +02:00
v5_udp.go p2p/discover: remove delay from discv5 RandomNodes (#32517) 2025-09-10 19:51:04 +02:00
v5_udp_test.go p2p/discover: fix flaky TestUDPv5_findnodeHandling (#34109) 2026-04-14 09:43:44 +02:00