* swarm: Integrate encryption into dpa and chunker
This is not a fully functional change.
Chunker api changed: it uses Putter and Getter interfaces to put and get
chunks from the store and to do the encryption/decryption if needed. These
putters and getters should also the hashing, so hashing related code
could be removed from chunker.
Some things are temporary, pyramid chunker doesn't work yet (only tree
chunker), and hashing and putting in dpa is not parallelized.
* swarm/storage: Use TreeChunker in DPA
PyramydChunker doesn't work yet with Putter/Getter
* swarm/storage: HasherStore needs a new hasher instance on every chunk
creation
This fixes the tests with pyramid chunker
* swarm/storage: Add back tree/pyramid comparison in chunker test
* swarm: Refactor chunker, new chunker instance is needed for each job
Chunker code is much simpler if all data of the job is stored on the
chunker object itself.
* swarm/storage: Remove commented code
* swarm/storage: Added comments
* cmd/swarm: Remove Branches from config
* swarm/storage: Remove commented code
* swarm/network: Remove chunker from syncer
* swarm/storage: Remove Chunker/Splitter/Joiner interfaces
They are not used anymore
* swarm/storage: Fix comment
* swarm/storage: Reenable chunker_test.TestDataAppend
* swarm/storage: Fix comments
* swarm/storage, cmd/swarm: Add FakeChunkStore
Instead of using a nil ChunkStore in hasherStore when we don't want the
hasherStore to actually store the chunk data, now a FakeChunkStore
instance should be used. Because of this the nil checks of the hasherStore.store
could be removed.
* swarm/storage: Use DefaultChunkSize az constant, not DefaultBranches
The branches what is changing, and it is dependent on chunk size and
hash size
* swarm/storage: Remove unnecessary return statement
* swarm/storage: Rename unfinishedChunk to unfinishedChunkData
* swarm/storage: Move NewTreeSplitterParams to chunker.go
It was in pyramid.go, although TreeSplitter is in chunker.go
* swarm/storage: Minor cleanup
incrementWorkerCount can be moved into runWorker
remove obsolete comment
* swarm/storage: Add TODO comment to chunker depth parameter
Depth can only be 0 because of a bug: https://github.com/ethersphere/go-ethereum/issues/344
* swarm/storage: Fix padding in hasherStore
There was a bug that data was not lengthened to padding
* swarm/storage: Add unit test for hasherStore
* swarm/storage: Decrease initial counter in span encryption
* swarm/storage: Renames in benchmark tests
* swarm/storage: Fixed pyramid benchmark test
* swarm/storage: Convert TestHasherStore to table-driven test
* swarm/storage, cmd/swarm: New MapChunkStore, simple ChunkStore implementation
It is not just the test which needs this kind of implementation
* swarm/storage: Cleanup on hashSize/chunkSize handling
- Fixes#16271. What was appeneded was a pointer to
an object that changes during the iteration.
- The topic is allocated as a 4-byte array, fill partial topics
with 0s. Partial topics are currently disabled, but would
crash as they rely on the presence of byte number 3.
- NewRegistry creates a goroutine that updates sync streams based on changes in kademlia. It calls updateSyncing method to create new and quit unintended sync streams. To early subscriptions are prevented by waiting until no new peers are added for a configured period of time SyncUpdateDelay.
- Implements two new methods on Kademlia:
- NeighbourhoodDepthC which returns a channel that returns kademlia neighbourhood depth on each change
- AddressBookSizeC which returns address book size when it changes
This methods are required for making a decision when to request sync stream subscriptions update in a dynamic network.
- To avoid the parsing of stream strings to distinguish the SYNC ones, Registry now uses Stream structure as the key for clients, servers and clientParams maps. This required that the Stream must be comparable and Key field is changed from []byte to string. The consequence is that key is changed to string in all function signatures that accepted it. Also, new functions FormatSyncBinKey and ParseSyncBinKey are introduced to explicitly handle the parsing and formating of bin numbers that are used as Key values in Stream.
- Implements RegistryOptions that holds optional values for stream Registry initialization. The number of arguments for NewRegistry is a bit high with adding another option SyncUpdateDelay, and optional and required ones are separated with RegistryOptions.
- Range now has constructor NewRange for better initialization and also has the String method for better logging.
- Registry now has the Quit method that sends QuitMsg to the peer to terminate the stream client on its side. This is required to terminate not needed sync streams when kademlia table changes by adding new peers.
- Changes the RequestSubscription to skip the request if the stream is already registered. This ensures that no request subscriptions are made if the server already exists.
- New SyncUpdateDelay cli flag is added to control the delay of syncing update. Default is set to 15s.
* crypto/bn256: full switchover to cloudflare's code
* crypto/bn256: only use cloudflare for optimized architectures
* crypto/bn256: upstream fallback for non-optimized code
* .travis, build: drop support for Go 1.8 (need type aliases)
* crypto/bn256/cloudflare: enable curve mul lattice optimization
