go-ethereum/swarm/storage/localstore.go
zelig 0460fbad11 SWORM - swarm poc 0.2 stability RC3
rebase on current develop branch 2016-07-09

swarm/services/ens:
* Rework ENS functionality to use the current spec.
* Refactor ens.go to set up sessions as needed
* Refactoring to support independent registrar and personal resolver contracts
* fixed tests

swarm/examples/album: improve upload UX
* No more unhandled errors and loose ends in the UI.
* Forward/Back buttons work as undo/redo
* all long operations display an informative modal dialog
* minor problems with cursor movements fixed
* reorder labels
* throbber as default image
* change browser history after delete or upload photo

swarm/storage:
* optimise splitter
* simplify reader
* benchmarks, tests improved
* simplify IO, remove chunkreader.go and io.SectionReader clone
* simplify error/timeout handling
* port pyramid splitter by karalabe, adapt to Splitter/DPA interface, rework params etc
* make NetStore.Put -> cloud.Store synchronous
* chunker join fix process leak and keep parallelisation limited to depth 1
* reset the base hash to SHA3. !!hard fork = no backward compatibility :)
* SectionReader -> LazySectionReader: Size method signature change
* introduce abort channel to joiner to fix process leak
* add abort channel   context to all manifest retrieval methods
* global waitgroup fixes intermittent upload test failures due to unfinished storage of chunks
* streamline joiner logic, contexts now unique to each readAt call
* LazyChunkReader seeker complains about missing size only if whence=2
* chunker join fix process leak and keep parallelisation limited to depth 1

swarm/network/kademlia:
* simplify code
* now really fix prox limit adjustment + tests and comments
* simplify and make readable findclosest algo code + comments
* reset initial time interval settings for kaddb findbest
* fix bucket replace scheme to optimise stability and availability
* absolute idle peers are disconnected after maxIdleInterval
* fix index out of range when deleting last idle peer from kaddb
* unwanted peers (due to full kad bucket) are now properly dropped with ErrUnwanted
* improve logging and reorg in kademlia
* simplify and fix code that deletes expired/unconnectable nodes
* kaddb.findBest does not get stuck on empty row but finds an actual missing node
* must replace node if bucket is full otherwise nodes will get stuck on empty rows
* kademlia: timer fix
* remove logging the state when syncing (resulting in deadlock)
* improved logs, fix potential deadlock in String()
* native go time marshalling
* tests fixed

swarm/network:
* log node address consistently in syncdb
* syncer logs queue cardinalities properly in syncUnSyncedKeys loop
* fix hive stop issue leading to send on closed chan + minor logging fixes
* better logging
* hive interface change
* fix process leak by adding select to state.synced <- false

swarm/cmd: swarm control cli improvements
* no more alias, swarm  is executable so it can be called via ssh
* environment vars now set to default, no need to preconfigure
* local and remote hive monitoring
* gethup.sh script now merged into swarm script and nice modularised
* simplify bash code and fix e2e tests in swarm/test
* stop method falls back to kill -9 after 10s
* enode method now supplies ip addr via ipecho request
* execute, options, rawoptions, setup, create-account addpeers and hive subcommands
* update-src -> update
* remote-update-scripts, remote-update-bin and remote-run
* local and remote monitoring of kademlia
* extensive documentation in  swarm/cmd/README.md
* add cleanlog, cleanbzz, update-src, remote-update-scripts, remote-update-bin, remote-run
* for remote binary update, take scripts from swarm/cmd/swarm
* raise default maxpeers to 40
* add latency loggingo to upload and download
* add checkdownload and checkaccess subcommands
* add mem/cpu/disk-info commands
* include randomfile cmd from test
* add enode, connect subcommands, improve startup (only one round needed)
* TODO: simplify tests using new checks

swarm/api/http: server handler: check protocol substring length to fix slice out of bounds crash

swarm/api:
* fix filesystem API - upload/Modify tests
* simplify downloader code and fix process leak
* change KAD defaults bucketsize 4, minproxbinsize 2
* add back final slash to paths in manifest matching
* fix filesystem api tests
2016-07-27 12:18:26 +02:00

58 lines
1.5 KiB
Go

package storage
import (
"encoding/binary"
)
// LocalStore is a combination of inmemory db over a disk persisted db
// implements a Get/Put with fallback (caching) logic using any 2 ChunkStores
type LocalStore struct {
memStore ChunkStore
DbStore ChunkStore
}
// This constructor uses MemStore and DbStore as components
func NewLocalStore(hash Hasher, params *StoreParams) (*LocalStore, error) {
dbStore, err := NewDbStore(params.ChunkDbPath, hash, params.DbCapacity, params.Radius)
if err != nil {
return nil, err
}
return &LocalStore{
memStore: NewMemStore(dbStore, params.CacheCapacity),
DbStore: dbStore,
}, nil
}
// LocalStore is itself a chunk store
// unsafe, in that the data is not integrity checked
func (self *LocalStore) Put(chunk *Chunk) {
chunk.dbStored = make(chan bool)
self.memStore.Put(chunk)
if chunk.wg != nil {
chunk.wg.Add(1)
}
go func() {
self.DbStore.Put(chunk)
if chunk.wg != nil {
chunk.wg.Done()
}
}()
}
// Get(chunk *Chunk) looks up a chunk in the local stores
// This method is blocking until the chunk is retrieved
// so additional timeout may be needed to wrap this call if
// ChunkStores are remote and can have long latency
func (self *LocalStore) Get(key Key) (chunk *Chunk, err error) {
chunk, err = self.memStore.Get(key)
if err == nil {
return
}
chunk, err = self.DbStore.Get(key)
if err != nil {
return
}
chunk.Size = int64(binary.LittleEndian.Uint64(chunk.SData[0:8]))
self.memStore.Put(chunk)
return
}