mirror of
https://github.com/ethereum/go-ethereum.git
synced 2026-07-25 14:16:44 +00:00
swarm/cmd: swarm cntrol cli improvements
* no more alias, swarm is executable so it can be called via ssh * environment vars now set to default, no need to preconfigure * stop method falls back to kill -9 after 10s * enode method now supplies ip addr via ipecho request * execute, options, rawoptions, setup, create-account addpeers and hive methods and subcommands * local and remote hive monitoring * gethup.sh script now merged into swarm script and nice modularised * simplify bash code and fix e2e tests in swarm/test * update-src -> update * remote-update-scripts, remote-update-bin and remote-run * local and remote monitoring of kademlia * extensive documentation in swarm/cmd/README.md swarm/network: fix hive stop issue leading to send on closed chan + minor logging fixes swarm/network/kademlia: * simplify code * now really fix prox limit adjustment + tests and comments * simplify and make readable findclosest algo code + comments * reset initial time interval settings for kaddb findbest * fix bucket replace scheme to optimise stability and availability * absolute idle peers are disconnected after maxIdleInterval
This commit is contained in:
zelig
parent
0de16bd17e
commit
38028fc8e9
20 changed files with 1100 additions and 831 deletions
|
|
@ -26,7 +26,8 @@ var (
|
||||||
"ProxBinSize": 4,
|
"ProxBinSize": 4,
|
||||||
"BucketSize": 3,
|
"BucketSize": 3,
|
||||||
"PurgeInterval": 151200000000000,
|
"PurgeInterval": 151200000000000,
|
||||||
"InitialRetryInterval": 4200000000,
|
"InitialRetryInterval": 42000000,
|
||||||
|
"MaxIdleInterval": 4200000000,
|
||||||
"ConnRetryExp": 2,
|
"ConnRetryExp": 2,
|
||||||
"Swap": {
|
"Swap": {
|
||||||
"BuyAt": 20000000000,
|
"BuyAt": 20000000000,
|
||||||
|
|
|
||||||
218
swarm/cmd/README.md
Normal file
218
swarm/cmd/README.md
Normal file
|
|
@ -0,0 +1,218 @@
|
||||||
|
|
||||||
|
# install and setup swarm
|
||||||
|
|
||||||
|
swarm is developed on a branch of the ethereum/go-ethereum repo
|
||||||
|
at this stage of the project there is no packages or binary distro, you need to a dev environment and compile from source.
|
||||||
|
[This document spells out a complete server setup on ubuntu linux](https://gist.github.com/zelig/74eb365752ceaacf15e860fb80eacb3e) including git/ssh/screen config, golang and compilation, node/npm and network monitoring (might contain a few bits that are tangential to swarm).
|
||||||
|
|
||||||
|
Assuming you got your setup working, you will use the `swarm` command line tool to control your swarm of instances.
|
||||||
|
This command line tool is at the moment geared towards developers and testing.
|
||||||
|
It is likely that it will be replaced by two different tools, one for devel/testing and one for end users
|
||||||
|
|
||||||
|
|
||||||
|
The command can be used to update the code
|
||||||
|
|
||||||
|
```shell
|
||||||
|
swarm update upstream/swarm
|
||||||
|
```
|
||||||
|
|
||||||
|
Then compile with
|
||||||
|
|
||||||
|
```shell
|
||||||
|
godep go build -v ./cmd/geth
|
||||||
|
```
|
||||||
|
|
||||||
|
Make sure you have `GOPATH` variable set and also that the `swarm` executable is in your PATH.
|
||||||
|
These environment variables are relevant and set to the following defaults.
|
||||||
|
Make sure you are happy with them, otherwise change them, in which case best to put these lines in your `~/.profile`.
|
||||||
|
|
||||||
|
```
|
||||||
|
export GETH_DIR=$GOPATH/src/github.com/ethereum/go-ethereum
|
||||||
|
export GETH=$GETH_DIR/geth
|
||||||
|
export SWARM_DIR=~/bzz
|
||||||
|
export SWARM_NETWORK_ID=322
|
||||||
|
```
|
||||||
|
|
||||||
|
* `GETH_DIR` points to your git working copy (given `GOPATH` its standardly under `$GOPATH/src/github.com/ethereum/go-ethereum`)
|
||||||
|
* `GETH` points to the `geth` executable compiled from the swarm branch. If you have systemwide install or use multiple geths you may need to change this, otherwise it is assumed you compile to the working copy of the repo.
|
||||||
|
* `SWARM_DIR` is the root directory for all swarm related stuff: logs, configs, as well as geth datadirs, make sure this dir is on a device with sufficient disk space
|
||||||
|
* `SWARM_NETWORK_ID`: this is by default the network id of the swarm testnet. If you run your own swarm, you need to change it, choose a number that is not likely chosen by others to avoid others joining you.
|
||||||
|
|
||||||
|
# Deploying and remote control
|
||||||
|
|
||||||
|
the swarm command supports remote update and remote control of your instances.
|
||||||
|
In our setting we assume you want to run a cluster of potentially remote swarm nodes each running a local cluster of instances
|
||||||
|
The only assumption is that you have (passwordless) ssh access set up to your swarm servers.
|
||||||
|
Assume `nodes.lst` is a list of nodes in the format of `username@ip` one per line. blank lines and lines commented out with `#` are ignored.
|
||||||
|
|
||||||
|
|
||||||
|
This copies the scripts found in `swarm/cmd/swarm` on all remote nodes listed in `nodes.lst`
|
||||||
|
|
||||||
|
```
|
||||||
|
swarm remote-update-scripts nodes.lst
|
||||||
|
```
|
||||||
|
|
||||||
|
If you just want to deploy a locally compiled binaries to all your remote nodes, this will fail if the remote instances are running, so make sure you stop them beforehand
|
||||||
|
|
||||||
|
```shell
|
||||||
|
swarm remote-run nodes.lst swarm stop all
|
||||||
|
swarm remote-update-bin nodes.lst
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Once you deployed the executables to the nodes, you can control them all with one command. For instance the following line initialises a cluster of two test swarm instances on each remote node.
|
||||||
|
Watch out, this will wipe your storage and all swarm related data
|
||||||
|
|
||||||
|
```shell
|
||||||
|
swarm remote-run nodes.lst swarm init 2
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
To (re) start a particular instance on a specific remote node with alternative options (for instance mining and different logging verbosity), you can just:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
swarm remote-run cicada@3.3.0.1 'swarm restart 01 --mine --verbosity=0 --vmodule=swarm/*=5'
|
||||||
|
```
|
||||||
|
|
||||||
|
# Logging
|
||||||
|
|
||||||
|
To check logs
|
||||||
|
|
||||||
|
```shell
|
||||||
|
swarm log 00 # taillog flow
|
||||||
|
swarm remote-run cicada@3.3.0.1 swarm log 00
|
||||||
|
```
|
||||||
|
|
||||||
|
You can view the log with a pager for an instance with
|
||||||
|
|
||||||
|
```
|
||||||
|
swarm viewlog 00
|
||||||
|
```
|
||||||
|
|
||||||
|
Logs are preserved and viewable with the above commands even when nodes are offline
|
||||||
|
Each new run logs to a different file
|
||||||
|
|
||||||
|
To purge logs
|
||||||
|
|
||||||
|
```
|
||||||
|
swarm cleanlog 01
|
||||||
|
```
|
||||||
|
|
||||||
|
To remove all logs on all nodes:
|
||||||
|
|
||||||
|
```
|
||||||
|
swarm remote-run nodes.lst swarm cleanlog all
|
||||||
|
```
|
||||||
|
|
||||||
|
# upload and dowload
|
||||||
|
|
||||||
|
upload and download via a running local instance
|
||||||
|
|
||||||
|
```shell
|
||||||
|
swarm up 00 /path/to/file/or/directory
|
||||||
|
swarm down 01 hash /path/to/destination
|
||||||
|
```
|
||||||
|
|
||||||
|
upload via remote swarm proxy or public gateway
|
||||||
|
|
||||||
|
```shell
|
||||||
|
swarm remote-up gateway-url /path/to/file/or/directory
|
||||||
|
wget -O- gateway-url/bzz:/swarm-url
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
# Further examples
|
||||||
|
|
||||||
|
```shell
|
||||||
|
|
||||||
|
# display CLI options given to geth used to launch swarm instance 02
|
||||||
|
swarm options 02
|
||||||
|
|
||||||
|
# restart swarm instance 00 with alternatiev options
|
||||||
|
swarm restart 00 --mine --bzznosync --verbosity=0 --vmodule=swarm/*=6
|
||||||
|
|
||||||
|
# attach console to a running swarm instance
|
||||||
|
swarm attach 00
|
||||||
|
|
||||||
|
# execute a command; e.g., start mining on a running instance
|
||||||
|
swarm execute 00 'miner.stop(1)'
|
||||||
|
|
||||||
|
# display static info about a instance (even if its offline)
|
||||||
|
swarm info 00
|
||||||
|
|
||||||
|
# displays the enode url of a running instance
|
||||||
|
swarm enode 01
|
||||||
|
|
||||||
|
# add peers to a running swarm instance
|
||||||
|
swarm addpeers 00 "enode://1033c1cada...@3.3.0.1:30301"
|
||||||
|
|
||||||
|
# to compile a list of enodes from all instances on all remote nodes:
|
||||||
|
swarm remote-run nodes.lst 'swarm enode all' > enodes.lst
|
||||||
|
|
||||||
|
# to add all peers to all instances on each node
|
||||||
|
for node in `cat nodes.lst|grep -v '^#'`; do scp enodes.lst $node:; done
|
||||||
|
swarm remote-run nodes.lst 'swarm addpeers enodes.lst'
|
||||||
|
|
||||||
|
# stop all running instances on the node
|
||||||
|
swarm stop all
|
||||||
|
|
||||||
|
# stop all running instances on all remote nodes
|
||||||
|
swarm remote-run nodes.lst swarm stop all
|
||||||
|
|
||||||
|
# display peer connection table of running instance 00
|
||||||
|
swarm hive 00
|
||||||
|
|
||||||
|
# display peer connection table for a running instance and continually refresh every 4 seconds
|
||||||
|
swarm monitor 00 4
|
||||||
|
|
||||||
|
# display peer connection table for all running instance on a remote node and continually refresh every 10 seconds
|
||||||
|
swarm monitor cicada@3.3.0.1 all 10
|
||||||
|
|
||||||
|
|
||||||
|
# configure eth-net-intelligence-api network monitoring client API for a node (the name argument appears as a prefix for all instances in your cluster)
|
||||||
|
swarm netstatconf cicada-sworm
|
||||||
|
|
||||||
|
# restart the net monitor client API
|
||||||
|
swarm netstatun
|
||||||
|
|
||||||
|
# configure eth-net-intelligence-api network monitoring client API and (re)start the monitor tool on all remote nodes
|
||||||
|
swarm remote-run nodes.lst 'swarm netstatconf cicada-sworm; swarm netstatrun'
|
||||||
|
|
||||||
|
|
||||||
|
swarm remote-run-all nodes.lst 'swarm netstatconf cicada-sworm; swarm netstatrun'
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
see also:
|
||||||
|
|
||||||
|
* https://github.com/ethereum/go-ethereum/tree/swarm/swarm/test
|
||||||
|
* https://github.com/ethereum/go-ethereum/tree/swarm/swarm/cmd
|
||||||
|
|
||||||
|
# ethereum netstats client setup
|
||||||
|
|
||||||
|
## install
|
||||||
|
|
||||||
|
nodejs and npm are prerequisites
|
||||||
|
|
||||||
|
```shell
|
||||||
|
# MAC
|
||||||
|
brew install node npm
|
||||||
|
# ubuntu
|
||||||
|
sudo apt-get install npm nodejs
|
||||||
|
```
|
||||||
|
|
||||||
|
clone the git repo and install:
|
||||||
|
|
||||||
|
```
|
||||||
|
git clone git@github.com:cubedro/eth-net-intelligence-api.git
|
||||||
|
cd eth-net-intelligence-api
|
||||||
|
npm install
|
||||||
|
npm install -g pm2
|
||||||
|
```
|
||||||
|
|
||||||
|
## configure and run netstats client for each node
|
||||||
|
|
||||||
|
```shell
|
||||||
|
swarm remote-run-all nodes.lst 'swarm netstatconf cicada-sworm; swarm netstatrun'
|
||||||
|
```
|
||||||
|
|
@ -1,9 +1,6 @@
|
||||||
export GOPATH=~/go
|
export PATH=$HOME/bin:$PATH
|
||||||
export PATH=~/bin:$GOPATH/bin:$PATH
|
export GETH_DIR=$HOME/bin
|
||||||
|
export SWARM_DIR=
|
||||||
|
|
||||||
if [ -f ~/.bash_aliases ]; then
|
export NVM_DIR=$HOME/.nvm
|
||||||
. ~/.bash_aliases
|
|
||||||
fi
|
|
||||||
|
|
||||||
export NVM_DIR="/home/ubuntu/.nvm"
|
|
||||||
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm
|
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm
|
||||||
|
|
@ -1,138 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
# Usage:
|
|
||||||
# bash /path/to/eth-utils/gethup.sh <datadir> <instance_name> <ip_addr>
|
|
||||||
|
|
||||||
root=$1 # base directory to use for datadir and logs
|
|
||||||
shift
|
|
||||||
id=$1 # double digit instance id like 00 01 02
|
|
||||||
shift
|
|
||||||
ip_addr=$1 # ip address to substitute
|
|
||||||
shift
|
|
||||||
|
|
||||||
# logs are output to a date-tagged file for each run , while a link is
|
|
||||||
# created to the latest, so that monitoring be easier with the same filename
|
|
||||||
# TODO: use this if GETH not set
|
|
||||||
# GETH=geth
|
|
||||||
# echo "ls -l $GETH"
|
|
||||||
# ls -l $GETH
|
|
||||||
|
|
||||||
# geth CLI params e.g., (dd=04, run=09)
|
|
||||||
datetag=`date "+%Y-%m-%d-%H:%M:%S"`
|
|
||||||
datadir=$root/data/$id # /tmp/eth/04
|
|
||||||
log=$root/log/$id.$datetag.log # /tmp/eth/04.09.log
|
|
||||||
linklog=$root/log/$id.log # /tmp/eth/04.09.log
|
|
||||||
password=$id # 04
|
|
||||||
port=303$id # 34504
|
|
||||||
bzzport=322$id # 3 2204
|
|
||||||
rpcport=302$id # 3204
|
|
||||||
|
|
||||||
mkdir -p $root/data
|
|
||||||
mkdir -p $root/enodes
|
|
||||||
mkdir -p $root/pids
|
|
||||||
mkdir -p $root/log
|
|
||||||
# if we do not have an account, create one
|
|
||||||
# will not prompt for password, we use the double digit instance id as passwd
|
|
||||||
# NEVER EVER USE THESE ACCOUNTS FOR INTERACTING WITH A LIVE CHAIN
|
|
||||||
keystoredir="$datadir/keystore/"
|
|
||||||
# echo "KeyStore dir: $keystoredir"
|
|
||||||
if [ ! -d "$keystoredir" ]; then
|
|
||||||
# echo "create an account with password $id [DO NOT EVER USE THIS ON LIVE]"
|
|
||||||
# mkdir -p $datadir/keystore
|
|
||||||
$GETH --datadir $datadir --password <(echo -n $id) account new >/dev/null 2>&1
|
|
||||||
# create account with password 00, 01, ...
|
|
||||||
# note that the account key will be stored also separately outside
|
|
||||||
# datadir
|
|
||||||
# this way you can safely clear the data directory and still keep your key
|
|
||||||
# under <rootdir>/keystore/dd
|
|
||||||
# LS=$(ls $datadir/keystore)
|
|
||||||
# echo $LS
|
|
||||||
while [ ! -d "$keystoredir" ]; do
|
|
||||||
echo "."
|
|
||||||
((i++))
|
|
||||||
if ((i>10)); then break; fi
|
|
||||||
sleep 1
|
|
||||||
done
|
|
||||||
# echo "copying keys $datadir/keystore $root/keystore/$id"
|
|
||||||
mkdir -p $root/keystore/$id
|
|
||||||
cp -R "$datadir/keystore/" $root/keystore/$id
|
|
||||||
fi
|
|
||||||
|
|
||||||
# query node's enode url
|
|
||||||
if [ $ip_addr="" ]; then
|
|
||||||
pattern='\d+\.\d+\.\d+\.\d+'
|
|
||||||
ip_addr="[::]"
|
|
||||||
else
|
|
||||||
pattern='\[\:\:\]'
|
|
||||||
fi
|
|
||||||
|
|
||||||
geth="$GETH --datadir $datadir --port $port"
|
|
||||||
|
|
||||||
# echo -n "enode for instance $id... "
|
|
||||||
if [ ! "$GETH" = "" ] && [ ! -f $root/enodes/$id.enode ]; then
|
|
||||||
cmd="$geth js <(echo 'console.log(admin.nodeInfo.enode); exit();') "
|
|
||||||
# echo $cmd '2>/dev/null |grep enode | perl -pe "s/'$pattern'/'$ip_addr'/g" | perl -pe "s/^/\"/; s/\s*$/\"/;" > '$root/enodes/$id.enode
|
|
||||||
eval $cmd 2>/dev/null |grep enode | perl -pe "s/$pattern/$ip_addr/g" | perl -pe "s/^/\"/; s/\s*\$/\"/;" > $root/enodes/$id.enode
|
|
||||||
fi
|
|
||||||
# cat $root/enodes/$id.enode
|
|
||||||
echo
|
|
||||||
|
|
||||||
# copy cluster enodes list to node's static node list
|
|
||||||
# echo "copy cluster enodes list to node's static node list"
|
|
||||||
if [ -f $root/enodes.all ]; then
|
|
||||||
cp $root/enodes.all $datadir/static-nodes.json
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ ! -f $root/pids/$id.pid ]; then
|
|
||||||
# bring up node `dd` (double digit)
|
|
||||||
# - using <rootdir>/<dd>
|
|
||||||
# - listening on port 303dd, (like 30300, 30301, ...)
|
|
||||||
# - with the account unlocked
|
|
||||||
# - launching json-rpc server on port 81dd (like 8100, 8101, 8102, ...)
|
|
||||||
# echo "BZZKEY=$geth account list|head -n1|perl -ne '/([a-f0-9]{40})/ && print \$1'"
|
|
||||||
BZZKEY=`$geth account list|head -n1|perl -ne '/([a-f0-9]{40})/ && print \$1'`
|
|
||||||
echo -n "starting instance $id ($BZZKEY @ $datadir )..."
|
|
||||||
# echo "$geth \
|
|
||||||
# --identity=$id \
|
|
||||||
# --bzzaccount=$BZZKEY --bzzport=$bzzport \
|
|
||||||
# --unlock=$BZZKEY \
|
|
||||||
# --password=<(echo -n $id) \
|
|
||||||
# --rpc --rpcport=$rpcport --rpccorsdomain='*' $* \
|
|
||||||
# 2>&1 | tee "$stablelog" > "$log" & # comment out if you pipe it to a tty etc.
|
|
||||||
# " >&2
|
|
||||||
|
|
||||||
|
|
||||||
$GETH --datadir=$datadir \
|
|
||||||
--identity=$id \
|
|
||||||
--bzzaccount=$BZZKEY --bzzport=$bzzport \
|
|
||||||
--port=$port \
|
|
||||||
--unlock=$BZZKEY \
|
|
||||||
--password=<(echo -n $id) \
|
|
||||||
--rpc --rpcport=$rpcport --rpccorsdomain='*' $* \
|
|
||||||
> "$log" 2>&1 & # comment out if you pipe it to a tty etc.
|
|
||||||
|
|
||||||
ln -sf "$log" "$linklog"
|
|
||||||
|
|
||||||
# wait until ready
|
|
||||||
# pid=`ps auxwww|grep geth|grep "ty=$id"|grep -v grep|awk '{print $2}'`
|
|
||||||
# echo "pid: $pid"
|
|
||||||
# ps auxwww|grep geth|grep "ty=$id"|grep -v grep
|
|
||||||
# echo $pid > $root/pids/$id.pid
|
|
||||||
#echo $! > $root/pids/$id.pid
|
|
||||||
while true; do
|
|
||||||
$GETH --exec="net" attach ipc:$datadir/geth.ipc > /dev/null 2>&1 && break
|
|
||||||
sleep 1
|
|
||||||
echo -n "."
|
|
||||||
if ((i++>10)); then
|
|
||||||
echo "instance $id failed to start"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
echo -n "started - "
|
|
||||||
pid=`ps auxwww|grep geth|grep "ty=$id"|grep -v grep|awk '{print $2}'`
|
|
||||||
echo "pid: $pid"
|
|
||||||
# ps auxwww|grep geth|grep "ty=$id"|grep -v grep
|
|
||||||
echo $pid > $root/pids/$id.pid
|
|
||||||
fi
|
|
||||||
|
|
||||||
# to bring up logs, uncomment
|
|
||||||
# tail -f $log
|
|
||||||
|
|
@ -1,2 +1,664 @@
|
||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
bash ~/bin/swarm.sh $SWARM_DIR $SWARM_NETWORK_ID $*
|
if [ "$GETH_DIR" = "" ]; then
|
||||||
|
if [ "$GOPATH" = "" ]; then echo "either GETH_DIR or GOPATH environment variable must be set"; exit 1; fi
|
||||||
|
export GETH_DIR=$GOPATH/src/github.com/ethereum/go-ethereum
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$GETH" = "" ]; then
|
||||||
|
export GETH=$GETH_DIR/geth
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$SWARM_NETWORK_ID" = "" ]; then export SWARM_NETWORK_ID=322; fi
|
||||||
|
|
||||||
|
if [ "$SWARM_DIR" = "" ]; then export SWARM_DIR=$HOME/bzz; fi
|
||||||
|
|
||||||
|
if [ "$IP_ADDR" = "" ]; then
|
||||||
|
export IP_ADDR=`curl ipecho.net/plain 2>/dev/null;echo `
|
||||||
|
fi
|
||||||
|
|
||||||
|
root=$SWARM_DIR
|
||||||
|
network_id=$SWARM_NETWORK_ID
|
||||||
|
cmd=$1
|
||||||
|
shift
|
||||||
|
|
||||||
|
dir="$root/$network_id"
|
||||||
|
|
||||||
|
tmpdir=/tmp
|
||||||
|
|
||||||
|
# swarm attach 00 brings up a console attached to a running instance
|
||||||
|
function attach {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
echo "attaching console to instance $id"
|
||||||
|
cmd="$GETH --datadir=$dir/data/$id $* attach ipc:$dir/data/$id/geth.ipc"
|
||||||
|
# echo $cmd
|
||||||
|
eval $cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm attach 00 brings up a console attached to a running instance
|
||||||
|
function execute {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
# attach $id --exec "'$*' "
|
||||||
|
cmd="$GETH --datadir=$dir/data/$id --exec '$*' attach ipc:$dir/data/$id/geth.ipc"
|
||||||
|
# echo $cmd
|
||||||
|
eval $cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm hive 00 displays the kademlia table of the given running instance
|
||||||
|
function hive {
|
||||||
|
if [ "$1" = "all" ]; then
|
||||||
|
N=`ls -1 -d $dir/data/* |wc -l`
|
||||||
|
for ((i=0;i<N;++i)); do
|
||||||
|
instance=`printf "%02d" $i`
|
||||||
|
hive $instance
|
||||||
|
done
|
||||||
|
else
|
||||||
|
# echo "kademlia table of instance $id"
|
||||||
|
execute $1 'console.log(bzz.hive)'|grep -v undefined
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm log 00 shows the running tail logs of an instance
|
||||||
|
function log {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
echo "streaming logs for instance $id"
|
||||||
|
cmd="tail -f $dir/log/$id.log"
|
||||||
|
echo $cmd
|
||||||
|
eval $cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm cleanlog 00 removes the old log files for a given instance (all for every instance)
|
||||||
|
function cleanlog {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
if [ $id = "all" ]; then
|
||||||
|
echo "remove logs for all instances"
|
||||||
|
rm -rf "$dir/log/"
|
||||||
|
else
|
||||||
|
echo "remove logs for instance $id"
|
||||||
|
rm -rf $dir/log/$id*
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# display kademlia tables of istances
|
||||||
|
function monitor {
|
||||||
|
if [ "$3" = "" ]; then
|
||||||
|
id=$1
|
||||||
|
period=$2
|
||||||
|
while true; do
|
||||||
|
hive $id
|
||||||
|
sleep $period
|
||||||
|
done
|
||||||
|
else
|
||||||
|
node=$1
|
||||||
|
id=$2
|
||||||
|
period=$3
|
||||||
|
while true; do
|
||||||
|
remote-run $node swarm hive $id
|
||||||
|
sleep $period
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# swarm cleanbzz 00 removes the bzz subdirectory for a given instance (all for every instance)
|
||||||
|
function cleanbzz {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
if [ $id = "all" ]; then
|
||||||
|
echo "remove bzz data for all instances"
|
||||||
|
rm -rf $dir/data/*/bzz
|
||||||
|
else
|
||||||
|
echo "remove bzz data for instance $id"
|
||||||
|
rm -rf "$dir/data/$id"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm less/viewlogd 00 displays the last (current) log for the given instance (in a pager)
|
||||||
|
function viewlog {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
echo "viewing logs for instance $id"
|
||||||
|
cmd="/usr/bin/less $dir/log/$id.log"
|
||||||
|
echo $cmd
|
||||||
|
eval $cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
# display the swawrm base account for an instance
|
||||||
|
function key {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
mkdir -p $dir/data/$id/
|
||||||
|
$GETH --datadir=$dir/data/$id account list|head -n1|perl -ne '/([a-f0-9]{40})/ && print $1'
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm options 00 displays the geth command line options used to start the swarm
|
||||||
|
function rawoptions {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
|
||||||
|
globaloptions="
|
||||||
|
--dev
|
||||||
|
--maxpeers=40
|
||||||
|
--shh=false
|
||||||
|
--nodiscover
|
||||||
|
--networkid=$network_id
|
||||||
|
--bzznoswap
|
||||||
|
--verbosity=0
|
||||||
|
--vmodule=swarm/*=5"
|
||||||
|
|
||||||
|
datadir=$dir/data/$id
|
||||||
|
password=$id
|
||||||
|
port=303$id
|
||||||
|
bzzport=322$id
|
||||||
|
rpcport=302$id
|
||||||
|
key=`swarm key $id`
|
||||||
|
|
||||||
|
instanceoptions="
|
||||||
|
--datadir=$datadir
|
||||||
|
--identity=$id
|
||||||
|
--bzzaccount=$key
|
||||||
|
--unlock=$key
|
||||||
|
--bzzport=$bzzport
|
||||||
|
--port=$port
|
||||||
|
--rpc
|
||||||
|
--rpcport=$rpcport
|
||||||
|
--rpccorsdomain='*'"
|
||||||
|
|
||||||
|
echo "$globaloptions"
|
||||||
|
echo "$instanceoptions"
|
||||||
|
echo "$*"
|
||||||
|
}
|
||||||
|
|
||||||
|
function options {
|
||||||
|
echo "The command line options passed to geth are the following:"
|
||||||
|
echo "use 'geth help' to see further options"
|
||||||
|
rawoptions $*
|
||||||
|
}
|
||||||
|
|
||||||
|
function start {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
|
||||||
|
if [ -f $dir/pids/$id.pid ]; then
|
||||||
|
echo "instance $id already running"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
datetag=`date "+%Y-%m-%d-%H:%M:%S"`
|
||||||
|
log=$dir/log/$id.$datetag.log
|
||||||
|
linklog=$dir/log/$id.log
|
||||||
|
opts=`swarm rawoptions $id $*|tr '\r' ' '`
|
||||||
|
# echo; echo "$GETH $opts > $log 2>&1 &"
|
||||||
|
|
||||||
|
$GETH $opts --password=<(echo -n $id) > "$log" 2>&1 & # comment out if you pipe it to a tty etc.
|
||||||
|
ln -sf "$log" "$linklog"
|
||||||
|
|
||||||
|
# wait until ready
|
||||||
|
|
||||||
|
((j=0))
|
||||||
|
while true; do
|
||||||
|
execute $id "net" > /dev/null 2>&1 && break
|
||||||
|
sleep 1
|
||||||
|
echo -n "."
|
||||||
|
if ((j++>10)); then
|
||||||
|
echo "instance $id failed to start"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo -n "started - "
|
||||||
|
pid=`ps auxwww|grep geth|grep "ty=$id"|grep -v grep|awk '{print $2}'`
|
||||||
|
echo "pid: $pid"
|
||||||
|
echo $pid > $dir/pids/$id.pid
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# setup 00 creates the direcories for instance
|
||||||
|
function setup {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
mkdir -p $dir/data/$id
|
||||||
|
mkdir -p $dir/enodes
|
||||||
|
mkdir -p $dir/pids
|
||||||
|
mkdir -p $dir/log
|
||||||
|
}
|
||||||
|
|
||||||
|
# creates the swarm base account for an instance
|
||||||
|
function create-account {
|
||||||
|
id=$1
|
||||||
|
datadir=$dir/data/$id
|
||||||
|
# if we do not have an account, create one
|
||||||
|
# will not prompt for password, we use the double digit instance id as passwd
|
||||||
|
# NEVER EVER USE THESE ACCOUNTS FOR INTERACTING WITH A LIVE CHAIN
|
||||||
|
keystoredir="$datadir/keystore/"
|
||||||
|
# echo "KeyStore dir: $keystoredir"
|
||||||
|
if [ ! -d "$keystoredir" ]; then
|
||||||
|
# echo "create an account with password $id [DO NOT EVER USE THIS ON LIVE]"
|
||||||
|
# mkdir -p $datadir/keystore
|
||||||
|
$GETH --datadir=$datadir --password=<(echo -n $id) account new >/dev/null 2>&1
|
||||||
|
# create account with password 00, 01, ...
|
||||||
|
# note that the account key will be stored also separately outside
|
||||||
|
# datadir
|
||||||
|
# this way you can safely clear the data directory and still keep your key
|
||||||
|
# under <rootdir>/keystore/dd
|
||||||
|
# LS=$(ls $datadir/keystore)
|
||||||
|
# echo $LS
|
||||||
|
while [ ! -d "$keystoredir" ]; do
|
||||||
|
echo "."
|
||||||
|
((i++))
|
||||||
|
if ((i>10)); then break; fi
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
# echo "copying keys $datadir/keystore $root/keystore/$id"
|
||||||
|
mkdir -p $dir/keystore/$id
|
||||||
|
cp -R "$keystoredir" $dir/keystore/$id
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# shuts down a running instance, cleans the pid
|
||||||
|
function stop {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
if [ $id = "all" ]; then
|
||||||
|
procs=`cat $dir/pids/*.pid 2>/dev/null |perl -pe 's/^\s+//;s/\s+\\$//;s/\s+/\n/g'`
|
||||||
|
# echo "stopping processes $procs"
|
||||||
|
for p in $procs; do
|
||||||
|
shutdown $p
|
||||||
|
done
|
||||||
|
rm -rf $dir/pids/*
|
||||||
|
else
|
||||||
|
pid=$dir/pids/$id.pid
|
||||||
|
if [ -f $pid ]; then
|
||||||
|
echo "stopping instance $id, pid="`cat $pid`
|
||||||
|
shutdown `cat $pid`
|
||||||
|
rm $pid
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# shutdown kills the node with interrupt 2 - if it resits falls back to -9 after 10s
|
||||||
|
function shutdown {
|
||||||
|
echo -n "stopping $1..."
|
||||||
|
kill -2 $1
|
||||||
|
while true; do
|
||||||
|
ps auxwww|grep geth|grep -v grep|awk '{print $2}'|grep -ql $1 || break
|
||||||
|
if ((i++>5)); then
|
||||||
|
echo "not stopping. killing it"
|
||||||
|
kill -QUIT $1
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
echo '.'
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
echo "stopped"
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm restart 00 calls stop and start
|
||||||
|
function restart {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
if [ $id = "all" ]; then
|
||||||
|
stop all
|
||||||
|
N=`ls -d1 $dir/data/*|wc -l`
|
||||||
|
cluster $N $*
|
||||||
|
else
|
||||||
|
stop $id
|
||||||
|
start $id $*
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm init X sets up and starts a new client instance
|
||||||
|
##########################################################
|
||||||
|
#
|
||||||
|
# IT WIPES THE DATABASE
|
||||||
|
#
|
||||||
|
##########################################################
|
||||||
|
function init {
|
||||||
|
killall geth
|
||||||
|
reset all
|
||||||
|
cluster $*
|
||||||
|
enode all
|
||||||
|
connect all
|
||||||
|
}
|
||||||
|
|
||||||
|
# reset wipes the datadirs of the instance
|
||||||
|
function reset {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
if [ $id = "all" ]; then
|
||||||
|
rm -rf $dir
|
||||||
|
else
|
||||||
|
rm -rf$dir/*/$id*
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# enode displays the instance's enode address
|
||||||
|
# swarm enode all writes all instances' enodes in a file
|
||||||
|
function enode {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
|
||||||
|
if [ $id = "all" ]; then
|
||||||
|
json=$dir/static-nodes.json
|
||||||
|
enodes=$dir/enodes.lst
|
||||||
|
cmd=$dir/connect.js
|
||||||
|
rm -f $enodes $json $cmd
|
||||||
|
# build a static nodes(-like) list of all enodes of the local cluster
|
||||||
|
echo "[" >> $json
|
||||||
|
N=`ls -1 -d $dir/data/* |wc -l`
|
||||||
|
for ((i=0;i<N;++i)); do
|
||||||
|
id=`printf "%02d" $i`
|
||||||
|
f=$dir/enodes/$id.enode
|
||||||
|
enode $id > $f
|
||||||
|
echo -n "admin.addPeer(" >> $cmd
|
||||||
|
cat "$f" | perl -pe 's/\s*$//' >> $cmd
|
||||||
|
echo ");" >> $cmd
|
||||||
|
cat $f |perl -pe 's/"//g'>> $enodes
|
||||||
|
cat $f >> $json
|
||||||
|
echo "," >> $json
|
||||||
|
done
|
||||||
|
echo "\"\"]" >> $json
|
||||||
|
cat $enodes
|
||||||
|
else
|
||||||
|
# echo "local IP: $ip_addr "
|
||||||
|
execute $id 'admin.nodeInfo.enode' |perl -pe "s/\[\:\:\]/$IP_ADDR/ "
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# connect sources the local or remote set of peers and connects the node to the peers
|
||||||
|
function connect {
|
||||||
|
id=$1
|
||||||
|
shift
|
||||||
|
if [ $id = "all" ]; then
|
||||||
|
N=`ls -1 -d $dir/data/* |wc -l`
|
||||||
|
for ((i=0;i<N;++i)); do
|
||||||
|
id=`printf "%02d" $i`
|
||||||
|
connect $id
|
||||||
|
done
|
||||||
|
else
|
||||||
|
echo -n 'peer count: '
|
||||||
|
attach $id --preload $dir/connect.js --exec net.peerCount
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm cluster N launches N nodes; 00 01 02 ...
|
||||||
|
function cluster {
|
||||||
|
N=$1
|
||||||
|
shift
|
||||||
|
echo "launching cluster of $N instances"
|
||||||
|
for ((i=0;i<N;++i)); do
|
||||||
|
id=`printf "%02d" $i`
|
||||||
|
setup $id $*
|
||||||
|
create-account $id
|
||||||
|
echo "launching node $i/$N.."
|
||||||
|
start $id $*
|
||||||
|
# info $id
|
||||||
|
enode $id
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm needs instance keyfile destination
|
||||||
|
# tests if the content for the key is available for the intance
|
||||||
|
#
|
||||||
|
function needs {
|
||||||
|
id=$1
|
||||||
|
keyfile=$2
|
||||||
|
|
||||||
|
target=$3
|
||||||
|
dest=$tmpdir/down
|
||||||
|
mkdir -p $dest
|
||||||
|
file=$dest/`basename $target`
|
||||||
|
rm -f $file
|
||||||
|
echo -n "waiting for root hash in '$keyfile'..."
|
||||||
|
while true; do
|
||||||
|
if [ ! -z $keyfile ]; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 1
|
||||||
|
echo -n "."
|
||||||
|
done
|
||||||
|
key=`cat $keyfile|tr -d \"`
|
||||||
|
echo " => $key"
|
||||||
|
download $id $key $dest && cmp --silent $file $target && echo "PASS" || echo "FAIL"
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm up 00 file uploads file via instances CLI
|
||||||
|
function up { #port, file
|
||||||
|
echo "Upload file '$2' to node $1... " 1>&2
|
||||||
|
file=`basename $2`
|
||||||
|
execute $1 "bzz.upload(\"$2\", \"$file\")"|tail -n1> /tmp/key
|
||||||
|
cat /tmp/key
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm download 00 file download file via instances CLI
|
||||||
|
function download {
|
||||||
|
echo "download '$2' from node $1 to '$3'"
|
||||||
|
execute $1 "bzz.download(\"$2\", \"$3\")" >/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
# swarm down issues bzz.get to download the content 10 attempts
|
||||||
|
function down {
|
||||||
|
echo -n "Download hash '$2' from node $1... "
|
||||||
|
while true; do
|
||||||
|
execute $1 "bzz.get(\"$2\")" 2> /dev/null |grep -qil "status" && break
|
||||||
|
sleep 1
|
||||||
|
echo -n "."
|
||||||
|
if ((i++>10)); then
|
||||||
|
echo "not found"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo "found OK"
|
||||||
|
}
|
||||||
|
|
||||||
|
# static info about an instance (available even if node is off)
|
||||||
|
function info {
|
||||||
|
echo "swarm node information"
|
||||||
|
echo "ROOTDIR: $root"
|
||||||
|
echo "DATADIR: $dir/data/$1"
|
||||||
|
echo "LOGFILE: $dir/log/$1.log"
|
||||||
|
echo "HTTPAPI: http://localhost:322$1"
|
||||||
|
echo "ETHPORT: 303$1"
|
||||||
|
echo "RPCPORT: 302$1"
|
||||||
|
echo "ACCOUNT:" 0x`ls -1 $dir/data/$1/bzz`
|
||||||
|
echo "CHEQUEB:" `cat $dir/data/$1/bzz/*/config.json|grep Contract|awk -F\" '{print $4}'`
|
||||||
|
echo "ROOTDIR: $root"
|
||||||
|
echo "DATADIR: $dir/data/$1"
|
||||||
|
echo "LOGFILE: $dir/log/$1.log"
|
||||||
|
}
|
||||||
|
|
||||||
|
# live into about an instance
|
||||||
|
function status {
|
||||||
|
echo -n "account balance: "
|
||||||
|
execute $1 'eth.getBalance(eth.accounts[0])'
|
||||||
|
echo -n "swap contract balance: "
|
||||||
|
execute $1 "eth.getBalance(bzz.info.Swap.Contract)"
|
||||||
|
echo -n "chequebook balance: "
|
||||||
|
execute $1 "chequebook.balance"
|
||||||
|
echo -n "peer count: "
|
||||||
|
execute $1 'net.peerCount'
|
||||||
|
echo -n "latest block number: "
|
||||||
|
execute $1 "eth.blockNumber"
|
||||||
|
}
|
||||||
|
|
||||||
|
# display peers for an instance
|
||||||
|
function peers {
|
||||||
|
execute $1 'admin.peers'
|
||||||
|
}
|
||||||
|
|
||||||
|
# add peers into an instance (connection not guaranteed)
|
||||||
|
function addpeers {
|
||||||
|
id=$1
|
||||||
|
peers=$2
|
||||||
|
if [ $id = "all" ]; then
|
||||||
|
N=`ls -1 -d $dir/data/* |wc -l`
|
||||||
|
for ((i=0;i<N;++i)); do
|
||||||
|
id=`printf "%02d" $i`
|
||||||
|
addpeers $id $peers
|
||||||
|
done
|
||||||
|
else
|
||||||
|
echo "addpeer to instance $id"
|
||||||
|
for peer in `cat $peers|grep -v '^#'`; do
|
||||||
|
execute $id "admin.addPeer(\"$peer\")"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# configures the eth-net-intelligence-api network monitor
|
||||||
|
function netstatconf {
|
||||||
|
group=$1
|
||||||
|
N=`ls -1 -d $dir/data/* |wc -l`
|
||||||
|
ip=`curl ipecho.net/plain 2>/dev/null;echo `
|
||||||
|
ws_server="ws://146.185.130.117:3000"
|
||||||
|
ws_secret=BZZ322
|
||||||
|
conf="$dir/$group-$ip.netstat.json"
|
||||||
|
|
||||||
|
echo "writing netstat conf for cluster $group-$ip ($N instances) -> $conf"
|
||||||
|
|
||||||
|
echo -e "[" > $conf
|
||||||
|
|
||||||
|
for ((i=0;i<$N;++i)); do
|
||||||
|
id=`printf "%02d" $i`
|
||||||
|
single_template=" {\n \"name\" : \"$group-$ip-$i\",\n \"cwd\" : \".\",\n \"script\" : \"app.js\",\n \"log_date_format\" : \"YYYY-MM-DD HH:mm Z\",\n \"merge_logs\" : false,\n \"watch\" : false,\n \"exec_interpreter\" : \"node\",\n \"exec_mode\" : \"fork_mode\",\n \"env\":\n {\n \"NODE_ENV\" : \"production\",\n \"RPC_HOST\" : \"localhost\",\n \"RPC_PORT\" : \"302$id\",\n \"INSTANCE_NAME\" : \"$group-$ip-$i\",\n \"WS_SERVER\" : \"$ws_server\",\n \"WS_SECRET\" : \"$ws_secret\",\n }\n }"
|
||||||
|
|
||||||
|
endline=""
|
||||||
|
if ((i<$N-1)); then
|
||||||
|
# if [ "$i" -ne "$N" ]; then
|
||||||
|
endline=","
|
||||||
|
fi
|
||||||
|
echo -e "$single_template$endline" >> $conf
|
||||||
|
done
|
||||||
|
echo "]" >> $conf
|
||||||
|
}
|
||||||
|
|
||||||
|
# (re)starts the eth-net-intelligence-api network monitor
|
||||||
|
function netstatrun {
|
||||||
|
cd $GETH_DIR/../eth-net-intelligence-api
|
||||||
|
pm2 kill
|
||||||
|
pm2 start $dir/*.netstat.json
|
||||||
|
}
|
||||||
|
|
||||||
|
# kills the eth-net-intelligence-api network monitor
|
||||||
|
function netstatkill {
|
||||||
|
cd $GETH_DIR/../eth-net-intelligence-api
|
||||||
|
pm2 kill
|
||||||
|
}
|
||||||
|
|
||||||
|
# copies the swarm control script to the remote node(s)
|
||||||
|
function remote-update-scripts {
|
||||||
|
scriptdir=$GETH_DIR/swarm/cmd/swarm/
|
||||||
|
remotes=$1
|
||||||
|
for remote in `cat $remotes|grep -v '^#'`; do echo "updating scripts on $remote..."; ssh $remote mkdir -p bin && scp -r $scriptdir/* $remote:bin/; done
|
||||||
|
}
|
||||||
|
|
||||||
|
# copies the geth executable to the remote nodes
|
||||||
|
function remote-update-bin {
|
||||||
|
remotes=$1
|
||||||
|
# remote-update-scripts $remotes
|
||||||
|
for remote in `cat $remotes|grep -v '^#'`; do echo "updating binary on $remote..."; scp -r $GETH_DIR/geth $remote:bin/; done
|
||||||
|
}
|
||||||
|
|
||||||
|
# runs a command on remote node or nodes from a file
|
||||||
|
function remote-run {
|
||||||
|
remotes=$1
|
||||||
|
shift
|
||||||
|
if `echo "$remotes" | grep -qil @`; then
|
||||||
|
ssh $remotes '. $HOME/bin/env.sh;' "$*"
|
||||||
|
else
|
||||||
|
for remote in `cat $remotes|grep -v '^#'`; do echo "running on $remote..."; remote-run $remote "$*"; done
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# updates the code from the given branch
|
||||||
|
function update {
|
||||||
|
branch=$1
|
||||||
|
echo "cd $GETH_DIR && git remote update && git reset --hard $branch"
|
||||||
|
(cd $GETH_DIR && git remote update && git reset --hard $branch)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
case $cmd in
|
||||||
|
"info" )
|
||||||
|
info $*;;
|
||||||
|
"enode" )
|
||||||
|
enode $*;;
|
||||||
|
"status" )
|
||||||
|
status $*;;
|
||||||
|
"peers" )
|
||||||
|
peers $*;;
|
||||||
|
"addpeers" )
|
||||||
|
addpeers $*;;
|
||||||
|
"clean" )
|
||||||
|
clean $*;;
|
||||||
|
"needs" )
|
||||||
|
needs $*;;
|
||||||
|
"up" )
|
||||||
|
up $*;;
|
||||||
|
"key" )
|
||||||
|
key $*;;
|
||||||
|
"down" )
|
||||||
|
down $*;;
|
||||||
|
"download" )
|
||||||
|
download $*;;
|
||||||
|
"init" )
|
||||||
|
init $*;;
|
||||||
|
"exec" )
|
||||||
|
execute $*;;
|
||||||
|
"hive" )
|
||||||
|
hive $*;;
|
||||||
|
"start" )
|
||||||
|
start $*;;
|
||||||
|
"stop" )
|
||||||
|
stop $* ;;
|
||||||
|
"restart" )
|
||||||
|
restart $*;;
|
||||||
|
"reset" )
|
||||||
|
reset $*;;
|
||||||
|
"cluster" )
|
||||||
|
cluster $*;;
|
||||||
|
"attach" )
|
||||||
|
attach $*;;
|
||||||
|
"execute" )
|
||||||
|
execute $*;;
|
||||||
|
"exec" )
|
||||||
|
execute $*;;
|
||||||
|
"cleanbzz" )
|
||||||
|
cleanbzz $*;;
|
||||||
|
"cleanlog" )
|
||||||
|
cleanlog $*;;
|
||||||
|
"log" )
|
||||||
|
log $*;;
|
||||||
|
"viewlog" )
|
||||||
|
viewlog $*;;
|
||||||
|
"less" )
|
||||||
|
viewlog $*;;
|
||||||
|
"connect" )
|
||||||
|
connect $*;;
|
||||||
|
"monitor" )
|
||||||
|
monitor $*;;
|
||||||
|
"remote-update-scripts" )
|
||||||
|
remote-update-scripts $*;;
|
||||||
|
"remote-update-bin" )
|
||||||
|
remote-update-bin $*;;
|
||||||
|
"update-src" )
|
||||||
|
update-src $*;;
|
||||||
|
"remote-run" )
|
||||||
|
remote-run $*;;
|
||||||
|
"netstatconf" )
|
||||||
|
netstatconf $*;;
|
||||||
|
"netstatkill" )
|
||||||
|
netstatkill $*;;
|
||||||
|
"netstatrun" )
|
||||||
|
netstatrun $*;;
|
||||||
|
"options" )
|
||||||
|
options $*;;
|
||||||
|
"rawoptions" )
|
||||||
|
rawoptions $*;;
|
||||||
|
"setup" )
|
||||||
|
setup $* ;;
|
||||||
|
"create-account" )
|
||||||
|
create-account $*;;
|
||||||
|
|
||||||
|
esac
|
||||||
|
|
|
||||||
|
|
@ -1,432 +0,0 @@
|
||||||
# !/bin/bash
|
|
||||||
# bash cluster <root> <network_id> <number_of_nodes> <runid> <local_IP> [[params]...]
|
|
||||||
# https://github.com/ethereum/go-ethereum/wiki/Setting-up-monitoring-on-local-cluster
|
|
||||||
|
|
||||||
# sets up a local ethereum network cluster of nodes
|
|
||||||
# - <number_of_nodes> is the number of nodes in cluster
|
|
||||||
# - <root> is the root directory for the cluster, the nodes are set up
|
|
||||||
# with datadir `<root>/<network_id>/00`, `<root>/ <network_id>/01`, ...
|
|
||||||
# - new accounts are created for each node
|
|
||||||
# - they launch on port 30300, 30301, ...
|
|
||||||
# - they star rpc on port 8100, 8101, ...
|
|
||||||
# - by collecting the nodes nodeUrl, they get connected to each other
|
|
||||||
# - if enode has no IP, `<local_IP>` is substituted
|
|
||||||
# - if `<network_id>` is not 0, they will not connect to a default client,
|
|
||||||
# resulting in a private isolated network
|
|
||||||
# - the nodes log into `<root>/<network_id>/00.<runid>.log`, `<root>/<network_id>/01.<runid>.log`, ...
|
|
||||||
# - The nodes launch in mining mode
|
|
||||||
# - the cluster can be killed with `killall geth` (FIXME: should record PIDs)
|
|
||||||
# and restarted from the same state
|
|
||||||
# - if you want to interact with the nodes, use rpc
|
|
||||||
# - you can supply additional params on the command line which will be passed
|
|
||||||
# to each node, for instance `-mine`
|
|
||||||
|
|
||||||
if [ "$GETH" = "" ]; then
|
|
||||||
echo "env var GETH not set "
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
srcdir=`dirname $0`
|
|
||||||
|
|
||||||
root=$1
|
|
||||||
shift
|
|
||||||
network_id=$1
|
|
||||||
shift
|
|
||||||
cmd=$1
|
|
||||||
shift
|
|
||||||
# ip_addr=`curl ipecho.net/plain 2>/dev/null;echo `
|
|
||||||
|
|
||||||
# echo "external IP: $ip_addr"
|
|
||||||
swarmoptions='--dev --maxpeers=40 --shh=false --nodiscover'
|
|
||||||
tmpdir=/tmp
|
|
||||||
|
|
||||||
function attach {
|
|
||||||
id=$1
|
|
||||||
shift
|
|
||||||
echo "attaching console to instance $id"
|
|
||||||
cmd="$GETH $* attach ipc:$root/$network_id/data/$id/geth.ipc"
|
|
||||||
echo $cmd
|
|
||||||
eval $cmd
|
|
||||||
}
|
|
||||||
|
|
||||||
function log {
|
|
||||||
id=$1
|
|
||||||
shift
|
|
||||||
echo "streaming logs for instance $id"
|
|
||||||
cmd="tail -f $root/$network_id/log/$id.log"
|
|
||||||
echo $cmd
|
|
||||||
eval $cmd
|
|
||||||
}
|
|
||||||
|
|
||||||
function cleanlog {
|
|
||||||
id=$1
|
|
||||||
shift
|
|
||||||
if [ $id = "all" ]; then
|
|
||||||
echo "remove logs for all instances"
|
|
||||||
rm -rf "$root/$network_id/log/"
|
|
||||||
else
|
|
||||||
echo "remove logs for instance $id"
|
|
||||||
rm -rf $root/$network_id/log/$id*
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
function cleanbzz {
|
|
||||||
id=$1
|
|
||||||
shift
|
|
||||||
if [ $id = "all" ]; then
|
|
||||||
echo "remove bzz data for all instances"
|
|
||||||
rm -rf $root/$network_id/data/*/bzz
|
|
||||||
else
|
|
||||||
echo "remove bzz data for instance $id"
|
|
||||||
rm -rf "$root/$network_id/data/$id"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
function less {
|
|
||||||
id=$1
|
|
||||||
shift
|
|
||||||
echo "viewing logs for instance $id"
|
|
||||||
cmd="/usr/bin/less $root/$network_id/log/$id.log"
|
|
||||||
echo $cmd
|
|
||||||
eval $cmd
|
|
||||||
}
|
|
||||||
|
|
||||||
function start {
|
|
||||||
id=$1
|
|
||||||
shift
|
|
||||||
# echo -n "starting instance $id - "
|
|
||||||
cmd="bash $srcdir/gethup.sh $root/$network_id/ $id '$ip_addr' --networkid=$network_id $swarmoptions $*"
|
|
||||||
# echo "pid="`cat $root/$network_id/pids/$id.pid`
|
|
||||||
# echo $cmd
|
|
||||||
eval $cmd
|
|
||||||
}
|
|
||||||
|
|
||||||
function stop {
|
|
||||||
id=$1
|
|
||||||
shift
|
|
||||||
if [ $id = "all" ]; then
|
|
||||||
procs=`cat $root/$network_id/pids/*.pid 2>/dev/null |perl -pe 's/^\s+//;s/\s+\\$//;s/\s+/\n/g'`
|
|
||||||
# echo "stopping processes $procs"
|
|
||||||
for p in $procs; do
|
|
||||||
shutdown $p
|
|
||||||
done
|
|
||||||
rm -rf $root/$network_id/pids/*
|
|
||||||
else
|
|
||||||
pid=$root/$network_id/pids/$id.pid
|
|
||||||
if [ -f $pid ]; then
|
|
||||||
echo "stopping instance $id, pid="`cat $pid`
|
|
||||||
shutdown `cat $pid`
|
|
||||||
rm $pid
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
# ps auxwww|grep geth|grep bzz|grep -v grep
|
|
||||||
}
|
|
||||||
|
|
||||||
function shutdown {
|
|
||||||
echo -n "stopping $1..."
|
|
||||||
kill -2 $1
|
|
||||||
while true ;do
|
|
||||||
ps auxwww|grep geth|grep -v grep|awk '{print $2}'|grep -ql $1 || break
|
|
||||||
sleep 1
|
|
||||||
done
|
|
||||||
echo "stopped"
|
|
||||||
}
|
|
||||||
|
|
||||||
function restart {
|
|
||||||
id=$1
|
|
||||||
shift
|
|
||||||
stop $id
|
|
||||||
start $id $*
|
|
||||||
}
|
|
||||||
|
|
||||||
function init {
|
|
||||||
stop all
|
|
||||||
killall geth
|
|
||||||
reset all
|
|
||||||
cluster $*
|
|
||||||
enode all
|
|
||||||
connect all
|
|
||||||
}
|
|
||||||
|
|
||||||
function reset {
|
|
||||||
id=$1
|
|
||||||
shift
|
|
||||||
if [ $id = "all" ]; then
|
|
||||||
rm -rf $root/$network_id
|
|
||||||
else
|
|
||||||
rm -rf$root/$network_id/*/$id*
|
|
||||||
fi
|
|
||||||
|
|
||||||
}
|
|
||||||
|
|
||||||
function enode {
|
|
||||||
dir=$root/$network_id
|
|
||||||
id=$1
|
|
||||||
shift
|
|
||||||
if [ $id = "all" ]; then
|
|
||||||
N=`ls -1 $dir/enodes/|wc -l`
|
|
||||||
enodes=$dir/enodes.all
|
|
||||||
rm -f $enodes
|
|
||||||
# build a static nodes(-like) list of all enodes of the local cluster
|
|
||||||
echo "[" >> $enodes
|
|
||||||
for ((i=0;i<N;++i)); do
|
|
||||||
id=`printf "%02d" $i`
|
|
||||||
enode=$dir/enodes/$id.enode
|
|
||||||
enode $id
|
|
||||||
if [ -f "$enode" ] && [ ! -z "$enode" ]; then
|
|
||||||
cat "$enode" >> $enodes
|
|
||||||
echo "," >> $enodes
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
echo "\"\"]" >> $enodes
|
|
||||||
cmd=$dir/connect.js
|
|
||||||
for ((i=0;i<N;++i)); do
|
|
||||||
id=`printf "%02d" $i`
|
|
||||||
enode=$dir/enodes/$id.enode
|
|
||||||
if [ -f "$enode" ] && [ ! -z "$enode" ]; then
|
|
||||||
echo -n "admin.addPeer(" >> $cmd
|
|
||||||
cat "$enode" >> $cmd
|
|
||||||
echo ");" >> $cmd
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
else
|
|
||||||
enode=$dir/enodes/$id.enode
|
|
||||||
attach $id --exec "'console.log(admin.nodeInfo.enode)'" |head -2 |tail -1| perl -pe 's/^/"/;s/$/"/'|perl -pe 's/\s*$//' > $enode
|
|
||||||
# cat $enode
|
|
||||||
fi
|
|
||||||
|
|
||||||
}
|
|
||||||
|
|
||||||
function connect {
|
|
||||||
dir=$root/$network_id
|
|
||||||
id=$1
|
|
||||||
shift
|
|
||||||
if [ $id = "all" ]; then
|
|
||||||
for ((i=0;i<N;++i)); do
|
|
||||||
id=`printf "%02d" $i`
|
|
||||||
connect $id
|
|
||||||
done
|
|
||||||
else
|
|
||||||
cmd="$GETH --preload $dir/connect.js --exec '\"admin.peers\"' attach ipc:$root/$network_id/data/$id/geth.ipc $dir/connect.js"
|
|
||||||
# echo $cmd
|
|
||||||
eval $cmd
|
|
||||||
cat $dir/connect.js
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
function cluster {
|
|
||||||
N=$1
|
|
||||||
shift
|
|
||||||
echo "launching cluster of $N instances"
|
|
||||||
# cmd="bash $srcdir/gethcluster.sh $root $network_id $N '' $swarmoptions $*"
|
|
||||||
# echo $cmd
|
|
||||||
# eval $cmd
|
|
||||||
dir=$root/$network_id
|
|
||||||
mkdir -p $dir/data
|
|
||||||
mkdir -p $dir/enodes
|
|
||||||
mkdir -p $dir/pids
|
|
||||||
mkdir -p $dir/log
|
|
||||||
|
|
||||||
for ((i=0;i<N;++i)); do
|
|
||||||
id=`printf "%02d" $i`
|
|
||||||
mkdir -p $dir/data/$id
|
|
||||||
echo "launching node $i/$N ---> tail -f $dir/log/$id.log"
|
|
||||||
start $id $vmodule $*
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
function needs {
|
|
||||||
id=$1
|
|
||||||
keyfile=$2
|
|
||||||
target=$3
|
|
||||||
dir=`dirname $3`
|
|
||||||
dest=$tmpdir/down
|
|
||||||
mkdir -p $dest
|
|
||||||
file=$dest/`basename $target`
|
|
||||||
rm -f $file
|
|
||||||
echo -n "waiting for root hash in '$keyfile'..."
|
|
||||||
while true; do
|
|
||||||
if [ -f $keyfile ] && [ ! -z $keyfile ]; then
|
|
||||||
break
|
|
||||||
fi
|
|
||||||
sleep 1
|
|
||||||
echo -n "."
|
|
||||||
done
|
|
||||||
key=`cat $keyfile|tr -d \"`
|
|
||||||
echo " => $key"
|
|
||||||
download $id $key $dest && cmp --silent $file $target && echo "PASS" || echo "FAIL"
|
|
||||||
# && ls -l $keyfile $file $target
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
function up { #port, file
|
|
||||||
echo "Upload file '$2' to node $1... " 1>&2
|
|
||||||
file=`basename $2`
|
|
||||||
attach $1 "--exec 'bzz.upload(\"$2\", \"$file\")'"|tail -n1> /tmp/key
|
|
||||||
# key=`bash swarm/cmd/bzzup.sh $2 86$1`
|
|
||||||
cat /tmp/key
|
|
||||||
}
|
|
||||||
|
|
||||||
function download {
|
|
||||||
echo "download '$2' from node $1 to '$3'"
|
|
||||||
# echo attach $1 "--exec 'bzz.download(\"$2\", \"$3\")'"
|
|
||||||
attach $1 "--exec 'bzz.download(\"$2\", \"$3\")'" > /dev/null
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
function down {
|
|
||||||
echo -n "Download hash '$2' from node $1... "
|
|
||||||
# echo "wget -O- http://localhost:86$1/$2 > /dev/null 2>&1 && echo 'got it' || echo 'not found'"
|
|
||||||
# wget -O- http://localhost:86$1/$2 > /dev/null 2>&1 && echo "got it" || echo "not found"
|
|
||||||
while true; do
|
|
||||||
attach $1 "--exec 'bzz.get(\"$2\")'" 2> /dev/null |grep -qil "status" && break
|
|
||||||
sleep 1
|
|
||||||
echo -n "."
|
|
||||||
if ((i++>10)); then
|
|
||||||
echo "not found"
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
echo "found OK"
|
|
||||||
}
|
|
||||||
|
|
||||||
function clean { #index
|
|
||||||
echo "Clean up for $1"
|
|
||||||
rm -rf $root/$network_id/data/$1/{bzz/*/chunks,bzz/*/requests/,bzz/*/bzz-peers.json,chaindata,nodes}
|
|
||||||
}
|
|
||||||
|
|
||||||
function info {
|
|
||||||
echo "swarm node information"
|
|
||||||
echo "ROOTDIR: $root"
|
|
||||||
echo "DATADIR: $root/$network_id/data/$1"
|
|
||||||
echo "LOGFILE: $root/$network_id/log/$1.log"
|
|
||||||
echo "HTTPAPI: http://localhost:322$1"
|
|
||||||
echo "ETHPORT: 303$1"
|
|
||||||
echo "RPCPORT: 302$1"
|
|
||||||
echo "ACCOUNT:" 0x`ls -1 $root/$network_id/data/$1/bzz`
|
|
||||||
echo "CHEQUEB:" `cat $root/$network_id/data/$1/bzz/*/config.json|grep Contract|awk -F\" '{print $4}'`
|
|
||||||
echo "ROOTDIR: $root"
|
|
||||||
echo "DATADIR: $root/$network_id/data/$1"
|
|
||||||
echo "LOGFILE: $root/$network_id/log/$1.log"
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
function status {
|
|
||||||
attach 00 -exec "'console.log(eth.getBalance(eth.accounts[0])); console.log(eth.getBalance(bzz.info().Swap.Contract)); console.log(chequebook.balance)'"
|
|
||||||
}
|
|
||||||
|
|
||||||
function netstatconf {
|
|
||||||
begin=$1
|
|
||||||
N=$2
|
|
||||||
name_prefix=$3
|
|
||||||
ws_server=$4
|
|
||||||
ws_secret=$5
|
|
||||||
conf="$root/$network_id/$name_prefix.netstat.json"
|
|
||||||
|
|
||||||
echo "writing netstat conf for cluster $name_prefix to $conf"
|
|
||||||
|
|
||||||
echo -e "[" > $conf
|
|
||||||
|
|
||||||
for ((i=$begin;i<$start+$N;++i)); do
|
|
||||||
id=`printf "%02d" $i`
|
|
||||||
single_template=" {\n \"name\" : \"$name_prefix-$i\",\n \"cwd\" : \".\",\n \"script\" : \"app.js\",\n \"log_date_format\" : \"YYYY-MM-DD HH:mm Z\",\n \"merge_logs\" : false,\n \"watch\" : false,\n \"exec_interpreter\" : \"node\",\n \"exec_mode\" : \"fork_mode\",\n \"env\":\n {\n \"NODE_ENV\" : \"production\",\n \"RPC_HOST\" : \"localhost\",\n \"RPC_PORT\" : \"302$id\",\n \"INSTANCE_NAME\" : \"$name_prefix-$i\",\n \"WS_SERVER\" : \"$ws_server\",\n \"WS_SECRET\" : \"$ws_secret\",\n }\n }"
|
|
||||||
|
|
||||||
endline=""
|
|
||||||
if (($i<$N-1)); then
|
|
||||||
# if [ "$i" -ne "$N" ]; then
|
|
||||||
endline=","
|
|
||||||
fi
|
|
||||||
echo -e "$single_template$endline" >> $conf
|
|
||||||
done
|
|
||||||
|
|
||||||
echo "]" >> $conf
|
|
||||||
}
|
|
||||||
|
|
||||||
function remote-update-scripts {
|
|
||||||
scriptdir=$1
|
|
||||||
remotes=$2
|
|
||||||
cd $GETH_DIR
|
|
||||||
for remote in `cat $remotes|grep -v '^#'`; do echo "updating scripts on $remote..."; ssh $remote mkdir -p bin && scp -r $scriptdir/* $remote:bin/; done
|
|
||||||
}
|
|
||||||
|
|
||||||
function remote-update-bin {
|
|
||||||
remotes=$1
|
|
||||||
remote-update-scripts $GETH_DIR/swarm/cmd/swarm/ $remotes
|
|
||||||
for remote in `cat $remotes|grep -v '^#'`; do echo "updating binary on $remote..."; scp -r $GETH_DIR/geth $remote:bin/; done
|
|
||||||
}
|
|
||||||
|
|
||||||
function remote-run {
|
|
||||||
remotes=$1
|
|
||||||
shift
|
|
||||||
for remote in `cat $remotes|grep -v '^#'`; do echo "running on $remote..."; ssh $remote ". ~/bin/env.sh; $*"; done
|
|
||||||
}
|
|
||||||
|
|
||||||
function update-src {
|
|
||||||
branch=$1
|
|
||||||
echo "cd $GETH_DIR && git remote update && git reset --hard $branch"
|
|
||||||
(cd $GETH_DIR && git remote update && git reset --hard $branch)
|
|
||||||
}
|
|
||||||
|
|
||||||
function netstatrun {
|
|
||||||
cd ~/eth-net-intelligence-api
|
|
||||||
pm2 kill
|
|
||||||
pm2 start $root/$network_id/*.netstat.json
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
case $cmd in
|
|
||||||
"info" )
|
|
||||||
info $*;;
|
|
||||||
"enode" )
|
|
||||||
enode $*;;
|
|
||||||
"connect" )
|
|
||||||
connect $*;;
|
|
||||||
"status" )
|
|
||||||
status $*;;
|
|
||||||
"clean" )
|
|
||||||
clean $*;;
|
|
||||||
"needs" )
|
|
||||||
needs $*;;
|
|
||||||
"up" )
|
|
||||||
up $*;;
|
|
||||||
"down" )
|
|
||||||
down $*;;
|
|
||||||
"download" )
|
|
||||||
download $*;;
|
|
||||||
"init" )
|
|
||||||
init $*;;
|
|
||||||
"start" )
|
|
||||||
start $*;;
|
|
||||||
"stop" )
|
|
||||||
stop $* ;;
|
|
||||||
"restart" )
|
|
||||||
restart $*;;
|
|
||||||
"reset" )
|
|
||||||
reset $*;;
|
|
||||||
"cluster" )
|
|
||||||
cluster $*;;
|
|
||||||
"attach" )
|
|
||||||
attach $*;;
|
|
||||||
"cleanbzz" )
|
|
||||||
cleanbzz $*;;
|
|
||||||
"cleanlog" )
|
|
||||||
cleanlog $*;;
|
|
||||||
"log" )
|
|
||||||
log $*;;
|
|
||||||
"less" )
|
|
||||||
less $*;;
|
|
||||||
"remote-update-scripts" )
|
|
||||||
remote-update-scripts $*;;
|
|
||||||
"remote-update-bin" )
|
|
||||||
remote-update-bin $*;;
|
|
||||||
"update-src" )
|
|
||||||
update-src $*;;
|
|
||||||
"remote-run" )
|
|
||||||
remote-run $*;;
|
|
||||||
"netstatconf" )
|
|
||||||
netstatconf $*;;
|
|
||||||
"netstatrun" )
|
|
||||||
netstatrun $*;;
|
|
||||||
|
|
||||||
esac
|
|
||||||
|
|
@ -1,28 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
|
|
||||||
TEST_DIR=`dirname $0`
|
|
||||||
TEST_NAME=`basename $0 .sh`
|
|
||||||
TEST_TYPE=`basename $TEST_DIR`
|
|
||||||
|
|
||||||
|
|
||||||
export SWARM_BIN=$TEST_DIR/../../cmd/swarm
|
|
||||||
export GETH=$SWARM_BIN/../../../geth
|
|
||||||
export NETWORKID=322$TEST_NAME
|
|
||||||
export TMPDIR=~/BZZ/test/$TEST_TYPE
|
|
||||||
export DATA_ROOT=$TMPDIR/$NETWORKID
|
|
||||||
# alias swarm='bash $SWARM_BIN/swarm.sh $DATA_ROOT $NETWORKID'
|
|
||||||
EXTRA_ARGS=$*
|
|
||||||
|
|
||||||
rm -rf $DATA_ROOT
|
|
||||||
|
|
||||||
wait=1
|
|
||||||
|
|
||||||
function swarm {
|
|
||||||
# echo bash $SWARM_BIN/swarm.sh $TMPDIR $NETWORKID $* $EXTRA_ARGS
|
|
||||||
bash $SWARM_BIN/swarm.sh $TMPDIR $NETWORKID $* $EXTRA_ARGS
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
function randomfile {
|
|
||||||
dd if=/dev/urandom of=/dev/stdout bs=1024 count=$1 2>/dev/null
|
|
||||||
}
|
|
||||||
|
|
@ -30,6 +30,7 @@ type Hive struct {
|
||||||
addr kademlia.Address
|
addr kademlia.Address
|
||||||
kad *kademlia.Kademlia
|
kad *kademlia.Kademlia
|
||||||
path string
|
path string
|
||||||
|
quit chan bool
|
||||||
toggle chan bool
|
toggle chan bool
|
||||||
more chan bool
|
more chan bool
|
||||||
|
|
||||||
|
|
@ -106,6 +107,7 @@ func (self *Hive) Addr() kademlia.Address {
|
||||||
func (self *Hive) Start(id discover.NodeID, listenAddr func() string, connectPeer func(string) error) (err error) {
|
func (self *Hive) Start(id discover.NodeID, listenAddr func() string, connectPeer func(string) error) (err error) {
|
||||||
self.toggle = make(chan bool)
|
self.toggle = make(chan bool)
|
||||||
self.more = make(chan bool)
|
self.more = make(chan bool)
|
||||||
|
self.quit = make(chan bool)
|
||||||
self.id = id
|
self.id = id
|
||||||
self.listenAddr = listenAddr
|
self.listenAddr = listenAddr
|
||||||
err = self.kad.Load(self.path, nil)
|
err = self.kad.Load(self.path, nil)
|
||||||
|
|
@ -145,11 +147,14 @@ func (self *Hive) Start(id discover.NodeID, listenAddr func() string, connectPee
|
||||||
} else {
|
} else {
|
||||||
glog.V(logger.Warn).Infof("[BZZ] KΛÐΞMLIΛ hive: no peer")
|
glog.V(logger.Warn).Infof("[BZZ] KΛÐΞMLIΛ hive: no peer")
|
||||||
}
|
}
|
||||||
self.toggle <- true
|
|
||||||
glog.V(logger.Detail).Infof("[BZZ] KΛÐΞMLIΛ hive: buzz kept alive")
|
glog.V(logger.Detail).Infof("[BZZ] KΛÐΞMLIΛ hive: buzz kept alive")
|
||||||
} else {
|
} else {
|
||||||
glog.V(logger.Info).Infof("[BZZ] KΛÐΞMLIΛ hive: no need for more bees")
|
glog.V(logger.Info).Infof("[BZZ] KΛÐΞMLIΛ hive: no need for more bees")
|
||||||
self.toggle <- false
|
}
|
||||||
|
select {
|
||||||
|
case self.toggle <- need:
|
||||||
|
case <-self.quit:
|
||||||
|
return
|
||||||
}
|
}
|
||||||
glog.V(logger.Debug).Infof("[BZZ] KΛÐΞMLIΛ hive: queen's address: %v, population: %d (%d)", self.addr, self.kad.Count(), self.kad.DBCount())
|
glog.V(logger.Debug).Infof("[BZZ] KΛÐΞMLIΛ hive: queen's address: %v, population: %d (%d)", self.addr, self.kad.Count(), self.kad.DBCount())
|
||||||
}
|
}
|
||||||
|
|
@ -174,11 +179,7 @@ func (self *Hive) keepAlive() {
|
||||||
default:
|
default:
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
case need, alive := <-self.toggle:
|
case need := <-self.toggle:
|
||||||
if !alive {
|
|
||||||
self.more <- false
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if alarm == nil && need {
|
if alarm == nil && need {
|
||||||
alarm = time.NewTicker(time.Duration(self.callInterval)).C
|
alarm = time.NewTicker(time.Duration(self.callInterval)).C
|
||||||
}
|
}
|
||||||
|
|
@ -186,13 +187,15 @@ func (self *Hive) keepAlive() {
|
||||||
alarm = nil
|
alarm = nil
|
||||||
|
|
||||||
}
|
}
|
||||||
|
case <-self.quit:
|
||||||
|
return
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (self *Hive) Stop() error {
|
func (self *Hive) Stop() error {
|
||||||
// closing toggle channel quits the updateloop
|
// closing toggle channel quits the updateloop
|
||||||
close(self.toggle)
|
close(self.quit)
|
||||||
return self.kad.Save(self.path, saveSync)
|
return self.kad.Save(self.path, saveSync)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -26,7 +26,6 @@ type NodeRecord struct {
|
||||||
Meta *json.RawMessage // arbitrary metadata saved for a peer
|
Meta *json.RawMessage // arbitrary metadata saved for a peer
|
||||||
|
|
||||||
node Node
|
node Node
|
||||||
connected bool
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (self *NodeRecord) setSeen() {
|
func (self *NodeRecord) setSeen() {
|
||||||
|
|
@ -45,7 +44,7 @@ type KadDb struct {
|
||||||
Nodes [][]*NodeRecord
|
Nodes [][]*NodeRecord
|
||||||
index map[Address]*NodeRecord
|
index map[Address]*NodeRecord
|
||||||
cursors []int
|
cursors []int
|
||||||
lock sync.Mutex
|
lock sync.RWMutex
|
||||||
purgeInterval time.Duration
|
purgeInterval time.Duration
|
||||||
initialRetryInterval time.Duration
|
initialRetryInterval time.Duration
|
||||||
connRetryExp int
|
connRetryExp int
|
||||||
|
|
@ -161,10 +160,10 @@ func (self *KadDb) findBest(maxBinSize int, binSize func(int) int) (node *NodeRe
|
||||||
|
|
||||||
var interval time.Duration
|
var interval time.Duration
|
||||||
var found bool
|
var found bool
|
||||||
var count int
|
|
||||||
var purge []bool
|
var purge []bool
|
||||||
var delta time.Duration
|
var delta time.Duration
|
||||||
var cursor int
|
var cursor int
|
||||||
|
var count int
|
||||||
var after time.Time
|
var after time.Time
|
||||||
|
|
||||||
// iterate over columns maximum bucketsize times
|
// iterate over columns maximum bucketsize times
|
||||||
|
|
@ -181,7 +180,6 @@ func (self *KadDb) findBest(maxBinSize int, binSize func(int) int) (node *NodeRe
|
||||||
proxLimit = po
|
proxLimit = po
|
||||||
need = true
|
need = true
|
||||||
}
|
}
|
||||||
cursor = self.cursors[po]
|
|
||||||
purge = make([]bool, len(dbrow))
|
purge = make([]bool, len(dbrow))
|
||||||
|
|
||||||
// there is a missing slot - finding a node to connect to
|
// there is a missing slot - finding a node to connect to
|
||||||
|
|
@ -192,7 +190,7 @@ func (self *KadDb) findBest(maxBinSize int, binSize func(int) int) (node *NodeRe
|
||||||
node = dbrow[cursor]
|
node = dbrow[cursor]
|
||||||
|
|
||||||
// skip already connected nodes
|
// skip already connected nodes
|
||||||
if node.connected {
|
if node.node != nil {
|
||||||
glog.V(logger.Debug).Infof("[KΛÐ]: kaddb record %v (PO%03d:%d/%d) already connected", node.Addr, po, cursor, len(dbrow))
|
glog.V(logger.Debug).Infof("[KΛÐ]: kaddb record %v (PO%03d:%d/%d) already connected", node.Addr, po, cursor, len(dbrow))
|
||||||
continue ROW
|
continue ROW
|
||||||
}
|
}
|
||||||
|
|
@ -223,7 +221,6 @@ func (self *KadDb) findBest(maxBinSize int, binSize func(int) int) (node *NodeRe
|
||||||
glog.V(logger.Debug).Infof("[KΛÐ]: kaddb record %v (PO%03d:%d) selected as candidate connection %v. seen at %v (%v ago), selectable since %v, retry after %v (in %v)", node.Addr, po, cursor, rounds, node.Seen, delta, node.After, after, interval)
|
glog.V(logger.Debug).Infof("[KΛÐ]: kaddb record %v (PO%03d:%d) selected as candidate connection %v. seen at %v (%v ago), selectable since %v, retry after %v (in %v)", node.Addr, po, cursor, rounds, node.Seen, delta, node.After, after, interval)
|
||||||
node.After = after
|
node.After = after
|
||||||
found = true
|
found = true
|
||||||
break ROW
|
|
||||||
} // ROW
|
} // ROW
|
||||||
self.cursors[po] = cursor
|
self.cursors[po] = cursor
|
||||||
self.delete(po, purge)
|
self.delete(po, purge)
|
||||||
|
|
|
||||||
|
|
@ -16,12 +16,13 @@ const (
|
||||||
proxBinSize = 4
|
proxBinSize = 4
|
||||||
maxProx = 8
|
maxProx = 8
|
||||||
connRetryExp = 2
|
connRetryExp = 2
|
||||||
|
maxPeers = 100
|
||||||
)
|
)
|
||||||
|
|
||||||
var (
|
var (
|
||||||
purgeInterval = 42 * time.Hour
|
purgeInterval = 42 * time.Hour
|
||||||
initialRetryInterval = 42 * 100 * time.Millisecond
|
initialRetryInterval = 42 * time.Millisecond
|
||||||
maxIdleInterval = 42 * 10 * time.Second
|
maxIdleInterval = 42 * 100 * time.Millisecond
|
||||||
)
|
)
|
||||||
|
|
||||||
type KadParams struct {
|
type KadParams struct {
|
||||||
|
|
@ -31,6 +32,7 @@ type KadParams struct {
|
||||||
BucketSize int
|
BucketSize int
|
||||||
PurgeInterval time.Duration
|
PurgeInterval time.Duration
|
||||||
InitialRetryInterval time.Duration
|
InitialRetryInterval time.Duration
|
||||||
|
MaxIdleInterval time.Duration
|
||||||
ConnRetryExp int
|
ConnRetryExp int
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -41,6 +43,7 @@ func NewKadParams() *KadParams {
|
||||||
BucketSize: bucketSize,
|
BucketSize: bucketSize,
|
||||||
PurgeInterval: purgeInterval,
|
PurgeInterval: purgeInterval,
|
||||||
InitialRetryInterval: initialRetryInterval,
|
InitialRetryInterval: initialRetryInterval,
|
||||||
|
MaxIdleInterval: maxIdleInterval,
|
||||||
ConnRetryExp: connRetryExp,
|
ConnRetryExp: connRetryExp,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -52,7 +55,7 @@ type Kademlia struct {
|
||||||
proxLimit int // state, the PO of the first row of the most proximate bin
|
proxLimit int // state, the PO of the first row of the most proximate bin
|
||||||
proxSize int // state, the number of peers in the most proximate bin
|
proxSize int // state, the number of peers in the most proximate bin
|
||||||
count int // number of active peers (w live connection)
|
count int // number of active peers (w live connection)
|
||||||
buckets []*bucket // the actual bins
|
buckets [][]Node // the actual bins
|
||||||
db *KadDb // kaddb, node record database
|
db *KadDb // kaddb, node record database
|
||||||
lock sync.RWMutex // mutex to access buckets
|
lock sync.RWMutex // mutex to access buckets
|
||||||
}
|
}
|
||||||
|
|
@ -68,14 +71,9 @@ type Node interface {
|
||||||
// add is the base address of the table
|
// add is the base address of the table
|
||||||
// params is KadParams configuration
|
// params is KadParams configuration
|
||||||
func New(addr Address, params *KadParams) *Kademlia {
|
func New(addr Address, params *KadParams) *Kademlia {
|
||||||
buckets := make([]*bucket, params.MaxProx+1)
|
buckets := make([][]Node, params.MaxProx+1)
|
||||||
for i, _ := range buckets {
|
glog.V(logger.Info).Infof("[KΛÐ] base address %v", addr.String()[:6])
|
||||||
buckets[i] = &bucket{size: params.BucketSize} // will initialise bucket{int(0),[]Node(nil),sync.Mutex}
|
|
||||||
}
|
|
||||||
glog.V(logger.Info).Infof("[KΛÐ] base address %v", addr)
|
|
||||||
|
|
||||||
// ! temporary hack fixme:
|
|
||||||
params.ProxBinSize = 4
|
|
||||||
return &Kademlia{
|
return &Kademlia{
|
||||||
addr: addr,
|
addr: addr,
|
||||||
KadParams: params,
|
KadParams: params,
|
||||||
|
|
@ -109,9 +107,6 @@ func (self *Kademlia) On(node Node, cb func(*NodeRecord, Node) error) (err error
|
||||||
|
|
||||||
index := self.proximityBin(node.Addr())
|
index := self.proximityBin(node.Addr())
|
||||||
record := self.db.findOrCreate(index, node.Addr(), node.Url())
|
record := self.db.findOrCreate(index, node.Addr(), node.Url())
|
||||||
// callback on add node
|
|
||||||
// setting the node on the record, set it checked (for connectivity)
|
|
||||||
record.node = node
|
|
||||||
|
|
||||||
if cb != nil {
|
if cb != nil {
|
||||||
err = cb(record, node)
|
err = cb(record, node)
|
||||||
|
|
@ -119,34 +114,46 @@ func (self *Kademlia) On(node Node, cb func(*NodeRecord, Node) error) (err error
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("unable to add node %v, callback error: %v", node.Addr(), err)
|
return fmt.Errorf("unable to add node %v, callback error: %v", node.Addr(), err)
|
||||||
}
|
}
|
||||||
glog.V(logger.Info).Infof("[KΛÐ]: add node record %v with node %v", record, node)
|
glog.V(logger.Debug).Infof("[KΛÐ]: add node record %v with node %v", record, node)
|
||||||
}
|
}
|
||||||
record.connected = true
|
|
||||||
|
|
||||||
// insert in kademlia table of active nodes
|
// insert in kademlia table of active nodes
|
||||||
bucket := self.buckets[index]
|
bucket := self.buckets[index]
|
||||||
// if bucket is full insertion replaces the worst node
|
// if bucket is full insertion replaces the worst node
|
||||||
// TODO: give priority to peers with active traffic
|
// TODO: give priority to peers with active traffic
|
||||||
replaced, err := bucket.insert(node)
|
if len(bucket) >= self.BucketSize { // >= allows us to add peers beyond the bucketsize limitation
|
||||||
if err != nil {
|
// always rotate peers
|
||||||
glog.V(logger.Debug).Infof("[KΛÐ]: node %v not needed: %v", node, err)
|
idle := self.MaxIdleInterval
|
||||||
return err
|
var pos int
|
||||||
// no prox adjustment needed
|
var replaced Node
|
||||||
// do not change count
|
for i, p := range bucket {
|
||||||
|
idleInt := time.Since(p.LastActive())
|
||||||
|
if idleInt > idle {
|
||||||
|
idle = idleInt
|
||||||
|
pos = i
|
||||||
|
replaced = p
|
||||||
}
|
}
|
||||||
if replaced != nil {
|
}
|
||||||
glog.V(logger.Debug).Infof("[KΛÐ]: node %v replaced by %v ", replaced, node)
|
if replaced == nil {
|
||||||
|
glog.V(logger.Debug).Infof("[KΛÐ]: all peers wanted, PO%03d bucket full", index)
|
||||||
|
return fmt.Errorf("bucket full")
|
||||||
|
}
|
||||||
|
glog.V(logger.Debug).Infof("[KΛÐ]: node %v replaced by %v (idle for %v > %v)", replaced, node, idle, self.MaxIdleInterval)
|
||||||
replaced.Drop()
|
replaced.Drop()
|
||||||
|
self.buckets[index] = append(bucket[:pos], bucket[(pos+1):]...)
|
||||||
|
// there is no change in bucket cardinalities so no prox limit adjustment is needed
|
||||||
return nil
|
return nil
|
||||||
}
|
} else {
|
||||||
// new node added
|
self.buckets[index] = append(bucket, node)
|
||||||
glog.V(logger.Info).Infof("[KΛÐ]: add node %v to table", node)
|
glog.V(logger.Debug).Infof("[KΛÐ]: add node %v to table", node)
|
||||||
self.count++
|
self.count++
|
||||||
self.setProxLimit(index, false)
|
self.setProxLimit(index, true)
|
||||||
|
}
|
||||||
|
record.node = node
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// is the entrypoint called when a node is taken offline
|
// Off is the called when a node is taken offline (from the protocol main loop exit)
|
||||||
func (self *Kademlia) Off(node Node, cb func(*NodeRecord, Node)) (err error) {
|
func (self *Kademlia) Off(node Node, cb func(*NodeRecord, Node)) (err error) {
|
||||||
self.lock.Lock()
|
self.lock.Lock()
|
||||||
defer self.lock.Unlock()
|
defer self.lock.Unlock()
|
||||||
|
|
@ -154,68 +161,70 @@ func (self *Kademlia) Off(node Node, cb func(*NodeRecord, Node)) (err error) {
|
||||||
var found bool
|
var found bool
|
||||||
index := self.proximityBin(node.Addr())
|
index := self.proximityBin(node.Addr())
|
||||||
bucket := self.buckets[index]
|
bucket := self.buckets[index]
|
||||||
for i := 0; i < len(bucket.nodes); i++ {
|
for i := 0; i < len(bucket); i++ {
|
||||||
if node.Addr() == bucket.nodes[i].Addr() {
|
if node.Addr() == bucket[i].Addr() {
|
||||||
found = true
|
found = true
|
||||||
bucket.nodes = append(bucket.nodes[:i], bucket.nodes[(i+1):]...)
|
self.buckets[index] = append(bucket[:i], bucket[(i+1):]...)
|
||||||
|
break
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if !found {
|
if !found {
|
||||||
// gracefully return without error if peer already offline
|
// gracefully return without error if peer already unregistered
|
||||||
|
glog.V(logger.Warn).Infof("[KΛÐ]: remove node %v not in table, population now is %v", node, self.count)
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
glog.V(logger.Info).Infof("[KΛÐ]: remove node %v from table", node)
|
|
||||||
|
|
||||||
self.count--
|
self.count--
|
||||||
if len(bucket.nodes) < bucket.size {
|
glog.V(logger.Debug).Infof("[KΛÐ]: remove node %v from table, population now is %v", node, self.count)
|
||||||
err = fmt.Errorf("insufficient nodes (%v) in bucket %v", len(bucket.nodes), index)
|
self.setProxLimit(index, false)
|
||||||
}
|
|
||||||
|
|
||||||
self.setProxLimit(index, true)
|
record := self.db.index[node.Addr()]
|
||||||
|
|
||||||
r := self.db.index[node.Addr()]
|
|
||||||
// callback on remove
|
// callback on remove
|
||||||
if cb != nil {
|
if cb != nil {
|
||||||
cb(r, r.node)
|
cb(record, record.node)
|
||||||
}
|
}
|
||||||
r.node = nil
|
record.node = nil
|
||||||
r.connected = false
|
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// proxLimit is dynamically adjusted so that
|
// proxLimit is dynamically adjusted so that
|
||||||
// 1) there is no empty buckets in bin < proxLimit and
|
// 1) there is no empty buckets in bin < proxLimit and
|
||||||
// 2) the sum of all items sare the maximpossible but lower than ProxBinSize
|
// 2) the sum of all items are the minimum possible but higher than ProxBinSize
|
||||||
// adjust Prox (proxLimit and proxSize after an insertion/removal of nodes)
|
// adjust Prox (proxLimit and proxSize after an insertion/removal of nodes)
|
||||||
// caller holds the lock
|
// caller holds the lock
|
||||||
func (self *Kademlia) setProxLimit(r int, off bool) {
|
func (self *Kademlia) setProxLimit(r int, on bool) {
|
||||||
// glog.V(logger.Info).Infof("[KΛÐ]: adjust proxbin for (bin: %v, off: %v)", r, off)
|
// if the change is outside the core (PO lower)
|
||||||
if r < self.proxLimit && len(self.buckets[r].nodes) > 0 {
|
// and the change does not leave a bucket empty then
|
||||||
|
// no adjustment needed
|
||||||
|
if r < self.proxLimit && len(self.buckets[r]) > 0 {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
glog.V(logger.Detail).Infof("[KΛÐ]: set proxbin (size: %v, limit: %v, bin: %v, off: %v)", self.proxSize, self.proxLimit, r, off)
|
// if on=a node was added, then r must be within prox limit so increment cardinality
|
||||||
if off {
|
if on {
|
||||||
|
self.proxSize++
|
||||||
|
curr := len(self.buckets[self.proxLimit])
|
||||||
|
// if now core is big enough without the furthest bucket, then contract
|
||||||
|
// this can never result in more than one bucket change
|
||||||
|
if self.proxSize >= self.ProxBinSize+curr && curr > 0 {
|
||||||
|
self.proxSize -= curr
|
||||||
|
self.proxLimit++
|
||||||
|
glog.V(logger.Detail).Infof("[KΛÐ]: proxbin contraction (size: %v, limit: %v, bin: %v)", self.proxSize, self.proxLimit, r)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// otherwise
|
||||||
|
if r >= self.proxLimit {
|
||||||
self.proxSize--
|
self.proxSize--
|
||||||
|
}
|
||||||
|
// expand core by lowering prox limit until hit zero or cover the empty bucket or reached target cardinality
|
||||||
for (self.proxSize < self.ProxBinSize || r < self.proxLimit) &&
|
for (self.proxSize < self.ProxBinSize || r < self.proxLimit) &&
|
||||||
self.proxLimit > 0 {
|
self.proxLimit > 0 {
|
||||||
//
|
//
|
||||||
self.proxLimit--
|
self.proxLimit--
|
||||||
self.proxSize += len(self.buckets[self.proxLimit].nodes)
|
self.proxSize += len(self.buckets[self.proxLimit])
|
||||||
glog.V(logger.Detail).Infof("[KΛÐ]: proxbin expansion (size: %v, limit: %v, bin: %v, off: %v)", self.proxSize, self.proxLimit, r, off)
|
glog.V(logger.Detail).Infof("[KΛÐ]: proxbin expansion (size: %v, limit: %v, bin: %v)", self.proxSize, self.proxLimit, r)
|
||||||
}
|
|
||||||
// glog.V(logger.Detail).Infof("%v", self)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
self.proxSize++
|
|
||||||
for self.proxLimit < self.MaxProx &&
|
|
||||||
len(self.buckets[self.proxLimit].nodes) > 0 &&
|
|
||||||
self.proxSize-len(self.buckets[self.proxLimit].nodes) >= self.ProxBinSize {
|
|
||||||
//
|
|
||||||
self.proxSize -= len(self.buckets[self.proxLimit].nodes)
|
|
||||||
self.proxLimit++
|
|
||||||
glog.V(logger.Detail).Infof("[KΛÐ]: proxbin contraction (size: %v, limit: %v, bin: %v, off: %v)", self.proxSize, self.proxLimit, r, off)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -227,60 +236,55 @@ proxLimit and MaxProx.
|
||||||
func (self *Kademlia) FindClosest(target Address, max int) []Node {
|
func (self *Kademlia) FindClosest(target Address, max int) []Node {
|
||||||
defer self.lock.RUnlock()
|
defer self.lock.RUnlock()
|
||||||
self.lock.RLock()
|
self.lock.RLock()
|
||||||
|
|
||||||
r := nodesByDistance{
|
r := nodesByDistance{
|
||||||
target: target,
|
target: target,
|
||||||
}
|
}
|
||||||
index := self.proximityBin(target)
|
|
||||||
|
|
||||||
start := index
|
po := self.proximityBin(target)
|
||||||
var down bool
|
index := po
|
||||||
if index >= self.proxLimit {
|
step := 1
|
||||||
|
|
||||||
|
// set proxbin iteration full
|
||||||
|
if index > self.proxLimit {
|
||||||
index = self.proxLimit
|
index = self.proxLimit
|
||||||
start = self.MaxProx
|
|
||||||
down = true
|
|
||||||
}
|
}
|
||||||
var n int
|
|
||||||
|
// if max is set to 0, just want a full bucket, dynamic number
|
||||||
|
min := max
|
||||||
|
// set limit to max
|
||||||
limit := max
|
limit := max
|
||||||
if max == 0 {
|
if max == 0 {
|
||||||
limit = 1000
|
min = 1
|
||||||
|
limit = maxPeers
|
||||||
}
|
}
|
||||||
for {
|
|
||||||
|
|
||||||
bucket := self.buckets[start].nodes
|
var n int
|
||||||
for i := 0; i < len(bucket); i++ {
|
for index >= 0 {
|
||||||
r.push(bucket[i], limit)
|
// add entire bucket
|
||||||
|
for _, p := range self.buckets[index] {
|
||||||
|
r.push(p, limit)
|
||||||
n++
|
n++
|
||||||
}
|
}
|
||||||
if max == 0 && start <= index && (n > 0 || start == 0) || max > 0 && down && start <= index && (n >= limit || n == self.count || start == 0) {
|
// terminate if index reached the bottom or enough peers > min
|
||||||
|
if n >= min && (step < 0 || max == 0) {
|
||||||
break
|
break
|
||||||
}
|
}
|
||||||
if down {
|
// reach top most non-empty PO bucket, turn around
|
||||||
start--
|
if index == self.MaxProx {
|
||||||
} else {
|
index = po
|
||||||
if start == self.MaxProx {
|
step = -1
|
||||||
if index == 0 {
|
|
||||||
break
|
|
||||||
}
|
}
|
||||||
start = index - 1
|
index += step
|
||||||
down = true
|
|
||||||
} else {
|
|
||||||
start++
|
|
||||||
}
|
}
|
||||||
}
|
glog.V(logger.Detail).Infof("[KΛÐ]: serve %d (<=%d) nodes for target lookup %v (PO%03d)", n, max, target, po)
|
||||||
}
|
|
||||||
glog.V(logger.Detail).Infof("[KΛÐ]: serve %d (=<%d) nodes for target lookup %v (PO%d)", n, self.MaxProx, target, index)
|
|
||||||
return r.nodes
|
return r.nodes
|
||||||
}
|
}
|
||||||
|
|
||||||
func (self *Kademlia) binsize(p int) int {
|
|
||||||
b := self.buckets[p]
|
|
||||||
defer b.lock.RUnlock()
|
|
||||||
b.lock.RLock()
|
|
||||||
return len(b.nodes)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (self *Kademlia) Suggest() (*NodeRecord, bool, int) {
|
func (self *Kademlia) Suggest() (*NodeRecord, bool, int) {
|
||||||
return self.db.findBest(self.BucketSize, self.binsize)
|
defer self.lock.RUnlock()
|
||||||
|
self.lock.RLock()
|
||||||
|
return self.db.findBest(self.BucketSize, func(i int) int { return len(self.buckets[i]) })
|
||||||
}
|
}
|
||||||
|
|
||||||
// adds node records to kaddb (persisted node record db)
|
// adds node records to kaddb (persisted node record db)
|
||||||
|
|
@ -288,13 +292,6 @@ func (self *Kademlia) Add(nrs []*NodeRecord) {
|
||||||
self.db.add(nrs, self.proximityBin)
|
self.db.add(nrs, self.proximityBin)
|
||||||
}
|
}
|
||||||
|
|
||||||
// in situ mutable bucket
|
|
||||||
type bucket struct {
|
|
||||||
size int
|
|
||||||
nodes []Node
|
|
||||||
lock sync.RWMutex
|
|
||||||
}
|
|
||||||
|
|
||||||
// nodesByDistance is a list of nodes, ordered by distance to target.
|
// nodesByDistance is a list of nodes, ordered by distance to target.
|
||||||
type nodesByDistance struct {
|
type nodesByDistance struct {
|
||||||
nodes []Node
|
nodes []Node
|
||||||
|
|
@ -331,31 +328,6 @@ func (h *nodesByDistance) push(node Node, max int) {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// insert adds a peer to a bucket either by appending to existing items if
|
|
||||||
// bucket length does not exceed bucketSize, or by replacing the worst
|
|
||||||
// Node in the bucket
|
|
||||||
func (self *bucket) insert(node Node) (replaced Node, err error) {
|
|
||||||
self.lock.Lock()
|
|
||||||
defer self.lock.Unlock()
|
|
||||||
if len(self.nodes) >= self.size { // >= allows us to add peers beyond the bucketsize limitation
|
|
||||||
// dev p2p kicks out nodes idle for > 30 s, so here we never replace nodes if
|
|
||||||
// bucket is full
|
|
||||||
// update, it seems we need to replace nodes
|
|
||||||
// return nil, fmt.Errorf("bucket full")
|
|
||||||
replaced := self.nodes[0]
|
|
||||||
self.nodes = append(self.nodes[1:], node)
|
|
||||||
return replaced, nil
|
|
||||||
}
|
|
||||||
self.nodes = append(self.nodes, node)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
func (self *bucket) length(node Node) int {
|
|
||||||
self.lock.Lock()
|
|
||||||
defer self.lock.Unlock()
|
|
||||||
return len(self.nodes)
|
|
||||||
}
|
|
||||||
|
|
||||||
/*
|
/*
|
||||||
Taking the proximity order relative to a fix point x classifies the points in
|
Taking the proximity order relative to a fix point x classifies the points in
|
||||||
the space (n byte long byte sequences) into bins. Items in each are at
|
the space (n byte long byte sequences) into bins. Items in each are at
|
||||||
|
|
@ -400,43 +372,46 @@ func (self *Kademlia) Load(path string, cb func(*NodeRecord, Node) error) (err e
|
||||||
}
|
}
|
||||||
|
|
||||||
// kademlia table + kaddb table displayed with ascii
|
// kademlia table + kaddb table displayed with ascii
|
||||||
// callerholds the lock
|
|
||||||
func (self *Kademlia) String() string {
|
func (self *Kademlia) String() string {
|
||||||
|
defer self.lock.RUnlock()
|
||||||
|
self.lock.RLock()
|
||||||
|
defer self.db.lock.RUnlock()
|
||||||
|
self.db.lock.RLock()
|
||||||
|
|
||||||
var rows []string
|
var rows []string
|
||||||
rows = append(rows, "=========================================================================")
|
rows = append(rows, "=========================================================================")
|
||||||
rows = append(rows, fmt.Sprintf("KΛÐΞMLIΛ hive: queen's address: %v, population: %d (%d)", self.addr, self.count, self.DBCount()))
|
rows = append(rows, fmt.Sprintf("%v KΛÐΞMLIΛ hive: queen's address: %v", time.Now().UTC().Format(time.UnixDate), self.addr.String()[:6]))
|
||||||
rows = append(rows, fmt.Sprintf("%v : MaxProx: %d, ProxBinSize: %d, BucketSize: %d, proxLimit: %d, proxSize: %d", time.Now(), self.MaxProx, self.ProxBinSize, self.BucketSize, self.proxLimit, self.proxSize))
|
rows = append(rows, fmt.Sprintf("population: %d (%d), proxLimit: %d, proxSize: %d", self.count, len(self.db.index), self.proxLimit, self.proxSize))
|
||||||
|
rows = append(rows, fmt.Sprintf("MaxProx: %d, ProxBinSize: %d, BucketSize: %d", self.MaxProx, self.ProxBinSize, self.BucketSize))
|
||||||
|
|
||||||
for i, b := range self.buckets {
|
for i, bucket := range self.buckets {
|
||||||
|
|
||||||
if i == self.proxLimit {
|
if i == self.proxLimit {
|
||||||
rows = append(rows, fmt.Sprintf("===================== PROX LIMIT: %d =================================", i))
|
rows = append(rows, fmt.Sprintf("============ PROX LIMIT: %d ==========================================", i))
|
||||||
}
|
}
|
||||||
row := []string{fmt.Sprintf("%03d", i), fmt.Sprintf("%2d", len(b.nodes))}
|
row := []string{fmt.Sprintf("%03d", i), fmt.Sprintf("%2d", len(bucket))}
|
||||||
var k int
|
var k int
|
||||||
c := self.db.cursors[i]
|
c := self.db.cursors[i]
|
||||||
for ; k < len(b.nodes); k++ {
|
for ; k < len(bucket); k++ {
|
||||||
p := b.nodes[(c+k)%len(b.nodes)]
|
p := bucket[(c+k)%len(bucket)]
|
||||||
row = append(row, fmt.Sprintf("%s", p.Addr().String()[:8]))
|
row = append(row, p.Addr().String()[:6])
|
||||||
if k == 3 {
|
if k == 4 {
|
||||||
break
|
break
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
for ; k < 3; k++ {
|
for ; k < 4; k++ {
|
||||||
row = append(row, " ")
|
row = append(row, " ")
|
||||||
}
|
}
|
||||||
row = append(row, fmt.Sprintf("| %2d %2d", len(self.db.Nodes[i]), self.db.cursors[i]))
|
row = append(row, fmt.Sprintf("| %2d %2d", len(self.db.Nodes[i]), self.db.cursors[i]))
|
||||||
|
|
||||||
for j, p := range self.db.Nodes[i] {
|
for j, p := range self.db.Nodes[i] {
|
||||||
row = append(row, fmt.Sprintf("%08x", p.Addr[:4]))
|
row = append(row, p.Addr.String()[:6])
|
||||||
if j == 2 {
|
if j == 3 {
|
||||||
break
|
break
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
rows = append(rows, strings.Join(row, " "))
|
rows = append(rows, strings.Join(row, " "))
|
||||||
if i == self.MaxProx {
|
if i == self.MaxProx {
|
||||||
break
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
rows = append(rows, "=========================================================================")
|
rows = append(rows, "=========================================================================")
|
||||||
|
|
|
||||||
|
|
@ -78,7 +78,7 @@ func TestBootstrap(t *testing.T) {
|
||||||
n := 0
|
n := 0
|
||||||
for n < 100 {
|
for n < 100 {
|
||||||
err = kad.On(node, nil)
|
err = kad.On(node, nil)
|
||||||
if err != nil && err.Error() != "bucket full" {
|
if err != nil {
|
||||||
t.Fatalf("backend not accepting node: %v", err)
|
t.Fatalf("backend not accepting node: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -150,7 +150,7 @@ func TestFindClosest(t *testing.T) {
|
||||||
// check that the result nodes have minimum distance to target.
|
// check that the result nodes have minimum distance to target.
|
||||||
farthestResult := nodes[len(nodes)-1].Addr()
|
farthestResult := nodes[len(nodes)-1].Addr()
|
||||||
for i, b := range kad.buckets {
|
for i, b := range kad.buckets {
|
||||||
for j, n := range b.nodes {
|
for j, n := range b {
|
||||||
if contains(nodes, n.Addr()) {
|
if contains(nodes, n.Addr()) {
|
||||||
continue // don't run the check below for nodes in result
|
continue // don't run the check below for nodes in result
|
||||||
}
|
}
|
||||||
|
|
@ -235,12 +235,12 @@ func TestSaveLoad(t *testing.T) {
|
||||||
nodes := kad.FindClosest(self, 100)
|
nodes := kad.FindClosest(self, 100)
|
||||||
path := "/tmp/bzz.peers"
|
path := "/tmp/bzz.peers"
|
||||||
err = kad.Save(path, nil)
|
err = kad.Save(path, nil)
|
||||||
if err != nil {
|
if err != nil && err.Error() != "bucket full" {
|
||||||
t.Fatalf("unepected error saving kaddb: %v", err)
|
t.Fatalf("unepected error saving kaddb: %v", err)
|
||||||
}
|
}
|
||||||
kad = New(self, params)
|
kad = New(self, params)
|
||||||
err = kad.Load(path, nil)
|
err = kad.Load(path, nil)
|
||||||
if err != nil {
|
if err != nil && err.Error() != "bucket full" {
|
||||||
t.Fatalf("unepected error loading kaddb: %v", err)
|
t.Fatalf("unepected error loading kaddb: %v", err)
|
||||||
}
|
}
|
||||||
for _, b := range kad.db.Nodes {
|
for _, b := range kad.db.Nodes {
|
||||||
|
|
@ -260,30 +260,30 @@ func TestSaveLoad(t *testing.T) {
|
||||||
}
|
}
|
||||||
|
|
||||||
func (self *Kademlia) proxCheck(t *testing.T) bool {
|
func (self *Kademlia) proxCheck(t *testing.T) bool {
|
||||||
var sum, i int
|
var sum int
|
||||||
var b *bucket
|
for i, b := range self.buckets {
|
||||||
for i, b = range self.buckets {
|
l := len(b)
|
||||||
l := len(b.nodes)
|
|
||||||
// if we are in the high prox multibucket
|
// if we are in the high prox multibucket
|
||||||
if i >= self.proxLimit {
|
if i >= self.proxLimit {
|
||||||
sum += l
|
sum += l
|
||||||
} else if l == 0 {
|
} else if l == 0 {
|
||||||
t.Errorf("bucket %d empty, yet proxLimit is %d\n%v", len(b.nodes), self.proxLimit, self)
|
t.Errorf("bucket %d empty, yet proxLimit is %d\n%v", len(b), self.proxLimit, self)
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
// check if merged high prox bucket does not exceed size
|
// check if merged high prox bucket does not exceed size
|
||||||
if sum > 0 {
|
if sum > 0 {
|
||||||
// if sum > self.ProxBinSize {
|
|
||||||
// t.Errorf("bucket %d is empty, yet proxSize is %d\n%v", i, self.proxSize, self)
|
|
||||||
// return false
|
|
||||||
// }
|
|
||||||
if sum != self.proxSize {
|
if sum != self.proxSize {
|
||||||
t.Errorf("proxSize incorrect, expected %v, got %v", sum, self.proxSize)
|
t.Errorf("proxSize incorrect, expected %v, got %v", sum, self.proxSize)
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
if self.proxLimit > 0 && sum+len(self.buckets[self.proxLimit-1].nodes) < self.ProxBinSize {
|
last := len(self.buckets[self.proxLimit])
|
||||||
t.Errorf("proxBinSize incorrect, expected %v got %v", sum, self.proxSize)
|
if last > 0 && sum >= self.ProxBinSize+last {
|
||||||
|
t.Errorf("proxLimit %v incorrect, redundant non-empty bucket %d added to proxBin with %v (target %v)", self.proxLimit, last, sum-last, self.ProxBinSize)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if self.proxLimit > 0 && sum < self.ProxBinSize {
|
||||||
|
t.Errorf("proxLimit %v incorrect. proxSize %v is less than target %v, yet there is more peers", self.proxLimit, sum, self.ProxBinSize)
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -181,7 +181,8 @@ func run(requestDb *storage.LDBDatabase, depo StorageHandler, backend bind.Backe
|
||||||
// the main forever loop that handles incoming requests
|
// the main forever loop that handles incoming requests
|
||||||
for {
|
for {
|
||||||
if self.hive.blockRead {
|
if self.hive.blockRead {
|
||||||
time.Sleep(1 * time.Second)
|
glog.V(logger.Warn).Infof("[BZZ] Cannot read network")
|
||||||
|
time.Sleep(100 * time.Millisecond)
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
err = self.handle()
|
err = self.handle()
|
||||||
|
|
@ -225,7 +226,7 @@ func (self *bzz) handle() error {
|
||||||
if err := msg.Decode(&req); err != nil {
|
if err := msg.Decode(&req); err != nil {
|
||||||
return self.protoError(ErrDecode, "<- %v: %v", msg, err)
|
return self.protoError(ErrDecode, "<- %v: %v", msg, err)
|
||||||
}
|
}
|
||||||
glog.V(logger.Debug).Infof("[BZZ] incoming store request: %s", req.String())
|
glog.V(logger.Detail).Infof("[BZZ] incoming store request: %s", req.String())
|
||||||
// swap accounting is done within forwarding
|
// swap accounting is done within forwarding
|
||||||
self.storage.HandleStoreRequestMsg(&req, &peer{bzz: self})
|
self.storage.HandleStoreRequestMsg(&req, &peer{bzz: self})
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -464,11 +464,11 @@ LOOP:
|
||||||
// send the unsynced keys
|
// send the unsynced keys
|
||||||
stateCopy := *state
|
stateCopy := *state
|
||||||
err := self.unsyncedKeys(unsynced, &stateCopy)
|
err := self.unsyncedKeys(unsynced, &stateCopy)
|
||||||
self.state = state
|
|
||||||
glog.V(logger.Debug).Infof("[BZZ] syncer[%v]: --> %v keys sent: (total: %v (%v), history: %v), sent sync state: %v", self.key.Log(), len(unsynced), keyCounts, keyCount, historyCnt, stateCopy)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
glog.V(logger.Warn).Infof("[BZZ] syncer[%v]: unable to send unsynced keys: %v", err)
|
glog.V(logger.Warn).Infof("[BZZ] syncer[%v]: unable to send unsynced keys: %v", err)
|
||||||
}
|
}
|
||||||
|
self.state = state
|
||||||
|
glog.V(logger.Debug).Infof("[BZZ] syncer[%v]: --> %v keys sent: (total: %v (%v), history: %v), sent sync state: %v", self.key.Log(), len(unsynced), keyCounts, keyCount, historyCnt, stateCopy)
|
||||||
unsynced = nil
|
unsynced = nil
|
||||||
keys = nil
|
keys = nil
|
||||||
}
|
}
|
||||||
|
|
@ -585,7 +585,7 @@ func (self *syncer) syncDeliveries() {
|
||||||
total++
|
total++
|
||||||
msg, err = self.newStoreRequestMsgData(req)
|
msg, err = self.newStoreRequestMsgData(req)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
glog.V(logger.Warn).Infof("[BZZ] syncer[%v]: failed to deliver %v: %v", self.key.Log(), req, err)
|
glog.V(logger.Warn).Infof("[BZZ] syncer[%v]: failed to create store request for %v: %v", self.key.Log(), req, err)
|
||||||
} else {
|
} else {
|
||||||
err = self.store(msg)
|
err = self.store(msg)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|
|
||||||
|
|
@ -1,33 +1,25 @@
|
||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
|
|
||||||
dir=`dirname $0`
|
|
||||||
source $dir/../../cmd/swarm/test.sh
|
|
||||||
|
|
||||||
swarm init 4
|
swarm init 4
|
||||||
echo "expect each node to have 3 peers"
|
echo "expect each node to have 3 peers"
|
||||||
cmd="'net.peerCount'"
|
cmd="net.peerCount"
|
||||||
sleep 5
|
sleep 5
|
||||||
swarm attach 00 --exec "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
swarm execute 00 "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
||||||
swarm attach 01 --exec "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
swarm execute 01 "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
||||||
swarm attach 02 --exec "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
swarm execute 02 "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
||||||
swarm attach 03 --exec "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
swarm execute 03 "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
||||||
|
|
||||||
swarm stop all
|
swarm stop all
|
||||||
|
|
||||||
echo "after static nodes is deleted, connections are recovered from kaddb in bzz-peers.json"
|
echo "connections are recovered from kaddb in bzz-peers.json"
|
||||||
# echo rm -rf $DATA_ROOT/enodes\*
|
|
||||||
# echo rm -rf $DATA_ROOT/data/\*/static-nodes.json
|
|
||||||
rm -rf $DATA_ROOT/enodes*
|
|
||||||
rm -rf $DATA_ROOT/data/*/static-nodes.json
|
|
||||||
|
|
||||||
swarm cluster 4
|
swarm cluster 4
|
||||||
echo "expect each node to have 3 peers"
|
echo "expect each node to have 3 peers"
|
||||||
cmd="'net.peerCount'"
|
sleep 5
|
||||||
sleep 10
|
swarm execute 00 "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
||||||
swarm attach 00 --exec "$cmd" |tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
swarm execute 01 "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
||||||
swarm attach 01 --exec "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
swarm execute 02 "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
||||||
swarm attach 02 --exec "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
swarm execute 03 "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
||||||
swarm attach 03 --exec "$cmd"|tail -n1|grep -ql 3&& echo "PASS"||echo "FAIL"
|
|
||||||
|
|
||||||
swarm stop all
|
swarm stop all
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@ echo " two nodes that do not sync and do not have any funds"
|
||||||
echo " cannot retrieve content from each other"
|
echo " cannot retrieve content from each other"
|
||||||
|
|
||||||
dir=`dirname $0`
|
dir=`dirname $0`
|
||||||
source $dir/../../cmd/swarm/test.sh
|
source $dir/../test.sh
|
||||||
|
|
||||||
FILE_00=/tmp/1K.0
|
FILE_00=/tmp/1K.0
|
||||||
randomfile 1 > $FILE_00
|
randomfile 1 > $FILE_00
|
||||||
|
|
|
||||||
|
|
@ -3,19 +3,20 @@ echo " two nodes that do not sync but have enough funds"
|
||||||
echo " can retrieve content from each other"
|
echo " can retrieve content from each other"
|
||||||
|
|
||||||
dir=`dirname $0`
|
dir=`dirname $0`
|
||||||
source $dir/../../cmd/swarm/test.sh
|
source $dir/..s/test.sh
|
||||||
|
|
||||||
file=/tmp/test.file
|
file=/tmp/test.file
|
||||||
mininginterval=120
|
mininginterval=120
|
||||||
key=/tmp/key
|
key=/tmp/key
|
||||||
logargs="--verbosity=0 --vmodule='swarm/*=6'"
|
# logargs="--verbosity=0 --vmodule='swarm/*=6'"
|
||||||
# logargs='--verbosity=6'
|
logargs='--verbosity=6'
|
||||||
|
|
||||||
|
# swarm init 2 --mine --bzznosync --bzznoswap=false $logargsc
|
||||||
|
# echo "Mining some ether..."
|
||||||
|
# sleep $mininginterval
|
||||||
|
|
||||||
swarm init 2 --mine --bzznosync $logargs
|
swarm cluster 2 --mine --bzznosync --bzznoswap=false $logargsc
|
||||||
|
|
||||||
echo "Mining some ether..."
|
|
||||||
sleep $mininginterval
|
|
||||||
|
|
||||||
randomfile 10 > $file
|
randomfile 10 > $file
|
||||||
swarm up 00 $file|tail -n1 > $key
|
swarm up 00 $file|tail -n1 > $key
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@ echo " two nodes that sync (no swap and do not have any funds)"
|
||||||
echo " can be in sync content with each other"
|
echo " can be in sync content with each other"
|
||||||
|
|
||||||
dir=`dirname $0`
|
dir=`dirname $0`
|
||||||
source $dir/../../cmd/swarm/test.sh
|
source $dir/../test.sh
|
||||||
|
|
||||||
mkdir -p /tmp/swarm-test-files
|
mkdir -p /tmp/swarm-test-files
|
||||||
FILE_00=/tmp/swarm-test-files/00
|
FILE_00=/tmp/swarm-test-files/00
|
||||||
|
|
@ -28,7 +28,6 @@ swarm needs 00 $key $FILE_00
|
||||||
swarm needs 01 $key $FILE_00
|
swarm needs 01 $key $FILE_00
|
||||||
swarm stop 01
|
swarm stop 01
|
||||||
|
|
||||||
# exit 1;
|
|
||||||
|
|
||||||
swarm up 00 $FILE_01|tail -n1 > $key
|
swarm up 00 $FILE_01|tail -n1 > $key
|
||||||
swarm needs 00 $key $FILE_01
|
swarm needs 00 $key $FILE_01
|
||||||
|
|
@ -46,7 +45,8 @@ swarm needs 00 $key $FILE_03
|
||||||
swarm stop 00
|
swarm stop 00
|
||||||
swarm up 01 $FILE_04|tail -n1 > $key
|
swarm up 01 $FILE_04|tail -n1 > $key
|
||||||
swarm needs 01 $key $FILE_04
|
swarm needs 01 $key $FILE_04
|
||||||
swarm start 00 #--bzznoswap
|
swarm start 00
|
||||||
|
sleep $wait
|
||||||
swarm needs 00 $key $FILE_04
|
swarm needs 00 $key $FILE_04
|
||||||
|
|
||||||
swarm stop all
|
swarm stop all
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@ echo " two nodes that do not have any funds"
|
||||||
echo " can still sync content with each other"
|
echo " can still sync content with each other"
|
||||||
|
|
||||||
dir=`dirname $0`
|
dir=`dirname $0`
|
||||||
source $dir/../../cmd/swarm/test.sh
|
source $dir/../test.sh
|
||||||
key=/tmp/key
|
key=/tmp/key
|
||||||
|
|
||||||
long=/tmp/10M
|
long=/tmp/10M
|
||||||
|
|
|
||||||
|
|
@ -5,26 +5,26 @@ echo " two nodes that sync (no swap and do not have any funds)"
|
||||||
echo " can sync content with each other even with intermittent network connection"
|
echo " can sync content with each other even with intermittent network connection"
|
||||||
|
|
||||||
dir=`dirname $0`
|
dir=`dirname $0`
|
||||||
source $dir/../../cmd/swarm/test.sh
|
source $dir/../test.sh
|
||||||
|
|
||||||
long=/tmp/10M
|
long=/tmp/10M
|
||||||
key=/tmp/key
|
key=/tmp/key
|
||||||
randomfile 10000 > $long
|
randomfile 100000 > $long
|
||||||
ls -l $long
|
ls -l $long
|
||||||
|
|
||||||
swarm init 2
|
swarm init 2 --vmodule='swarm/*=5'
|
||||||
sleep $wait
|
|
||||||
swarm up 00 $long |tail -n1 > $key &
|
swarm up 00 $long |tail -n1 > $key &
|
||||||
sleep $wait
|
sleep 1
|
||||||
swarm attach 01 -exec "'bzz.blockNetworkRead(true)'"
|
swarm execute 01 'bzz.blockNetworkRead(true)'
|
||||||
sleep $wait
|
sleep 3
|
||||||
swarm attach 01 -exec "'bzz.blockNetworkRead(false)'"
|
swarm execute 01 'bzz.blockNetworkRead(false)'
|
||||||
sleep $wait
|
# sleep $wait
|
||||||
swarm attach 01 -exec "'bzz.blockNetworkRead(true)'"
|
# swarm attach 01 -exec "'bzz.blockNetworkRead(true)'"
|
||||||
sleep $wait
|
# sleep $wait
|
||||||
swarm stop 01
|
swarm stop 01
|
||||||
|
|
||||||
swarm start 01
|
# swarm start 01
|
||||||
swarm needs 01 $key $long
|
# sleep $wait
|
||||||
|
# swarm needs 01 $key $long
|
||||||
|
# sleep 3
|
||||||
swarm stop all
|
swarm stop all
|
||||||
20
swarm/test/test.sh
Normal file
20
swarm/test/test.sh
Normal file
|
|
@ -0,0 +1,20 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
TEST_DIR=`dirname $0`
|
||||||
|
TEST_NAME=`basename $0 .sh`
|
||||||
|
TEST_TYPE=`basename $TEST_DIR`
|
||||||
|
export IP_ADDR="[::]"
|
||||||
|
|
||||||
|
|
||||||
|
export SWARM_NETWORK_ID=322$TEST_NAME
|
||||||
|
export SWARM_DIR=~/bzz/test/$TEST_TYPE
|
||||||
|
|
||||||
|
rm -rf $SWARM_DIR/$SWARM_NETWORK_ID
|
||||||
|
|
||||||
|
wait=1
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
function randomfile {
|
||||||
|
dd if=/dev/urandom of=/dev/stdout bs=1024 count=$1 2>/dev/null
|
||||||
|
}
|
||||||
Loading…
Reference in a new issue