forks/go-ethereum
1
0
Fork 1
mirror of https://github.com/ethereum/go-ethereum.git synced 2026-08-19 02:12:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Interim ShyftGeth Docker Complete - PG Docker Complete - Persistent PG Db Storage

Browse source
This commit is contained in:
David Krett 2018-07-27 16:29:58 -04:00
parent 122c62ca81
commit 3f6f14bc12
5 changed files with 125 additions and 96 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

95
README.md
View file

@ -2,9 +2,7 @@
Official golang implementation of the Ethereum protocol. Official golang implementation of the Ethereum protocol.
[![API Reference]( [![API Reference](https://camo.githubusercontent.com/915b7be44ada53c290eb157634330494ebe3e30a/68747470733a2f2f676f646f632e6f72672f6769746875622e636f6d2f676f6c616e672f6764646f3f7374617475732e737667)](https://godoc.org/github.com/empyrean/go-ethereum)
https://camo.githubusercontent.com/915b7be44ada53c290eb157634330494ebe3e30a/68747470733a2f2f676f646f632e6f72672f6769746875622e636f6d2f676f6c616e672f6764646f3f7374617475732e737667
)](https://godoc.org/github.com/empyrean/go-ethereum)
[![Go Report Card](https://goreportcard.com/badge/github.com/empyrean/go-ethereum)](https://goreportcard.com/report/github.com/empyrean/go-ethereum) [![Go Report Card](https://goreportcard.com/badge/github.com/empyrean/go-ethereum)](https://goreportcard.com/report/github.com/empyrean/go-ethereum)
[![Build Status](https://travis-ci.org/ShyftNetwork/shyft_go-ethereum.svg?branch=master)](https://travis-ci.org/ShyftNetwork/shyft_go-ethereum) [![Build Status](https://travis-ci.org/ShyftNetwork/shyft_go-ethereum.svg?branch=master)](https://travis-ci.org/ShyftNetwork/shyft_go-ethereum)
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/ethereum/go-ethereum?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) [![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/ethereum/go-ethereum?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
@ -33,7 +31,7 @@ or, to build the full suite of utilities:
The go-ethereum project comes with several wrappers/executables found in the `cmd` directory. The go-ethereum project comes with several wrappers/executables found in the `cmd` directory.
| Command | Description | | Command | Description |
|:----------:|-------------| | :-----------: | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`geth`** | Our main Ethereum CLI client. It is the entry point into the Ethereum network (main-, test- or private net), capable of running as a full node (default) archive node (retaining all historical state) or a light node (retrieving data live). It can be used by other processes as a gateway into the Ethereum network via JSON RPC endpoints exposed on top of HTTP, WebSocket and/or IPC transports. `geth --help` and the [CLI Wiki page](https://github.com/empyrean/go-ethereum/wiki/Command-Line-Options) for command line options. | | **`geth`** | Our main Ethereum CLI client. It is the entry point into the Ethereum network (main-, test- or private net), capable of running as a full node (default) archive node (retaining all historical state) or a light node (retrieving data live). It can be used by other processes as a gateway into the Ethereum network via JSON RPC endpoints exposed on top of HTTP, WebSocket and/or IPC transports. `geth --help` and the [CLI Wiki page](https://github.com/empyrean/go-ethereum/wiki/Command-Line-Options) for command line options. |
| `abigen` | Source code generator to convert Ethereum contract definitions into easy to use, compile-time type-safe Go packages. It operates on plain [Ethereum contract ABIs](https://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI) with expanded functionality if the contract bytecode is also available. However it also accepts Solidity source files, making development much more streamlined. Please see our [Native DApps](https://github.com/empyrean/go-ethereum/wiki/Native-DApps:-Go-bindings-to-Ethereum-contracts) wiki page for details. | | `abigen` | Source code generator to convert Ethereum contract definitions into easy to use, compile-time type-safe Go packages. It operates on plain [Ethereum contract ABIs](https://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI) with expanded functionality if the contract bytecode is also available. However it also accepts Solidity source files, making development much more streamlined. Please see our [Native DApps](https://github.com/empyrean/go-ethereum/wiki/Native-DApps:-Go-bindings-to-Ethereum-contracts) wiki page for details. |
| `bootnode` | Stripped down version of our Ethereum client implementation that only takes part in the network node discovery protocol, but does not run any of the higher level application protocols. It can be used as a lightweight bootstrap node to aid in finding peers in private networks. | | `bootnode` | Stripped down version of our Ethereum client implementation that only takes part in the network node discovery protocol, but does not run any of the higher level application protocols. It can be used as a lightweight bootstrap node to aid in finding peers in private networks. |
@ -63,10 +61,10 @@ $ geth console
This command will: This command will:
* Start geth in fast sync mode (default, can be changed with the `--syncmode` flag), causing it to - Start geth in fast sync mode (default, can be changed with the `--syncmode` flag), causing it to
download more data in exchange for avoiding processing the entire history of the Ethereum network, download more data in exchange for avoiding processing the entire history of the Ethereum network,
which is very CPU intensive. which is very CPU intensive.
* Start up Geth's built-in interactive [JavaScript console](https://github.com/empyrean/go-ethereum/wiki/JavaScript-Console), - Start up Geth's built-in interactive [JavaScript console](https://github.com/empyrean/go-ethereum/wiki/JavaScript-Console),
(via the trailing `console` subcommand) through which you can invoke all official [`web3` methods](https://github.com/ethereum/wiki/wiki/JavaScript-API) (via the trailing `console` subcommand) through which you can invoke all official [`web3` methods](https://github.com/ethereum/wiki/wiki/JavaScript-API)
as well as Geth's own [management APIs](https://github.com/empyrean/go-ethereum/wiki/Management-APIs). as well as Geth's own [management APIs](https://github.com/empyrean/go-ethereum/wiki/Management-APIs).
This too is optional and if you leave it out you can always attach to an already running Geth instance This too is optional and if you leave it out you can always attach to an already running Geth instance
@ -88,22 +86,22 @@ testnet too. Please see above for their explanations if you've skipped to here.
Specifying the `--testnet` flag however will reconfigure your Geth instance a bit: Specifying the `--testnet` flag however will reconfigure your Geth instance a bit:
* Instead of using the default data directory (`~/.ethereum` on Linux for example), Geth will nest - Instead of using the default data directory (`~/.ethereum` on Linux for example), Geth will nest
itself one level deeper into a `testnet` subfolder (`~/.ethereum/testnet` on Linux). Note, on OSX itself one level deeper into a `testnet` subfolder (`~/.ethereum/testnet` on Linux). Note, on OSX
and Linux this also means that attaching to a running testnet node requires the use of a custom and Linux this also means that attaching to a running testnet node requires the use of a custom
endpoint since `geth attach` will try to attach to a production node endpoint by default. E.g. endpoint since `geth attach` will try to attach to a production node endpoint by default. E.g.
`geth attach <datadir>/testnet/geth.ipc`. Windows users are not affected by this. `geth attach <datadir>/testnet/geth.ipc`. Windows users are not affected by this.
* Instead of connecting the main Ethereum network, the client will connect to the test network, - Instead of connecting the main Ethereum network, the client will connect to the test network,
which uses different P2P bootnodes, different network IDs and genesis states. which uses different P2P bootnodes, different network IDs and genesis states.
*Note: Although there are some internal protective measures to prevent transactions from crossing _Note: Although there are some internal protective measures to prevent transactions from crossing
over between the main network and test network, you should make sure to always use separate accounts over between the main network and test network, you should make sure to always use separate accounts
for play-money and real-money. Unless you manually move accounts, Geth will by default correctly for play-money and real-money. Unless you manually move accounts, Geth will by default correctly
separate the two networks and will not make any accounts available between them.* separate the two networks and will not make any accounts available between them._
### Full node on the Rinkeby test network ### Full node on the Rinkeby test network
The above test network is a cross client one based on the ethash proof-of-work consensus algorithm. As such, it has certain extra overhead and is more susceptible to reorganization attacks due to the network's low difficulty / security. Go Ethereum also supports connecting to a proof-of-authority based test network called [*Rinkeby*](https://www.rinkeby.io) (operated by members of the community). This network is lighter, more secure, but is only supported by go-ethereum. The above test network is a cross client one based on the ethash proof-of-work consensus algorithm. As such, it has certain extra overhead and is more susceptible to reorganization attacks due to the network's low difficulty / security. Go Ethereum also supports connecting to a proof-of-authority based test network called [_Rinkeby_](https://www.rinkeby.io) (operated by members of the community). This network is lighter, more secure, but is only supported by go-ethereum.
``` ```
$ geth --rinkeby console $ geth --rinkeby console
@ -123,7 +121,7 @@ To get an idea how the file should look like you can use the `dumpconfig` subcom
$ geth --your-favourite-flags dumpconfig $ geth --your-favourite-flags dumpconfig
``` ```
*Note: This works only with geth v1.6.0 and above.* _Note: This works only with geth v1.6.0 and above._
#### Docker quick start #### Docker quick start
@ -153,19 +151,19 @@ These can be turned on/off and configured as you'd expect.
HTTP based JSON-RPC API options: HTTP based JSON-RPC API options:
* `--rpc` Enable the HTTP-RPC server - `--rpc` Enable the HTTP-RPC server
* `--rpcaddr` HTTP-RPC server listening interface (default: "localhost") - `--rpcaddr` HTTP-RPC server listening interface (default: "localhost")
* `--rpcport` HTTP-RPC server listening port (default: 8545) - `--rpcport` HTTP-RPC server listening port (default: 8545)
* `--rpcapi` API's offered over the HTTP-RPC interface (default: "eth,net,web3") - `--rpcapi` API's offered over the HTTP-RPC interface (default: "eth,net,web3")
* `--rpccorsdomain` Comma separated list of domains from which to accept cross origin requests (browser enforced) - `--rpccorsdomain` Comma separated list of domains from which to accept cross origin requests (browser enforced)
* `--ws` Enable the WS-RPC server - `--ws` Enable the WS-RPC server
* `--wsaddr` WS-RPC server listening interface (default: "localhost") - `--wsaddr` WS-RPC server listening interface (default: "localhost")
* `--wsport` WS-RPC server listening port (default: 8546) - `--wsport` WS-RPC server listening port (default: 8546)
* `--wsapi` API's offered over the WS-RPC interface (default: "eth,net,web3") - `--wsapi` API's offered over the WS-RPC interface (default: "eth,net,web3")
* `--wsorigins` Origins from which to accept websockets requests - `--wsorigins` Origins from which to accept websockets requests
* `--ipcdisable` Disable the IPC-RPC server - `--ipcdisable` Disable the IPC-RPC server
* `--ipcapi` API's offered over the IPC-RPC interface (default: "admin,debug,eth,miner,net,personal,shh,txpool,web3") - `--ipcapi` API's offered over the IPC-RPC interface (default: "admin,debug,eth,miner,net,personal,shh,txpool,web3")
* `--ipcpath` Filename for IPC socket/pipe within the datadir (explicit paths escape it) - `--ipcpath` Filename for IPC socket/pipe within the datadir (explicit paths escape it)
You'll need to use your own programming environments' capabilities (libraries, tools, etc) to connect You'll need to use your own programming environments' capabilities (libraries, tools, etc) to connect
via HTTP, WS or IPC to a Geth node configured with the above flags and you'll need to speak [JSON-RPC](http://www.jsonrpc.org/specification) via HTTP, WS or IPC to a Geth node configured with the above flags and you'll need to speak [JSON-RPC](http://www.jsonrpc.org/specification)
@ -200,8 +198,10 @@ and agree upon. This consists of a small JSON file (e.g. call it `genesis.json`)
"extraData": "", "extraData": "",
"gasLimit": "0x2fefd8", "gasLimit": "0x2fefd8",
"nonce": "0x0000000000000042", "nonce": "0x0000000000000042",
"mixhash" : "0x0000000000000000000000000000000000000000000000000000000000000000", "mixhash":
"parentHash" : "0x0000000000000000000000000000000000000000000000000000000000000000", "0x0000000000000000000000000000000000000000000000000000000000000000",
"parentHash":
"0x0000000000000000000000000000000000000000000000000000000000000000",
"timestamp": "0x00" "timestamp": "0x00"
} }
``` ```
@ -228,7 +228,8 @@ $ geth init path/to/genesis.json
#### Creating the rendezvous point #### Creating the rendezvous point
With all nodes that you want to run initialized to the desired genesis state, you'll need to start a With all nodes that you want to run initialized to the desired genesis state, you'll need to start a
bootstrap node that others can use to find each other in your network and/or over the internet. The bootstrap node that others can use to find each other in your
network and/or over the internet. The
clean way is to configure and run a dedicated bootnode: clean way is to configure and run a dedicated bootnode:
``` ```
@ -241,7 +242,7 @@ that other nodes can use to connect to it and exchange peer information. Make su
displayed IP address information (most probably `[::]`) with your externally accessible IP to get the displayed IP address information (most probably `[::]`) with your externally accessible IP to get the
actual `enode` URL. actual `enode` URL.
*Note: You could also use a full fledged Geth node as a bootnode, but it's the less recommended way.* _Note: You could also use a full fledged Geth node as a bootnode, but it's the less recommended way._
#### Starting up your member nodes #### Starting up your member nodes
@ -254,8 +255,8 @@ private network separated, so do also specify a custom `--datadir` flag.
$ geth --datadir=path/to/custom/data/folder --bootnodes=<bootnode-enode-url-from-above> $ geth --datadir=path/to/custom/data/folder --bootnodes=<bootnode-enode-url-from-above>
``` ```
*Note: Since your network will be completely cut off from the main and test networks, you'll also _Note: Since your network will be completely cut off from the main and test networks, you'll also
need to configure a miner to process transactions and create new blocks for you.* need to configure a miner to process transactions and create new blocks for you._
#### Running a private miner #### Running a private miner
@ -280,6 +281,7 @@ limit blocks converge to (`--targetgaslimit`) and the price transactions are acc
## SHYFT NOTES ## SHYFT NOTES
#### CLI #### CLI
Run `./shyft-geth.sh` with one of the following flags: Run `./shyft-geth.sh` with one of the following flags:
- `--setup` - Setups postgres and the shyft chain db. - `--setup` - Setups postgres and the shyft chain db.
@ -287,6 +289,27 @@ Run `./shyft-geth.sh` with one of the following flags:
- `--reset` - Drops postgress and chain db, and reinstantiates both. - `--reset` - Drops postgress and chain db, and reinstantiates both.
- `--js [web3 filename]` - Executes web3 calls with a passed file name. If the file name is `sendTransactions.js`, `./shyft-geth.sh --js sendTransactions`. - `--js [web3 filename]` - Executes web3 calls with a passed file name. If the file name is `sendTransactions.js`, `./shyft-geth.sh --js sendTransactions`.
#### Docker Images
Docker Images are available for ShyftGeth and the Postgresql Database which can be used for development and testing. To launch these containers you will need to have docker-compose installed on your computer. Installation instructions for docker-compose are available [here](https://docs.docker.com/install/).
To launch ShyftGeth and PG - issue the following command from the root of the project directory:
`docker-compose up`
If you would like to reinitialize/rebuild the docker images you can issue the following command:
`docker-compose up --build`
The Postgresql Database Container will persist the database data to the the pg-data/ directory. So if you do want to reinitialize the database you should delete the content of this directory prior to launching the docker containers. From your local machine you can view the database by connecting to the database in the container at 127.0.0.1:8001
_TODO_
- Dockerize shyftBlockExplorerAPI
- Dockerize shyftBlockExplorerUI
- Adjust docker scripts and ports to facilitate sending of test transactions
- Modify Docker scripts to facilitate hot reloading during development
## Contribution ## Contribution
Thank you for considering to help out with the source code! We welcome contributions from Thank you for considering to help out with the source code! We welcome contributions from
@ -301,11 +324,11 @@ procedures quick and simple.
Please make sure your contributions adhere to our coding guidelines: Please make sure your contributions adhere to our coding guidelines:
* Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting) guidelines (i.e. uses [gofmt](https://golang.org/cmd/gofmt/)). - Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting) guidelines (i.e. uses [gofmt](https://golang.org/cmd/gofmt/)).
* Code must be documented adhering to the official Go [commentary](https://golang.org/doc/effective_go.html#commentary) guidelines. - Code must be documented adhering to the official Go [commentary](https://golang.org/doc/effective_go.html#commentary) guidelines.
* Pull requests need to be based on and opened against the `master` branch. - Pull requests need to be based on and opened against the `master` branch.
* Commit messages should be prefixed with the package(s) they modify. - Commit messages should be prefixed with the package(s) they modify.
* E.g. "eth, rpc: make trace configs optional" - E.g. "eth, rpc: make trace configs optional"
Please see the [Developers' Guide](https://github.com/empyrean/go-ethereum/wiki/Developers'-Guide) Please see the [Developers' Guide](https://github.com/empyrean/go-ethereum/wiki/Developers'-Guide)
for more details on configuring your environment, managing project dependencies and testing procedures. for more details on configuring your environment, managing project dependencies and testing procedures.

3
containers/docker/develop-alpine/pg/Dockerfile
View file

@ -4,6 +4,5 @@ FROM postgres:10.4-alpine
WORKDIR / WORKDIR /
COPY ./shyft-cli/postgres_setup/docker_init_user_db.sh /docker-entrypoint-initdb.d/docker_init_user_db.sh COPY ./shyft-cli/postgres_setup/docker_init_user_db.sh /docker-entrypoint-initdb.d/docker_init_user_db.sh
RUN cd /docker-entrypoint-initdb.d && ls -a RUN cd /docker-entrypoint-initdb.d && ls -a
VOLUME ["/var/lib/postgresql"]
EXPOSE 5432 EXPOSE 5432

19
containers/docker/develop-alpine/shyftApi/Dockerfile
View file

@ -0,0 +1,19 @@
FROM golang:1.10.3-alpine
COPY ./shyftBlockExplorerApi /go/src/github.com/ShyftNetwork/go-empyrean/shyftBlockExplorerApi
# COPY ./vendor /go/src/github.com/ShyftNetwork/go-empyrean/vendor
COPY ./core /go/src/github.com/ShyftNetwork/go-empyrean/core
# COPY ./common /go/src/github.com/ShyftNetwork/go-empyrean/common
COPY .git /go/src/github.com/ShyftNetwork/go-empyrean/.git
WORKDIR /go/src/github.com/ShyftNetwork/go-empyrean/shyftBlockExplorerApi
RUN apk add --update git make gcc musl-dev linux-headers ca-certificates
RUN go get ./
RUN go build
CMD go get github.com/pilu/fresh && fresh
EXPOSE 8080

9
containers/docker/develop-alpine/shyftGeth/Dockerfile
View file

@ -8,10 +8,7 @@ RUN \
apk add --update git make gcc musl-dev linux-headers ca-certificates && \ apk add --update git make gcc musl-dev linux-headers ca-certificates && \
(cd /go/src/github.com/ShyftEthereum/go-empyrean && make geth) && \ (cd /go/src/github.com/ShyftEthereum/go-empyrean && make geth) && \
cp -v /go/src/github.com/ShyftEthereum/go-empyrean/build/bin/geth /bin cp -v /go/src/github.com/ShyftEthereum/go-empyrean/build/bin/geth /bin
COPY ./shyft-cli/initShyftGeth.sh /
COPY ./shyft-cli/resetShyftGeth.sh /
COPY ./shyft-cli/startShyftGeth.sh /
# COPY ./containers/docker/develop-alpine/shyftGeth/wait-for-postgres.sh /
COPY config.toml / COPY config.toml /
COPY unlockPasswords.txt / COPY unlockPasswords.txt /
COPY ShyftNetwork.json / COPY ShyftNetwork.json /
@ -22,6 +19,6 @@ COPY UTC--2018-03-18T22-37-04.212101982Z--5bd738164c61fb50eb12e227846cbaef2de965
COPY UTC--2018-03-18T22-37-12.082606217Z--c04ee4131895f1d0c294d508af65d94060aa42bb / COPY UTC--2018-03-18T22-37-12.082606217Z--c04ee4131895f1d0c294d508af65d94060aa42bb /
COPY UTC--2018-03-18T22-37-19.650676366Z--07d899c4ac0c1725c35c5f816e60273b33a964f7 / COPY UTC--2018-03-18T22-37-19.650676366Z--07d899c4ac0c1725c35c5f816e60273b33a964f7 /
WORKDIR / WORKDIR /go/src/github.com/ShyftEthereum/go-empyrean
CMD ["./initShyftGeth.sh", "/go/src/ShyftNetwork/go-empyrean/shyft-cli/initShyftGeth.sh/startShyftGeth.sh"] CMD ["./shyft-cli/initShyftGeth.sh", "./shyft-cli/startShyftGeth.sh"]

19
docker-compose.yml
View file

@ -11,7 +11,6 @@ services:
- "30301:30301" - "30301:30301"
- "30303:30303/udp" - "30303:30303/udp"
- "30304:30304" - "30304:30304"
- "5432:5432"
volumes: volumes:
- .:/go/src/ShyftNetwork/go-empyrean - .:/go/src/ShyftNetwork/go-empyrean
working_dir: /go/src/ShyftNetwork/go-empyrean working_dir: /go/src/ShyftNetwork/go-empyrean
@ -20,16 +19,17 @@ services:
networks: networks:
- shyftnet - shyftnet
command: > command: >
sh -c '/go/src/ShyftNetwork/go-empyrean/wait-for.sh pg:5432 && sh -c 'cd /go/src/ShyftNetwork/go-empyrean &&
./wait-for.sh pg:5432 &&
DBENV=docker export DBENV && DBENV=docker export DBENV &&
/initShyftGeth.sh && ./shyft-cli/initShyftGeth.sh &&
/startShyftGeth.sh' ./shyft-cli/startShyftGeth.sh'
pg: pg:
build: build:
context: $PWD context: $PWD
dockerfile: containers/docker/develop-alpine/pg/Dockerfile dockerfile: containers/docker/develop-alpine/pg/Dockerfile
volumes: volumes:
- ./pg-data:/var/lib/postgresql/ - ./pg-data:/var/lib/postgresql/data
ports: ports:
- "8001:5432" - "8001:5432"
networks: networks:
@ -41,18 +41,9 @@ services:
# build: # build:
# context: $PWD # context: $PWD
# dockerfile: containers/docker/develop-alpine/shyftApi/Dockerfile # dockerfile: containers/docker/develop-alpine/shyftApi/Dockerfile
# volumes: -
networks: networks:
shyftnet: shyftnet:
driver: bridge driver: bridge
# volumes:
# pg-data:
# api:
# build:
# context: ./containers/docker/develop-alpine/shyftApi