forks/go-ethereum
1
0
Fork 1
mirror of https://github.com/ethereum/go-ethereum.git synced 2026-08-19 18:32:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Fix broken links and colon-separated pages missing from the wiki git repo

Browse source
This commit is contained in:
Adam Schmideg 2018-12-20 13:50:13 +01:00
parent 3b03b71349
commit 5d05be100d
12 changed files with 1148 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
_config.yml
View file

@ -27,7 +27,7 @@ navbar-links:
- Community and support: docs/geth#community-and-support
- Issue handling workflow: docs/Issue-handling-workflow
Developers:
- Mobile Clients: docs/Mobile-Clients:-Libraries-and-Inproc-Ethereum-Nodes
- Native DApps: docs/Native-DApps:-Go-bindings-to-Ethereum-contracts
- Mobile Clients: docs/Mobile-Clients/Libraries-and-Inproc-Ethereum-Nodes
- Native DApps: docs/Native-DApps/Go-bindings-to-Ethereum-contracts
- Active go-ethereum projects: docs/Active-go-ethereum-projects

4
docs/Contract-Tutorial.md
View file

@ -106,7 +106,7 @@ If you have the SolC Solidity Compiler installed, you need now reformat by remo
#### Linking your compiler in Geth
Now [go back to the console](../geth) and type this command to install solC, replacing _path/to/solc_ to the path that you got on the last command you did:
Now [go back to the console](Geth) and type this command to install solC, replacing _path/to/solc_ to the path that you got on the last command you did:
admin.setSolc("path/to/solc")
@ -157,7 +157,7 @@ Now you can paste the resulting text on your geth window. Wait up to thirty seco
You will probably be asked for the password you picked in the beginning, because you need to pay for the gas costs to deploying your contract. This contract is estimated to need 172 thousand gas to deploy (according to the [online solidity compiler](https://chriseth.github.io/cpp-ethereum/)), at the time of writing, gas on the test net is priced at 1 to 10 microethers per unit of gas (nicknamed "szabo" = 1 followed by 12 zeroes in wei). To know the latest price in ether all you can see the [latest gas prices at the network stats page](https://stats.ethdev.com) and multiply both terms.
**Notice that the cost is not paid to the [ethereum developers](../foundation), instead it goes to the _Miners_, people who are running computers who keep the network running. Gas price is set by the market of the current supply and demand of computation. If the gas prices are too high, you can be a miner and lower your asking price.**
**Notice that the cost is not paid to the [ethereum developers](https://ethereum.org), instead it goes to the _Miners_, people who are running computers who keep the network running. Gas price is set by the market of the current supply and demand of computation. If the gas prices are too high, you can be a miner and lower your asking price.**
After less than a minute, you should have a log with the contract address, this means you've sucessfully deployed your contract. You can verify the deployed code (compiled) by using this command:

2
docs/Geth.md
View file

@ -1,5 +1,5 @@
`geth` is the the command line interface for running a full ethereum node implemented in Go.
It is the main deliverable of the [Frontier Release](Frontier)
It is the main deliverable of the [Frontier Release](https://ethereum.gitbooks.io/frontier-guide/content/frontier.html)
## Capabilities

2
docs/JavaScript-Console.md
View file

@ -15,7 +15,7 @@ The attach node accepts an endpoint in case the geth node is running with a non
$ geth attach http://191.168.1.1:8545
$ geth attach ws://191.168.1.1:8546
Note that by default the geth node doesn't start the http and weboscket service and not all functionality is provided over these interfaces due to security reasons. These defaults can be overridden when the `--rpcapi` and `--wsapi` arguments when the geth node is started, or with [admin.startRPC](admin_startRPC) and [admin.startWS](admin_startWS).
Note that by default the geth node doesn't start the http and weboscket service and not all functionality is provided over these interfaces due to security reasons. These defaults can be overridden when the `--rpcapi` and `--wsapi` arguments when the geth node is started, or with [admin.startRPC](Management-APIs#admin_startrpc) and [admin.startWS](Management-APIs#admin_startws).
If you need log information, start with:

2
docs/Mining.md
View file

@ -288,4 +288,4 @@ eth -m on -G -a <coinbase> -i -v 8 //
* [Benchmarking results for GPU mining](https://forum.ethereum.org/discussion/2134/gpu-mining-is-out-come-and-let-us-know-of-your-bench-scores)
* [historic moment](https://twitter.com/gavofyork/status/586623875577937922)
* [live mining statistic](https://etherapps.info/stats/mining)
* [netstat ethereum network monitor](stats.ethdev.com)
* [netstat ethereum network monitor](https://stats.ethdev.com)

88
docs/Mobile-Clients/Libraries-and-Inproc-Ethereum-Nodes.md Normal file
View file

@ -0,0 +1,88 @@
**This page has been obsoleted. An new guide is in the progress at [[Mobile: Introduction]]**
---
*This page is meant to be a guide on using go-ethereum from mobile platforms. Since neither the mobile libraries nor the light client protocol is finalized, the content here will be sparse, with emphasis being put on how to get your hands dirty. As the APIs stabilize this section will be expanded accordingly.*
### Changelog
* 30th September, 2016: Create initial page, upload Android light bundle.
### Background
Before reading further, please skim through the slides of a Devcon2 talk: [Import Geth: Ethereum from Go and beyond](https://ethereum.karalabe.com/talks/2016-devcon.html), which introduces the basic concepts behind using go-ethereum as a library, and also showcases a few code snippets on how you can do various client side tasks, both on classical computing nodes as well as Android devices. A recording of the talk will be linked when available.
*Please note, the Android and iOS library bundles linked in the presentation will not be updated (for obvious posterity reasons), so always grab latest bundles from this page (until everything is merged into the proper build infrastructure).*
### Mobile bundles
You can download the latest bundles at:
* [Android (30th September, 2016)](https://bintray.com/karalabe/ethereum/download_file?file_path=geth.aar) - `SHA1: 753e334bf61fa519bec83bcb487179e36d58fc3a`
* iOS: *light client has not yet been bundled*
### Android quickstart
We assume you are using Android Studio for your development. Please download the latest Android `.aar` bundle from above and import it into your Android Studio project via `File -> New -> New Module`. This will result in a `geth` sub-project inside your work-space. To use the library in your project, please modify your apps `build.gradle` file, adding a dependency to the Geth library:
```gradle
dependencies {
// All your previous dependencies
compile project(':geth')
}
```
To get you hands dirty, here's a code snippet that will
* Start up an in-process light node inside your Android application
* Display some initial infos about your node
* Subscribe to new blocks and display them live as they arrive
<img src="http://i.imgur.com/LyTCCqg.png" width="512px" />
```java
import org.ethereum.geth.*;
public class MainActivity extends AppCompatActivity {
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
setTitle("Android In-Process Node");
final TextView textbox = (TextView) findViewById(R.id.textbox);
Context ctx = new Context();
try {
Node node = Geth.newNode(getFilesDir() + "/.ethereum", new NodeConfig());
node.start();
NodeInfo info = node.getNodeInfo();
textbox.append("My name: " + info.getName() + "\n");
textbox.append("My address: " + info.getListenerAddress() + "\n");
textbox.append("My protocols: " + info.getProtocols() + "\n\n");
EthereumClient ec = node.getEthereumClient();
textbox.append("Latest block: " + ec.getBlockByNumber(ctx, -1).getNumber() + ", syncing...\n");
NewHeadHandler handler = new NewHeadHandler() {
@Override public void onError(String error) { }
@Override public void onNewHead(final Header header) {
MainActivity.this.runOnUiThread(new Runnable() {
public void run() { textbox.append("#" + header.getNumber() + ": " + header.getHash().getHex().substring(0, 10) + ".\n"); }
});
}
};
ec.subscribeNewHead(ctx, handler, 16);
} catch (Exception e) {
e.printStackTrace();
}
}
}
```
#### Known quirks
* Many constructors (those that would throw exceptions) are of the form `Geth.newXXX()`, instead of simply the Java style `new XXX()` This is an upstream limitation of the [gomobile](https://github.com/golang/mobile) project, one which is currently being worked on to resolve.
* There are zero documentations attached to the Java library methods. This too is a limitation of the [gomobile](https://github.com/golang/mobile) project. We will try to propose a fix upstream to make our docs from the Go codebase available in Java.

196
docs/Mobile/Account-management.md Normal file
View file

@ -0,0 +1,196 @@
To provide Ethereum integration for your mobile applications, the very first thing you should be interested in doing is account management.
Although all current leading Ethereum implementations provide account management built in, it is ill advised to keep accounts in any location that is shared between multiple applications and/or multiple people. The same way you do not entrust your ISP (who is after all your gateway into the internet) with your login credentials; you should not entrust an Ethereum node (who is your gateway into the Ethereum network) with your credentials either.
The proper way to handle user accounts in your mobile applications is to do client side account management, everything self-contained within your own application. This way you can ensure as fine grained (or as coarse) access permissions to the sensitive data as deemed necessary, without relying on any third party application's functionality and/or vulnerabilities.
To support this, `go-ethereum` provides a simple, yet thorough accounts library that gives you all the tools to do properly secured account management via encrypted keystores and passphrase protected accounts. You can leverage all the security of the `go-ethereum` crypto implementation while at the same time running everything in your own application.
## Encrypted keystores
Although handling your users' accounts locally on their own mobile device does provide certain security guarantees, access keys to Ethereum accounts should never lay around in clear-text form. As such, we provide an encrypted keystore that provides the proper security guarantees for you without requiring a thorough understanding from your part of the associated cryptographic primitives.
The important thing to know when using the encrypted keystore is that the cryptographic primitives used within can operate either in *standard* or *light* mode. The former provides a higher level of security at the cost of increased computational burden and resource consumption:
* *standard* needs 256MB memory and 1 second processing on a modern CPU to access a key
* *light* needs 4MB memory and 100 millisecond processing on a modern CPU to access a key
As such, *light* is more suitable for mobile applications, but you should be aware of the trade-offs nonetheless.
*For those interested in the cryptographic and/or implementation details, the key-store uses the `secp256k1` elliptic curve as defined in the [Standards for Efficient Cryptography](http://www.secg.org/sec2-v2.pdf), implemented by the [`libsecp256k`](https://github.com/bitcoin-core/secp256k1) library and wrapped by [`github.com/ethereum/go-ethereum/accounts`](https://godoc.org/github.com/ethereum/go-ethereum/accounts). Accounts are stored on disk in the [Web3 Secret Storage](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) format.*
### Keystores on Android (Java)
The encrypted keystore on Android is implemented by the `KeyStore` class from the `org.ethereum.geth` package. The configuration constants (for the *standard* or *light* security modes described above) are located in the `Geth` abstract class, similarly from the `org.ethereum.geth` package. Hence to do client side account management on Android, you'll need to import two classes into your Java code:
```java
import org.ethereum.geth.Geth;
import org.ethereum.geth.KeyStore;
```
Afterwards you can create a new encrypted keystore via:
```java
KeyStore ks = new KeyStore("/path/to/keystore", Geth.LightScryptN, Geth.LightScryptP);
```
The path to the keystore folder needs to be a location that is writable by the local mobile application but non-readable for other installed applications (for security reasons obviously), so we'd recommend placing it inside your app's data directory. If you are creating the `KeyStore` from within a class extending an Android object, you will most probably have access to the `Context.getFilesDir()` method via `this.getFilesDir()`, so you could set the keystore path to `this.getFilesDir() + "/keystore"`.
The last two arguments of the `KeyStore` constructor are the crypto parameters defining how resource-intensive the keystore encryption should be. You can choose between `Geth.StandardScryptN, Geth.StandardScryptP`, `Geth.LightScryptN, Geth.LightScryptP` or specify your own numbers (please make sure you understand the underlying cryptography for this). We recommend using the *light* version.
### Keystores on iOS (Swift 3)
The encrypted keystore on iOS is implemented by the `GethKeyStore` class from the `Geth` framework. The configuration constants (for the *standard* or *light* security modes described above) are located in the same namespace as global variables. Hence to do client side account management on iOS, you'll need to import the framework into your Swift code:
```swift
import Geth
```
Afterwards you can create a new encrypted account manager via:
```swift
let ks = GethNewKeyStore("/path/to/keystore", GethLightScryptN, GethLightScryptP);
```
The path to the keystore folder needs to be a location that is writable by the local mobile application but non-readable for other installed applications (for security reasons obviously), so we'd recommend placing it inside your app's document directory. You should be able to retrieve the document directory via `let datadir = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]`, so you could set the keystore path to `datadir + "/keystore"`.
The last two arguments of the `GethNewKeyStore` factory method are the crypto parameters defining how resource-intensive the keystore encryption should be. You can choose between `GethStandardScryptN, GethStandardScryptP`, `GethLightScryptN, GethLightScryptP` or specify your own numbers (please make sure you understand the underlying cryptography for this). We recommend using the *light* version.
## Account lifecycle
Having created an encrypted keystore for your Ethereum accounts, you can use this for the entire account lifecycle requirements of your mobile application. This includes the basic functionality of creating new accounts and deleting existing ones; as well as the more advanced functionality of updating access credentials, exporting existing accounts, and importing them on another device.
Although the keystore defines the encryption strength it uses to store your accounts, there is no global master password that can grant access to all of them. Rather each account is maintained individually, and stored on disk in its [encrypted format](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) individually, ensuring a much cleaner and stricter separation of credentials.
This individuality however means that any operation requiring access to an account will need to provide the necessary authentication credentials for that particular account in the form of a passphrase:
* When creating a new account, the caller must supply a passphrase to encrypt the account with. This passphrase will be required for any subsequent access, the lack of which will forever forfeit using the newly created account.
* When deleting an existing account, the caller must supply a passphrase to verify ownership of the account. This isn't cryptographically necessary, rather a protective measure against accidental loss of accounts.
* When updating an existing account, the caller must supply both current and new passphrases. After completing the operation, the account will not be accessible via the old passphrase any more.
* When exporting an existing account, the caller must supply both the current passphrase to decrypt the account, as well as an export passphrase to re-encrypt it with before returning the key-file to the user. This is required to allow moving accounts between devices without sharing original credentials.
* When importing a new account, the caller must supply both the encryption passphrase of the key-file being imported, as well as a new passhprase with which to store the account. This is required to allow storing account with different credentials than used for moving them around.
*Please note, there is no recovery mechanisms for losing the passphrases. The cryptographic properties of the encrypted keystore (if using the provided parameters) guarantee that account credentials cannot be brute forced in any meaningful time.*
### Accounts on Android (Java)
An Ethereum account on Android is implemented by the `Account` class from the `org.ethereum.geth` package. Assuming we already have an instance of a `KeyStore` called `ks` from the previous section, we can easily execute all of the described lifecycle operations with a handful of function calls.
```java
// Create a new account with the specified encryption passphrase.
Account newAcc = ksm.newAccount("Creation password");
// Export the newly created account with a different passphrase. The returned
// data from this method invocation is a JSON encoded, encrypted key-file.
byte[] jsonAcc = ks.exportKey(newAcc, "Creation password", "Export password");
// Update the passphrase on the account created above inside the local keystore.
ks.updateAccount(newAcc, "Creation password", "Update password");
// Delete the account updated above from the local keystore.
ks.deleteAccount(newAcc, "Update password");
// Import back the account we've exported (and then deleted) above with yet
// again a fresh passphrase.
Account impAcc = ks.importKey(jsonAcc, "Export password", "Import password");
```
*Although instances of `Account` can be used to access various information about specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or private keys), rather act solely as identifiers for client code and the keystore.*
### Accounts on iOS (Swift 3)
An Ethereum account on iOS is implemented by the `GethAccount` class from the `Geth` framework. Assuming we already have an instance of a `GethKeyStore` called `ks` from the previous section, we can easily execute all of the described lifecycle operations with a handful of function calls.
```swift
// Create a new account with the specified encryption passphrase.
let newAcc = try! ks?.newAccount("Creation password")
// Export the newly created account with a different passphrase. The returned
// data from this method invocation is a JSON encoded, encrypted key-file.
let jsonKey = try! ks?.exportKey(newAcc!, passphrase: "Creation password", newPassphrase: "Export password")
// Update the passphrase on the account created above inside the local keystore.
try! ks?.update(newAcc, passphrase: "Creation password", newPassphrase: "Update password")
// Delete the account updated above from the local keystore.
try! ks?.delete(newAcc, passphrase: "Update password")
// Import back the account we've exported (and then deleted) above with yet
// again a fresh passphrase.
let impAcc = try! ks?.importKey(jsonKey, passphrase: "Export password", newPassphrase: "Import password")
```
*Although instances of `GethAccount` can be used to access various information about specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or private keys), rather act solely as identifiers for client code and the keystore.*
## Signing authorization
As mentioned above, account objects do not hold the sensitive private keys of the associated Ethereum accounts, but are merely placeholders to identify the cryptographic keys with. All operations that require authorization (e.g. transaction signing) are performed by the account manager after granting it access to the private keys.
There are a few different ways one can authorize the account manager to execute signing operations, each having its advantages and drawbacks. Since the different methods have wildly different security guarantees, it is essential to be clear on how each works:
* **Single authorization**: The simplest way to sign a transaction via the keystore is to provide the passphrase of the account every time something needs to be signed, which will ephemerally decrypt the private key, execute the signing operation and immediately throw away the decrypted key. The drawbacks are that the passphrase needs to be queried from the user every time, which can become annoying if done frequently; or the application needs to keep the passphrase in memory, which can have security consequences if not done properly; and depending on the keystore's configured strength, constantly decrypting keys can result in non-negligible resource requirements.
* **Multiple authorizations**: A more complex way of signing transactions via the keystore is to unlock the account via its passphrase once, and allow the account manager to cache the decrypted private key, enabling all subsequent signing requests to complete without the passphrase. The lifetime of the cached private key may be managed manually (by explicitly locking the account back up) or automatically (by providing a timeout during unlock). This mechanism is useful for scenarios where the user may need to sign many transactions or the application would need to do so without requiring user input. The crucial aspect to remember is that **anyone with access to the account manager can sign transactions while a particular account is unlocked** (e.g. device left unattended; application running untrusted code).
*Note, creating transactions is out of scope here, so the remainder of this section will assume we already have a transaction to sign, and will focus only on creating an authorized version of it. Creating an actually meaningful transaction will be covered later.*
### Signing on Android (Java)
Assuming we already have an instance of a `KeyStore` called `ks` from the previous sections, we can create a new account to sign transactions with via it's already demonstrated `newAccount` method; and to avoid going into transaction creation for now, we can hard-code a random transaction to sign instead.
```java
// Create a new account to sign transactions with
Account signer = ks.newAccount("Signer password");
Transaction tx = new Transaction(
1, new Address("0x0000000000000000000000000000000000000000"),
new BigInt(0), new BigInt(0), new BigInt(1), null); // Random empty transaction
BigInt chain = new BigInt(1); // Chain identifier of the main net
```
With the boilerplate out of the way, we can now sign transaction using the authorization mechanisms described above:
```java
// Sign a transaction with a single authorization
Transaction signed = ks.signTxPassphrase(signer, "Signer password", tx, chain);
// Sign a transaction with multiple manually cancelled authorizations
ks.unlock(signer, "Signer password");
signed = ks.signTx(signer, tx, chain);
ks.lock(signer.getAddress());
// Sign a transaction with multiple automatically cancelled authorizations
ks.timedUnlock(signer, "Signer password", 1000000000);
signed = ks.signTx(signer, tx, chain);
```
### Signing on iOS (Swift 3)
Assuming we already have an instance of a `GethKeyStore` called `ks` from the previous sections, we can create a new account to sign transactions with via it's already demonstrated `newAccount` method; and to avoid going into transaction creation for now, we can hard-code a random transaction to sign instead.
```swift
// Create a new account to sign transactions with
var error: NSError?
let signer = try! ks?.newAccount("Signer password")
let to = GethNewAddressFromHex("0x0000000000000000000000000000000000000000", &error)
let tx = GethNewTransaction(1, to, GethNewBigInt(0), GethNewBigInt(0), GethNewBigInt(0), nil) // Random empty transaction
let chain = GethNewBigInt(1) // Chain identifier of the main net
```
*Note, although Swift usually rewrites `NSError` returns to throws, this particular instance seems to have been missed for some reason (possibly due to it being a constructor). It will be fixed in a later version of the iOS bindings when the appropriate fixed are implemented upstream in the `gomobile` project.*
With the boilerplate out of the way, we can now sign transaction using the authorization methods described above:
```swift
// Sign a transaction with a single authorization
var signed = try! ks?.signTxPassphrase(signer, passphrase: "Signer password", tx: tx, chainID: chain)
// Sign a transaction with multiple manually cancelled authorizations
try! ks?.unlock(signer, passphrase: "Signer password")
signed = try! ks?.signTx(signer, tx: tx, chainID: chain)
try! ks?.lock(signer?.getAddress())
// Sign a transaction with multiple automatically cancelled authorizations
try! ks?.timedUnlock(signer, passphrase: "Signer password", timeout: 1000000000)
signed = try! ks?.signTx(signer, tx: tx, chainID: chain)
```

122
docs/Mobile/Introduction.md Normal file
View file

@ -0,0 +1,122 @@
The Ethereum blockchain along with its two extension protocols Whisper and Swarm was originally conceptualized to become the supporting pillar of web3, providing the consensus, messaging and storage backbone for a new generation of distributed (actually, decentralized) applications called DApps.
The first incarnation towards this dream of web3 was a command line client providing an RPC interface into the peer-to-peer protocols. The client was soon enough extended with a web-browser-like graphical user interface, permitting developers to write DApps based on the tried and proven HTML/CSS/JS technologies.
As many DApps have more complex requirements than what a browser environment can handle, it became apparent that providing programmatic access to the web3 pillars would open the door towards a new class of applications. As such, the second incarnation of the web3 dream is to open up all our technologies for other projects as reusable components.
Starting with the 1.5 release family of `go-ethereum`, we transitioned away from providing only a full blown Ethereum client and started shipping official Go packages that could be embedded into third party desktop and server applications. It took only a small leap from here to begin porting our code to mobile platforms.
## Quick overview
Similarly to our reusable Go libraries, the mobile wrappers also focus on four main usage areas:
- Simplified client side account management
- Remote node interfacing via different transports
- Contract interactions through auto-generated bindings
- In-process Ethereum, Whisper and Swarm peer-to-peer node
You can watch a quick overview about these in Peter's (@karalabe) talk titled "Import Geth: Ethereum from Go and beyond", presented at the Ethereum Devcon2 developer conference in September, 2016 (Shanghai). Slides are [available here](https://ethereum.karalabe.com/talks/2016-devcon.html).
[![Peter's Devcon2 talk](https://img.youtube.com/vi/R0Ia1U9Gxjg/0.jpg)](https://www.youtube.com/watch?v=R0Ia1U9Gxjg)
## Library bundles
The `go-ethereum` mobile library is distributed either as an Android `.aar` archive (containing binaries for `arm-7`, `arm64`, `x86` and `x64`); or as an iOS XCode framework (containing binaries for `arm-7`, `arm64` and `x86`). We do not provide library bundles for Windows phone the moment.
### Android archive
The simplest way to use `go-ethereum` in your Android project is through a Maven dependency. We provide bundles of all our stable releases (starting from v1.5.0) through Maven Central, and also provide the latest develop bundle through the Sonatype OSS repository.
#### Stable dependency (Maven Central)
To add an Android dependency to the **stable** library release of `go-ethereum`, you'll need to ensure that the Maven Central repository is enabled in your Android project, and that the `go-ethereum` code is listed as a required dependency of your application. You can do both of these by editing the `build.gradle` script in your Android app's folder:
```gradle
repositories {
mavenCentral()
}
dependencies {
// All your previous dependencies
compile 'org.ethereum:geth:1.5.2' // Change the version to the latest release
}
```
#### Develop dependency (Sonatype)
To add an Android dependency to the current **develop** version of `go-ethereum`, you'll need to ensure that the Sonatype snapshot repository is enabled in your Android project, and that the `go-ethereum` code is listed as a required `SNAPSHOT` dependency of your application. You can do both of these by editing the `build.gradle` script in your Android app's folder:
```gradle
repositories {
maven {
url "https://oss.sonatype.org/content/groups/public"
}
}
dependencies {
// All your previous dependencies
compile 'org.ethereum:geth:1.5.3-SNAPSHOT' // Change the version to the latest release
}
```
#### Custom dependency
If you prefer not to depend on Maven Central or Sonatype; or would like to access an older develop build not available any more as an online dependency, you can download any bundle directly from [our website](https://geth.ethereum.org/downloads/) and insert it into your project in Android Studio via `File -> New -> New module... -> Import .JAR/.AAR Package`.
You will also need to configure `gradle` to link the mobile library bundle to your application. This can be done by adding a new entry to the `dependencies` section of your `build.gradle` script, pointing it to the module you just added (named `geth` by default).
```gradle
dependencies {
// All your previous dependencies
compile project(':geth')
}
```
#### Manual builds
Lastly, if you would like to make modifications to the `go-ethereum` mobile code and/or build it yourself locally instead of downloading a pre-built bundle, you can do so using a `make` command. This will create an Android archive called `geth.aar` in the `build/bin` folder that you can import into your Android Studio as described above.
```bash
$ make android
[...]
Done building.
Import "build/bin/geth.aar" to use the library.
```
### iOS framework
The simplest way to use `go-ethereum` in your iOS project is through a [CocoaPods](https://cocoapods.org/) dependency. We provide bundles of all our stable releases (starting from v1.5.3) and also latest develop versions.
#### Automatic dependency
To add an iOS dependency to the current stable or latest develop version of `go-ethereum`, you'll need to ensure that your iOS XCode project is configured to use CocoaPods. Detailing that is out of scope in this document, but you can find a guide in the upstream [Using CocoaPods](https://guides.cocoapods.org/using/using-cocoapods.html) page. Afterwards you can edit your `Podfile` to list `go-ethereum` as a dependency:
```ruby
target 'MyApp' do
# All your previous dependencies
pod 'Geth', '1.5.4' # Change the version to the latest release
end
```
Alternatively, if you'd like to use the latest develop version, replace the package version `1.5.4` with `~> 1.5.5-unstable` to switch to pre-releases and to always pull in the latest bundle from a particular release family.
#### Custom dependency
If you prefer not to depend on CocoaPods; or would like to access an older develop build not available any more as an online dependency, you can download any bundle directly from [our website](https://geth.ethereum.org/downloads/) and insert it into your project in XCode via `Project Settings -> Build Phases -> Link Binary With Libraries`.
Do not forget to extract the framework from the compressed `.tar.gz` archive. You can do that either using a GUI tool or from the command line via (replace the archive with your downloaded file):
```
tar -zxvf geth-ios-all-1.5.3-unstable-e05d35e6.tar.gz
```
#### Manual builds
Lastly, if you would like to make modifications to the `go-ethereum` mobile code and/or build it yourself locally instead of downloading a pre-built bundle, you can do so using a `make` command. This will create an iOS XCode framework called `Geth.framework` in the `build/bin` folder that you can import into XCode as described above.
```bash
$ make ios
[...]
Done building.
Import "build/bin/Geth.framework" to use the library.
```

438
docs/Native-DApps/Go-bindings-to-Ethereum-contracts.md Normal file
View file

@ -0,0 +1,438 @@
**[Please note, events are not yet implemented as they need some RPC subscription
features that are still under review.]**
The original roadmap and/or dream of the Ethereum platform was to provide a solid, high
performing client implementation of the consensus protocol in various languages, which
would provide an RPC interface for JavaScript DApps to communicate with, pushing towards
the direction of the Mist browser, through which users can interact with the blockchain.
Although this was a solid plan for mainstream adoption and does cover quite a lot of use
cases that people come up with (mostly where people manually interact with the blockchain),
it eludes the server side (backend, fully automated, devops) use cases where JavaScript is
usually not the language of choice given its dynamic nature.
This page introduces the concept of server side native Dapps: Go language bindings to any
Ethereum contract that is compile time type safe, highly performant and best of all, can
be generated fully automatically from a contract ABI and optionally the EVM bytecode.
*This page is written in a more beginner friendly tutorial style to make it easier for
people to start out with writing Go native Dapps. The used concepts will be introduced
gradually as a developer would need/encounter them. However, we do assume the reader
is familiar with Ethereum in general, has a fair understanding of Solidity and can code
Go.*
## Token contract
To avoid falling into the fallacy of useless academic examples, we're going to take the
official [Token contract](https://ethereum.org/token) as the base for introducing the Go
native bindings. If you're unfamiliar with the contract, skimming the linked page should
probably be enough, the details aren't relevant for now. *In short the contract implements
a custom token that can be deployed on top of Ethereum.* To make sure this tutorial doesn't
go stale if the linked website changes, the Solidity source code of the Token contract is
also available at [`token.sol`](https://gist.github.com/karalabe/08f4b780e01c8452d989).
### Go binding generator
Interacting with a contract on the Ethereum blockchain from Go (or any other language for
a matter of fact) is already possible via the RPC interfaces exposed by Ethereum clients.
However, writing the boilerplate code that translates decent Go language constructs into
RPC calls and back is extremely time consuming and also extremely brittle: implementation
bugs can only be detected during runtime and it's almost impossible to evolve a contract
as even a tiny change in Solidity can be painful to port over to Go.
To avoid all this mess, the go-ethereum implementation introduces a source code generator
that can convert Ethereum ABI definitions into easy to use, type-safe Go packages. Assuming
you have a valid Go development environment set up, `godep` installed and the go-ethereum
repository checked out correctly, you can build the generator with:
```
$ cd $GOPATH/src/github.com/ethereum/go-ethereum
$ godep go install ./cmd/abigen
```
### Generating the bindings
The single essential thing needed to generate a Go binding to an Ethereum contract is the
contract's ABI definition `JSON` file. For our `Token` contract tutorial you can obtain this
either by compiling the Solidity code yourself (e.g. via @chriseth's [online Solidity compiler](https://chriseth.github.io/browser-solidity/)), or you can download our pre-compiled [`token.abi`](https://gist.github.com/karalabe/b8dfdb6d301660f56c1b).
To generate a binding, simply call:
```
$ abigen --abi token.abi --pkg main --type Token --out token.go
```
Where the flags are:
* `--abi`: Mandatory path to the contract ABI to bind to
* `--pgk`: Mandatory Go package name to place the Go code into
* `--type`: Optional Go type name to assign to the binding struct
* `--out`: Optional output path for the generated Go source file (not set = stdout)
This will generate a type-safe Go binding for the Token contract. The generated code will
look something like [`token.go`](https://gist.github.com/karalabe/5839509295afa4f7e2215bc4116c7a8f),
but please generate your own as this will change as more work is put into the generator.
### Accessing an Ethereum contract
To interact with a contract deployed on the blockchain, you'll need to know the `address`
of the contract itself, and need to specify a `backend` through which to access Ethereum.
The binding generator provides out of the box an RPC backend through which you can attach
to an existing Ethereum node via IPC, HTTP or WebSockets.
We'll use the foundation's [Unicorn](https://ethereum.org/donate) token contract deployed
on the testnet to demonstrate calling contract methods. It is deployed at the address
`0x21e6fc92f93c8a1bb41e2be64b4e1f88a54d3576`.
To run the snippet below, please ensure a Geth instance is running and attached to the
Morden test network where the above mentioned contract was deployed. Also please update
the path to the IPC socket below to the one reported by your own local Geth node.
```go
package main
import (
"fmt"
"log"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/ethclient"
)
func main() {
// Create an IPC based RPC connection to a remote node
conn, err := ethclient.Dial("/home/karalabe/.ethereum/testnet/geth.ipc")
if err != nil {
log.Fatalf("Failed to connect to the Ethereum client: %v", err)
}
// Instantiate the contract and display its name
token, err := NewToken(common.HexToAddress("0x21e6fc92f93c8a1bb41e2be64b4e1f88a54d3576"), conn)
if err != nil {
log.Fatalf("Failed to instantiate a Token contract: %v", err)
}
name, err := token.Name(nil)
if err != nil {
log.Fatalf("Failed to retrieve token name: %v", err)
}
fmt.Println("Token name:", name)
}
```
And the output (yay):
```
Token name: Testnet Unicorn
```
If you look at the method invoked to read the token name `token.Name(nil)`, it required
a parameter to be passed, even though the original Solidity contract requires none. This
is a `*bind.CallOpts` type, which can be used to fine tune the call.
* `Pending`: Whether to access pending contract state or the current stable one
* `GasLimit`: Place a limit on the computing resources the call might consume
### Transacting with an Ethereum contract
Invoking a method that changes contract state (i.e. transacting) is a bit more involved,
as a live transaction needs to be authorized and broadcast into the network. **Opposed
to the conventional way of storing accounts and keys in the node we attach to, Go bindings
require signing transactions locally and do not delegate this to a remote node.** This is
done so to facilitate the general direction of the Ethereum community where accounts are
kept private to DApps, and not shared (by default) between them.
Thus to allow transacting with a contract, your code needs to implement a method that
given an input transaction, signs it and returns an authorized output transaction. Since
most users have their keys in the [Web3 Secret Storage](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) format, the `bind` package contains a small utility method
(`bind.NewTransactor(keyjson, passphrase)`) that can create an authorized transactor from
a key file and associated password, without the user needing to implement key signing himself.
Changing the previous code snippet to send one unicorn to the zero address:
```go
package main
import (
"fmt"
"log"
"math/big"
"strings"
"github.com/ethereum/go-ethereum/accounts/abi/bind"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/ethclient"
)
const key = `paste the contents of your *testnet* key json here`
func main() {
// Create an IPC based RPC connection to a remote node and instantiate a contract binding
conn, err := ethclient.Dial("/home/karalabe/.ethereum/testnet/geth.ipc")
if err != nil {
log.Fatalf("Failed to connect to the Ethereum client: %v", err)
}
token, err := NewToken(common.HexToAddress("0x21e6fc92f93c8a1bb41e2be64b4e1f88a54d3576"), conn)
if err != nil {
log.Fatalf("Failed to instantiate a Token contract: %v", err)
}
// Create an authorized transactor and spend 1 unicorn
auth, err := bind.NewTransactor(strings.NewReader(key), "my awesome super secret password")
if err != nil {
log.Fatalf("Failed to create authorized transactor: %v", err)
}
tx, err := token.Transfer(auth, common.HexToAddress("0x0000000000000000000000000000000000000000"), big.NewInt(1))
if err != nil {
log.Fatalf("Failed to request token transfer: %v", err)
}
fmt.Printf("Transfer pending: 0x%x\n", tx.Hash())
}
```
And the output (yay):
```
Transfer pending: 0x4f4aaeb29ed48e88dd653a81f0b05d4df64a86c99d4e83b5bfeb0f0006b0e55b
```
*Note, with high probability you won't have any testnet unicorns available to spend, so the
above program will fail with an error. Send at least 2.014 testnet(!) Ethers to the foundation
testnet tipjar `0xDf7D0030bfed998Db43288C190b63470c2d18F50` to receive a unicorn token and
you'll be able to see the above code run without an error!*
Similar to the method invocations in the previous section which only read contract state,
transacting methods also require a mandatory first parameter, a `*bind.TransactOpts` type,
which authorizes the transaction and potentially fine tunes it:
* `From`: Address of the account to invoke the method with (mandatory)
* `Signer`: Method to sign a transaction locally before broadcasting it (mandatory)
* `Nonce`: Account nonce to use for the transaction ordering (optional)
* `GasLimit`: Place a limit on the computing resources the call might consume (optional)
* `GasPrice`: Explicitly set the gas price to run the transaction with (optional)
* `Value`: Any funds to transfer along with the method call (optional)
The two mandatory fields are automatically set by the `bind` package if the auth options are
constructed using `bind.NewTransactor`. The nonce and gas related fields are automatically
derived by the binding if they are not set. An unset value is assumed to be zero.
### Pre-configured contract sessions
As mentioned in the previous two sections, both reading as well as state modifying contract
calls require a mandatory first parameter which can both authorize as well as fine tune some
of the internal parameters. However, most of the time we want to use the same parameters and
issue transactions with the same account, so always constructing the call/transact options or
passing them along with the binding can become unwieldy.
To avoid these scenarios, the generator also creates specialized wrappers that can be pre-
configured with tuning and authorization parameters, allowing all the Solidity defined methods
to be invoked without needing an extra parameter.
These are named analogous to the original contract type name, just suffixed with `Sessions`:
```go
// Wrap the Token contract instance into a session
session := &TokenSession{
Contract: token,
CallOpts: bind.CallOpts{
Pending: true,
},
TransactOpts: bind.TransactOpts{
From: auth.From,
Signer: auth.Signer,
GasLimit: big.NewInt(3141592),
},
}
// Call the previous methods without the option parameters
session.Name()
session.Transfer("0x0000000000000000000000000000000000000000"), big.NewInt(1))
```
### Deploying contracts to Ethereum
Interacting with existing contracts is nice, but let's take it up a notch and deploy
a brand new contract onto the Ethereum blockchain! To do so however, the contract ABI
we used to generate the binding is not enough. We need the compiled bytecode too to
allow deploying it.
To get the bytecode, either go back to the online compiler with which you may generate it,
or alternatively download our [`token.bin`](https://gist.github.com/karalabe/026548f6a5f5f97b54de).
You'll need to rerun the Go generator with the bytecode included for it to create deploy
code too:
```
$ abigen --abi token.abi --pkg main --type Token --out token.go --bin token.bin
```
This will generate something similar to [`token.go`](https://gist.github.com/karalabe/2153b087c1f80f651fd87dd4c439fac4).
If you quickly skim this file, you'll find an extra `DeployToken` function that was just
injected compared to the previous code. Beside all the parameters specified by Solidity,
it also needs the usual authorization options to deploy the contract with and the Ethereum
backend to deploy the contract through.
Putting it all together would result in:
```go
package main
import (
"fmt"
"log"
"math/big"
"strings"
"time"
"github.com/ethereum/go-ethereum/accounts/abi/bind"
"github.com/ethereum/go-ethereum/ethclient"
)
const key = `paste the contents of your *testnet* key json here`
func main() {
// Create an IPC based RPC connection to a remote node and an authorized transactor
conn, err := rpc.NewIPCClient("/home/karalabe/.ethereum/testnet/geth.ipc")
if err != nil {
log.Fatalf("Failed to connect to the Ethereum client: %v", err)
}
auth, err := bind.NewTransactor(strings.NewReader(key), "my awesome super secret password")
if err != nil {
log.Fatalf("Failed to create authorized transactor: %v", err)
}
// Deploy a new awesome contract for the binding demo
address, tx, token, err := DeployToken(auth, conn), new(big.Int), "Contracts in Go!!!", 0, "Go!")
if err != nil {
log.Fatalf("Failed to deploy new token contract: %v", err)
}
fmt.Printf("Contract pending deploy: 0x%x\n", address)
fmt.Printf("Transaction waiting to be mined: 0x%x\n\n", tx.Hash())
// Don't even wait, check its presence in the local pending state
time.Sleep(250 * time.Millisecond) // Allow it to be processed by the local node :P
name, err := token.Name(&bind.CallOpts{Pending: true})
if err != nil {
log.Fatalf("Failed to retrieve pending name: %v", err)
}
fmt.Println("Pending name:", name)
}
```
And the code performs as expected: it requests the creation of a brand new Token contract
on the Ethereum blockchain, which we can either wait for to be mined or as in the above code
start calling methods on it in the pending state :)
```
Contract pending deploy: 0x46506d900559ad005feb4645dcbb2dbbf65e19cc
Transaction waiting to be mined: 0x6a81231874edd2461879b7280ddde1a857162a744e3658ca7ec276984802183b
Pending name: Contracts in Go!!!
```
## Bind Solidity directly
If you've followed the tutorial along until this point you've probably realized that
every contract modification needs to be recompiled, the produced ABIs and bytecodes
(especially if you need multiple contracts) individually saved to files and then the
binding executed for them. This can become a quite bothersome after the Nth iteration,
so the `abigen` command supports binding from Solidity source files directly (`--sol`),
which first compiles the source code (via `--solc`, defaulting to `solc`) into it's
constituent components and binds using that.
Binding the official Token contract [`token.sol`](https://gist.github.com/karalabe/08f4b780e01c8452d989)
would then entail to running:
```
$ abigen --sol token.sol --pkg main --out token.go
```
*Note: Building from Solidity (`--sol`) is mutually exclusive with individually setting
the bind components (`--abi`, `--bin` and `--type`), as all of them are extracted from
the Solidity code and produced build results directly.*
Building a contract directly from Solidity has the nice side effect that all contracts
contained within a Solidity source file are built and bound, so if your file contains many
contract sources, each and every one of them will be available from Go code. The sample
Token solidity file results in [`token.go`](https://gist.github.com/karalabe/c22aab73194ba7da834ab5b379621031).
### Project integration (i.e. `go generate`)
The `abigen` command was made in such a way as to play beautifully together with existing
Go toolchains: instead of having to remember the exact command needed to bind an Ethereum
contract into a Go project, we can leverage `go generate` to remember all the nitty-gritty
details.
Place the binding generation command into a Go source file before the package definition:
```
//go:generate abigen --sol token.sol --pkg main --out token.go
```
After which whenever the Solidity contract is modified, instead of needing to remember and
run the above command, we can simply call `go generate` on the package (or even the entire
source tree via `go generate ./...`), and it will correctly generate the new bindings for us.
## Blockchain simulator
Being able to deploy and access already deployed Ethereum contracts from within native Go
code is an extremely powerful feature, but there is one facet with developing native code
that not even the testnet lends itself well to: *automatic unit testing*. Using go-ethereum
internal constructs it's possible to create test chains and verify them, but it is unfeasible
to do high level contract testing with such low level mechanisms.
To sort out this last issue that would make it hard to run (and test) native DApps, we've also
implemented a *simulated blockchain*, that can be set as a backend to native contracts the same
way as a live RPC backend could be: `backends.NewSimulatedBackend(genesisAccounts)`.
```go
package main
import (
"fmt"
"log"
"math/big"
"github.com/ethereum/go-ethereum/accounts/abi/bind"
"github.com/ethereum/go-ethereum/accounts/abi/bind/backends"
"github.com/ethereum/go-ethereum/core"
"github.com/ethereum/go-ethereum/crypto"
)
func main() {
// Generate a new random account and a funded simulator
key, _ := crypto.GenerateKey()
auth := bind.NewKeyedTransactor(key)
sim := backends.NewSimulatedBackend(core.GenesisAccount{Address: auth.From, Balance: big.NewInt(10000000000)})
// Deploy a token contract on the simulated blockchain
_, _, token, err := DeployMyToken(auth, sim, new(big.Int), "Simulated blockchain tokens", 0, "SBT")
if err != nil {
log.Fatalf("Failed to deploy new token contract: %v", err)
}
// Print the current (non existent) and pending name of the contract
name, _ := token.Name(nil)
fmt.Println("Pre-mining name:", name)
name, _ = token.Name(&bind.CallOpts{Pending: true})
fmt.Println("Pre-mining pending name:", name)
// Commit all pending transactions in the simulator and print the names again
sim.Commit()
name, _ = token.Name(nil)
fmt.Println("Post-mining name:", name)
name, _ = token.Name(&bind.CallOpts{Pending: true})
fmt.Println("Post-mining pending name:", name)
}
```
And the output (yay):
```
Pre-mining name:
Pre-mining pending name: Simulated blockchain tokens
Post-mining name: Simulated blockchain tokens
Post-mining pending name: Simulated blockchain tokens
```
Note, that we don't have to wait for a local private chain miner, or testnet miner to
integrate the currently pending transactions. When we decide to mine the next block,
we simply `Commit()` the simulator.

119
docs/Native/Account-management.md Normal file
View file

@ -0,0 +1,119 @@
To provide Ethereum integration for your native applications, the very first thing you should be interested in doing is account management.
Although all current leading Ethereum implementations provide account management built in, it is ill advised to keep accounts in any location that is shared between multiple applications and/or multiple people. The same way you do not entrust your ISP (who is after all your gateway into the internet) with your login credentials; you should not entrust an Ethereum node (who is your gateway into the Ethereum network) with your credentials either.
The proper way to handle user accounts in your native applications is to do client side account management, everything self-contained within your own application. This way you can ensure as fine grained (or as coarse) access permissions to the sensitive data as deemed necessary, without relying on any third party application's functionality and/or vulnerabilities.
To support this, `go-ethereum` provides a simple, yet thorough accounts package that gives you all the tools to do properly secured account management via encrypted keystores and passphrase protected accounts. You can leverage all the security of the `go-ethereum` crypto implementation while at the same time running everything in your own application.
## Encrypted keystores
Although handling accounts locally to an application does provide certain security guarantees, access keys to Ethereum accounts should never lay around in clear-text form. As such, we provide an encrypted keystore that provides the proper security guarantees for you without requiring a thorough understanding from your part of the associated cryptographic primitives.
The important thing to know when using the encrypted keystore is that the cryptographic primitives used within can operate either in *standard* or *light* mode. The former provides a higher level of security at the cost of increased computational burden and resource consumption:
* *standard* needs 256MB memory and 1 second processing on a modern CPU to access a key
* *light* needs 4MB memory and 100 millisecond processing on a modern CPU to access a key
As such, *standard* is more suitable for native applications, but you should be aware of the trade-offs nonetheless in case you you're targeting more resource constrained environments.
*For those interested in the cryptographic and/or implementation details, the key-store uses the `secp256k1` elliptic curve as defined in the [Standards for Efficient Cryptography](http://www.secg.org/sec2-v2.pdf), implemented by the [`libsecp256k`](https://github.com/bitcoin-core/secp256k1) library and wrapped by [`github.com/ethereum/go-ethereum/accounts`](https://godoc.org/github.com/ethereum/go-ethereum/accounts). Accounts are stored on disk in the [Web3 Secret Storage](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) format.*
### Keystores from Go
The encrypted keystore is implemented by the [`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager) struct from the [`github.com/ethereum/go-ethereum/accounts`](https://godoc.org/github.com/ethereum/go-ethereum/accounts) package, which also contains the configuration constants for the *standard* or *light* security modes described above. Hence to do client side account management from Go, you'll need to import only the `accounts` package into your code:
```go
import "github.com/ethereum/go-ethereum/accounts"
```
Afterwards you can create a new encrypted account manager via:
```go
am := accounts.NewManager("/path/to/keystore", accounts.StandardScryptN, accounts.StandardScryptP);
```
The path to the keystore folder needs to be a location that is writable by the local user but non-readable for other system users (for security reasons obviously), so we'd recommend placing it either inside your user's home directory or even more locked down for backend applications.
The last two arguments of [`accounts.NewManager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#NewManager) are the crypto parameters defining how resource-intensive the keystore encryption should be. You can choose between [`accounts.StandardScryptN, accounts.StandardScryptP`, `accounts.LightScryptN, accounts.LightScryptP`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#pkg-constants) or specify your own numbers (please make sure you understand the underlying cryptography for this). We recommend using the *standard* version.
## Account lifecycle
Having created an encrypted keystore for your Ethereum accounts, you can use this account manager for the entire account lifecycle requirements of your native application. This includes the basic functionality of creating new accounts and deleting existing ones; as well as the more advanced functionality of updating access credentials, exporting existing accounts, and importing them on another device.
Although the keystore defines the encryption strength it uses to store your accounts, there is no global master password that can grant access to all of them. Rather each account is maintained individually, and stored on disk in its [encrypted format](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) individually, ensuring a much cleaner and stricter separation of credentials.
This individuality however means that any operation requiring access to an account will need to provide the necessary authentication credentials for that particular account in the form of a passphrase:
* When creating a new account, the caller must supply a passphrase to encrypt the account with. This passphrase will be required for any subsequent access, the lack of which will forever forfeit using the newly created account.
* When deleting an existing account, the caller must supply a passphrase to verify ownership of the account. This isn't cryptographically necessary, rather a protective measure against accidental loss of accounts.
* When updating an existing account, the caller must supply both current and new passphrases. After completing the operation, the account will not be accessible via the old passphrase any more.
* When exporting an existing account, the caller must supply both the current passphrase to decrypt the account, as well as an export passphrase to re-encrypt it with before returning the key-file to the user. This is required to allow moving accounts between machines and applications without sharing original credentials.
* When importing a new account, the caller must supply both the encryption passphrase of the key-file being imported, as well as a new passhprase with which to store the account. This is required to allow storing account with different credentials than used for moving them around.
*Please note, there is no recovery mechanisms for losing the passphrases. The cryptographic properties of the encrypted keystore (if using the provided parameters) guarantee that account credentials cannot be brute forced in any meaningful time.*
### Accounts from Go
An Ethereum account is implemented by the [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) struct from the [`github.com/ethereum/go-ethereum/accounts`](https://godoc.org/github.com/ethereum/go-ethereum/accounts) package. Assuming we already have an instance of an [`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager) called `am` from the previous section, we can easily execute all of the described lifecycle operations with a handful of function calls (error handling omitted).
```go
// Create a new account with the specified encryption passphrase.
newAcc, _ := am.NewAccount("Creation password");
// Export the newly created account with a different passphrase. The returned
// data from this method invocation is a JSON encoded, encrypted key-file.
jsonAcc, _ := am.Export(newAcc, "Creation password", "Export password");
// Update the passphrase on the account created above inside the local keystore.
am.Update(newAcc, "Creation password", "Update password");
// Delete the account updated above from the local keystore.
am.Delete(newAcc, "Update password");
// Import back the account we've exported (and then deleted) above with yet
// again a fresh passphrase.
impAcc, _ := am.Import(jsonAcc, "Export password", "Import password");
```
*Although instances of [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) can be used to access various information about specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or private keys), rather act solely as identifiers for client code and the keystore.*
## Signing authorization
As mentioned above, account objects do not hold the sensitive private keys of the associated Ethereum accounts, but are merely placeholders to identify the cryptographic keys with. All operations that require authorization (e.g. transaction signing) are performed by the account manager after granting it access to the private keys.
There are a few different ways one can authorize the account manager to execute signing operations, each having its advantages and drawbacks. Since the different methods have wildly different security guarantees, it is essential to be clear on how each works:
* **Single authorization**: The simplest way to sign a transaction via the account manager is to provide the passphrase of the account every time something needs to be signed, which will ephemerally decrypt the private key, execute the signing operation and immediately throw away the decrypted key. The drawbacks are that the passphrase needs to be queried from the user every time, which can become annoying if done frequently; or the application needs to keep the passphrase in memory, which can have security consequences if not done properly; and depending on the keystore's configured strength, constantly decrypting keys can result in non-negligible resource requirements.
* **Multiple authorizations**: A more complex way of signing transactions via the account manager is to unlock the account via its passphrase once, and allow the account manager to cache the decrypted private key, enabling all subsequent signing requests to complete without the passphrase. The lifetime of the cached private key may be managed manually (by explicitly locking the account back up) or automatically (by providing a timeout during unlock). This mechanism is useful for scenarios where the user may need to sign many transactions or the application would need to do so without requiring user input. The crucial aspect to remember is that **anyone with access to the account manager can sign transactions while a particular account is unlocked** (e.g. application running untrusted code).
*Note, creating transactions is out of scope here, so the remainder of this section will assume we already have a transaction hash to sign, and will focus only on creating a cryptographic signature authorizing it. Creating an actual transaction and injecting the authorization signature into it will be covered later.*
### Signing from Go
Assuming we already have an instance of an [`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager) called `am` from the previous sections, we can create a new account to sign transactions with via it's already demonstrated [`NewAccount`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.NewAccount) method; and to avoid going into transaction creation for now, we can hard-code a random [`common.Hash`](https://godoc.org/github.com/ethereum/go-ethereum/common#Hash) to sign instead.
```go
// Create a new account to sign transactions with
signer, _ := am.NewAccount("Signer password");
txHash := common.HexToHash("0x0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef");
```
With the boilerplate out of the way, we can now sign transaction using the authorization mechanisms described above:
```go
// Sign a transaction with a single authorization
signature, _ := am.SignWithPassphrase(signer, "Signer password", txHash.Bytes());
// Sign a transaction with multiple manually cancelled authorizations
am.Unlock(signer, "Signer password");
signature, _ = am.Sign(signer.Address, txHash.Bytes());
am.Lock(signer.Address);
// Sign a transaction with multiple automatically cancelled authorizations
am.TimedUnlock(signer, "Signer password", time.Second);
signature, _ = am.Sign(signer.Address, txHash.Bytes());
```
You may wonder why [`SignWithPassphrase`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.SignWithPassphrase) takes an [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) as the signer, whereas [`Sign`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.Sign) takes only a [`common.Address`](https://godoc.org/github.com/ethereum/go-ethereum/common#Address). The reason is that an [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) object may also contain a custom key-path, allowing [`SignWithPassphrase`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.SignWithPassphrase) to sign using accounts outside of the keystore; however [`Sign`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.Sign) relies on accounts already unlocked within the keystore, so it cannot specify custom paths.

40
docs/Native/Introduction.md Normal file
View file

@ -0,0 +1,40 @@
The Ethereum blockchain along with its two extension protocols Whisper and Swarm was originally conceptualized to become the supporting pillar of web3, providing the consensus, messaging and storage backbone for a new generation of distributed (actually, decentralized) applications called DApps.
The first incarnation towards this dream of web3 was a command line client providing an RPC interface into the peer-to-peer protocols. The client was soon enough extended with a web-browser-like graphical user interface, permitting developers to write DApps based on the tried and proven HTML/CSS/JS technologies.
As many DApps have more complex requirements than what a browser environment can handle, it became apparent that providing programmatic access to the web3 pillars would open the door towards a new class of applications. As such, the second incarnation of the web3 dream is to open up all our technologies for other projects as reusable components.
Starting with the 1.5 release family of `go-ethereum`, we transitioned away from providing only a full blown Ethereum client and started shipping official Go packages that could be embedded into third party desktop and server applications.
*Note, this guide will assume you are familiar with Go development. It will make no attempts to cover general topics about Go project layouts, import paths or any other standard methodologies. If you are new to Go, consider reading its [getting started guides](https://github.com/golang/go/wiki#getting-started-with-go) first.*
## Quick overview
Our reusable Go libraries focus on four main usage areas:
- Simplified client side account management
- Remote node interfacing via different transports
- Contract interactions through auto-generated bindings
- In-process Ethereum, Whisper and Swarm peer-to-peer node
You can watch a quick overview about these in Peter's (@karalabe) talk titled "Import Geth: Ethereum from Go and beyond", presented at the Ethereum Devcon2 developer conference in September, 2016 (Shanghai). Slides are [available here](https://ethereum.karalabe.com/talks/2016-devcon.html).
[![Peter's Devcon2 talk](https://img.youtube.com/vi/R0Ia1U9Gxjg/0.jpg)](https://www.youtube.com/watch?v=R0Ia1U9Gxjg)
## Go packages
The `go-ethereum` library is distributed as a collection of standard Go packages straight from our GitHub repository. The packages can be used directly via the official Go toolkit, without needing any third party tools. External dependencies are vendored locally into `vendor`, ensuring both self-containment as well as code stability. If you reuse `go-ethereum` in your own project, please follow these best practices and vendor it yourself too to avoid any accidental API breakages!
The canonical import path for `go-ethereum` is `github.com/ethereum/go-ethereum`, with all packages residing underneath. Although there are [quite a number](https://godoc.org/github.com/ethereum/go-ethereum#pkg-subdirectories) of them, you'll only need to care about a limited subset, each of which will be properly introduced in their relevant section.
You can download all our packages via:
```
$ go get -d github.com/ethereum/go-ethereum/...
```
You may also need Go's original context package. Although this was moved into the official Go SDK in Go 1.7, `go-ethereum` will depend on the original `golang.org/x/net/context` package until we officially drop support for Go 1.5 and Go 1.6.
```
$ go get -u golang.org/x/net/context
```

138
docs/Tracing/Introduction.md Normal file
View file

@ -0,0 +1,138 @@
There are two different types of transactions in Ethereum: plain value transfers and contract executions. A plain value transfer just moves Ether from one account to another and as such is uninteresting from this guide's perspective. If however the recipient of a transaction is a contract account with associated EVM (Ethereum Virtual Machine) bytecode - beside transferring any Ether - the code will also be executed as part of the transaction.
Having code associated with Ethereum accounts permits transactions to do arbitrarilly complex data storage and enables them to act on the previously stored data by further transacting internally with outside accounts and contracts. This creates an intertwined ecosystem of contracts, where a single transaction can interact with tens or hunderds of accounts.
The downside of contract execution is that it is very hard to say what a transaction actually did. A transaction receipt does contain a status code to check whether execution succeeded or not, but there's no way to see what data was modified, nor what external contracts where invoked. In order to introspect a transaction, we need to trace its execution.
## Tracing prerequisites
In its simplest form, tracing a transaction entails requesting the Ethereum node to reexecute the desired transaction with varying degrees of data collection and have it return the aggregated summary for post processing. Reexecuting a transaction however has a few prerequisites to be met.
In order for an Ethereum node to reexecute a transaction, it needs to have available all historical state accessed by the transaction:
* Balance, nonce, bytecode and storage of both the recipient as well as all internally invoked contracts.
* Block metadata referenced during execution of both the outer as well as all internally created transactions.
* Intermediate state generated by all preceding transactions contained in the same block as the one being traced.
Depending on your node's mode of synchronization and pruning, different configurations result in different capabilities:
* An **archive** node retaining **all historical data** can trace arbitrary transactions at any point in time. Tracing a single transaction also entails reexecuting all preceding transactions in the same block.
* A **fast synced** node retaining **all historical data** after initial sync can only trace transactions from blocks following the initial sync point. Tracing a single transaction also entails reexecuting all preceding transactions in the same block.
* A **fast synced** node retaining only **periodic state data** after initial sync can only trace transactions from blocks following the initial sync point. Tracing a single transaction entails reexecuting all preceding transactions **both** in the same block, as well as all preceding blocks until the previous stored snapshot.
* A **light synced** node retrieving data **on demand** can in theory trace transactions for which all required historical state is readily available in the network. In practice, data availability is **not** a feasible assumption.
*There are exceptions to the above rules when running batch traces of entire blocks or chain segments. Those will be detailed later.*
## Basic traces
The simplest type of transaction trace that `go-ethereum` can generate are raw EVM opcode traces. For every VM instruction the transaction executes, a structured log entry is emitted, containing all contextual metadata deemed useful. This includes the *program counter*, *opcode name*, *opcode cost*, *remaining gas*, *execution depth* and any *occurred error*. The structured logs can optionally also contain the content of the *execution stack*, *execution memory* and *contract storage*.
An example log entry for a single opcode looks like:
```json
{
"pc": 48,
"op": "DIV",
"gasCost": 5,
"gas": 64532,
"depth": 1,
"error": null,
"stack": [
"00000000000000000000000000000000000000000000000000000000ffffffff",
"0000000100000000000000000000000000000000000000000000000000000000",
"2df07fbaabbe40e3244445af30759352e348ec8bebd4dd75467a9f29ec55d98d"
],
"memory": [
"0000000000000000000000000000000000000000000000000000000000000000",
"0000000000000000000000000000000000000000000000000000000000000000",
"0000000000000000000000000000000000000000000000000000000000000060"
],
"storage": {
}
}
```
The entire output of an raw EVM opcode trace is a JSON object having a few metadata fields: *consumed gas*, *failure status*, *return value*; and a list of *opcode entries* that take the above form:
```json
{
"gas": 25523,
"failed": false,
"returnValue": "",
"structLogs": []
}
```
### Generating basic traces
To generate a raw EVM opcode trace, `go-ethereum` provides a few [RPC API endpoints](https://github.com/ethereum/go-ethereum/wiki/Management-APIs), out of which the most commonly used is [`debug_traceTransaction`](https://github.com/ethereum/go-ethereum/wiki/Management-APIs#debug_tracetransaction).
In its simplest form, `traceTransaction` accepts a transaction hash as its sole argument, traces the transaction, aggregates all the generated data and returns it as a **large** JSON object. A sample invocation from the Geth console would be:
```js
debug.traceTransaction("0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f")
```
The same call can of course be invoked from outside the node too via HTTP RPC. In this case, please make sure the HTTP endpoint is enabled via `--rpc` and the `debug` API namespace exposed via `--rpcapi=debug`.
```
$ curl -H "Content-Type: application/json" -d '{"id": 1, "method": "debug_traceTransaction", "params": ["0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f"]}' localhost:8545
```
Running the above operation on the Rinkeby network (with a node retaining enough history) will result in this [trace dump](https://gist.github.com/karalabe/c91f95ac57f5e57f8b950ec65ecc697f).
### Tuning basic traces
By default the raw opcode tracer emits all relevant events that occur within the EVM while processing a transaction, such as *EVM stack*, *EVM memory* and *updated storage slots*. Certain use cases however may not need some of these data fields reported. To cater for those use cases, these massive fields may be omitted using a second *options* parameter for the tracer:
```json
{
"disableStack": true,
"disableMemory": true,
"disableStorage": true
}
```
Running the previous tracer invocation from the Geth console with the data fields disabled:
```js
debug.traceTransaction("0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f", {disableStack: true, disableMemory: true, disableStorage: true})
```
Analogously running the filtered tracer from outside the node too via HTTP RPC:
```
$ curl -H "Content-Type: application/json" -d '{"id": 1, "method": "debug_traceTransaction", "params": ["0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f", {"disableStack": true, "disableMemory": true, "disableStorage": true}]}' localhost:8545
```
Running the above operation on the Rinkeby network will result in this significantly shorter [trace dump](https://gist.github.com/karalabe/d74a7cb33a70f2af75e7824fc772c5b4).
### Limits of basic traces
Although the raw opcode traces we've generated above have their use, this basic way of tracing is problematic in the real world. Having an individual log entry for every single opcode is too low level for most use cases, and will require developers to create additional tools to post-process the traces. Additionally, a full opcode trace can easily go into the hundreds of megabytes, making them very resource intensive to get out of the node and process externally.
To avoid all of the previously mentioned issues, `go-ethereum` supports running custom JavaScript tracers *within* the Ethereum node, which have full access to the EVM stack, memory and contract storage. This permits developers to only gather the data they need, and do any processing **at** the data. Please see the next section for our *custom in-node tracers*.
### Pruning
Geth by default does in-memory pruning of state, discarding state entries that it deems is no longer necessary to maintain. This is configured via the `--gcmode` option. Often, people run into the error that state is not available.
Say you want to do a trace on block `B`. Now there are a couple of cases:
1. You have done a fast-sync, pivot block `P` where `P <= B`.
2. You have done a fast-sync, pivot block `P` where `P > B`.
3. You have done a full-sync, with pruning
4. You have done a full-sync, without pruning (`--gcmode=archive`)
Here's what happens in each respective case:
1. Geth will regenerate the desired state by replaying blocks from the closest point in time before `B` where it has full state. This defaults to [`128`](https://github.com/ethereum/go-ethereum/blob/master/eth/api_tracer.go#L52) blocks max, but you can specify more in the actual call `... "reexec":1000 .. }` to the tracer.
2. Sorry, can't be done without replaying from genesis.
3. Same as 1)
4. Does not need to replay anything, can immediately load up the state and serve the request.
There is one other option available to you, which may or may not suit your needs. That is to use [Evmlab]( https://github.com/holiman/evmlab).
```
docker pull holiman/evmlab && docker run -it holiman/evmlab
```
There you can use the reproducer. The reproducer will incrementally fetch data from infura until it has all the information required to create the trace locally on an evm which is bundled with the image. It will create a custom genesis containing the state that the transaction touches (balances, code, nonce etc). It should be mentioned that the evmlab reproducer is strictly guaranteed to be totally exact with regards to gascosts incurred by the outer transaction, as evmlab does not fully calculate the gascosts for nonzero data etc, but is usually sufficient to analyze contracts and events.