forks/go-ethereum
1
0
Fork 1
mirror of https://github.com/ethereum/go-ethereum.git synced 2026-08-20 10:52:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

move go, mobile, tracing docs into dapp category

Browse source
This commit is contained in:
Felix Lange 2019-10-31 12:37:45 +01:00
parent 771ad7e92d
commit be778966f7
11 changed files with 1045 additions and 631 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

324
docs/_dapp/mobile-accounts.md Normal file
View file

@ -0,0 +1,324 @@
---
title: Mobile Account Management
---
To provide Ethereum integration for your mobile applications, the very first thing you
should be interested in doing is account management.
Although all current leading Ethereum implementations provide account management built in,
it is ill advised to keep accounts in any location that is shared between multiple
applications and/or multiple people. The same way you do not entrust your ISP (who is
after all your gateway into the internet) with your login credentials; you should not
entrust an Ethereum node (who is your gateway into the Ethereum network) with your
credentials either.
The proper way to handle user accounts in your mobile applications is to do client side
account management, everything self-contained within your own application. This way you
can ensure as fine grained (or as coarse) access permissions to the sensitive data as
deemed necessary, without relying on any third party application's functionality and/or
vulnerabilities.
To support this, `go-ethereum` provides a simple, yet thorough accounts library that gives
you all the tools to do properly secured account management via encrypted keystores and
passphrase protected accounts. You can leverage all the security of the `go-ethereum`
crypto implementation while at the same time running everything in your own application.
## Encrypted keystores
Although handling your users' accounts locally on their own mobile device does provide
certain security guarantees, access keys to Ethereum accounts should never lay around in
clear-text form. As such, we provide an encrypted keystore that provides the proper
security guarantees for you without requiring a thorough understanding from your part of
the associated cryptographic primitives.
The important thing to know when using the encrypted keystore is that the cryptographic
primitives used within can operate either in *standard* or *light* mode. The former
provides a higher level of security at the cost of increased computational burden and
resource consumption:
* *standard* needs 256MB memory and 1 second processing on a modern CPU to access a key
* *light* needs 4MB memory and 100 millisecond processing on a modern CPU to access a key
As such, *light* is more suitable for mobile applications, but you should be aware of the
trade-offs nonetheless.
*For those interested in the cryptographic and/or implementation details, the key-store
uses the `secp256k1` elliptic curve as defined in the [Standards for Efficient
Cryptography](sec2), implemented by the [`libsecp256k`](secp256k1) library and wrapped by
[`github.com/ethereum/go-ethereum/accounts`](accounts-go). Accounts are stored on disk in
the [Web3 Secret Storage](secstore) format.*
[sec2]: http://www.secg.org/sec2-v2.pdf
[accounts-go]: https://godoc.org/github.com/ethereum/go-ethereum/accounts
[secp256k1]: https://github.com/bitcoin-core/secp256k1
[secstore]: https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition
### Keystores on Android (Java)
The encrypted keystore on Android is implemented by the `KeyStore` class from the
`org.ethereum.geth` package. The configuration constants (for the *standard* or *light*
security modes described above) are located in the `Geth` abstract class, similarly from
the `org.ethereum.geth` package. Hence to do client side account management on Android,
you'll need to import two classes into your Java code:
```java
import org.ethereum.geth.Geth;
import org.ethereum.geth.KeyStore;
```
Afterwards you can create a new encrypted keystore via:
```java
KeyStore ks = new KeyStore("/path/to/keystore", Geth.LightScryptN, Geth.LightScryptP);
```
The path to the keystore folder needs to be a location that is writable by the local
mobile application but non-readable for other installed applications (for security reasons
obviously), so we'd recommend placing it inside your app's data directory. If you are
creating the `KeyStore` from within a class extending an Android object, you will most
probably have access to the `Context.getFilesDir()` method via `this.getFilesDir()`, so
you could set the keystore path to `this.getFilesDir() + "/keystore"`.
The last two arguments of the `KeyStore` constructor are the crypto parameters defining
how resource-intensive the keystore encryption should be. You can choose between
`Geth.StandardScryptN, Geth.StandardScryptP`, `Geth.LightScryptN, Geth.LightScryptP` or
specify your own numbers (please make sure you understand the underlying cryptography for
this). We recommend using the *light* version.
### Keystores on iOS (Swift 3)
The encrypted keystore on iOS is implemented by the `GethKeyStore` class from the `Geth`
framework. The configuration constants (for the *standard* or *light* security modes
described above) are located in the same namespace as global variables. Hence to do client
side account management on iOS, you'll need to import the framework into your Swift code:
```swift
import Geth
```
Afterwards you can create a new encrypted account manager via:
```swift
let ks = GethNewKeyStore("/path/to/keystore", GethLightScryptN, GethLightScryptP);
```
The path to the keystore folder needs to be a location that is writable by the local
mobile application but non-readable for other installed applications (for security reasons
obviously), so we'd recommend placing it inside your app's document directory. You should
be able to retrieve the document directory via `let datadir =
NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]`, so you
could set the keystore path to `datadir + "/keystore"`.
The last two arguments of the `GethNewKeyStore` factory method are the crypto parameters
defining how resource-intensive the keystore encryption should be. You can choose between
`GethStandardScryptN, GethStandardScryptP`, `GethLightScryptN, GethLightScryptP` or
specify your own numbers (please make sure you understand the underlying cryptography for
this). We recommend using the *light* version.
## Account lifecycle
Having created an encrypted keystore for your Ethereum accounts, you can use this for the
entire account lifecycle requirements of your mobile application. This includes the basic
functionality of creating new accounts and deleting existing ones; as well as the more
advanced functionality of updating access credentials, exporting existing accounts, and
importing them on another device.
Although the keystore defines the encryption strength it uses to store your accounts,
there is no global master password that can grant access to all of them. Rather each
account is maintained individually, and stored on disk in its [encrypted
format](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition)
individually, ensuring a much cleaner and stricter separation of credentials.
This individuality however means that any operation requiring access to an account will
need to provide the necessary authentication credentials for that particular account in
the form of a passphrase:
* When creating a new account, the caller must supply a passphrase to encrypt the account
with. This passphrase will be required for any subsequent access, the lack of which
will forever forfeit using the newly created account.
* When deleting an existing account, the caller must supply a passphrase to verify
ownership of the account. This isn't cryptographically necessary, rather a protective
measure against accidental loss of accounts.
* When updating an existing account, the caller must supply both current and new
passphrases. After completing the operation, the account will not be accessible via the
old passphrase any more.
* When exporting an existing account, the caller must supply both the current passphrase
to decrypt the account, as well as an export passphrase to re-encrypt it with before
returning the key-file to the user. This is required to allow moving accounts between
devices without sharing original credentials.
* When importing a new account, the caller must supply both the encryption passphrase of
the key-file being imported, as well as a new passhprase with which to store the
account. This is required to allow storing account with different credentials than used
for moving them around.
*Please note, there is no recovery mechanisms for losing the passphrases. The
cryptographic properties of the encrypted keystore (if using the provided parameters)
guarantee that account credentials cannot be brute forced in any meaningful time.*
### Accounts on Android (Java)
An Ethereum account on Android is implemented by the `Account` class from the
`org.ethereum.geth` package. Assuming we already have an instance of a `KeyStore` called
`ks` from the previous section, we can easily execute all of the described lifecycle
operations with a handful of function calls.
```java
// Create a new account with the specified encryption passphrase.
Account newAcc = ksm.newAccount("Creation password");
// Export the newly created account with a different passphrase. The returned
// data from this method invocation is a JSON encoded, encrypted key-file.
byte[] jsonAcc = ks.exportKey(newAcc, "Creation password", "Export password");
// Update the passphrase on the account created above inside the local keystore.
ks.updateAccount(newAcc, "Creation password", "Update password");
// Delete the account updated above from the local keystore.
ks.deleteAccount(newAcc, "Update password");
// Import back the account we've exported (and then deleted) above with yet
// again a fresh passphrase.
Account impAcc = ks.importKey(jsonAcc, "Export password", "Import password");
```
*Although instances of `Account` can be used to access various information about specific
Ethereum accounts, they do not contain any sensitive data (such as passphrases or private
keys), rather act solely as identifiers for client code and the keystore.*
### Accounts on iOS (Swift 3)
An Ethereum account on iOS is implemented by the `GethAccount` class from the `Geth`
framework. Assuming we already have an instance of a `GethKeyStore` called `ks` from the
previous section, we can easily execute all of the described lifecycle operations with a
handful of function calls.
```swift
// Create a new account with the specified encryption passphrase.
let newAcc = try! ks?.newAccount("Creation password")
// Export the newly created account with a different passphrase. The returned
// data from this method invocation is a JSON encoded, encrypted key-file.
let jsonKey = try! ks?.exportKey(newAcc!, passphrase: "Creation password", newPassphrase: "Export password")
// Update the passphrase on the account created above inside the local keystore.
try! ks?.update(newAcc, passphrase: "Creation password", newPassphrase: "Update password")
// Delete the account updated above from the local keystore.
try! ks?.delete(newAcc, passphrase: "Update password")
// Import back the account we've exported (and then deleted) above with yet
// again a fresh passphrase.
let impAcc = try! ks?.importKey(jsonKey, passphrase: "Export password", newPassphrase: "Import password")
```
*Although instances of `GethAccount` can be used to access various information about
specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or
private keys), rather act solely as identifiers for client code and the keystore.*
## Signing authorization
As mentioned above, account objects do not hold the sensitive private keys of the
associated Ethereum accounts, but are merely placeholders to identify the cryptographic
keys with. All operations that require authorization (e.g. transaction signing) are
performed by the account manager after granting it access to the private keys.
There are a few different ways one can authorize the account manager to execute signing
operations, each having its advantages and drawbacks. Since the different methods have
wildly different security guarantees, it is essential to be clear on how each works:
* **Single authorization**: The simplest way to sign a transaction via the keystore is to
provide the passphrase of the account every time something needs to be signed, which
will ephemerally decrypt the private key, execute the signing operation and immediately
throw away the decrypted key. The drawbacks are that the passphrase needs to be queried
from the user every time, which can become annoying if done frequently; or the
application needs to keep the passphrase in memory, which can have security
consequences if not done properly; and depending on the keystore's configured strength,
constantly decrypting keys can result in non-negligible resource requirements.
* **Multiple authorizations**: A more complex way of signing transactions via the
keystore is to unlock the account via its passphrase once, and allow the account
manager to cache the decrypted private key, enabling all subsequent signing requests to
complete without the passphrase. The lifetime of the cached private key may be managed
manually (by explicitly locking the account back up) or automatically (by providing a
timeout during unlock). This mechanism is useful for scenarios where the user may need
to sign many transactions or the application would need to do so without requiring user
input. The crucial aspect to remember is that **anyone with access to the account
manager can sign transactions while a particular account is unlocked** (e.g. device
left unattended; application running untrusted code).
*Note, creating transactions is out of scope here, so the remainder of this section will
assume we already have a transaction to sign, and will focus only on creating an
authorized version of it. Creating an actually meaningful transaction will be covered
later.*
### Signing on Android (Java)
Assuming we already have an instance of a `KeyStore` called `ks` from the previous
sections, we can create a new account to sign transactions with via it's already
demonstrated `newAccount` method; and to avoid going into transaction creation for now, we
can hard-code a random transaction to sign instead.
```java
// Create a new account to sign transactions with
Account signer = ks.newAccount("Signer password");
Transaction tx = new Transaction(
1, new Address("0x0000000000000000000000000000000000000000"),
new BigInt(0), new BigInt(0), new BigInt(1), null); // Random empty transaction
BigInt chain = new BigInt(1); // Chain identifier of the main net
```
With the boilerplate out of the way, we can now sign transaction using the authorization
mechanisms described above:
```java
// Sign a transaction with a single authorization
Transaction signed = ks.signTxPassphrase(signer, "Signer password", tx, chain);
// Sign a transaction with multiple manually cancelled authorizations
ks.unlock(signer, "Signer password");
signed = ks.signTx(signer, tx, chain);
ks.lock(signer.getAddress());
// Sign a transaction with multiple automatically cancelled authorizations
ks.timedUnlock(signer, "Signer password", 1000000000);
signed = ks.signTx(signer, tx, chain);
```
### Signing on iOS (Swift 3)
Assuming we already have an instance of a `GethKeyStore` called `ks` from the previous
sections, we can create a new account to sign transactions with via it's already
demonstrated `newAccount` method; and to avoid going into transaction creation for now, we
can hard-code a random transaction to sign instead.
```swift
// Create a new account to sign transactions with
var error: NSError?
let signer = try! ks?.newAccount("Signer password")
let to = GethNewAddressFromHex("0x0000000000000000000000000000000000000000", &error)
let tx = GethNewTransaction(1, to, GethNewBigInt(0), GethNewBigInt(0), GethNewBigInt(0), nil) // Random empty transaction
let chain = GethNewBigInt(1) // Chain identifier of the main net
```
*Note, although Swift usually rewrites `NSError` returns to throws, this particular
instance seems to have been missed for some reason (possibly due to it being a
constructor). It will be fixed in a later version of the iOS bindings when the appropriate
fixed are implemented upstream in the `gomobile` project.*
With the boilerplate out of the way, we can now sign transaction using the authorization
methods described above:
```swift
// Sign a transaction with a single authorization
var signed = try! ks?.signTxPassphrase(signer, passphrase: "Signer password", tx: tx, chainID: chain)
// Sign a transaction with multiple manually cancelled authorizations
try! ks?.unlock(signer, passphrase: "Signer password")
signed = try! ks?.signTx(signer, tx: tx, chainID: chain)
try! ks?.lock(signer?.getAddress())
// Sign a transaction with multiple automatically cancelled authorizations
try! ks?.timedUnlock(signer, passphrase: "Signer password", timeout: 1000000000)
signed = try! ks?.signTx(signer, tx: tx, chainID: chain)
```

179
docs/_dapp/mobile.md Normal file
View file

@ -0,0 +1,179 @@
---
title: Mobile API
---
The Ethereum blockchain along with its two extension protocols Whisper and Swarm was
originally conceptualized to become the supporting pillar of web3, providing the
consensus, messaging and storage backbone for a new generation of distributed (actually,
decentralized) applications called DApps.
The first incarnation towards this dream of web3 was a command line client providing an
RPC interface into the peer-to-peer protocols. The client was soon enough extended with a
web-browser-like graphical user interface, permitting developers to write DApps based on
the tried and proven HTML/CSS/JS technologies.
As many DApps have more complex requirements than what a browser environment can handle,
it became apparent that providing programmatic access to the web3 pillars would open the
door towards a new class of applications. As such, the second incarnation of the web
dream is to open up all our technologies for other projects as reusable components.
Starting with the 1.5 release family of `go-ethereum`, we transitioned away from providing
only a full blown Ethereum client and started shipping official Go packages that could be
embedded into third party desktop and server applications. It took only a small leap from
here to begin porting our code to mobile platforms.
## Quick overview
Similarly to our reusable Go libraries, the mobile wrappers also focus on four main usage
areas:
- Simplified client side account management
- Remote node interfacing via different transports
- Contract interactions through auto-generated bindings
- In-process Ethereum, Whisper and Swarm peer-to-peer node
You can watch a quick overview about these in Peter's (@karalabe) talk titled "Import
Geth: Ethereum from Go and beyond", presented at the Ethereum Devcon2 developer conference
in September, 2016 (Shanghai). Slides are [available
here](https://ethereum.karalabe.com/talks/2016-devcon.html).
[![Peter's Devcon2 talk](https://img.youtube.com/vi/R0Ia1U9Gxjg/0.jpg)](https://www.youtube.com/watch?v=R0Ia1U9Gxjg)
## Library bundles
The `go-ethereum` mobile library is distributed either as an Android `.aar` archive
(containing binaries for `arm-7`, `arm64`, `x86` and `x64`); or as an iOS XCode framework
(containing binaries for `arm-7`, `arm64` and `x86`). We do not provide library bundles
for Windows phone the moment.
### Android archive
The simplest way to use `go-ethereum` in your Android project is through a Maven
dependency. We provide bundles of all our stable releases (starting from v1.5.0) through
Maven Central, and also provide the latest develop bundle through the Sonatype OSS
repository.
#### Stable dependency (Maven Central)
To add an Android dependency to the **stable** library release of `go-ethereum`, you'll
need to ensure that the Maven Central repository is enabled in your Android project, and
that the `go-ethereum` code is listed as a required dependency of your application. You
can do both of these by editing the `build.gradle` script in your Android app's folder:
```gradle
repositories {
mavenCentral()
}
dependencies {
// All your previous dependencies
compile 'org.ethereum:geth:1.5.2' // Change the version to the latest release
}
```
#### Develop dependency (Sonatype)
To add an Android dependency to the current version of `go-ethereum`, you'll need to
ensure that the Sonatype snapshot repository is enabled in your Android project, and that
the `go-ethereum` code is listed as a required `SNAPSHOT` dependency of your application.
You can do both of these by editing the `build.gradle` script in your Android app's
folder:
```gradle
repositories {
maven {
url "https://oss.sonatype.org/content/groups/public"
}
}
dependencies {
// All your previous dependencies
compile 'org.ethereum:geth:1.5.3-SNAPSHOT' // Change the version to the latest release
}
```
#### Custom dependency
If you prefer not to depend on Maven Central or Sonatype; or would like to access an older
develop build not available any more as an online dependency, you can download any bundle
directly from [our website](https://geth.ethereum.org/downloads/) and insert it into your
project in Android Studio via `File -> New -> New module... -> Import .JAR/.AAR Package`.
You will also need to configure `gradle` to link the mobile library bundle to your
application. This can be done by adding a new entry to the `dependencies` section of your
`build.gradle` script, pointing it to the module you just added (named `geth` by default).
```gradle
dependencies {
// All your previous dependencies
compile project(':geth')
}
```
#### Manual builds
Lastly, if you would like to make modifications to the `go-ethereum` mobile code and/or
build it yourself locally instead of downloading a pre-built bundle, you can do so using a
`make` command. This will create an Android archive called `geth.aar` in the `build/bin`
folder that you can import into your Android Studio as described above.
```bash
$ make android
[...]
Done building.
Import "build/bin/geth.aar" to use the library.
```
### iOS framework
The simplest way to use `go-ethereum` in your iOS project is through a
[CocoaPods](https://cocoapods.org/) dependency. We provide bundles of all our stable
releases (starting from v1.5.3) and also latest develop versions.
#### Automatic dependency
To add an iOS dependency to the current stable or latest develop version of `go-ethereum`,
you'll need to ensure that your iOS XCode project is configured to use CocoaPods.
Detailing that is out of scope in this document, but you can find a guide in the upstream
[Using CocoaPods](https://guides.cocoapods.org/using/using-cocoapods.html) page.
Afterwards you can edit your `Podfile` to list `go-ethereum` as a dependency:
```ruby
target 'MyApp' do
# All your previous dependencies
pod 'Geth', '1.5.4' # Change the version to the latest release
end
```
Alternatively, if you'd like to use the latest develop version, replace the package
version `1.5.4` with `~> 1.5.5-unstable` to switch to pre-releases and to always pull in
the latest bundle from a particular release family.
#### Custom dependency
If you prefer not to depend on CocoaPods; or would like to access an older develop build
not available any more as an online dependency, you can download any bundle directly from
[our website](https://geth.ethereum.org/downloads/) and insert it into your project in
XCode via `Project Settings -> Build Phases -> Link Binary With Libraries`.
Do not forget to extract the framework from the compressed `.tar.gz` archive. You can do
that either using a GUI tool or from the command line via (replace the archive with your
downloaded file):
```
tar -zxvf geth-ios-all-1.5.3-unstable-e05d35e6.tar.gz
```
#### Manual builds
Lastly, if you would like to make modifications to the `go-ethereum` mobile code and/or
build it yourself locally instead of downloading a pre-built bundle, you can do so using a
`make` command. This will create an iOS XCode framework called `Geth.framework` in the
`build/bin` folder that you can import into XCode as described above.
```bash
$ make ios
[...]
Done building.
Import "build/bin/Geth.framework" to use the library.
```

252
docs/_dapp/native-accounts.md Normal file
View file

@ -0,0 +1,252 @@
---
title: Go Account Management
---
To provide Ethereum integration for your native applications, the very first thing you
should be interested in doing is account management.
Although all current leading Ethereum implementations provide account management built in,
it is ill advised to keep accounts in any location that is shared between multiple
applications and/or multiple people. The same way you do not entrust your ISP (who is
after all your gateway into the internet) with your login credentials; you should not
entrust an Ethereum node (who is your gateway into the Ethereum network) with your
credentials either.
The proper way to handle user accounts in your native applications is to do client side
account management, everything self-contained within your own application. This way you
can ensure as fine grained (or as coarse) access permissions to the sensitive data as
deemed necessary, without relying on any third party application's functionality and/or
vulnerabilities.
To support this, `go-ethereum` provides a simple, yet thorough accounts package that gives
you all the tools to do properly secured account management via encrypted keystores and
passphrase protected accounts. You can leverage all the security of the `go-ethereum`
crypto implementation while at the same time running everything in your own application.
## Encrypted keystores
Although handling accounts locally to an application does provide certain security
guarantees, access keys to Ethereum accounts should never lay around in clear-text form.
As such, we provide an encrypted keystore that provides the proper security guarantees for
you without requiring a thorough understanding from your part of the associated
cryptographic primitives.
The important thing to know when using the encrypted keystore is that the cryptographic
primitives used within can operate either in *standard* or *light* mode. The former
provides a higher level of security at the cost of increased computational burden and
resource consumption:
* *standard* needs 256MB memory and 1 second processing on a modern CPU to access a key
* *light* needs 4MB memory and 100 millisecond processing on a modern CPU to access a key
As such, *standard* is more suitable for native applications, but you should be aware of
the trade-offs nonetheless in case you you're targeting more resource constrained
environments.
*For those interested in the cryptographic and/or implementation details, the key-store
uses the `secp256k1` elliptic curve as defined in the [Standards for Efficient
Cryptography](sec2), implemented by the [`libsecp256k`](secp256k1) library and wrapped by
[`github.com/ethereum/go-ethereum/accounts`](accounts-go). Accounts are stored on disk in
the [Web3 Secret Storage](secstore) format.*
[sec2]: http://www.secg.org/sec2-v2.pdf
[accounts-go]: https://godoc.org/github.com/ethereum/go-ethereum/accounts
[secp256k1]: https://github.com/bitcoin-core/secp256k1
[secstore]: https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition
### Keystores from Go
The encrypted keystore is implemented by the
[`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager)
struct from the
[`github.com/ethereum/go-ethereum/accounts`](https://godoc.org/github.com/ethereum/go-ethereum/accounts)
package, which also contains the configuration constants for the *standard* or *light*
security modes described above. Hence to do client side account management from Go, you'll
need to import only the `accounts` package into your code:
```go
import "github.com/ethereum/go-ethereum/accounts"
```
Afterwards you can create a new encrypted account manager via:
```go
am := accounts.NewManager("/path/to/keystore", accounts.StandardScryptN, accounts.StandardScryptP);
```
The path to the keystore folder needs to be a location that is writable by the local user
but non-readable for other system users (for security reasons obviously), so we'd
recommend placing it either inside your user's home directory or even more locked down for
backend applications.
The last two arguments of
[`accounts.NewManager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#NewManager)
are the crypto parameters defining how resource-intensive the keystore encryption should
be. You can choose between [`accounts.StandardScryptN, accounts.StandardScryptP`,
`accounts.LightScryptN,
accounts.LightScryptP`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#pkg-constants)
or specify your own numbers (please make sure you understand the underlying cryptography
for this). We recommend using the *standard* version.
## Account lifecycle
Having created an encrypted keystore for your Ethereum accounts, you can use this account
manager for the entire account lifecycle requirements of your native application. This
includes the basic functionality of creating new accounts and deleting existing ones; as
well as the more advanced functionality of updating access credentials, exporting existing
accounts, and importing them on another device.
Although the keystore defines the encryption strength it uses to store your accounts,
there is no global master password that can grant access to all of them. Rather each
account is maintained individually, and stored on disk in its [encrypted
format](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition)
individually, ensuring a much cleaner and stricter separation of credentials.
This individuality however means that any operation requiring access to an account will
need to provide the necessary authentication credentials for that particular account in
the form of a passphrase:
* When creating a new account, the caller must supply a passphrase to encrypt the account
with. This passphrase will be required for any subsequent access, the lack of which
will forever forfeit using the newly created account.
* When deleting an existing account, the caller must supply a passphrase to verify
ownership of the account. This isn't cryptographically necessary, rather a protective
measure against accidental loss of accounts.
* When updating an existing account, the caller must supply both current and new
passphrases. After completing the operation, the account will not be accessible via the
old passphrase any more.
* When exporting an existing account, the caller must supply both the current passphrase
to decrypt the account, as well as an export passphrase to re-encrypt it with before
returning the key-file to the user. This is required to allow moving accounts between
machines and applications without sharing original credentials.
* When importing a new account, the caller must supply both the encryption passphrase of
the key-file being imported, as well as a new passhprase with which to store the
account. This is required to allow storing account with different credentials than used
for moving them around.
*Please note, there is no recovery mechanisms for losing the passphrases. The
cryptographic properties of the encrypted keystore (if using the provided parameters)
guarantee that account credentials cannot be brute forced in any meaningful time.*
### Accounts from Go
An Ethereum account is implemented by the
[`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account)
struct from the
[`github.com/ethereum/go-ethereum/accounts`](https://godoc.org/github.com/ethereum/go-ethereum/accounts)
package. Assuming we already have an instance of an
[`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager)
called `am` from the previous section, we can easily execute all of the described
lifecycle operations with a handful of function calls (error handling omitted).
```go
// Create a new account with the specified encryption passphrase.
newAcc, _ := am.NewAccount("Creation password");
// Export the newly created account with a different passphrase. The returned
// data from this method invocation is a JSON encoded, encrypted key-file.
jsonAcc, _ := am.Export(newAcc, "Creation password", "Export password");
// Update the passphrase on the account created above inside the local keystore.
am.Update(newAcc, "Creation password", "Update password");
// Delete the account updated above from the local keystore.
am.Delete(newAcc, "Update password");
// Import back the account we've exported (and then deleted) above with yet
// again a fresh passphrase.
impAcc, _ := am.Import(jsonAcc, "Export password", "Import password");
```
*Although instances of
[`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account)
can be used to access various information about specific Ethereum accounts, they do not
contain any sensitive data (such as passphrases or private keys), rather act solely as
identifiers for client code and the keystore.*
## Signing authorization
As mentioned above, account objects do not hold the sensitive private keys of the
associated Ethereum accounts, but are merely placeholders to identify the cryptographic
keys with. All operations that require authorization (e.g. transaction signing) are
performed by the account manager after granting it access to the private keys.
There are a few different ways one can authorize the account manager to execute signing
operations, each having its advantages and drawbacks. Since the different methods have
wildly different security guarantees, it is essential to be clear on how each works:
* **Single authorization**: The simplest way to sign a transaction via the account
manager is to provide the passphrase of the account every time something needs to be
signed, which will ephemerally decrypt the private key, execute the signing operation
and immediately throw away the decrypted key. The drawbacks are that the passphrase
needs to be queried from the user every time, which can become annoying if done
frequently; or the application needs to keep the passphrase in memory, which can have
security consequences if not done properly; and depending on the keystore's configured
strength, constantly decrypting keys can result in non-negligible resource
requirements.
* **Multiple authorizations**: A more complex way of signing transactions via the account
manager is to unlock the account via its passphrase once, and allow the account manager
to cache the decrypted private key, enabling all subsequent signing requests to
complete without the passphrase. The lifetime of the cached private key may be managed
manually (by explicitly locking the account back up) or automatically (by providing a
timeout during unlock). This mechanism is useful for scenarios where the user may need
to sign many transactions or the application would need to do so without requiring user
input. The crucial aspect to remember is that **anyone with access to the account
manager can sign transactions while a particular account is unlocked** (e.g.
application running untrusted code).
*Note, creating transactions is out of scope here, so the remainder of this section will
assume we already have a transaction hash to sign, and will focus only on creating a
cryptographic signature authorizing it. Creating an actual transaction and injecting the
authorization signature into it will be covered later.*
### Signing from Go
Assuming we already have an instance of an
[`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager)
called `am` from the previous sections, we can create a new account to sign transactions
with via it's already demonstrated
[`NewAccount`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.NewAccount)
method; and to avoid going into transaction creation for now, we can hard-code a random
[`common.Hash`](https://godoc.org/github.com/ethereum/go-ethereum/common#Hash) to sign
instead.
```go
// Create a new account to sign transactions with
signer, _ := am.NewAccount("Signer password");
txHash := common.HexToHash("0x0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef");
```
With the boilerplate out of the way, we can now sign transaction using the authorization
mechanisms described above:
```go
// Sign a transaction with a single authorization
signature, _ := am.SignWithPassphrase(signer, "Signer password", txHash.Bytes());
// Sign a transaction with multiple manually cancelled authorizations
am.Unlock(signer, "Signer password");
signature, _ = am.Sign(signer.Address, txHash.Bytes());
am.Lock(signer.Address);
// Sign a transaction with multiple automatically cancelled authorizations
am.TimedUnlock(signer, "Signer password", time.Second);
signature, _ = am.Sign(signer.Address, txHash.Bytes());
```
You may wonder why
[`SignWithPassphrase`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.SignWithPassphrase)
takes an
[`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account)
as the signer, whereas
[`Sign`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.Sign) takes
only a
[`common.Address`](https://godoc.org/github.com/ethereum/go-ethereum/common#Address). The
reason is that an
[`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account)
object may also contain a custom key-path, allowing
[`SignWithPassphrase`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.SignWithPassphrase)
to sign using accounts outside of the keystore; however
[`Sign`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.Sign) relies
on accounts already unlocked within the keystore, so it cannot specify custom paths.

3
docs/_developers/Go-bindings-to-Ethereum-contracts.md → docs/_dapp/native-bindings.md
View file

@ -1,6 +1,7 @@
--- ---
title: Native DApps / Go bindings to Ethereum contracts title: Go Contract Bindings
--- ---
**[Please note, events are not yet implemented as they need some RPC subscription **[Please note, events are not yet implemented as they need some RPC subscription
features that are still under review.]** features that are still under review.]**

72
docs/_dapp/native.md Normal file
View file

@ -0,0 +1,72 @@
---
title: Go API
---
The Ethereum blockchain along with its two extension protocols Whisper and Swarm was
originally conceptualized to become the supporting pillar of web3, providing the
consensus, messaging and storage backbone for a new generation of distributed (actually,
decentralized) applications called DApps.
The first incarnation towards this dream of web3 was a command line client providing an
RPC interface into the peer-to-peer protocols. The client was soon enough extended with a
web-browser-like graphical user interface, permitting developers to write DApps based on
the tried and proven HTML/CSS/JS technologies.
As many DApps have more complex requirements than what a browser environment can handle,
it became apparent that providing programmatic access to the web3 pillars would open the
door towards a new class of applications. As such, the second incarnation of the web3
dream is to open up all our technologies for other projects as reusable components.
Starting with the 1.5 release family of `go-ethereum`, we transitioned away from providing
only a full blown Ethereum client and started shipping official Go packages that could be
embedded into third party desktop and server applications.
*Note, this guide will assume you are familiar with Go development. It will make no
attempts to cover general topics about Go project layouts, import paths or any other
standard methodologies. If you are new to Go, consider reading its [getting started
guides](https://github.com/golang/go/wiki#getting-started-with-go) first.*
## Quick overview
Our reusable Go libraries focus on four main usage areas:
- Simplified client side account management
- Remote node interfacing via different transports
- Contract interactions through auto-generated bindings
- In-process Ethereum, Whisper and Swarm peer-to-peer node
You can watch a quick overview about these in Peter's (@karalabe) talk titled "Import
Geth: Ethereum from Go and beyond", presented at the Ethereum Devcon2 developer conference
in September, 2016 (Shanghai). Slides are [available
here](https://ethereum.karalabe.com/talks/2016-devcon.html).
[![Peter's Devcon2 talk](https://img.youtube.com/vi/R0Ia1U9Gxjg/0.jpg)](https://www.youtube.com/watch?v=R0Ia1U9Gxjg)
## Go packages
The `go-ethereum` library is distributed as a collection of standard Go packages straight
from our GitHub repository. The packages can be used directly via the official Go toolkit,
without needing any third party tools. External dependencies are vendored locally into
`vendor`, ensuring both self-containment as well as code stability. If you reuse
`go-ethereum` in your own project, please follow these best practices and vendor it
yourself too to avoid any accidental API breakages!
The canonical import path for `go-ethereum` is `github.com/ethereum/go-ethereum`, with all
packages residing underneath. Although there are [quite a
number](https://godoc.org/github.com/ethereum/go-ethereum#pkg-subdirectories) of them,
you'll only need to care about a limited subset, each of which will be properly introduced
in their relevant section.
You can download all our packages via:
```
$ go get -d github.com/ethereum/go-ethereum/...
```
You may also need Go's original context package. Although this was moved into the official
Go SDK in Go 1.7, `go-ethereum` will depend on the original `golang.org/x/net/context`
package until we officially drop support for Go 1.5 and Go 1.6.
```
$ go get -u golang.org/x/net/context
```

216
docs/_dapp/tracing.md Normal file
View file

@ -0,0 +1,216 @@
---
title: EVM Tracing
---
There are two different types of transactions in Ethereum: plain value transfers and
contract executions. A plain value transfer just moves Ether from one account to another
and as such is uninteresting from this guide's perspective. If however the recipient of a
transaction is a contract account with associated EVM (Ethereum Virtual Machine)
bytecode - beside transferring any Ether - the code will also be executed as part of the
transaction.
Having code associated with Ethereum accounts permits transactions to do arbitrarilly
complex data storage and enables them to act on the previously stored data by further
transacting internally with outside accounts and contracts. This creates an intertwined
ecosystem of contracts, where a single transaction can interact with tens or hunderds of
accounts.
The downside of contract execution is that it is very hard to say what a transaction
actually did. A transaction receipt does contain a status code to check whether execution
succeeded or not, but there's no way to see what data was modified, nor what external
contracts where invoked. In order to introspect a transaction, we need to trace its
execution.
## Tracing prerequisites
In its simplest form, tracing a transaction entails requesting the Ethereum node to
reexecute the desired transaction with varying degrees of data collection and have it
return the aggregated summary for post processing. Reexecuting a transaction however has a
few prerequisites to be met.
In order for an Ethereum node to reexecute a transaction, it needs to have available all
historical state accessed by the transaction:
* Balance, nonce, bytecode and storage of both the recipient as well as all internally invoked contracts.
* Block metadata referenced during execution of both the outer as well as all internally created transactions.
* Intermediate state generated by all preceding transactions contained in the same block as the one being traced.
Depending on your node's mode of synchronization and pruning, different configurations
result in different capabilities:
* An **archive** node retaining **all historical data** can trace arbitrary transactions
at any point in time. Tracing a single transaction also entails reexecuting all
preceding transactions in the same block.
* A **fast synced** node retaining **all historical data** after initial sync can only
trace transactions from blocks following the initial sync point. Tracing a single
transaction also entails reexecuting all preceding transactions in the same block.
* A **fast synced** node retaining only **periodic state data** after initial sync can
only trace transactions from blocks following the initial sync point. Tracing a single
transaction entails reexecuting all preceding transactions **both** in the same block,
as well as all preceding blocks until the previous stored snapshot.
* A **light synced** node retrieving data **on demand** can in theory trace transactions
for which all required historical state is readily available in the network. In
practice, data availability is **not** a feasible assumption.
*There are exceptions to the above rules when running batch traces of entire blocks or
chain segments. Those will be detailed later.*
## Basic traces
The simplest type of transaction trace that `go-ethereum` can generate are raw EVM opcode
traces. For every VM instruction the transaction executes, a structured log entry is
emitted, containing all contextual metadata deemed useful. This includes the *program
counter*, *opcode name*, *opcode cost*, *remaining gas*, *execution depth* and any
*occurred error*. The structured logs can optionally also contain the content of the
*execution stack*, *execution memory* and *contract storage*.
An example log entry for a single opcode looks like:
```json
{
"pc": 48,
"op": "DIV",
"gasCost": 5,
"gas": 64532,
"depth": 1,
"error": null,
"stack": [
"00000000000000000000000000000000000000000000000000000000ffffffff",
"0000000100000000000000000000000000000000000000000000000000000000",
"2df07fbaabbe40e3244445af30759352e348ec8bebd4dd75467a9f29ec55d98d"
],
"memory": [
"0000000000000000000000000000000000000000000000000000000000000000",
"0000000000000000000000000000000000000000000000000000000000000000",
"0000000000000000000000000000000000000000000000000000000000000060"
],
"storage": {
}
}
```
The entire output of an raw EVM opcode trace is a JSON object having a few metadata
fields: *consumed gas*, *failure status*, *return value*; and a list of *opcode entries*
that take the above form:
```json
{
"gas": 25523,
"failed": false,
"returnValue": "",
"structLogs": []
}
```
### Generating basic traces
To generate a raw EVM opcode trace, `go-ethereum` provides a few [RPC API
endpoints](debug-api), out of which the most commonly used is
[`debug_traceTransaction`](trace-tx).
In its simplest form, `traceTransaction` accepts a transaction hash as its sole argument,
traces the transaction, aggregates all the generated data and returns it as a **large**
JSON object. A sample invocation from the Geth console would be:
```js
debug.traceTransaction("0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f")
```
The same call can of course be invoked from outside the node too via HTTP RPC. In this
case, please make sure the HTTP endpoint is enabled via `--rpc` and the `debug` API
namespace exposed via `--rpcapi=debug`.
```
$ curl -H "Content-Type: application/json" -d '{"id": 1, "method": "debug_traceTransaction", "params": ["0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f"]}' localhost:8545
```
Running the above operation on the Rinkeby network (with a node retaining enough history)
will result in this [trace dump](rinkeby-example-trace-big).
### Tuning basic traces
By default the raw opcode tracer emits all relevant events that occur within the EVM while
processing a transaction, such as *EVM stack*, *EVM memory* and *updated storage slots*.
Certain use cases however may not need some of these data fields reported. To cater for
those use cases, these massive fields may be omitted using a second *options* parameter
for the tracer:
```json
{
"disableStack": true,
"disableMemory": true,
"disableStorage": true
}
```
Running the previous tracer invocation from the Geth console with the data fields
disabled:
```js
debug.traceTransaction("0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f", {disableStack: true, disableMemory: true, disableStorage: true})
```
Analogously running the filtered tracer from outside the node too via HTTP RPC:
```
$ curl -H "Content-Type: application/json" -d '{"id": 1, "method": "debug_traceTransaction", "params": ["0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f", {"disableStack": true, "disableMemory": true, "disableStorage": true}]}' localhost:8545
```
Running the above operation on the Rinkeby network will result in this significantly
shorter [trace dump](rinkeby-example-trace).
### Limits of basic traces
Although the raw opcode traces we've generated above have their use, this basic way of
tracing is problematic in the real world. Having an individual log entry for every single
opcode is too low level for most use cases, and will require developers to create
additional tools to post-process the traces. Additionally, a full opcode trace can easily
go into the hundreds of megabytes, making them very resource intensive to get out of the
node and process externally.
To avoid all of the previously mentioned issues, `go-ethereum` supports running custom
JavaScript tracers *within* the Ethereum node, which have full access to the EVM stack,
memory and contract storage. This permits developers to only gather the data they need,
and do any processing **at** the data. Please see the next section for our *custom in-node
tracers*.
### Pruning
Geth by default does in-memory pruning of state, discarding state entries that it deems is
no longer necessary to maintain. This is configured via the `--gcmode` option. Often,
people run into the error that state is not available.
Say you want to do a trace on block `B`. Now there are a couple of cases:
1. You have done a fast-sync, pivot block `P` where `P <= B`.
2. You have done a fast-sync, pivot block `P` where `P > B`.
3. You have done a full-sync, with pruning
4. You have done a full-sync, without pruning (`--gcmode=archive`)
Here's what happens in each respective case:
1. Geth will regenerate the desired state by replaying blocks from the closest point ina
time before `B` where it has full state. This defaults to `128` blocks max, but you can
specify more in the actual call `... "reexec":1000 .. }` to the tracer.
2. Sorry, can't be done without replaying from genesis.
3. Same as 1)
4. Does not need to replay anything, can immediately load up the state and serve the request.
There is one other option available to you, which may or may not suit your needs. That is
to use [Evmlab](evmlab).
docker pull holiman/evmlab && docker run -it holiman/evmlab
There you can use the reproducer. The reproducer will incrementally fetch data from infura
until it has all the information required to create the trace locally on an evm which is
bundled with the image. It will create a custom genesis containing the state that the
transaction touches (balances, code, nonce etc). It should be mentioned that the evmlab
reproducer is strictly guaranteed to be totally exact with regards to gascosts incurred by
the outer transaction, as evmlab does not fully calculate the gascosts for nonzero data
etc, but is usually sufficient to analyze contracts and events.
[evmlab]: https://github.com/holiman/evmlab
[rinkeby-example-trace]: https://gist.github.com/karalabe/d74a7cb33a70f2af75e7824fc772c5b4
[rinkeby-example-trace-big]: https://gist.github.com/karalabe/c91f95ac57f5e57f8b950ec65ecc697f
[debug-api]: ../rpc/ns-debug
[trace-tx]: ../rpc/ns-debug#debug_tracetransaction

141
docs/_developers/Tracing_Introduction.md
View file

@ -1,141 +0,0 @@
---
title: Tracing / Introduction
---
There are two different types of transactions in Ethereum: plain value transfers and contract executions. A plain value transfer just moves Ether from one account to another and as such is uninteresting from this guide's perspective. If however the recipient of a transaction is a contract account with associated EVM (Ethereum Virtual Machine) bytecode - beside transferring any Ether - the code will also be executed as part of the transaction.
Having code associated with Ethereum accounts permits transactions to do arbitrarilly complex data storage and enables them to act on the previously stored data by further transacting internally with outside accounts and contracts. This creates an intertwined ecosystem of contracts, where a single transaction can interact with tens or hunderds of accounts.
The downside of contract execution is that it is very hard to say what a transaction actually did. A transaction receipt does contain a status code to check whether execution succeeded or not, but there's no way to see what data was modified, nor what external contracts where invoked. In order to introspect a transaction, we need to trace its execution.
## Tracing prerequisites
In its simplest form, tracing a transaction entails requesting the Ethereum node to reexecute the desired transaction with varying degrees of data collection and have it return the aggregated summary for post processing. Reexecuting a transaction however has a few prerequisites to be met.
In order for an Ethereum node to reexecute a transaction, it needs to have available all historical state accessed by the transaction:
* Balance, nonce, bytecode and storage of both the recipient as well as all internally invoked contracts.
* Block metadata referenced during execution of both the outer as well as all internally created transactions.
* Intermediate state generated by all preceding transactions contained in the same block as the one being traced.
Depending on your node's mode of synchronization and pruning, different configurations result in different capabilities:
* An **archive** node retaining **all historical data** can trace arbitrary transactions at any point in time. Tracing a single transaction also entails reexecuting all preceding transactions in the same block.
* A **fast synced** node retaining **all historical data** after initial sync can only trace transactions from blocks following the initial sync point. Tracing a single transaction also entails reexecuting all preceding transactions in the same block.
* A **fast synced** node retaining only **periodic state data** after initial sync can only trace transactions from blocks following the initial sync point. Tracing a single transaction entails reexecuting all preceding transactions **both** in the same block, as well as all preceding blocks until the previous stored snapshot.
* A **light synced** node retrieving data **on demand** can in theory trace transactions for which all required historical state is readily available in the network. In practice, data availability is **not** a feasible assumption.
*There are exceptions to the above rules when running batch traces of entire blocks or chain segments. Those will be detailed later.*
## Basic traces
The simplest type of transaction trace that `go-ethereum` can generate are raw EVM opcode traces. For every VM instruction the transaction executes, a structured log entry is emitted, containing all contextual metadata deemed useful. This includes the *program counter*, *opcode name*, *opcode cost*, *remaining gas*, *execution depth* and any *occurred error*. The structured logs can optionally also contain the content of the *execution stack*, *execution memory* and *contract storage*.
An example log entry for a single opcode looks like:
```json
{
"pc": 48,
"op": "DIV",
"gasCost": 5,
"gas": 64532,
"depth": 1,
"error": null,
"stack": [
"00000000000000000000000000000000000000000000000000000000ffffffff",
"0000000100000000000000000000000000000000000000000000000000000000",
"2df07fbaabbe40e3244445af30759352e348ec8bebd4dd75467a9f29ec55d98d"
],
"memory": [
"0000000000000000000000000000000000000000000000000000000000000000",
"0000000000000000000000000000000000000000000000000000000000000000",
"0000000000000000000000000000000000000000000000000000000000000060"
],
"storage": {
}
}
```
The entire output of an raw EVM opcode trace is a JSON object having a few metadata fields: *consumed gas*, *failure status*, *return value*; and a list of *opcode entries* that take the above form:
```json
{
"gas": 25523,
"failed": false,
"returnValue": "",
"structLogs": []
}
```
### Generating basic traces
To generate a raw EVM opcode trace, `go-ethereum` provides a few [RPC API endpoints](https://github.com/ethereum/go-ethereum/wiki/Management-APIs), out of which the most commonly used is [`debug_traceTransaction`](https://github.com/ethereum/go-ethereum/wiki/Management-APIs#debug_tracetransaction).
In its simplest form, `traceTransaction` accepts a transaction hash as its sole argument, traces the transaction, aggregates all the generated data and returns it as a **large** JSON object. A sample invocation from the Geth console would be:
```js
debug.traceTransaction("0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f")
```
The same call can of course be invoked from outside the node too via HTTP RPC. In this case, please make sure the HTTP endpoint is enabled via `--rpc` and the `debug` API namespace exposed via `--rpcapi=debug`.
```
$ curl -H "Content-Type: application/json" -d '{"id": 1, "method": "debug_traceTransaction", "params": ["0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f"]}' localhost:8545
```
Running the above operation on the Rinkeby network (with a node retaining enough history) will result in this [trace dump](https://gist.github.com/karalabe/c91f95ac57f5e57f8b950ec65ecc697f).
### Tuning basic traces
By default the raw opcode tracer emits all relevant events that occur within the EVM while processing a transaction, such as *EVM stack*, *EVM memory* and *updated storage slots*. Certain use cases however may not need some of these data fields reported. To cater for those use cases, these massive fields may be omitted using a second *options* parameter for the tracer:
```json
{
"disableStack": true,
"disableMemory": true,
"disableStorage": true
}
```
Running the previous tracer invocation from the Geth console with the data fields disabled:
```js
debug.traceTransaction("0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f", {disableStack: true, disableMemory: true, disableStorage: true})
```
Analogously running the filtered tracer from outside the node too via HTTP RPC:
```
$ curl -H "Content-Type: application/json" -d '{"id": 1, "method": "debug_traceTransaction", "params": ["0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f", {"disableStack": true, "disableMemory": true, "disableStorage": true}]}' localhost:8545
```
Running the above operation on the Rinkeby network will result in this significantly shorter [trace dump](https://gist.github.com/karalabe/d74a7cb33a70f2af75e7824fc772c5b4).
### Limits of basic traces
Although the raw opcode traces we've generated above have their use, this basic way of tracing is problematic in the real world. Having an individual log entry for every single opcode is too low level for most use cases, and will require developers to create additional tools to post-process the traces. Additionally, a full opcode trace can easily go into the hundreds of megabytes, making them very resource intensive to get out of the node and process externally.
To avoid all of the previously mentioned issues, `go-ethereum` supports running custom JavaScript tracers *within* the Ethereum node, which have full access to the EVM stack, memory and contract storage. This permits developers to only gather the data they need, and do any processing **at** the data. Please see the next section for our *custom in-node tracers*.
### Pruning
Geth by default does in-memory pruning of state, discarding state entries that it deems is no longer necessary to maintain. This is configured via the `--gcmode` option. Often, people run into the error that state is not available.
Say you want to do a trace on block `B`. Now there are a couple of cases:
1. You have done a fast-sync, pivot block `P` where `P <= B`.
2. You have done a fast-sync, pivot block `P` where `P > B`.
3. You have done a full-sync, with pruning
4. You have done a full-sync, without pruning (`--gcmode=archive`)
Here's what happens in each respective case:
1. Geth will regenerate the desired state by replaying blocks from the closest point in time before `B` where it has full state. This defaults to [`128`](https://github.com/ethereum/go-ethereum/blob/master/eth/api_tracer.go#L52) blocks max, but you can specify more in the actual call `... "reexec":1000 .. }` to the tracer.
2. Sorry, can't be done without replaying from genesis.
3. Same as 1)
4. Does not need to replay anything, can immediately load up the state and serve the request.
There is one other option available to you, which may or may not suit your needs. That is to use [Evmlab]( https://github.com/holiman/evmlab).
```
docker pull holiman/evmlab && docker run -it holiman/evmlab
```
There you can use the reproducer. The reproducer will incrementally fetch data from infura until it has all the information required to create the trace locally on an evm which is bundled with the image. It will create a custom genesis containing the state that the transaction touches (balances, code, nonce etc). It should be mentioned that the evmlab reproducer is strictly guaranteed to be totally exact with regards to gascosts incurred by the outer transaction, as evmlab does not fully calculate the gascosts for nonzero data etc, but is usually sufficient to analyze contracts and events.

199
docs/_doc/Mobile_Account-management.md
View file

@ -1,199 +0,0 @@
---
title: Mobile / Account management
---
To provide Ethereum integration for your mobile applications, the very first thing you should be interested in doing is account management.
Although all current leading Ethereum implementations provide account management built in, it is ill advised to keep accounts in any location that is shared between multiple applications and/or multiple people. The same way you do not entrust your ISP (who is after all your gateway into the internet) with your login credentials; you should not entrust an Ethereum node (who is your gateway into the Ethereum network) with your credentials either.
The proper way to handle user accounts in your mobile applications is to do client side account management, everything self-contained within your own application. This way you can ensure as fine grained (or as coarse) access permissions to the sensitive data as deemed necessary, without relying on any third party application's functionality and/or vulnerabilities.
To support this, `go-ethereum` provides a simple, yet thorough accounts library that gives you all the tools to do properly secured account management via encrypted keystores and passphrase protected accounts. You can leverage all the security of the `go-ethereum` crypto implementation while at the same time running everything in your own application.
## Encrypted keystores
Although handling your users' accounts locally on their own mobile device does provide certain security guarantees, access keys to Ethereum accounts should never lay around in clear-text form. As such, we provide an encrypted keystore that provides the proper security guarantees for you without requiring a thorough understanding from your part of the associated cryptographic primitives.
The important thing to know when using the encrypted keystore is that the cryptographic primitives used within can operate either in *standard* or *light* mode. The former provides a higher level of security at the cost of increased computational burden and resource consumption:
* *standard* needs 256MB memory and 1 second processing on a modern CPU to access a key
* *light* needs 4MB memory and 100 millisecond processing on a modern CPU to access a key
As such, *light* is more suitable for mobile applications, but you should be aware of the trade-offs nonetheless.
*For those interested in the cryptographic and/or implementation details, the key-store uses the `secp256k1` elliptic curve as defined in the [Standards for Efficient Cryptography](http://www.secg.org/sec2-v2.pdf), implemented by the [`libsecp256k`](https://github.com/bitcoin-core/secp256k1) library and wrapped by [`github.com/ethereum/go-ethereum/accounts`](https://godoc.org/github.com/ethereum/go-ethereum/accounts). Accounts are stored on disk in the [Web3 Secret Storage](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) format.*
### Keystores on Android (Java)
The encrypted keystore on Android is implemented by the `KeyStore` class from the `org.ethereum.geth` package. The configuration constants (for the *standard* or *light* security modes described above) are located in the `Geth` abstract class, similarly from the `org.ethereum.geth` package. Hence to do client side account management on Android, you'll need to import two classes into your Java code:
```java
import org.ethereum.geth.Geth;
import org.ethereum.geth.KeyStore;
```
Afterwards you can create a new encrypted keystore via:
```java
KeyStore ks = new KeyStore("/path/to/keystore", Geth.LightScryptN, Geth.LightScryptP);
```
The path to the keystore folder needs to be a location that is writable by the local mobile application but non-readable for other installed applications (for security reasons obviously), so we'd recommend placing it inside your app's data directory. If you are creating the `KeyStore` from within a class extending an Android object, you will most probably have access to the `Context.getFilesDir()` method via `this.getFilesDir()`, so you could set the keystore path to `this.getFilesDir() + "/keystore"`.
The last two arguments of the `KeyStore` constructor are the crypto parameters defining how resource-intensive the keystore encryption should be. You can choose between `Geth.StandardScryptN, Geth.StandardScryptP`, `Geth.LightScryptN, Geth.LightScryptP` or specify your own numbers (please make sure you understand the underlying cryptography for this). We recommend using the *light* version.
### Keystores on iOS (Swift 3)
The encrypted keystore on iOS is implemented by the `GethKeyStore` class from the `Geth` framework. The configuration constants (for the *standard* or *light* security modes described above) are located in the same namespace as global variables. Hence to do client side account management on iOS, you'll need to import the framework into your Swift code:
```swift
import Geth
```
Afterwards you can create a new encrypted account manager via:
```swift
let ks = GethNewKeyStore("/path/to/keystore", GethLightScryptN, GethLightScryptP);
```
The path to the keystore folder needs to be a location that is writable by the local mobile application but non-readable for other installed applications (for security reasons obviously), so we'd recommend placing it inside your app's document directory. You should be able to retrieve the document directory via `let datadir = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]`, so you could set the keystore path to `datadir + "/keystore"`.
The last two arguments of the `GethNewKeyStore` factory method are the crypto parameters defining how resource-intensive the keystore encryption should be. You can choose between `GethStandardScryptN, GethStandardScryptP`, `GethLightScryptN, GethLightScryptP` or specify your own numbers (please make sure you understand the underlying cryptography for this). We recommend using the *light* version.
## Account lifecycle
Having created an encrypted keystore for your Ethereum accounts, you can use this for the entire account lifecycle requirements of your mobile application. This includes the basic functionality of creating new accounts and deleting existing ones; as well as the more advanced functionality of updating access credentials, exporting existing accounts, and importing them on another device.
Although the keystore defines the encryption strength it uses to store your accounts, there is no global master password that can grant access to all of them. Rather each account is maintained individually, and stored on disk in its [encrypted format](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) individually, ensuring a much cleaner and stricter separation of credentials.
This individuality however means that any operation requiring access to an account will need to provide the necessary authentication credentials for that particular account in the form of a passphrase:
* When creating a new account, the caller must supply a passphrase to encrypt the account with. This passphrase will be required for any subsequent access, the lack of which will forever forfeit using the newly created account.
* When deleting an existing account, the caller must supply a passphrase to verify ownership of the account. This isn't cryptographically necessary, rather a protective measure against accidental loss of accounts.
* When updating an existing account, the caller must supply both current and new passphrases. After completing the operation, the account will not be accessible via the old passphrase any more.
* When exporting an existing account, the caller must supply both the current passphrase to decrypt the account, as well as an export passphrase to re-encrypt it with before returning the key-file to the user. This is required to allow moving accounts between devices without sharing original credentials.
* When importing a new account, the caller must supply both the encryption passphrase of the key-file being imported, as well as a new passhprase with which to store the account. This is required to allow storing account with different credentials than used for moving them around.
*Please note, there is no recovery mechanisms for losing the passphrases. The cryptographic properties of the encrypted keystore (if using the provided parameters) guarantee that account credentials cannot be brute forced in any meaningful time.*
### Accounts on Android (Java)
An Ethereum account on Android is implemented by the `Account` class from the `org.ethereum.geth` package. Assuming we already have an instance of a `KeyStore` called `ks` from the previous section, we can easily execute all of the described lifecycle operations with a handful of function calls.
```java
// Create a new account with the specified encryption passphrase.
Account newAcc = ksm.newAccount("Creation password");
// Export the newly created account with a different passphrase. The returned
// data from this method invocation is a JSON encoded, encrypted key-file.
byte[] jsonAcc = ks.exportKey(newAcc, "Creation password", "Export password");
// Update the passphrase on the account created above inside the local keystore.
ks.updateAccount(newAcc, "Creation password", "Update password");
// Delete the account updated above from the local keystore.
ks.deleteAccount(newAcc, "Update password");
// Import back the account we've exported (and then deleted) above with yet
// again a fresh passphrase.
Account impAcc = ks.importKey(jsonAcc, "Export password", "Import password");
```
*Although instances of `Account` can be used to access various information about specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or private keys), rather act solely as identifiers for client code and the keystore.*
### Accounts on iOS (Swift 3)
An Ethereum account on iOS is implemented by the `GethAccount` class from the `Geth` framework. Assuming we already have an instance of a `GethKeyStore` called `ks` from the previous section, we can easily execute all of the described lifecycle operations with a handful of function calls.
```swift
// Create a new account with the specified encryption passphrase.
let newAcc = try! ks?.newAccount("Creation password")
// Export the newly created account with a different passphrase. The returned
// data from this method invocation is a JSON encoded, encrypted key-file.
let jsonKey = try! ks?.exportKey(newAcc!, passphrase: "Creation password", newPassphrase: "Export password")
// Update the passphrase on the account created above inside the local keystore.
try! ks?.update(newAcc, passphrase: "Creation password", newPassphrase: "Update password")
// Delete the account updated above from the local keystore.
try! ks?.delete(newAcc, passphrase: "Update password")
// Import back the account we've exported (and then deleted) above with yet
// again a fresh passphrase.
let impAcc = try! ks?.importKey(jsonKey, passphrase: "Export password", newPassphrase: "Import password")
```
*Although instances of `GethAccount` can be used to access various information about specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or private keys), rather act solely as identifiers for client code and the keystore.*
## Signing authorization
As mentioned above, account objects do not hold the sensitive private keys of the associated Ethereum accounts, but are merely placeholders to identify the cryptographic keys with. All operations that require authorization (e.g. transaction signing) are performed by the account manager after granting it access to the private keys.
There are a few different ways one can authorize the account manager to execute signing operations, each having its advantages and drawbacks. Since the different methods have wildly different security guarantees, it is essential to be clear on how each works:
* **Single authorization**: The simplest way to sign a transaction via the keystore is to provide the passphrase of the account every time something needs to be signed, which will ephemerally decrypt the private key, execute the signing operation and immediately throw away the decrypted key. The drawbacks are that the passphrase needs to be queried from the user every time, which can become annoying if done frequently; or the application needs to keep the passphrase in memory, which can have security consequences if not done properly; and depending on the keystore's configured strength, constantly decrypting keys can result in non-negligible resource requirements.
* **Multiple authorizations**: A more complex way of signing transactions via the keystore is to unlock the account via its passphrase once, and allow the account manager to cache the decrypted private key, enabling all subsequent signing requests to complete without the passphrase. The lifetime of the cached private key may be managed manually (by explicitly locking the account back up) or automatically (by providing a timeout during unlock). This mechanism is useful for scenarios where the user may need to sign many transactions or the application would need to do so without requiring user input. The crucial aspect to remember is that **anyone with access to the account manager can sign transactions while a particular account is unlocked** (e.g. device left unattended; application running untrusted code).
*Note, creating transactions is out of scope here, so the remainder of this section will assume we already have a transaction to sign, and will focus only on creating an authorized version of it. Creating an actually meaningful transaction will be covered later.*
### Signing on Android (Java)
Assuming we already have an instance of a `KeyStore` called `ks` from the previous sections, we can create a new account to sign transactions with via it's already demonstrated `newAccount` method; and to avoid going into transaction creation for now, we can hard-code a random transaction to sign instead.
```java
// Create a new account to sign transactions with
Account signer = ks.newAccount("Signer password");
Transaction tx = new Transaction(
1, new Address("0x0000000000000000000000000000000000000000"),
new BigInt(0), new BigInt(0), new BigInt(1), null); // Random empty transaction
BigInt chain = new BigInt(1); // Chain identifier of the main net
```
With the boilerplate out of the way, we can now sign transaction using the authorization mechanisms described above:
```java
// Sign a transaction with a single authorization
Transaction signed = ks.signTxPassphrase(signer, "Signer password", tx, chain);
// Sign a transaction with multiple manually cancelled authorizations
ks.unlock(signer, "Signer password");
signed = ks.signTx(signer, tx, chain);
ks.lock(signer.getAddress());
// Sign a transaction with multiple automatically cancelled authorizations
ks.timedUnlock(signer, "Signer password", 1000000000);
signed = ks.signTx(signer, tx, chain);
```
### Signing on iOS (Swift 3)
Assuming we already have an instance of a `GethKeyStore` called `ks` from the previous sections, we can create a new account to sign transactions with via it's already demonstrated `newAccount` method; and to avoid going into transaction creation for now, we can hard-code a random transaction to sign instead.
```swift
// Create a new account to sign transactions with
var error: NSError?
let signer = try! ks?.newAccount("Signer password")
let to = GethNewAddressFromHex("0x0000000000000000000000000000000000000000", &error)
let tx = GethNewTransaction(1, to, GethNewBigInt(0), GethNewBigInt(0), GethNewBigInt(0), nil) // Random empty transaction
let chain = GethNewBigInt(1) // Chain identifier of the main net
```
*Note, although Swift usually rewrites `NSError` returns to throws, this particular instance seems to have been missed for some reason (possibly due to it being a constructor). It will be fixed in a later version of the iOS bindings when the appropriate fixed are implemented upstream in the `gomobile` project.*
With the boilerplate out of the way, we can now sign transaction using the authorization methods described above:
```swift
// Sign a transaction with a single authorization
var signed = try! ks?.signTxPassphrase(signer, passphrase: "Signer password", tx: tx, chainID: chain)
// Sign a transaction with multiple manually cancelled authorizations
try! ks?.unlock(signer, passphrase: "Signer password")
signed = try! ks?.signTx(signer, tx: tx, chainID: chain)
try! ks?.lock(signer?.getAddress())
// Sign a transaction with multiple automatically cancelled authorizations
try! ks?.timedUnlock(signer, passphrase: "Signer password", timeout: 1000000000)
signed = try! ks?.signTx(signer, tx: tx, chainID: chain)
```

125
docs/_doc/Mobile_Introduction.md
View file

@ -1,125 +0,0 @@
---
title: Mobile / Introduction
---
The Ethereum blockchain along with its two extension protocols Whisper and Swarm was originally conceptualized to become the supporting pillar of web3, providing the consensus, messaging and storage backbone for a new generation of distributed (actually, decentralized) applications called DApps.
The first incarnation towards this dream of web3 was a command line client providing an RPC interface into the peer-to-peer protocols. The client was soon enough extended with a web-browser-like graphical user interface, permitting developers to write DApps based on the tried and proven HTML/CSS/JS technologies.
As many DApps have more complex requirements than what a browser environment can handle, it became apparent that providing programmatic access to the web3 pillars would open the door towards a new class of applications. As such, the second incarnation of the web3 dream is to open up all our technologies for other projects as reusable components.
Starting with the 1.5 release family of `go-ethereum`, we transitioned away from providing only a full blown Ethereum client and started shipping official Go packages that could be embedded into third party desktop and server applications. It took only a small leap from here to begin porting our code to mobile platforms.
## Quick overview
Similarly to our reusable Go libraries, the mobile wrappers also focus on four main usage areas:
- Simplified client side account management
- Remote node interfacing via different transports
- Contract interactions through auto-generated bindings
- In-process Ethereum, Whisper and Swarm peer-to-peer node
You can watch a quick overview about these in Peter's (@karalabe) talk titled "Import Geth: Ethereum from Go and beyond", presented at the Ethereum Devcon2 developer conference in September, 2016 (Shanghai). Slides are [available here](https://ethereum.karalabe.com/talks/2016-devcon.html).
[![Peter's Devcon2 talk](https://img.youtube.com/vi/R0Ia1U9Gxjg/0.jpg)](https://www.youtube.com/watch?v=R0Ia1U9Gxjg)
## Library bundles
The `go-ethereum` mobile library is distributed either as an Android `.aar` archive (containing binaries for `arm-7`, `arm64`, `x86` and `x64`); or as an iOS XCode framework (containing binaries for `arm-7`, `arm64` and `x86`). We do not provide library bundles for Windows phone the moment.
### Android archive
The simplest way to use `go-ethereum` in your Android project is through a Maven dependency. We provide bundles of all our stable releases (starting from v1.5.0) through Maven Central, and also provide the latest develop bundle through the Sonatype OSS repository.
#### Stable dependency (Maven Central)
To add an Android dependency to the **stable** library release of `go-ethereum`, you'll need to ensure that the Maven Central repository is enabled in your Android project, and that the `go-ethereum` code is listed as a required dependency of your application. You can do both of these by editing the `build.gradle` script in your Android app's folder:
```gradle
repositories {
mavenCentral()
}
dependencies {
// All your previous dependencies
compile 'org.ethereum:geth:1.5.2' // Change the version to the latest release
}
```
#### Develop dependency (Sonatype)
To add an Android dependency to the current **develop** version of `go-ethereum`, you'll need to ensure that the Sonatype snapshot repository is enabled in your Android project, and that the `go-ethereum` code is listed as a required `SNAPSHOT` dependency of your application. You can do both of these by editing the `build.gradle` script in your Android app's folder:
```gradle
repositories {
maven {
url "https://oss.sonatype.org/content/groups/public"
}
}
dependencies {
// All your previous dependencies
compile 'org.ethereum:geth:1.5.3-SNAPSHOT' // Change the version to the latest release
}
```
#### Custom dependency
If you prefer not to depend on Maven Central or Sonatype; or would like to access an older develop build not available any more as an online dependency, you can download any bundle directly from [our website](https://geth.ethereum.org/downloads/) and insert it into your project in Android Studio via `File -> New -> New module... -> Import .JAR/.AAR Package`.
You will also need to configure `gradle` to link the mobile library bundle to your application. This can be done by adding a new entry to the `dependencies` section of your `build.gradle` script, pointing it to the module you just added (named `geth` by default).
```gradle
dependencies {
// All your previous dependencies
compile project(':geth')
}
```
#### Manual builds
Lastly, if you would like to make modifications to the `go-ethereum` mobile code and/or build it yourself locally instead of downloading a pre-built bundle, you can do so using a `make` command. This will create an Android archive called `geth.aar` in the `build/bin` folder that you can import into your Android Studio as described above.
```bash
$ make android
[...]
Done building.
Import "build/bin/geth.aar" to use the library.
```
### iOS framework
The simplest way to use `go-ethereum` in your iOS project is through a [CocoaPods](https://cocoapods.org/) dependency. We provide bundles of all our stable releases (starting from v1.5.3) and also latest develop versions.
#### Automatic dependency
To add an iOS dependency to the current stable or latest develop version of `go-ethereum`, you'll need to ensure that your iOS XCode project is configured to use CocoaPods. Detailing that is out of scope in this document, but you can find a guide in the upstream [Using CocoaPods](https://guides.cocoapods.org/using/using-cocoapods.html) page. Afterwards you can edit your `Podfile` to list `go-ethereum` as a dependency:
```ruby
target 'MyApp' do
# All your previous dependencies
pod 'Geth', '1.5.4' # Change the version to the latest release
end
```
Alternatively, if you'd like to use the latest develop version, replace the package version `1.5.4` with `~> 1.5.5-unstable` to switch to pre-releases and to always pull in the latest bundle from a particular release family.
#### Custom dependency
If you prefer not to depend on CocoaPods; or would like to access an older develop build not available any more as an online dependency, you can download any bundle directly from [our website](https://geth.ethereum.org/downloads/) and insert it into your project in XCode via `Project Settings -> Build Phases -> Link Binary With Libraries`.
Do not forget to extract the framework from the compressed `.tar.gz` archive. You can do that either using a GUI tool or from the command line via (replace the archive with your downloaded file):
```
tar -zxvf geth-ios-all-1.5.3-unstable-e05d35e6.tar.gz
```
#### Manual builds
Lastly, if you would like to make modifications to the `go-ethereum` mobile code and/or build it yourself locally instead of downloading a pre-built bundle, you can do so using a `make` command. This will create an iOS XCode framework called `Geth.framework` in the `build/bin` folder that you can import into XCode as described above.
```bash
$ make ios
[...]
Done building.
Import "build/bin/Geth.framework" to use the library.
```

122
docs/_doc/Native_Account-management.md
View file

@ -1,122 +0,0 @@
---
title: Native / Account management
---
To provide Ethereum integration for your native applications, the very first thing you should be interested in doing is account management.
Although all current leading Ethereum implementations provide account management built in, it is ill advised to keep accounts in any location that is shared between multiple applications and/or multiple people. The same way you do not entrust your ISP (who is after all your gateway into the internet) with your login credentials; you should not entrust an Ethereum node (who is your gateway into the Ethereum network) with your credentials either.
The proper way to handle user accounts in your native applications is to do client side account management, everything self-contained within your own application. This way you can ensure as fine grained (or as coarse) access permissions to the sensitive data as deemed necessary, without relying on any third party application's functionality and/or vulnerabilities.
To support this, `go-ethereum` provides a simple, yet thorough accounts package that gives you all the tools to do properly secured account management via encrypted keystores and passphrase protected accounts. You can leverage all the security of the `go-ethereum` crypto implementation while at the same time running everything in your own application.
## Encrypted keystores
Although handling accounts locally to an application does provide certain security guarantees, access keys to Ethereum accounts should never lay around in clear-text form. As such, we provide an encrypted keystore that provides the proper security guarantees for you without requiring a thorough understanding from your part of the associated cryptographic primitives.
The important thing to know when using the encrypted keystore is that the cryptographic primitives used within can operate either in *standard* or *light* mode. The former provides a higher level of security at the cost of increased computational burden and resource consumption:
* *standard* needs 256MB memory and 1 second processing on a modern CPU to access a key
* *light* needs 4MB memory and 100 millisecond processing on a modern CPU to access a key
As such, *standard* is more suitable for native applications, but you should be aware of the trade-offs nonetheless in case you you're targeting more resource constrained environments.
*For those interested in the cryptographic and/or implementation details, the key-store uses the `secp256k1` elliptic curve as defined in the [Standards for Efficient Cryptography](http://www.secg.org/sec2-v2.pdf), implemented by the [`libsecp256k`](https://github.com/bitcoin-core/secp256k1) library and wrapped by [`github.com/ethereum/go-ethereum/accounts`](https://godoc.org/github.com/ethereum/go-ethereum/accounts). Accounts are stored on disk in the [Web3 Secret Storage](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) format.*
### Keystores from Go
The encrypted keystore is implemented by the [`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager) struct from the [`github.com/ethereum/go-ethereum/accounts`](https://godoc.org/github.com/ethereum/go-ethereum/accounts) package, which also contains the configuration constants for the *standard* or *light* security modes described above. Hence to do client side account management from Go, you'll need to import only the `accounts` package into your code:
```go
import "github.com/ethereum/go-ethereum/accounts"
```
Afterwards you can create a new encrypted account manager via:
```go
am := accounts.NewManager("/path/to/keystore", accounts.StandardScryptN, accounts.StandardScryptP);
```
The path to the keystore folder needs to be a location that is writable by the local user but non-readable for other system users (for security reasons obviously), so we'd recommend placing it either inside your user's home directory or even more locked down for backend applications.
The last two arguments of [`accounts.NewManager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#NewManager) are the crypto parameters defining how resource-intensive the keystore encryption should be. You can choose between [`accounts.StandardScryptN, accounts.StandardScryptP`, `accounts.LightScryptN, accounts.LightScryptP`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#pkg-constants) or specify your own numbers (please make sure you understand the underlying cryptography for this). We recommend using the *standard* version.
## Account lifecycle
Having created an encrypted keystore for your Ethereum accounts, you can use this account manager for the entire account lifecycle requirements of your native application. This includes the basic functionality of creating new accounts and deleting existing ones; as well as the more advanced functionality of updating access credentials, exporting existing accounts, and importing them on another device.
Although the keystore defines the encryption strength it uses to store your accounts, there is no global master password that can grant access to all of them. Rather each account is maintained individually, and stored on disk in its [encrypted format](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) individually, ensuring a much cleaner and stricter separation of credentials.
This individuality however means that any operation requiring access to an account will need to provide the necessary authentication credentials for that particular account in the form of a passphrase:
* When creating a new account, the caller must supply a passphrase to encrypt the account with. This passphrase will be required for any subsequent access, the lack of which will forever forfeit using the newly created account.
* When deleting an existing account, the caller must supply a passphrase to verify ownership of the account. This isn't cryptographically necessary, rather a protective measure against accidental loss of accounts.
* When updating an existing account, the caller must supply both current and new passphrases. After completing the operation, the account will not be accessible via the old passphrase any more.
* When exporting an existing account, the caller must supply both the current passphrase to decrypt the account, as well as an export passphrase to re-encrypt it with before returning the key-file to the user. This is required to allow moving accounts between machines and applications without sharing original credentials.
* When importing a new account, the caller must supply both the encryption passphrase of the key-file being imported, as well as a new passhprase with which to store the account. This is required to allow storing account with different credentials than used for moving them around.
*Please note, there is no recovery mechanisms for losing the passphrases. The cryptographic properties of the encrypted keystore (if using the provided parameters) guarantee that account credentials cannot be brute forced in any meaningful time.*
### Accounts from Go
An Ethereum account is implemented by the [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) struct from the [`github.com/ethereum/go-ethereum/accounts`](https://godoc.org/github.com/ethereum/go-ethereum/accounts) package. Assuming we already have an instance of an [`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager) called `am` from the previous section, we can easily execute all of the described lifecycle operations with a handful of function calls (error handling omitted).
```go
// Create a new account with the specified encryption passphrase.
newAcc, _ := am.NewAccount("Creation password");
// Export the newly created account with a different passphrase. The returned
// data from this method invocation is a JSON encoded, encrypted key-file.
jsonAcc, _ := am.Export(newAcc, "Creation password", "Export password");
// Update the passphrase on the account created above inside the local keystore.
am.Update(newAcc, "Creation password", "Update password");
// Delete the account updated above from the local keystore.
am.Delete(newAcc, "Update password");
// Import back the account we've exported (and then deleted) above with yet
// again a fresh passphrase.
impAcc, _ := am.Import(jsonAcc, "Export password", "Import password");
```
*Although instances of [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) can be used to access various information about specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or private keys), rather act solely as identifiers for client code and the keystore.*
## Signing authorization
As mentioned above, account objects do not hold the sensitive private keys of the associated Ethereum accounts, but are merely placeholders to identify the cryptographic keys with. All operations that require authorization (e.g. transaction signing) are performed by the account manager after granting it access to the private keys.
There are a few different ways one can authorize the account manager to execute signing operations, each having its advantages and drawbacks. Since the different methods have wildly different security guarantees, it is essential to be clear on how each works:
* **Single authorization**: The simplest way to sign a transaction via the account manager is to provide the passphrase of the account every time something needs to be signed, which will ephemerally decrypt the private key, execute the signing operation and immediately throw away the decrypted key. The drawbacks are that the passphrase needs to be queried from the user every time, which can become annoying if done frequently; or the application needs to keep the passphrase in memory, which can have security consequences if not done properly; and depending on the keystore's configured strength, constantly decrypting keys can result in non-negligible resource requirements.
* **Multiple authorizations**: A more complex way of signing transactions via the account manager is to unlock the account via its passphrase once, and allow the account manager to cache the decrypted private key, enabling all subsequent signing requests to complete without the passphrase. The lifetime of the cached private key may be managed manually (by explicitly locking the account back up) or automatically (by providing a timeout during unlock). This mechanism is useful for scenarios where the user may need to sign many transactions or the application would need to do so without requiring user input. The crucial aspect to remember is that **anyone with access to the account manager can sign transactions while a particular account is unlocked** (e.g. application running untrusted code).
*Note, creating transactions is out of scope here, so the remainder of this section will assume we already have a transaction hash to sign, and will focus only on creating a cryptographic signature authorizing it. Creating an actual transaction and injecting the authorization signature into it will be covered later.*
### Signing from Go
Assuming we already have an instance of an [`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager) called `am` from the previous sections, we can create a new account to sign transactions with via it's already demonstrated [`NewAccount`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.NewAccount) method; and to avoid going into transaction creation for now, we can hard-code a random [`common.Hash`](https://godoc.org/github.com/ethereum/go-ethereum/common#Hash) to sign instead.
```go
// Create a new account to sign transactions with
signer, _ := am.NewAccount("Signer password");
txHash := common.HexToHash("0x0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef");
```
With the boilerplate out of the way, we can now sign transaction using the authorization mechanisms described above:
```go
// Sign a transaction with a single authorization
signature, _ := am.SignWithPassphrase(signer, "Signer password", txHash.Bytes());
// Sign a transaction with multiple manually cancelled authorizations
am.Unlock(signer, "Signer password");
signature, _ = am.Sign(signer.Address, txHash.Bytes());
am.Lock(signer.Address);
// Sign a transaction with multiple automatically cancelled authorizations
am.TimedUnlock(signer, "Signer password", time.Second);
signature, _ = am.Sign(signer.Address, txHash.Bytes());
```
You may wonder why [`SignWithPassphrase`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.SignWithPassphrase) takes an [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) as the signer, whereas [`Sign`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.Sign) takes only a [`common.Address`](https://godoc.org/github.com/ethereum/go-ethereum/common#Address). The reason is that an [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) object may also contain a custom key-path, allowing [`SignWithPassphrase`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.SignWithPassphrase) to sign using accounts outside of the keystore; however [`Sign`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.Sign) relies on accounts already unlocked within the keystore, so it cannot specify custom paths.

43
docs/_doc/Native_Introduction.md
View file

@ -1,43 +0,0 @@
---
title: Native / Introduction
---
The Ethereum blockchain along with its two extension protocols Whisper and Swarm was originally conceptualized to become the supporting pillar of web3, providing the consensus, messaging and storage backbone for a new generation of distributed (actually, decentralized) applications called DApps.
The first incarnation towards this dream of web3 was a command line client providing an RPC interface into the peer-to-peer protocols. The client was soon enough extended with a web-browser-like graphical user interface, permitting developers to write DApps based on the tried and proven HTML/CSS/JS technologies.
As many DApps have more complex requirements than what a browser environment can handle, it became apparent that providing programmatic access to the web3 pillars would open the door towards a new class of applications. As such, the second incarnation of the web3 dream is to open up all our technologies for other projects as reusable components.
Starting with the 1.5 release family of `go-ethereum`, we transitioned away from providing only a full blown Ethereum client and started shipping official Go packages that could be embedded into third party desktop and server applications.
*Note, this guide will assume you are familiar with Go development. It will make no attempts to cover general topics about Go project layouts, import paths or any other standard methodologies. If you are new to Go, consider reading its [getting started guides](https://github.com/golang/go/wiki#getting-started-with-go) first.*
## Quick overview
Our reusable Go libraries focus on four main usage areas:
- Simplified client side account management
- Remote node interfacing via different transports
- Contract interactions through auto-generated bindings
- In-process Ethereum, Whisper and Swarm peer-to-peer node
You can watch a quick overview about these in Peter's (@karalabe) talk titled "Import Geth: Ethereum from Go and beyond", presented at the Ethereum Devcon2 developer conference in September, 2016 (Shanghai). Slides are [available here](https://ethereum.karalabe.com/talks/2016-devcon.html).
[![Peter's Devcon2 talk](https://img.youtube.com/vi/R0Ia1U9Gxjg/0.jpg)](https://www.youtube.com/watch?v=R0Ia1U9Gxjg)
## Go packages
The `go-ethereum` library is distributed as a collection of standard Go packages straight from our GitHub repository. The packages can be used directly via the official Go toolkit, without needing any third party tools. External dependencies are vendored locally into `vendor`, ensuring both self-containment as well as code stability. If you reuse `go-ethereum` in your own project, please follow these best practices and vendor it yourself too to avoid any accidental API breakages!
The canonical import path for `go-ethereum` is `github.com/ethereum/go-ethereum`, with all packages residing underneath. Although there are [quite a number](https://godoc.org/github.com/ethereum/go-ethereum#pkg-subdirectories) of them, you'll only need to care about a limited subset, each of which will be properly introduced in their relevant section.
You can download all our packages via:
```
$ go get -d github.com/ethereum/go-ethereum/...
```
You may also need Go's original context package. Although this was moved into the official Go SDK in Go 1.7, `go-ethereum` will depend on the original `golang.org/x/net/context` package until we officially drop support for Go 1.5 and Go 1.6.
```
$ go get -u golang.org/x/net/context
```