## Why this should be merged
The `temporary.WithTempRegisteredExtras()` global lock introduced in
#234 wasn't fit for purpose when used in `coreth` as it required central
coordination of registration, types, and usage of the payload accessor.
## How this works
Instead of a central registration point, the new
`libevm.WithTemporaryExtrasLock()` function takes out a global lock and
provides the caller with a handle that proves the lock is held. All of
the override functions, e.g. `params.WithTempRegisteredExtras()` now
require a current lock, which will be propagated by the respective
`coreth` functions.
See https://github.com/ava-labs/coreth/pull/1328 for intended usage in
`coreth` and `subnet-evm`. A consumer of both of these can then safely
do the following:
```go
import (
"github.com/ava-labs/libevm/libevm"
coreth "github.com/ava-labs/coreth/plugin/evm"
subnet "github.com/ava-labs/subnet-evm/plugin/evm"
)
// asCChain calls `fn` while emulating `coreth`. It is safe for concurrent usage with [asSubnetEVM].
func asCChain(fn func() error) error {
return libevm.WithTemporaryExtrasLock(func(l libevm.ExtrasLock) error {
return coreth.WithTempRegisteredLibEVMExtras(l, fn)
})
}
// asSubnetEVM calls `fn` while emulating `subnet-evm`. It is safe for concurrent usage with [asCChain].
func asSubnetEVM(fn func() error) error {
return libevm.WithTemporaryExtrasLock(func(l libevm.ExtrasLock) error {
return subnet.WithTempRegisteredLibEVMExtras(l, fn)
})
}
```
## How this was tested
Unit test of the new function plus existing integration tests of all
modified code.
## Why this should be merged
Version increase across all `ava-labs` EVM-related code.
## How this works
Bump to 1.24.8 in both `go.mod` files + requisite version increase for
`golangci-lint`.
## How this was tested
Existing tests.
---------
Co-authored-by: Austin Larson <austin.larson@avalabs.org>
## Why this should be merged
The `vm.PrecompileEnvironment.Call()` method requires careful usage
because of reentrancy vulnerabilities (this is common to all outgoing
`CALL`s in the EVM, not just stateful precompiles). This package
provides a common method of protection, a reentrancy guard.
## How this works
Provides a function that returns `vm.ErrExecutionReverted` if called
twice, by the same contract, in the same transaction, with the same
identifier.
## How this was tested
Unit and integration tests.
## Why this should be merged
Registration of extras requires runtime enforcement of payload types. In
production, only a single set of types can be registered and any
attempts at re-registration will panic (by design). Although this makes
production usage safe, it doesn't allow downstream consumers (e.g. data
indexers) to use extras from different chains (e.g. `coreth` _and_
`subnet-evm`) at the same time.
## How this works
1. The `libevm/register.AtMostOnce` type can now be overridden
temporarily.
2. `params`, `core/types`, `core/vm`, and `state` packages introduce
`WithTempRegisteredExtras()` functions.
3. `libevm/temporary.WithRegisteredExtras()` provides "atomic" override
of all extras.
In all cases, the scope of the override is limited to the life of a
single function call.
## How this was tested
Relative to numbered list above:
1. Unit test of new and existing functionality.
2. Integration tests of both packages, demonstrating both payload and
behavioural override.
## Why this should be merged
Completeness.
## How this works
n/a
## How this was tested
Originally the `DELEGATECALL` test asserted that the `CALLVALUE` was
zero, but this didn't demonstrate that it was inherited (e.g.
`STATICCALL` always receives zero). The value sent to the caller of the
precompile is now non-zero, to precisely demonstrate correct behaviour
of all call types.
## Why this should be merged
Allows direct access to the P-256 verification precompile. Currently the
only alternative is to rely on the planned but not guaranteed address
that Ethereum will use.
## How this works
Embed the non-exported `p256verify` type in an exported struct to allow
its methods to be exposed.
## How this was tested
Run the test suite for `p256verify` on `P256Verify`.
## Why this should be merged
The need for examples to be in `package rawdb_test` made it difficult to
test other code that relied on non-exported `rawdb` identifiers.
## How this works
1. Move all testable examples from `database.libevm_test.go` to
`examples.libevm_test.go` _without any change_.
2. Change `database.libevm_test.go` to be in `package rawdb` and extend
the `WithSkipFreezers()` test.
## How this was tested
No change at all to testable examples. The `WithSkipFreezers()` test now
touches both the happy and (specific) error paths to demonstrate exact
behaviour (under the old approach it could have just been that there was
no error, even without the option).
## Why this should be merged
As described in https://github.com/ava-labs/coreth/issues/1137, database
inspection of a Coreth chain database fails because `InspectDatabase()`
expects for freezers to be used. This PR fixes this bug by adding an
option to skip freezer inspection.
This PR should be followed with a downstream PR in Coreth to pass in an
option to skip freezer inspection during database inspection.
## How this works
Extends `inspectDatabaseConfig` with a `skipFreezers` field and adds the
associated option function for it.
## How this was tested
CI + ran `InspectDatabase()` against Coreth locally
## Why this should be merged
Improved clarity.
## How this works
Semantically equivalent refactor. These values aren't accessible /
needed during execution of a stateful precompile, but are important when
making outgoing calls so were originally only set in the
`PrecompileEnvironment.Call()` implementation. This was confusing and
led to a false-positive bug report.
## How this was tested
Existing tests.
## Why this should be merged
Provides precompiles with unambiguous access to contextual addresses,
without the consumer needing to understand how they change under
different call types.
## How this works
The `libevm.AddressContext` type, which used to carry 3 addresses, now
provides different versions of `Caller` and `Self`. The EVM-semantic
versions are as defined by the rules of the EVM (and available before
this change). The raw versions are the unmodified caller and self.
## How this was tested
Extension of existing UTs to include raw addresses in addition to
existing, EVM-semantic ones.
## Why this should be merged
In case a tx's execution (specifically a registered precompile) should
be invalidated, this allows the EVM to find this error.
## How this works
Adds a setter and getter that can be called from a precompile.
## How this was tested
UT
## Why this should be merged
`ava-labs/coreth` has a partitioned state-address space, achieved by
setting or clearing a specific bit in the hash used to key the space.
This change allows such behaviour to be achieved with pure `libevm`
instead of the `StateDB` wrapping that `coreth` currently uses.
## How this works
Introduction of `state.StateDBHooks` interface, including a
`TransformStateKey()` method that allows for arbitrary change of state
key. If registered, this hook will be honoured by
`StateDB.{Get,GetCommitted,State}Key()` methods unless they receive a
`stateconf.SkipStateKeyTransformation` option.
## How this was tested
Unit test of `SetState() -> GetState() + GetCommittedState()` round trip
with and without options to skip.
## Why this should be merged
Coreth and Subnet-EVM require access to the current tx hash in the Warp
precompile (for selecting the correct predicate results).
This can not be exposed as a wrapper through the `OverrideEVMResetArgs`
because `SetTxContext` is called _prior_ to resetting the EVM instance.
Therefore, the `SetTxContext` function on any DB wrapper would never be
called.
## How this works
Exposes the existing `txHash` through a `TxHash()` function.
## How this was tested
Added a trivial unit test.
## Why this should be merged
`goheader` linter apparently hasn't been running but now decided to and
is blocking another PR. It's not worth the time to investigatie why
(probably a version update).
## How this works
Update header comments as required.
## How this was tested
N/A
## Why this should be merged
To allow more thorough handling of duplicate state roots (or any other
info other users would like), additional information can be provided to
a call of `statedb.Commit`. This change allows arbitrary types to be
sent to `triedb` as well as the `SnapshotTree`. However, this is a
breaking change for those using the functionality already, since the
snapshot commit option is wrapped with another call.
## How this works
See the edited libevm test for usage.
## How this was tested
Edited test case to include `TrieDBUpdateOption` and ensures the payload
is sent.
## Why this should be merged
#185 was allowed to merge despite CI requiring the `lint` job.
## How this works
Fix linter issues.
## How this was tested
The first commit is an empty commit, expected to fail CI, to confirm
that the job is in fact required.
## Why this should be merged
Required for
[ACP-194](https://github.com/avalanche-foundation/ACPs/tree/main/ACPs/194-streaming-asynchronous-execution#gas-charged)
$\lambda$ bound on gas consumption.
## How this works
Hook into `core.StateTransition.TransitionDb()` as this is the bottom of
all execution paths (e.g. `core.ApplyTransaction()` as used in SAE,
`core.StateProcessor.Process(*Block,...)`, etc.). Once consumed gas is
no longer changing (i.e. after all spends and refunds), the transaction
limit is passed to the hook to determine the minimum consumption, which
is applied.
## How this was tested
Unit test via `core.ApplyTransaction()` as this is our entry point in
SAE.
---------
Signed-off-by: Arran Schlosberg <519948+ARR4N@users.noreply.github.com>
Co-authored-by: Stephen Buttolph <stephen@avalabs.org>
## Why this should be merged
Importing some `avalanchego` test fixtures automatically imports
`coreth` packages that register extras, which affects testing of other
VMs that depend on `libevm`. All other registered extras can be cleared
with the pattern in this PR.
## How this works
Uses the `register.AtMostOnce` type already used in `params` and
`types`. I forgot to update the `vm` package when implementing the type.
## How this was tested
Existing unit tests.
> [!NOTE]
> This will be merged into `main` once the current target branch has
been merged.
## Why this should be merged
My original implementation was back to front and it's been bugging me.
## How this works
Originally `params.LibEVMVersion` was a `var` (because it needed to be
computed) and the test used a `const`. This change simply inverts the
two and moves some code around without any change in logic.
I bumped the minor version to 2 (a no-op when not on a release branch)
to bring it in line with the latest release candidate and to avoid
forgetting to do so when performing an actual release.
## How this was tested
Unit test of version-string constant.
## Why this should be merged
Allows `core/types/hashing.go` to be deleted from `coreth`.
## How this works
Exported function acts as a proxy for unexported upstream function.
## How this was tested
NA
## Why this should be merged
Adds logging of `libevm` modification of default behaviour.
## How this works
1. Introduces `log.Lazy` functions to allow expensive logging operations
to be computed i.f.f. required by the logging level.
2. Adds `Info` logging for registration of types and `Debug` logging for
all else.
3. Only paths that change behaviour in a potentially unpredictable
manner are logged; of note, RLP / JSON encoding is _not_ considered
unpredictable given that registered extras are logged.
4. The minimal viable package, `set`, was necessary because we don't
want to depend on `avalanchego` and the `hashicorp/go-set` latest
version requires a later version of Go. #153 tracks a swap to the latter
when possible.
The `eth/tracers/internal/tracetest` test flaked at least twice
(unrelated to these changes) so I've marked it as such since it's not
worth a separate PR.
## How this was tested
New unit test on `log.Lazy` + `set` methods. Existing CI for the rest as
it's a refactor.
## Why this should be merged
Support for configurable `core/types.Block` with RLP encoding, including
interplay with `Body`.
## How this works
`Block` doesn't export most of its fields so relies on an internal type,
`extblock`, for RLP encoding. This type is modified to implement the
`rlp.Encoder` and `Decoder` methods as a point to inject hooks using
`rlp.Fields` (as in #120 for `Body`).
`Block` shares the same registered extra type as `Body`. Unlike
`Header`, which has its own field in a `Block`, the fields in `Body` are
promoted to be carried directly. This suggests that (at least for pure
data payloads) the modifications might be equivalent (and
`ava-labs/coreth` evidences this). Should different payloads be
absolutely required in the future, we can split the types—the
`RegisterExtras` signature is already too verbose though 😢.
## How this was tested
Explicit inclusion of a backwards-compatibility test for
`NOOPBlockBodyHooks` + implicit testing via the multiple upstream tests
in `block_test.go`. Re implicit testing: default behaviour is now to use
the noop hooks even when no registration is performed, but if we change
this then the tests in `block_test.go` can still be called as subtests
from a test that explicitly registers noops.
---------
Signed-off-by: Arran Schlosberg <519948+ARR4N@users.noreply.github.com>
Co-authored-by: Quentin McGaw <quentin.mcgaw@avalabs.org>
## Why this should be merged
Simplify testing in #130.
## How this works
Backwards-compatibility tests were originally created in `package
types_test` because the first one uses `ethtest` and would have caused a
circular dependency. None of the later tests have this issue so they
were moved into `package types`. The `cmpeth` test utility isn't
currently needed anywhere else so its functionality is collapsed inside
this new file.
## How this was tested
N/A as simply moving existing tests to different files.
Breaking change refactoring `core/types.Block.WithBody()` method from signature
WithBody(transactions []*Transaction, uncles []*Header) *Block
to signature
WithBody(body Body) *Block
such that block and body extras can be used within `WithBody`.
Note `geth` made the same change in method signature so the deltas introduced in this PR will disappear once we sync.
## Why this should be merged
Allows for `ava-labs/coreth` equivalent modifications of
`rawdb.InspectDatabase()` through external logic injection.
## How this works
Variadic options to:
1. Record a database statistic;
2. Mark a database statistic as metadata;
3. Filter statistics for printing.
## How this was tested
Testable example acting as a unit test.
---------
Signed-off-by: Arran Schlosberg <519948+ARR4N@users.noreply.github.com>
Co-authored-by: Quentin McGaw <quentin.mcgaw@avalabs.org>
## Why this should be merged
Simplification of `types.Body` RLP overriding, resulting in reduced code
at both the implementation and consumer ends.
## How this works
Introduction of `rlp.Fields` type, to mirror regular RLP encoding of a
struct. The RLP override hook now only needs to return the fields of
interest, which MAY come from either the `Body` or the registered extra.
This pattern allows for arbitrary modification of upstream fields via
(1) reordering; (2) addition; (3) deletion; and (4) inverting required
vs optional status. While less important for `Body`, this allows for
complete support of `ava-labs/coreth` `Header` modifications, which make
use of 1-3.
## How this was tested
Existing backwards-compatibility tests + new unit tests for introduced
functionality.
---------
Signed-off-by: Arran Schlosberg <519948+ARR4N@users.noreply.github.com>
Co-authored-by: Quentin McGaw <quentin.mcgaw@avalabs.org>
## Why this should be merged
Allows for modification of `types.Body` payload data + RLP encoding
without placing the entire RLP burden on the `libevm` user as we did
with `types.HeaderHooks`.
## How this works
RLP encoding of a struct is simply a concatenation of RLP encodings of
fields, encompassed by an RLP "list". The
`AppendRLPFields(rlp.EncoderBuffer, ...)` hook exploits this and plugs
in before all `rlp:"optional"`-tagged fields to allow for inclusion of
any new fields. The `EncoderBuffer` SHOULD be used as the `io.Writer`
passed when encoding each field: `rlp.Encode(buffer, fieldValue)`.
`Body` doesn't have `{En,De}codeRLP` methods so they are implemented to
identically replicate original behaviour when a no-op hook is present.
This pattern is sufficient for the `ava-labs/coreth` modifications of
`Body` but can be modified / extended for more complex scenarios, like
`Header`.
> [!NOTE]
> This PR does not include registration of the hooks as that was not the
initial goal and adding them would create too much PR bloat. There is a
placeholder `var todoRegisteredBodyHooks` global variable that can only
be set in tests.
## How this was tested
- Backwards compatibility: the new methods are fuzzed against a `type
withoutMethods Body` passed directly to `rlp.{En,De}code()`
- `coreth` compatibility: unit test of a local implementation of
`BodyHooks` demonstrating reproducibility of RLP encoding.
---------
Signed-off-by: Arran Schlosberg <519948+ARR4N@users.noreply.github.com>
Co-authored-by: Quentin McGaw <quentin.mcgaw@avalabs.org>
## Why this should be merged
JSON equivalent of #89.
## How this works
The check for registered extras, previously used in `{En,De}codeRLP()`
methods is abstracted into a `Header.hooks() HeaderHooks` method that
either returns (a) an instance of the registered type or (b) a
`NOOPHeaderHooks` if no registration was performed. This is then used
for all hooks, new (JSON) and old (RLP).
## How this was tested
Extension of existing unit tests.
## Why this should be merged
Aligns precompiled contracts with the pattern used in
`vm.EVMInterpreter` for regular contracts and simplifies gas accounting
by using existing mechanisms. The most notable simplification occurs
when there are multiple error paths that return early and have to
account for consumed gas (any local solution would likely just mirror
this one).
This forms part of a broader, long-term direction of feature parity
between precompiled and bytecode contracts, exposed via
`vm.PrecompileEnvironment`.
The `vm.Contract.Value()` method is also exposed as a natural
accompaniment to this change.
## How this works
The original signature for `vm.PrecompiledStatefulContract` received the
amount of gas available and then returned the amount remaining; this has
been removed in lieu of exposing the existing `vm.Contract.UseGas()`
method via the `vm.PrecompileEnvironment`. A new `legacy` package wraps
the old signature and converts it to a new one so `ava-labs/coreth`
doesn't need to be refactored.
```go
func oldPrecompile(env vm.PrecompileEnvironment, input []byte, gas uint64) ([]byte, uint64, error) {
// ...
if err != nil {
return nil, gas - gasCost, err // pattern susceptible to bugs; should it be `nil, gas, err` ?
}
// ...
return output, gas - gasCost, nil
}
func newPrecompile(env vm.PrecompileEnvironment, input []byte) ([]byte, error) {
// The original `gas` argument is still available as `env.Gas()`
// ...
if !env.UseGas(gasCost) { // an explicit point at which gas is consumed
return nil, vm.ErrOutOfGas
}
// ...
if err != nil {
return nil, err
}
// ...
return output, nil
}
```
## How this was tested
Existing unit test modified to use the `legacy` adaptor.
---------
Signed-off-by: Arran Schlosberg <519948+ARR4N@users.noreply.github.com>
Co-authored-by: Quentin McGaw <quentin.mcgaw@gmail.com>
## Why this should be merged
Replaces #86. That approach couldn't be replicated for generated JSON
marshalling, and this once also reduces the upstream delta.
## How this works
AST manipulation of a specified file, which finds specified methods and
modifies their names to be unexported.
## How this was tested
Inspection of the output (`core/types/gen_header_rlp.go` remains
unchanged).
## Why this should be merged
The `types.Header` fields of both
[`coreth`](https://pkg.go.dev/github.com/ava-labs/coreth/core/types#Header)
and
[`subnet-evm`](https://pkg.go.dev/github.com/ava-labs/subnet-evm/core/types#Header)
have been modified such that their RLP encodings (i.e. block hashes)
aren't compatible with vanilla `geth` nor each other. This PR adds
support for arbitrary RLP encoding coupled with type-safe extra
payloads.
## How this works
Equivalent to #1 (`params`) and #44 (`types.StateAccount`) registration
of pseudo-generic payloads. The only major difference is the guarantee
of a non-nil payload pointer, which means that the payload hooks are
never called on nil pointers as this would make it difficult to decode
RLP into them.
## How this was tested
Round-trip RLP {en,de}coding via a registered stub hook.
---------
Signed-off-by: Arran Schlosberg <519948+ARR4N@users.noreply.github.com>
Co-authored-by: Quentin McGaw <quentin.mcgaw@gmail.com>
## Why this should be merged
We're making changes to very sensitive code that, if broken, can result
in block hashes being different.
## How this works
Change-detector test.
## How this was tested
A randomly filled `types.Header` with expected RLP locked as a constant
in the test.
## Why this should be merged
This is a precursor to being able to override `types.Header` RLP
{en,de}coding. As there is already a `Header.EncodeRLP()` method we
either have to modify the generated code or rename the generated
method—this PR does the latter.
## How this works
The `rlpgen -internal_methods` flag changes the generated methods from
`EncodeRLP()` and `DecodeRLP()` to `encodeRLP()` and `decodeRLP()`,
respectively. A new CI job checks that generated code is up to date. We
can then implement our own `Header.EncodeRLP()` that either overrides or
falls back on the original.
It appears that `core/gen_genesis.go` was out of date but only because
of formatting.
## How this was tested
I deliberately excluded the change to `core/types/gen_header_rlp.go` to
confirm that the new workflow
[detects](https://github.com/ava-labs/libevm/actions/runs/12259667481/job/34202386378?pr=86#step:5:92)
the change and fails. The actual change can be inspected via the code
diff.
## Why this should be merged
There are 3 places at which we perform the same, sensitive logic to
access registered payloads and as we modify more types this is likely to
expand. (e.g. `types.Header`).
## How this works
Introduces `pseudo.Accessor` to abstract the reusable code.
## How this was tested
Existing unit tests. Note that the `types.StateAccount` tests needed a
minor refactor to provide the assertions with access to the
`ExtraPayloads[T]` without introducing generic types anywhere.
## Why this should be merged
Consolidates duplicated logic. Similar rationale to #84.
## How this works
New `register.AtMostOnce[T]` type is responsible for limiting calls to
`Register()`.
## How this was tested
Existing unit tests of `params`. Note that the equivalent functionality
in `types` wasn't tested but now is.
## Why this should be merged
Provides access to extra payloads registered on `types.SlimAccount`, not
just on regular `types.StateAccount`. This is a breaking syntax change
to have the method reflect the behaviour.
## How this works
Modify existing method to accept the `ExtraPayloadCarrier` interface
instead of `*StateAccount`. The interface is implemented by both
`*StateAccount` and `*SlimAccount`.
## How this was tested
Covered by existing testing.
## Why this should be merged
Allows for a drop-in replacement of `snapshot.Tree` (i.e. one that uses
block hashes instead of state roots). This is intended as a temporary
solution while we investigate having the state root affected by the
block hash to remove path ambiguity.
## How this works
Introduction of:
1. `state.SnapshotTree` interface to match methods required on
`snapshot.Tree` as used by `state.StateDB`; and
2. `stateconf` package for variadic options plumbed by
`StateDB.Commit()` through to `SnapshotTree.Update()`.
Although variadic (to maintain function call-signature compatibility)
only the `stateconf.WithUpdatePayload(any)` is expected to be used.
Recipients of the options can access the payload with
`stateconf.ExtractUpdatePayload()`.
## How this was tested
Unit test demonstrating propagation of `stateconf.UpdateOption` payload.
## Why this should be merged
Performs trie prefetching concurrently, required for equivalent
performance with `coreth` / `subnet-evm` implementations.
## How this works
`StateDB.StartPrefetcher()` accepts variadic options (for backwards
compatibility of function signatures). An option to specify a
`WorkerPool` is provided which, if present, is used to call
`Trie.Get{Account,Storage}()`; the pool is responsible for concurrency
but does not need to be able to wait on the work as that is handled by
this change.
## How this was tested
Unit test demonstrating hand-off of work to a `WorkerPool` as well as
API-guaranteed ordering of events.
## Why this should be merged
Simplifies the creation and application of variadic options; this is
also applicable to many of my other WIP branches (e.g. `snapshot`).
## How this works
See `libevm/options/options.go`, which is very simple.
## How this was tested
Existing tests of `vm.PrecompileEnvironment.Call()`, which itself uses
refactored options.
