## Why this should be merged
Fixes a potential race in parallel handler where the goroutine that
closes `whenProcessed` could observe a re-assigned channel for the next
block, leading to an incorrect close and possible double-close panic.
## How this works
`finishBlock()` waits for the channel to be closed by the goroutine,
instead of for the `WaitGroup` that closes _just_ before it, which is
what allowed the race.
## How this was tested
It's not possible to include a specific test. This PR therefore also
includes thorough documentation of every per-block goroutine and when it
is guaranteed to complete.
---------
Co-authored-by: Arran Schlosberg <me@arranschlosberg.com>
## Why this should be merged
EVM parallel co-processors that interface with the regular transaction
path via precompiles.
## How this works
Introduces the `parallel.Processor`, which orchestrates a set of
`parallel.Handler`s. Each `Handler` performs arbitrary, strongly typed
processing of any sub-set of transactions in a block and makes its
results available to a precompile and/or a post-block method for
persisting state. Although stateful, `Handler`s can only read the
pre-block and post-block state, which isolates them from conflicts with
the regular transaction path.
There is deliberately no support for a precompile to "write" to a
`Handler`, only to "read". This is because the transaction might still
revert, which would also have to be communicated to the `Handler`,
resulting in unnecessary complexity. Logs/events are the recommended
approach for precompile -> `Handler` communication, to be read from the
`types.Receipts` at the end of the block.
## How this was tested
Integration tests covering:
1. Selection of transactions to process + end-to-end plumbing of data
through a `Handler`.
2. Registration as a precompile, exercised with actual transaction
processing, and demonstrating log + return-data correctness.
---------
Signed-off-by: Arran Schlosberg <519948+ARR4N@users.noreply.github.com>
## Why this should be merged
See ava-labs/strevm#120. To get bloom filters for a block when the
header doesn't contain the correct bloom because of SAE's delayed
settlement, you need a custom accessor to provide arbitrary bloom.
## How this works
Introduces an optional interface which, if satisfied by
`filters.Backend` implementations, is used to override the Bloom filter
instead of retrieving it from the `types.Header`.
## How this was tested
Existing tests unchanged. Integration test demonstrates calling of the
override method when present.
---------
Co-authored-by: Arran Schlosberg <me@arranschlosberg.com>
This adds a test logger for libevm, that treats errors as they _should_
be treated.
- Logs at `log.LevelWarn` and above trigger `t.Errorf()`, causing tests
to fail
- Critical logs fail immediately: Logs at `log.LevelCrit` trigger
`t.Fatalf()`
- Info/debug logs pass through: Lower severity logs use `t.Logf()` for
informational output
You can use the logger like this:
```go
import (
"github.com/ava-labs/libevm/ethtest"
"github.com/ava-labs/libevm/log"
)
func TestSomething(t *testing.T) {
logger := log.NewLogger(ethtest.NewTBLogHandler(t, log.LevelDebug))
// Or to set globally
log.SetDefault(log.NewLogger(ethtest.NewTBLogHandler(t, log.LevelDebug))
}
```
I was thinking about adding tests to show this works, but it seemed
silly to test a testing utility.
---------
Signed-off-by: Jonathan Oppenheimer <147infiniti@gmail.com>
Co-authored-by: Arran Schlosberg <519948+ARR4N@users.noreply.github.com>
## Why this should be merged
Currently, `eth_close` is being registered as an available RPC due to
`Close` being an exported function on the `FilterAPI`.
## How this works
Converts the method to a function that takes in the API.
## How this was tested
Updated unit test.
## Why this should be merged
This allows closing the `timeoutLoop` goroutine. Closing goroutines is
the best-practice. But, more importantly, this causes the
`filters.Backend` to never be GCed. Which is causing GC tests to fail in
SAE.
## How this works
I tried to make this as minimal of a diff as possible.
## How this was tested
With this change, the SAE tests are passing:
https://github.com/ava-labs/strevm/pull/64
## Why this should be merged
Allow calls to `core.SetupGenesisBlock()` when `hookstest.Stub` is
registered as a `ChainConfig` extra.
## How this works
Implements `json.Marshaler` and `json.Unmarshaler` interfaces.
## How this was tested
Unit test of the root cause and the desired usage.
## Why this should be merged
In lieu of modifications to `core.IntrinsicGas()`, required for support
of Warp in SAE.
If we were to introduce variadic `options.Option`s to `IntrinsicGas()`,
it's impossible to guarantee that they would always be passed. The
`types.Transaction` itself isn't actually passed to `IntrinsicGas()`
either, so we can't rely on it to carry payloads/hooks.
## How this works
`vm.Hooks` are extended to include `PreprocessingGasCharge(tx
common.Hash) (uint64, error)`, which returns the amount of gas to be
charged between `core.IntrinsicGas` and `vm.EVM` execution charges. The
two entry points to execution, `vm.EVM.Call()` and `vm.EVM.Create()` are
modified to first spend said charge before running upstream logic.
The new hook is defined on a separate interface definition, embedded in
`vm.Hooks`, to allow types to implement just that method and enforce it
with the `var _ vm.Preprocessor = (*impl)(nil)` pattern.
## How this was tested
Integration tests via both `core.ApplyTransaction()` and
`core.ApplyMessage()`. The former is more comprehensive, while the
latter allows for inspection of interim error values.
---------
Signed-off-by: Arran Schlosberg <519948+ARR4N@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Austin Larson <78000745+alarso16@users.noreply.github.com>
## Why this should be merged
Allows ICM code to use the `abi` package from `libevm` instead of
`subnet-evm`, as part of a broader transition.
## How this works
Port and refactor of the following `abi.ABI` methods (with differences
described) from `subnet-evm`:
1. `PackEvent()` returns topics and packed data for events. Unlike
`subnet-evm`, the returned `[]common.Hash` topic slice MAY be nil
instead of the non-nil but empty equivalent. This is in keeping with the
upstream `abi.MakeTopics()`.
2. `PackOutput()` returns packed output for methods.
3. `UnpackInputIntoInterface()` unpacks method inputs or event arguments
into an arbitrary interface. Unlike the `subnet-evm` implementation, it
doesn't perform any checks on the length of the input buffer as these
are an Avalanche-specific feature that can be performed by the consumer.
## How this was tested
Unit tests from `subnet-evm`, refactored for clarity.
## Why this should be merged
Required for SAE to support more of the required APIs. I left out the
account and personal APIs, as we really shouldn't be managing any keys
on the node.
## How this works
Follows the same style that already existed. Additionally, copied the
upstream description for the types.
## How this was tested
Only local testing (although the prior functions were also not tested)
## Why this should be merged
The ability to deterministically generate private keys (and
corresponding EOAs) is useful in testing.
## How this works
A `[]byte` seed is written to a Keccak state, which is rejection-sampled
to find a scalar `<` the order of `S256` to use as the private key.
## How this was tested
Unit test that demonstrates (a) determinism of generation; and (b)
correct computation of the public key as proven by ECDSA recovery using
the standard tx-sender functionality.
## Why this should be merged
The `temporary.WithTempRegisteredExtras()` global lock introduced in
#234 wasn't fit for purpose when used in `coreth` as it required central
coordination of registration, types, and usage of the payload accessor.
## How this works
Instead of a central registration point, the new
`libevm.WithTemporaryExtrasLock()` function takes out a global lock and
provides the caller with a handle that proves the lock is held. All of
the override functions, e.g. `params.WithTempRegisteredExtras()` now
require a current lock, which will be propagated by the respective
`coreth` functions.
See https://github.com/ava-labs/coreth/pull/1328 for intended usage in
`coreth` and `subnet-evm`. A consumer of both of these can then safely
do the following:
```go
import (
"github.com/ava-labs/libevm/libevm"
coreth "github.com/ava-labs/coreth/plugin/evm"
subnet "github.com/ava-labs/subnet-evm/plugin/evm"
)
// asCChain calls `fn` while emulating `coreth`. It is safe for concurrent usage with [asSubnetEVM].
func asCChain(fn func() error) error {
return libevm.WithTemporaryExtrasLock(func(l libevm.ExtrasLock) error {
return coreth.WithTempRegisteredLibEVMExtras(l, fn)
})
}
// asSubnetEVM calls `fn` while emulating `subnet-evm`. It is safe for concurrent usage with [asCChain].
func asSubnetEVM(fn func() error) error {
return libevm.WithTemporaryExtrasLock(func(l libevm.ExtrasLock) error {
return subnet.WithTempRegisteredLibEVMExtras(l, fn)
})
}
```
## How this was tested
Unit test of the new function plus existing integration tests of all
modified code.
## Why this should be merged
Version increase across all `ava-labs` EVM-related code.
## How this works
Bump to 1.24.8 in both `go.mod` files + requisite version increase for
`golangci-lint`.
## How this was tested
Existing tests.
---------
Co-authored-by: Austin Larson <austin.larson@avalabs.org>
## Why this should be merged
The current rlpgen partially supports named types with basic underlying,
and it generates the rlp without correct conversion i.e:
`w.WriteUint64(obj.Uint64NewT)`
This PR adds the full support it
## How this works
Adds check for `isNamedWithBasicUnderlying` and then returns a `op` with
`makeNamedBasicOp` which is basically a `basicOp` but with the `typ`
refers to the named type so that it can correctly encode decode with the
named type.
## How this was tested
Added UT tests
---------
Signed-off-by: Ceyhun Onur <ceyhunonur54@gmail.com>
## Why this should be merged
The `vm.PrecompileEnvironment.Call()` method requires careful usage
because of reentrancy vulnerabilities (this is common to all outgoing
`CALL`s in the EVM, not just stateful precompiles). This package
provides a common method of protection, a reentrancy guard.
## How this works
Provides a function that returns `vm.ErrExecutionReverted` if called
twice, by the same contract, in the same transaction, with the same
identifier.
## How this was tested
Unit and integration tests.
## Why this should be merged
Registration of extras requires runtime enforcement of payload types. In
production, only a single set of types can be registered and any
attempts at re-registration will panic (by design). Although this makes
production usage safe, it doesn't allow downstream consumers (e.g. data
indexers) to use extras from different chains (e.g. `coreth` _and_
`subnet-evm`) at the same time.
## How this works
1. The `libevm/register.AtMostOnce` type can now be overridden
temporarily.
2. `params`, `core/types`, `core/vm`, and `state` packages introduce
`WithTempRegisteredExtras()` functions.
3. `libevm/temporary.WithRegisteredExtras()` provides "atomic" override
of all extras.
In all cases, the scope of the override is limited to the life of a
single function call.
## How this was tested
Relative to numbered list above:
1. Unit test of new and existing functionality.
2. Integration tests of both packages, demonstrating both payload and
behavioural override.
## Why this should be merged
Completeness.
## How this works
n/a
## How this was tested
Originally the `DELEGATECALL` test asserted that the `CALLVALUE` was
zero, but this didn't demonstrate that it was inherited (e.g.
`STATICCALL` always receives zero). The value sent to the caller of the
precompile is now non-zero, to precisely demonstrate correct behaviour
of all call types.
## Why this should be merged
Adds default-value support to the `options` package, which could
previously only create configured types from zero values.
## How this works
Exposes the option-application loop as the `ApplyTo()` function and
refactors `As()` to share this code path.
## How this was tested
Testable example acting as a unit test.
## Why this should be merged
Allows direct access to the P-256 verification precompile. Currently the
only alternative is to rely on the planned but not guaranteed address
that Ethereum will use.
## How this works
Embed the non-exported `p256verify` type in an exported struct to allow
its methods to be exposed.
## How this was tested
Run the test suite for `p256verify` on `P256Verify`.
## Why this should be merged
The need for examples to be in `package rawdb_test` made it difficult to
test other code that relied on non-exported `rawdb` identifiers.
## How this works
1. Move all testable examples from `database.libevm_test.go` to
`examples.libevm_test.go` _without any change_.
2. Change `database.libevm_test.go` to be in `package rawdb` and extend
the `WithSkipFreezers()` test.
## How this was tested
No change at all to testable examples. The `WithSkipFreezers()` test now
touches both the happy and (specific) error paths to demonstrate exact
behaviour (under the old approach it could have just been that there was
no error, even without the option).
## Why this should be merged
As described in https://github.com/ava-labs/coreth/issues/1137, database
inspection of a Coreth chain database fails because `InspectDatabase()`
expects for freezers to be used. This PR fixes this bug by adding an
option to skip freezer inspection.
This PR should be followed with a downstream PR in Coreth to pass in an
option to skip freezer inspection during database inspection.
## How this works
Extends `inspectDatabaseConfig` with a `skipFreezers` field and adds the
associated option function for it.
## How this was tested
CI + ran `InspectDatabase()` against Coreth locally
## Why this should be merged
Improved clarity.
## How this works
Semantically equivalent refactor. These values aren't accessible /
needed during execution of a stateful precompile, but are important when
making outgoing calls so were originally only set in the
`PrecompileEnvironment.Call()` implementation. This was confusing and
led to a false-positive bug report.
## How this was tested
Existing tests.
## Why this should be merged
Provides precompiles with unambiguous access to contextual addresses,
without the consumer needing to understand how they change under
different call types.
## How this works
The `libevm.AddressContext` type, which used to carry 3 addresses, now
provides different versions of `Caller` and `Self`. The EVM-semantic
versions are as defined by the rules of the EVM (and available before
this change). The raw versions are the unmodified caller and self.
## How this was tested
Extension of existing UTs to include raw addresses in addition to
existing, EVM-semantic ones.
## Why this should be merged
In case a tx's execution (specifically a registered precompile) should
be invalidated, this allows the EVM to find this error.
## How this works
Adds a setter and getter that can be called from a precompile.
## How this was tested
UT
## Why this should be merged
`ava-labs/coreth` has a partitioned state-address space, achieved by
setting or clearing a specific bit in the hash used to key the space.
This change allows such behaviour to be achieved with pure `libevm`
instead of the `StateDB` wrapping that `coreth` currently uses.
## How this works
Introduction of `state.StateDBHooks` interface, including a
`TransformStateKey()` method that allows for arbitrary change of state
key. If registered, this hook will be honoured by
`StateDB.{Get,GetCommitted,State}Key()` methods unless they receive a
`stateconf.SkipStateKeyTransformation` option.
## How this was tested
Unit test of `SetState() -> GetState() + GetCommittedState()` round trip
with and without options to skip.
## Why this should be merged
Coreth and Subnet-EVM require access to the current tx hash in the Warp
precompile (for selecting the correct predicate results).
This can not be exposed as a wrapper through the `OverrideEVMResetArgs`
because `SetTxContext` is called _prior_ to resetting the EVM instance.
Therefore, the `SetTxContext` function on any DB wrapper would never be
called.
## How this works
Exposes the existing `txHash` through a `TxHash()` function.
## How this was tested
Added a trivial unit test.
## Why this should be merged
Avoids false positives from `goheader`.
## How this works
Checks out entire history to give the linter a reference point for new
issues only.
## How this was tested
With great frustration by waiting for CI.
## Why this should be merged
`goheader` linter apparently hasn't been running but now decided to and
is blocking another PR. It's not worth the time to investigatie why
(probably a version update).
## How this works
Update header comments as required.
## How this was tested
N/A
## Why this should be merged
Addition of recent, functionality constitutes a bump to the minor
version. Doing this on `main` first for consistency and will cut a
release candidate after.
## How this works
Magic
## How this was tested
Super magic
## Why this should be merged
To allow more thorough handling of duplicate state roots (or any other
info other users would like), additional information can be provided to
a call of `statedb.Commit`. This change allows arbitrary types to be
sent to `triedb` as well as the `SnapshotTree`. However, this is a
breaking change for those using the functionality already, since the
snapshot commit option is wrapped with another call.
## How this works
See the edited libevm test for usage.
## How this was tested
Edited test case to include `TrieDBUpdateOption` and ensures the payload
is sent.
## Why this should be merged
#185 was allowed to merge despite CI requiring the `lint` job.
## How this works
Fix linter issues.
## How this was tested
The first commit is an empty commit, expected to fail CI, to confirm
that the job is in fact required.
## Why this should be merged
Required for
[ACP-194](https://github.com/avalanche-foundation/ACPs/tree/main/ACPs/194-streaming-asynchronous-execution#gas-charged)
$\lambda$ bound on gas consumption.
## How this works
Hook into `core.StateTransition.TransitionDb()` as this is the bottom of
all execution paths (e.g. `core.ApplyTransaction()` as used in SAE,
`core.StateProcessor.Process(*Block,...)`, etc.). Once consumed gas is
no longer changing (i.e. after all spends and refunds), the transaction
limit is passed to the hook to determine the minimum consumption, which
is applied.
## How this was tested
Unit test via `core.ApplyTransaction()` as this is our entry point in
SAE.
---------
Signed-off-by: Arran Schlosberg <519948+ARR4N@users.noreply.github.com>
Co-authored-by: Stephen Buttolph <stephen@avalabs.org>
