eth_call and debug_traceCall allow users to override various block fields, among them base fee. However the overriden base fee was not considered for computing the effective gas price of that message, and instead base fee of the base block was used. This has been fixed in this commit.
This PR fixes an overflow which can could happen if inconsistent blockchain rules were configured. Additionally, it tries to prevent such inconsistencies from occurring by making sure that merge cannot be enabled unless previous fork(s) are also enabled.
As mentioned in #26621, the block index format for era1 is not in line with the regular era block index. This change modifies the index so all relative offsets are based against the beginning of the block index record.
This change adds support for blob-transaction in certain API-endpoints, e.g. eth_fillTransaction. A follow-up PR will add support for signing such transactions.
* all: implement era format, add history importer/export
* internal/era/e2store: refactor e2store to provide ReadAt interface
* internal/era/e2store: export HeaderSize
* internal/era: refactor era to use ReadAt interface
* internal/era: elevate anonymous func to named
* cmd/utils: don't store entire era file in-memory during import / export
* internal/era: better abstraction between era and e2store
* cmd/era: properly close era files
* cmd/era: don't let defers stack
* cmd/geth: add description for import-history
* cmd/utils: better bytes buffer
* internal/era: error if accumulator has more records than max allowed
* internal/era: better doc comment
* internal/era/e2store: rm superfluous reader, rm superfluous testcases, add fuzzer
* internal/era: avoid some repetition
* internal/era: simplify clauses
* internal/era: unexport things
* internal/era,cmd/utils,cmd/era: change to iterator interface for reading era entries
* cmd/utils: better defer handling in history test
* internal/era,cmd: add number method to era iterator to get the current block number
* internal/era/e2store: avoid double allocation during write
* internal/era,cmd/utils: fix lint issues
* internal/era: add ReaderAt func so entry value can be read lazily
Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
* internal/era: improve iterator interface
* internal/era: fix rlp decode of header and correctly read total difficulty
* cmd/era: fix rebase errors
* cmd/era: clearer comments
* cmd,internal: fix comment typos
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
This change makes use of uin256 to represent balance in state. It touches primarily upon statedb, stateobject and state processing, trying to avoid changes in transaction pools, core types, rpc and tracers.
* changed parallel universe HF to napoli HF
* integration-tests: update ipc path on ci tests (#1127)
* integration-tests: update ipc path on ci tests
* added devnetBorFlags in matic-cli-config.yml
---------
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
* Revert "integration-tests: update ipc path on ci tests (#1127)"
This reverts commit 0660fac39b.
* using cancun for block-stm metadata instead of napoli
* added a check to verify tx dependencies and test cases
* fix in snapshot.chainConfig
* removed snapshot.*params.BorConfig using snapshot.chainConfig.Bor instead
* removed unnecessary if statement in ParallelStateProcessor.Process
* addressed comment
* small fix in commitTransactions
* dependency calculation bug fix in miner/worker.go
---------
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Arpit Temani <temaniarpit27@gmail.com>
This change simplifies the logic for indexing transactions and enhances the UX when transaction is not found by returning more information to users.
Transaction indexing is now considered as a part of the initial sync, and `eth.syncing` will thus be `true` if transaction indexing is not yet finished. API consumers can use the syncing status to determine if the node is ready to serve users.
EIP-4844 adds a new transaction type for blobs. Users can submit such transactions via `eth_sendRawTransaction`. In this PR we refrain from adding support to `eth_sendTransaction` and in fact it will fail if the user passes in a blob hash.
However since the chain can handle such transactions it makes sense to allow simulating them. E.g. an L2 operator should be able to simulate submitting a rollup blob and updating the L2 state. Most methods that take in a transaction object should recognize blobs. The change boils down to adding `blobVersionedHashes` and `maxFeePerBlobGas` to `TransactionArgs`. In summary:
- `eth_sendTransaction`: will fail for blob txes
- `eth_signTransaction`: will fail for blob txes
The methods that sign txes does not, as of this PR, add support the for new EIP-4844 transaction types. Resuming the summary:
- `eth_sendRawTransaction`: can send blob txes
- `eth_fillTransaction`: will fill in a blob tx. Note: here we simply fill in normal transaction fields + possibly `maxFeePerBlobGas` when blobs are present. One can imagine a more elaborate set-up where users can submit blobs themselves and we fill in proofs and commitments and such. Left for future PRs if desired.
- `eth_call`: can simulate blob messages
- `eth_estimateGas`: blobs have no effect here. They have a separate unit of gas which is not tunable in the transaction.
Given the discussions around deprecating pending (see #28623 or ethereum/execution-apis#495), we can move away from using the pending block internally, and use latest instead
Here we update the eth and snap protocol test suites with a new test chain,
created by the hivechain tool. The new test chain uses proof-of-stake. As such,
tests using PoW block propagation in the eth protocol are removed. The test suite
now connects to the node under test using the engine API in order to make it
accept transactions.
The snap protocol test suite has been rewritten to output test descriptions and
log requests more verbosely.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This is primarily to make lint work again on macOS 14. The older version of golangci-lint kept crashing.
Also included is a fix for a goroutine leak in the recently-introduced function MustRunCommandWithOutput.
Certain flags, such as `--rpc.txfeecap` currently do not have an env-var auto-generated for them. This change adds three missing cli flag types to the auto env-var helper function to fix this.
This change
- Removes interface `log.Format`,
- Removes method `log.FormatFunc`,
- unexports `TerminalHandler.TerminalFormat` formatting methods (renamed to `TerminalHandler.format`)
- removes the notion of `log.Lazy` values
The lazy handler was useful in the old log package, since it
could defer the evaluation of costly attributes until later in the
log pipeline: thus, if the logging was done at 'Trace', we could
skip evaluation if logging only was set to 'Info'.
With the move to slog, this way of deferring evaluation is no longer
needed, since slog introduced 'Enabled': the caller can thus do
the evaluate-or-not decision at the callsite, which is much more
straight-forward than dealing with lazy reflect-based evaluation.
Also, lazy evaluation would not work with 'native' slog, as in, these
two statements would be evaluated differently:
```golang
log.Info("foo", "my lazy", lazyObj)
slog.Info("foo", "my lazy", lazyObj)
```
These changes improves the performance of the non-coloured terminal formatting, _quite a lot_.
```
name old time/op new time/op delta
TerminalHandler-8 10.2µs ±15% 5.4µs ± 9% -47.02% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
TerminalHandler-8 2.17kB ± 0% 0.40kB ± 0% -81.46% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
TerminalHandler-8 33.0 ± 0% 5.0 ± 0% -84.85% (p=0.008 n=5+5)
```
I tried to _somewhat_ organize the commits, but the it might still be a bit chaotic. Some core insights:
- The function `terminalHandler.Handl` uses a mutex, and writes all output immediately to 'upstream'. Thus, it can reuse a scratch-buffer every time.
- This buffer can be propagated internally, making all the internal formatters either write directly to it,
- OR, make use of the `tmp := buf.AvailableBuffer()` in some cases, where a byte buffer "extra capacity" can be temporarily used.
- The `slog` package uses `Attr` by value. It makes sense to minimize operating on them, since iterating / collecting into a new slice, iterating again etc causes copy-on-heap. Better to operate on them only once.
- If we want to do padding, it's better to copy from a constant `space`-buffer than to invoke `bytes.Repeat` every single time.
This PR replaces Geth's logger package (a fork of [log15](https://github.com/inconshreveable/log15)) with an implementation using slog, a logging library included as part of the Go standard library as of Go1.21.
Main changes are as follows:
* removes any log handlers that were unused in the Geth codebase.
* Json, logfmt, and terminal formatters are now slog handlers.
* Verbosity level constants are changed to match slog constant values. Internal translation is done to make this opaque to the user and backwards compatible with existing `--verbosity` and `--vmodule` options.
* `--log.backtraceat` and `--log.debug` are removed.
The external-facing API is largely the same as the existing Geth logger. Logger method signatures remain unchanged.
A small semantic difference is that a `Handler` can only be set once per `Logger` and not changed dynamically. This just means that a new logger must be instantiated every time the handler of the root logger is changed.
----
For users of the `go-ethereum/log` module. If you were using this module for your own project, you will need to change the initialization. If you previously did
```golang
log.Root().SetHandler(log.LvlFilterHandler(log.LvlInfo, log.StreamHandler(os.Stderr, log.TerminalFormat(true))))
```
You now instead need to do
```golang
log.SetDefault(log.NewLogger(log.NewTerminalHandlerWithLevel(os.Stderr, log.LevelInfo, true)))
```
See more about reasoning here: https://github.com/ethereum/go-ethereum/issues/28558#issuecomment-1820606613
* eth/gasestimator: early exit for plain transfer and error allowance
* core, eth/gasestimator: hard guess at a possible required gas
* internal/ethapi: update estimation tests with the error ratio
* eth/gasestimator: I hate you linter
* graphql: fix gas estimation test
---------
Co-authored-by: Oren <orenyomtov@users.noreply.github.com>
* core/vm: set basefee to 0 internally on eth_call
* core: nicer 0-basefee, make it work for blob fees too
* internal/ethapi: make tests a bit more complex
* core: fix blob fee checker
* core: make code a bit more readable
* core: fix some test error strings
* core/vm: Get rid of weird comment
* core: dict wrong typo
a little copying is better than a little dependency
-- go proverb
We have this dependency on docker, a.k.a moby: a gigantic library, and we only need ~70 LOC,
so here I tried moving it inline instead.
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR is a bit in preparation for the slog work in #28187 .
Our current test re logging mostly test the internals, but we have no real end-to-end test of the logging output. This PR introduces a simple reexec-based log tester. This also relies upon a special mode in geth, which can be made to eject a set of predefined log messages (only available if the build-tag `integrationtests` is used
e.g. go run --tags=integrationtests ./cmd/geth --log.format terminal logtest
While working on this, I also noticed a quirk in the setup: when geth was configured to use a file output, then two separate handlers were used (one handler for the file, one handler for the console). Using two separate handlers means that two formatters are used, thus the formatting of any/all records happened twice. This PR changes the mechanism to use two separate io.Writers instead, which is both more optimal and fixes a bug which occurs due to a global statefulness in the formatter.
This change fixes#28355, where eth_getProof failed to return the correct codehash under certain conditions. This PR changes the logic to unconditionally look up the codehash, and also adds some more tests.
* build: upgrade to golang 1.21.2
* build: verify checksums via tool
* deps: upgrade go to 1.21.3
* build: move more build metadata into checksum file
* build: move gobootsrc to checksums
So apparently in the spec the base block parameter of eth_call is optional.
I agree that "latest" is a sane default for this that most people would use.
* internal/cli: add support to overwrite config.toml via cli flags
* fix lint and refactor
* add extensive tests for flagset
* fix type conversion for big.Int
* add more tests for coverage
* add t.parallel
* internal/cli/flagset: handle flag conversion using interface
* internal/cli/flagset: fix test
* added new api to support conditional transactions (EIP-4337) (#700)
* Refactored the code and updated the miner to check for the validity of options (#793)
* refactored the code and updated the miner to check for the validity of options
* added new errors -32003 and -32005
* added unit tests
* addressed comments
* Aa 4337 update generics (#799)
* poc
* minor bug fix
* use common.Hash
* updated UnmarshalJSON function (reference - tynes)
* fix
* done
* linters
* with test
* undo some unintentional changes
---------
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
* handelling the block range and timestamp range, also made timestamp a pointer
---------
Co-authored-by: Evgeny Danilenko <6655321@bk.ru>
* Added filtering of conditional transactions in txpool (#920)
* added filtering of conditional transactions in txpool
* minor fix in ValidateKnownAccounts
* bug fix
* Supporting nil knownAccounts
* lints
* bundled transactions are not announced/broadcasted to the peers
* fixed after upstream merge
* few fixes
* sentry reject conditional transaction
* Changed the namespace of conditional transaction API from `eth` to `bor` (#985)
* added conditional transaction to bor namespace
* test comit
* test comit
* added conditional transaction
* namespapce changed to bor
* cleanup
* cleanup
* addressed comments
* reverted changes in ValidateKnownAccounts
* addressed comments and removed unwanted code
* addressed comments
* bug fix
* lint
* removed licence from core/types/transaction_conditional_test.go
---------
Co-authored-by: Evgeny Danilenko <6655321@bk.ru>
* Add CLI flags to config LevelDB table/total sizes
I wired up CLI flags to allow configuring LevelDB table and total sizes:
- `--leveldb.compaction.table.size`, LevelDB SSTable file size factor in MiB (default: 2)
- `--leveldb.compaction.table.multiplier`, multiplier on LevelDB SSTable file size (default: 1)
- `--leveldb.compaction.total.size`, total size factor in MiB of LevelDB levels (default: 10)
- `--leveldb.compaction.total.multiplier`, multiplier on LevelDB total level size (default: 10)
N.B. that the default values for these configs are exactly the same as
before this changset and so Bor behavior should not change unless these
flags are deliberately overridden. Bor/Geth inherited the default values
from [the `goleveldb`
defaults](126854af5e/leveldb/opt/options.go).
We (Alchemy) found it necessary to override these configs as follows to
keep Bor archive nodes tracking the canonical chain:
- `--leveldb.compaction.table.size=4`
- `--leveldb.compaction.total.size=20`
These overrides double the size of LevelDB SSTable files (2 MiB -> 4
MiB) and also the total amount of data in each level (100 MiB -> 200
MiB, 1,000 MiB -> 2,000 MiB, etc.). The idea is to have LevelDB read and
write data in larger chunks while keeping the proportional frequency of
compaction operations the same as in the original defaults defined by
Dean and Ghemawat.
Without these overrides we found that our archive nodes would tend to
fall into a "LevelDB compaction loop of death" where the incoming stream
of blockchain data could not be flowed into LevelDB's structure quickly
enough, resulting in the node blocking writes for long periods of time
while LevelDB's single-threaded compaction organized the data. Over
time the nodes would fall farther and farther behind the canonical chain
head, metaphorically dying a slow node's death.
These configs can be changed on existing node databases (resyncing is
not necessary). LevelDB appears to work correctly with SSTable files of
different sizes. Note that the database does not undergo any sort of
migration when changing these configs. Only newly-written files (due to
new data or compaction) are affected by these configs.
* Update docs
* Adjust line spacing for linter
* Replace map with `ExtraDBConfig`
* Rename `LevelDbConfig` to `ExtraDBConfig`
* Regenerate docs
* Milestone Implementation
* Merge branch 'POS-347' into reciept-e2e-test
* Changes for testing, will be removed after testing
* Debugged the error
* Just for testing purpose
* refactor debug api methods, rename whitelist -> checkpoint
* remove first iteration based vars
* fix linters
* Rewind Changes
* Error changes
* RewindBack function in bor_checkpoint_verifier
* Testcases
* Added the fetch test for milestone and checkpoint
* Debugged the lint changes
* Debugged the lint changes
* Debugged the lint changes
* Debugged the lint changes
* Improved the error in Miner test file
* Improved the error of pointing to the wrong function
* Locking the sprint after the vote has been made on it.
* Adding more logs for testing
* Adding more logs for testing
* Adding more logs for testing
* Implemented the NoAckMilestone fetching mechanism
* Testcases for milestone implementation
* Testing code for fetchNoAckMilestone and fetchLastNoAckMilestone
* Testing changes
* refactor else-if
* Corrected the number of params in bor_ext.go
* Dummy API for testing
* Defined the GetVoteOnRootHash in interface
* Defined the GetVoteOnRootHash in interface
* Made changes in the web3ext file
* Added the GetVoteOnRootHash in PublicBlockChain API
* Added the GetVoteOnRootHash in filterBackend
* Added the log of Root and RooHash
* Removed the 0x from rootHash
* Just for testing purpose
* "GetVoteOnRootHash" mock implementation
* bor_test.go
* Added the test for milestone implementation
* Added service for fetching milestone by ID
* Improved the comments
* Removed the duplicate code
* use setter for borVerifier
* use setter for borVerifier
* refactor handleNoAckMilestone
* remove code repetition with retry function
* Converged the repetitive code
* after CR
* persistence
* persistence implementation
* feature flag
* Persistence Changes
* cr
* initial
* fix
* fix
* Whitelist Flag
* 1 Add:Included the milestone flag 2.Add:Hardlimit the rewind to maximum of 255 blocks
* Chg:Updated go.mod file
* Remove:Dubai Hardfork code
* Add:checked errors for call functions to the Db, Rmv: Remote Header variable from the IsValidPeer() function
* Fix:Linting issues'
* Add:MilestoneGRPC functions
* Fix:Lint issues
* Fix:Lint issues
* Fix: TestFetchMilestoneFromMockHeimdall
* Fix:Integrations tests
* Add:Test for sprint length and milestone changes
* Add:Functionality to fetch the finalized block
* Chg:Changed default val of TriesInmemory to 1024
* fix:Some functions of heimdallGRPC client
* Restored the GRPC functionality, was commented out for developing purpose
* Fix:Bor_checkpoint_Verfier function
* Test:Added the chain Rewinding test
* Test:Added the Sprint Length + Milestone merge test
* Add:Implemented the future milestone
* Add:Future milestone changes
* Add:Future milestone changes
* Chg: Voting on endBlockHash rather than rootHash
* Chg: Changed the logic of future milestone from rootHash checking to endBlockHash checking
* Fix:Using endBockHash while verifying the incoming milestone
* Chg:Variable names for better readiblity
* Fix:Testing changes
* Add:metrics for milestone implementation
* Add:Metrics for milestone implementatian
* Fix:Order of statements in a function for better optimization
* Chg:Removed unrequired file
* Fix:new variable intialization
* Add:Comment to increase readiblity
* Fix:Logs
* Chg:Name of GetVoteOnRootHash to GetVoteOnHash
* Fix:Linting issues
* Fixed linting issues
* Rmv: Unnecessary logs and Add:Skip test for long tests
* Fix:Checking current chain with whitelisted milestone or checkpoint in Finalized block function
* Fix:Test
* Fix:Whitelisting of Milestone and Checkpoint process
* Fix: Milestone JSON structure
* Chg:Testcases changes
* Fix:Change from VoteOnRootHash to VoteOnHash
* Fix:Variable name fix
* Fix:Finalized API
* internal/jsre/deps: update web3.js bundle
* Fix:milestone verifier
* Chg:Handling the long future chain import issue
* Fix:Lint issues
* Fix:TestLowDiffLongChain and TestPrunedImportSide tests, used hardcoded value 128 instead of DefaultTriesInMemory value
* Chg:Testcode for producing metrics
* Chg:Milestong polling value to 32 secs
* Add:Testcases
* Add:Implemented the check to fetch the milestoneId from heimdall before locking the fork
* Added GRPC method for FetchMilestoneID
* Fix:lint issue
* Fix:lint issue
* Skiped out the tests which were mainly used to produce the supporting data
* remove vcs build when running snyk
* Add:Improved the logs and comments
* fix linters
* Skipped some test as they are panic due to timeout issue in github
* Chg:Variable name LockerSprintNumber to LockedMilestoneNumber for better readablity and clarity
* Chg:Conflicting variable names in milestone test file
* Chg:Conflicting function names in milestone test file
* fix : minor fix in TestInsertingSpanSizeBlocks
* Fix:Mocking issue in TestInsertingSpanSizeBlocks
* Fix:GRPC Polyproto Version
* eth/downloader: skip peer drop due to whitelisting err
* eth, tests/bor: bug fixes and minor refactor
* Add:Implemented the milestone related functions in the HeimdallApp
* Fix:Lint Errors & Remove:Redundant Code
* Fix:Testing Errors
* Fix:Bor integeration tests
* Fix:Test errors
* update heimdall client mock files
* remove unused arguments
* remove redundant code
* Chg:Changed the milestone polling intervals
* Add: added block finality from whitelisted checkpoint
* skip future chain validation
* Add:confirmation check of 16 blocks over the end block while voting for the milestone in GetVoteHash() function
* Chg:Included endBlockNum in UnlockMutex function
* Add:Property based test for milestone
* Fix:Opening the lock while processing future milestone
* Add:Property based test for futureMilestone
* Defined the value of TempTriesInMemory
* Fixed the finalized api
* Fixed lint issues
* eth: add logs while fetching and rewinding
* fix linters: use default returns instead of recursive calls
* Fix:Milestone intergration test
* Add:GetVoteHash fn in mock backend
* tests/bor: fix mock span
* tests/bor: remove t.Parallel()
* use bor namespace in ethclient, fix mock function
---------
Co-authored-by: Vaibhav Jindal <vaibhavjindal29@gmail.com>
Co-authored-by: VaibhavJindal <74560896+VAIBHAVJINDAL3012@users.noreply.github.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Evgeny Danienko <6655321@bk.ru>
Co-authored-by: Shivam Sharma <shivam691999@gmail.com>
Co-authored-by: Anshal Shukla <shukla.anshal85@gmail.com>
This PR makes EIP-4788 work in the engine API and miner. It also fixes some bugs related to
EIP-4844 block processing and mining. Changes in detail:
- Header.BeaconRoot has been renamed to ParentBeaconRoot.
- The engine API now implements forkchoiceUpdatedV3
- newPayloadV3 method has been updated with the parentBeaconBlockRoot parameter
- beacon root is now applied to new blocks in miner
- For EIP-4844, block creation now updates the blobGasUsed field of the header
Optimizations:
- Previously, if a transaction was reverting, EstimateGas would exhibit worst-case behavior and binary search up to the max gas limit (~40 state-clone + tx executions). This change allows EstimateGas to return after only a single unconstrained execution in this scenario.
- Uses the gas used from the unconstrained execution to bias the remaining binary search towards the likely solution in a simple way that doesn't impact the worst case. For a typical contract-invoking transaction, this reduces the median number of state-clone+executions from 25 to 18 (28% reduction).
Cleanup:
- added & improved function + code comments
- correct the EstimateGas documentation to clarify the gas limit determination is at latest block, not pending, if the blockNr is unspecified.
The Go authors updated golang/x/ext to change the function signature of the slices sort method.
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just
picked a new version that some other dep depends on, causing our code to fail building.
This PR updates the dep on our code too and does all the refactorings to follow upstream...
The Go authors updated golang/x/ext to change the function signature of the slices sort method.
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just
picked a new version that some other dep depends on, causing our code to fail building.
This PR updates the dep on our code too and does all the refactorings to follow upstream...
