forks/go-ethereum
1
0
Fork 1
mirror of https://github.com/ethereum/go-ethereum.git synced 2026-07-28 07:36:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
go-ethereum/docs
History
rroblak ebc7dc231a
Add CLI flags to config LevelDB table/total sizes (#981) 
* Add CLI flags to config LevelDB table/total sizes

I wired up CLI flags to allow configuring LevelDB table and total sizes:
  - `--leveldb.compaction.table.size`, LevelDB SSTable file size factor in MiB (default: 2)
  - `--leveldb.compaction.table.multiplier`, multiplier on LevelDB SSTable file size (default: 1)
  - `--leveldb.compaction.total.size`, total size factor in MiB of LevelDB levels (default: 10)
  - `--leveldb.compaction.total.multiplier`, multiplier on LevelDB total level size (default: 10)

N.B. that the default values for these configs are exactly the same as
before this changset and so Bor behavior should not change unless these
flags are deliberately overridden. Bor/Geth inherited the default values
from [the `goleveldb`
defaults](126854af5e/leveldb/opt/options.go).

We (Alchemy) found it necessary to override these configs as follows to
keep Bor archive nodes tracking the canonical chain:
  - `--leveldb.compaction.table.size=4`
  - `--leveldb.compaction.total.size=20`

These overrides double the size of LevelDB SSTable files (2 MiB -> 4
MiB) and also the total amount of data in each level (100 MiB -> 200
MiB, 1,000 MiB -> 2,000 MiB, etc.). The idea is to have LevelDB read and
write data in larger chunks while keeping the proportional frequency of
compaction operations the same as in the original defaults defined by
Dean and Ghemawat.

Without these overrides we found that our archive nodes would tend to
fall into a "LevelDB compaction loop of death" where the incoming stream
of blockchain data could not be flowed into LevelDB's structure quickly
enough, resulting in the node blocking writes for long periods of time
while LevelDB's single-threaded compaction organized the data.  Over
time the nodes would fall farther and farther behind the canonical chain
head, metaphorically dying a slow node's death.

These configs can be changed on existing node databases (resyncing is
not necessary). LevelDB appears to work correctly with SSTable files of
different sizes. Note that the database does not undergo any sort of
migration when changing these configs. Only newly-written files (due to
new data or compaction) are affected by these configs.

* Update docs

* Adjust line spacing for linter

* Replace map with `ExtraDBConfig`

* Rename `LevelDbConfig` to `ExtraDBConfig`

* Regenerate docs
2023-09-12 12:09:26 +02:00
..
audits Update Bor to geth-v1.9.16 (#78) 2020-07-24 19:01:41 +05:30
cli Add CLI flags to config LevelDB table/total sizes (#981) 2023-09-12 12:09:26 +02:00
postmortems docs/postmortems: remove wrong parentheses (#26066) 2022-11-01 14:50:03 +01:00
README.md Merge branch 'qa' and 'master' into develop (#663) 2023-01-13 14:47:14 +05:30

README.md

Documentation

The new command line interface (CLI) in this version of Bor aims to give users more control over the codebase when interacting with and starting a node. We have made every effort to keep most of the flags similar to the old CLI, except for a few notable changes. One major change is the use of the --config flag, which previously represented fields without available flags. It now represents all flags available to the user, and will overwrite any other flags if provided. As a node operator, you still have the flexibility to modify flags as needed. Please note that this change does not affect the internal functionality of the node, and it remains compatible with Geth and the Ethereum Virtual Machine (EVM).

Additional notes

  • The new entrypoint to run the Bor client is server.

    $ bor server <flags>

    See here for more flag details.

  • The bor dumpconfig sub-command prints the default configurations, in the TOML format, on the terminal. One can pipe (>) this to a file (say config.toml) and use it to start bor.

  • A toml file now can be used instead of flags and can contain all configuration for the node to run. To simply run bor with a configuration file, the following command can be used.

    $ bor server --config <path_to_config.toml>

  • You can find an example config file here to know more about what each flag is used for, what are the defaults and recommended values for different networks.

  • Toml files used earlier (with --config flag) to configure additional fields (like static and trusted nodes) are being deprecated and have been converted to flags.