diff --git a/docs/fundamentals/blsync.md b/docs/fundamentals/blsync.md index 796432f71d..8e66ef383a 100644 --- a/docs/fundamentals/blsync.md +++ b/docs/fundamentals/blsync.md @@ -9,7 +9,26 @@ description: Running geth with integrated beacon light client ## Usage -### Integrated mode +### Using Dynamic Checkpoint Fetch + +This will automatically fetch the latest finalized checkpoint and launch Geth in snap sync mode with light client support (blsync). + +Replace `` with your select beacon api provider from the provided list of [lightsync endpoints](https://s1na.github.io/light-sync-endpoints/). Replace `` with a url from the list of maintained [checkpoint sync endpoints](https://eth-clients.github.io/checkpoint-sync-endpoints/). The following command can be used to run geth client with blsync directly. + +```terminal +export BEACON= && \ +export CHECKPOINT= && \ +geth --beacon.api=$BEACON --beacon.checkpoint=$(curl -s $CHECKPOINT/checkpointz/v1/status | jq -r '.data.finality.finalized.root') +``` + +#### Example +```terminal +export BEACON=https://lodestar-sepolia.chainsafe.io && \ +export CHECKPOINT=https://sepolia.beaconstate.info && \ +geth --sepolia --beacon.api=$BEACON --beacon.checkpoint=$(curl -s $CHECKPOINT/checkpointz/v1/status | jq -r '.data.finality.finalized.root') +``` + +### Running with Manual Checkpoint Fetch To run blsync as part of Geth, you need to specify a public HTTP endpoint and a checkpoint: @@ -29,10 +48,26 @@ A checkpoint is the block root of the first proposed slot of a finalized beacon - Copy the block root field. ![Copy the block root](/images/docs/blsync3.png) + +## Testing a Beacon API Endpoint + +To verify that your Beacon API is reachable and returning valid data, you can use a simple `curl` command to request the light client bootstrap header for a given block root. + +Replace `` with your Beacon API domain, and `` with the hex-encoded block root you want to test. + +```terminal +curl -H "Accept: application/json" http:///eth/v1/beacon/light_client/bootstrap/ +``` + #### Example +```terminal +curl -H "Accept: application/json" http://testing.holesky.beacon-api.nimbus.team/eth/v1/beacon/light_client/bootstrap/0x82e6ba0e288db5eb79c328fc6cb03a6aec921b00af6888bd51d6b000e68e75ac +``` The following command can be used to start Geth with blsync on the Sepolia network. Note that the checkpoint root will be outdated two weeks after the writing of this page and a recent one will have to be picked according to the guide above: +#### Example + ```terminal ./build/bin/geth --sepolia --beacon.api https://sepolia.lightclient.xyz --beacon.checkpoint 0x0014732c89a02315d2ada0ed2f63b32ecb8d08751c01bea39011b31ad9ecee36 ``` diff --git a/docs/fundamentals/pruning.md b/docs/fundamentals/dbpruning.md similarity index 99% rename from docs/fundamentals/pruning.md rename to docs/fundamentals/dbpruning.md index 5f7582ce01..1d16f41a53 100644 --- a/docs/fundamentals/pruning.md +++ b/docs/fundamentals/dbpruning.md @@ -1,5 +1,5 @@ --- -title: Pruning +title: Database Pruning description: Instructions for pruning a Geth node --- diff --git a/docs/fundamentals/downloadera.md b/docs/fundamentals/downloadera.md new file mode 100644 index 0000000000..7adc11f6bb --- /dev/null +++ b/docs/fundamentals/downloadera.md @@ -0,0 +1,49 @@ +--- +title: Download Era +description: Instructions for downloading Era files +--- + +Era downloads are used to retrieve historical block bodies and receipts data that have been expired or pruned from a Geth node. Era files allow operators to efficiently reconstruct history without a full sync. Geth nodes with history expiry enabled can prune historical block bodies and receipts to significantly reduce storage requirements. However, in some cases operators may want to selectively restore some of this history for research, debugging, or compliance purposes. Era files provide an efficient way to retrieve historical data directly from trusted servers without needing to re-sync the entire chain. The geth `download-era` command enables targeted retrieval of this data. + +## Requirements {#requirements} +Before downloading Era files, ensure sufficient disk space is available for storing the downloaded files. You may download era files while your node is running. + +Era files are indexed by block or epoch range. When downloading: +1. Geth queries the era server for the file corresponding to the requested range. +2. Downloaded files are automatically verified against known checksums. +3. Verified files are placed into the ancient store directory, ready for Geth to use. + +## Download Era Command {#command} +```sh +geth download-era --server [--block | --epoch | --all] --datadir +``` + +| Flag | Description | +| ----------- | -------------------------------------------------------- | +| `--server` | (Required) URL of the era server | +| `--block` | Block number or range to download (e.g. `100000-200000`) | +| `--epoch` | Epoch number or range to download (e.g. `100-200`) | +| `--all` | Download all available era files | +| `--datadir` | Geth datadir where era files will be stored | + + +Range formats: + - Single value: `500` → downloads only block or epoch `500` + - Range: `100-200` → downloads inclusive range from `100`to `200` + +Servers can be found at the following link: [Ethereum History Endpoints](https://eth-clients.github.io/history-endpoints/). This link will have the most updates list of clients serving era files. Currently, geth only supports era files of the `Era1` specification, so make sure download accordingly. + +## Example {#example} +Download epochs `100-300`: +```sh +geth download-era --server https://mainnet.era1.nimbus.team --epoch 100-300 --datadir /mnt/geth-data +``` + +Download all: +```sh +geth download-era --server https://mainnet.era1.nimbus.team --all --datadir /mnt/geth-data +``` + + + + diff --git a/docs/fundamentals/historypruning.md b/docs/fundamentals/historypruning.md new file mode 100644 index 0000000000..49ea7181f4 --- /dev/null +++ b/docs/fundamentals/historypruning.md @@ -0,0 +1,130 @@ +--- +title: History Pruning +description: Guide to pruning history on geth to decrease space consumption +--- + +Running a full Geth node consumes significant disk space over time. Much of this storage is occupied by historical block bodies and receipts that are rarely accessed. Geth provides a history pruning tool to safely remove this old history while preserving full chain validation. + +## Pruning prerequisites {#prerequisites} + +Before running the pruning command, you must ensure: +- Geth is fully stopped +- No other process is accessing the database + +Running history pruning while Geth is online will fail and may corrupt data. + +## Running the pruning command {#running} + +Once Geth is stopped, run: + +```sh +geth prune-history --datadir /path/to/your/geth/datadir +``` + +This command will scan your database and delete historical block bodies and receipts up to the merge block (i.e. historical data from the proof-of-work chain that is no longer needed for consensus). + +The state trie and all headers are preserved. Your node remains fully functional for consensus. + +After pruning is completed, you may continue running geth! + +## Measuring the storage savings {#measuring} + +Before and after pruning you can inspect your database to measure storage savings with. This should take around 30 minutes to run, and will provide a detailed measure of your storage: + +```sh +geth db inspect --datadir /mnt/history_expiry_vol_data/gdata +``` + +Example output before pruning (mainnet): +```sh ++-----------------------+-----------------------------+------------+------------+ +| DATABASE | CATEGORY | SIZE | ITEMS | ++-----------------------+-----------------------------+------------+------------+ +| Key-Value store | Headers | 59.92 KiB | 90 | +| Key-Value store | Bodies | 7.63 MiB | 90 | +| Key-Value store | Receipt lists | 7.65 MiB | 90 | +| Key-Value store | Difficulties (deprecated) | 0.00 B | 0 | +| Key-Value store | Block number->hash | 3.69 KiB | 90 | +| Key-Value store | Block hash->number | 889.35 MiB | 22745224 | +| Key-Value store | Transaction index | 13.85 GiB | 401894661 | +| Key-Value store | Log index filter-map rows | 13.31 GiB | 136688603 | +| Key-Value store | Log index last-block-of-map | 2.79 MiB | 61014 | +| Key-Value store | Log index block-lv | 45.18 MiB | 2368590 | +| Key-Value store | Log bloombits (deprecated) | 0.00 B | 0 | +| Key-Value store | Contract codes | 10.16 GiB | 1673786 | +| Key-Value store | Hash trie nodes | 0.00 B | 0 | +| Key-Value store | Path trie state lookups | 229.86 KiB | 5741 | +| Key-Value store | Path trie account nodes | 47.29 GiB | 410145969 | +| Key-Value store | Path trie storage nodes | 180.07 GiB | 1791897413 | +| Key-Value store | Verkle trie nodes | 0.00 B | 0 | +| Key-Value store | Verkle trie state lookups | 0.00 B | 0 | +| Key-Value store | Trie preimages | 0.00 B | 0 | +| Key-Value store | Account snapshot | 13.76 GiB | 299492286 | +| Key-Value store | Storage snapshot | 95.46 GiB | 1323423323 | +| Key-Value store | Beacon sync headers | 654.00 B | 1 | +| Key-Value store | Clique snapshots | 0.00 B | 0 | +| Key-Value store | Singleton metadata | 302.46 MiB | 19 | +| Ancient store (Chain) | Hashes | 824.28 MiB | 22745135 | +| Ancient store (Chain) | Bodies | 655.94 GiB | 22745135 | +| Ancient store (Chain) | Receipts | 253.70 GiB | 22745135 | +| Ancient store (Chain) | Headers | 10.93 GiB | 22745135 | +| Ancient store (State) | History.Meta | 442.68 KiB | 5738 | +| Ancient store (State) | Account.Index | 61.89 MiB | 5738 | +| Ancient store (State) | Storage.Index | 80.13 MiB | 5738 | +| Ancient store (State) | Account.Data | 68.90 MiB | 5738 | +| Ancient store (State) | Storage.Data | 27.08 MiB | 5738 | ++-----------------------+-----------------------------+------------+------------+ +| TOTAL | 1.27 TIB | | ++-----------------------+-----------------------------+------------+------------+ +``` + +Example output after pruning (mainnet): +```sh ++-----------------------+-----------------------------+------------+------------+ +| DATABASE | CATEGORY | SIZE | ITEMS | ++-----------------------+-----------------------------+------------+------------+ +| Key-Value store | Headers | 59.92 KiB | 90 | +| Key-Value store | Bodies | 7.63 MiB | 90 | +| Key-Value store | Receipt lists | 7.65 MiB | 90 | +| Key-Value store | Difficulties (deprecated) | 0.00 B | 0 | +| Key-Value store | Block number->hash | 3.69 KiB | 90 | +| Key-Value store | Block hash->number | 889.35 MiB | 22745224 | +| Key-Value store | Transaction index | 13.85 GiB | 401894661 | +| Key-Value store | Log index filter-map rows | 13.31 GiB | 136688603 | +| Key-Value store | Log index last-block-of-map | 2.79 MiB | 61014 | +| Key-Value store | Log index block-lv | 45.18 MiB | 2368590 | +| Key-Value store | Log bloombits (deprecated) | 0.00 B | 0 | +| Key-Value store | Contract codes | 10.16 GiB | 1673786 | +| Key-Value store | Hash trie nodes | 0.00 B | 0 | +| Key-Value store | Path trie state lookups | 229.86 KiB | 5741 | +| Key-Value store | Path trie account nodes | 47.29 GiB | 410145969 | +| Key-Value store | Path trie storage nodes | 180.07 GiB | 1791897413 | +| Key-Value store | Verkle trie nodes | 0.00 B | 0 | +| Key-Value store | Verkle trie state lookups | 0.00 B | 0 | +| Key-Value store | Trie preimages | 0.00 B | 0 | +| Key-Value store | Account snapshot | 13.76 GiB | 299492286 | +| Key-Value store | Storage snapshot | 95.46 GiB | 1323423323 | +| Key-Value store | Beacon sync headers | 654.00 B | 1 | +| Key-Value store | Clique snapshots | 0.00 B | 0 | +| Key-Value store | Singleton metadata | 302.46 MiB | 19 | +| Ancient store (Chain) | Bodies | 412.23 GiB | 7207742 | +| Ancient store (Chain) | Receipts | 137.01 GiB | 7207742 | +| Ancient store (Chain) | Headers | 10.93 GiB | 7207742 | +| Ancient store (Chain) | Hashes | 824.28 MiB | 7207742 | +| Ancient store (State) | History.Meta | 442.68 KiB | 5738 | +| Ancient store (State) | Account.Index | 61.89 MiB | 5738 | +| Ancient store (State) | Storage.Index | 80.13 MiB | 5738 | +| Ancient store (State) | Account.Data | 68.90 MiB | 5738 | +| Ancient store (State) | Storage.Data | 27.08 MiB | 5738 | ++-----------------------+-----------------------------+------------+------------+ +| TOTAL | 936.34 GIB | | ++-----------------------+-----------------------------+------------+------------+ +``` + + + + + + + + diff --git a/src/data/documentation-links.yaml b/src/data/documentation-links.yaml index 50680eae35..9c8b74b903 100644 --- a/src/data/documentation-links.yaml +++ b/src/data/documentation-links.yaml @@ -32,8 +32,12 @@ to: /docs/fundamentals/logs - id: Connecting to peers to: /docs/fundamentals/peer-to-peer - - id: Pruning - to: /docs/fundamentals/pruning + - id: History Pruning + to: /docs/fundamentals/historypruning + - id: Database Pruning + to: /docs/fundamentals/dbpruning + - id: Download Eras + to: /docs/fundamentals/downloadera - id: Private networks via Kurtosis to: /docs/fundamentals/kurtosis - id: Private networks