From 08a03c77618e279afe47292ad3958ebb4bd3e1de Mon Sep 17 00:00:00 2001 From: rjl493456442 Date: Mon, 9 Jun 2025 18:01:38 +0800 Subject: [PATCH] ethdb/pebble: adjust the number of memory tables (#31970) This pull request adjusts the number of allowed memory tables in Pebble. Pebble allows configuring an arbitrary number of memory tables to hold unflushed data. When the current memtable becomes full, it is scheduled for flushing, and a new memtable is allocated to accept subsequent writes. However, if too many memtables accumulate and are waiting to be flushed, subsequent writes will stall. Originally, only two memtables were configured, each with a size of 512 MB for Ethereum mainnet. While this setup works well under normal conditions, it becomes problematic under heavy write loads. In such scenarios, flushing is only triggered when more than 512 MB of data is pending, which may not be responsive enough. Even worse, if compactions are running concurrently, flushing memtables can become slow due to the heavy IO overhead, leading to write stalls across the system. This pull request tries to mitigate the performance degradation by having more memory tables but with a smaller size. In this case, the pending writes can be flushed more smoothly and responsively. --- ethdb/pebble/pebble.go | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/ethdb/pebble/pebble.go b/ethdb/pebble/pebble.go index 5c851af910..aa9eca9c92 100644 --- a/ethdb/pebble/pebble.go +++ b/ethdb/pebble/pebble.go @@ -199,9 +199,12 @@ func New(file string, cache int, handles int, namespace string, readonly bool) ( // Taken from https://github.com/cockroachdb/pebble/blob/master/internal/constants/constants.go maxMemTableSize := (1<<31)<<(^uint(0)>>63) - 1 - // Two memory tables is configured which is identical to leveldb, - // including a frozen memory table and another live one. - memTableLimit := 2 + // Four memory tables are configured, each with a default size of 256 MB. + // Having multiple smaller memory tables while keeping the total memory + // limit unchanged allows writes to be flushed more smoothly. This helps + // avoid compaction spikes and mitigates write stalls caused by heavy + // compaction workloads. + memTableLimit := 4 memTableSize := cache * 1024 * 1024 / 2 / memTableLimit // The memory table size is currently capped at maxMemTableSize-1 due to a