go-ethereum/metrics
Daniel Liu dddd6c57cd metrics: improve reading Go runtime metrics (#25886)
This changes how we read performance metrics from the Go runtime. Instead
of using runtime.ReadMemStats, we now rely on the API provided by package
runtime/metrics.

runtime/metrics provides more accurate information. For example, the new
interface has better reporting of memory use. In my testing, the reported
value of held memory more accurately reflects the usage reported by the OS.

The semantics of metrics system/memory/allocs and system/memory/frees have
changed to report amounts in bytes. ReadMemStats only reported the count of
allocations in number-of-objects. This is imprecise: 'tiny objects' are not
counted because the runtime allocates them in batches; and certain
improvements in allocation behavior, such as struct size optimizations,
will be less visible when the number of allocs doesn't change.

Changing allocation reports to be in bytes makes it appear in graphs that
lots more is being allocated. I don't think that's a problem because this
metric is primarily interesting for geth developers.

The metric system/memory/pauses has been changed to report statistical
values from the histogram provided by the runtime. Its name in influxdb has
changed from geth.system/memory/pauses.meter to
geth.system/memory/pauses.histogram.

We also have a new histogram metric, system/cpu/schedlatency, reporting the
Go scheduler latency.
2024-12-13 14:00:13 +08:00
..
exp metrics/exp: allow configuring metrics HTTP server on separate endpoint (#21290) 2024-12-13 14:00:12 +08:00
influxdb metrics: improve reading Go runtime metrics (#25886) 2024-12-13 14:00:13 +08:00
librato metrics: fix issues reported by staticcheck (#20365) 2024-12-13 14:00:12 +08:00
prometheus metrics: fix some typos (#25551) 2024-12-13 14:00:12 +08:00
config.go metrics/influxdb: support V2 (#23194) 2024-12-13 14:00:12 +08:00
counter.go metrics: added NewCounterForced (#17919) 2024-12-13 14:00:11 +08:00
counter_test.go metrics: fix staticcheck warning ST1017: don't use yoda conditions 2024-10-24 16:07:11 +08:00
cpu.go metrics: disable CPU stats (gosigar) on iOS (#20816) 2024-12-13 14:00:12 +08:00
cpu_disabled.go metrics: add go:build lines (#23468) 2024-12-13 14:00:12 +08:00
cpu_enabled.go metrics: fix compilation for GOOS=js (#23449) 2024-12-13 14:00:12 +08:00
cputime_nop.go metrics: add go:build lines (#23468) 2024-12-13 14:00:12 +08:00
cputime_unix.go metrics: fix compilation for GOOS=js (#23449) 2024-12-13 14:00:12 +08:00
debug.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
debug_test.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
disk.go all: fix license headers one more time 2015-07-23 18:35:11 +02:00
disk_linux.go all: fix ineffectual assignments and remove uses of crypto.Sha3 2017-01-09 16:24:42 +01:00
disk_nop.go all: fix staticcheck warning ST1005: incorrectly formatted error string 2024-10-24 09:48:20 +08:00
doc.go metrics: not compare float numbers directly (#20219) 2024-12-13 14:00:12 +08:00
ewma.go metrics: make meter updates lock-free (#21446) 2024-12-13 14:00:12 +08:00
ewma_test.go metrics: not compare float numbers directly (#20219) 2024-12-13 14:00:12 +08:00
FORK.md metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
gauge.go core, metrics: switch some invalid counters to gauges (#20047) 2024-05-10 19:48:10 +08:00
gauge_float64.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
gauge_float64_test.go metrics: fix some typos (#25551) 2024-12-13 14:00:12 +08:00
gauge_test.go metrics: fix some typos (#25551) 2024-12-13 14:00:12 +08:00
graphite.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
graphite_test.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
healthcheck.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
histogram.go metrics: add handler performance metrics (#22581) 2024-12-13 14:00:12 +08:00
histogram_test.go metrics: fix staticcheck warning ST1017: don't use yoda conditions 2024-10-24 16:07:11 +08:00
init_test.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
json.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
json_test.go all: fix staticcheck warning SA1006 2024-10-28 15:13:21 +08:00
LICENSE metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
log.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
memory.md metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
meter.go metrics: zero temp variable in updateMeter (#21470) 2024-12-13 14:00:12 +08:00
meter_test.go metrics: zero temp variable in updateMeter (#21470) 2024-12-13 14:00:12 +08:00
metrics.go metrics: improve reading Go runtime metrics (#25886) 2024-12-13 14:00:13 +08:00
metrics_test.go metrics: improve reading Go runtime metrics (#25886) 2024-12-13 14:00:13 +08:00
opentsdb.go metrics: remove redundant type specifiers (#19090) 2024-12-13 14:00:11 +08:00
opentsdb_test.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
README.md metrics: change links in README.md to https (#20182) 2024-12-13 14:00:11 +08:00
registry.go FIx Bad block error. 2021-09-17 17:59:06 +05:30
registry_test.go metrics: fix issues reported by staticcheck (#20365) 2024-12-13 14:00:12 +08:00
resetting_sample.go metrics: use resetting histograms for rare packets (#22586) 2024-12-13 14:00:12 +08:00
resetting_timer.go FIx Bad block error. 2021-09-17 17:59:06 +05:30
resetting_timer_test.go FIx Bad block error. 2021-09-17 17:59:06 +05:30
runtimehistogram.go metrics: improve reading Go runtime metrics (#25886) 2024-12-13 14:00:13 +08:00
runtimehistogram_test.go metrics: improve reading Go runtime metrics (#25886) 2024-12-13 14:00:13 +08:00
sample.go metrics: fix staticcheck warning ST1017: don't use yoda conditions 2024-10-24 16:07:11 +08:00
sample_test.go metrics: add missing calls to Ticker.Stop in tests (#20866) 2024-12-13 14:00:12 +08:00
syslog.go metrics: add go:build lines (#23468) 2024-12-13 14:00:12 +08:00
timer.go metrics: fix issues reported by staticcheck (#20365) 2024-12-13 14:00:12 +08:00
timer_test.go metrics: make flawed test less flawed (#20818) 2024-12-13 14:00:12 +08:00
validate.sh metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
writer.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00
writer_test.go metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910) 2018-02-23 11:56:08 +02:00

go-metrics

travis build status

Go port of Coda Hale's Metrics library: https://github.com/dropwizard/metrics.

Documentation: https://godoc.org/github.com/rcrowley/go-metrics.

Usage

Create and update metrics:

c := metrics.NewCounter()
metrics.Register("foo", c)
c.Inc(47)

g := metrics.NewGauge()
metrics.Register("bar", g)
g.Update(47)

r := NewRegistry()
g := metrics.NewRegisteredFunctionalGauge("cache-evictions", r, func() int64 { return cache.getEvictionsCount() })

s := metrics.NewExpDecaySample(1028, 0.015) // or metrics.NewUniformSample(1028)
h := metrics.NewHistogram(s)
metrics.Register("baz", h)
h.Update(47)

m := metrics.NewMeter()
metrics.Register("quux", m)
m.Mark(47)

t := metrics.NewTimer()
metrics.Register("bang", t)
t.Time(func() {})
t.Update(47)

Register() is not threadsafe. For threadsafe metric registration use GetOrRegister:

t := metrics.GetOrRegisterTimer("account.create.latency", nil)
t.Time(func() {})
t.Update(47)

NOTE: Be sure to unregister short-lived meters and timers otherwise they will leak memory:

// Will call Stop() on the Meter to allow for garbage collection
metrics.Unregister("quux")
// Or similarly for a Timer that embeds a Meter
metrics.Unregister("bang")

Periodically log every metric in human-readable form to standard error:

go metrics.Log(metrics.DefaultRegistry, 5 * time.Second, log.New(os.Stderr, "metrics: ", log.Lmicroseconds))

Periodically log every metric in slightly-more-parseable form to syslog:

w, _ := syslog.Dial("unixgram", "/dev/log", syslog.LOG_INFO, "metrics")
go metrics.Syslog(metrics.DefaultRegistry, 60e9, w)

Periodically emit every metric to Graphite using the Graphite client:


import "github.com/cyberdelia/go-metrics-graphite"

addr, _ := net.ResolveTCPAddr("tcp", "127.0.0.1:2003")
go graphite.Graphite(metrics.DefaultRegistry, 10e9, "metrics", addr)

Periodically emit every metric into InfluxDB:

NOTE: this has been pulled out of the library due to constant fluctuations in the InfluxDB API. In fact, all client libraries are on their way out. see issues #121 and #124 for progress and details.

import "github.com/vrischmann/go-metrics-influxdb"

go influxdb.InfluxDB(metrics.DefaultRegistry,
  10e9, 
  "127.0.0.1:8086", 
  "database-name", 
  "username", 
  "password"
)

Periodically upload every metric to Librato using the Librato client:

Note: the client included with this repository under the librato package has been deprecated and moved to the repository linked above.

import "github.com/mihasya/go-metrics-librato"

go librato.Librato(metrics.DefaultRegistry,
    10e9,                  // interval
    "example@example.com", // account owner email address
    "token",               // Librato API token
    "hostname",            // source
    []float64{0.95},       // percentiles to send
    time.Millisecond,      // time unit
)

Periodically emit every metric to StatHat:

import "github.com/rcrowley/go-metrics/stathat"

go stathat.Stathat(metrics.DefaultRegistry, 10e9, "example@example.com")

Maintain all metrics along with expvars at /debug/metrics:

This uses the same mechanism as the official expvar but exposed under /debug/metrics, which shows a json representation of all your usual expvars as well as all your go-metrics.

import "github.com/rcrowley/go-metrics/exp"

exp.Exp(metrics.DefaultRegistry)

Installation

go get github.com/rcrowley/go-metrics

StatHat support additionally requires their Go client:

go get github.com/stathat/go

Publishing Metrics

Clients are available for the following destinations: