Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

go collector: add default metrics acceptance tests; adding more context to HELP #1568

Merged
merged 2 commits into from
Aug 1, 2024
Merged

go collector: add default metrics acceptance tests; adding more context to HELP #1568

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cd96dc8
go collector: add default metrics acceptance tests; adding more conte…
bwplotka Jul 30, 2024
39dcda8
Update prometheus/go_collector.go
bwplotka Aug 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 44 additions & 1 deletion prometheus/collectors/go_collector_latest_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,31 @@ var baseMetrics = []string{
"go_threads",
}

var memstatMetrics = []string{
"go_memstats_alloc_bytes",
"go_memstats_alloc_bytes_total",
"go_memstats_buck_hash_sys_bytes",
"go_memstats_frees_total",
"go_memstats_gc_sys_bytes",
"go_memstats_heap_alloc_bytes",
"go_memstats_heap_idle_bytes",
"go_memstats_heap_inuse_bytes",
"go_memstats_heap_objects",
"go_memstats_heap_released_bytes",
"go_memstats_heap_sys_bytes",
"go_memstats_lookups_total",
"go_memstats_mallocs_total",
"go_memstats_mcache_inuse_bytes",
"go_memstats_mcache_sys_bytes",
"go_memstats_mspan_inuse_bytes",
"go_memstats_mspan_sys_bytes",
"go_memstats_next_gc_bytes",
"go_memstats_other_sys_bytes",
"go_memstats_stack_inuse_bytes",
"go_memstats_stack_sys_bytes",
"go_memstats_sys_bytes",
}

func TestGoCollectorMarshalling(t *testing.T) {
reg := prometheus.NewRegistry()
reg.MustRegister(NewGoCollector(
Expand All @@ -55,6 +80,24 @@ func TestGoCollectorMarshalling(t *testing.T) {
}
}

func TestWithGoCollectorDefault(t *testing.T) {
reg := prometheus.NewRegistry()
reg.MustRegister(NewGoCollector())
result, err := reg.Gather()
if err != nil {
t.Fatal(err)
}

got := []string{}
for _, r := range result {
got = append(got, r.GetName())
}

if diff := cmp.Diff(got, withBaseMetrics(memstatMetrics)); diff != "" {
t.Errorf("[IMPORTANT, those are default metrics, can't change in 1.x] missmatch (-want +got):\n%s", diff)
}
}

func TestWithGoCollectorMemStatsMetricsDisabled(t *testing.T) {
reg := prometheus.NewRegistry()
reg.MustRegister(NewGoCollector(
Expand Down Expand Up @@ -192,7 +235,7 @@ func TestGoCollectorDenyList(t *testing.T) {
func ExampleGoCollector() {
reg := prometheus.NewRegistry()

// Register the GoCollector with the default options. Only the base metrics will be enabled.
// Register the GoCollector with the default options. Only the base metrics and memstats are enabled.
reg.MustRegister(NewGoCollector())

http.Handle("/metrics", promhttp.HandlerFor(reg, promhttp.HandlerOpts{}))
Expand Down
17 changes: 9 additions & 8 deletions prometheus/go_collector.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,21 +22,21 @@ import (
// goRuntimeMemStats provides the metrics initially provided by runtime.ReadMemStats.
// From Go 1.17 those similar (and better) statistics are provided by runtime/metrics, so
// while eval closure works on runtime.MemStats, the struct from Go 1.17+ is
// populated using runtime/metrics.
// populated using runtime/metrics. Those are the defaults we can't alter.
func goRuntimeMemStats() memStatsMetrics {
return memStatsMetrics{
{
desc: NewDesc(
memstatNamespace("alloc_bytes"),
"Number of bytes allocated and still in use.",
"Number of bytes allocated and currently in use.",
nil, nil,
),
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Alloc) },
valType: GaugeValue,
}, {
desc: NewDesc(
memstatNamespace("alloc_bytes_total"),
"Total number of bytes allocated, even if freed.",
"Total number of bytes allocated until now, even if released already.",
nil, nil,
),
eval: func(ms *runtime.MemStats) float64 { return float64(ms.TotalAlloc) },
Expand All @@ -60,23 +60,24 @@ func goRuntimeMemStats() memStatsMetrics {
}, {
desc: NewDesc(
memstatNamespace("mallocs_total"),
"Total number of mallocs.",
// TODO(bwplotka): We could add go_memstats_heap_objects, probably useful for discovery. Let's gather more feedback, kind of waste of bytes for everybody for compatibility reason.
"Total number of heap objects allocated, semantically a counter version for go_memstats_heap_objects gauge.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose "Total number of heap objects allocated, both live and gc-ed. Semantically a counter version for go_memstats_heap_objects gauge." Apologies for the lack of clarity :)

bwplotka marked this conversation as resolved.
Show resolved Hide resolved
nil, nil,
),
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Mallocs) },
valType: CounterValue,
}, {
desc: NewDesc(
memstatNamespace("frees_total"),
"Total number of frees.",
"Total number of heap objects frees.",
nil, nil,
),
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Frees) },
valType: CounterValue,
}, {
desc: NewDesc(
memstatNamespace("heap_alloc_bytes"),
"Number of heap bytes allocated and still in use.",
"Number of heap bytes allocated and currently in use.",
nil, nil,
),
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapAlloc) },
Expand Down Expand Up @@ -116,7 +117,7 @@ func goRuntimeMemStats() memStatsMetrics {
}, {
desc: NewDesc(
memstatNamespace("heap_objects"),
"Number of allocated objects.",
"Number of currently allocated objects.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether here we could add something to the effect of "(regardless of released)" (like in pprof) or "(both live and garbage collected)" to "Number of currently allocated objects" to be on the super-safe side. It's just a detail, of course.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, so this is current number, so it does not include released objects. Did you mean to add more details on some _total metric here? Like mallocs_total or alloc_bytes_total? Feel free to suggest with

image

the exact description for those, we could add that both live and gc-ed line if you want, good idea 💪🏽

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, it should be for the counter version, mallocs_total.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you propose a description that you want? (:

nil, nil,
),
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapObjects) },
Expand Down Expand Up @@ -225,7 +226,7 @@ func newBaseGoCollector() baseGoCollector {
nil, nil),
gcDesc: NewDesc(
"go_gc_duration_seconds",
"A summary of the pause duration of garbage collection cycles.",
"A summary of the wall-time pause (stop-the-world) duration in garbage collection cycles.",
bwplotka marked this conversation as resolved.
Show resolved Hide resolved
nil, nil),
gcLastTimeDesc: NewDesc(
"go_memstats_last_gc_time_seconds",
Expand Down
Loading