-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Centos Stream 9: error getting disk usage ("/sys/kernel/debug/tracing"): permission denied #10897
Comments
Same Issue on Raspbian (64bit) on Raspberrypi, Kernel: 5.10.103-v8+: |
Same here on Ubuntu 20.04.4 using telegraf output: |
@shorton3 @CalvinSchwartz @arestifo https://access.redhat.com/solutions/5914171 |
This is happening on Debian
Permissions on the directory
|
@reimda it looks like in #10527 you changed a debug to an error, which is causing these messages to show up now. As an example, on my local system with no configuration of
This does not stop the plugin from correctly collecting data, only produces the above messages. Running on older versions does not produce such a message without Should this message get switched back to a debug? Thanks! |
You have to add telegraf to dkcer group --> usermod -aG docker telegraf |
Refers to: https://access.redhat.com/solutions/5914171 Try this: [[inputs.disk]] |
In plugins/input/system/ps.go https://github.com/powersj/telegraf/blob/b9e66f8b9a2d97e09a7e2c1ff5b2bc08bc4eeebc/plugins/inputs/system/ps.go#L157 It looks like we need to be more careful about the log level we're using. I switched everything from debug level to error level, but a 'permission denied' should be a warning as it doesn't cause the plugin to stop working, and only makes it stop producing metrics for the filesystem involved. |
It makes sense to me to add tracefs to the ignore_fs default. I don't think it's a filesystem anyone is going to want to know the free space of. |
Same problem with telegraf 1.22.0-1 on clean Debian 11 UPD. Adding |
added |
The tracefs filesystem is showing up more and is not likely something a user wishes to have enabled by default. This adds it to the ignore in the config by default. This also reduces the error message level from error to warning. This was previously a debug message, but was bumped all the way to an error. This message does not prevent any messages from getting read by Telegraf and not an error that halts Telegraf. Users may still wish to address the messages so reducing to a warning is better than hiding them. Fixes: influxdata#10897
I have put up #10925 which will reduce the error message to a warning and correctly add tracefs to the list of default ignored filesystem types. If anyone has time, it would be great if you could download a build artifact and ensure the messages look right. Thanks! |
I have a similar problem, but the difference is that the problem is occurring on bind mounts that Nomad is creating:
Is there any current way to exclude these? |
The tracefs filesystem is showing up more and is not likely something a user wishes to have enabled by default. This adds it to the ignore in the config by default. This also reduces the error message level from error to debug. This was previously a debug message, but was bumped all the way to an error. This message does not prevent any messages from getting read by Telegraf and not an error that halts Telegraf. Users may still wish to address the messages so reducing to a debug is better than hiding them. Fixes: influxdata#10897
i had to add "fuse.gvfsd-fuse" to my ignores on xfce |
In addition to adding the filesystem type to the ignore list, v1.22.3 of telegraf, released yesterday, has reduced these log messages back to debug level. |
@0xTH0R is this a nested mount? If so, can you please open a new issue? |
Same problem on Debian 12 and adding It seems that the thing in brackets is a file system location that corresponds to something in It's now complaining about
but However I see
so adding |
@rwb196884 please open a new issue with the information above and an (redacted) excerpt of the log. |
If you run telegraf service not under root user than your debug command is not correct, you shoud specify telegraf user: sudo -u telegraf telegraf --debug --config /etc/telegraf/telegraf.conf --input-filter disk --test |
Relevant telegraf.conf
Logs from Telegraf
System info
telegraf-1.22.0-1.x86_64
Docker
No response
Steps to reproduce
1.start telegraf and look in /var/log/messages
2.
3.
...
Expected behavior
no permission denied logs
Actual behavior
permission denied logs
Additional info
No response
The text was updated successfully, but these errors were encountered: