-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Postgres output plugin show error with custom "create_templates" #15273
Comments
Hi,
I think this error message is also important, no? It sounds like you may be trying to access an existing table with schema. Since you are asking Telegraf to set the schema, you probably need to add Similar to this comment: #8651 (comment) |
Hi, Thanks for responding! 2024-05-01T19:06:23Z I! [outputs.postgresql] PG Exec - map[args:] commandTag:CREATE TABLE pid:435887 sql:CREATE TABLE "public"."sanbi.wurnet.nl" ("time" timestamp without time zone, "cpu" text, "device" tex │ But if I add the "create_template" property the error occurs. |
Right and I didn't say public, I said |
I'm probably just not understanding it properly. The schema "telegraf" does not exist in the database, the schema "public" does. And that's the scheme I want to use. Without the "create_templates" property, the table is created in the "public" schema as expected. |
My reading of the comment I linked to was that you needed to use the telegraf schema after creating the schema. Similar to the original PR comment:
|
I see what you mean. Unfortunately I get the same error after setting schema = "telegraf" |
Somehow it looks like the name of the table, and perhaps other values also, is not passed to the template. |
can you provide an example data point please? |
I'm quite new to telegraf so I hope this is an example of a data point: sanbi.wurnet.nl,host=sanbi.wurnet.nl,topic=unlock/fdp/sanbi.wurnet.nl/system uptime="7985729" 1714594920000000000 Without the create_templates this is saved successful in a Postgres table "sanbi.wurnet.nl" in the public schema. |
Launched postgres via: docker run -it --rm --net host -e POSTGRES_PASSWORD=password postgres My config: [agent]
debug = true
omit_hostname = true
[[outputs.file]]
[[inputs.file]]
files = ["data"]
data_format = "influx"
[[outputs.postgresql]]
connection = "host=localhost user=postgres password=password sslmode=disable dbname=postgres"
schema = "public"
create_templates = [
'''CREATE TABLE {{.table}} ({{.columns}})''',
] ❯ ./telegraf --config config.toml --once
2024-05-01T20:43:17Z I! Loading config: config.toml
2024-05-01T20:43:17Z I! Starting Telegraf 1.31.0-553d972c brought to you by InfluxData the makers of InfluxDB
2024-05-01T20:43:17Z I! Available plugins: 234 inputs, 9 aggregators, 32 processors, 25 parsers, 60 outputs, 6 secret-stores
2024-05-01T20:43:17Z I! Loaded inputs: file
2024-05-01T20:43:17Z I! Loaded aggregators:
2024-05-01T20:43:17Z I! Loaded processors:
2024-05-01T20:43:17Z I! Loaded secretstores:
2024-05-01T20:43:17Z I! Loaded outputs: file postgresql
2024-05-01T20:43:17Z I! Tags enabled:
2024-05-01T20:43:17Z D! [agent] Initializing plugins
2024-05-01T20:43:17Z D! [agent] Connecting outputs
2024-05-01T20:43:17Z D! [agent] Attempting connection to [outputs.file]
2024-05-01T20:43:17Z D! [agent] Successfully connected to outputs.file
2024-05-01T20:43:17Z D! [agent] Attempting connection to [outputs.postgresql]
2024-05-01T20:43:18Z D! [agent] Successfully connected to outputs.postgresql
2024-05-01T20:43:18Z D! [agent] Starting service inputs
2024-05-01T20:43:18Z D! [agent] Stopping service inputs
2024-05-01T20:43:18Z D! [agent] Input channel closed
2024-05-01T20:43:18Z I! [agent] Hang on, flushing any cached metrics before shutdown
sanbi.wurnet.nl,host=sanbi.wurnet.nl,topic=unlock/fdp/sanbi.wurnet.nl/system uptime="7985729" 1714594920000000000
2024-05-01T20:43:18Z D! [outputs.file] Wrote batch of 1 metrics in 17.33µs
2024-05-01T20:43:18Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2024-05-01T20:43:18Z D! [outputs.postgresql] Wrote batch of 1 metrics in 18.449166ms
2024-05-01T20:43:18Z D! [outputs.postgresql] Buffer fullness: 0 / 10000 metrics
2024-05-01T20:43:18Z I! [agent] Stopping running outputs
2024-05-01T20:43:18Z D! [agent] Stopped Successfully
|
Thanks for testing. I's really strange, I have the exact same config. Especially the fact that it is working without the "create_templates". I will keep on trying and keep you posted. Thanks a lot so far! |
Hi, I just found out what my problem is. I'm using a helm chart for deploying Telegraf to kubernetes. Helm sees the accolades as part of its own templating and removes them. So only this is put in the config file: create_templates = [ And then everything makes sense. So now I only have to find out how to make helm ignore those accolades. |
using the following works: create_templates = [ |
oh wow - TIL! Thank you very much for digging into it! I'll go ahead and close this then. |
Relevant telegraf.conf
Logs from Telegraf
System info
Telegraf: 1.30.2, Kubernetes: v1.25.16, Postgres: 16
Docker
No response
Steps to reproduce
...
Expected behavior
Without any configuration of the Postgres output plugin everything works as expected. But using a custom "create_templates" template I'm receiving the shown error message:
syntax error at or near "(" (SQLSTATE 42601) pid:426127 sql:CREATE TABLE () time:604.871µs]
Even if the "create_template" configuration is the same as the default (without the "create_template").
Actual behavior
The expected behavior is that the the sql statements in "create_template" are executed successfully. Starting with the one that, reading the documentation, should be the same as the default.
Additional info
No response
The text was updated successfully, but these errors were encountered: