Telegraf 1.8 + Cassandra serializing issue w/ Jolokia2

[AlexGilliland]

Relevant telegraf.conf:

System info:

[Include Telegraf version, operating system name, and other relevant details]
CentOS 7
telegraf 1.8.1 and 1.8.3
Jolokia2 setup

Steps to reproduce:

1. Upgrade telegraf from 1.7.3 to 1.8.1
2. Start receiving " [serializers.influx] could not serialize metric:" errors for "cassandra_Column" and "cassandra_Cache" within /var/log/messages
3. Revert back to 1.7.3 resolves the issue

Expected behavior:

Still be able to serialize the metric

Actual behavior:

Receive an error "no serializable fields; discarding metric"

Additional info:

Identical setup between our cassandra nodes, if I upgrade telegraf to 1.8+ it has issues serializing all keyspaces on cache and columnfamily.
Using the config from https://github.com/influxdata/telegraf/blob/master/plugins/inputs/jolokia2/examples/cassandra.conf

[glinton]

Can you paste the relevant (sanitized) sections of your telegraf.conf please?

Possibly more helpful, can you check the settings against comments on #4748 ?

[AlexGilliland]

I took a look at the settings in #4748 and if I'm understanding that correctly, doesn't apply, but please correct me if I'm wrong here.

Cassandra version - working on upgrading.

cqlsh> show version
[cqlsh 5.0.1 | Cassandra 2.1.11 | CQL spec 3.2.1 | Native protocol v3]

Errors:

[serializers.influx] could not serialize metric: "cassandra_Cache,host=removed,jolokia_agent_url=http://localhost:8778/jolokia,name=HitRate,scope=CounterCache": no serializable fields; discarding metric

This occurs for all keyspaces

[serializers.influx] could not serialize metric: "cassandra_ColumnFamily,host=removed,jolokia_agent_url=http://localhost:8778/jolokia,keyspace=redactedName,name=EstimatedColumnCountHistogram,scope=attribute_name_index": no serializable fields; discarding metric

Telegraf.conf - just your default from the jolokia2 conf

  [[inputs.jolokia2_agent.metric]]
    name  = "ColumnFamily"
    mbean = "org.apache.cassandra.metrics:keyspace=*,name=*,scope=*,type=ColumnFamily"
    tag_keys = ["keyspace", "name", "scope"]
    field_prefix = "$2_"

 [[inputs.jolokia2_agent.metric]]
    name  = "FileCache"
    mbean = "org.apache.cassandra.metrics:name=*,type=FileCache"
    tag_keys = ["name"]
    field_prefix = "$1_"

[2ban]

Hi, guys. We use 1.8.3 telegraf and cassandra 3.0.16 + jolokia 1.6.0 java agent.
But still get similar errors:

2019-02-06T12:07:00Z E! [serializers.influx] could not serialize metric: "cassandra_Cache,host=telegraf.cassandra,jolokia_agent_url=http://cassandra-jolokia.cassandra.svc:8778/jolokia/,name=OneMinuteHitRate,scope=CounterCache": no serializable fields; discarding metric

2019-02-06T12:10:00Z E! [serializers.influx] could not serialize metric: "cassandra_Cache,host=telegraf.cassandra,jolokia_agent_url=http://cassandra-jolokia.cassandra.svc:8778/jolokia/,name=FiveMinuteHitRate,scope=RowCache": no serializable fields; discarding metric

Telegraf config was taken from your example:

    [[inputs.jolokia2_agent.metric]]
      name  = "Cache"
      mbean = "org.apache.cassandra.metrics:name=*,scope=*,type=Cache"
      tag_keys = ["name", "scope"]
      field_prefix = "$1_" 

Any advice on that?
thanks.

[danielnelson]

@2ban Can you test with 1.9.4? I suspect the error won't be displayed anymore but it could be that no metrics are produced by this section either, if you run into either of these issues can you open a new issue?