Add tags for nodes

[XavM]

Tags can be used with services but not with nodes

Adding tags for nodes and the associated filter for /v1/catalog/nodes HTTP API would be useful

[armon]

So you would want to filter nodes on that endpoint using tags? I'm trying to get a better understanding of the use case. This seems reasonably useful.

[XavM]

Use case example :

Imagine we have containers running on distinct Hardware Nodes, we could use node tags to describe the HN they are running on

This would expose to discovery a clear inventory of the current topology for a given datacenter

The way we do it now is by creating a fake service for each container that holds the corresponding set of tags, but it is much more a trick than a native solution

Node tags could be used for things that are not relative to services but are specific to nodes :

• OS release
• Allocated resources (Allocated CPU, mem, disk)
• Virtualization type (KVM, Xen, LXC, Docker, ...)
• Hardware node
• Last update of the ssh key
• Time zone
• What ever ...

Being able to filter nodes using tags, could allow us to trigger a deploy or configuration change on a specific set of nodes (independent of the services that are exposed on those nodes), or give some clues about different pool of hosts that are not behaving the same

If available, consul could be the main inventory for Dev and Ops

[armon]

@XavM Again, makes sense to me. Thanks!

[pr0xr]

I really like this idea. I believe that Consul would benefit from having a method for retrieving "facts" about the node consul is running on. This would help identifying the node it's running on.

[nickwales]

I would like to third this idea! It would fulfill the workflow that has been germinating in my head recently.

[janitha]

It would also be useful to filter services based on the agent tags (which would would also resolve #545 )

[jhmartin]

For fact-retrieval, one might consider integrating with Chef Ohai -- that'd bring in a lot of node facts very easily.

[johannegger]

i would also like to express my interest in this feature.

[dellis23]

To throw in another use-case along with my 👍, I could see this being used for tagging things as being in maintenance mode, which would then be provided to consul template for updating configuration.

[ryanuber]

@dellis23 Consul 0.5 has built-in support for node-level and service-level maintenance modes. When maintenance mode is enabled, the node or service will be automatically excluded in all queries for nodes/services with passing health checks. Check out the maint command and http endpoints to get a feel for how it works!

[ryanuber]

I agree with @armon, having node-level tags seems reasonably useful. I think this would provide enough functionality to add "facts" about nodes by use of tags without needing to support specific fact generation systems like Ohai, facter, etc.

[sitano]

+1 - we use the same look up patterns based on tags

What do you think if it will be implemented by someone else? (or you would like to impl it yourself)

Actually, i think we can consider any node have a tag if its service have a tag. In that sense, there is also another issue corresponding to nodes look up - there is no easy way to ask consul for a list of nodes that have subset of tags on services they belongs to. What i would like to have too.

Additionally, its not very good i have to ask each DataCenter the same query just to obtain any info across the whole world. It will be good to have something like ?dc=*.

Another thing, there is no way to ask for a nodes list in the DC with full info. Now /catalog/nodes returns almost nothing but simple list. Imagine 100 nodes, how long should i ask consul for tag info on them. And /health/service/{service} which returns nodes, do not have a simple variant of just /health/service request which should be the same as /health/service/*

[ryanuber]

Hi @sitano,

What do you think if it will be implemented by someone else? (or you would like to impl it yourself)

We are always open to implementations from the community! We should agree on the design and functionality first, though, and I don't think we're quite there yet.

Actually, i think we can consider any node have a tag if its service have a tag.

I think we will run into some odd collision cases by going that route. This would also still require a service to be registered before any tag system would be available, which would promote registering a service just to add tags to the node. While it would work, the UX doesn't quite feel right to me.

its not very good i have to ask each DataCenter the same query just to obtain any info across the whole world. It will be good to have something like ?dc=*.

We added some plumbing not long ago that would make this type of query possible internally, but honestly from an API perspective I think that the current design works well and lends itself easily to obtaining global information across datacenters. If you have a more specific use case, please open a separate issue and we can discuss it there.

there is no way to ask for a nodes list in the DC with full info.

I think the API's were designed this way to avoid absolutely massive responses. Consider a case where there are hundreds of nodes in the same datacenter. Returning each node with its full list of services and tags inline will be a very large response, and contain lots of duplicated information. As they are, the API's are composable so that they can help clients build any kind of response they need.

Since the original issue here is about adding tags support to nodes, please open separate issues if you would like to discuss any of the unrelated stuff above in more detail.

[sitano]

The idea was not to emulate tag system for nodes through applying custom services to them.

But to have an API which can:

1. get a list of nodes filtered by nodes-tags in certain dc.
2. get a list of nodes filtered by service-tags those nodes are part of in certain dc.

Currently for my production i need (2). But the (2) is a subset of this Issue.

a) For (1),(2) we can extend: /v1/catalog/nodes with ?tag= filter, to filter based on ServiceTags first, and then with NodeTags second, when impl support.

b) For (2) /v1/catalog/service/ with ?tag= filter, to filter nodes based on ServiceTags

c) It is better if this ?tag will support something like ?tag=First,Second,LastTag

d) For (2) /v1/catalog/service/ to be able to support prefix style request of the form /v1/catalog/service/[service_prefix]*?tag=blah - what means select all nodes filtered by services which are filtered by wildcard or prefix, and then filter them by ServiceTags they have.

What do you think? I can implement it if we agree on api changes before i start.

[EricMCornelius]

I would like to chime in and say that I'm currently evaluating consul to replace some of our home-grown service discovery systems.

One of the biggest pain points I'm currently hitting is the inability for flexible tagging and querying of service instances.

In particular, there's no easy way for us to version services with a semver-style major.minor(.patch) and return filtered results of healthy nodes which adhere to a given policy. It would be extremely useful to have the ability to query richer metadata about services (e.g. where metadata.major_ver=1 and metadata.minor_ver>=3).

We can roll our own protocol on top of the existing service tags, but the restriction that they're a flat array of strings is very limiting in its own right. It leaves us with little alternative other than to dump stringified json directly into the tags array...

Are there any recommendations on how to tackle this aside from waiting for a richer set of metadata + associated filtering/querying apis?

[EricMCornelius]

Note that we could obviously hit the k/v store and use some opaque tag or service unique id, but that's an awful lot of extra complexity for what usually amounts to a simple question: which services can fulfill my request?

I'm happy to tackle contributing a more flexible feature like this back into core if that's the best approach.

[sitano]

From my point of view, it looks like, Consul API do not have any space to be extended to support flexible filtering / querying of nodes / services / hc information. It seems like it is not even close to use case patterns HashiCorp is trying to match for Consul. So, i have developed web service with SQL REST endpoint to query in-memory database which is caching all the information from all consuls across all datacenters. It similar to something like facebook`s osquery. I hope someday Consul will support more flexible querying api.

As far as this message is kinda off topic, i vote for Node Tags functionality +1.

[ybogdanov]

+1 for Node Tags

@sitano can you share it?

[sitano]

@ybogdanov actually i am trying to discuss it with my employer. it turned out quite useful thing for us.

[ybogdanov]

@sitano as a temporary solution, I'm thinking of something like tail -f /dev/null service on every node, to which tags can be assigned. And this will be a stub node tags service, by which you can easily query nodes by tags. Will it work, or am I missing something?

[sitano]

@ybogdanov , it is possible. it was the way i was thinking about one of workarounds for it. but the api which do selects for you is very restricted. my advice - check in advance whether it fits your future usage patterns. we were filtering nodes based on tags / services over 10 datacenters, and it took a lot of time (actually we end up making a lot of http requests to each dc master / per node). So, then i did write my sql cache for consul.

[mainframe]

+1 for Node Tags - would be VERY useful feature

[ssenaria]

This is one that is holding us back from adopting Consul. Any idea as to when this will be implemented. Any workarounds you guys can share?

[armon]

@ssenaria Likely this would not be until a V2 API which would allow us to change various representations and rethink the APIs a bit. People instead either just create a virtual "service" on every node and attach tags to that or create entries in the KV store.

[pearkes]

👍 this would be great!

[scalp42]

Our main use is to have a single shared cluster and use node tags to differentiate between environments.

[sflint]

+1 to environments

[GonZo]

+1 node tags

[matthewhartstonge]

👍 node tags.

[dky]

+1 need node tags.

[plorenz]

Would it be possible to make serf tags manageable from consul? I would like to be able to use node level tags with serf semantics.

Or would these node level tags have semantics different from serf tags?

[JoeOrtiz]

+1 node tags

[slackpad]

Hi @plorenz I don't think we'd implement this using Serf tags. Consul makes use of them internally for some operations, but there are limits to the number of tags, etc. that would make this a tricky feature to use if we exposed Serf tags. I think these tags would be Consul-level and managed between the Consul agent and the catalog.

[bfgoodrich]

This would definitely be a useful feature

[ajvb]

👍 Would be useful for me as well. Would like to be able to grab all nodes related to a specific environment (production servers for application X) that span many service definitions.

[scalp42]

Went with the approach of having every node with a "generic" /dev/null check every 30-60s and adding tags for the environment if someone wondering about this.

[sauragrawal]

+1 for node tags

[yosefy]

+1 node tags

becuse:
for inventory on AWS ot Goo
say ansible uses ec2 instances tags as groups
also prometheus may use it as inventory

and all those guys have tags for instance

and then on colocation we install consul and it works in different system
you see the "pain"

[yosefy]

hi , is this topic out of question?
or shoud we try serf for that?
please could you make a bit of clear on that :)
thanks

[123BLiN]

+1

[phyber]

This was literally one of the first things I tried to do after having Consul up and running on a set of test nodes, only to find that it's not a feature. Seems odd.

[ssenaria]

@yosefy We ended up using Serf but now have a need to use Consul. I can't abandon Serf for Consul and I don't want to have to run both!

[devaos]

+1 could use node tags as well as node key/value pairs

[scalp42]

Again, use "generic" /dev/null checks to work around the issue and you'll have node tags until it's actually implemented.

[gamefundas]

I didn't quite follow the /dev/null approach. Does this refer to registering the service with a dummy check and tag the service to our needs? In this case I am assuming the service names can be the same.

Particularly interested in the behavior, where lets say

Service A was tagged with Tag1, Tag2 & Tag3 in Node1
Service A was tagged with Tag 2, Tag3, Tag4 in Node 2

Will querying for Service A and Tag 1 yield Node 1, Service A and Tag 2 return Node 1 & 2?

Looking to try this out today but @ybogdanov and @scalp42 if you are aware of the behavior please let me know.

[gamefundas]

I found out that if both node1 and node2 are registered as Service A with different tagging scheme then Consul rolls up the tags from each node's service definition and forms a global set of tags. So essentially there is no way to apply the filters discretely. Also observed that not all events got propagated, events were getting missed out.

Alternately I tried adding different service id's for each node and that makes it work. Just that there is multiple registrations for the same service. Another thing to note is that the tag names are matched as contains, for example, tag1, tag2, tag3 can all be matched if one queries for "-tag tag" or "-tag a" etc. This is very strange as one has to define tags that are very much exclusive of each other.

Not sure if I am missing something basic here.

[elnoxgdl]

+1 for this functionallity!