Beats central management: index templates and built-in roles

[ycombinator]

We are starting on a Central Management feature for Beats (elastic/beats#7028). This will be an X-Pack feature and will require certain elements in X-Pack Elasticsearch, specifically:

Index Templates

An index template for an internal index, say .management-beats-agents, that will be used to persist information from Beats agents when they enroll themselves or update the same information periodically after enrollment. It will also be used to persist the central configuration (after merging) for each beat. Each document in this index will represent an enrolled Beat and its merged central configuration, if any.

PUT _template/.management-beats-agents
{
  "version": 7000001,
  "index_patterns": [
    ".management-beats-agents"
  ],
  "settings": {
    "index": {
      "number_of_shards": 1,
      "auto_expand_replicas": "0-1",
      "codec": "best_compression"
    }
  },
  "mappings": {
    "_doc": {
      "dynamic": "strict",
      "properties": {
        "beat": {
          "properties": {
            "id": {
              "type": "keyword"
            },
            "enrollment_token": {
              "type": "keyword"
            },
            "access_token": {
              "type": "keyword"
            },
            "verified_on": {
              "type": "date"
            },
            "type": {
              "type": "keyword"
            },
            "ephemeral_id": {
              "type": "keyword"
            },
            "local_configuration_yml": {
              "type": "text"
            },
            "central_configuration_yml": {
               "type": "text"
            },
            "metadata": {
              "dynamic": "true",
              "type": "object"
            }
          }
        }
      }
    }
  }
}

An index template for an internal index, say .management-beats-admin, that will be used to persist enrollment tokens and configuration for Beats. There may be two different types of documents in this index, which will be distinguished by the value in the type field, either enrollment_token or configuration_fragment. Documents with type = enrollment_token are expected to have an enrollment_token field, containing sub-fields necessary to represent enrollment tokens. Documents with type = configuration_fragment are expected to have a configuration_fragment field, containing sub-fields necessary to represent configuration fragments.

PUT _template/.management-beats-admin
{
  "version": 7000001,
  "index_patterns": [
    ".management-beats-admin"
  ],
  "settings": {
    "index": {
      "number_of_shards": 1,
      "auto_expand_replicas": "0-1",
      "codec": "best_compression"
    }
  },
  "mappings": {
    "_doc": {
      "dynamic": "strict",
      "properties": {
        "type": {
          "type": "keyword"
        },
        "enrollment_token": {
          "properties": {
            "token": {
              "type": "keyword"
            }
          }
        },
        "configuration_fragment": {
          "properties": {
            "tag": {
              "type": "keyword"
            },
            "fragment_yml": {
              "type": "text"
            }
          }
        }
      }
    }
  }
}

Security Roles

A beats_agent role, which should:

• be used by Beats agents (indirectly, via a user configured in kibana.yml) for initial enrollment, periodic updates to Beat information, and to pull down configuration for the agent.
• have create_index, read, and index privileges on the .management-beats-agents index

PUT _xpack/security/role/beats_agent
{
  "indices": [
    {
      "names": [
        ".management-beats-agents"
      ],
      "privileges": [
        "create_index",
        "index",
        "read"
      ],
      "field_security": {
        "grant": [
          "*"
        ]
      }
    }
  ]
}

A beats_admin role, which should:

• be used by administrative users to sign in to Kibana and use the Beats Management UI to centrally-manage beat configurations, including verifying enrolled beats.
• have all index privileges on all .management-beats-* indices.

PUT _xpack/security/role/beats_admin
{
  "indices": [
    {
      "names": [
        ".management-beats-*"
      ],
      "privileges": [
        "all"
      ],
      "field_security": {
        "grant": [
          "*"
        ]
      }
    }
  ]
}

[elasticmachine]

Pinging @elastic/es-security