Provide introductory docs for API Keys

[tvernum]

It is common for people to stumble across the API key documentation and assume that API Keys is the preferred/recommended way to authenticate to Elasticsearch

I often come across comments on discuss or community-slack where a conversation starts with "I'm trying to use API Keys and ..." and after digging it turns out that API keys aren't a good fit for their needs, but they assumed API Keys are how they're supposed to interact with ES.

It would be helpful to have an introductory guide to API Keys that explain the problems that they're intended to solve, how they work (and why) and potential traps.

[elasticmachine]

Pinging @elastic/es-docs (Team:Docs)

[elasticmachine]

Pinging @elastic/es-security (Team:Security)

[lockewritesdocs]

@tvernum, this sounds like a worthy cause, especially because I hear folks from other teams internally indicate that API keys are the "preferred" method of authenticating with Elasticsearch. If there are internal resources that provide some of this background information and problem resolution, please let me know.

[tvernum]

With the sort of changes we're working through at the moment, it probably makes sense to plan for this to eventually form part of a "Understanding authentication & authorization models in Elasticsearch" section.

That would want to cover:

• Users
  • Password based authentication
    • Reserved Users
    • ES managed users (file, native)
    • Externally managed users (ldap, AD)
  • SSO (SAML, OIDC, Kerberos)
  • Other (PKI, Custom Realms?)
• Token Authentication
  • Access Tokens
  • API Keys
• Service Accounts

And then for each part think about options such as:

• Interactive login to Kibana
• Use within other Elastic products
• Access via official language clients
• Access via curl
• Access via other tools / clients

But if we try to write all of that now, it will be a huge task and spend a lot of time in reviews. I think it makes sense to approach it in pieces, but with an idea of how it might come together at the end.

[tvernum]

In terms of content for the short term:

Password-based Users

Strengths / Features

• Allow you to assign the same permissions (roles) to multiple users
• Allow you to modify permissions (either assign different roles, or edit the roles themselves) as needs change
• Allow you to integrate with an external identity system (e.g. LDAP)
• Are easily supported by a variety of clients tools ("HTTP basic authentication")

Limitations

• If you have multiple clients (e.g filebeat running on a lot of hosts), you need to share a single password across each host, or create multiple users.
• In order to create new users, you must have the manage_security privilege, but that privilege is very powerful and gives the ability to create users with any possible role (even superuser).

API Keys

Strengths / Features

• The ability to create API keys only requires manage_own_api_key and the API keys that are created cannot have more access than the owning user.
• Using a unique API key for each client (host) is easy, because a single user can have multiple API Keys, and the specific API key that performed an operation can be recorded in audit logs.

Limitations

• API keys cannot be linked to a central identity store - it is not possible to revoke API key access by de-activating a user in a corporate LDAP directory.
• API keys only support header-based authentication. You cannot authenticate using other mechanisms such as PKI.
• API keys are immutable. You cannot change an API Key's access, you need to create a new API key instead.
• API keys cannot create additional API keys.
• AP keys require a special authentication header that may not be supported (or may be less easily used) in some tools. E.g. in curl, user authentication is as simple as -u {username}, but API key authentication requires a custom header -H "Authorization: ApiKey {string}"

Non-differences

• API Key authentication is not inherently more secure than password authentication.
  • Notably administrators should treat API key credentials with the same level of care as they would give to a password. The credentials are effectively equivalent (assuming the same set of privileges)
  • The fact that API keys can be created with less privileges than the owning user may allow for more secure setups. However, the fact the users can be centrally managed (LDAP) may be more secure in some environments.
• With the exception of creating API keys (and features such as alerting that depend on creating new API keys), there is nothing that a User can do that an API key can't - or vice versa.