[APM] Fleet: Inject Kibana URL + API Key into APM integration policy

[simitt]

This is an interim solution. The permanent solution is #95501

---

Hook into Fleet extension points to create and inject a Kibana API Key and the Kibana URL to the APM integration policy. The API Key need to be checked on every packagePolicyUpdate as the key might be invalidated by users, in which case a new key needs to be created.
Permissions for key creation should not be an issue, as Fleet requires superuser privileges.

---

We will have to either inject secondary credentials for querying Kibana, or somehow extend the privileges of the primary API Key provided by Fleet to APM Server.

Fleet provides extension points for the package policy create and update APIs.

Information courtesy of Paul Tavares:

API hooks available for Package policies (aka: integration policies): one for Create and another for Update. To use them, a dependency must be set on your kibana Plugin to Fleet, which will then expose FleetStartContract during the start phase of the plugin's lifecycle. This interface includes a method named registerExternalCallback() which allows you to register a callback function for the following:

fleetStart.registerExternalCallback('packagePolicyCreate', async () => {});

fleetStart.registerExternalCallback('packagePolicyUpdate', async () => {});

The Types for the callbacks supported are defined here:

kibana/x-pack/plugins/fleet/server/plugin.ts

Lines 132 to 148 in a99ccc2

	export type ExternalCallback =
	| [
	'packagePolicyCreate',
	(
	newPackagePolicy: NewPackagePolicy,
	context: RequestHandlerContext,
	request: KibanaRequest
	) => Promise<NewPackagePolicy>
	]
	| [
	'packagePolicyUpdate',
	(
	newPackagePolicy: UpdatePackagePolicy,
	context: RequestHandlerContext,
	request: KibanaRequest
	) => Promise<UpdatePackagePolicy>
	];

The APM app should hook into these API calls, and inject one or more additional API Keys. This could be done without #89311.

---

This issue supersedes #89311
Related discussion: elastic/apm-server#4573 (comment)

[elasticmachine]

Pinging @elastic/apm-ui (Team:apm)

[felixbarny]

Alex already created an issue for that: #88822

[axw]

To try and help get things started, I've prototyped a couple of things:

• Injecting an API Key into the APM package policies in Fleet: https://github.com/axw/kibana/tree/apm-fleet-apikey
• Injecting agent config directly into APM package policies in Fleet: https://github.com/axw/kibana/tree/apm-agent-config-fleet

It's all hacks, just use it for inspiration if anything.

The approach of injecting config is probably what we should eventually take. Since with that approach APM Server wouldn't be connecting to Kibana at all, we would need to come up with a new way of indicating that agent config has been applied. I'm also not sure if this will work with our security model.

[sorenlouv]

Thanks @axw, much appreciated! This provides some great pointers for what to do next

[simitt]

update: I deleted a bunch of comments on this issue and moved them over to #89989 (comment). Apologies for this mess!

[simitt]

Since we decided to not work on an iterims solution for 7.13, closing this issue in favor of #95501.