Kubernetes attachment

[felixbarny]

• Publish a docker image that contains the agent distribution, similar to what the Java agent is already doing (see build and the observability/apm-agent-java image in our Docker registry)
• Integrate with the webhook in APM Server (Kubernetes admission controller for APM agent attachment apm-server#7386) to copy the agent from the image to a volume via an init container and add environment variables to attach the agent.

[trentm]

I think we should consider starting a package-lock.json file for the Node.js APM agent for this and for the AWS Lambda layers that we are publishing for releases now, so that we have repeatable builds. Fits in with SLSA as well.

(Or we could go deeper and consider using a npm-shrinkwrap.json that would get published so that those using elastic-apm-node as a library would also get locked-down/stable transitive dep versions ... even though the npm docs discourage that. https://docs.npmjs.com/cli/v8/configuring-npm/package-lock-json#package-lockjson-vs-npm-shrinkwrapjson)

[astorm]

Spent a bit of last week learning enough to define some concrete deliverable for this. Specifically --

• We need to build a docker container that contains all the source files needed for the Node Agent agent, similar to what we do when building the lambda layer. The end deliverable here is a Dockerfile that can build a an image with the source files located in a /usr/agent/node_modules folder on the container

• Once we have the ability to build this Docker container, we need to automatically build and publish it whenever a new version of the agent is built. This build and publish should include a latest tag

• Finally, we need to extend the mutating web hook such that

  • The default configuration includes a section that ensures the above published docker container is mounted when a user has the mutating webhook deployed into their system
  • The command that copies the agent from the above container to the shared mounted volume can copy a folder (right now it's setup to copy a single file only -- hopefully as simple as adding a -r option)
  • User containers have a NODE_OPTIONS=-r elastic-apm-agent/start env. var set to that ensure the agent is started automatically
  • If necessary -- user containers have a NODE_PATH env var set that points to the mounted volume path such that the copied agent is available for requiring from user code.

[AlexanderWert]

Note: This can also be reused to support Elastic APM with container-based AWS Lambda functions.

[trentm]

For comparison, my (very naive) read of the "OTel Operator" is that it sets (or appends to) NODE_OPTIONS=" --require /otel-auto-instrumentation/autoinstrumentation.js"
https://github.com/open-telemetry/opentelemetry-operator/blob/main/pkg/instrumentation/nodejs.go#L24-L27

[astorm]

Request made to productivity team for a published Dockerfile: https://github.com/elastic/observability-robots/issues/1229

I'm proceeding to making the adjustments to the mutating webhook to enable attachment.

[trentm]

see #2742

[astorm]

Completed the minor changes needed to the mutating webhook, see elastic/apm-k8s-attacher#31. @trentm per last week's planning I decided to go absolute NODE_OPTIONS path and left out NODE_PATH for now.

[astorm]

Final task to finish this will be fore the Node.js Docker image to be listed as a public docker image. Right now it appears to require some sort of elastic auth to fetch. See: https://github.com/elastic/sre/issues/194

[trentm]

@astorm This can be closed now, right?