Add a kubernetes-log4j module

CHANGELOG.md

@@ -21,6 +21,7 @@
 * Fix #5636: Add new extension `open-virtual-networking` to manage resources in `k8s.ovn.org/v1` API group.
 * Fix #5711: Kube API Test - Kubernetes API Server JUnit Test Support
 * Fix #5772: Add openshift model `io.fabric8.openshift.api.model.DeploymentConfigRollback`
+* Add a `kubernetes-log4j` module to lookup Kubernetes attributes in a Log4j Core configuration.
 
 #### _**Note**_: Breaking changes
 * KubeSchema and Validation Schema generated classes are no longer annotated with Jackson, Lombok, and Sundrio annotations.

doc/KubernetesLog4j.md

@@ -0,0 +1,97 @@
+# Kubernetes Log4j Lookup
+
+The Kubernetes Log4j Lookup provides a [Log4j Core Lookup](https://logging.apache.org/log4j/2.x/manual/lookups) that
+can be used to logs files data specific to the Kubernetes container in which the application is running.
+
+## Usage
+
+In order to use it, you only need to add the following artifact to your Maven dependencies:
+
+```xml
+
+<dependencies>
+  <dependency>
+    <groupId>io.fabric8</groupId>
+    <artifactId>kubernetes-log4j</artifactId>
+    <version>${fabric8.version}</version>
+    <scope>runtime</scope>
+  </dependency>
+  ...
+</dependencies>
+```
+
+The following lookups can be use in a `log4j2.xml` configuration file.
+
+| Supported keys                | Description                                |
+|-------------------------------|--------------------------------------------|
+| `${k8s:masterUrl}`            | the master URL of the Kubernetes cluster   |
+| `${k8s:namespaceId}`          | the id of the namespace                    |
+| `${k8s:namespaceName}`        | the name of the namespace                  |
+| `${k8s:namespaceAnnotations}` | the annotations of the namespace           |
+| `${k8s:namespaceLabels}`      | the labels of the namespace                |
+| `${k8s:podId}`                | the id of the pod                          |
+| `${k8s:podIp}`                | the IP of the pod                          |
+| `${k8s:podName}`              | the name of the pod                        |
+| `${k8s:accountName}`          | the name of the pod service account        |
+| `${k8s:annotations}`          | the annotations of the pod                 |
+| `${k8s:labels}`               | the labels of the pod                      |
+| `${k8s:labels.<name>}`        | the value of the `<name>` label of the pod |
+| `${k8s:containerId}`          | the id of the container                    |
+| `${k8s:containerName}`        | the name of the container                  |
+| `${k8s:imageId}`              | the id of the container image              |
+| `${k8s:imageName}`            | the name of the container image            |
+| `${k8s:host}`                 | the node name of the pod                   |
+| `${k8s:hostIp}`               | the IP of the pod                          |
+
+## Configuration
+
+In order to access data from the Kubernetes cluster, the Kubernetes Log4j lookup uses the automatic configuration
+procedure of the Fabric8 Kubernetes client.
+
+### Automatic configuration
+
+See [Configuring the client](https://github.com/fabric8io/kubernetes-client/tree/main?tab=readme-ov-file#configuring-the-client)
+
+### Legacy configuration
+
+To ease the transition between the [`log4j-kubernetes`](https://logging.apache.org/log4j/2.x/log4j-kubernetes)
+artifact and Fabric8's Log4j lookup, the Kubernetes client can also be configured via one of the [Log4j property
+sources](https://logging.apache.org/log4j/2.x/manual/configuration#SystemProperties).
+
+To enable the legacy configuration, the Java System property `kubernetes.log4j.useProperties` must be set to `true`.
+
+The following configuration properties are recognized.
+
+| Log4j Property Name                                                                                           |                Default |                        Description |
+|---------------------------------------------------------------------------------------------------------------|-----------------------:|-----------------------------------:|
+| `log4j2.kubernetes.client.apiVersion`<br/>`spring.cloud.kubernetes.client.apiVersion`                         |                     v1 |             Kubernetes API Version |
+| `log4j2.kubernetes.client.caCertData`<br/>`spring.cloud.kubernetes.client.caCertData`                         |                        |          Kubernetes API CACertData |
+| `log4j2.kubernetes.client.caCertFile`<br/>`spring.cloud.kubernetes.client.caCertFile`                         |                        |          Kubernetes API CACertFile |
+| `log4j2.kubernetes.client.clientCertData`<br/>`spring.cloud.kubernetes.client.clientCertData`                 |                        |      Kubernetes API ClientCertData |
+| `log4j2.kubernetes.client.clientCertFile`<br/>`spring.cloud.kubernetes.client.clientCertFile`                 |                        |      Kubernetes API ClientCertFile |
+| `log4j2.kubernetes.client.clientKeyAlgo`<br/>`spring.cloud.kubernetes.client.clientKeyAlgo`                   |                    RSA |       Kubernetes API ClientKeyAlgo |
+| `log4j2.kubernetes.client.clientKeyData`<br/>`spring.cloud.kubernetes.client.clientKeyData`                   |                        |       Kubernetes API ClientKeyData |
+| `log4j2.kubernetes.client.clientKeyFile`<br/>`spring.cloud.kubernetes.client.clientKeyFile`                   |                        |       Kubernetes API ClientKeyFile |
+| `log4j2.kubernetes.client.clientKeyPassPhrase`<br/>`spring.cloud.kubernetes.client.clientKeyPassphrase`       |               changeit | Kubernetes API ClientKeyPassphrase |
+| `log4j2.kubernetes.client.connectionTimeout`<br/>`spring.cloud.kubernetes.client.connectionTimeout`           |                    10s |                 Connection timeout |
+| `log4j2.kubernetes.client.httpProxy`<br/>`spring.cloud.kubernetes.client.http-proxy`                          |                        |                                    |
+| `log4j2.kubernetes.client.httpsProxy`<br/>`spring.cloud.kubernetes.client.https-proxy`                        |                        |                                    |
+| `log4j2.kubernetes.client.loggingInterval`</br>`spring.cloud.kubernetes.client.loggingInterval`               |                    20s |                   Logging interval |
+| `log4j2.kubernetes.client.masterUrl`<br/>`spring.cloud.kubernetes.client.masterUrl`                           | kubernetes.default.svc |     Kubernetes API Master Node URL |
+| `log4j2.kubernetes.client.namespace`<br/>`spring.cloud.kubernetes.client.namespace`                           |                default |               Kubernetes Namespace |
+| `log4j2.kubernetes.client.noProxy`<br/>`spring.cloud.kubernetes.client.noProxy`                               |                        |                                    |
+| `log4j2.kubernetes.client.password`<br/>`spring.cloud.kubernetes.client.password`                             |                        |            Kubernetes API Password |
+| `log4j2.kubernetes.client.proxyPassword`<br/>`spring.cloud.kubernetes.client.proxyPassword`                   |                        |                                    |
+| `log4j2.kubernetes.client.proxyUsername`<br/>`spring.cloud.kubernetes.client.proxyUsername`                   |                        |                                    |
+| `log4j2.kubernetes.client.requestTimeout`<br/>`spring.cloud.kubernetes.client.requestTimeout`                 |                    10s |                    Request timeout |
+| `log4j2.kubernetes.client.rollingTimeout`<br/>`spring.cloud.kubernetes.client.rollingTimeout`                 |                   900s |                    Rolling timeout |
+| `log4j2.kubernetes.client.trustCerts`<br/>`spring.cloud.kubernetes.client.trustCerts`                         |                  false |  Kubernetes API Trust Certificates |
+| `log4j2.kubernetes.client.username`<br/>`spring.cloud.kubernetes.client.username`                             |                        |            Kubernetes API Username |
+| `log4j2.kubernetes.client.watchReconnectInterval`<br/>`spring.cloud.kubernetes.client.watchReconnectInterval` |                     1s |                 Reconnect Interval |
+| `log4j2.kubernetes.client.watchReconnectLimit`<br/>`spring.cloud.kubernetes.client.watchReconnectLimit`       |                     -1 |   Reconnect Interval limit retries |
+
+### Usage in Spring Boot
+
+Note that Log4j Core is initialized at least twice by Spring Boot and since the Spring `Environment` is only
+available
+during the last Log4j initialization Spring properties will only be available to Log4j in the last initialization.

log4j/pom.xml

@@ -0,0 +1,81 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+    Copyright (C) 2015 Red Hat, Inc.
+
+    Licensed under the Apache License, Version 2.0 (the "License");
+    you may not use this file except in compliance with the License.
+    You may obtain a copy of the License at
+
+            http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+  <parent>
+    <groupId>io.fabric8</groupId>
+    <artifactId>kubernetes-client-project</artifactId>
+    <version>6.11-SNAPSHOT</version>
+  </parent>
+
+  <artifactId>kubernetes-log4j</artifactId>
+  <packaging>bundle</packaging>
+  <name>Fabric8 :: Kubernetes :: Log4j Core components</name>
+  <description>Provides a lookup to use Kubernetes attributes in a Log4j Core configuration.</description>
+
+  <properties>
+    <osgi.export>io.fabric8.kubernetes.log4j.*</osgi.export>
+    <osgi.import>*</osgi.import>
+  </properties>
+
+  <dependencies>
+    <dependency>
+      <groupId>io.fabric8</groupId>
+      <artifactId>kubernetes-client</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.logging.log4j</groupId>
+      <artifactId>log4j-api</artifactId>
+      <version>${log4j.version}</version>
+      <scope>compile</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.logging.log4j</groupId>
+      <artifactId>log4j-core</artifactId>
+      <version>${log4j.version}</version>
+      <scope>compile</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.assertj</groupId>
+      <artifactId>assertj-core</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.junit.jupiter</groupId>
+      <artifactId>junit-jupiter-api</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.junit.jupiter</groupId>
+      <artifactId>junit-jupiter-params</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>io.fabric8</groupId>
+      <artifactId>kubernetes-server-mock</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.mockito</groupId>
+      <artifactId>mockito-core</artifactId>
+      <scope>test</scope>
+    </dependency>
+  </dependencies>
+
+</project>

log4j/src/main/java/io/fabric8/kubernetes/log4j/lookup/ClientBuilder.java

@@ -0,0 +1,85 @@
+/*
+ * Copyright (C) 2015 Red Hat, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *         http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.fabric8.kubernetes.log4j.lookup;
+
+import io.fabric8.kubernetes.client.Config;
+import io.fabric8.kubernetes.client.ConfigBuilder;
+import io.fabric8.kubernetes.client.KubernetesClient;
+import io.fabric8.kubernetes.client.KubernetesClientBuilder;
+import org.apache.logging.log4j.status.StatusLogger;
+import org.apache.logging.log4j.util.PropertiesUtil;
+
+import static io.fabric8.kubernetes.client.utils.Utils.getSystemPropertyOrEnvVar;
+
+/**
+ * Builds a Kubernetes Client.
+ */
+final class ClientBuilder {
+
+  /**
+   * If this system property is set to {@code true}, the client configuration is retrieved from Log4j Properties.
+   */
+  public static final String KUBERNETES_LOG4J_USE_PROPERTIES = "kubernetes.log4j.useProperties";
+
+  private ClientBuilder() {
+  }
+
+  public static KubernetesClient createClient() {
+    final Config config = kubernetesClientConfig(PropertiesUtil.getProperties());
+    return config != null ? new KubernetesClientBuilder()
+        .withConfig(config).build() : null;
+  }
+
+  static Config kubernetesClientConfig(final PropertiesUtil props) {
+    try {
+      final Config base = Config.autoConfigure(null);
+      if (getSystemPropertyOrEnvVar(KUBERNETES_LOG4J_USE_PROPERTIES, false)) {
+        final Log4jConfig log4jConfig = new Log4jConfig(props, base);
+        return new ConfigBuilder()
+            .withApiVersion(log4jConfig.getApiVersion())
+            .withCaCertData(log4jConfig.getCaCertData())
+            .withCaCertFile(log4jConfig.getCaCertFile())
+            .withClientCertData(log4jConfig.getClientCertData())
+            .withClientCertFile(log4jConfig.getClientCertFile())
+            .withClientKeyAlgo(log4jConfig.getClientKeyAlgo())
+            .withClientKeyData(log4jConfig.getClientKeyData())
+            .withClientKeyFile(log4jConfig.getClientKeyFile())
+            .withClientKeyPassphrase(log4jConfig.getClientKeyPassphrase())
+            .withConnectionTimeout(log4jConfig.getConnectionTimeout())
+            .withHttpProxy(log4jConfig.getHttpProxy())
+            .withHttpsProxy(log4jConfig.getHttpsProxy())
+            .withLoggingInterval(log4jConfig.getLoggingInterval())
+            .withMasterUrl(log4jConfig.getMasterUrl())
+            .withNamespace(log4jConfig.getNamespace())
+            .withNoProxy(log4jConfig.getNoProxy())
+            .withPassword(log4jConfig.getPassword())
+            .withProxyPassword(log4jConfig.getProxyPassword())
+            .withProxyUsername(log4jConfig.getProxyUsername())
+            .withRequestTimeout(log4jConfig.getRequestTimeout())
+            .withTrustCerts(log4jConfig.isTrustCerts())
+            .withUsername(log4jConfig.getUsername())
+            .withWatchReconnectInterval(log4jConfig.getWatchReconnectInterval())
+            .withWatchReconnectLimit(log4jConfig.getWatchReconnectLimit())
+            .build();
+      }
+      return base;
+    } catch (final Exception e) {
+      StatusLogger.getLogger().warn("An error occurred while retrieving Kubernetes Client configuration: {}.",
+          e.getMessage(), e);
+    }
+    return null;
+  }
+}

log4j/src/main/java/io/fabric8/kubernetes/log4j/lookup/ContainerUtil.java

@@ -0,0 +1,116 @@
+/*
+ * Copyright (C) 2015 Red Hat, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *         http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.fabric8.kubernetes.log4j.lookup;
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.status.StatusLogger;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Stream;
+
+/**
+ * Locate the current docker container.
+ */
+final class ContainerUtil {
+
+  private static final Logger LOGGER = StatusLogger.getLogger();
+  private static final Pattern DOCKER_ID_PATTERN = Pattern.compile("[0-9a-fA-F]{64}");
+  static final Path CGROUP_PATH = Paths.get("/proc/self/cgroup");
+
+  private ContainerUtil() {
+  }
+
+  /**
+   * Returns the container id when running in a Docker container.
+   * <p>
+   * This inspects /proc/self/cgroup looking for a Kubernetes Control Group. Once it finds one it attempts
+   * to isolate just the docker container id. There doesn't appear to be a standard way to do this, but
+   * it seems to be the only way to determine what the current container is in a multi-container pod. It would have
+   * been much nicer if Kubernetes would just put the container id in a standard environment variable.
+   * </p>
+   * 
+   * @param path Path to a {@code /proc/pid/cgroup} file.
+   * @return A container id or {@code null} if not found.
+   */
+  public static String getContainerId(Path path) {
+    try {
+      if (Files.exists(path)) {
+        try (final Stream<String> lines = Files.lines(path)) {
+          final String id = lines
+              .map(ContainerUtil::getContainerId)
+              .filter(Objects::nonNull)
+              .findFirst()
+              .orElse(null);
+          LOGGER.debug("Found container id {}", id);
+          return id;
+        }
+      }
+      LOGGER.warn("Unable to access container information");
+    } catch (IOException ioe) {
+      LOGGER.warn("Error obtaining container id: {}", ioe.getMessage());
+    }
+    return null;
+  }
+
+  private static String getContainerId(String line) {
+    return Optional.ofNullable(getCGroupPath(line))
+        .map(ContainerUtil::getDockerId)
+        .orElse(null);
+  }
+
+  /**
+   * Retrieves a container id from a hierarchy of CGroups
+   * <p>
+   * Based on
+   * <a href=
+   * "https://github.com/jenkinsci/docker-workflow-plugin/blob/master/src/main/java/org/jenkinsci/plugins/docker/workflow/client/ControlGroup.java">ControlGroup.java</a>
+   * </p>
+   * 
+   * @param cgroupPath a slash-separated hierarchy of CGroups.
+   * @return a Docker ID
+   */
+  private static String getDockerId(String cgroupPath) {
+    String[] elements = cgroupPath.split("/", -1);
+    String dockerId = null;
+    for (String element : elements) {
+      Matcher matcher = DOCKER_ID_PATTERN.matcher(element);
+      if (matcher.find()) {
+        dockerId = matcher.group();
+      }
+    }
+    return dockerId;
+  }
+
+  /**
+   * Retrieves the full hierarchy of CGroups the process belongs
+   * <p>
+   * See <a href="https://man7.org/linux/man-pages/man7/cgroups.7.html">/proc/pid/cgroups</a>
+   * </p>
+   * 
+   * @param line A line from a {@code /proc/pid/cgroups} file
+   */
+  private static String getCGroupPath(String line) {
+    String[] fields = line.split(":", -1);
+    return fields.length > 2 ? fields[2] : null;
+  }
+}