change default FLUSH_INTERVAL to be 5s to be in sync with Helm Chart.…

… Also fix timeout issue that can occur with Concat plugin.

Dockerfile

@@ -23,7 +23,7 @@ RUN mkdir -p /fluentd/conf.d && \
 
 # Default settings
 ENV LOG_FORMAT "json"
-ENV FLUSH_INTERVAL "30s"
+ENV FLUSH_INTERVAL "5s"
 ENV NUM_THREADS "1"
 ENV SOURCE_CATEGORY "%{namespace}/%{pod_name}"
 ENV SOURCE_CATEGORY_PREFIX "kubernetes/"

README.md

@@ -90,7 +90,7 @@ Environment | Variable Description
 `EXCLUDE_UNIT_REGEX` |A regular expression for `systemd` units. Matching units will be excluded from Sumo. The logs will still be sent to FluentD.
 `FLUENTD_SOURCE`|Fluentd can tail files or query `systemd`. Allowable values: `file`, `Systemd`. <br/><br/>Default: `file` 
 `FLUENTD_USER_CONFIG_DIR`|A directory of user-defined fluentd configuration files, which must be in the  `*.conf` directory in the container.
-`FLUSH_INTERVAL` |How frequently to push logs to Sumo.<br/><br/>Default: `30s`
+`FLUSH_INTERVAL` |How frequently to push logs to Sumo.<br/><br/>Default: `5s`
 `KUBERNETES_META`|Include or exclude Kubernetes metadata such as `namespace` and `pod_name` if using JSON log format. <br/><br/>Default: `true`
 `LOG_FORMAT`|Format in which to post logs to Sumo. Allowable values:<br/><br/>`text`—Logs will appear in SumoLogic in text format.<br/>`json`—Logs will appear in SumoLogic in json format.<br/>`json_merge`—Same as json but if the container logs in json format to stdout it will merge in the container json log at the root level and remove the log field.<br/><br/>Default: `json`
 `MULTILINE_START_REGEXP`|The regular expression for the `concat` plugin to use when merging multi-line messages. Defaults to Julian dates, for example, Jul 29, 2017.

conf.d/file/source.containers.conf

@@ -15,27 +15,37 @@
   key log
   multiline_start_regexp "#{ENV['MULTILINE_START_REGEXP']}"
   separator "#{ENV['CONCAT_SEPARATOR']}"
+  timeout_label @NORMAL
 </filter>
 
-<filter containers.**>
-  @type kubernetes_metadata
-  annotation_match ["sumologic\.com.*"]
-  de_dot false
-  tag_to_kubernetes_name_regexp '.+?\.containers\.(?<pod_name>[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?<namespace>[^_]+)_(?<container_name>.+)-(?<docker_id>[a-z0-9]{64})\.log$'
-  merge_json_log false
-</filter>
+<match containers.**>
+  @type relabel
+  @label @NORMAL
+</match>
 
-<filter containers.**>
-  @type kubernetes_sumologic
-  source_name "#{ENV['SOURCE_NAME']}"
-  source_host "#{ENV['SOURCE_HOST']}"
-  log_format "#{ENV['LOG_FORMAT']}"
-  kubernetes_meta "#{ENV['KUBERNETES_META']}"
-  source_category "#{ENV['SOURCE_CATEGORY']}"
-  source_category_prefix "#{ENV['SOURCE_CATEGORY_PREFIX']}"
-  source_category_replace_dash "#{ENV['SOURCE_CATEGORY_REPLACE_DASH']}"
-  exclude_namespace_regex "#{ENV['EXCLUDE_NAMESPACE_REGEX']}"
-  exclude_pod_regex "#{ENV['EXCLUDE_POD_REGEX']}"
-  exclude_container_regex "#{ENV['EXCLUDE_CONTAINER_REGEX']}"
-  exclude_host_regex "#{ENV['EXCLUDE_HOST_REGEX']}"
-</filter>
+<label @NORMAL>
+  <filter containers.**>
+    @type kubernetes_metadata
+    annotation_match ["sumologic\.com.*"]
+    de_dot false
+    tag_to_kubernetes_name_regexp '.+?\.containers\.(?<pod_name>[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?<namespace>[^_]+)_(?<container_name>.+)-(?<docker_id>[a-z0-9]{64})\.log$'
+    merge_json_log false
+  </filter>
+
+  <filter containers.**>
+    @type kubernetes_sumologic
+    source_name "#{ENV['SOURCE_NAME']}"
+    source_host "#{ENV['SOURCE_HOST']}"
+    log_format "#{ENV['LOG_FORMAT']}"
+    kubernetes_meta "#{ENV['KUBERNETES_META']}"
+    source_category "#{ENV['SOURCE_CATEGORY']}"
+    source_category_prefix "#{ENV['SOURCE_CATEGORY_PREFIX']}"
+    source_category_replace_dash "#{ENV['SOURCE_CATEGORY_REPLACE_DASH']}"
+    exclude_namespace_regex "#{ENV['EXCLUDE_NAMESPACE_REGEX']}"
+    exclude_pod_regex "#{ENV['EXCLUDE_POD_REGEX']}"
+    exclude_container_regex "#{ENV['EXCLUDE_CONTAINER_REGEX']}"
+    exclude_host_regex "#{ENV['EXCLUDE_HOST_REGEX']}"
+  </filter>
+
+  @include /fluentd/conf.d/out.sumo.conf
+</label>