[Filebeat-8.1] module is configured with datasets enabled, but unable to setup

[renzedj]

I am experiencing the issue described in #29175, where I am unable to load Elastic Ingest pipelines using filebeat setup .... Per the instructions in the referenced issue, I have enabled the modules and set the datasets to enabled (e.g., syslog.enabled: true), however I am still getting the following error when attempting to do the setup, as follows:

filebeat@8d9084383041:~$ filebeat modules enable apache system
Module apache is already enabled
Module system is already enabled
filebeat@8d9084383041:~$ filebeat setup --index-management --pipelines --modules apache,system
Exiting: module apache is configured but has no enabled filesets

Here is the contents of my system.yml file, as an example:

# Module: system
# Docs: https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-module-system.html

- module: system
  # Syslog
  syslog:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

  # Authorization logs
  auth:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

Note that I am attempting to use the filebeat container image to execute this (I plan to automate this functionality), however I am mounting a local directory with enabled modules to ${path.config}/modules.d, and have validated that they're showing in the correct place inside the container.

The contents of my filebeat.yml is:

---
filebeat:
  config:
    modules:
      path: '${path.config}/modules.d/*.yml'
      reload:
        enabled: false

setup:
  kibana:
    host: '${KIBANA_URL}'
    protocol: 'https'
    username: '${ELASTIC_ADMIN_USER}'
    password: '${ELASTIC_ADMIN_PASSWORD}'
    ssl:
      enabled: true

  ilm:
    policy_name: 'filebeat-ilm-policy'
    enabled: 'true'
    pattern: '000001'

  template:
    name: 'filebeat'
    pattern: 'filebeat-*'
    settings:
      index:
        final_pipeline: '${SETUP_TEMPLATE_SETTINGS_INDEX_FINAL_PIPELINE:filebeat-final-pipeline}'

output:
  elasticsearch:
    index: 'filebeat-%{[agent.version]}'
    hosts:
      - '${ELASTICSEARCH_URL}'
    protocol: 'https'
    username: '${ELASTIC_ADMIN_USER}'
    password: '${ELASTIC_ADMIN_PASSWORD}'
    ssl:
      enabled: true

[renzedj]

Note that I am attempting to use the filebeat container image to execute this (I plan to automate this functionality), however I am mounting a local directory with enabled modules to ${path.config}/modules.d, and have validated that they're showing in the correct place inside the container.

Interesting. When I do this from a container, it fails (even when doing it manually). When I download the agent to a Linux host in our environment, everything functions as expected.

The only thing I have to do differently is run it with --strict.perms=false when executing it in a container. Could this be an issue?

Here are the commands I use on the Linux host to make things work as expected:

filebeat --path.home $(pwd) --path.config $(pwd) --path.data $(pwd) --path.logs $(pwd) modules enable apache
filebeat --path.home $(pwd) --path.config $(pwd) --path.data $(pwd) --path.logs $(pwd) setup --index-management --pipelines --modules=apache,system

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[cmacknz]

What version are you using? We changed the base image of the container recently and it had the unexpected side effect of breaking the setup command.

There is a fix coming in 7.17.2: #30435

[renzedj]

What version are you using? We changed the base image of the container recently and it had the unexpected side effect of breaking the setup command.

There is a fix coming in 7.17.2: #30435

I'm using 8.1.0. In fact, I was incorrect - when I thought it was working correctly on a host, I was doing a setup for 7.15.0 in error, because I wasn't doing ./filebeat .... I'm trying to do the setup from RHEL.

Is there a way to force the setup? While I can appreciate the reasoning behind not loading pipelines, etc. for filesets that aren't enabled, in a use case where I'm managing pipelines, etc. separately, it's more than a slight nuisance to do have to enable them in order to do the load.

[weslambert]

What version are you using? We changed the base image of the container recently and it had the unexpected side effect of breaking the setup command.
There is a fix coming in 7.17.2: #30435

I'm using 8.1.0. In fact, I was incorrect - when I thought it was working correctly on a host, I was doing a setup for 7.15.0 in error, because I wasn't doing ./filebeat .... I'm trying to do the setup from RHEL.

Is there a way to force the setup? While I can appreciate the reasoning behind not loading pipelines, etc. for filesets that aren't enabled, in a use case where I'm managing pipelines, etc. separately, it's more than a slight nuisance to do have to enable them in order to do the load.

I agree that it would be nice to have a --force flag to setup the pipelines for a module, even if it is not enabled. We relied on this functionality previously (then it broke with 8.1.0), and we have had to work around it using module/pipeline specific variable (enabled) overrides using -M.

[jvalente-salemstate]

I'm having the same issue with the microsoft module and the o365 module the latter of which which had worked in the past when I set up on 7x.

``> .\filebeat.exe version

filebeat version 8.1.1 (amd64), libbeat 8.1.1 [7f30bb3 built 2022-03-17 23:13:40 +0000 UTC]``

Both modules are outputting correctly to Logstash as I can see them when filtering on the event datasets within Kibana.

Also, wanted to edit in that I tested this with 7.15.0, which i still had backed up on my system and it did work, though the pipelines don't match what the version of everything else now.

[cmacknz]

I think there are now two separate threads going on in this issue:

1. A request for a --force flag when performing setup to bypass any validation we have.
2. A possible bug where filesets have been enabled, but setup still fails.

@kvch any thoughts on what might be causing problem 2 here?

[kvch]

We made sure all filesets are disabled by default in #28818. Since the PR was merged we cannot load assets using setup.

This problem is somewhat complex. Users can enable modules in 3 ways: in filebeat.yml, in modules.d and using the -modules flag. When we introduced the restriction above we did not consider the last method. So we broke accidentally broke the --modules flag. When setting the modules flag, we just enable a module, but there is no way to enable a fileset/dataset.

I see a few workarounds. One is adding a --force flag as @cmacknz suggested. Alternatively, we can add a new --dataset flag, where users could configure a list of datasets like nginx/access. The issue I see is especially with loading Ingest pipelines is that some pipelines require fileset/dataset configuration because they rely contains template expressions that are substituted based on fileset/dataset configuration. So in general, adding a new flag --dataset is not ideal (or --force).

In general, there is no need to load pipelines separately using setup. (Initially, setup did not even load pipelines. Many people requested it in the past for some reason and we gave in after a few years.) Every required pipeline is loaded before Filebeat ships the first event to Elasticsearch. So there is no need to run the command setup --pipelines -modules apache,system. It's enough to run filebeat setup --index-management and let Filebeat load pipelines on startup right before data collection starts.

[weslambert]

What if we don't want the templates loaded, and only want the pipelines?

Will --index-management allow for that? In our case, we only want the pipelines, and not the template(s).

At the moment, my workaround is to do the following (for every module/fileset):

filebeat setup --pipelines --modules $module -M "$module.$fileset.enabled=true" -c $FB_YML

[kvch]

Why are you loading only the pipelines?

[weslambert]

Because I am managing the templates externally, adding additional/custom mappings, etc.

[renzedj]

Because I am managing the templates externally, adding additional/custom mappings, etc.

This is my use case as well. I don't want to configure the individual filebeats to load pipelines, etc. In fact, your own documentation cautions against having multiple beats configured to load templates/modules/pipelines in order to prevent the Elasticsearch server from getting bogged down. My solution to this is to load them manually.

[renzedj]

What if we don't want the templates loaded, and only want the pipelines?

Will --index-management allow for that? In our case, we only want the pipelines, and not the template(s).

At the moment, my workaround is to do the following (for every module/fileset):

filebeat setup --pipelines --modules $module -M "$module.$fileset.enabled=true" -c $FB_YML

Thanks for the tip. Not wild about it, but it works.

[jvalente-salemstate]

What if we don't want the templates loaded, and only want the pipelines?
Will --index-management allow for that? In our case, we only want the pipelines, and not the template(s).
At the moment, my workaround is to do the following (for every module/fileset):
filebeat setup --pipelines --modules $module -M "$module.$fileset.enabled=true" -c $FB_YML

Thanks for the tip. Not wild about it, but it works.

Yeah, after testing it in 7.15 I also gave the -M flag a try in 8.1.1 and this also worked (and better for my purpose since evyerhthing loaded would be using the same version as my beats agent)

[kvch]

Because I am managing the templates externally, adding additional/custom mappings, etc.

Why is loading pipelines required for that? On startup Filebeat loads all required pipelines if necessary. I do not see any reason why you have to load pipelines separately.

In fact, your own documentation cautions against having multiple beats configured to load templates/modules/pipelines in order to prevent the Elasticsearch server from getting bogged down.

Where is that in our documentation? Our documentation should say that do not enable template and ILM loading for every Beat. The only reference I could find mentions that you should not enable setup.template.overwrite to avoid overloading ES.

For the record, Filebeat never tries to overwrite Ingest pipelines. You have to force overwriting pipelines by setting filebeat.overwrite_pipeline. You cannot overload Elasticsearch by starting multiple Filebeats if you do not load pipelines beforehand. Only one will load the pipeline unless you start all of the Filebeats at the same time.

Also, I do not think this warning applies to pipelines. There is no way to disable pipeline loading. You can only disable ILM and template loading. So it does not really protect you from anything. Filebeat will always check if the pipeline is available.

@drenze-athene Have you overloaded your ES with pipelines before?

[weslambert]

In our case, we don't use the Elasticsearch output for Filebeat. Instead, we use Logstash. We actually manually invoke (via a script running docker exec) the pipeline load using a custom Filebeat config to load the pipelines separately from the actual config used by the agent for sending logs.

[renzedj]

Where is that in our documentation? Our documentation should say that do not enable template and ILM loading for every Beat. The only reference I could find mentions that you should not enable setup.template.overwrite to avoid overloading ES.

Sorry - that's exactly what I was referring to. I may have misread.

@drenze-athene Have you overloaded your ES with pipelines before?

No. But then I've also always manually loaded templates and pipelines prior to deploying a new beat version.

[kvch]

Thank you @drenze-athene @jvalente-salemstate @weslambert for your input.

We should address this problem by adding a new flag named --force to enable loading pipelines again. In the documentation we should point out that it might be bad option in some cases, so everyone should be careful with it.

[stefnestor]

👋 howdy, Beats team! May I suggest there's ongoing confusion where this doc page still says the description's steps should work. This makes it failing appear to be a regression bug rather than unintended functionality / enhancement.

Also noting because it took me an extra minute, the workaround setting override described here looks like

$ pwd
/Users/stef/downloads/filebeat-8.2.2-darwin-x86_64
+ $ ./filebeat setup -e --pipelines --modules aws -M "aws.vpcflow.enabled=true"
{"log.level":"info","@timestamp":"2022-06-23T15:08:56.384-0600","log.origin":{"file.name":"instance/beat.go","file.line":685},"message":"Home path: [/Users/stef/downloads/filebeat-8.2.2-darwin-x86_64] Config path: [/Users/stef/downloads/filebeat-8.2.2-darwin-x86_64] Data path: [/Users/stef/downloads/filebeat-8.2.2-darwin-x86_64/data] Logs path: [/Users/stef/downloads/filebeat-8.2.2-darwin-x86_64/logs]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:08:56.384-0600","log.origin":{"file.name":"instance/beat.go","file.line":693},"message":"Beat ID: 4a7e721e-b86a-483b-bc09-8ba7cf5a5eee","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-06-23T15:08:59.387-0600","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":80},"message":"read token request for getting IMDSv2 token returns empty: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.389-0600","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1063},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/Users/stef/downloads/filebeat-8.2.2-darwin-x86_64","data":"/Users/stef/downloads/filebeat-8.2.2-darwin-x86_64/data","home":"/Users/stef/downloads/filebeat-8.2.2-darwin-x86_64","logs":"/Users/stef/downloads/filebeat-8.2.2-darwin-x86_64/logs"},"type":"filebeat","uuid":"4a7e721e-b86a-483b-bc09-8ba7cf5a5eee"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.390-0600","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1072},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"2f1e50cc31b960b1a975f2ebe08bd17be9a5e575","libbeat":"8.2.2","time":"2022-05-25T13:25:34.000Z","version":"8.2.2"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.390-0600","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1075},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"darwin","arch":"amd64","max_procs":8,"version":"go1.17.10"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.390-0600","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1079},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-06-15T21:27:48.851274-06:00","name":"Stef-Elastic.local","ip":["127.0.0.1/8","::1/128","fe80::1/64","fe80::aede:48ff:fe00:1122/64","fe80::14c4:aeea:3cd8:3f42/64","192.168.1.4/24","fe80::e02f:b8ff:feaf:5209/64","fe80::e02f:b8ff:feaf:5209/64","fe80::170c:18ef:b7f:4b1d/64","fe80::d2c3:fd6b:12b8:2da0/64","fe80::ce81:b1c:bd2c:69e/64","fe80::393d:d8a0:40c1:548a/64","fe80::f901:c684:7e1c:3339/64"],"kernel_version":"21.5.0","mac":["ac:de:48:00:11:22","36:7d:da:a3:4e:17","14:7d:da:a3:4e:17","e2:2f:b8:af:52:09","e2:2f:b8:af:52:09","96:d6:40:54:da:c1","96:d6:40:54:da:c5","96:d6:40:54:da:c0","96:d6:40:54:da:c4","96:d6:40:54:da:c1"],"os":{"type":"macos","family":"darwin","platform":"darwin","name":"Mac OS X","version":"10.16","major":10,"minor":16,"patch":0,"build":"21F79"},"timezone":"MDT","timezone_offset_sec":-21600,"id":"3CD1C3CD-207F-5502-BBB1-DC4EDC6AEEED"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.391-0600","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1108},"message":"Process info","service.name":"filebeat","system_info":{"process":{"cwd":"/Users/stef/Downloads/filebeat-8.2.2-darwin-x86_64","exe":"./filebeat","name":"filebeat","pid":30075,"ppid":26775,"start_time":"2022-06-23T15:08:56.279-0600"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.391-0600","log.origin":{"file.name":"instance/beat.go","file.line":325},"message":"Setup Beat: filebeat; Version: 8.2.2","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.392-0600","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: https://5103cce8eff845cf89df55783f5f5e35.us-west-2.aws.found.io:443","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.392-0600","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: Stef-Elastic.local","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.399-0600","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: aws (vpcflow)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.400-0600","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: https://5103cce8eff845cf89df55783f5f5e35.us-west-2.aws.found.io:443","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.920-0600","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 7.17.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.924-0600","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: aws (vpcflow)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:08:59.925-0600","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: https://5103cce8eff845cf89df55783f5f5e35.us-west-2.aws.found.io:443","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:09:00.185-0600","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 7.17.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:09:00.268-0600","log.logger":"modules","log.origin":{"file.name":"fileset/pipelines.go","file.line":133},"message":"Elasticsearch pipeline loaded.","service.name":"filebeat","pipeline":"filebeat-8.2.2-aws-vpcflow-pipeline","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:09:00.268-0600","log.origin":{"file.name":"cfgfile/reload.go","file.line":262},"message":"Loading of config files completed.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:09:00.269-0600","log.logger":"load","log.origin":{"file.name":"cfgfile/list.go","file.line":129},"message":"Stopping 1 runners ...","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-06-23T15:09:00.316-0600","log.logger":"modules","log.origin":{"file.name":"fileset/pipelines.go","file.line":133},"message":"Elasticsearch pipeline loaded.","service.name":"filebeat","pipeline":"filebeat-8.2.2-aws-vpcflow-pipeline","ecs.version":"1.6.0"}
Loaded Ingest pipelines

[matutter]

I just noticed that the -M flag was required. The docs (https://www.elastic.co/guide/en/beats/filebeat/current/load-ingest-pipelines.html) really need to be updated to show the -M flag being used because some of those examples do not do anything without it and it is misleading.

[ker2x]

filebeat 8.4.2 : unknown flag: --enable-all-filesets

[leehinman]

filebeat 8.4.2 : unknown flag: --enable-all-filesets

8.4.3 will have it

[willemdh]

@leehinman We just updated to Elastic 8.5.1 and stumbled again on this issue....

So we normally do this:

sudo /usr/share/filebeat/bin/filebeat setup --pipelines --modules elasticsearch,kibana,logstash

And expect the logstash pipelines to be loaded.. But hey actually don't. We would preferr not to use "--enable-all-filesets", as we only use the logstash modules..

So in what version will this be fixed? Currently we need for example to manually add certain pipelines or we get millions of errors these don't exist..

sudo /usr/share/filebeat/bin/filebeat setup --pipelines --modules logstash -M "logstash.slowlog.enabled=true"
sudo /usr/share/filebeat/bin/filebeat setup --pipelines --modules logstash -M "logstash.log.enabled=true"

[renzedj]

@leehinman We just updated to Elastic 8.5.1 and stumbled again on this issue....

So we normally do this:

sudo /usr/share/filebeat/bin/filebeat setup --pipelines --modules elasticsearch,kibana,logstash

And expect the logstash pipelines to be loaded.. But hey actually don't. We would preferr not to use "--enable-all-filesets", as we only use the logstash modules..

So in what version will this be fixed? Currently we need for example to manually add certain pipelines or we get millions of errors these don't exist..

sudo /usr/share/filebeat/bin/filebeat setup --pipelines --modules logstash -M "logstash.slowlog.enabled=true"
sudo /usr/share/filebeat/bin/filebeat setup --pipelines --modules logstash -M "logstash.log.enabled=true"

I keep hoping for a fix to this as well. In the meantime, as a workaround to make things easier, this is a bash script I use to just enable and upload all filesets for all modules. I run this on a copy of filebeat that I save in my home directory, so you'll have to change the path, but hope this helps.

#!/bin/bash
export FILESETS=""
export MODULES="$(find filebeat-${VERSION}/modules.d -type f | sed 's|^.*/||g;s|\.yml.*$||g')"
export MODULES="$(echo $MODULES | tr ' ' ',')"

pushd ${HOME}/filebeat-${VERSION}

printf "Uploading template: "
${HOME}/filebeat-${VERSION}/filebeat --path.home=${HOME}/filebeat-${VERSION} --path.config=${HOME}/beat-config --path.data=${HOME}/beat-config/data --path.logs=${HOME}/beat-config/logs setup --strict.perms=false --index-management

for m in $(echo "${MODULES}" | sed 's|,| |g'); do
  for fs in $(grep -E '^  [^# ]' modules.d/$m.* | sed -E 's|^  (.*):.*$|\1|g'); do
    FILESET="-M '${m}.${fs}.enabled=true'"
    printf "Uploading ${m}.${fs}: "
    ${HOME}/filebeat-${VERSION}/filebeat --path.home=${HOME}/filebeat-${VERSION} --path.config=${HOME}/beat-config --path.data=${HOME}/beat-config/data --path.logs=${HOME}/beat-config/logs setup --strict.perms=false --pipelines --modules=${m} -M "${m}.${fs}.enabled=true"
  done
done

[ker2x]

Thank you @willemdh @drenze-athene . I ended up upgrading ELK to v8 while keeping filbeat v7. I guess I'll keep it like this for some more month. it's a weird world when GitHub users on a closed issue are being more helpful than platinum support 🙃

[willemdh]

@ker2x @drenze-athene

I've been wondering many times lately if Elastic test their own upgrades decently.... So many newly introduced features but even more deprecated stuff with no clear migration path. I've been trying to escalate some problems before v8 was even released, but except for a few support engineer who acknowledge there is a problem, there doesn't seem to be a real fix coming for many 'major' updates..

Biggest problems imho ATM are
Datastreams aliases
Template overriding
Perfmon counter indexing

For years Elastic is telling us to use aliases and then release datastreams for which aliases are not compatible with index aliases...... This breaks everything ever built in Kibana when attempting to switch. See elastic/elasticsearch#66163

There is currently NO supported way to add / modify mappings or normalizers on index / component templates provided by Elastic, which survives an update. This makes upgrading very painful. Sth which worked flawlessly with the legacy templates order nrs. I think there is an internal 'enhancement request' for this no idea on the status, I guess it isnt even on their radar anymore. The result is we need to manually update the builtin templates and add our own modified settings and mappings component template. Every update.

No support for special chars in perfmon counters?? Monitoring based on Perfmon counters is so basic but this doesn't work for ages now... Will this ever get fixed, who knows.. See #31516

Sorry for the complaining @elasticmachine I needed to get this off my chest.

[willemdh]

Any news about a real solution for this problem?

We need to execute every update the setup for every module / fileset we use.... Very error prone and makes updating even harder then it already is.

[ker2x]

@leehinman can you reopen the bug until we have a proper solution instead of a workaround ?

[amitkanfer]

Are we planning to properly fix this issue? @jlind23

[jlind23]

Adding this to one of our following sprint to work on a better technical as it seems the current one didn't fix all the issues.
@leehinman do you recall what the remaining work or was our PR supposed to fix all the problems mentioned here?

Should we fix this by added a --force flag as suggested in this comment: #30916 (comment) ?

[leehinman]

Adding this to one of our following sprint to work on a better technical as it seems the current one didn't fix all the issues. @leehinman do you recall what the remaining work or was our PR supposed to fix all the problems mentioned here?

Should we fix this by added a --force flag as suggested in this comment: #30916 (comment) ?

Yes it will be a "force" flag, but I think it just needs to "enable module filesets", but it should only work in setup and only when "--modules" is listed, and only for the modules in the "--modules" argument list. Then you would run something like:

filebeat setup --pipelines --modules elasticsearch,kibana,logstash  --force-enable-module-filesets

With the intent that it would load the same resources as the following did in the past.

sudo /usr/share/filebeat/bin/filebeat setup --pipelines --modules elasticsearch,kibana,logstash

[jlind23]

I believe this is what @ker2x and others are looking if I am not mistaken.

[NITEMAN]

FTR: This seems to fail on 8.8.1:

$ docker run --rm docker.elastic.co/beats/filebeat:8.8.1 filebeat setup -e -v --enable-all-filesets --pipelines --modules system -v
{"log.level":"info","@timestamp":"2023-06-23T10:04:59.115Z","log.origin":{"file.name":"instance/beat.go","file.line":779},"message":"Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-23T10:04:59.119Z","log.origin":{"file.name":"instance/beat.go","file.line":787},"message":"Beat ID: 87680abe-f55a-47ff-b96f-c03ff87d6581","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2023-06-23T10:05:02.121Z","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":81},"message":"read token request for getting IMDSv2 token returns empty: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-23T10:05:02.122Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1299},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/usr/share/filebeat","data":"/usr/share/filebeat/data","home":"/usr/share/filebeat","logs":"/usr/share/filebeat/logs"},"type":"filebeat","uuid":"87680abe-f55a-47ff-b96f-c03ff87d6581"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-06-23T10:05:02.122Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1308},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"7ba375a8778fe6c1a61376a6c015e8cea71caf21","libbeat":"8.8.1","time":"2023-06-05T20:27:02.000Z","version":"8.8.1"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-06-23T10:05:02.122Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1311},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":8,"version":"go1.19.9"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-06-23T10:05:02.123Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1317},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2023-06-16T15:58:37Z","containerized":true,"name":"f32fe1d1bce3","ip":["127.0.0.1","172.17.0.2"],"kernel_version":"4.19.0-24-amd64","mac":["02:42:ac:11:00:02"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"20.04.6 LTS (Focal Fossa)","major":20,"minor":4,"patch":6,"codename":"focal"},"timezone":"UTC","timezone_offset_sec":0},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-06-23T10:05:02.123Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1346},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":null,"effective":null,"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null},"cwd":"/usr/share/filebeat","exe":"/usr/share/filebeat/filebeat","name":"filebeat","pid":7,"ppid":1,"seccomp":{"mode":"filter","no_new_privs":false},"start_time":"2023-06-23T10:04:58.430Z"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-06-23T10:05:02.123Z","log.origin":{"file.name":"instance/beat.go","file.line":330},"message":"Setup Beat: filebeat; Version: 8.8.1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-23T10:05:02.127Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":108},"message":"elasticsearch url: http://elasticsearch:9200","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-23T10:05:02.128Z","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":105},"message":"Beat name: f32fe1d1bce3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-23T10:05:02.128Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":120},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-06-23T10:05:02.128Z","log.origin":{"file.name":"instance/beat.go","file.line":1274},"message":"Exiting: module system is configured but has no enabled filesets","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: module system is configured but has no enabled filesets

[Terkea]

The same issue here in 8.8.2

{"log.level":"error","@timestamp":"2023-07-07T11:09:58.914Z","log.origin":{"file.name":"instance/beat.go","file.line":1274},"message":"Exiting: module elasticsearch is configured but has no enabled filesets","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: module elasticsearch is configured but has no enabled filesets

Interestingly, this error seems to kick in regardless of the module that I enable. I tried this with auditd and system and I would have the exact same error.

Any solution for this? It seems to be a really old issue

[Curry-rose]

8.8.2 中的相同问题

{"log.level":"error","@timestamp":"2023-07-07T11:09:58.914Z","log.origin":{"file.name":"instance/beat.go","file.line":1274},"message":"Exiting: module elasticsearch is configured but has no enabled filesets","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: module elasticsearch is configured but has no enabled filesets

有趣的是，无论我启用哪个模块，此错误似乎都会启动。我用审计和系统尝试了这个，我会有完全相同的错误。

有什么解决方案吗？这似乎是一个非常古老的问题

me too bro

[Anatr0p]

Use sudo filebeat setup --pipelines --modules nginx -M "nginx.error.enabled=true" - example for nginx module