Functionbeat can't find default credentials

[PatriceDouge]

- Steps to Reproduce:

1. Add in set aws creds in powershell
2. Add configs:
`functionbeat.provider.aws.endpoint: "s3.amazonaws.com"
functionbeat.provider.aws.deploy_bucket: "fn-deploy"

functionbeat.provider.aws.functions:

• name: cloudwatch-to-es
  enabled: true
  type: cloudwatch_logs

  triggers:

  • log_group_name: group
    filter_pattern: "ERROR"

setup.kibana:
host: "000:5601"

output.elasticsearch:
hosts: ["000:9200"]
ssl.certificate: "path/to/pem"
protocol: "https"
username: "elastic"
password: "changeme"`

3. In powershell as admin: ./functionbeat -v -e -d "*" deploy cloudwatch-to-es

Logs:
2020-03-30T09:44:40.610-0400 INFO instance/beat.go:622 Home path: [C:\Program Files\Functionbeat] Config path: [C:\Program Files\Functionbeat] Data path: [/tmp] Logs path: [/tmp/logs] 2020-03-30T09:44:40.611-0400 DEBUG [beat] instance/beat.go:674 Beat metadata path: \tmp\meta.json 2020-03-30T09:44:40.612-0400 INFO instance/beat.go:630 Beat ID: c0fcb17c-41c2-4384-bd71-a0ad65082391 2020-03-30T09:44:40.636-0400 DEBUG [filters] add_cloud_metadata/providers.go:126 add_cloud_metadata: starting to fetch metadata, timeout=3s 2020-03-30T09:44:40.640-0400 DEBUG [filters] add_cloud_metadata/providers.go:162 add_cloud_metadata: received disposition for openstack after 1.0006ms. result=[provider:openstack, error=failed requesting openstack metadata: Get http://169.254.169.254/2009-04-04/meta-data/instance-id: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network., metadata={}] 2020-03-30T09:44:40.640-0400 DEBUG [filters] add_cloud_metadata/providers.go:162 add_cloud_metadata: received disposition for digitalocean after 2.9992ms. result=[provider:digitalocean, error=failed requesting digitalocean metadata: Get http://169.254.169.254/metadata/v1.json: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network., metadata={}] 2020-03-30T09:44:40.640-0400 DEBUG [filters] add_cloud_metadata/providers.go:162 add_cloud_metadata: received disposition for aws after 2.9992ms. result=[provider:aws, error=failed requesting aws metadata: Get http://169.254.169.254/2014-02-25/dynamic/instance-identity/document: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network., metadata={}] 2020-03-30T09:44:40.640-0400 DEBUG [filters] add_cloud_metadata/providers.go:162 add_cloud_metadata: received disposition for gcp after 2.9992ms. result=[provider:gcp, error=failed requesting gcp metadata: Get http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network., metadata={}] 2020-03-30T09:44:40.641-0400 DEBUG [filters] add_cloud_metadata/providers.go:162 add_cloud_metadata: received disposition for az after 4.0017ms. result=[provider:az, error=failed requesting az metadata: Get http://169.254.169.254/metadata/instance/compute?api-version=2017-04-02: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network., metadata={}] 2020-03-30T09:44:40.641-0400 DEBUG [filters] add_cloud_metadata/providers.go:129 add_cloud_metadata: fetchMetadata ran for 4.0017ms 2020-03-30T09:44:40.641-0400 INFO add_cloud_metadata/add_cloud_metadata.go:89 add_cloud_metadata: hosting provider type not detected. 2020-03-30T09:44:40.641-0400 DEBUG [processors] processors/processor.go:101 Generated new processors: add_host_metadata=[netinfo.enabled=[false], cache.ttl=[5m0s]], add_cloud_metadata=null **error while creating CLIManager: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information**.

• I am using AWS, so im not sure why google is looking for credentials. I removed the gcp config info from the yaml file but its still giving me that error as an output.

• My AWS credentials are in the .aws file in "C::/user/" as [ default ] and also in my environment variables.

• I have confirmed that aws cli works so my credentials should be working and be accessible by functionbeat.

• I have confirmed that my network has access to aws from cli so im not sure why its saying "A socket operation was attempted to an unreachable network" in the logs

• I have tried everything to get it working, is there any other things I can try to see if I am doing something wrong? At this point this seems like a bug with finding my AWS credentials in my environment.

• Version: 7.6.1

• Operating System: Windows 10

• Discuss Forum URL: https://discuss.elastic.co/t/functionbeat-not-finding-default-credentials/222784

Thanks for you help,

Patrice

[ppf2]

Re-opening to address this in product.

The default functionbeat.yml file has various templates in place with enabled: false set.

I have only enabled the Cloudwatch function in the yml.

But functionbeat throws the following error on deploy related to Google services:

2020-05-11T13:01:02.707-0700	DEBUG	[processors]	processors/processor.go:101	Generated new processors: add_host_metadata=[netinfo.enabled=[false], cache.ttl=[5m0s]], add_cloud_metadata=null
error while creating CLIManager: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

Not until I commented out the entire section below (even though this section already has enabled: false set) did the error go away.

# Configure functions to run on Google Cloud Platform, currently we assume that the credentials
# are present in the environment to correctly create the function when using the CLI.
#
# Configure which region your project is located in.
#functionbeat.provider.gcp.location_id: "europe-west2"
# Configure which Google Cloud project to deploy your functions.
#functionbeat.provider.gcp.project_id: "my-project-123456"
# Configure the Google Cloud Storage we should upload the function artifact.
#functionbeat.provider.gcp.storage_name: "functionbeat-deploy"

#functionbeat.provider.gcp.functions:
  # Define the list of function availables, each function required to have a unique name.
  # Create a function that accepts events coming from Google Pub/Sub.
#  - name: pubsub
#    enabled: false
#    type: pubsub

    # Description of the method to help identify them when you run multiples functions.
#    description: "Google Cloud Function for Pub/Sub"

    # The maximum memory allocated for this function.
    # Default is 256MB.
    #memory_size: 256MB

    # Execution timeout in seconds. If the function does not finish in time,
    # it is considered failed and terminated. Default is 60s.
    #timeout: 60s

    # Email of the service account of the function. Defaults to {projectid}@appspot.gserviceaccount.com
    #service_account_email: {projectid}@appspot.gserviceaccount.com

    # Labels of the function.
    #labels:
    # mylabel: label

    # VPC Connector this function can connect to.
    # Format: projects/*/locations/*/connectors/* or fully-qualified URI
    #vpc_connector: ""

    # Number of maximum instances running at the same time. Default is unlimited.
    #maximum_instances: 0

#    trigger:
#      resource: "projects/_/pubsub/myPubSub"
      #service: "pubsub.googleapis.com"

We should address this in product instead of having end users workaround this by commenting out sections of the yml.

[elasticmachine]

Pinging @elastic/integrations-services (Team:Services)