[7.x](backport #25671) Update Go version to 1.16.4 (#25994)

* Update Go version to 1.16.4 (#25671)

Update to go v1.16.4. Change MacOS cross compile to use
debian 10 image and 10.14 of the SDK. Fix arm7 packaging.

(cherry picked from commit 4dc5285)

# Conflicts:
#	libbeat/docs/version.asciidoc

* update version.asciidoc

Co-authored-by: Michel Laterman <[email protected]>

.go-version

@@ -1 +1 @@
-1.15.12
+1.16.4

CHANGELOG-developer.next.asciidoc

@@ -50,6 +50,8 @@ The list below covers the major changes between 7.0.0-rc2 and master only.
 - Remove global ACK handler support via `SetACKHandler` from publisher pipeline. {pull}19632[19632]
 - Make implementing `Close` required for `reader.Reader` interfaces. {pull}20455[20455]
 - Remove `NumCPU` as clients should update the CPU count on the fly in case of config changes in a VM. {pull}23154[23154]
+- Remove Metricbeat EventFetcher and EventsFetcher interface. Use the reporter interface instead. {pull}25093[25093]
+- Update Darwin build image to a debian 10 base that increases the MacOS SDK and minimum supported version used in build to 10.14. {issue}24193[24193]
 
 ==== Bugfixes
 
@@ -105,4 +107,5 @@ The list below covers the major changes between 7.0.0-rc2 and master only.
 - Update Go version to 1.15.9. {pull}24442[24442]
 - Update Go version to 1.15.10. {pull}24606[24606]
 - Update Go version to 1.15.12. {pull}25629[25629]
+- Update Go version to 1.16.4. {issue}25346[25346] {pull}25671[25671]
 - Add sorting to array fields for generated data files (*-generated.json) {pull}25320[25320]

CHANGELOG.next.asciidoc

@@ -19,6 +19,29 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Introduce APM libbeat instrumentation, active when running the beat with ELASTIC_APM_ACTIVE=true. {pull}17938[17938]
 - Make error message about locked data path actionable. {pull}18667[18667]
 - Fix panic with inline SSL when the certificate or key were small than 256 bytes. {pull}23820[23820]
+- Ensure dynamic template names are unique for the same field. {pull}18849[18849]
+- Remove the deprecated `xpack.monitoring.*` settings. Going forward only `monitoring.*` settings may be used. {issue}9424[9424] {pull}18608[18608]
+- Added `certificate` TLS verification mode to ignore server name mismatch. {issue}12283[12283] {pull}20293[20293]
+- Autodiscover doesn't generate any configuration when a variable is missing. Previously it generated an incomplete configuration. {pull}20898[20898]
+- Remove redundant `cloudfoundry.*.timestamp` fields. This value is set in `@timestamp`. {pull}21175[21175]
+- Allow embedding of CAs, Certificate of private keys for anything that support TLS in ouputs and inputs. {pull}21179[21179]
+- Update to Golang 1.12.1. {pull}11330[11330]
+- Disable Alibaba Cloud and Tencent Cloud metadata providers by default. {pull}13812[12812]
+- API address is a required setting in `add_cloudfoundry_metadata`. {pull}21759[21759]
+- Update to ECS 1.7.0. {pull}22571[22571]
+- Add support for SCRAM-SHA-512 and SCRAM-SHA-256 in Kafka output. {pull}12867[12867]
+- Remove id_field_data {pull}25239[25239]
+- MacOSX minimum supported version set to 10.14 {issue}24193{24193}
+
+*Auditbeat*
+
+- File integrity dataset (macOS): Replace unnecessary `file.origin.raw` (type keyword) with `file.origin.text` (type `text`). {issue}12423[12423] {pull}15630[15630]
+- Change event.kind=error to event.kind=event to comply with ECS. {issue}18870[18870] {pull}20685[20685]
+- Change network.direction values to ECS recommended values (inbound, outbound). {issue}12445[12445] {pull}20695[20695]
+- Docker container needs to be explicitly run as user root for auditing. {pull}21202[21202]
+- File integrity dataset no longer includes the leading dot in `file.extension` values (e.g. it will report "png" instead of ".png") to comply with ECS. {pull}21644[21644]
+- Use ECS 1.7 ingress/egress network directions instead of inbound/outbound. {pull}22991[22991]
+- Use ingress/egress instead of inbound/outbound for ECS 1.7 in auditd module. {pull}23000[23000]
 
 *Auditbeat*
 

NOTICE.txt

@@ -37548,37 +37548,6 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 
 
---------------------------------------------------------------------------------
-Dependency : github.com/poy/eachers
-Version: v0.0.0-20181020210610-23942921fe77
-Licence type (autodetected): MIT
---------------------------------------------------------------------------------
-
-Contents of probable licence file $GOMODCACHE/github.com/poy/[email protected]/LICENSE.md:
-
-The MIT License (MIT)
-
-Copyright (c) 2016 Andrew Poydence
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-
-
 --------------------------------------------------------------------------------
 Dependency : github.com/prometheus/client_golang
 Version: v1.1.1-0.20190913103102-20428fa0bffc

auditbeat/Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.15.12
+FROM golang:1.16.4
 
 RUN \
     apt-get update \

dev-tools/mage/crossbuild.go

@@ -198,7 +198,9 @@ func crossBuildImage(platform string) (string, error) {
 
 	switch {
 	case strings.HasPrefix(platform, "darwin"):
-		tagSuffix = "darwin"
+		tagSuffix = "darwin-debian10"
+	case strings.HasPrefix(platform, "linux/armv7"):
+		tagSuffix = "armhf"
 	case strings.HasPrefix(platform, "linux/arm"):
 		tagSuffix = "arm"
 		if runtime.GOARCH == "arm64" {

dev-tools/mage/gotest_test.go

@@ -364,7 +364,7 @@ func TestGoTest_Helper_WithWrongPanic(t *testing.T) {
 	t.Run("setup failing go-routine", func(t *testing.T) {
 		go func() {
 			time.Sleep(1 * time.Second)
-			t.Fatal("oops")
+			t.Error("oops")
 		}()
 	})
 

filebeat/Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.15.12
+FROM golang:1.16.4
 
 RUN \
     apt-get update \

generator/common/beatgen/setup/setup.go

@@ -151,5 +151,9 @@ func GitAdd() error {
 
 // Update updates the generated files (aka make update).
 func Update() error {
-	return sh.Run("make", "update")
+	err := sh.Run("make", "update")
+	if err != nil {
+		return err
+	}
+	return gotool.Mod.Tidy()
 }

go.mod

@@ -1,6 +1,6 @@
 module github.com/elastic/beats/v7
 
-go 1.15
+go 1.16
 
 require (
 	4d63.com/tz v1.1.1-0.20191124060701-6d37baae851b
@@ -132,7 +132,6 @@ require (
 	github.com/pierrre/gotestcover v0.0.0-20160517101806-924dca7d15f0
 	github.com/pkg/errors v0.9.1
 	github.com/pmezard/go-difflib v1.0.0
-	github.com/poy/eachers v0.0.0-20181020210610-23942921fe77 // indirect
 	github.com/prometheus/client_golang v1.1.1-0.20190913103102-20428fa0bffc // indirect
 	github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
 	github.com/prometheus/common v0.7.0

go.sum

@@ -674,8 +674,6 @@ github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/poy/eachers v0.0.0-20181020210610-23942921fe77 h1:SNdqPRvRsVmYR0gKqFvrUKhFizPJ6yDiGQ++VAJIoDg=
-github.com/poy/eachers v0.0.0-20181020210610-23942921fe77/go.mod h1:x1vqpbcMW9T/KRcQ4b48diSiSVtYgvwQ5xzDByEg4WE=
 github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
 github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
 github.com/prometheus/client_golang v1.1.0/go.mod h1:I1FGZT9+L76gKKOs5djB6ezCbFQP1xR9D75/vuwEF3g=

heartbeat/Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.15.12
+FROM golang:1.16.4
 
 RUN \
     apt-get update \

journalbeat/Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.15.12
+FROM golang:1.16.4
 
 RUN \
     apt-get update \

journalbeat/magefile.go

@@ -137,6 +137,8 @@ func selectImage(platform string) (string, error) {
 	tagSuffix := "main"
 
 	switch {
+	case strings.HasPrefix(platform, "linux/armv7"):
+		tagSuffix = "armhf"
 	case strings.HasPrefix(platform, "linux/arm"):
 		tagSuffix = "arm"
 		if runtime.GOARCH == "arm64" {

libbeat/Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.15.12
+FROM golang:1.16.4
 
 RUN \
     apt-get update \

libbeat/docs/version.asciidoc

@@ -1,6 +1,6 @@
 :stack-version: 7.14.0
 :doc-branch: 7.x
-:go-version: 1.15.12
+:go-version: 1.16.4
 :release-state: unreleased
 :python: 3.7
 :docker: 1.12

metricbeat/Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.15.12
+FROM golang:1.16.4
 
 RUN \
     apt-get update \

metricbeat/module/http/_meta/Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.15.12
+FROM golang:1.16.4
 
 COPY test/main.go main.go
 

packetbeat/Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.15.12
+FROM golang:1.16.4
 
 RUN \
     apt-get update \

x-pack/elastic-agent/Dockerfile

@@ -1,4 +1,4 @@
-ARG GO_VERSION=1.12.7
+ARG GO_VERSION=1.16.4
 FROM circleci/golang:${GO_VERSION}
 
 

x-pack/functionbeat/Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.15.12
+FROM golang:1.16.4
 
 RUN \
     apt-get update \

x-pack/libbeat/Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.15.12
+FROM golang:1.16.4
 
 RUN \
     apt-get update \