Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[infra] Introduce "language" attribute in the project.yaml (#3297). #3299

Merged
merged 4 commits into from
Jan 30, 2020
Merged

[infra] Introduce "language" attribute in the project.yaml (#3297). #3299

merged 4 commits into from
Jan 30, 2020

Conversation

Dor1s
Copy link
Contributor

@Dor1s Dor1s commented Jan 28, 2020

If this looks good, I'll include all other non C/C++ projects in this PR before landing.

@TravisBuddy
Copy link

Travis tests have failed

Hey @Dor1s,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

2nd Build

View build log

./infra/travis/travis_build.py 
Building project mtail
Running command: python /home/travis/build/google/oss-fuzz/infra/helper.py build_fuzzers mtail --engine libfuzzer --sanitizer address --architecture x86_64
Running: docker build -t gcr.io/oss-fuzz/mtail projects/mtail

Step 1/6 : FROM gcr.io/oss-fuzz-base/base-builder
latest: Pulling from oss-fuzz-base/base-builder





















Digest: sha256:c0466abc5bd8515aaceb3fa06725c982522c1127d3559da8615ad22fc43c78a7
Status: Downloaded newer image for gcr.io/oss-fuzz-base/base-builder:latest
 ---> 5c065059a5cc
Step 2/6 : MAINTAINER [email protected]
 ---> Running in e42904987955
Removing intermediate container e42904987955
 ---> b1adf79f9b4e
Step 3/6 : RUN mkdir -p github.com/google
 ---> Running in a1508c99f624
Removing intermediate container a1508c99f624
 ---> 4db5d7bb9c17
Step 4/6 : RUN git clone --depth 1 https://github.com/google/mtail github.com/google/mtail
 ---> Running in e251e24e1d7c
Cloning into 'github.com/google/mtail'...
Removing intermediate container e251e24e1d7c
 ---> d5fd1342d5ca
Step 5/6 : WORKDIR github.com/google/mtail
 ---> Running in 366a6a94c9a3
Removing intermediate container 366a6a94c9a3
 ---> c5f86e9e6433
Step 6/6 : COPY build.sh $SRC/
 ---> 19ec75d7dd5e
Successfully built 19ec75d7dd5e
Successfully tagged gcr.io/oss-fuzz/mtail:latest
Keeping existing build artifacts as-is (if any).
Running: docker run --rm --privileged -i --cap-add SYS_PTRACE -e FUZZING_ENGINE=libfuzzer -e SANITIZER=address -e ARCHITECTURE=x86_64 -v /home/travis/build/google/oss-fuzz/build/out/mtail:/out -v /home/travis/build/google/oss-fuzz/build/work/mtail:/work -t gcr.io/oss-fuzz/mtail
---------------------------------------------------------------
Compiling libFuzzer to /usr/lib/libFuzzingEngine.a... ar: creating /usr/lib/libFuzzingEngine.a
 done.
CC=clang
CXX=clang++
CFLAGS=-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link
CXXFLAGS=-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link -stdlib=libc++
---------------------------------------------------------------
+ export GOPATH=/root/go:/
+ GOPATH=/root/go:/
+ make GO111MODULE=off --debug install_deps
GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading go.opencensus.io v0.22.2
go: downloading github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7
go: downloading github.com/prometheus/client_golang v1.3.0
go: downloading github.com/fsnotify/fsnotify v1.4.7
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/google/go-cmp v0.4.0
go: extracting github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: extracting github.com/pkg/errors v0.9.1
go: extracting github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7
go: downloading github.com/prometheus/common v0.9.0
go: downloading contrib.go.opencensus.io/exporter/jaeger v0.2.0
go: extracting github.com/prometheus/client_golang v1.3.0
go: extracting go.opencensus.io v0.22.2
go: extracting github.com/fsnotify/fsnotify v1.4.7
go: extracting contrib.go.opencensus.io/exporter/jaeger v0.2.0
go: downloading google.golang.org/api v0.15.0
go: downloading github.com/uber/jaeger-client-go v2.22.1+incompatible
go: extracting github.com/google/go-cmp v0.4.0
go: extracting github.com/prometheus/common v0.9.0
go: downloading github.com/prometheus/procfs v0.0.8
go: downloading github.com/cespare/xxhash/v2 v2.1.1
go: extracting github.com/cespare/xxhash/v2 v2.1.1
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/prometheus/client_model v0.2.0
go: extracting github.com/prometheus/procfs v0.0.8
go: extracting github.com/beorn7/perks v1.0.1
go: downloading golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
go: extracting golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
go: extracting github.com/prometheus/client_model v0.2.0
go: downloading google.golang.org/grpc v1.26.0
go: downloading golang.org/x/sys v0.0.0-20200117145432-59e60aa80a0c
go: extracting github.com/golang/protobuf v1.3.2
go: extracting github.com/uber/jaeger-client-go v2.22.1+incompatible
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: extracting github.com/matttproud/golang_protobuf_extensions v1.0.1
go: extracting google.golang.org/grpc v1.26.0
go: downloading google.golang.org/genproto v0.0.0-20200117163144-32f20d992d24
go: extracting golang.org/x/sys v0.0.0-20200117145432-59e60aa80a0c
go: extracting google.golang.org/genproto v0.0.0-20200117163144-32f20d992d24
go: extracting google.golang.org/api v0.15.0
go: downloading golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e
go: extracting golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: finding contrib.go.opencensus.io/exporter/jaeger v0.2.0
go: finding go.opencensus.io v0.22.2
go: finding github.com/pkg/errors v0.9.1
go: finding github.com/fsnotify/fsnotify v1.4.7
go: finding github.com/prometheus/client_golang v1.3.0
go: finding github.com/google/go-cmp v0.4.0
go: finding github.com/uber/jaeger-client-go v2.22.1+incompatible
go: finding github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7
go: finding golang.org/x/sys v0.0.0-20200117145432-59e60aa80a0c
go: finding golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
go: finding google.golang.org/api v0.15.0
go: finding github.com/beorn7/perks v1.0.1
go: finding github.com/cespare/xxhash/v2 v2.1.1
go: finding github.com/prometheus/common v0.9.0
go: finding github.com/golang/protobuf v1.3.2
go: finding golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e
go: finding github.com/prometheus/client_model v0.2.0
go: finding github.com/prometheus/procfs v0.0.8
go: finding github.com/matttproud/golang_protobuf_extensions v1.0.1
go: finding google.golang.org/grpc v1.26.0
go: finding google.golang.org/genproto v0.0.0-20200117163144-32f20d992d24
Updating goal targets....
 File 'install_deps' does not exist.
   File '.dep-stamp' does not exist.
     File 'print-version' does not exist.
    Must remake target 'print-version'.
go version go1.13.6 linux/amd64
    Successfully remade target file 'print-version'.
       File '/root/go/bin/goyacc' does not exist.
      Must remake target '/root/go/bin/goyacc'.
go get -v golang.org/x/tools/cmd/goyacc
golang.org/x/tools/cmd/goyacc
      Successfully remade target file '/root/go/bin/goyacc'.
       File '/root/go/bin/togo' does not exist.
      Must remake target '/root/go/bin/togo'.
go get -v github.com/flazz/togo
github.com/flazz/togo (download)
github.com/flazz/togo
      Successfully remade target file '/root/go/bin/togo'.
  Must remake target '.dep-stamp'.
Install all dependencies, ensuring they're updated
go get -v bufio bytes context contrib.go.opencensus.io/exporter/jaeger crypto/rand encoding/json expvar flag fmt github.com/fsnotify/fsnotify github.com/golang/glog github.com/golang/groupcache/lru github.com/google/go-cmp/cmp github.com/google/go-cmp/cmp/cmpopts github.com/pkg/errors github.com/prometheus/client_golang/prometheus github.com/prometheus/client_golang/prometheus/promhttp github.com/prometheus/common/version go.opencensus.io/trace go.opencensus.io/zpages html/template io io/ioutil math math/big math/rand net net/http net/http/pprof os os/exec os/signal path path/filepath reflect regexp regexp/syntax runtime runtime/debug sort strconv strings sync sync/atomic syscall testing text/tabwriter time unicode unicode/utf8
get "contrib.go.opencensus.io/exporter/jaeger": found meta tag get.metaImport{Prefix:"contrib.go.opencensus.io/exporter/jaeger", VCS:"git", RepoRoot:"https://github.com/census-ecosystem/opencensus-go-exporter-jaeger"} at //contrib.go.opencensus.io/exporter/jaeger?go-get=1
contrib.go.opencensus.io/exporter/jaeger (download)
github.com/uber/jaeger-client-go (download)
get "go.opencensus.io/trace": found meta tag get.metaImport{Prefix:"go.opencensus.io", VCS:"git", RepoRoot:"https://github.com/census-instrumentation/opencensus-go"} at //go.opencensus.io/trace?go-get=1
get "go.opencensus.io/trace": verifying non-authoritative meta tag
go.opencensus.io (download)
github.com/golang/groupcache (download)
get "google.golang.org/api/support/bundler": found meta tag get.metaImport{Prefix:"google.golang.org/api", VCS:"git", RepoRoot:"https://code.googlesource.com/google-api-go-client"} at //google.golang.org/api/support/bundler?go-get=1
get "google.golang.org/api/support/bundler": verifying non-authoritative meta tag
google.golang.org/api (download)
get "golang.org/x/sync/semaphore": found meta tag get.metaImport{Prefix:"golang.org/x/sync", VCS:"git", RepoRoot:"https://go.googlesource.com/sync"} at //golang.org/x/sync/semaphore?go-get=1
get "golang.org/x/sync/semaphore": verifying non-authoritative meta tag
golang.org/x/sync (download)
github.com/fsnotify/fsnotify (download)
get "golang.org/x/sys/unix": found meta tag get.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at //golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
golang.org/x/sys (download)
github.com/golang/glog (download)
github.com/google/go-cmp (download)
get "golang.org/x/xerrors": found meta tag get.metaImport{Prefix:"golang.org/x/xerrors", VCS:"git", RepoRoot:"https://go.googlesource.com/xerrors"} at //golang.org/x/xerrors?go-get=1
golang.org/x/xerrors (download)
github.com/pkg/errors (download)
github.com/prometheus/client_golang (download)
github.com/beorn7/perks (download)
github.com/cespare/xxhash (download)
github.com/golang/protobuf (download)
github.com/prometheus/client_model (download)
github.com/prometheus/common (download)
github.com/matttproud/golang_protobuf_extensions (download)
github.com/prometheus/procfs (download)
get "google.golang.org/grpc/codes": found meta tag get.metaImport{Prefix:"google.golang.org/grpc", VCS:"git", RepoRoot:"https://github.com/grpc/grpc-go"} at //google.golang.org/grpc/codes?go-get=1
get "google.golang.org/grpc/codes": verifying non-authoritative meta tag
google.golang.org/grpc (download)
get "google.golang.org/grpc/grpclog": found meta tag get.metaImport{Prefix:"google.golang.org/grpc", VCS:"git", RepoRoot:"https://github.com/grpc/grpc-go"} at //google.golang.org/grpc/grpclog?go-get=1
get "google.golang.org/grpc/grpclog": verifying non-authoritative meta tag
get "google.golang.org/grpc/metadata": found meta tag get.metaImport{Prefix:"google.golang.org/grpc", VCS:"git", RepoRoot:"https://github.com/grpc/grpc-go"} at //google.golang.org/grpc/metadata?go-get=1
get "google.golang.org/grpc/metadata": verifying non-authoritative meta tag
get "google.golang.org/grpc/stats": found meta tag get.metaImport{Prefix:"google.golang.org/grpc", VCS:"git", RepoRoot:"https://github.com/grpc/grpc-go"} at //google.golang.org/grpc/stats?go-get=1
get "google.golang.org/grpc/stats": verifying non-authoritative meta tag
get "google.golang.org/grpc/status": found meta tag get.metaImport{Prefix:"google.golang.org/grpc", VCS:"git", RepoRoot:"https://github.com/grpc/grpc-go"} at //google.golang.org/grpc/status?go-get=1
get "google.golang.org/grpc/status": verifying non-authoritative meta tag
get "google.golang.org/genproto/googleapis/rpc/status": found meta tag get.metaImport{Prefix:"google.golang.org/genproto", VCS:"git", RepoRoot:"https://github.com/google/go-genproto"} at //google.golang.org/genproto/googleapis/rpc/status?go-get=1
get "google.golang.org/genproto/googleapis/rpc/status": verifying non-authoritative meta tag
google.golang.org/genproto (download)
github.com/golang/groupcache/lru
github.com/uber/jaeger-client-go/thrift
go.opencensus.io
go.opencensus.io/internal
go.opencensus.io/trace/internal
go.opencensus.io/trace/tracestate
go.opencensus.io/trace
golang.org/x/sync/semaphore
google.golang.org/api/support/bundler
github.com/uber/jaeger-client-go/thrift-gen/jaeger
github.com/uber/jaeger-client-go/thrift-gen/zipkincore
github.com/uber/jaeger-client-go/thrift-gen/agent
runtime/cgo
golang.org/x/sys/unix
net
github.com/fsnotify/fsnotify
os/user
github.com/golang/glog
github.com/google/go-cmp/cmp/internal/diff
github.com/google/go-cmp/cmp/internal/flags
github.com/google/go-cmp/cmp/internal/function
github.com/google/go-cmp/cmp/internal/value
github.com/google/go-cmp/cmp
golang.org/x/xerrors/internal
golang.org/x/xerrors
github.com/google/go-cmp/cmp/cmpopts
crypto/x509
net/textproto
vendor/golang.org/x/net/http/httpguts
vendor/golang.org/x/net/http/httpproxy
github.com/pkg/errors
github.com/beorn7/perks/quantile
github.com/cespare/xxhash
crypto/tls
github.com/golang/protobuf/proto
github.com/golang/protobuf/ptypes/any
github.com/golang/protobuf/ptypes/duration
net/http/httptrace
github.com/golang/protobuf/ptypes/timestamp
github.com/golang/protobuf/ptypes
net/http
github.com/prometheus/client_model/go
github.com/prometheus/client_golang/prometheus/internal
github.com/matttproud/golang_protobuf_extensions/pbutil
github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg
github.com/prometheus/common/model
github.com/prometheus/procfs/internal/fs
github.com/prometheus/procfs/internal/util
github.com/prometheus/procfs
go.opencensus.io/resource
go.opencensus.io/metric/metricdata
go.opencensus.io/tag
go.opencensus.io/stats/internal
go.opencensus.io/stats
go.opencensus.io/internal/tagencoding
go.opencensus.io/metric/metricproducer
go.opencensus.io/stats/view
google.golang.org/grpc/codes
google.golang.org/grpc/grpclog
google.golang.org/grpc/metadata
google.golang.org/grpc/stats
google.golang.org/genproto/googleapis/rpc/status
google.golang.org/grpc/connectivity
google.golang.org/grpc/internal
google.golang.org/grpc/status
contrib.go.opencensus.io/exporter/jaeger
github.com/prometheus/common/expfmt
go.opencensus.io/trace/propagation
go.opencensus.io/plugin/ocgrpc
go.opencensus.io/zpages/internal
github.com/prometheus/client_golang/prometheus
go.opencensus.io/zpages
github.com/prometheus/common/version
github.com/prometheus/client_golang/prometheus/promhttp
go get -v bufio context encoding/json errors expvar fmt github.com/golang/glog github.com/pkg/errors github.com/prometheus/client_golang/prometheus/testutil golang.org/x/sys/unix io io/ioutil math math/rand net net/http net/http/httptest os os/user path path/filepath reflect regexp regexp/syntax runtime sort strconv strings sync syscall testing testing/quick time
github.com/prometheus/client_golang/prometheus/testutil
net/http/httptest
touch .dep-stamp
  Successfully remade target file '.dep-stamp'.
Must remake target 'install_deps'.
Successfully remade target file 'install_deps'.
+ go mod vendor
+ make GO111MODULE=off --debug /out/vm-fuzzer.dict /out/vm-fuzzer_seed_corpus.zip /out/vm-fuzzer
GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Updating goal targets....
 File '/out/vm-fuzzer.dict' does not exist.
   File 'mgen' does not exist.
     File 'print-version' does not exist.
    Must remake target 'print-version'.
go version go1.13.6 linux/amd64
    Successfully remade target file 'print-version'.
     Prerequisite 'print-version' of target '.dep-stamp' does not exist.
    Must remake target '.dep-stamp'.
Install all dependencies, ensuring they're updated
go get -v bufio bytes context contrib.go.opencensus.io/exporter/jaeger crypto/rand encoding/json expvar flag fmt github.com/fsnotify/fsnotify github.com/golang/glog github.com/golang/groupcache/lru github.com/google/go-cmp/cmp github.com/google/go-cmp/cmp/cmpopts github.com/pkg/errors github.com/prometheus/client_golang/prometheus github.com/prometheus/client_golang/prometheus/promhttp github.com/prometheus/common/version go.opencensus.io/trace go.opencensus.io/zpages html/template io io/ioutil math math/big math/rand net net/http net/http/pprof os os/exec os/signal path path/filepath reflect regexp regexp/syntax runtime runtime/debug sort strconv strings sync sync/atomic syscall testing text/tabwriter time unicode unicode/utf8
go get -v bufio context encoding/json errors expvar fmt github.com/golang/glog github.com/pkg/errors github.com/prometheus/client_golang/prometheus/testutil golang.org/x/sys/unix io io/ioutil math math/rand net net/http net/http/httptest os os/user path path/filepath reflect regexp regexp/syntax runtime sort strconv strings sync syscall testing testing/quick time
touch .dep-stamp
    Successfully remade target file '.dep-stamp'.
  Must remake target 'mgen'.
echo "mgen: $(go list -f '{{if not .Standard}}{{.Dir}}{{end}}' $(go list -f '{{ join .Deps "\n" }}' print-version) | sed -e 's@$@/*.go@' | tr "\n" " " )" > .d/mgen.d
can't load package: package print-version: cannot find package "print-version" in any of:
	/root/.go/src/print-version (from $GOROOT)
	/root/go/src/print-version (from $GOPATH)
	/src/print-version
can't load package: package github.com/google/mtail: no Go files in /src/github.com/google/mtail
go build -ldflags "-X main.Branch=master -X main.Version=3f0c4da -X main.Revision=3f0c4daf06c8e2923f61689ffd5f8c0e0d0fa039" -o mgen print-version
can't load package: package print-version: cannot find package "print-version" in any of:
	/root/.go/src/print-version (from $GOROOT)
	/root/go/src/print-version (from $GOPATH)
	/src/print-version
Makefile:124: recipe for target 'mgen' failed
make: *** [mgen] Error 1
Building fuzzers failed.
Failed projects: mtail
TravisBuddy Request Identifier: c3b44500-41ec-11ea-a094-3d03eb93742b

@Dor1s
Copy link
Contributor Author

Dor1s commented Jan 28, 2020

Travis is unhappy because our build scripts are python2 and do not yet conform to the linter expectations. The mtail project's build also appears to be broken, but none of these is related to the changes introduced in the PR. PTAL

oliverchang
oliverchang approved these changes Jan 28, 2020
View reviewed changes
Copy link
Collaborator

@oliverchang oliverchang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

jonathanmetzman
jonathanmetzman approved these changes Jan 29, 2020
View reviewed changes
Copy link
Contributor

@jonathanmetzman jonathanmetzman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Dor1s
Copy link
Contributor Author

Dor1s commented Jan 30, 2020

Added the attribute for the following projects:

projects/go-attestation/project.yaml
projects/go-json-iterator/project.yaml
projects/golang-protobuf/project.yaml
projects/golang/project.yaml
projects/gonids/project.yaml
projects/kubernetes/project.yaml
projects/mtail/project.yaml
projects/syzkaller/project.yaml
projects/wasmtime/project.yaml  -- Rust, all others are Go

merging

@Dor1s Dor1s merged commit 7751ab5 into master Jan 30, 2020
@TravisBuddy
Copy link

Hey @Dor1s,
Something went wrong with the build.

TravisCI finished with status errored, which means the build failed because of something unrelated to the tests, such as a problem with a dependency or the build process itself.

View build log

TravisBuddy Request Identifier: 96aea7d0-43bd-11ea-9d70-2f474a28a725

@TravisBuddy
Copy link

Hey @Dor1s,
Something went wrong with the build.

TravisCI finished with status errored, which means the build failed because of something unrelated to the tests, such as a problem with a dependency or the build process itself.

View build log

TravisBuddy Request Identifier: 080bff80-43bf-11ea-9235-e14405c2c5dd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@oliverchang oliverchang oliverchang approved these changes

@jonathanmetzman jonathanmetzman jonathanmetzman approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Dor1s @TravisBuddy @oliverchang @jonathanmetzman