Serve openapi spec

[johanneswuerbach]

Use kubernetes-sigs/custom-metrics-apiserver#65 to serve an openapi spec.

Based on #316

Fixes #257
Fixes #254

[lilic]

Thanks for the PR, is there anything else left to do there mainly asking as its draft PR? I will give it a try if this fixes the bug for us as well, thanks!

[lilic]

I get the following errors when trying to build:

➜  k8s-prometheus-adapter git:(openapi) ✗ make docker-build
docker run -it -v /Users/lili/go/src/github.com/DirectXMan12/k8s-prometheus-adapter/_output:/build -v /Users/lili/go/src/github.com/DirectXMan12/k8s-prometheus-adapter:/go/src/github.com/directxman12/k8s-prometheus-adapter -e GOARCH=amd64 golang:1.13 /bin/bash -c "\
		CGO_ENABLED=0 go build -tags netgo -o /build/amd64/adapter github.com/directxman12/k8s-prometheus-adapter/cmd/adapter"
# github.com/directxman12/k8s-prometheus-adapter/cmd/adapter
src/github.com/directxman12/k8s-prometheus-adapter/cmd/adapter/adapter.go:249:5: cmd.OpenAPIConfig undefined (type *PrometheusAdapter has no field or method OpenAPIConfig)
src/github.com/directxman12/k8s-prometheus-adapter/cmd/adapter/adapter.go:250:5: cmd.OpenAPIConfig undefined (type *PrometheusAdapter has no field or method OpenAPIConfig)
src/github.com/directxman12/k8s-prometheus-adapter/cmd/adapter/adapter.go:251:5: cmd.OpenAPIConfig undefined (type *PrometheusAdapter has no field or method OpenAPIConfig)
make: *** [docker-build] Error 2

Do you mind just detailing the steps you used to build it, thanks!

[johanneswuerbach]

Sorry, you need to change the custom-metrics-apiserver to my fork to include kubernetes-sigs/custom-metrics-apiserver#65.

I added

replace (
  // forced by the inclusion of sigs.k8s.io/metrics-server's use of this in their go.mod
  k8s.io/kubernetes/pkg/kubelet/apis/stats/v1alpha1 => ./localvendor/k8s.io/kubernetes/pkg/kubelet/apis/stats/v1alpha1
  github.com/kubernetes-incubator/custom-metrics-apiserver v0.0.0-20200618121405-54026617ec44 => github.com/johanneswuerbach/custom-metrics-apiserver openapi
)

and then it should just work ™️ . No idea why this is actually gone.

[lilic]

Thanks that worked compiling it, but I still get apiserver errors in the log, maybe I have a wrong setup can you maybe share yours if those do not happen to you? Thanks!

W0929 10:23:15.312722 1 handler_proxy.go:102] no RequestInfo found in the context
E0929 10:23:15.313665 1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0929 10:23:15.314782 1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
W0929 10:23:30.928722 1 handler_proxy.go:102] no RequestInfo found in the context
E0929 10:23:30.928802 1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0929 10:23:30.929206 1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0929 10:23:40.994410 1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.97.168.59:443/apis/metrics.k8s.io/v1beta1: bad status from https://10.97.168.59:443/apis/metrics.k8s.io/v1beta1: 403
E0929 10:23:40.995736 1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.97.168.59:443/apis/metrics.k8s.io/v1beta1: bad status from https://10.97.168.59:443/apis/metrics.k8s.io/v1beta1: 403
E0929 10:23:41.002959 1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.97.168.59:443/apis/metrics.k8s.io/v1beta1: bad status from https://10.97.168.59:443/apis/metrics.k8s.io/v1beta1: 403
E0929 10:23:41.026396 1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.97.168.59:443/apis/metrics.k8s.io/v1beta1: bad status from https://10.97.168.59:443/apis/metrics.k8s.io/v1beta1: 403
E0929 10:23:42.004090 1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exist
I0929 10:23:42.004106 1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0929 10:24:42.010031 1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exist
I0929 10:24:42.010081 1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.

[johanneswuerbach]

Sorry, I was mainly testing with the test-adapter here kubernetes-sigs/custom-metrics-apiserver#65 (comment), but will try to find some time to test this adapter directly.

[lilic]

@johanneswuerbach no worries, feel free to reach out if you need any help or me to test anything! Thanks again!

[johanneswuerbach]

Hey @lilic I pushed a version pointing directly to my fork and it is working for me (no more errors, and output with kubectl get --raw /openapi/v2 | jq . | grep 'custom.metrics')

[lilic]

@johanneswuerbach yay, thanks for the ping, will test it out today!! 🎉

[lilic]

Tested, and no more errors in apiserver logs!
But I cannot get "kubectl get --raw /openapi/v2 | jq . | grep 'custom.metrics'" to return. 🤷‍♀️ But there seem to be

Once is merged kubernetes-sigs/custom-metrics-apiserver#65 we can make this nondraft, correct or are there any other blockers? Thanks again!

[johanneswuerbach]

No, we can either merge this PR directly or first #316 and then this PR once kubernetes-sigs/custom-metrics-apiserver#65 is merged.

[s-urbaniak]

@johanneswuerbach if you feel this can be merged, please let me know and thank you for the contribution! Please squash commits in logical packets please if possible. If possible please make separate commits for:

• pure vendoring
• pure code changes (possibly two separate commits for the bump changes and a separate one for enabling OpenAPI)

[johanneswuerbach]

@s-urbaniak, sure. Should #316 be merged first or do you prefer to close 316 and make all changes only in this PR?

[s-urbaniak]

@johanneswuerbach I think doing it in one PR does the trick and reduces cross-PR coordination. So closing #316 and doing everything here is fine.

Thank you so much!!!! 🎉

[johanneswuerbach]

@s-urbaniak rebased, does the split make sense or should I split the commits differently?

[s-urbaniak]

@johanneswuerbach this looks perfect, I am happy to merge, but I am just asking one last thing if you don't mind :-) We are in the process of moving this project to kubernetes-sigs (see kubernetes/org#2182). So it would be great if you could sign the CNCF CLA (see kubernetes/org#2182 (comment)) so we can continue with a smooth transition :-)

Thank you! 🎉

[johanneswuerbach]

I've been contributing to kubernetes before, so the CLA should already be signed kubernetes/enhancements#2022

Or is there something different here?

[s-urbaniak]

Ah, no this looks perfect then, merging, and thank you so much! 🎉

[renan]

@s-urbaniak Would you please release a new version as well? 🙏

[s-urbaniak]

@renan yes, I just released 0.8.0 yesterday https://github.com/DirectXMan12/k8s-prometheus-adapter/releases/tag/v0.8.0 :-)

[shreben]

Hi @s-urbaniak

could you please take a look at the following issue of the release you've mentioned
#329

Thank you in advance.