Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

403 on new install #118

Open
bn-cforkner opened this issue Apr 24, 2024 · 1 comment
Open

403 on new install #118

bn-cforkner opened this issue Apr 24, 2024 · 1 comment

Comments

@bn-cforkner
Copy link

bn-cforkner commented Apr 24, 2024 •
edited
Loading

Did a fresh helm install on EKS

=========================================================================

  CHT Containers Collector Environment

  CHT_API_TOKEN: ****

  CHT_CLUSTER_NAME: dev-us-east-2

  JAVA_OPTS:
    -XX:+ExitOnOutOfMemoryError -Xms10M -Xmx891M

  CHT_INTERVAL: 900

=========================================================================

2024-04-24T22:41:32.215Z [main] WARN FilenoUtil : Native subprocess control requires open access to the JDK IO subsystem
Pass '--add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED' to enable.
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85: warning: parentheses after method name is interpreted as an argument list, not a decomposed argument
CHT Containers Collector : version 5f1b5a2 starting
D, [2024-04-24T22:41:46.848458 #10] DEBUG -- : Ensuring cache directory is present: /tmp/cache
D, [2024-04-24T22:41:46.862460 #10] DEBUG -- : Ensuring metrics cache directory is present: /tmp/metrics_cache
D, [2024-04-24T22:41:46.935558 #10] DEBUG -- : Fetching collector agent version
D, [2024-04-24T22:41:46.943248 #10] DEBUG -- : Kubernetes collector agent version found : 1398
D, [2024-04-24T22:41:47.028912 #10] DEBUG -- : loaded K8S config from  with master @ https://172.22.0.1:443/ with ca certificate /var/run/secrets/kubernetes.io/serviceaccount/ca.crt with client_cert_file  with client key file  with trust_certs false with trust store file  with proxy username 
D, [2024-04-24T22:41:47.635460 #10] DEBUG -- : Fetching Cluster UID
D, [2024-04-24T22:41:47.640977 #10] DEBUG -- : Connecting to URL: https://172.22.0.1:443/api/v1/namespaces/kube-system
E, [2024-04-24T22:41:48.137155 #10] ERROR -- : [Java::JavaIo::IOException]: Server returned HTTP response code: 403 for URL: https://172.22.0.1:443/api/v1/namespaces/kube-system
E, [2024-04-24T22:41:48.138684 #10] ERROR -- : Backtrace:
	jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(jdk/internal/reflect/NativeConstructorAccessorImpl.java:77)
	jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(jdk/internal/reflect/DelegatingConstructorAccessorImpl.java:45)
	java.lang.reflect.Constructor.newInstanceWithCaller(java/lang/reflect/Constructor.java:499)
	java.lang.reflect.Constructor.newInstance(java/lang/reflect/Constructor.java:480)
	sun.net.www.protocol.http.HttpURLConnection$10.run(sun/net/www/protocol/http/HttpURLConnection.java:2057)
	sun.net.www.protocol.http.HttpURLConnection$10.run(sun/net/www/protocol/http/HttpURLConnection.java:2052)
	java.security.AccessController.doPrivileged(java/security/AccessController.java:569)
	sun.net.www.protocol.http.HttpURLConnection.getChainedException(sun/net/www/protocol/http/HttpURLConnection.java:2051)
	sun.net.www.protocol.http.HttpURLConnection.getInputStream0(sun/net/www/protocol/http/HttpURLConnection.java:1609)
	sun.net.www.protocol.http.HttpURLConnection.getInputStream(sun/net/www/protocol/http/HttpURLConnection.java:1589)
	sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(sun/net/www/protocol/https/HttpsURLConnectionImpl.java:224)
	jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	jdk.internal.reflect.NativeMethodAccessorImpl.invoke(jdk/internal/reflect/NativeMethodAccessorImpl.java:77)
	jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)
	java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:568)
	org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:329)
	org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:193)
	RUBY.get_input_stream(uri:classloader:/mesos-collector/lib/common/fetcher.rb:37)
	RUBY.fetch_uid(uri:classloader:/mesos-collector/lib/common/fetcher.rb:50)
	RUBY.do_fetch_cluster_uid(uri:classloader:/mesos-collector/lib/common/collection_cycle.rb:104)
	RUBY.do_upload(uri:classloader:/mesos-collector/lib/common/collection_cycle.rb:24)
	RUBY.upload_k8s_state_v4(uri:classloader:/mesos-collector/bin/container-collector.rb:88)
	org.jruby.RubyClass.finvokeWithRefinements(org/jruby/RubyClass.java:522)
	org.jruby.RubyBasicObject.send(org/jruby/RubyBasicObject.java:1687)
	org.jruby.RubyKernel.send(org/jruby/RubyKernel.java:2315)
	org.jruby.RubyKernel$INVOKER$s$send.call(org/jruby/RubyKernel$INVOKER$s$send.gen)
	RUBY.delegate_runner_methods(uri:classloader:/gems/boson-1.3.0/lib/boson/runner_library.rb:12)
	org.jruby.RubyProc.call(org/jruby/RubyProc.java:378)
	org.jruby.RubyMethod.call(org/jruby/RubyMethod.java:144)
	org.jruby.RubyMethod$INVOKER$i$call.call(org/jruby/RubyMethod$INVOKER$i$call.gen)
	RUBY.redefine_command_block(uri:classloader:/gems/boson-1.3.0/lib/boson/scientist.rb:87)
	org.jruby.RubyProc.call(org/jruby/RubyProc.java:333)
	org.jruby.RubyProc$INVOKER$i$call.call(org/jruby/RubyProc$INVOKER$i$call.gen)
	RUBY.call_original_command(uri:classloader:/gems/boson-1.3.0/lib/boson/scientist.rb:123)
	RUBY.during_analyze(uri:classloader:/gems/boson-1.3.0/lib/boson/scientist.rb:115)
	RUBY.analyze(uri:classloader:/gems/boson-1.3.0/lib/boson/scientist.rb:107)
	RUBY.redefine_command_block(uri:classloader:/gems/boson-1.3.0/lib/boson/scientist.rb:86)
	org.jruby.RubyProc.call(org/jruby/RubyProc.java:378)
	org.jruby.RubyClass.finvokeWithRefinements(org/jruby/RubyClass.java:522)
	org.jruby.RubyBasicObject.send(org/jruby/RubyBasicObject.java:1687)
	org.jruby.RubyKernel.send(org/jruby/RubyKernel.java:2315)
	org.jruby.RubyKernel$INVOKER$s$send.call(org/jruby/RubyKernel$INVOKER$s$send.gen)
	RUBY.full_invoke(uri:classloader:/gems/boson-1.3.0/lib/boson.rb:79)
	RUBY.execute_command(uri:classloader:/gems/boson-1.3.0/lib/boson/bare_runner.rb:42)
	RUBY.execute_command(uri:classloader:/gems/boson-1.3.0/lib/boson/runner.rb:35)
	RUBY.execute(uri:classloader:/gems/boson-1.3.0/lib/boson/runner.rb:31)
	RUBY.start(uri:classloader:/gems/boson-1.3.0/lib/boson/runner.rb:26)
	RUBY.<main>(uri:classloader:/mesos-collector/bin/container-collector.rb:120)
	org.jruby.Ruby.runInterpreter(org/jruby/Ruby.java:1290)
	org.jruby.Ruby.loadFile(org/jruby/Ruby.java:2964)
	org.jruby.RubyKernel.loadCommon(org/jruby/RubyKernel.java:1121)
	org.jruby.RubyKernel.load(org/jruby/RubyKernel.java:1091)
	org.jruby.RubyKernel$INVOKER$s$load.call(org/jruby/RubyKernel$INVOKER$s$load.gen)
	RUBY.<main>(uri:classloader:/META-INF/main.rb:1)
	org.jruby.Ruby.runInterpreter(org/jruby/Ruby.java:1290)
	org.jruby.Ruby.loadFile(org/jruby/Ruby.java:2964)
	org.jruby.RubyKernel.requireCommon(org/jruby/RubyKernel.java:1064)
	org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1057)
	org.jruby.RubyKernel$INVOKER$s$1$0$require.call(org/jruby/RubyKernel$INVOKER$s$1$0$require.gen)
	RUBY.require(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85)
	RUBY.<main>(<script>:3)
	org.jruby.Ruby.runInterpreter(org/jruby/Ruby.java:1312)
	org.jruby.Ruby.runInterpreter(org/jruby/Ruby.java:1316)
	org.jruby.embed.internal.EmbedEvalUnitImpl.run(org/jruby/embed/internal/EmbedEvalUnitImpl.java:119)
	org.jruby.embed.ScriptingContainer.runUnit(org/jruby/embed/ScriptingContainer.java:1296)
	org.jruby.embed.ScriptingContainer.runScriptlet(org/jruby/embed/ScriptingContainer.java:1289)
	jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	jdk.internal.reflect.NativeMethodAccessorImpl.invoke(jdk/internal/reflect/NativeMethodAccessorImpl.java:77)
	jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)
	java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:568)
	JarMain.invokeMethod(JarMain.java:263)
	JarMain.invokeMethod(JarMain.java:256)
	JarMain.launchJRuby(JarMain.java:141)
	JarMain.start(JarMain.java:158)
	JarMain.doStart(JarMain.java:233)
	JarMain.main(JarMain.java:227)
E, [2024-04-24T22:41:48.139484 #10] ERROR -- : Could not open an input stream to cluster's Kubernetes API: Server returned HTTP response code: 403 for URL: https://172.22.0.1:443/api/v1/namespaces/kube-system

I've validated that the service account and bindings were made as expected in the chart
@kscherme
Copy link
Contributor

kscherme commented Jul 3, 2024

@bn-cforkner Hi! From the line Kubernetes collector agent version found : 1398, it looks like you may be using an old version of the helm chart. Can you double check that you are using helm chart https://github.com/CloudHealth/helm/releases/tag/cloudhealth-collector-4.6.2?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kscherme @bn-cforkner