Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SECURITY: Kubeclient::Config: return ssl_options[:verify_ssl] correctly #557

Merged
merged 20 commits into from
Mar 23, 2022

Conversation

cben
Copy link
Collaborator

@cben cben commented Mar 23, 2022

Fixing #554 and #555 on master branch.

(also merges tests & changelog added in #552)

cben added 14 commits March 14, 2022 14:37
…verify

[v4.y] Test VERIFY_PEER / VERIFY_NONE work against real cluster
Test sandwitches the real CA cert between two unrelated CA certs
(another-ca1.pem, another-ca2.pem, simply copied from two runs of
update_certs_k0s.rb).
No test for root+intermediate scenario.

Fails before the backport of ManageIQ#461:

KubeclientConfigTest#test_concatenated_ca [/home/beni/kubeclient/test/test_config.rb:196]:
Expected false to be truthy.

(some experimenting with order suggests only first cert is honored.)
Passes with the fix.
…-add-file

Load cluster ca certificates using OpenSSL::X509::Store#add_file

(cherry picked from commit 53408c1)
- Helps confirm suspected OS-specific failures.
- Normally when one build fails, most other builds already started and
  some almost complete `bundle install` / started tests.
  So it's not a big "waste" expensive to let them finish, arguably it's
  actually a waste to abort them! (sunken cost fallacy? :-)
- In case rubocop complains, while I do consider it a merge blocker,
  it's better contributor (and maintainer) experience to also see test results.
…-file

[v4.y] Load cluster ca certificates using OpenSSL::X509::Store#add_file
- VULNERABILITY FIX: Previously, whenever kubeconfig did not define
  custom CA (normal situation for production clusters with public domain
  and certificate!), `Config` was returning hard-coded `VERIFY_NONE` :-(

  Assuming you passed those ssl_options to Kubeclient::Client,
  this means that instead of checking server's certificate against
  your system CA store, it would accept ANY certificate, allowing easy
  man-in-the middle attacks.

  This is especially dangerous with user/password or token credentials
  because MITM attacker could simply steal those credentials to the
  cluster and do anything you could do on the cluster.

- Bug fix: kubeconfig `insecure-skip-tls-verify` field was ignored.
  When kubeconfig did define custom CA, `Config` was returning hard-coded
  `VERIFY_PEER`.

  Now we honor it, return `VERIFY_NONE` iff kubeconfig has explicit
  `insecure-skip-tls-verify: true`, otherwise `VERIFY_PEER`.

These don't affect code that supplies `Client` parameters directly,
only code that uses `Config`.

(To ease back-porting, this commit is rebased directly on the 6-year-old
PR that introduced Kubeclient::Config - this was broken from day 1!
ManageIQ#127
Tests come in separate commits based on later points.)
…verify_ssl]

- Removed `insecure-skip-tls-verify: true` from most test configs
  (that was one of the reasons the bug went unnoticed, VERIFY_NONE
  was what the unit tests expected.)

- Added new kubeconfig files + `Config` unit tests covering:
  - custom CA, omitted `insecure-skip-tls-verify`
  - custom CA, `insecure-skip-tls-verify: false`
  - custom CA, `insecure-skip-tls-verify: true`
  - no custom CA, omitted `insecure-skip-tls-verify`
  - no custom CA, `insecure-skip-tls-verify: false`
  - no custom CA, `insecure-skip-tls-verify: true`
…rify

[v4.y] SECURITY: Kubeclient::Config: return ssl_options[:verify_ssl] correctly
@cben cben force-pushed the security-config-ssl_verify branch from 5c20fe2 to 144894c Compare March 23, 2022 11:24
@cben
Copy link
Collaborator Author

cben commented Mar 23, 2022

cc @grosser @agrare @jperville @lnacshonredhat

The git history has become hard to read here (due to my attempt to keep it "clean" in the "prefer merges to rebases" sense, LOL)...
But this is a straight forward merge of #556 into master, only conflicts were CHANGELOG and github actions.yml (over continue-on-errorb1824ed).

@cben
Copy link
Collaborator Author

cben commented Mar 23, 2022

🔴 truffleruby-head on ubuntu always times out on real-cluster tests, I guess watch #finish doesn't work there? Not new, not merge blocker.

🔴 this time ruby 2.6 on ubuntu also timed out. It did pass real-cluster tests ✔️, just was too slow. Perhaps I should increase 3min timeout 🤔
Re-running 2.6... => passed in 40sec

Tiny followup to ManageIQ#556.
Sorry for noise, had these locally but forgot to push before merging.
If I start backporting, CHANGELOG.md on master branch might not always be updated
with all backports (it SHOULD, but it will require separate merges to master).
So I prefer pointing to the vulnerability issue as the "source of truth".
Also, security impact will be better discussed on the issue.
@cben cben force-pushed the security-config-ssl_verify branch from 144894c to 55709fa Compare March 23, 2022 12:34
@cben cben merged commit 9ca1153 into ManageIQ:master Mar 23, 2022
@@ -10,8 +10,8 @@ on:
- '**'
jobs:
build:
continue-on-error: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is wrong and will result in commits marked as passing when they failed CI

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty certain the overall status is still failed 🔴 (as on all recent PRs where truffleruby fails), its just that individual runs continue till the end.

@@ -10,6 +10,12 @@ The client supports GET, POST, PUT, DELETE on all the entities available in kube
The client currently supports Kubernetes REST api version v1.
To learn more about groups and versions in kubernetes refer to [k8s docs](https://kubernetes.io/docs/api/)

## VULNERABILITY❗

If you use `Kubeclient::Config`, all gem versions released before 2022 could return incorrect `ssl_options[:verify_ssl]`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should say before va.b.c ?
... ideally also change heading to ## VULNERABILITY in <va.b.c so I can quickly ignore the heading

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see #560

Copy link
Contributor

@grosser grosser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx for the fix and great summary :)

@cben cben added the bug label Mar 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants