Update windows.md to run as a windows service (in support of containerd) #1874

jayunit100 · 2021-02-14T21:08:49Z

quick update for antrea, as a windows service

codecov-io · 2021-02-14T21:12:49Z

Codecov Report

❗ No coverage uploaded for pull request base (main@f93242a). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1874   +/-   ##
=======================================
  Coverage        ?   65.00%           
=======================================
  Files           ?      197           
  Lines           ?    17507           
  Branches        ?        0           
=======================================
  Hits            ?    11380           
  Misses          ?     4941           
  Partials        ?     1186

Flag	Coverage Δ
kind-e2e-tests	`56.02% <0.00%> (?)`
unit-tests	`41.61% <0.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

jayunit100 · 2021-02-14T21:13:29Z

@ruicao93 just a first draft of a patch for docs ,

docs/windows.md

ruicao93 · 2021-02-23T01:23:43Z

Fixes: #1889

@jayunit100 Thanks for your PR. Do you need me continue the PR if you have no time to address the comments?

ruicao93

Thanks Zhecheng.

Have you tried the service installation method?
Another thing we need to consider is that the agent config may need to be updated in some cases(e.g. new antrea-controller Pod is deployed). If the case happens, antrea-agent service will cannot start. [Windows] Always update agent configs on startup #1847

docs/windows.md

antoninbas · 2021-02-26T00:54:29Z

docs/windows.md

+
+mv antrea-agent-windows-x86_64.exe c:/k/antrea/bin/antrea-agent.exe
+Import-Module ./helper.psm1
+Install-AntreaAgent -KubernetesVersion "v1.20.2" -KubernetesHome "c:/k" -KubeConfig "C:/etc/kubernetes/kubelet.conf" -AntreaVersion "${TAG}" -AntreaHome "c:/k/antrea"


what's the requirement for the K8s version? And should we update the default K8s version in the existing powershell scripts? Maybe the scripts should use the value returned by https://storage.googleapis.com/kubernetes-release/release/stable.txt by default?

KubeConfig is important here as well, we've prototyped with this being the kubelet.conf but by default the node doesn't have access to everything it needs, is there a way that we can allow the node config to do what it needs to or is that a security problem and can we think of a way around it (avoiding manually creating the kubeconfig)

Thank you @antoninbas and @perithompson . I am investigating these issues. Yes, on my testbed, kubelet.conf is not enough to get secret in Install-AntreaAgent().

Discussed with @ruicao93. As for the K8s version, there's no requirement. So here v1.20.4 is just an example, users should keep it consistent with K8s version on other nodes. I changed it to <KubernetesVersion> and users should fill it up.
As for kubeconfig, it is users' responsibility to provide a valid kubeconfig. Whether using admin.conf or a customed one, it's up to the user. I changed it to <KubeconfigPath> and users should fill it up.

understood, it's just incredibly difficult to automate this at node creation time so I was wondering if there was a way around this to help? Either way, have we got a method of generating this kubeconfig, either on Linux or on Windows that we could add to this document as there is no explanation of where the kubeconfig comes from.

I have mentioned that kubeconfig can be copied from /etc/kubernetes/admin.conf on ControlPlane node. I suppose it is enough for users.

An alternative could be to provide a script to generate all the required kubeconfig files (2 for antrea-agent and 1 for kube-proxy) using the "admin" kubeconfig. This process would need to be done only once per cluster instead of on every Node. We could then use a DaemonSet to distribute and install the generated kubeconfig files to each Windows Node in the cluster. As a new Node joins the cluster, the files would be installed automatically. But maybe that's an overkill...

It seems that currently we recommend running kube-proxy with the "admin" kubeconfig (/etc/kubernetes/admin.conf copied from the control-plane mode), which seems like too many permissions.

I agree it is an overkill. A simple script is enough to distribute kubeconfig to nodes. Then how about this? Admin kubeconfig is used to Install-AntreaAgent because such permission is required to get secret and kubelet one is used to install kube-proxy. I have updated this doc.

In the context of capi, when you create a node, you only have control of actions that happen on the host as an administrator, so in theory I guess you could get it from another host from the cluster but securing that is a nightmare too!

docs/windows.md

ruicao93 · 2021-03-20T08:39:26Z

docs/windows.md

+New-KubeProxyServiceInterface
+
+mkdir "${KubernetesHome}/antrea/logs"
+nssm install kube-proxy "${KubernetesHome}/kube-proxy.exe" "--proxy-mode=userspace --kubeconfig=$KubeletKubeConfig --log-dir=c:/var/log/kube-proxy --logtostderr=false --alsologtostderr"


What about use config file like KubeProxyKubeConfig instead of KubeletKubeConfig. It means let user specify a kubeconfig file used by kube-proxy. The kubelet configfile contains too many permissions.

For example, user could generate the KubeProxyKubeConfig use the kube-proxy service account.

@lzhecheng any thoughts on that, is there any upstream reference on running kube-proxy as a service?

@antoninbas this doc uses sc.exe but nssm is similar here.
https://kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#troubleshooting

KubeletKubeConfig is changed to KubeProxyKubeconfigPath.

docs/windows.md

antoninbas · 2021-04-07T22:20:13Z

docs/windows.md

+New-KubeProxyServiceInterface
+
+mkdir "${KubernetesHome}/antrea/logs"
+nssm install kube-proxy "${KubernetesHome}/kube-proxy.exe" "--proxy-mode=userspace --kubeconfig=$KubeletKubeConfig --log-dir=c:/var/log/kube-proxy --logtostderr=false --alsologtostderr"


@lzhecheng any thoughts on that, is there any upstream reference on running kube-proxy as a service?

docs/windows.md

jianjuns · 2021-04-08T01:01:21Z

docs/windows.md

+
+### Installation via wins (docker based runtimes)
+
+Installing Antrea as a wins service gives you a lot of flexibility to manage it as a Pod, but currently this only works


Installing Antrea using wins..

I just feel "as wins service" might confuse people - thinking it means Windows service.

I think @jayunit100 is referring to https://github.com/rancher/wins. Is it correct?

I know. But not all readers understand what is a "wins service", so I suggest to rephrase the sentence as: "Installing Antrea using wins gives.."

Got it. I updated the sentence and added a link to the repo.

Co-authored-by: Zhecheng Li <[email protected]>

antoninbas

LGTM. I would like to include this in the v1.0 release so these changes are part of the documentation for this release. We can refine them as needed in the future.

antoninbas · 2021-04-08T18:26:30Z

/skip-all

antoninbas · 2021-04-08T19:03:24Z

The "Verify docs and spelling" is failing because of an issue that has been fixed in "main". I will use admin privileges to merge.

antoninbas · 2021-04-08T19:06:26Z

Thanks again @lzhecheng for picking up this PR!

Update windows.md

7bb0d60

vmwclabot added the cla-not-required label Feb 14, 2021

jayunit100 changed the title ~~Update windows.md~~ Update windows.md to run as a windows service (in support of containerd) Feb 14, 2021

antoninbas requested a review from ruicao93 February 16, 2021 20:20

antoninbas reviewed Feb 16, 2021

View reviewed changes

ruicao93 reviewed Feb 18, 2021

View reviewed changes

docs/windows.md Outdated Show resolved Hide resolved

ruicao93 linked an issue Feb 24, 2021 that may be closed by this pull request

Antrea running as a service on windows #1889

Closed

lzhecheng force-pushed the patch-14 branch from 5fdb903 to 30a9608 Compare February 25, 2021 07:25

lzhecheng requested review from ruicao93 and antoninbas February 25, 2021 07:30

ruicao93 reviewed Feb 25, 2021

View reviewed changes

docs/windows.md Outdated Show resolved Hide resolved

docs/windows.md Outdated Show resolved Hide resolved

docs/windows.md Outdated Show resolved Hide resolved

antoninbas reviewed Feb 26, 2021

View reviewed changes

lzhecheng force-pushed the patch-14 branch 4 times, most recently from 13830c1 to 362abd4 Compare March 9, 2021 03:28

perithompson reviewed Mar 9, 2021

View reviewed changes

docs/windows.md Outdated Show resolved Hide resolved

lzhecheng force-pushed the patch-14 branch 2 times, most recently from fd52d61 to 803689e Compare March 11, 2021 02:54

ruicao93 reviewed Mar 20, 2021

View reviewed changes

ruicao93 mentioned this pull request Mar 20, 2021

Create windows-containerd.md #1968

Closed

antoninbas added this to the Antrea v1.0 release milestone Mar 30, 2021

lzhecheng reviewed Mar 31, 2021

View reviewed changes

docs/windows.md Outdated Show resolved Hide resolved

lzhecheng reviewed Mar 31, 2021

View reviewed changes

docs/windows.md Outdated Show resolved Hide resolved

antoninbas reviewed Apr 7, 2021

View reviewed changes

jianjuns reviewed Apr 8, 2021

View reviewed changes

lzhecheng force-pushed the patch-14 branch from e0f0b0f to fa06d92 Compare April 8, 2021 01:59

lzhecheng force-pushed the patch-14 branch from fa06d92 to fcb7f28 Compare April 8, 2021 02:17

Addressed comments

310efd2

Co-authored-by: Zhecheng Li <[email protected]>

lzhecheng force-pushed the patch-14 branch from fcb7f28 to 310efd2 Compare April 8, 2021 03:38

antoninbas approved these changes Apr 8, 2021

View reviewed changes

jianjuns approved these changes Apr 8, 2021

View reviewed changes

antoninbas merged commit 3be5642 into antrea-io:main Apr 8, 2021

lzhecheng mentioned this pull request Apr 13, 2021

Containerd Windows nodes RBAC #2066

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update windows.md to run as a windows service (in support of containerd) #1874

Update windows.md to run as a windows service (in support of containerd) #1874

jayunit100 commented Feb 14, 2021

codecov-io commented Feb 14, 2021 •

edited

Loading

jayunit100 commented Feb 14, 2021

ruicao93 commented Feb 23, 2021

ruicao93 left a comment •

edited

Loading

antoninbas Feb 26, 2021

perithompson Feb 26, 2021

lzhecheng Mar 3, 2021

lzhecheng Mar 9, 2021 •

edited

Loading

perithompson Mar 9, 2021 •

edited

Loading

lzhecheng Mar 10, 2021

antoninbas Mar 10, 2021

lzhecheng Mar 11, 2021

perithompson Mar 11, 2021

ruicao93 Mar 20, 2021

antoninbas Apr 7, 2021

lzhecheng Apr 8, 2021

lzhecheng Apr 8, 2021

antoninbas Apr 7, 2021

jianjuns Apr 8, 2021

jianjuns Apr 8, 2021

lzhecheng Apr 8, 2021

jianjuns Apr 8, 2021

lzhecheng Apr 8, 2021

antoninbas left a comment

antoninbas commented Apr 8, 2021

antoninbas commented Apr 8, 2021

antoninbas commented Apr 8, 2021


		### Installation via wins (docker based runtimes)

		Installing Antrea as a wins service gives you a lot of flexibility to manage it as a Pod, but currently this only works

Update windows.md to run as a windows service (in support of containerd) #1874

Update windows.md to run as a windows service (in support of containerd) #1874

Conversation

jayunit100 commented Feb 14, 2021

codecov-io commented Feb 14, 2021 • edited Loading

Codecov Report

jayunit100 commented Feb 14, 2021

ruicao93 commented Feb 23, 2021

ruicao93 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lzhecheng Mar 9, 2021 • edited Loading

Choose a reason for hiding this comment

perithompson Mar 9, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antoninbas left a comment

Choose a reason for hiding this comment

antoninbas commented Apr 8, 2021

antoninbas commented Apr 8, 2021

antoninbas commented Apr 8, 2021

codecov-io commented Feb 14, 2021 •

edited

Loading

ruicao93 left a comment •

edited

Loading

lzhecheng Mar 9, 2021 •

edited

Loading

perithompson Mar 9, 2021 •

edited

Loading