Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(platform): support create superedge edge cluster #1864

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

feat(platform): support create superedge edge cluster #1864

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
66b6556
fix(platform): add deploy superedge apps
attlee-wang Apr 11, 2022
4d61a58
fix(platform): add addon superedge apps
attlee-wang Apr 14, 2022
6a59b42
fix(platform): fix more k8s version conflict
attlee-wang Apr 15, 2022
a8009b6
fix(platform): create superedge cluster success
attlee-wang Apr 15, 2022
4652618
fix(platform): delete rubbish file
attlee-wang Apr 15, 2022
dc4b003
fix(platform): prepare superedge edge cluster
attlee-wang Apr 19, 2022
607bbfe
fix(platform): prepare superedge insecure registries
attlee-wang Apr 20, 2022
2ab2f57
fix(platform): add addon superedge docs
attlee-wang Apr 24, 2022
faa9010
fix(platform): add deploy edge flannel
attlee-wang Apr 24, 2022
538fd90
feat(platform): support create superedge edge cluster
attlee-wang Apr 24, 2022
3b70081
feat(platform): fix ci ete ui faild
attlee-wang Apr 24, 2022
f5c3cce
feat(platform): fix edge info config alreay
attlee-wang Apr 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 103 additions & 11 deletions docs/design-proposals/edge-cluster.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 支持TKE创建边缘集群
# 支持TKE创建SuperEdge边缘集群

**Author**: [attlee wang](https://github.com/attlee-wang)

Expand Down Expand Up @@ -63,25 +63,117 @@
p.EnsurePreClusterInstallHook,
p.EnsurePreInstallHook,
...
// 准备创建SuperEdge边缘节点的逻辑
EnsurePrepareEgdeCluster
// Add-on SuperEdge 组件的逻辑
EnsureApplyEdgeApps
...
p.EnsurePostInstallHook,
p.EnsurePostClusterInstallHook,

// Addon SuperEdge 组件的逻辑
....
// 准备添加边缘节点的逻辑
},
```

## 4. Plan
## 4. 使用Demo

### 4.1 创建边缘集群

创建SuperEdge的边缘集群的demo superedge_edge.yaml具体如下：

```yaml
apiVersion: platform.tkestack.io/v1
kind: Cluster
metadata:
name: attlee-superedge
spec:
displayName: attlee-superedge-demo
etcd:
local:
dataDir: ""
serverCertSANs:
- etcd
- etcd.kube-system
features:
containerRuntime: docker
machines:
- ip: 10.0.200.98
password: PasswordBase64
port: 22
username: root
networkDevice: eth0
clusterCIDR: 192.168.0.0/16
properties:
maxClusterServiceNum: 256
maxNodePodNum: 256
publicAlternativeNames:
- 106.52.199.103
tenantID: default
type: Edge ## 集群类型，标识创建的是SuperEdge的边缘集群
version: 1.21.4-tke.1
```

更多参数的详细解析请参考TKEStack的官方文档，用`kubectl apply -f superedge_edge.yaml ` 提交集群yaml，集群处于`Running`表示创建SuperEdge 边缘集群成功，

```powershell
[root@attlee]# kubectl apply -f superedge_edge.yaml ## 创建superedge edge边缘集群
cluster.platform.tkestack.io/cls-tptvzrdk created

[root@attlee]# kubectl get cluster ## 查看边缘集群的状态
NAME TYPE VERSION STATUS AGE
cls-tptvzrdk Edge 1.21.4-tke.1 Running 8m ## 处于Running表示创建边缘集群成功
```

失败可`kubectl get cluster cls-tptvzrdk -o yaml `查看边缘集群创建失败细节。

### 4.2 添加边缘节点

**<1>.下载静态安装包**

至[SuperEdge](https://github.com/superedge/superedge)社区在需要加入的节点上下载自己需要的K8s 静态安装包：

> 注意修改"arch=amd64"参数，目前支持[amd64, arm64], kubernetes版本可选[1.20.6, 1.18.2], 其他参数不变

```powershell
arch=amd64 version=v0.7.0 kubernetesVersion=1.20.6 && rm -rf edgeadm-linux-* && wget https://superedge-1253687700.cos.ap-guangzhou.myqcloud.com/$version/$arch/edgeadm-linux-$arch-$version-k8s-$kubernetesVersion.tgz && tar -xzvf edgeadm-linux-* && cd edgeadm-linux-$arch-$version-k8s-$kubernetesVersion && ./edgeadm
```

**<2>. Join边缘节点**

在自己新创建的SuperEdge边缘集群的Master执行如下命令获取加入边缘节点的Token：

```powershell
[root@attlee]# kubeadm token create --print-join-command
kubeadm join 127.0.0.1:6443 --token n6hdw6.jbr9778iqoebekyv --discovery-token-ca-cert-hash sha256:b1a00ff33fba4f9a6e63ebbee777120bfbe252a27eeb86d78eaa44799c7a9415
```

> - kubeadm命令master默认已安装；
> - 127.0.0.1:6443 为访问边缘集群kube-api的地址，加入边缘节点可换成`<Master节点外网IP/Master节点内网IP/域名>`，端口、token、discovery-token-ca-cert-hash sha256 不变；

执行如下命令加入边缘节点：

```powershell
[root@attlee]# ./edgeadm join <Master节点外网IP/Master节点内网IP/域名>:Port --token xxxx --discovery-token-ca-cert-hash sha256:xxxxxxxxxx --install-pkg-path <edgeadm kube-*静态安装包路径> --enable-edge=true
```

> 注意通过 `--install-pkg-path` 带edgeadm join的kube-*静态安装包径路

比如：

```powershell
[root@attlee]# ./edgeadm join 106.52.199.103:6443 --token n6hdw6.jbr9778iqoebekyv --discovery-token-ca-cert-hash sha256:b1a00ff33fba4f9a6e63ebbee777120bfbe252a27eeb86d78eaa44799c7a9415 --install-pkg-path ./kube-linux-amd64-v1.20.6.tar.gz
```

更好资料可产看[Addon SuperEdge 让原生Kuberntes集群具备边缘能力](https://github.com/superedge/superedge/blob/main/docs/installation/addon_superedge_CN.md)。

## 5. Plan

| 时间 | 关键节点 | 相关人员 | 进度 |
| :--------: | ---------------------------------------------------- | :----------: | :----: |
| 2022-01-27 | 输出Proposals, review 方案 | @attlee-wang | 已完成 |
| 2022-02-11 | 输出提交代码框架，分出steps，创建tasks | @attlee-wang | |
| 2022-02-18 | 边缘独立集群能够创建出来 | | |
| 2022-02-25 | 边缘节点能够添加 | | |
| 2022-03-04 | 能够把SuperEdge 打入TKEStack的Releases包 | | |
| 2022-03-11 | 提供边缘集群的部署使用方式，补充入TKEStack的使用文档 | | |
| 2022-02-11 | 输出提交代码框架，分出steps，创建tasks | @attlee-wang | 已完成 |
| 2022-02-18 | 边缘独立集群能够创建出来 | @attlee-wang | 已完成 |
| 2022-02-25 | 边缘节点能够添加 | @attlee-wang | 已完成 |
| | 能够把SuperEdge 打入TKEStack的Releases包 | | |
| | 提供边缘集群的部署使用方式，补充入TKEStack的使用文档 | | |
| | 添加必要的单元测试和e2e测试 | | |
| | 支持TKEStack 创建边缘集群页面 | | |

39 changes: 33 additions & 6 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,35 @@ replace (
github.com/NetEase-Object-Storage/nos-golang-sdk => github.com/karuppiah7890/nos-golang-sdk v0.0.0-20191116042345-0792ba35abcc
github.com/chartmuseum/storage => github.com/leoryu/chartmuseum-storage v0.11.1-0.20211104032734-9da39e8f5170
github.com/deislabs/oras => github.com/deislabs/oras v0.8.0
github.com/superedge/superedge => github.com/attlee-wang/superedge v0.8.13
google.golang.org/grpc => google.golang.org/grpc v1.38.0
k8s.io/api => k8s.io/api v0.22.3
k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.22.3
k8s.io/apimachinery => k8s.io/apimachinery v0.22.3
// this replace will be removed if https://github.com/kubernetes/kubernetes/pull/104920 is merged in 1.22
k8s.io/apiserver => github.com/leoryu/k8s-apiserver v0.22.4-0.20211110063743-0341ac1e5801
k8s.io/cli-runtime => k8s.io/cli-runtime v0.22.3
k8s.io/client-go => k8s.io/client-go v0.22.3
k8s.io/cloud-provider => k8s.io/cloud-provider v0.22.3
k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.22.3
k8s.io/code-generator => k8s.io/code-generator v0.22.3
k8s.io/component-base => k8s.io/component-base v0.22.3
k8s.io/component-helpers => k8s.io/component-helpers v0.22.3
k8s.io/controller-manager => k8s.io/controller-manager v0.22.3
k8s.io/cri-api => k8s.io/cri-api v0.22.3
k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.22.3
k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.22.3
k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.22.3
k8s.io/kube-proxy => k8s.io/kube-proxy v0.22.3
k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.22.3
k8s.io/kubectl => k8s.io/kubectl v0.22.3
k8s.io/kubelet => k8s.io/kubelet v0.22.3
k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.22.3
k8s.io/metrics => k8s.io/metrics v0.22.3
k8s.io/mount-utils => k8s.io/mount-utils v0.22.3
k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.22.3
k8s.io/sample-cli-plugin => k8s.io/sample-cli-plugin v0.22.3
k8s.io/sample-controller => k8s.io/sample-controller v0.22.3
)

require (
Expand Down Expand Up @@ -42,7 +67,7 @@ require (
github.com/gorilla/websocket v1.4.2
github.com/gosuri/uitable v0.0.4
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
github.com/hashicorp/go-uuid v1.0.1
github.com/hashicorp/go-uuid v1.0.2
github.com/howeyc/fsnotify v0.9.0
github.com/imdario/mergo v0.3.12
github.com/influxdata/influxdb1-client v0.0.0-20191209144304-8bf82d3c094d
Expand All @@ -51,8 +76,8 @@ require (
github.com/json-iterator/go v1.1.11
github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de
github.com/moul/http2curl v1.0.0 // indirect
github.com/onsi/ginkgo v1.14.0
github.com/onsi/gomega v1.10.3
github.com/onsi/ginkgo v1.16.4
github.com/onsi/gomega v1.15.0
github.com/parnurzeal/gorequest v0.2.15
github.com/pkg/errors v0.9.1
github.com/pkg/sftp v1.10.1
Expand All @@ -67,13 +92,14 @@ require (
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.8.1
github.com/stretchr/testify v1.7.0
github.com/superedge/superedge v0.8.2
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/clb v1.0.194
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.194
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/cvm v1.0.194
github.com/thoas/go-funk v0.4.0
go.etcd.io/etcd/client/pkg/v3 v3.5.0
go.etcd.io/etcd/client/v3 v3.5.0
go.uber.org/zap v1.17.0
go.uber.org/zap v1.19.0
golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a
golang.org/x/net v0.0.0-20210903162142-ad29c8ab022f
golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f
Expand Down Expand Up @@ -101,11 +127,12 @@ require (
k8s.io/component-base v0.22.3
k8s.io/klog v1.0.0
k8s.io/kube-aggregator v0.22.3
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e // indirect
k8s.io/kubectl v0.22.3
k8s.io/kubernetes v1.19.14
k8s.io/metrics v0.22.3
k8s.io/utils v0.0.0-20210819203725-bdf08cb9a70a
sigs.k8s.io/controller-runtime v0.6.0
sigs.k8s.io/controller-runtime v0.10.3
sigs.k8s.io/yaml v1.2.0
yunion.io/x/pkg v0.0.0-20200603123312-ad58e621aec0
)
Loading