-
Notifications
You must be signed in to change notification settings - Fork 295
Creating cluster with an existing subnet #52
Comments
Provisioning in existing subnets creates a number of weird edge cases that i do not feel like supporting. This request keeps showing up though, so out of interest, can you clarify why it's not sufficient to use existing routetables and attaching those to the new subnets? A little bit more background info on what you're trying to do would also help to see why you need this feature.
Do you mean in AWS, or at your office? |
Hi @pieterlange. Thanks for your response. I don't understand:
What do you talking about? Have you some examples? As far I can see, I use existing subnet w/o issues.
I can't create subnets into the AWS account of my company. |
@sdouche Hi, would you mind letting me summarize!
|
I wanted to share my experience of shared subnets. Having encountered a slight different use case but same end result of wanting to share network resources I did a pull request here for shared subnets. I ended up replacing that with shared route tables. The main reason I ditched shared subnets is that k8s and kube-aws rely on some aspects of the subnets and to a lesser extent the route tables which meant that spinning up a new cluster was a little fragile. It's definitely related to what @mumoshu says above - dedicated subnets are ok but mixing them tends to case issues. I think it's definitely achievable still though with subnets, I chose the path of slightly lesser resistance. There would have to be detailed instructions on exactly what subnet and route table requirements there are across k8s and kube-aws. |
One of the side effects of not having the subnets managed by your deployment tool: kubernetes/kubernetes#29298 Opinion: organizations that cling on to old-world devision of labor with network teams and IT systems teams can do the necessary work for product integration themselves. (and deal with the edge cases themselves) |
Hi everyone. I never talk about shared networks (I use dedicated subnets for k8s), only to use existing subnets (created for me). At work, we've a complex network topology (8 AWS accounts, 2 datacenters, etc), that's why the AWS network objects are managed only by the sysadmin teams. EDIT: I'm ok if you don't want change the Cloudformation template but at least, please remove the subnet verification in
|
@pieterlange wrong example here, you created a deployment tool for internet facing only. I don't want to expose our Jenkins cluster to the world. |
Okay, this is a fair use-case but i don't think we should break the experience for other users based on this very specific example. I feel if we're going to support this, we shouldn't needlessly complicate the So you instead of defining: subnets:
- availabilityZone: us-east-1a
instanceCIDR: "10.50.1.0/24"
- availabilityZone: us-east-1b
instanceCIDR: "10.50.2.0/24"
- availabilityZone: us-east-1c
instanceCIDR: "10.50.3.0/24" You'd define the following: subnets:
- availabilityZone: us-east-1a
subnetId: "subnet-abcdef1"
- availabilityZone: us-east-1b
subnetId: "subnet-abcdef2"
- availabilityZone: us-east-1c
subnetId: "subnet-abcdef3" Just an example/proposal. What do you think @sdouche @mumoshu @c-knowles ? Maybe even keep the |
Hi @pieterlange. Seems good to me. I'm open to beta test this change :). Thanks. |
I need to be able to deploy k8n to existing subnet because that subnets are preconfigured with some concept in mind, regarding routing and IP ranges... |
Hi @aholbreich. |
Also if you are interested in checking out some code changes to see something right now, I have a branch of the older repo with some changes in. |
Not true @mumoshu, |
@sdouche Excuse me if I'm missing the point but I meant to use e.g. awscli to finally create stack from the stack template initially generated via |
@mumoshu oh i see. |
@mumoshu maybe i do not understand is well. But isn't the whole puint of CLI tool like this to be able to crate and manage the whole cluster from cli? without touching any intermediate artefacts? Otherways it is better to use something more declarative and define the whole set-up step by step. |
@aholbreich Not necessarily it can be used to generate cloudformation templates that you customize for your needs and store in a version control system. We need to deploy in existing private subnets and so far we have been using the cli tool to generate the template and be able to track changes in git. |
@Camsteack you can use your own routetable and set mapPublicIPs in order to deploy to private subnets. |
@pieterlange this is what we are doing 😃 |
This is supported since v0.9.4-rc.1. Please read the updated comments in |
kube-aws version: v0.9.7 Is it still true that using managed VS existing subnets is still preferred? Here's my use case. I'm using Terraform to build out a brand-new VPC and all of its associated objects (route tables, NAT gateways, subnets, etc...). I specifically created an "application" tier of subnets and now I want I'm really trying to assess the risks involved here. |
We run kube-aws in existing subnets without problem for a long time, pretty safe to use |
@redbaron One of the things we're trying to understand is how to get this working with existing subnets. In the case where our subnet is managed by subnets:
- name: ManagedPublicSubnet1
private: true # does some magic
availabilityZone: us-west-1a
instanceCIDR: "10.0.0.0/24" When we want to leverage an existing public subnet (configured to use a route table that is connected to a NAT gateway and confirmed working), how does the K8s cluster know it is public? |
You are missing ID:
|
keep That is how it works for us 100% right now, maybe |
I'm sorry, I was unclear. We're able to get existing private subnets working, but failing to configure existing PUBLIC subnets correctly. 😢 The documentation in
Even with that setup, every time I launch, say, a public-facing LoadBalancer service in K8s is stays in pending state with the following error message:
Another team member found that adding an AWS tag to the subnet will then allow the cluster to launch LBs in the public subnets. AWS tag key = kubernetes.io/cluster/my-cluster-name, value = true. Is that expected? I fear I'm missing something here. |
yes, tags need to be present on a subnet, existing subnets are not tagged by kube-aws. Following tags need to be present: VPC: Subnets where internal ELBs will be created: Subnets where external ELBs will be created: |
Thank you so much for your help. I was able to get this working and found this other issue that gives more context. NOTE: You cluster tag is valid for K8s v1.6, v1.5 uses |
…ues-with-nds-rollouts to hcom-flavour * commit 'f6c440e0654c605a24248ad6b58a04d097e49c56': Make sure that canal and flannel land on all nodes (including tainted ones). Correct addition syntax in add-node-cidrs.
Hello,
We can specify an existing VPC, but not an existing subnet. At work, I can't create network stuff. I have forked the file
stack-template.json
but It would be cool to have the option built-in.Thanks.
The text was updated successfully, but these errors were encountered: