Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor IPTable Rules #2697

Merged
merged 1 commit into from
Dec 12, 2023
Merged

Refactor IPTable Rules #2697

merged 1 commit into from
Dec 12, 2023

Conversation

jchen6585
Copy link
Contributor

@jchen6585 jchen6585 commented Dec 1, 2023

What type of PR is this?
bug

Which issue does this PR fix:
#2373

What does this PR do / Why do we need it:
nf_tables mode restricts the jump stack size to at most 16. We currently have (one jump per VPC CIDR/Excluded CIDR) + 1, so having 15+ CIDRS will result in a jump stack size greater than 16, causing failure. This pr refactors our iptable rules to retain current behavior while keeping the jump stack size to a size of 2.

If an issue # is not available please add repro steps and logs from IPAMD/CNI showing the issue:

Testing done on this change:

Manually checked iptable rules for 16 CIDRS, aws-node is up and running
Screenshot 2023-12-01 at 11 31 45 AM

Manually tested adding exclusion cidrs using AWS_VPC_K8S_CNI_EXCLUDE_SNAT_CIDRS

Also manually checked up to VPC Limit (50 CIDRS).
Manually checked packet using tcpdump

  • Leaving VPC goes out eth0
  • Routing within VPC goes out correct eth

Integration Tests:

  • CNI: Passing
Screenshot 2023-12-01 at 10 55 52 AM
  • IPAMD: Passing
Screenshot 2023-12-04 at 11 20 26 AM
  • SNAT: Passing
Screenshot 2023-12-05 at 2 21 07 PM

Ensured traffic to the internet aren't dropped while rules are being updated (deleted and added 10+ cidrs while ping in a pod was running)
Screenshot 2023-12-08 at 4 34 32 PM

Scale Testing
130 Pod
Screenshot 2023-12-01 at 4 25 34 PM

730 pod
Screenshot 2023-12-01 at 4 25 46 PM

5000 pod
Screenshot 2023-12-01 at 4 25 55 PM

Will this PR introduce any new dependencies?:

No

Will this break upgrades or downgrades? Has updating a running cluster been tested?:
No, Yes

Does this change require updates to the CNI daemonset config files to work?:

No

Does this PR introduce any user-facing change?:

No


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@jchen6585 jchen6585 requested a review from a team as a code owner December 1, 2023 19:38
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
@jchen6585 jchen6585 force-pushed the iptable-refactor branch 6 times, most recently from 45b54e0 to 1e181a3 Compare December 4, 2023 17:37
jdn5126
jdn5126 previously approved these changes Dec 4, 2023
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
@jchen6585 jchen6585 force-pushed the iptable-refactor branch 4 times, most recently from 700c562 to dc524ce Compare December 8, 2023 17:42
@jayanthvn
Copy link
Contributor

This will lead to non-zero chains being in iptables after upgrades even though we are not using them. I think it is better to delete the AWS chains during nodeInit and we anyway rebuild the chains so should be safe... Let's discuss it offline.

@jchen6585 jchen6585 force-pushed the iptable-refactor branch 3 times, most recently from 803671e to 929c112 Compare December 11, 2023 21:34
Copy link
Contributor

@jdn5126 jdn5126 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question, otherwise all changes look good to me

pkg/ipamd/ipamd.go Outdated Show resolved Hide resolved
Copy link
Contributor

@jdn5126 jdn5126 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few nits. I think we are good to update the test agent image in this PR now, assuming your testing shows everything passing

pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
pkg/networkutils/network.go Outdated Show resolved Hide resolved
jdn5126
jdn5126 previously approved these changes Dec 12, 2023
Copy link
Contributor

@jayanthvn jayanthvn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jchen6585 jchen6585 merged commit 21c4bd7 into aws:master Dec 12, 2023
6 checks passed
@jchen6585 jchen6585 mentioned this pull request Dec 13, 2023
jdn5126 added a commit to jdn5126/amazon-vpc-cni-k8s that referenced this pull request Dec 21, 2023
jdn5126 added a commit that referenced this pull request Dec 21, 2023
jchen6585 pushed a commit to jchen6585/amazon-vpc-cni-k8s that referenced this pull request Jan 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants