Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send NDP NA message upon assigning egress IP #2196

Merged
merged 1 commit into from
Aug 25, 2021
Merged

Conversation

wenqiq
Copy link
Contributor

@wenqiq wenqiq commented May 20, 2021

Send NDP NA message upon assigning egress IP

Implement Neighbor Discovery Protocol(NDP) neighbor advertisement for Egress IPv6 support.
Once an IPv6 IP address has been assigned to Node, an unsolicited Neighbor Advertisement ICMPv6
multicast packet will be sent, announcing the IP to all IPv6 nodes as per RFC4861.

Signed-off-by: Wenqi Qiu [email protected]

For #2128

@codecov-commenter
Copy link

codecov-commenter commented May 20, 2021

Codecov Report

Merging #2196 (3f9890e) into main (1829d3a) will increase coverage by 4.55%.
The diff coverage is 42.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2196      +/-   ##
==========================================
+ Coverage   60.76%   65.32%   +4.55%     
==========================================
  Files         286      287       +1     
  Lines       23096    26548    +3452     
==========================================
+ Hits        14034    17342    +3308     
- Misses       7592     7615      +23     
- Partials     1470     1591     +121     
Flag Coverage Δ
e2e-tests 56.07% <6.00%> (?)
kind-e2e-tests 47.46% <0.00%> (-0.29%) ⬇️
unit-tests 41.46% <47.36%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../controller/egress/ipassigner/ip_assigner_linux.go 62.50% <33.33%> (+62.50%) ⬆️
pkg/util/ip/ip.go 67.04% <40.00%> (-8.67%) ⬇️
pkg/agent/util/ndp/ndp_linux.go 44.44% <44.44%> (ø)
pkg/controller/egress/ipallocator/allocator.go 65.00% <0.00%> (-15.42%) ⬇️
pkg/controller/networkpolicy/endpoint_querier.go 77.64% <0.00%> (-13.79%) ⬇️
pkg/apis/controlplane/v1beta1/conversion.go 72.51% <0.00%> (-12.19%) ⬇️
pkg/legacyapis/core/v1alpha2/register.go 69.23% <0.00%> (-10.77%) ⬇️
pkg/apis/stats/register.go 71.42% <0.00%> (-10.39%) ⬇️
pkg/legacyapis/stats/register.go 71.42% <0.00%> (-10.39%) ⬇️
pkg/ovs/openflow/ofctrl_meter.go 33.84% <0.00%> (-10.16%) ⬇️
... and 274 more

@wenqiq wenqiq force-pushed the egress-ndp branch 2 times, most recently from 98d673b to e7362d9 Compare May 20, 2021 13:45
Copy link
Member

@tnqn tnqn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @wenqiq. Want to check with you if it can be simplified.

pkg/agent/util/ndp/ndp.go Outdated Show resolved Hide resolved
pkg/agent/util/ndp/ndp.go Outdated Show resolved Hide resolved
xliuxu
xliuxu previously requested changes Jul 2, 2021
go.mod Outdated Show resolved Hide resolved
@wenqiq wenqiq force-pushed the egress-ndp branch 3 times, most recently from 2745f9b to 888dc3d Compare July 3, 2021 12:42
@wenqiq wenqiq changed the title Add NDP protocol interface for ipv6 support Implement Neighbor Discovery Protocol neighbor advertisement for IPv6 support Jul 12, 2021
@wenqiq wenqiq marked this pull request as ready for review July 14, 2021 01:15
@wenqiq
Copy link
Contributor Author

wenqiq commented Jul 15, 2021

@tnqn @xliuxu I updated the patch and used a function to implement NDP neighbor advertisement. PTAL. Thanks.

pkg/agent/util/ndp/doc.go Outdated Show resolved Hide resolved
pkg/agent/util/ndp/ndp_linux.go Outdated Show resolved Hide resolved
return nil
}

// NeighborAdvertisement sends an NDP Neighbor Advertisement over interface 'iface' from 'srcIP'.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// NeighborAdvertisement sends an NDP Neighbor Advertisement over interface 'iface' from 'srcIP'.
// NeighborAdvertisement sends a NDP Neighbor Advertisement over interface 'iface' from 'srcIP'.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

return fmt.Errorf("interface address error: %v", err)
}

ipAddr := &net.IPAddr{}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ipAddr := &net.IPAddr{}
var ipAddr *net.IPAddr

Since you create a new instance in the loop.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

}
}

ic, err := icmp.ListenPacket("ip6:ipv6-icmp", ipAddr.String())
Copy link
Member

@tnqn tnqn Jul 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why it needs to listen incoming icmp packets? I thought it just sends an unsolicited NA?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, use syscall.Socket to send a NA message
syscall.Socket(syscall.AF_INET6, syscall.SOCK_RAW, syscall.IPPROTO_ICMPV6)

pkg/agent/util/ndp/ndp_linux.go Show resolved Hide resolved
}

cm := &ipv6.ControlMessage{
HopLimit: hopLimit,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the hopLimit is set multiple times? I assume only one takes effect.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


cm := &ipv6.ControlMessage{
HopLimit: hopLimit,
Src: ipAddr.IP,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't set src, will the OS pick one from the interface?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, PTAL.

@wenqiq wenqiq requested a review from xliuxu July 20, 2021 14:23
@wenqiq wenqiq force-pushed the egress-ndp branch 2 times, most recently from 1681d9c to 54171f3 Compare July 25, 2021 00:31
@wenqiq
Copy link
Contributor Author

wenqiq commented Aug 11, 2021

Do you have any comments or considerations about this PR? @tnqn

Copy link
Member

@tnqn tnqn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you have tested this PR in a IPv6 cluster and it worked? Have you checked whether the generated NDP packet can fresh an external Node's ip neighbor cache?

return fmt.Errorf("failed to send gratuitous ARP: %v", err)
}
klog.V(2).InfoS("Sent gratuitous ARP", "ip", parsedIP)
} else {
klog.ErrorS(ipv6NotSupportErr, "Failed to send Advertisement", "ip", parsedIP)
} else if addr.IP.To16() != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as we have verified the IP, maybe just use "else" to avoid a repeated conversion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

I assume you have tested this PR in a IPv6 cluster and it worked? Have you checked whether the generated NDP packet can fresh an external Node's ip neighbor cache?

I have tested and I will put some test data here soon.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The external Node can receive the NDP NA or GratuitousARP packets, but IP neighbor cache didn't fresh.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, receiving GARP will only flush the stale entry and won't add a new entry, is this your observation? or the stale entry was not deleted? You mean both NDP NA and GARP didn't work?

Copy link
Contributor Author

@wenqiq wenqiq Aug 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think both NDP NA and GARP can work, sent message packets successfully. The integration test seems back to normal.

I have tested the basic IPv6 SNAT function and put some info in my repo. PTAL. @tnqn

https://github.com/wenqiq/antrea/blob/egress-ndp-demonstration/docs/egress-ipv6/egress-ipv6-practice.md

}
}

func mustIPv6(s string) (net.IP, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe put it to pkg/util/ip/ip.go like

antrea/pkg/util/ip/ip.go

Lines 174 to 181 in fe4457f

// MustParseCIDR turns the given string into IPNet or panics, for tests or other cases where the string must be valid.
func MustParseCIDR(cidr string) *net.IPNet {
_, ipNet, err := net.ParseCIDR(cidr)
if err != nil {
panic(fmt.Errorf("cannot parse '%v': %v", cidr, err))
}
return ipNet
}

It can be used in other cases too. I think you don't need to check the result since it's only used when we know the IP must be valid.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

tnqn
tnqn previously approved these changes Aug 16, 2021
Copy link
Member

@tnqn tnqn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tnqn
Copy link
Member

tnqn commented Aug 16, 2021

integration test failed:

failed to send neighbor advertisement: new NDP Neighbor Advertisement Message error: hardwareAddr length error: 

@tnqn
Copy link
Member

tnqn commented Aug 16, 2021

It seems that the code for IPv4 is also wrong as HardwareAddr is always empty if the dummy device is newly created.

@wenqiq wenqiq dismissed xliuxu’s stale review August 17, 2021 08:50

Requested changed

@wenqiq
Copy link
Contributor Author

wenqiq commented Aug 17, 2021

It seems that the code for IPv4 is also wrong as HardwareAddr is always empty if the dummy device is newly created.

In the test case we use local IP "127.0.0.1" to get IPNetDevice and the interface doesn't have a HardwareAddr which cause testing fail.

ipassigner.NewIPAssigner(net.ParseIP("127.0.0.1"), dummyDeviceName)

func NewIPAssigner(nodeIPAddr net.IP, dummyDeviceName string) (*ipAssigner, error) {
	_, egressInterface, err := util.GetIPNetDeviceFromIP(nodeIPAddr)

@wenqiq wenqiq changed the title Implement Neighbor Discovery Protocol neighbor advertisement for IPv6 support [WIP][Do Not Merge]Implement Neighbor Discovery Protocol neighbor advertisement for IPv6 support Aug 17, 2021
@wenqiq
Copy link
Contributor Author

wenqiq commented Aug 24, 2021

Rebased upstream main branch and fixed grammer issues in commit message.
/test-all

@wenqiq wenqiq added this to the Antrea v1.3 release milestone Aug 24, 2021
tnqn
tnqn previously approved these changes Aug 24, 2021
Copy link
Member

@tnqn tnqn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, will wait for one day before merging in case anyone wants to take another look

@tnqn
Copy link
Member

tnqn commented Aug 24, 2021

/test-all

@tnqn
Copy link
Member

tnqn commented Aug 25, 2021

/test-ipv6-all

@tnqn
Copy link
Member

tnqn commented Aug 25, 2021

@wenqiq I just notice the tests still don't run on IPv6 clusters. Can you have a following-up PR to enable egress tests on IPv6 and dual-stack clusters? otherwise we wouldn't know when it's broken by accident.

@tnqn
Copy link
Member

tnqn commented Aug 25, 2021

"Go / Check tidy, code generation and manifest (pull_request)" failed, which may be because go 1.17 update. Please update your local go version and regenerate the code.

@wenqiq
Copy link
Contributor Author

wenqiq commented Aug 25, 2021

@wenqiq I just notice the tests still don't run on IPv6 clusters. Can you have a following-up PR to enable egress tests on IPv6 and dual-stack clusters? otherwise we wouldn't know when it's broken by accident.

Sure, will implement it.

"Go / Check tidy, code generation and manifest (pull_request)" failed, which may be because go 1.17 update. Please update your local go version and regenerate the code.

It seems there is no generated code or manifest changes in this PR.

Implement Neighbor Discovery Protocol(NDP) neighbor advertisement
for Egress IPv6 support.Once an IPv6 IP address has been assigned
to Node, an unsolicited Neighbor Advertisement ICMPv6 multicast
packet will be sent, announcing the IP to all IPv6 nodes
as per RFC4861.

Signed-off-by: Wenqi Qiu <[email protected]>
@wenqiq
Copy link
Contributor Author

wenqiq commented Aug 25, 2021

Rebased upstream/main.
/test-all

@tnqn
Copy link
Member

tnqn commented Aug 25, 2021

/test-ipv6-all

@tnqn
Copy link
Member

tnqn commented Aug 25, 2021

/test-ipv6-only-all

@tnqn
Copy link
Member

tnqn commented Aug 25, 2021

/test-e2e

@wenqiq
Copy link
Contributor Author

wenqiq commented Aug 25, 2021

It seems jenkins-ipv6-only-e2e test failed? How can we get the test result details?

@tnqn
Copy link
Member

tnqn commented Aug 25, 2021

I checked the IPv6 e2e tests failed on go build issue, which may be related to Go version upgrade. Since we haven't enabled egress test on IPv6, it shouldn't be affected in theory. I'm going to merge this. @wenqiq will have a follow-up PR for enabling egress test on IPv6 cluster.

@tnqn tnqn merged commit 01da0aa into antrea-io:main Aug 25, 2021
wenqiq added a commit to wenqiq/antrea that referenced this pull request Sep 1, 2021
…antrea-io#2196

Fixed agent start fail nil panic in pure IPv6 cluster. Related antrea-io#2436

Signed-off-by: Wenqi Qiu <[email protected]>
wenqiq added a commit to wenqiq/antrea that referenced this pull request Sep 1, 2021
…antrea-io#2196

Fixed agent start fail nil panic in pure IPv6 cluster. Related antrea-io#2436

Signed-off-by: Wenqi Qiu <[email protected]>
wenqiq added a commit to wenqiq/antrea that referenced this pull request Sep 1, 2021
…antrea-io#2196

Fixed agent start fail nil panic in pure IPv6 cluster. Related antrea-io#2436

Signed-off-by: Wenqi Qiu <[email protected]>
wenqiq added a commit to wenqiq/antrea that referenced this pull request Sep 1, 2021
The nodeConfig.NodeIPv4Addr is nil which would cause panic in agent,
when starting agent with Egress feature enabled in pure IPv6 cluster.

It also adds Egress IPv6 test cases in dual-stack and pure IPv6 cluster.

Related antrea-io#2196

Signed-off-by: Wenqi Qiu <[email protected]>
wenqiq added a commit to wenqiq/antrea that referenced this pull request Sep 2, 2021
The nodeConfig.NodeIPv4Addr is nil which would cause panic in agent,
when starting agent with Egress feature enabled in pure IPv6 cluster.

It also adds Egress IPv6 test cases in dual-stack and pure IPv6 cluster.

Related antrea-io#2196

Signed-off-by: Wenqi Qiu <[email protected]>
wenqiq added a commit to wenqiq/antrea that referenced this pull request Sep 2, 2021
The nodeConfig.NodeIPv4Addr is nil which would cause panic in agent,
when starting agent with Egress feature enabled in pure IPv6 cluster.

It also adds Egress IPv6 test cases in dual-stack and pure IPv6 cluster.

Related antrea-io#2196

Signed-off-by: Wenqi Qiu <[email protected]>
tnqn pushed a commit that referenced this pull request Sep 3, 2021
The nodeConfig.NodeIPv4Addr is nil which would cause panic in agent,
when starting agent with Egress feature enabled in pure IPv6 cluster.

It also adds Egress IPv6 test cases in dual-stack and pure IPv6 cluster.

Related #2196

Signed-off-by: Wenqi Qiu <[email protected]>
tnqn pushed a commit to tnqn/antrea that referenced this pull request Sep 3, 2021
The nodeConfig.NodeIPv4Addr is nil which would cause panic in agent,
when starting agent with Egress feature enabled in pure IPv6 cluster.

It also adds Egress IPv6 test cases in dual-stack and pure IPv6 cluster.

Related antrea-io#2196

Signed-off-by: Wenqi Qiu <[email protected]>
tnqn added a commit that referenced this pull request Sep 3, 2021
The nodeConfig.NodeIPv4Addr is nil which would cause panic in agent,
when starting agent with Egress feature enabled in pure IPv6 cluster.

It also adds Egress IPv6 test cases in dual-stack and pure IPv6 cluster.

Related #2196

Signed-off-by: Wenqi Qiu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants