How configuration change should be handled during healing?

[ljkiraly]

Question

A custom NSC is used to connect to NSM v1.9.0. This NSC has a side-car which auto-generate the interface name configured on NSC. During a robustness test the following happened:

• The NSMgr pod was restarted;
• The custom NSC tries to close and reopen the connections (kernel2kernel and kernel2ethernet2kernel type connections);
• NSMgr shortcut the close request and did not sent towards forwarder (forwarder-vpp);
• When the NSC tries to 'reopen' the connection a new interface name was generated, different from the previous one;
• Since the forwarder doesn't receive close for the previous connection this request was handled as a refresh ;
• This sequence ended in two faulty NSC interfaces (two connections toward two network service, but interfaces are mixed up and also the cross-connect to the remote connection). Also the routing tables for policy based routing are empty.

Can you suggest something to solve this issue? The data-path healing is not an option here. Currently the forwarder tolerate the configuration change (interface name change) for a connection, can that be changed to return with some fault when some specific parameters changed for an established connection?

[glazychev-art]

Hi @ljkiraly
Thanks for the report

Could you re-check this problem on the last main branch or on v1.10.0-rc.1 (it will be out soon)?
We have improved how the healing works and now it should cover this scenario - Close will be called on the forwarder

[denis-tingaikin]

@ljkiraly The problem should be fixed by networkservicemesh/sdk#1471

Now changes in datapath should work as you expect.

Feel free to test it with v1.10.0

If the problem is still actual, please feel free to re-open the ticket OR create another one if you find something else ;)