MCLAG feature for SONIC (#2514)

* MCLAG feature for sonic

* MCLAG feature for sonic

* remove binary file

* remove unused dockerfile

update docker-iccpd to stretch-based container

Signed-off-by: shine.chen <[email protected]>

* minor fix for isolation port setting

Signed-off-by: shine.chen <[email protected]>

* iccpd docker would start on demand

Signed-off-by: shine.chen <[email protected]>

* Add x attribute on mclagdctl file

Signed-off-by: shine.chen <[email protected]>

* add warm-reboot support for MCLAG

Signed-off-by: shine.chen <[email protected]>

* merge to master branch and reformat iccpd file

Signed-off-by: shine.chen <[email protected]>

* fix some bugs and make peer-link configuration optional

Signed-off-by: shine.chen <[email protected]>

* refactor code per Brcm review

Signed-off-by: shine.chen <[email protected]>

* correct a typo

Signed-off-by: shine.chen <[email protected]>

* * optimize iccpd arp/mac sync process
* refine code according to brcm opinoin
* unify function return value

Signed-off-by: shine.chen <[email protected]>

* * optimize warm-reboot process
* estabish iccpd connection with configurated src-ip

Signed-off-by: shine.chen <[email protected]>

* fix a typo

Signed-off-by: shine.chen <[email protected]>

* optimize some code
* add some debug info
* optimize bridge mac setting
* fix vlan mac sync issue on standby node

Signed-off-by: shine.chen <[email protected]>

* optimize some code

Signed-off-by: shine.chen <[email protected]>

* fix some bugs for warm-reboot

Signed-off-by: shine.chen <[email protected]>

* refine log level

Signed-off-by: shine.chen <[email protected]>

* refine iccpd syslog & skip arp packet whose src ip is local ip

Signed-off-by: shine.chen <[email protected]>

* remove iccpd dependency with teamd

Signed-off-by: shine.chen <[email protected]>

* print log level when dump mclag status

Signed-off-by: shine.chen <[email protected]>

* revise per community review

Signed-off-by: shine.chen <[email protected]>

Co-authored-by: shine.chen <[email protected]>
Co-authored-by: shine.chen <[email protected]>

dockers/docker-iccpd/Dockerfile.j2

@@ -0,0 +1,47 @@
+{% from "dockers/dockerfile-macros.j2" import install_debian_packages, install_python_wheels, copy_files %}
+FROM docker-config-engine-stretch
+
+ARG docker_container_name
+RUN [ -f /etc/rsyslog.conf ] && sed -ri "s/%syslogtag%/$docker_container_name#%syslogtag%/;" /etc/rsyslog.conf
+
+## Make apt-get non-interactive
+ENV DEBIAN_FRONTEND=noninteractive
+
+RUN apt-get update      && \
+    apt-get install -f -y  \
+    libdbus-1-3            \
+    libdaemon0             \
+    libpython2.7           \
+    # Install redis-tools dependencies
+    # TODO: implicitly install dependencies
+    libatomic1             \
+    libjemalloc1           \
+    liblua5.1-0            \
+    lua-bitop              \
+    lua-cjson
+
+RUN apt-get -y install ebtables
+RUN apt-get -y install -f kmod
+
+COPY \
+{% for deb in docker_iccpd_debs.split(' ') -%}
+debs/{{ deb }}{{' '}}
+{%- endfor -%}
+debs/
+
+RUN dpkg -i \
+{% for deb in docker_iccpd_debs.split(' ') -%}
+debs/{{ deb }}{{' '}}
+{%- endfor %}
+
+COPY ["start.sh", "iccpd.sh", "/usr/bin/"]
+COPY ["supervisord.conf", "/etc/supervisor/conf.d/"]
+COPY ["iccpd.j2", "/usr/share/sonic/templates/"]
+
+RUN chmod +x /usr/bin/start.sh /usr/bin/iccpd.sh
+RUN apt-get clean -y      && \
+    apt-get autoclean -y  && \
+    apt-get autoremove -y && \
+    rm -rf /debs
+
+ENTRYPOINT ["/usr/bin/supervisord"]

dockers/docker-iccpd/base_image_files/mclagdctl

@@ -0,0 +1,10 @@
+#!/bin/bash
+
+DOCKER_EXEC_FLAGS="i"
+
+# Determine whether stdout is on a terminal
+if [ -t 1 ] ; then
+    DOCKER_EXEC_FLAGS+="t"
+fi
+
+docker exec -$DOCKER_EXEC_FLAGS iccpd mclagdctl "$@"

dockers/docker-iccpd/iccpd.j2

@@ -0,0 +1,11 @@
+{% for mclag_id in MC_LAG %}
+mclag_id:{{mclag_id}}
+    local_ip:{{MC_LAG[mclag_id]['local_ip']}}
+    peer_ip:{{MC_LAG[mclag_id]['peer_ip']}}
+{% if MC_LAG[mclag_id].has_key('peer_link') %}
+    peer_link:{{MC_LAG[mclag_id]['peer_link']}}
+{% endif %}
+    mclag_interface:{{MC_LAG[mclag_id]['mclag_interface']}}
+{% endfor %}
+system_mac:{{DEVICE_METADATA['localhost']['mac']}}
+

dockers/docker-iccpd/iccpd.sh

@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+
+function start_app {
+    rm -f /var/run/iccpd/*
+    mclagsyncd &
+    iccpd
+}
+
+function clean_up {
+    pkill -9 mclagsyncd
+    pkill -9 iccpd
+    exit
+}
+
+trap clean_up SIGTERM SIGKILL
+start_app
+read

dockers/docker-iccpd/start.sh

@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+
+
+ICCPD_CONF_PATH=/etc/iccpd
+
+rm -rf $ICCPD_CONF_PATH
+mkdir -p $ICCPD_CONF_PATH
+
+sonic-cfggen -d -t /usr/share/sonic/templates/iccpd.j2 > $ICCPD_CONF_PATH/iccpd.conf
+
+mkdir -p /var/sonic
+echo "# Config files managed by sonic-config-engine" > /var/sonic/config_status
+
+rm -f /var/run/rsyslogd.pid
+
+supervisorctl start rsyslogd
+
+supervisorctl start iccpd

dockers/docker-iccpd/supervisord.conf

@@ -0,0 +1,29 @@
+[supervisord]
+logfile_maxbytes=1MB
+logfile_backups=2
+nodaemon=true
+
+[program:start.sh]
+command=/usr/bin/start.sh
+priority=1
+autostart=true
+autorestart=false
+stdout_logfile=syslog
+stderr_logfile=syslog
+
+[program:rsyslogd]
+command=/usr/sbin/rsyslogd -n
+priority=2
+autostart=false
+autorestart=false
+stdout_logfile=syslog
+stderr_logfile=syslog
+
+[program:iccpd]
+command=/usr/bin/iccpd.sh
+priority=3
+autostart=false
+autorestart=false
+stdout_logfile=syslog
+stderr_logfile=syslog
+

files/build_templates/docker_image_ctl.j2

@@ -231,6 +231,9 @@ start() {
 {%- if docker_container_name != "database" %}
         -v /usr/share/sonic/device/$PLATFORM/$HWSKU/$DEV:/usr/share/sonic/hwsku:ro \
 {%- endif %}
+{%- if docker_container_name == "iccpd" %}
+        -v /lib/modules:/lib/modules:ro \
+{%- endif %}
 {%- if sonic_asic_platform != "mellanox" %}
         --tmpfs /tmp \
 {%- endif %}

files/build_templates/iccpd.service.j2

@@ -0,0 +1,13 @@
+[Unit]
+Description=ICCPD container
+Requires=updategraph.service swss.service
+After=updategraph.service swss.service
+
+[Service]
+User={{ sonicadmin_user }}
+ExecStartPre=/usr/bin/{{docker_container_name}}.sh start
+ExecStart=/usr/bin/{{docker_container_name}}.sh wait
+ExecStop=/usr/bin/{{docker_container_name}}.sh stop
+
+[Install]
+WantedBy=multi-user.target swss.service

files/build_templates/sonic_debian_extension.j2

@@ -427,6 +427,9 @@ if [ -f {{service}} ]; then
     echo "{{service}}" | sudo tee -a $GENERATED_SERVICE_FILE
 fi
 {% endfor %}
+if [ -f iccpd.service ]; then
+    sudo LANG=C chroot $FILESYSTEM_ROOT systemctl disable iccpd.service
+fi
 sudo LANG=C chroot $FILESYSTEM_ROOT fuser -km /sys || true
 sudo LANG=C chroot $FILESYSTEM_ROOT umount -lf /sys
 {% endif %}

rules/config

@@ -115,6 +115,10 @@ SONIC_DPKG_CACHE_SOURCE ?= /var/cache/sonic/artifacts
 # Default VS build memory preparation
 DEFAULT_VS_PREPARE_MEM = yes
 
+
+# ENABLE_ICCPD - build docker-iccpd for mclag support
+ENABLE_ICCPD = y
+
 # ENABLE_SYSTEM_SFLOW - build docker-sonic-sflow for sFlow support
 ENABLE_SFLOW = y
 
@@ -126,3 +130,4 @@ ENABLE_RESTAPI = n
 
 # ENABLE_NAT - build docker-sonic-nat for nat support
 ENABLE_NAT = y
+

rules/docker-iccpd.mk

@@ -0,0 +1,18 @@
+# docker image for iccpd agent
+
+DOCKER_ICCPD = docker-iccpd.gz
+$(DOCKER_ICCPD)_PATH = $(DOCKERS_PATH)/docker-iccpd
+$(DOCKER_ICCPD)_DEPENDS += $(SWSS) $(REDIS_TOOLS) $(ICCPD)
+$(DOCKER_ICCPD)_LOAD_DOCKERS += $(DOCKER_CONFIG_ENGINE_STRETCH)
+
+ifeq ($(ENABLE_ICCPD), y)
+SONIC_DOCKER_IMAGES += $(DOCKER_ICCPD)
+SONIC_INSTALL_DOCKER_IMAGES += $(DOCKER_ICCPD)
+SONIC_STRETCH_DOCKERS += $(DOCKER_ICCPD)
+endif
+
+$(DOCKER_ICCPD)_CONTAINER_NAME = iccpd
+$(DOCKER_ICCPD)_RUN_OPT += --privileged -t
+$(DOCKER_ICCPD)_RUN_OPT += -v /etc/sonic:/etc/sonic:ro
+
+$(DOCKER_ICCPD)_BASE_IMAGE_FILES += mclagdctl:/usr/bin/mclagdctl

rules/iccpd.mk

@@ -0,0 +1,14 @@
+# iccpd package
+
+ICCPD_VERSION = 0.0.5
+
+ICCPD = iccpd_$(ICCPD_VERSION)_amd64.deb
+$(ICCPD)_DEPENDS += $(LIBNL_GENL3_DEV) $(LIBNL_CLI_DEV)
+$(ICCPD)_RDEPENDS += $(LIBNL_GENL3) $(LIBNL_CLI)
+$(ICCPD)_SRC_PATH = $(SRC_PATH)/iccpd
+SONIC_MAKE_DEBS += $(ICCPD)
+# SONIC_STRETCH_DEBS += $(ICCPD)
+
+# Export these variables so they can be used in a sub-make
+export ICCPD_VERSION
+export ICCPD

src/iccpd/Makefile

@@ -0,0 +1,35 @@
+.ONESHELL:
+SHELL = /bin/bash
+.SHELLFLAGS += -e
+
+MAIN_TARGET = iccpd_$(ICCPD_VERSION)_amd64.deb
+DEB_PATH = debian
+
+all: iccpd-build mclagdctl-build
+
+iccpd-build:
+	make -C src
+
+mclagdctl-build:
+	make -C src/mclagdctl
+
+$(addprefix $(DEST)/, $(MAIN_TARGET)): $(DEST)/% :
+	make all
+	if [ ! -d $(DEB_PATH)/usr/bin ]; then
+		mkdir -p $(DEB_PATH)/usr/bin
+	fi
+	cp iccpd $(DEB_PATH)/usr/bin/iccpd
+	cp src/mclagdctl/mclagdctl $(DEB_PATH)/usr/bin/mclagdctl
+	chmod +x $(DEB_PATH)/usr/bin/iccpd
+	chmod +x $(DEB_PATH)/usr/bin/mclagdctl
+	md5sum $(DEB_PATH)/usr/bin/iccpd > $(DEB_PATH)/DEBIAN/md5sums
+	md5sum $(DEB_PATH)/usr/bin/mclagdctl >> $(DEB_PATH)/DEBIAN/md5sums
+	dpkg-deb -b $(DEB_PATH) $(DEST)/$(MAIN_TARGET)
+
+clean: iccpd-clean mclagdctl-clean
+
+iccpd-clean:
+	make -C src clean
+
+mclagdctl-clean:
+	make -C src/mclagdctl clean

src/iccpd/debian/DEBIAN/control

@@ -0,0 +1,12 @@
+Package: iccpd-0.0.5-amd64
+Source: nps
+Version: 0.0.5
+Architecture: amd64
+Maintainer: Simon Ji <[email protected]>
+Installed-Size: 1508
+Depends:
+Section: main
+Priority: extra
+Homepage: https://github.com/NephosInc/SONiC
+Description:
+

src/iccpd/include/app_csm.h

@@ -0,0 +1,73 @@
+/*
+ * app_csm.h
+ *
+ * Copyright(c) 2016-2019 Nephos/Estinet.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, see <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ *  Maintainer: jianjun, grace Li from nephos
+ */
+
+#ifndef APP_CSM_H_
+#define APP_CSM_H_
+
+#include <sys/queue.h>
+
+#include "../include/mlacp_fsm.h"
+
+struct CSM;
+
+enum APP_CONNECTION_STATE
+{
+    APP_NONEXISTENT,
+    APP_RESET,
+    APP_CONNSENT,
+    APP_CONNREC,
+    APP_CONNECTING,
+    APP_OPERATIONAL
+};
+
+typedef enum APP_CONNECTION_STATE APP_CONNECTION_STATE_E;
+
+struct AppCSM
+{
+    struct mLACP mlacp;
+    APP_CONNECTION_STATE_E current_state;
+
+    uint32_t rx_connect_msg_id;
+    uint32_t tx_connect_msg_id;
+    uint32_t invalid_msg_id;
+
+    TAILQ_HEAD(app_msg_list, Msg) app_msg_list;
+
+    uint8_t invalid_msg : 1;
+    uint8_t nak_msg : 1;
+};
+
+void app_csm_init(struct CSM*, int all);
+void app_csm_finalize(struct CSM*);
+void app_csm_transit(struct CSM*);
+int app_csm_prepare_iccp_msg(struct CSM*, char*, size_t);
+void app_csm_enqueue_msg(struct CSM*, struct Msg*);
+struct Msg* app_csm_dequeue_msg(struct CSM*);
+void app_csm_correspond_from_msg(struct CSM*, struct Msg*);
+void app_csm_correspond_from_connect_msg(struct CSM*, struct Msg*);
+void app_csm_correspond_from_connect_ack_msg(struct CSM*, struct Msg*);
+int app_csm_prepare_nak_msg(struct CSM*, char*, size_t);
+int app_csm_prepare_connect_msg(struct CSM*, char*, size_t);
+int app_csm_prepare_connect_ack_msg(struct CSM*, char*, size_t);
+
+#endif /* APP_CSM_H_ */