topotests: 'sudo pytest bgp-ecmp-topo1/test_bgp_ecmp_topo1.py' always fail

[kwind]

Hi, I configure test according to "http://docs.frrouting.org/projects/dev-guide/en/latest/topotests.html", but 'sudo pytest bgp-ecmp-topo1/test_bgp_ecmp_topo1.py' always fail. ‘sudo pytest ospf-topo1/test_ospf_topo1.py’ is ok.

the following is error messages:

zfg@zfg-virtual-machine:~/frr/frr/tests/topotests$ sudo pytest bgp-ecmp-topo1/test_bgp_ecmp_topo1.py -s -vv

2019-05-23 01:15:59,043 INFO: Running environment diagnostics
2019-05-23 01:15:59,331 WARNING: failed to find exabgp or returned error
============================================================================ test session starts ============================================================================
platform linux2 -- Python 2.7.15rc1, pytest-3.3.2, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python2
cachedir: .cache
rootdir: /home/zfg/frr/frr/tests/topotests, inifile: pytest.ini
collected 2 items

bgp-ecmp-topo1/test_bgp_ecmp_topo1.py::test_bgp_convergence 2019-05-23 01:16:02,794 INFO: loading topology: test_bgp_ecmp_topo1
2019-05-23 01:16:02,794 INFO: starting topology: test_bgp_ecmp_topo1
2019-05-23 01:16:06,085 INFO: r1: running version: 7.0.1-MyOwnFRRVersion-g305c097
2019-05-23 01:16:06,088 INFO: r1: waiting for zebra to start (1 seconds)
2019-05-23 01:16:07,985 INFO: starting BGP on all 20 peers (10 seconds)
2019-05-23 01:16:18,251 INFO: peer1 exabgp started, output=bash: exabgp: Command not found

2019-05-23 01:16:18,251 INFO: peer1
2019-05-23 01:16:18,319 INFO: peer20 exabgp started, output=bash: exabgp: Command not found
....

2019-05-23 01:16:18,908 INFO: peer17
2019-05-23 01:16:18,930 INFO: peer16 exabgp started, output=bash: exabgp: Command not found

2019-05-23 01:16:18,930 INFO: peer16
2019-05-23 01:16:19,014 INFO: '_output_summary_cmp' polling started (interval 0.5 secs, maximum wait 30 secs)
2019-05-23 01:16:55,952 ERROR: '_output_summary_cmp' failed after 36.94 seconds
2019-05-23 01:16:56,037 ERROR: assert failed at "test_bgp_ecmp_topo1/test_bgp_convergence": BGP router network did not converge
assert json["ipv4Unicast"]["peers"]["10.0.2.107"]["state"] value is different (
--- Expected value
+++ Current value
@@ -1 +1 @@
-"Established"
+"Active")
json["ipv4Unicast"]["peers"]["10.0.2.107"]["prefixReceivedCount"] value is different (
--- Expected value
+++ Current value
@@ -1 +1 @@
-42
+0)
......

bgp-ecmp-topo1/test_bgp_ecmp_topo1.py:185: AssertionError
========================================================================= 2 failed in 68.50 seconds =========================================================================
zfg@zfg-virtual-machine:~/frr/frr/tests/topotests$

the following is my configure:
os: Ubuntu 18.04.2 LTS,
frr: frr-7.0.1-1-gcd305c097,
exabgp:
zfg@zfg-virtual-machine:/$ exabgp -v
ExaBGP : 3.4.17
Python : 2.7.15rc1 (default, Nov 12 2018, 14:31:15) [GCC 7.3.0]
Uname : #21~18.04.1-Ubuntu SMP Wed May 8 08:43:37 UTC 2019
zfg@zfg-virtual-machine:/etc/exabgp$ ll
total 32
drwxr-xr-x 2 exabgp exabgp 4096 5月 22 23:02 ./
drwxr-xr-x 136 root root 12288 5月 22 22:21 ../
-rw-r--r-- 1 exabgp exabgp 399 5月 22 22:59 exabgp.cfg
-rw-r--r-- 1 exabgp exabgp 702 5月 22 22:59 exabgp.env
-rwxr-xr-x 1 exabgp exabgp 853 5月 22 23:01 exa-receive.py*
-rwxr-xr-x 1 exabgp exabgp 1827 5月 22 23:02 exa-send.py*

[mjstapp]

I had a similar problem recently when I was switching between python2 and python3, and poking at topotests. I'd managed to harm my python2 installation of exabgp, and I had no /usr/local/bin/exabgp . I did a pip uninstall / install cycle, and the repaired installation works as expected. maybe give that a try?

[qlyoung]

Mark is correct, your exabgp installation is screwed up and needs to be fixed. This isn't a topotests bug. If you are still encountering issues after repairing your exabgp please report them in this issue.

@polychaeta autoclose in 2 days

[kwind]

I had a similar problem recently when I was switching between python2 and python3, and poking at topotests. I'd managed to harm my python2 installation of exabgp, and I had no /usr/local/bin/exabgp . I did a pip uninstall / install cycle, and the repaired installation works as expected. maybe give that a try?

Hi, Mark，I have a try that, but it still didn't work. I have /usr/local/bin/exabgp. the new error following:

zfg@zfg-virtual-machine:~/frr/frr/tests/topotests/bgp-ecmp-topo1$ sudo pytest test_bgp_ecmp_topo1.py -s -vv

2019-05-23 19:37:56,021 INFO: Running environment diagnostics
============================= test session starts =============================
platform linux2 -- Python 2.7.15rc1, pytest-3.3.2, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python2
cachedir: ../.cache
rootdir: /home/zfg/frr/frr/tests/topotests, inifile: pytest.ini
collected 2 items

test_bgp_ecmp_topo1.py::test_bgp_convergence 2019-05-23 19:37:58,769 INFO: loading topology: test_bgp_ecmp_topo1
2019-05-23 19:37:58,770 INFO: starting topology: test_bgp_ecmp_topo1
2019-05-23 19:37:58,961 INFO: r1: running version: 7.0.1-MyOwnFRRVersion-g305c097
2019-05-23 19:37:58,962 INFO: r1: waiting for zebra to start (1 seconds)
2019-05-23 19:38:00,263 INFO: starting BGP on all 20 peers (10 seconds)
2019-05-23 19:38:10,496 INFO: peer1 exabgp started, output=
2019-05-23 19:38:10,496 INFO: peer1
2019-05-23 19:38:10,670 INFO: peer20 exabgp started, output=
......
2019-05-23 19:38:13,296 INFO: peer16 exabgp started, output=
2019-05-23 19:38:13,296 INFO: peer16
2019-05-23 19:38:13,386 INFO: '_output_summary_cmp' polling started (interval 0.5 secs, maximum wait 30 secs)
2019-05-23 19:38:48,659 ERROR: '_output_summary_cmp' failed after 35.27 seconds
2019-05-23 19:38:48,661 ERROR: assert failed at "test_bgp_ecmp_topo1/test_bgp_convergence": BGP router network did not converge
assert json["ipv4Unicast"]["peers"]["10.0.1.101"]["state"] value is different (
--- Expected value
+++ Current value
@@ -1 +1 @@
-"Established"
+"Active")
json["ipv4Unicast"]["peers"]["10.0.1.101"]["prefixReceivedCount"] value is different (
--- Expected value
+++ Current value
@@ -1 +1 @@
-42
+0)
FAILED [ 50%]
test_bgp_ecmp_topo1.py::test_bgp_ecmp 2019-05-23 19:38:48,769 INFO: 'router_json_cmp' polling started (interval 0.5 secs, maximum wait 5 secs)
2019-05-23 19:38:48,873 INFO: 'router_json_cmp' succeeded after 0.10 seconds
PASSED [100%]2019-05-23 19:38:48,873 INFO: stopping topology: test_bgp_ecmp_topo1
2019-05-23 19:38:48,903 INFO: stopping "s3"
2019-05-23 19:38:48,904 INFO: stopping "s2"
2019-05-23 19:38:48,904 INFO: stopping "s1"
2019-05-23 19:38:48,904 INFO: stopping "s4"
2019-05-23 19:38:48,913 INFO: r1: stopping bgpd
2019-05-23 19:38:48,916 INFO: r1: stopping staticd
2019-05-23 19:38:48,919 INFO: r1: stopping zebra
2019-05-23 19:38:48,978 INFO: stopping "s3"
2019-05-23 19:38:48,978 INFO: stopping "s2"
2019-05-23 19:38:48,978 INFO: stopping "s1"
2019-05-23 19:38:48,978 INFO: stopping "s4"

================================== FAILURES ===================================
____________________________ test_bgp_convergence _____________________________

def test_bgp_convergence():
    "Test for BGP topology convergence"
    tgen = get_topogen()

    # Skip if previous fatal error condition is raised
    if tgen.routers_have_failure():
        pytest.skip(tgen.errors)

    # Expected result
    router = tgen.gears['r1']
    if router.has_version('<', '3.0'):
        reffile = os.path.join(CWD, 'r1/summary20.txt')
    else:
        reffile = os.path.join(CWD, 'r1/summary.txt')

    expected = json.loads(open(reffile).read())

    def _output_summary_cmp(router, cmd, data):
        """
        Runs `cmd` that returns JSON data (normally the command ends
        with 'json') and compare with `data` contents.
        """
        output = router.vtysh_cmd(cmd, isjson=True)
        if 'ipv4Unicast' in output:
            output['ipv4Unicast']['vrfName'] = \
                    output['ipv4Unicast']['vrfName'].replace(
                        'default', 'Default')
        elif 'vrfName' in output:
            output['vrfName'] = output['vrfName'].replace('default', 'Default')
        return topotest.json_cmp(output, data)

    test_func = functools.partial(
        _output_summary_cmp, router, 'show ip bgp summary json', expected)
    _, res = topotest.run_and_expect(test_func, None, count=60, wait=0.5)
    assertmsg = 'BGP router network did not converge'

  assert res is None, assertmsg

E AssertionError: BGP router network did not converge
E assert json["ipv4Unicast"]["peers"]["10.0.1.101"]["state"] value is different (
E --- Expected value
E +++ Current value
E @@ -1 +1 @@
E -"Established"
E +"Active")
E json["ipv4Unicast"]["peers"]["10.0.1.101"]["prefixReceivedCount"] value is different (
E --- Expected value
E +++ Current value
E @@ -1 +1 @@
E -42
E +0)

test_bgp_ecmp_topo1.py:157: AssertionError
===================== 1 failed, 1 passed in 57.82 seconds =====================

[polychaeta]

This issue will no longer be automatically closed.

[kwind]

Thanks for all of you! The problem has been solved. The installtion of exabgp is not correctly.