tools/testers/pg_prove_tests.sh fails when PostgreSQL port is not passed

[landryb]

with 2.6.3, i had no issue running the previous testing suite on OpenBSD. with 3.0.0, i made ports for pgtap and pg_prove, which do pass their own test suite without issue (w/ psql 12).

but running tools/testers/pg_prove_tests.sh postgres from the toplevel dir against a throwaway psql instance setup by our ports infrastructure, i get this:

server started
RELEASE_TYPE b

cd ./tools/testers/
psql -f setup_db.sql
OK
PANIC: could not determine iterator for input  at /usr/libdata/perl5/App/Prove.pm line 548.

from my understanding, the testing db is inited fine, but the pg_prove invocation badly fails. If i remove the --quiet from the pg_prove invocation, same thing without much more details.

i suppose this testing suite is supposed to work ? something missing in my env ?

[cvvergara]

PANIC: could not determine iterator for input  at /usr/libdata/perl5/App/Prove.pm line 548.

The panic is generated in pgtap, that is not pgRouting code.

I don't know your environment, never used OpenBSD.

• is pgtap installed correctly?

When I installed postgreSQL12 and had postgreSQL 10 (have different postgresql versions) it happened for example that:

pgsql --version

didnt match the same number as

SELECT version();

I cant not replicate your problem.
But maybe you can try to replicate how we test on github actions:
https://github.com/pgRouting/pgrouting/blob/release-3.0/.github/workflows/ubuntu.yml

or if you have a fork, then maybe you can add an action that replicates your problem.

[landryb]

after debugging a bit with @afresh1 (our resident perl expert) it turns out this is caused by $PGPORT being empty in tools/testers/pg_prove_tests.sh (i call the script with only the database username, and using our ksh as interpreter, not bash), which ends up passing an empty extra string to prove, which confuses it:

$prove ""
PANIC: could not determine iterator for input  at /usr/libdata/perl5/App/Prove.pm line 549.

the tests run if i pass the 5432 default port to the script. Some tests fail, some seem to kill the psql server, i'll report on those later on..

[landryb]

yeah, some tests crash the server:

psql:../../pgtap/vrp_basic/no_crash_test-vrpOneDepot.sql:43: server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
psql:../../pgtap/vrp_basic/no_crash_test-vrpOneDepot.sql:43: fatal: connection to server was lost

same thing with pgtap/pickDeliver/pickDeliver/innerQuery.sql

[landryb]

once i've removed those two tests killing the server, here's the complete report of the testsuite:

Test Summary Report
-------------------
../../pgtap/alpha_shape/s_shape-test.sql                                                             (Wstat: 0 Tests: 167 Failed: 0)
  TODO passed:   95
../../pgtap/common/has_v3_signatures.sql                                                             (Wstat: 0 Tests: 92 Failed: 2)
  Failed tests:  34, 39
../../pgtap/dijkstra/dijkstraCost-group-innerQuery.sql                                               (Wstat: 0 Tests: 226 Failed: 2)
  Failed tests:  5, 10
../../pgtap/dijkstra/dijkstraCost-types-check.sql                                                    (Wstat: 0 Tests: 43 Failed: 2)
  Failed tests:  9-10
../../pgtap/dijkstra/dijkstraCost_empty_combinations_empty_result.test.sql                           (Wstat: 768 Tests: 0 Failed: 0)
  Non-zero exit status: 3
  Parse errors: Bad plan.  You planned 1 tests but ran 0.
../../pgtap/dijkstra/dijkstra_empty_combinations_empty_result.test.sql                               (Wstat: 768 Tests: 0 Failed: 0)
  Non-zero exit status: 3
  Parse errors: Bad plan.  You planned 1 tests but ran 0.
../../pgtap/dijkstra/manyToMany_equiv_combinations.test.sql                                          (Wstat: 768 Tests: 0 Failed: 0)
  Non-zero exit status: 3
  Parse errors: Bad plan.  You planned 2 tests but ran 0.
../../pgtap/dijkstra/no_crash_test-dijkstraCost.sql                                                  (Wstat: 768 Tests: 5 Failed: 0)
  Non-zero exit status: 3
  Parse errors: Bad plan.  You planned 81 tests but ran 5.
../../pgtap/topology/createVerticesTable.test.sql                                                    (Wstat: 0 Tests: 95 Failed: 0)
  TODO passed:   33-35
../../pgtap/trsp/trsp/innerQuery.sql                                                                 (Wstat: 0 Tests: 162 Failed: 0)
  TODO passed:   4-6, 10-12, 16-18, 34-36, 40-42, 46-48
                58-60, 64-66, 70-72, 88-90, 94-96, 100-102
                112-114, 118-120, 124-126, 142-144, 148-150
                154-156
../../pgtap/trsp/trspViaEdges/innerQuery.sql                                                         (Wstat: 0 Tests: 162 Failed: 0)
  TODO passed:   4-6, 10-12, 16-18, 34-36, 40-42, 46-48
                58-60, 64-66, 70-72, 88-90, 94-96, 100-102
                112-114, 118-120, 124-126, 142-144, 148-150
                154-156
../../pgtap/trsp/trspViaVertices/innerQuery.sql                                                      (Wstat: 0 Tests: 162 Failed: 0)
  TODO passed:   4-6, 10-12, 16-18, 34-36, 40-42, 46-48
                58-60, 64-66, 70-72, 88-90, 94-96, 100-102
                112-114, 118-120, 124-126, 142-144, 148-150
                154-156
Files=293, Tests=30732, 620 wallclock secs ( 6.05 usr  4.37 sys +  1.54 cusr  7.58 csys = 19.54 CPU)

[cvvergara]

@bob-beck
About this ones, the functions tested are of 3.1 that is where combinations were added

./../pgtap/dijkstra/dijkstraCost_empty_combinations_empty_result.test.sql                           (Wstat: 768 Tests: 0 Failed: 0)
  Non-zero exit status: 3
  Parse errors: Bad plan.  You planned 1 tests but ran 0.
../../pgtap/dijkstra/dijkstra_empty_combinations_empty_result.test.sql                               (Wstat: 768 Tests: 0 Failed: 0)
  Non-zero exit status: 3
  Parse errors: Bad plan.  You planned 1 tests but ran 0.
../../pgtap/dijkstra/manyToMany_equiv_combinations.test.sql                                          (Wstat: 768 Tests: 0 Failed: 0)
  Non-zero exit status: 3
  Parse errors: Bad plan.  You planned 2 tests but ran 0.

Note that changes between minors add functions So 3.1 has functions that 3.0 does not have and 3.2 has functions that 3.1 does not have. Therefore running 3.1 tests on 3.0 version will create that kind of output

other topic

Because of these changes:

• We made a lot of fixes on the tests in pgtap tests: need to set the required input graph #1630
• We tried yet again to fix this server crash pgr_chinesePostman crash on test data #1300
  And the fact that we are about to release 3.0.3, 3.1.1

Can you check again the tests on openbsd

• 3.0 on release-3.0 branch
• 3.1 in master branch
• 3.2-dev on develop branch
  And open an issue if some crash the server and/or a test fails (mention openbsd on your title)

Of course it would be great to have a way to do a CI on openbsd, we are open for volunteers for that task :-)
otherwise we will be asking the favor of checking the tests when we fix the issues opened.

[landryb]

@bob-beck has nothing to do with that, i commited openbsd/ports@d4bd6a538 to cvs and he's managing the cvs to git conversion/github mirror.

will try to run all tests on those 3 branches from:

• dd17e25 for release-3.0
• 13923bd for master/3.1
• d9294ab for develop/3.2

but that doesnt look good as 3.0 failed the same (ie server dies):

psql:../../pgtap/pickDeliver/pickDeliver/innerQuery.sql:255: server closed the connection unexpectedly                                                                                                                                         
        This probably means the server terminated abnormally                                                                                                                                                                                   
        before or while processing the request.                                                                                                                                                                                                
psql:../../pgtap/pickDeliver/pickDeliver/innerQuery.sql:255: fatal: connection to server was lost            

note that it isnt the test fixed in #1300.

[cvvergara]

@landryb

Can you change the title to:
tools/testers/pg_prove_tests.sh fails when PostgreSQL port is not passed

[cvvergara]

The default port when port is not passed will be -p 5432

[cvvergara]

I took the liberty of changing the title.
Because I saw your workaround which was to add the port on the call to that file

I saw the way you are avoiding the failing tests on openBSD, and the corresponding issue is open.

[landryb]

thanks !