Allow PBS CI to create workers

[ocaisa]

No description provided.

[ocaisa]

@AdamWlodarczyk In 1c17093 I wanted to clean up some warnings and it has now triggered a failing test that I cannot figure out

[ocaisa]

@AdamWlodarczyk I can fix the problem with 7d0fba9 but I really don't understand why that is not allowed. Not closing the local cluster seems to kick a lot of warnings

[AdamWlodarczyk]

@ocaisa this is strange. I don't understand it either. I will take a look at it on the weekend.

[AdamWlodarczyk]

@ocaisa, I'm still debugging that and this is super strange. Somehow multiple closing and therefore multiple local cluster creation influences the mpi task subprocess execution since it returns the returncode == 1.

[AdamWlodarczyk]

@ocaisa, news from the front. If you'd comment out the test_flush_and_abort test and test_mpi_deserialize_and_execute then remaining test (especially the test_mpi_wrap_execution where problems appeared) are passing. So the problem lays in those two. Probably. I'll keep on looking.

[AdamWlodarczyk]

set_task_mpi_comm and flush_and_abort functions from mpi_wrapper.py seem to make troubles.
Both have in common the import line from mpi4py import MPI.

[ocaisa]

@AdamWlodarczyk Ok, maybe there is some sense here, there may be some environment variables set by mpi4py that don't allow mpiexec to run. I'll play around with it a bit

[ocaisa]

I think I could get this to work if I could enforce an order, I'm trying to that but I'm failing

[ocaisa]

@AdamWlodarczyk Ok, I have something that works, there is a warning that I would like to ignore but I don't know how to do that.

[AdamWlodarczyk]

@ocaisa ftobia/pytest-ordering#57 mentions that warning. Well, let's count on them to fix it.

[AdamWlodarczyk]

Added just small change.

[ocaisa]

@AdamWlodarczyk This still needs a small amount of work, I tried to execute an MPI job on the slaves and it didn't work, I'll figure it out tomorrow.

[ocaisa]

@AdamWlodarczyk ok, good to go now