Use in-memory communicator to test quantile

[rongou]

The in-memory communicator can run allreduce calls in process, thus making it easier to test distributed quantile sketches. We no longer need to rely on python or dask.

@trivialfis @hcho3

[trivialfis]

Thank you for cleaning up the tests! Could you please help remove the marker in tests/pytest.ini?

[rongou]

Done.

[trivialfis]

Is this still possible?

[rongou]

Not really. Removed.

[trivialfis]

@rongou Hi, I'm running into an error when testing the c++ code on a 2-GPU machine:

[ RUN      ] GPUQuantile.AllReduceBasic
terminate called after throwing an instance of 'dmlc::Error'
  what():  [17:36:08] /home/jiamingy/Workspace/XGBoost/xgboost/src/collective/nccl_device_communicator.cuh:49: Check failed: n_uniques == world (1 vs. 2) : Multiple processes within communication group running on same CUDA device is not supported. c8be5176b263d057236621713f361bc8

Stack trace:
  [bt] (0) ./testxgboost(dmlc::LogMessageFatal::~LogMessageFatal()+0x7e) [0x55c0b3375b5e]
  [bt] (1) ./testxgboost(xgboost::collective::NcclDeviceCommunicator::NcclDeviceCommunicator(int, xgboost::collective::Communicator*)+0x78e) [0x55c0b36e2dee]

Any chance you can take a look when you are available?

[rongou]

OK, let me see if I can reproduce it.