You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As discussed in #190-#191, our current parallelization scheme can be much improved by using the multiprocessing infrastructure to distribute individual profiles across a pool of workers as those workers become available, rather than sending fixed batches of profiles to independent processes. The first step to doing this effectively will be to eliminate the dependence of many of the qc tests on global variables which may not be reliable between workers. Some rough ideas on implementation are in branch no-global.
The text was updated successfully, but these errors were encountered:
As discussed in #190-#191, our current parallelization scheme can be much improved by using the multiprocessing infrastructure to distribute individual profiles across a pool of workers as those workers become available, rather than sending fixed batches of profiles to independent processes. The first step to doing this effectively will be to eliminate the dependence of many of the qc tests on global variables which may not be reliable between workers. Some rough ideas on implementation are in branch
no-global
.The text was updated successfully, but these errors were encountered: