Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use priority queue for job processing #486

Merged
merged 11 commits into from
Jul 24, 2019
Merged

use priority queue for job processing #486

merged 11 commits into from
Jul 24, 2019

Commits on Jul 18, 2019

  1. queue: submit jobs with priorities to queues 

    TODO: properly handle last_job
    @redshiftzero
    redshiftzero committed Jul 18, 2019
    Configuration menu
    Copy the full SHA
    4b2cbab View commit details
    Browse the repository at this point in the history

  2. queue: jobs must be sortable, i.e. must implement __lt__ 

    the next job in a priority queue is set by sorted(list(entries))[0]

this means that the job objects themselves must be sortable, i.e.
they must all implement __lt__.

in this commit I'm just implementing __lt__ for two job types where
we don't care about the order as long as all jobs of that type have
the same priority.

see related discussion in https://bugs.python.org/issue31145 and for
a solution using dataclasses (not in python 3.5)
    @redshiftzero
    redshiftzero committed Jul 18, 2019
    Configuration menu
    Copy the full SHA
    fbcfbea View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2019

  1. queue: workaround lack of sort stability in heapq

    @redshiftzero
    redshiftzero committed Jul 19, 2019
    Configuration menu
    Copy the full SHA
    9ad3313 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2019

  1. queue: get rid of last_job

    @redshiftzero
    redshiftzero committed Jul 22, 2019
    Configuration menu
    Copy the full SHA
    fb41b99 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2019

  1. queue: update type sigs for PriorityQueue and related ApiJob changes

    @redshiftzero
    redshiftzero committed Jul 23, 2019
    Configuration menu
    Copy the full SHA
    0a11a36 View commit details
    Browse the repository at this point in the history

  2. queue: make job_priorities dict a class variable for testability

    @redshiftzero
    redshiftzero committed Jul 23, 2019
    Configuration menu
    Copy the full SHA
    deba67d View commit details
    Browse the repository at this point in the history

  3. test: update existing tests to add job_priorities, remove last_job

    @redshiftzero
    redshiftzero committed Jul 23, 2019
    Configuration menu
    Copy the full SHA
    87bd329 View commit details
    Browse the repository at this point in the history

  4. test: new tests for ApiJob.__lt__ (ApiJobs are sortable now)

    @redshiftzero
    redshiftzero committed Jul 23, 2019
    Configuration menu
    Copy the full SHA
    94f92d7 View commit details
    Browse the repository at this point in the history

  5. test: queue jobs executed in proper order

    @redshiftzero
    redshiftzero committed Jul 23, 2019
    Configuration menu
    Copy the full SHA
    09851e6 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2019

  1. queue: address review comments from priorityqueue implementation

    @redshiftzero
    redshiftzero committed Jul 24, 2019
    Configuration menu
    Copy the full SHA
    c15426b View commit details
    Browse the repository at this point in the history

  2. refresh session whenever a reply succeeds

    Allie Crevier committed Jul 24, 2019
    Configuration menu
    Copy the full SHA
    bdd6a93 View commit details
    Browse the repository at this point in the history