-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use priority queue for job processing #486
Conversation
TODO: properly handle last_job
the next job in a priority queue is set by sorted(list(entries))[0] this means that the job objects themselves must be sortable, i.e. they must all implement __lt__. in this commit I'm just implementing __lt__ for two job types where we don't care about the order as long as all jobs of that type have the same priority. see related discussion in https://bugs.python.org/issue31145 and for a solution using dataclasses (not in python 3.5)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will run through some tests in the morning but looks good for the initial pass-through
aaebfd2
to
09851e6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This bug on master now does not occur. Please confirm that this PR fixes it.
However a similar but different bug is happening, STR:
- log into the client in Qubes
- cut connection to server
- send a few replies and wait until they timeout (you'll see the red bars)
- send a message as the source with the failed replies
- press refresh
- reconnect server
- wait until refresh is over
- send another message as the source and refresh
Expected:
The replies will be sent and no longer red. The messages from the source will show up in the conversation view.
Actual:
The replies are still red (failed). The messages from the source do not show up in the conversation view. When you close the client and reopen the red replies are blue and the messages appear, so this seems to only be a UI refresh issue.
Wait for at least one to time out. The actual behavior happens both when a reply is in the middle of being processed and when all replies are finished being processed and failed. |
8952d13
to
bdd6a93
Compare
I added a commit to test and fix the small UI bug I mentioned in the comment: bdd6a93 Now when replies begin succeeding again, the previous replies that failed show up as successful and new source messages appear. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At this point, all comments have been addressed so I think it's time to
Description
Fixes #423.
After investigation, #423 is a worthwhile architectural change as we get some major benefits:
Queue
with Python'sPriorityQueue
.RunnableQueue.last_job
while continuing to preserve job ordering in cases where jobs timeout/queue execution pauses.Other notes to be aware of:
PriorityQueue
is that it does not preserve FIFO ordering of objects with equal priorities. A counter was added to our job objects to ensure that the sort order of objects with equal priorities is stable (added notes inline and in commit messages to make this clear).Test Plan
I would follow the standard test plan for this one
Checklist
If these changes modify code paths involving cryptography, the opening of files in VMs, network (via the RPC service) traffic, or fine tuning of the graphical user interface, Qubes testing is required. Please check as applicable: