Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create more or faster workers #533

Closed
matthew-white opened this issue Jul 20, 2022 · 1 comment · Fixed by #1064
Closed

Create more or faster workers #533

matthew-white opened this issue Jul 20, 2022 · 1 comment · Fixed by #1064
Assignees

Comments

@matthew-white
Copy link
Member

At the moment, run-server.js spins up just one worker (I think). However, we want run-server.js to spin up multiple workers. (We've talked about 4 being a good number.)

On most machines, start-odk.sh in the central repository will use pm2 to spin up multiple instances. However, it seems like it'd be good to have more than one worker per instance.

@matthew-white
Copy link
Member Author

Right now, I think a Backend worker can wait up to 25 seconds to check for work to do:

// main loop. kicks off a check and attempts to process the result of the check.
// if there was something to do, takes a break while that happens; the runner will
// call back into the scheduler when it's done.
// if there was nothing to do, immediately schedules a subsequent check at a capped
// exponential backoff rate.
const worker = (container, jobMap = defaultJobMap, defaultDelay = 3000) => {
let enable = true; // we allow the caller to abort for testing.
const check = checker(container);
const run = runner(container, jobMap);
const status = new Status();
const withStatus = (x, chain) => { status.set(x); return chain; };
const report = reporter(container.Sentry);
// this is the main loop, which should already try to hermetically catch its own
// failures and restart itself.
const now = (delay = defaultDelay) => {
if (!enable) return;
const wait = () => { waitFor(min(delay * 2, 25000)); }; // eslint-disable-line no-use-before-define

25 seconds seems like a fair amount of time to wait. Though it's also not common for a user to be actively waiting for an event to be processed: most of the time, async work isn't needed immediately. But should we decrease 25 to a smaller number? Maybe that wouldn't be needed if we added more workers, as that would decrease the average wait time.

@matthew-white matthew-white changed the title Spin up multiple workers Create more or faster workers Sep 19, 2023
@github-project-automation github-project-automation bot moved this to 🕒 backlog in ODK Central Sep 26, 2023
@matthew-white matthew-white self-assigned this Dec 9, 2023
@matthew-white matthew-white moved this from 🕒 backlog to ✏️ in progress in ODK Central Dec 12, 2023
matthew-white added a commit that referenced this issue Dec 12, 2023
@github-project-automation github-project-automation bot moved this from ✏️ in progress to ✅ done in ODK Central Dec 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: ✅ done
Development

Successfully merging a pull request may close this issue.

1 participant