occ commands to manage the job list #24551

DeepDiver1975 · 2016-05-11T07:40:23Z

ToDo:

~~introduce select for update~~ table lock in query builder
use ~~select for update to~~ lock selected job
add state update call to the job queue to mark job as selected by executor
add occ command to remove jobs from the queue

mention-bot · 2016-05-11T07:40:25Z

By analyzing the blame information on this pull request, we identified @nickvergessen, @icewind1991 and @blizzz to be potential reviewers

nickvergessen · 2016-05-17T09:43:40Z

introduce select for update in query builder

I looked a bit into SELECT ... FOR UPDATE. I assume you want to use it in lib/private/BackgroundJob/JobList.php to get the new job and lock the row until you wrote the reserved flag to the database?

DeepDiver1975 · 2016-05-17T09:49:50Z

I assume you want to use it in lib/private/BackgroundJob/JobList.php to get the new job and lock the row until you wrote the reserved flag to the database?

exactly

PVince81 · 2016-05-21T17:08:56Z

@nickvergessen please update status, PR was merged. Raise doc / QA tickets if applicable.
Thanks

nickvergessen · 2016-05-23T08:50:27Z

Done, didn't update this PR, the worker still looks good, but the table change of course is in conflict now.

PVince81 · 2016-05-31T08:38:24Z

We're past feature freeze, move to 9.2 ? @DeepDiver1975 @nickvergessen

DeepDiver1975 · 2016-05-31T08:51:31Z

We're past feature freeze, move to 9.2 ? @DeepDiver1975 @nickvergessen

agreed

PVince81 · 2016-09-14T08:46:48Z

Is there still interest in getting this feature finished ?

Not sure how this fits in with the latest changes with bkg jobs

PVince81 · 2017-04-06T14:20:21Z

moving to backlog for now unless there is interest and time in reviving this soon and getting it completed

jvillafanez · 2017-07-25T17:10:42Z

core/Command/Background/Queue/Worker.php

+		$this->logger = new CommandLogger($output);
+
+		$waitTime = $input->getOption('sleep');
+		while (true) {


Any idea how this is expected to be executed? I'm not fan of infinite loops without any escape route, and if this is going to be run as a service the only way to stop the loop would be kill it with fire (aka, sigkill signal).

I think it would be good to capture signals if possible in order to try to make a smooth exit if possible. While this is possible in between the jobs during the sleep time, the challenge is to send some kind of notice to the background job in order to stop it nicely, otherwise the execution thread will likely be running inside the background job without any chance to process the incomming signal, which will lead to the admin to kill the process forcefully.

signal handling has need added

jvillafanez · 2017-07-25T17:11:52Z

core/Command/Background/Queue/Worker.php

+	protected function execute(InputInterface $input, OutputInterface $output) {
+		$this->logger = new CommandLogger($output);
+
+		$waitTime = $input->getOption('sleep');


Maybe we should force a minimum value. I'm not sure if setting a value of 0 will keep the CPU running near 100%

addressed - there is a minimum of 1 now

DeepDiver1975 · 2018-02-15T09:24:06Z

@ownclouders rebase

ownclouders · 2018-02-15T09:24:08Z

Hey! I'm GitMate.io! This pull request is being rebased automatically. Please DO NOT push while rebase is in progress or your changes would be lost permanently ⚠️

ownclouders · 2018-02-15T09:24:32Z

Automated rebase with GitMate.io was successful! 🎉

DeepDiver1975 · 2018-04-19T12:56:50Z

@ownclouders rebase

ownclouders · 2018-04-19T12:56:53Z

Hey! I'm GitMate.io! This pull request is being rebased automatically. Please DO NOT push while rebase is in progress or your changes would be lost permanently ⚠️

ownclouders · 2018-04-19T12:57:23Z

Automated rebase with GitMate.io was successful! 🎉

DeepDiver1975 · 2018-04-19T13:00:03Z

use owncloud/market#86 which introduces a command to download the latest revocation list from the marketplace.

mmattel · 2018-06-12T10:02:21Z

@DeepDiver1975 Is thisPR still valid?

Linking:

Merged: #31630 (Adding background:queue commands: status and delete)
and
Open: #31617 (Add background:execute occ command for running cron jobs manually)

tomneedham · 2018-06-12T11:01:00Z

No longer relevant, obsolete

lock · 2019-07-31T06:02:59Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

DeepDiver1975 added the 2 - Developing label May 11, 2016

DeepDiver1975 added this to the 9.1-current milestone May 11, 2016

DeepDiver1975 force-pushed the stand-alone-job-executer branch from 26e2d12 to 79d669c Compare May 17, 2016 09:34

nickvergessen mentioned this pull request May 18, 2016

Lock jobs while executing them, to allow multiple executors to run in… #24696

Merged

2 tasks

DeepDiver1975 force-pushed the stand-alone-job-executer branch 2 times, most recently from d639831 to 4dbf074 Compare May 29, 2016 19:44

DeepDiver1975 added 3 - To Review and removed 2 - Developing labels May 30, 2016

DeepDiver1975 changed the title ~~[WIP] stand alone job executor process~~ stand alone job executor process May 30, 2016

DeepDiver1975 modified the milestones: 9.2-next, 9.1-current May 31, 2016

DeepDiver1975 force-pushed the stand-alone-job-executer branch from 4dbf074 to d1d3b1e Compare July 1, 2016 10:14

PVince81 modified the milestones: backlog, 10.0 Apr 6, 2017

DeepDiver1975 changed the title ~~stand alone job executor process~~ occ commands to manage the job list Jul 19, 2017

DeepDiver1975 self-assigned this Jul 19, 2017

DeepDiver1975 requested a review from jvillafanez July 19, 2017 10:20

DeepDiver1975 force-pushed the stand-alone-job-executer branch from 903148e to 887821d Compare July 19, 2017 10:35

jvillafanez reviewed Jul 25, 2017

View reviewed changes

DeepDiver1975 force-pushed the stand-alone-job-executer branch from 887821d to 0296b45 Compare October 5, 2017 19:53

DeepDiver1975 mentioned this pull request Nov 17, 2017

[FR] occ maintenance:bgjobs #29603

Closed

ownclouders removed the status/STALE label Feb 15, 2018

ownclouders force-pushed the stand-alone-job-executer branch from b10eca9 to 71a5414 Compare February 15, 2018 09:24

ownclouders added the status/STALE label Feb 23, 2018

ownclouders removed the status/STALE label Apr 19, 2018

ownclouders force-pushed the stand-alone-job-executer branch from 71a5414 to f274363 Compare April 19, 2018 12:57

ownclouders added the status/STALE label Apr 27, 2018

PVince81 modified the milestones: backlog, triage Apr 27, 2018

PVince81 modified the milestones: triage, maybe some day May 22, 2018

DeepDiver1975 mentioned this pull request Jun 4, 2018

Add background:execute occ command for running cron jobs manually #31617

Merged

9 tasks

Adding background:queue commands: worker, status and delete

3929f66

DeepDiver1975 force-pushed the stand-alone-job-executer branch from f274363 to 3929f66 Compare June 4, 2018 12:25

ownclouders removed the status/STALE label Jun 4, 2018

ownclouders added the status/STALE label Jun 12, 2018

ownclouders removed the status/STALE label Jun 12, 2018

tomneedham closed this Jun 12, 2018

DeepDiver1975 deleted the stand-alone-job-executer branch June 12, 2018 11:02

ownclouders mentioned this pull request Oct 23, 2018

occ provisioning command background commands #33261

Closed

1 task

lock bot locked as resolved and limited conversation to collaborators Jul 31, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

occ commands to manage the job list #24551

occ commands to manage the job list #24551

DeepDiver1975 commented May 11, 2016 •

edited

Loading

mention-bot commented May 11, 2016

nickvergessen commented May 17, 2016

DeepDiver1975 commented May 17, 2016

PVince81 commented May 21, 2016

nickvergessen commented May 23, 2016

PVince81 commented May 31, 2016

DeepDiver1975 commented May 31, 2016

PVince81 commented Sep 14, 2016

PVince81 commented Apr 6, 2017

jvillafanez Jul 25, 2017

DeepDiver1975 Oct 5, 2017

jvillafanez Jul 25, 2017

DeepDiver1975 Oct 5, 2017

DeepDiver1975 commented Feb 15, 2018

ownclouders commented Feb 15, 2018

ownclouders commented Feb 15, 2018

DeepDiver1975 commented Apr 19, 2018

ownclouders commented Apr 19, 2018

ownclouders commented Apr 19, 2018

DeepDiver1975 commented Apr 19, 2018

mmattel commented Jun 12, 2018

tomneedham commented Jun 12, 2018

lock bot commented Jul 31, 2019

occ commands to manage the job list #24551

occ commands to manage the job list #24551

Conversation

DeepDiver1975 commented May 11, 2016 • edited Loading

mention-bot commented May 11, 2016

nickvergessen commented May 17, 2016

DeepDiver1975 commented May 17, 2016

PVince81 commented May 21, 2016

nickvergessen commented May 23, 2016

PVince81 commented May 31, 2016

DeepDiver1975 commented May 31, 2016

PVince81 commented Sep 14, 2016

PVince81 commented Apr 6, 2017

jvillafanez Jul 25, 2017

Choose a reason for hiding this comment

DeepDiver1975 Oct 5, 2017

Choose a reason for hiding this comment

jvillafanez Jul 25, 2017

Choose a reason for hiding this comment

DeepDiver1975 Oct 5, 2017

Choose a reason for hiding this comment

DeepDiver1975 commented Feb 15, 2018

ownclouders commented Feb 15, 2018

ownclouders commented Feb 15, 2018

DeepDiver1975 commented Apr 19, 2018

ownclouders commented Apr 19, 2018

ownclouders commented Apr 19, 2018

DeepDiver1975 commented Apr 19, 2018

mmattel commented Jun 12, 2018

tomneedham commented Jun 12, 2018

lock bot commented Jul 31, 2019

DeepDiver1975 commented May 11, 2016 •

edited

Loading