Cron generates the same jobs in the same time

[atsareva]

Hi guys,

Recently we upgraded magento with 2.2.0 version and faced with a strange issue for generating cron jobs.

Preconditions

1. Magento 2.2.0 / 2.2.1
2. Ubuntu 16.04 lts
3. PHP 7.0.22
4. mysql Ver 14.14 Distrib 5.7.20
5. apache

Steps to reproduce

1. I was only observing cron_schedule table about one hour.

Expected result

1. cron_schedule shouldn't contain repeated jobs in the same time

Actual result

But actually it contains wrong job's schedule, therefore some jobs will be run more often.

or

More serious problems can occurs due to repeated cron jobs. E.g we have import job which should be run each 15minutes, but sometimes it's run twice a minute. Therefore reindex process is working all time and we have a lot of deadlock issues.

Thanks,
Alena Tsareva

[magento-engcom-team]

@atsareva Thank you for your report! This issue is already addressed in internal ticket MAGETWO-83782 We are closing this issue as a duplicate of #11002

[tomasvinduska]

This is not duplicit its separate problem when cron create duplicit records on shedule time.

[vlauria]

on magento 2.2.2 i have same problem. This is not duplicate. It's another problem, cron run schedule more job at times.

[ccasciotti]

I agree with @tomasvinduska and @vlauria, I'm facing the same issue using Magento 2.2.2. The cron schedules job at the same time more than once for the same schedule time.

I'll reopen issue.

[developer-vtnetzwelt]

I am facing the same issue. Can we know why it is labelled as non-issue?

[harrigo]

Yeah this is a really frustrating issue for me also and it causing some crons to fail and get stuck in a running state. These then stack leading to issues like here: #11002

This also seems similar to #10279 however this is supposed to be a duplicate of this (however different and may be the cause of the following rather) #10650 and seems like its fixed in 2.2.5 so hopefully that clears up all our issues.

For now however i have created a little script that seems to be working for me and just run via cron (i know its just a plaster on a problem and doesn't fix but easy and better than nothing if its causing you issues like mine). This basically just groups all job_codes and loops through creating an array of scheduled times for that code. If the job_code already has a job set for that time it is deleted from the schedule so that these duplicates are removed. It takes about a seconds to run so i just run it every min:

<?php 
require '/home/sites/magento/public_html/app/bootstrap.php'; 
$bootstrap = \Magento\Framework\App\Bootstrap::create(BP, $_SERVER); 
$objectManager = $bootstrap->getObjectManager(); 
$resource = $objectManager->get('Magento\Framework\App\ResourceConnection'); 
$connection = $resource->getConnection(); 
$sql = 'SELECT job_code FROM cron_schedule GROUP BY job_code;'; 
$collection = $connection->query($sql); 

foreach ($collection as $job):
	$sql = 'SELECT scheduled_id, scheduled_at FROM cron_schedule WHERE job_code = "' . $job["job_code"] . '";';
	$jobcollection = $connection->query($sql);
	$jobarray = array();
	foreach ($jobcollection as $jobinstance):
		if(in_array($jobinstance['scheduled_at'], array_column($jobarray, 'scheduled_at'))) {
			$sql = 'DELETE FROM cron_schedule WHERE schedule_id = ' . $jobinstance['schedule_id'] . ';";'
			$connection->query($sql);
		} else {
			array_push($jobarray, array(
					"schedule_id" => $jobinstance['schedule_id'],
					"scheduled_at" => $jobinstance['scheduled_at'],
				));
		}
	endforeach;
endforeach;
?>

[bpoiss]

I fixed this with overriding vendor/magento/module-cron/Model/Schedule.php tryLockJob method with:

public function tryLockJob($oldStatus = self::STATUS_PENDING)
{
    if ( $this->_getResource()->trySetJobStatusAtomic($this->getId(), self::STATUS_RUNNING, $oldStatus)) {
        $this->setStatus(self::STATUS_RUNNING);
        return true;
    }
    return false;
}

[MrDaar]

I've experienced similar issues...

[rauberdaniel]

This is a major issue here as well, since it seems that sales_send_order_emails jobs are queued multiple times at the same time which causes order emails to be sent multiple times. I wonder if it could be simply "fixed" (workaround) by adding a UNIQUE constraint to the job_code and scheduled_at columns, but I assume that might break the whole scheduling stuff.

[rauberdaniel]

I created a mysql query that takes care of the duplicate jobs and removes them. Simply run this as a cron job every minute and you should be fine:

mysql -u $DBUSER -p$DBPASSWORD -h $DBHOST $DBDATABASE -e "DELETE FROM cron_schedule WHERE schedule_id IN (SELECT schedule_id FROM (SELECT MAX(schedule_id) as schedule_id, job_code, scheduled_at, COUNT(*) as num FROM cron_schedule WHERE status = \"pending\" GROUP BY job_code, scheduled_at) as a WHERE num > 1 ORDER BY scheduled_at DESC);"

(please note you have to replace the variables with your real credentials and Database Information.)

[marcinfr]

Hi,
Issue is because some of cron schedule groups are running as separate process, each of this process execute method to generate schedules for all of groups, so each job schedule is created few times.

I have found 2 solutions:

1. in cofiguration (stores->configuration->advanced->system->cron) set "Use Separate Process" to NO for all of cron groups.

2. in code, i have found that there is already fix for this issue: fcda71d
   , its merged into 2.2-develop, but its not merged into 2.3 and 2.4.

[crtl]

I am having the same issue in Magento 2.2.4 CE.
Some cronjobs are scheduled 20 times.

[engcom-backlog-nickolas]

Hello @atsareva, thank you for your report. The fix will be available with the upcoming 2.2.7 release.

[hostep]

@engcom-backlog-nickolas: any reference to a commit or PR for the fix in 2.2.7 you are referring to would be appreciated!

I was under the impression all big cron related issues got fixed in 2.2.5, but according to your comment, that's still not true?

[engcom-backlog-nickolas]

@hostep, #12497

[hostep]

@engcom-backlog-nickolas: correct me if I'm wrong but #12497 got included in Magento 2.2.5, see: f274f03

[Ctucker9233]

@engcom-backlog-nickolas What @hostep said is correct. If it is fixed, then why are we still seeing cron jobs created one on top of the other resulting in a server deadlock? Every few days I have to truncate the cron db table to keep everything running smoothly. If I don't, I get multiple notifications like this:

[Magento\Framework\DB\Adapter\DeadlockException]
SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction, query was: DELETE FROM mgit_cron_schedule WHERE (status = 'missed') AND (job_code in ('ddg_automation_customer_subscriber_guest_sync', 'ddg_automation_importer', 'ddg_automation_status', 'ddg_automation_abandonedcarts', 'ddg_automation_reviews_and_wishlist', 'ddg_automation_campaign', 'ddg_automation_order_sync', 'ddg_automation_cleaner', 'ddg_automation_catalog_sync', 'ddg_automation_email_templates')) AND (created_at < '2018-11-05 01:59:03')

Please provide the commit that fixes this issue and will be included in 2.2.7.

[tim-reynolds-visionet]

Is there confirmation from anyone that this was fixed in 2.2.7? We are affected by this presently and would very much like to have it stop.

[hostep]

@tim-reynolds-visionet : most big cron related issues should have been fixed in 2.2.5, if you still run against such issues in 2.2.5 or 2.2.6 or 2.2.7, can you open a new issue describing what you are experiencing? Thanks! :)

[blickcreative]

I am having the exact same issue after updated Magento to 2.2.8
Is there a patch or fix available?

[rizwan-i-khan]

@crtl
I am facing the same issue, any solutions for this?

[KLeval]

Same here, our task that send the ordered products to logistic service scheduled at 3:00AM is sometime sent 3 times ... We had to deport the task on linux crontab for the moment !

[rizwan-i-khan]

@KLeval
https://magento.stackexchange.com/questions/294926/magento-2-duplicate-cron-jobs-in-cron-schedule-table
maybe this workaround can solve your issue.