Cron generates the same jobs in the same time

[atsareva]

Hi guys,

Recently we upgraded magento with 2.2.0 version and faced with a strange issue for generating cron jobs.

Preconditions

1. Magento 2.2.0 / 2.2.1
2. Ubuntu 16.04 lts
3. PHP 7.0.22
4. mysql Ver 14.14 Distrib 5.7.20
5. apache

Steps to reproduce

1. I was only observing cron_schedule table about one hour.

Expected result

1. cron_schedule shouldn't contain repeated jobs in the same time

Actual result

But actually it contains wrong job's schedule, therefore some jobs will be run more often.

or

More serious problems can occurs due to repeated cron jobs. E.g we have import job which should be run each 15minutes, but sometimes it's run twice a minute. Therefore reindex process is working all time and we have a lot of deadlock issues.

Thanks,
Alena Tsareva

[magento-engcom-team]

@atsareva Thank you for your report! This issue is already addressed in internal ticket MAGETWO-83782 We are closing this issue as a duplicate of #11002

[tomasvinduska]

This is not duplicit its separate problem when cron create duplicit records on shedule time.

[vlauria]

on magento 2.2.2 i have same problem. This is not duplicate. It's another problem, cron run schedule more job at times.

[ccasciotti]

I agree with @tomasvinduska and @vlauria, I'm facing the same issue using Magento 2.2.2. The cron schedules job at the same time more than once for the same schedule time.

I'll reopen issue.

[developer-vtnetzwelt]

I am facing the same issue. Can we know why it is labelled as non-issue?

[harrigo]

Yeah this is a really frustrating issue for me also and it causing some crons to fail and get stuck in a running state. These then stack leading to issues like here: #11002

This also seems similar to #10279 however this is supposed to be a duplicate of this (however different and may be the cause of the following rather) #10650 and seems like its fixed in 2.2.5 so hopefully that clears up all our issues.

For now however i have created a little script that seems to be working for me and just run via cron (i know its just a plaster on a problem and doesn't fix but easy and better than nothing if its causing you issues like mine). This basically just groups all job_codes and loops through creating an array of scheduled times for that code. If the job_code already has a job set for that time it is deleted from the schedule so that these duplicates are removed. It takes about a seconds to run so i just run it every min:

<?php 
require '/home/sites/magento/public_html/app/bootstrap.php'; 
$bootstrap = \Magento\Framework\App\Bootstrap::create(BP, $_SERVER); 
$objectManager = $bootstrap->getObjectManager(); 
$resource = $objectManager->get('Magento\Framework\App\ResourceConnection'); 
$connection = $resource->getConnection(); 
$sql = 'SELECT job_code FROM cron_schedule GROUP BY job_code;'; 
$collection = $connection->query($sql); 

foreach ($collection as $job):
	$sql = 'SELECT scheduled_id, scheduled_at FROM cron_schedule WHERE job_code = "' . $job["job_code"] . '";';
	$jobcollection = $connection->query($sql);
	$jobarray = array();
	foreach ($jobcollection as $jobinstance):
		if(in_array($jobinstance['scheduled_at'], array_column($jobarray, 'scheduled_at'))) {
			$sql = 'DELETE FROM cron_schedule WHERE schedule_id = ' . $jobinstance['schedule_id'] . ';";'
			$connection->query($sql);
		} else {
			array_push($jobarray, array(
					"schedule_id" => $jobinstance['schedule_id'],
					"scheduled_at" => $jobinstance['scheduled_at'],
				));
		}
	endforeach;
endforeach;
?>

[bpoiss]

I fixed this with overriding vendor/magento/module-cron/Model/Schedule.php tryLockJob method with:

public function tryLockJob($oldStatus = self::STATUS_PENDING)
{
    if ( $this->_getResource()->trySetJobStatusAtomic($this->getId(), self::STATUS_RUNNING, $oldStatus)) {
        $this->setStatus(self::STATUS_RUNNING);
        return true;
    }
    return false;
}

[MrDaar]

I've experienced similar issues...

[rauberdaniel]

This is a major issue here as well, since it seems that sales_send_order_emails jobs are queued multiple times at the same time which causes order emails to be sent multiple times. I wonder if it could be simply "fixed" (workaround) by adding a UNIQUE constraint to the job_code and scheduled_at columns, but I assume that might break the whole scheduling stuff.

[rauberdaniel]

I created a mysql query that takes care of the duplicate jobs and removes them. Simply run this as a cron job every minute and you should be fine:

mysql -u $DBUSER -p$DBPASSWORD -h $DBHOST $DBDATABASE -e "DELETE FROM cron_schedule WHERE schedule_id IN (SELECT schedule_id FROM (SELECT MAX(schedule_id) as schedule_id, job_code, scheduled_at, COUNT(*) as num FROM cron_schedule WHERE status = \"pending\" GROUP BY job_code, scheduled_at) as a WHERE num > 1 ORDER BY scheduled_at DESC);"

(please note you have to replace the variables with your real credentials and Database Information.)