-
Notifications
You must be signed in to change notification settings - Fork 11.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[6.x] Use SKIP LOCKED for mysql 8.1 and pgsql 9.5 queue workers #31287
Conversation
$databaseEngine = $this->database->getPdo()->getAttribute(PDO::ATTR_DRIVER_NAME); | ||
$databaseVersion = $this->database->getPdo()->getAttribute(PDO::ATTR_SERVER_VERSION); | ||
|
||
if ($databaseEngine == 'mysql' && version_compare($databaseVersion, '8.0.1', '>=') || |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Kinda feels like a hack we need to look at the engine here. Feels like this should be the responsibility of the database component.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it is a hack :)
This breaks compatibility with MSSQL server @taylorotwell. It does not support the "SELECT FOR UPDATE" syntax. Whenever I now try to run the queue with
Queues are unusable in MSSQL now because of this commit. It should be reverted until a proper solution that covers all currently supported databases is implemented. |
@Krunch locks are skipped for SqlServer, same as Sqlite |
Not with this commit @themsaid. If I put |
With this commit the SQL query generated on MSSQL is this: Before this commit: As you can see the locking part is drastically different. @themsaid |
We're experiencing the same problem with SQLServer after updating to 6.14, the queue-workers crash and the queue is left unprocessed. We reverted back to 6.13.1 which works. |
return 'FOR UPDATE SKIP LOCKED'; | ||
} | ||
|
||
return 'FOR UPDATE'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
'FOR UPDATE' does not seem to work in MSSQL while not in a cursor.
I opened Issue #31368 for this specific bug. |
Using
SKIP LOCKED
on MySQL 8.0.19 I was able to get 516,783 jobs processed by 50 workers in 25 minutes on a 3GB RAM - 1 CPU core DigitalOcean server. No deadlocks.Running the same test without
SKIP LOCKED
I started getting deadlocks after a few minutes.