[WORKAROUND] NC14 RC2: occ upgrade hangs on running cronjob it states #10949

eggithub · 2018-08-31T15:05:30Z

I cant give to much info right now due to lack of time but this is the command line output of the upgrade process:
sudo -u www-data php /var/www/nextcloud/occ upgrade
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Set log level to debug
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …

A cronjob was running at that time, killed the cron process but occ upgrade keeps giving this output.

When im home I provide more info (mysql/php7/etc..).
Tell me what else you need apart from the template. I use my own upgrade script. perhaps that might help.

to be continued...

Cheers

nextcloud-bot · 2018-08-31T15:17:58Z

GitMate.io thinks possibly related issues are #6960 (occ upgrade is outdated), #10918 (Occ Upgrade forces to run once a day), #9562 (Check before running "upgrade" if cronjob is still running and wait for it to finish), #8092 (Upgrade notice should not be displayed when occ is run with --output=json), and #4726 (Upgrading Nextcloud sometimes causes Cronjob Deadlocks).

snopy234 · 2018-08-31T16:16:33Z

Try to change your current version in [nextcloud path]/config/config.php to a lower version. In my case i've changed 14.0.0.17 to 14.0.0.0
Then use ./occ upgrade to upgrade your nextcloud. I hope this helps for you and sorry for my language problems. Have a nice day!
PS: Please let me know if it has worked.

kesselb · 2018-08-31T19:12:02Z

See #9992 for more information about this issue

awrede · 2018-09-01T00:41:39Z

@snopy234 I ran into the same issue. Changing version in config.php from 14.0.0.17 to 14.0.0.0 and then running ./occ upgrade made the upgrade run to completion.

eggithub · 2018-09-01T09:01:00Z

disabled the nextcloud cronjob
waited for >12 hours
then I was able to update

snopy234 · 2018-09-01T15:13:03Z

@awrede nice to hear that it's working :)

hchristo · 2018-09-15T15:09:09Z

Try to change your current version in [nextcloud path]/config/config.php to a lower version. In my case i've changed 14.0.0.17 to 14.0.0.0
Then use ./occ upgrade to upgrade your nextcloud. I hope this helps for you and sorry for my language problems. Have a nice day!
PS: Please let me know if it has worked.

Thanks, this ben Worked, Working, yes Working... THX!

jgrete · 2018-09-26T04:55:49Z

This issue is still present in 14.0.1 and the workaround is the same.
Very strange...

snopy234 · 2018-09-26T11:15:03Z

Is my workaround still working? I havn't any problems since i changed the version

jgrete · 2018-09-26T20:06:03Z

@snopy234 👍
Yepp! THanks a lot!

Ra72xx · 2018-09-27T02:34:09Z

Workaround does not work for me::
Nextcloud or one of the apps require upgrade - only a limited number of commands are available You may use your browser or the occ upgrade command to do the upgrade Set log level to debug Updating database schema Updated database Checking for update of app bookmarks in appstore An unhandled exception has been thrown: Error: Call to undefined method GuzzleHttp\Client::request() in /srv/www/htdocs/nextcloud/lib/private/Http/Client/Client.php:161 Stack trace: #0 /srv/www/htdocs/nextcloud/lib/private/App/AppStore/Fetcher/Fetcher.php(106): OC\Http\Client\Client->get('https://apps.ne...', Array) #1 /srv/www/htdocs/nextcloud/lib/private/App/AppStore/Fetcher/AppFetcher.php(80): OC\App\AppStore\Fetcher\Fetcher->fetch('', '') #2 /srv/www/htdocs/nextcloud/lib/private/App/AppStore/Fetcher/Fetcher.php(170): OC\App\AppStore\Fetcher\AppFetcher->fetch('', '') #3 /srv/www/htdocs/nextcloud/lib/private/Installer.php(385): OC\App\AppStore\Fetcher\Fetcher->get() #4 /srv/www/htdocs/nextcloud/lib/private/Updater.php(445): OC\Installer->isUpdateAvailable('bookmarks') #5 /srv/www/htdocs/nextcloud/lib/private/Updater.php(256): OC\Updater->upgradeAppStoreApps(Array) #6 /srv/www/htdocs/nextcloud/lib/private/Updater.php(126): OC\Updater->doUpgrade('14.0.1.1', '14.0.0.1') #7 /srv/www/htdocs/nextcloud/core/Command/Upgrade.php(262): OC\Updater->upgrade() #8 /srv/www/htdocs/nextcloud/3rdparty/symfony/console/Command/Command.php(251): OC\Core\Command\Upgrade->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #9 /srv/www/htdocs/nextcloud/3rdparty/symfony/console/Application.php(946): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #10 /srv/www/htdocs/nextcloud/3rdparty/symfony/console/Application.php(248): Symfony\Component\Console\Application->doRunCommand(Object(OC\Core\Command\Upgrade), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #11 /srv/www/htdocs/nextcloud/3rdparty/symfony/console/Application.php(148): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #12 /srv/www/htdocs/nextcloud/lib/private/Console/Application.php(213): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #13 /srv/www/htdocs/nextcloud/console.php(96): OC\Console\Application->run() #14 /srv/www/htdocs/nextcloud/occ(11): require_once('/srv/www/htdocs...')

Here the bookmarks app causes the trouble, but before the activity and accessibility app caused the problem. I disabled them.

How to go on?

PS: Nextcloud causes trouble at almost every update. As much as I like it, it would be great if the update process would me much more robust.

Ra72xx · 2018-09-27T02:42:41Z

Disabling files_external_gdrive did the job... (wasn't there a time when nextcloud disabled all but the core apps by itself before updating?)

jgrete · 2018-09-27T10:41:05Z

I hadn't activated that plugin, so the cause of your trouble was different.
Yes, I'm a little bit fed up also with these ever crahing updates. On most updates cycles I wasn't able to do a smooth update on my home installation. I could recover from all chrashes, but on my companies installation I'm much more conservative when it comes to nextcloud updates...

cvandesande · 2018-10-01T19:08:03Z

Also got this after the 14.0.1 upgrade. The workaround from @hchristo worked

MorrisJobke · 2018-10-01T19:29:29Z

A cronjob was running at that time, killed the cron process but occ upgrade keeps giving this output.

Killing the cron job caused the trouble actually here, because we store a "is something running" timestamp for each cron job and it only get updated by the job itself once it finished. The cronjob also just finishes the currently running one and then updates the DB table.

Best would be the SQL query from #9992 (comment), but be aware that if the cron.php is still running somewhere that this could cause some issues.

MorrisJobke · 2018-10-01T19:30:20Z

Also this would be helpful to fix the cause of this issue:

SELECT * FROM oc_jobs WHERE reserved_at <> 0;

snopy234 · 2018-10-03T10:45:39Z

@cvandesande which workaround from @hchristo do you mean?

cvandesande · 2018-10-03T10:56:07Z

@snopy234 where you downgrade the version in config.php to 14.0.0.0

michidk · 2018-10-05T20:21:38Z

Same issue. Query (@MorrisJobke):

SELECT * FROM oc_jobs WHERE reserved_at <> 0;

oc_jobs

id	class	argument	last_run	last_checked	reserved_at	execution_duration
748	OC\Preview\BackgroundCleanupJob	null	1538766002	1538766001	1538766001	0

Also tried updating Nextcloud manually, following this guide: https://docs.nextcloud.com/server/13/admin_manual/maintenance/manual_upgrade.html - with the same result: stuck, waiting for cron to finish.

Is there an hotfix to get Nextcloud working again?

michidk · 2018-10-05T21:21:57Z

Fixed it by setting the results of the query SELECT * FROM oc_jobs WHERE reserved_at <> 0; at field reserved_at to 0.

MorrisJobke · 2018-10-08T13:22:33Z

cc @rullzer for the background job in previews :/

GLLM · 2018-10-11T11:48:40Z

Hi all, I also have this issue.

I got it while upgrading from 14.0.0 to 14.0.1. Solved it by changing version in config.php from 14.0.0.17 to 14.0.0.0 and then running ./occ upgrade

today, upgraded from 14.0.1 to 14.0.2 and also got it.
I'll try @michidk workaround ...

Feedback/Edit : solved by doing SELECT * FROM oc_jobs WHERE reserved_at <> 0;
and setting it to 0 on the one line that I got.

Thanks

moretocome · 2018-10-12T08:42:38Z

Saw this error for the first time when upgrading from 14.0.2 to 14.0.3 with commandline updater.phar. Solved by changing version to 14.0.0.0 in config.php

pkmkrishnakumar-zz · 2018-10-12T08:48:19Z

@GLLM

How to do this please give in detail here (For novice user)

This error occured during 14 ---> 14.1 & now 14.1 -----> 14.2

MorrisJobke · 2018-10-12T09:22:13Z

How to do this please give in detail here (For novice user)

This error occured during 14 ---> 14.1 & now 14.1 -----> 14.2

See my comment in #10949 (comment)

GLLM · 2018-10-12T10:27:31Z

Hi all
Got the same issue once again from 14.0.2 > 14.0.3

Solved (again) it by looking in the oc_jobs table, found the line with field reserved_at <> 0, and set it up back to 0.

Thanks

benneti · 2018-10-12T12:40:44Z

Hi,
Got the same issue (for the first time) from 14.0.2 > 14.0.3,
The fix suggested by @snopy234 worked, so thank you for this.

MorrisJobke · 2018-10-12T14:31:59Z

The big question is, why there are those entries. What type of cron job do you have? Do you run them in ajax mode?

eggithub · 2018-10-12T14:41:16Z

I just use the Cron option for background jobs.

Could it be related to the News App? I reinstalled that before I updated the nextcloud instance. I know from experience that News takes a long time to update after installing a new version because it happens dat my feeds aren't accessible for hours. And since this app also uses cron for updates, could it be that it is holding up something. Just guessing... But can I check this Cron update behaviour somewhere in the logs?

Running this nextcloud instance on an RPI3...

MorrisJobke · 2018-10-12T14:42:19Z

Could it be related to the News App? I reinstalled that before I updated the nextcloud instance. I know from experience that News takes a long time to update after installing a new version because it happens dat my feeds aren't accessible for hours. And since this app also uses cron for updates, could it be that it is holding up something. Just guessing... But can I check this Cron update behaviour somewhere in the logs?

Not really - except it is a news background job, but it seems to be many different ones in here.

benneti · 2018-10-12T15:00:16Z

I also use cron (provided by debian stable package) and have the news app installed

enoch85 · 2018-10-16T19:22:24Z

Sorry for bumping, just got this on one of the servers I maintain going from 14.0.1 -- 14.0.3.

root@cloud:/var/scripts# sudo -u www-data php /var/www/nextcloud/occ upgrade
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Set log level to debug
Turned on maintenance mode
Waiting for cron to finish (checks again in 5 seconds) …
Waiting for cron to finish (checks again in 5 seconds) …
^C
root@cloud:/var/scripts# sudo -u postgr^C
root@cloud:/var/scripts# mysql -u root 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 922369
Server version: 5.7.23-0ubuntu0.16.04.1 (Ubuntu)

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SELECT * FROM oc_jobs WHERE reserved_at <> 0;
ERROR 1046 (3D000): No database selected
mysql> use nextcloud_db
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> SELECT * FROM oc_jobs WHERE reserved_at <> 0;
+----+------------------------------------------------+----------+------------+--------------+-------------+--------------------+
| id | class                                          | argument | last_run   | last_checked | reserved_at | execution_duration |
+----+------------------------------------------------+----------+------------+--------------+-------------+--------------------+
| 20 | OCA\Survey_Client\BackgroundJobs\MonthlyReport | null     | 1539676802 |   1539676802 |  1539676802 |                 13 |
+----+------------------------------------------------+----------+------------+--------------+-------------+--------------------+
1 row in set (0,00 sec)

mysql> SELECT * FROM oc_jobs WHERE reserved_at <> 0;
+----+------------------------------------------------+----------+------------+--------------+-------------+--------------------+
| id | class                                          | argument | last_run   | last_checked | reserved_at | execution_duration |
+----+------------------------------------------------+----------+------------+--------------+-------------+--------------------+
| 20 | OCA\Survey_Client\BackgroundJobs\MonthlyReport | null     | 1539676802 |   1539676802 |  1539676802 |                 13 |
+----+------------------------------------------------+----------+------------+--------------+-------------+--------------------+
1 row in set (0,00 sec)

mysql> UPDATE oc_jobs SET reserved_at=0
    -> ;
Query OK, 1 row affected (0,00 sec)
Rows matched: 24  Changed: 1  Warnings: 0

mysql> SELECT * FROM oc_jobs WHERE reserved_at <> 0;
Empty set (0,00 sec)

mysql> exit
Bye
root@cloud:/var/scripts# nano nextcloud_update.sh 
root@cloud:/var/scripts# nano nextcloud_update.sh 
root@cloud:/var/scripts# bash nextcloud_update.sh 
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Set log level to debug
Updating database schema
Updated database
Disabled incompatible app: bruteforcesettings
Disabled incompatible app: calendar
Disabled incompatible app: contacts
Disabled incompatible app: impersonate
Checking for update of app accessibility in appstore
Checked for update of app "accessibility" in appstore 
Checking for update of app activity in appstore
Checked for update of app "activity" in appstore 
Checking for update of app cloud_federation_api in appstore
Checked for update of app "cloud_federation_api" in appstore 
Checking for update of app comments in appstore
Checked for update of app "comments" in appstore 
Checking for update of app dav in appstore
Checked for update of app "dav" in appstore 
Checking for update of app federatedfilesharing in appstore
Checked for update of app "federatedfilesharing" in appstore 
Checking for update of app federation in appstore
Checked for update of app "federation" in appstore 
Checking for update of app files in appstore
Checked for update of app "files" in appstore 
Checking for update of app files_external in appstore
Checked for update of app "files_external" in appstore

All good now.

Ra72xx · 2018-10-16T19:46:50Z

Any idea how to get rid of this? I'm kind of sick of going into manual mode to repair NC after every update. Please please make the updates more robust!!

GAS85 · 2018-10-22T13:04:21Z

Meet the same issue during update to 14.0.3

MorrisJobke · 2018-11-02T08:34:43Z

We removed this via #12188

Will be shipped in 14.0.4

sscholl · 2018-11-19T13:20:52Z

@snopy234 Worked indeed!

snopy234 · 2018-11-30T16:00:04Z

Thank you to share this with us @sscholl and @benneti :)

btw i found the solution for the update error from 15 beta 2 to 15 rc 2 ... what are you doing with the updater at each update @nextcloud-bot @karlitschek :(

eggithub changed the title ~~NC14 RC2: occ upgrade hangs running cronjob~~ NC14 RC2: occ upgrade hangs on running cronjob it states Aug 31, 2018

eggithub changed the title ~~NC14 RC2: occ upgrade hangs on running cronjob it states~~ [WORKAROUND] NC14 RC2: occ upgrade hangs on running cronjob it states Sep 1, 2018

tflidd added the 14-beta-feedback label Sep 2, 2018

nextcloud-bot mentioned this issue Sep 7, 2018

Upgrade all apps after/during running the upgrader with occ #11102

Closed

MorrisJobke added the needs info label Oct 1, 2018

Co0p3r21 mentioned this issue Oct 5, 2018

Upgrade from 14.0.1 to 14.0.2 RC1 waiting for cron to finish #11630

Closed

linucksrox mentioned this issue Oct 9, 2018

Stuck Waiting for cron to finish when upgrading from 14.0 to 14.0.1 #11392

Closed

MorrisJobke mentioned this issue Oct 12, 2018

Update to 14.02 from 14.01 various error about "Delete old files" #11777

Closed

MorrisJobke mentioned this issue Oct 12, 2018

Cronjob blocking update (forever) #9992

Closed

MorrisJobke closed this as completed Nov 2, 2018

MorrisJobke modified the milestones: Nextcloud 14.0.4, Nextcloud 15 Nov 2, 2018

janvlug mentioned this issue Feb 25, 2024

Error when upgrading nextcloud: table nextcloud.oc_jobs doesn't exist nextcloud/nextcloudpi#1818

Closed

[WORKAROUND] NC14 RC2: occ upgrade hangs on running cronjob it states #10949

[WORKAROUND] NC14 RC2: occ upgrade hangs on running cronjob it states #10949

Comments

eggithub commented Aug 31, 2018

nextcloud-bot commented Aug 31, 2018

snopy234 commented Aug 31, 2018 • edited Loading

kesselb commented Aug 31, 2018

awrede commented Sep 1, 2018

eggithub commented Sep 1, 2018

snopy234 commented Sep 1, 2018

hchristo commented Sep 15, 2018

jgrete commented Sep 26, 2018

snopy234 commented Sep 26, 2018

jgrete commented Sep 26, 2018

Ra72xx commented Sep 27, 2018

Ra72xx commented Sep 27, 2018

jgrete commented Sep 27, 2018

cvandesande commented Oct 1, 2018

MorrisJobke commented Oct 1, 2018

MorrisJobke commented Oct 1, 2018

snopy234 commented Oct 3, 2018 • edited Loading

cvandesande commented Oct 3, 2018 • edited Loading

michidk commented Oct 5, 2018 • edited Loading

michidk commented Oct 5, 2018 • edited Loading

MorrisJobke commented Oct 8, 2018

GLLM commented Oct 11, 2018 • edited Loading

moretocome commented Oct 12, 2018

pkmkrishnakumar-zz commented Oct 12, 2018 • edited Loading

MorrisJobke commented Oct 12, 2018

GLLM commented Oct 12, 2018

benneti commented Oct 12, 2018

MorrisJobke commented Oct 12, 2018

eggithub commented Oct 12, 2018

MorrisJobke commented Oct 12, 2018

benneti commented Oct 12, 2018

enoch85 commented Oct 16, 2018

Ra72xx commented Oct 16, 2018 • edited Loading

GAS85 commented Oct 22, 2018

MorrisJobke commented Nov 2, 2018

sscholl commented Nov 19, 2018

snopy234 commented Nov 30, 2018 • edited Loading

snopy234 commented Aug 31, 2018 •

edited

Loading

snopy234 commented Oct 3, 2018 •

edited

Loading

cvandesande commented Oct 3, 2018 •

edited

Loading

michidk commented Oct 5, 2018 •

edited

Loading

michidk commented Oct 5, 2018 •

edited

Loading

GLLM commented Oct 11, 2018 •

edited

Loading

pkmkrishnakumar-zz commented Oct 12, 2018 •

edited

Loading

Ra72xx commented Oct 16, 2018 •

edited

Loading

snopy234 commented Nov 30, 2018 •

edited

Loading