Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

can't scan #738

Closed
ippocratis opened this issue Jan 29, 2023 · 7 comments
Closed

can't scan #738

ippocratis opened this issue Jan 29, 2023 · 7 comments
Labels
ARM

Comments

@ippocratis
Copy link

ippocratis commented Jan 29, 2023 •
edited
Loading

running on raspberry pi 400

been using an older image for a long time

maybe a torchvision update broke something for arm devices?

sudo docker exec --user root backend python3 manage.py scan

throws error

/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
@ippocratis
Copy link
Author

downgraded torchvision on the running "backend"

docker exec -it backend pip install torchvision==0.11.3

full error of sudo docker exec --user root backend python3 manage.py scan

output

/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
use SECRET_KEY from env
    return self.cursor.execute(sql, params)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "api_longrunningjob_pkey"
DETAIL:  Key (id)=(16) already exists.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/code/manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 330, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 371, in execute
    output = self.handle(*args, **options)
  File "/code/api/management/commands/scan.py", line 55, in handle
    scan_photos(
  File "/code/api/directory_watcher.py", line 350, in scan_photos
    lrj = LongRunningJob.objects.create(
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 447, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 753, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 790, in save_base
    updated = self._save_table(
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 895, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 933, in _do_insert
    return manager._insert(
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 1254, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/compiler.py", line 1397, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "api_longrunningjob_pkey"
DETAIL:  Key (id)=(16) already exists.

@ippocratis
Copy link
Author

ippocratis commented Jan 31, 2023 •
edited
Loading

i starrted a new db coz off "api_longrunningjob_pkey" Key (id)=(16) already exists error

and

did a docker exec -it backend pip install --upgrade torch together with docker exec -it backend pip install --upgrade torchvision and docker exec -it backend python3 -m pip install --upgrade pip

/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")

persiists

but

scan managed to reach 100% this time

but the job is still unfinished

the "recycle" has not changed in to a "check mark" icon

last lines from docker logs backend says workers are timed out

[2023-01-31 11:20:17 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:895)
[2023-01-31 11:20:18 +0000] [146] [WARNING] Worker with pid 895 was terminated due to signal 9
[2023-01-31 11:20:18 +0000] [921] [INFO] Booting worker with pid: 921
use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
127.0.0.1 - - [2023-01-31 11:20:48] "POST /build/ HTTP/1.1" 200 142 4.873032
[2023-01-31 11:20:48 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:921)
[2023-01-31 11:20:49 +0000] [921] [INFO] Worker exiting (pid: 921)
[2023-01-31 11:20:50 +0000] [146] [WARNING] Worker with pid 921 was terminated due to signal 9
[2023-01-31 11:20:50 +0000] [928] [INFO] Booting worker with pid: 928

last rqworker.log lines says

django.db.utils.OperationalError: server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request.

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/rq/worker.py", line 1061, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.10/dist-packages/rq/job.py", line 821, in perform
    self._result = self._execute()
  File "/usr/local/lib/python3.10/dist-packages/rq/job.py", line 844, in _execute
    result = self.func(*self.args, **self.kwargs)
  File "/code/api/apps.py", line 19, in build_index
    build_image_similarity_index(user)
  File "/code/api/image_similarity.py", line 78, in build_image_similarity_index
    for photo in paginator.page(page).object_list:
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 287, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 1308, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/compiler.py", line 1156, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

10:51:21 default: api.apps.build_index() (51ccc167-f5c4-4428-be48-af487e1cd639)
11:06:01 default: Job OK (51ccc167-f5c4-4428-be48-af487e1cd639)
11:06:01 Result is kept for 60 seconds
11:06:01 Cleaning registries for queue: default
11:06:01 default: api.apps.build_index() (de02b31e-6ce1-40f4-aeb5-ab88cda94d32)
11:39:47 default: Job OK (de02b31e-6ce1-40f4-aeb5-ab88cda94d32)
11:39:47 Result is kept for 60 seconds
11:39:48 Cleaning registries for queue: default
11:39:48 default: api.apps.build_index() (8068c695-a4b5-4ae2-8c91-0aa95079a8d6)

last ownphotos.log lines looks normal

2023-01-31 11:07:18,916 : photo.py : _extract_faces : 626 : INFO : image 41923031e2e93e10566b774d0fa0037a1: 1 face(s) saved
2023-01-31 11:07:18,917 : directory_watcher.py : handle_new_image : 198 : INFO : job 9676e42e-026c-473d-80fe-4162261212cc: extract faces: /data/2023/01/IMG_20230105_161105.jpg, elapsed: 5.820637
2023-01-31 11:07:18,993 : directory_watcher.py : handle_new_image : 205 : INFO : job 9676e42e-026c-473d-80fe-4162261212cc: image processed: /data/2023/01/IMG_20230105_161105.jpg, elapsed: 5.896748
2023-01-31 11:07:19,199 : directory_watcher.py : scan_photos : 404 : INFO : Scanned 143 files in : /data/2023
2023-01-31 11:08:06,699 : directory_watcher.py : scan_photos : 406 : INFO : Finished updating album things
2023-01-31 11:18:23,053 : api_util.py : get_search_term_examples : 178 : INFO : 100 possible ids
2023-01-31 11:18:23,109 : api_util.py : get_search_term_examples : 191 : INFO : Getting search terms for user 1
2023-01-31 11:18:42,722 : api_util.py : get_search_term_examples : 192 : INFO : Found 100 photos
2023-01-31 11:39:47,795 : image_similarity.py : build_image_similarity_index : 90 : INFO : building similarity index took 2024.96 seconds
2023-01-31 11:39:50,644 : image_similarity.py : build_image_similarity_index : 64 : INFO : building similarity index for user deleted
2023-01-31 11:39:51,888 : image_similarity.py : build_image_similarity_index : 90 : INFO : building similarity index took 1.24 seconds
2023-01-31 11:39:51,889 : image_similarity.py : build_image_similarity_index : 64 : INFO : building similarity index for user ippo

@derneuere
Copy link
Member

It could be that this line here changed: https://github.com/LibrePhotos/librephotos-docker/blob/00d97785992b8417d82489614d187e38b481288c/backend/base/Dockerfile#L61

We need a custom pytorch dependency, because raspberry pis do not support the whole arm instruction set. We solved this issue by using this 3rd party pytorch dependency: https://github.com/KumaTea/pytorch-aarch64

@derneuere derneuere mentioned this issue Feb 6, 2023
Closed
2 tasks
@derneuere derneuere added the ARM label Mar 5, 2023
@derneuere
Copy link
Member

I changed a couple of things in the base image and in the entrypoint file. It would be great, if you could test, if it works again.

@ippocratis
Copy link
Author

ippocratis commented Mar 23, 2023 •
edited
Loading

I changed a couple of things in the base image and in the entrypoint file. It would be great, if you could test, if it works again.

Was using 2022w50 backend image so far.
scan was completed successfuly with that image

Just updated to latest and I'll wait for my cron to initiate a scan in the morning

I'll report back tomorow

Thanks

@ippocratis
Copy link
Author

ippocratis commented Mar 24, 2023 •
edited
Loading

From guicron_django.log looks like workers are exiting again due to torchvision

[2023-03-23 17:58:55 +0000] [226] [INFO] Booting worker with pid: 226
use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
[2023-03-23 17:59:53 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:226)
[2023-03-23 17:59:54 +0000] [146] [WARNING] Worker with pid 226 was terminated due to signal 9
[2023-03-23 17:59:54 +0000] [252] [INFO] Booting worker with pid: 252
use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
[2023-03-23 18:00:24 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:252)
[2023-03-23 18:00:24 +0000] [252] [INFO] Worker exiting (pid: 252)
[2023-03-23 18:00:25 +0000] [146] [WARNING] Worker with pid 252 was terminated due to signal 9
[2023-03-23 18:00:25 +0000] [259] [INFO] Booting worker with pid: 259
use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
Unauthorized: /api/searchtermexamples/
Unauthorized: /api/albums/place/list/
Unauthorized: /api/persons/
Unauthorized: /api/albums/thing/list/
Unauthorized: /api/albums/user/list/
[2023-03-23 18:03:40 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:259)
[2023-03-23 18:03:41 +0000] [146] [WARNING] Worker with pid 259 was terminated due to signal 9
[2023-03-23 18:03:41 +0000] [285] [INFO] Booting worker with pid: 285
use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
[2023-03-23 18:04:40 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:285)
[2023-03-23 18:04:41 +0000] [146] [WARNING] Worker with pid 285 was terminated due to signal 9
[2023-03-23 18:04:41 +0000] [311] [INFO] Booting worker with pid: 311
use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
Unauthorized: /api/sitesettings
Unauthorized: /api/rqavailable/
Unauthorized: /api/user/1/
Unauthorized: /api/user/
Unauthorized: /api/searchtermexamples/
Unauthorized: /api/albums/thing/list/
Unauthorized: /api/albums/user/list/
Unauthorized: /api/persons/
Unauthorized: /api/dirtree/
Unauthorized: /api/albums/place/list/
Unauthorized: /api/albums/date/list/
[2023-03-24 07:59:07 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:311)
[2023-03-24 07:59:08 +0000] [146] [WARNING] Worker with pid 311 was terminated due to signal 9

From the admin page of the web ui I also see that the scan stuck at 95%

The rqworker.log

use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
17:44:48 Worker rq:worker:ea2e93bb090e4fab9716304800f2d562: started, version 1.10.1
17:44:48 Subscribing to channel rq:pubsub:ea2e93bb090e4fab9716304800f2d562
17:44:48 *** Listening on default...
17:44:48 Cleaning registries for queue: default
17:44:48 default: api.apps.build_index() (37bd72f5-7c2b-44e1-b8a4-1acb7714869f)
18:11:45 default: Job OK (37bd72f5-7c2b-44e1-b8a4-1acb7714869f)
18:11:45 Result is kept for 60 seconds
18:11:45 Cleaning registries for queue: default
18:11:45 default: api.apps.build_index() (1645d968-8ae7-47a2-9ef7-4335a2da84cf)
18:32:38 default: Job OK (1645d968-8ae7-47a2-9ef7-4335a2da84cf)
18:32:38 Result is kept for 60 seconds
18:32:38 Cleaning registries for queue: default
18:32:38 default: api.apps.build_index() (8e557a7f-f65a-4a91-9d21-a1980dda8f20)
18:47:08 default: Job OK (8e557a7f-f65a-4a91-9d21-a1980dda8f20)
18:47:08 Result is kept for 60 seconds
18:47:09 default: api.apps.build_index() (616a193d-3076-430c-a9b3-f89264278a37)
19:07:22 default: Job OK (616a193d-3076-430c-a9b3-f89264278a37)
19:07:22 Result is kept for 60 seconds
19:07:22 Cleaning registries for queue: default
19:07:22 default: api.apps.build_index() (c9b44047-4919-4d5c-b284-7789d9cd5820)
19:23:53 default: Job OK (c9b44047-4919-4d5c-b284-7789d9cd5820)
19:23:53 Result is kept for 60 seconds
19:23:53 Cleaning registries for queue: default
19:23:53 default: api.apps.build_index() (9d296957-6099-4fdd-977d-742448a02699)
19:41:32 default: Job OK (9d296957-6099-4fdd-977d-742448a02699)
19:41:32 Result is kept for 60 seconds
19:41:32 Cleaning registries for queue: default
19:41:32 default: api.apps.build_index() (1c939256-332d-4826-90ef-fd1d35704484)
20:01:57 default: Job OK (1c939256-332d-4826-90ef-fd1d35704484)
20:01:57 Result is kept for 60 seconds
20:01:58 Cleaning registries for queue: default
20:01:58 default: api.apps.build_index() (cc9c0ff6-f1c7-43ec-a79e-004d9855c07d)
20:18:03 default: Job OK (cc9c0ff6-f1c7-43ec-a79e-004d9855c07d)
20:18:03 Result is kept for 60 seconds
20:18:03 Cleaning registries for queue: default
20:45:03 Cleaning registries for queue: default
21:12:03 Cleaning registries for queue: default
21:39:03 Cleaning registries for queue: default
22:06:04 Cleaning registries for queue: default
22:33:04 Cleaning registries for queue: default
23:00:04 Cleaning registries for queue: default
23:27:04 Cleaning registries for queue: default
23:54:05 Cleaning registries for queue: default
23:59:25 default: api.apps.build_index() (d1e31451-b473-463e-b738-db377e4945c0)
00:01:55 Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.OperationalError: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/rq/worker.py", line 1061, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.10/dist-packages/rq/job.py", line 821, in perform
    self._result = self._execute()
  File "/usr/local/lib/python3.10/dist-packages/rq/job.py", line 844, in _execute
    result = self.func(*self.args, **self.kwargs)
  File "/code/api/apps.py", line 19, in build_index
    build_image_similarity_index(user)
  File "/code/api/image_similarity.py", line 78, in build_image_similarity_index
    for photo in paginator.page(page).object_list:
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 287, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 1308, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/compiler.py", line 1156, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.OperationalError: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/rq/worker.py", line 1061, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.10/dist-packages/rq/job.py", line 821, in perform
    self._result = self._execute()
  File "/usr/local/lib/python3.10/dist-packages/rq/job.py", line 844, in _execute
    result = self.func(*self.args, **self.kwargs)
  File "/code/api/apps.py", line 19, in build_index
    build_image_similarity_index(user)
  File "/code/api/image_similarity.py", line 78, in build_image_similarity_index
    for photo in paginator.page(page).object_list:
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 287, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 1308, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/compiler.py", line 1156, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

00:22:10 Cleaning registries for queue: default
00:49:10 Cleaning registries for queue: default
01:16:10 Cleaning registries for queue: default
01:43:11 Cleaning registries for queue: default
02:10:11 Cleaning registries for queue: default
02:37:11 Cleaning registries for queue: default
03:04:11 Cleaning registries for queue: default
03:31:12 Cleaning registries for queue: default
03:58:12 Cleaning registries for queue: default
04:25:12 Cleaning registries for queue: default
04:52:12 Cleaning registries for queue: default
05:19:13 Cleaning registries for queue: default
05:46:13 Cleaning registries for queue: default
06:13:13 Cleaning registries for queue: default
06:40:13 Cleaning registries for queue: default
07:07:13 Cleaning registries for queue: default
07:34:13 Cleaning registries for queue: default
07:59:33 default: api.apps.build_index() (b165fe1a-a1f8-495f-af7d-64980af66cd2)
08:25:15 default: Job OK (b165fe1a-a1f8-495f-af7d-64980af66cd2)
08:25:15 Result is kept for 60 seconds
08:25:15 Cleaning registries for queue: default
08:25:15 default: api.apps.build_index() (4415b9c6-e52b-4688-9092-063c15c9987d)

My docker-compose.yml

version: "3.8"
services:
  proxy:
    image: reallibrephotos/librephotos-proxy:${tag}
    container_name: proxy
    restart: unless-stopped
    volumes:
      - ${scanDirectory}:/data
      - ${data}/protected_media:/protected_media
    ports:
      - ${httpPort}:80
    depends_on:
      - backend
      - frontend

  db:
    image: postgres:13
    container_name: db
    restart: unless-stopped
    environment:
      - POSTGRES_USER=${dbUser}
      - POSTGRES_PASSWORD=${dbPass}
      - POSTGRES_DB=${dbName}
    volumes:
      - ${data}/protected_media/db:/var/lib/postgresql/data
    command: postgres -c fsync=off -c synchronous_commit=off -c full_page_writes=off -c random_page_cost=1.0
    #Checking health of Postgres db
    healthcheck:
      test: psql -U ${dbUser} -d ${dbName} -c "SELECT 1;"
      interval: 5s
      timeout: 5s
      retries: 5

  frontend:
    image: reallibrephotos/librephotos-frontend:${tag}
    container_name: frontend
    restart: unless-stopped
    depends_on:
      - backend

  backend:
    image: reallibrephotos/librephotos:${tag}
    container_name: backend
    restart: unless-stopped
    volumes:
      - ${scanDirectory}:/data
      - /run/media/ippo/TOSHIBA/protected_media:/protected_media
      - /run/media/ippo/TOSHIBA/uhuru-logs:/logs
      - ${data}/cache:/root/.cache
    environment:
      - SECRET_KEY=${shhhhKey}
      - BACKEND_HOST=backend
      - ADMIN_EMAIL=${adminEmail}
      - ADMIN_USERNAME=${userName}
      - ADMIN_PASSWORD=${userPass}
      - DB_BACKEND=postgresql
      - DB_NAME=${dbName}
      - DB_USER=${dbUser}
      - DB_PASS=${dbPass}
      - DB_HOST=${dbHost}
      - DB_PORT=5432
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - MAPBOX_API_KEY=${mapApiKey}
      - WEB_CONCURRENCY=${gunniWorkers}
      - SKIP_PATTERNS=${skipPatterns}
      - ALLOW_UPLOAD=${allowUpload}
      - DEBUG=0
      - HEAVYWEIGHT_PROCESS=${HEAVYWEIGHT_PROCESS}

    # Wait for Postgres
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy

  redis:
    image: redis:6
    container_name: redis
    restart: unless-stopped
    healthcheck:
      test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ]
      interval: 5s
      timeout: 5s
      retries: 12

@derneuere
Copy link
Member

The warning is normal and also happens on the normal setup. It looks like it's a timeout issue, similar to how #744 (comment) described it. Would be great, if you could open a separate issue for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ARM
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@derneuere @ippocratis