Scan from scratch crashes #720
Comments
|
In fact even with my ops base or test base, I'm no more able to scan an additional picture. I get the following error: |
|
@sickelap @derneuere It looks as if today fix has solved the issue? isn't it? |
|
Did you run this on a fresh installation? If not, please let me know the details how this could be reproduced. |
|
very simple to reproduce for me. |
|
Recreated everything from scratch and it works fine. My guess is that you used an existing database. If so, could you please remove it and start over? |
|
I said : "up from scratch, I mean librephotos directory containing only 1 directory picture containing 3 files in this test." |
|
I guess major error from myself was to still use too old docker-compose.yml and librephotos.env files! They were dated oct 21. I restarted from scratch with uptodate files and I went at the end of the process. |
|
I reopen this subject... I started scan from scratch but with uptodate .yml and .env files in dev mode. Scan crashed with the following error. In addition I don't see any "Unknown - Other' cluster despite the fact that searching for "Unknown - Other" return many photos. I known that faces clustering was discussed here in oct / nov but I don't know clearly the final result and if the subject is supposed to be closed or not. |
|
Don't know if it can help but here is the end of rqworker.log. |
All clusters are cleaned so that when facial recognition is done, it is more accurate, or so I think.
In my case it seems all fine!
The Unknown - Other faces appears in the labeled faces, not in interred and the search in all branches are erroring, see #686, im trying to setup the dev environment to try to resolve that! |
|
Oh and this is my docker.yml and the .env: # DO NOT EDIT
# The .env file has everything you need to edit.
# Run options:
# 1. Use prebuilt images (preferred method):
# run cmd: docker-compose up -d
# 2. Build images on your own machine:
# build cmd: COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose build
# run cmd: docker-compose up -d
version: "3.7"
services:
proxy:
image: reallibrephotos/librephotos-proxy:${tag}
container_name: lb_proxy
restart: unless-stopped
volumes:
- ${myPhotos}:/data
- ${data}/protected_media:/protected_media
ports:
- ${httpPort}:80
networks:
- proxy
- default
labels:
- traefik.enable=true
- traefik.http.routers.librephotos.entrypoints=https
- traefik.http.routers.librephotos.rule=Host(`PRIVATE`)
- traefik.http.routers.librephotos.tls=true
- traefik.http.routers.librephotos.middlewares=defaultMiddlewares@file
- traefik.http.routers.librephotos.service=librephotos
- traefik.http.services.librephotos.loadbalancer.server.port=80
- traefik.docker.network=proxy
depends_on:
- backend
- frontend
db:
image: postgres:13
container_name: lb_db
restart: unless-stopped
environment:
- POSTGRES_USER=${dbUser}
- POSTGRES_PASSWORD=${dbPass}
- POSTGRES_DB=${dbName}
volumes:
- ${data}/db:/var/lib/postgresql/data
command: postgres -c fsync=off -c synchronous_commit=off -c full_page_writes=off -c random_page_cost=1.0
healthcheck:
test: psql -U ${dbUser} -d ${dbName} -c "SELECT 1;"
interval: 5s
timeout: 5s
retries: 5
frontend:
image: reallibrephotos/librephotos-frontend:${tag}
#image: reallibrephotos/librephotos-frontend:latest
container_name: lb_frontend
restart: unless-stopped
depends_on:
- backend
backend:
image: reallibrephotos/librephotos:${tag}
container_name: lb_backend
restart: unless-stopped
deploy:
resources:
limits:
memory: 4096M
reservations:
memory: 1024M
volumes:
- ${myPhotos}:/data
- ${data}/protected_media:/protected_media
- ${data}/logs:/logs
- ${data}/cache:/root/.cache
environment:
- SECRET_KEY=${shhhhKey:-}
- BACKEND_HOST=backend
- ADMIN_EMAIL=${adminEmail:-}
- ADMIN_USERNAME=${userName:-}
- ADMIN_PASSWORD=${userPass:-}
- DB_BACKEND=postgresql
- DB_NAME=${dbName}
- DB_USER=${dbUser}
- DB_PASS=${dbPass}
- DB_HOST=${dbHost}
- DB_PORT=5432
- REDIS_HOST=redis
- REDIS_PORT=6379
- MAPBOX_API_KEY="PRIVATE"
- WEB_CONCURRENCY=${gunniWorkers:-1}
- SKIP_PATTERNS=${skipPatterns:-}
- ALLOW_UPLOAD=${allowUpload:-false}
- DEBUG=0
- HEAVYWEIGHT_PROCESS=${HEAVYWEIGHT_PROCESS:-3}
depends_on:
- db
- redis
redis:
image: redis:6
container_name: lb_redis
restart: unless-stopped
healthcheck:
test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ]
interval: 5s
timeout: 5s
retries: 12
networks:
proxy:
external: true# This file contains all the things you need to change to set up your Libre Photos.
# There are a few items that must be set for it to work such as the location of your photos.
# After the mandatory entry's there are some optional ones that you may set.
# Start of mandatory changes.
# Location of your photos.
myPhotos=./librephotos/pictures
# Internal data of LibrePhotos
data=./librephotos/data
# ------------------------------------------------------------------------------------------------
# Wow, we are at the optional now. Pretty easy so far. You do not have to change any of the below.
#What port should Libre Photos be accessed at (Default 3000)
httpPort=3000
# What branch should we install the latest weekly build or the development branch (dev)
tag=dev
# Number of workers, which take care of the request to the api. This setting can dramatically affect the ram usage.
# A positive integer generally in the 2-4 x $(NUM_CORES) range.
# You’ll want to vary this a bit to find the best for your particular workload.
# Each worker needs 800MB of RAM. Change at your own will. Default is 2.
gunniWorkers=3
# You can set the database name. Did you know Libre Photos was forked from OwnPhotos?
dbName=PRIVATE
# Here you can change the user name for the database.
dbUser=PRIVATE
# The password used by the database.
dbPass=PRIVATE
# Database host. Only change this if you want to use your own existing Postgres server. If using your own server, you can remove the 'db' container from docker-compose.yml. If you're changing the name of the DB's container name (DB_CONT_NAME further down), you need to set this variable to match that name too.
dbHost=db
# Default minimum rating to interpret as favorited. This default value is used when creating a new user.
# Users can change this in their settings (Dashboards > Library).
DEFAULT_FAVORITE_MIN_RATING=4
# ---------------------------------------------------------------------------------------------
# If you are not a developer ignore the following parameters: you will never need them.
# Where shall we store the backend and frontend code files.
codedir=./librephotos/code
# Location for pgAdmin
pgAdminLocation=./librephotos/pgadmin
|
|
I use default values (ie without any modification) for docker-compose.yml (3.8) as well as for .env DO NOT EDITThe .env file has everything you need to edit.Run options:1. Use prebuilt images (preferred method):run cmd: docker-compose up -d2. Build images on your own machine:build cmd: COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose buildrun cmd: docker-compose up -dversion: "3.8" db: frontend: backend: redis: This file contains all the things you need to change to set up your Libre Photos.There are a few items that must be set for it to work such as the location of your photos.After the mandatory entries there are some optional ones that you may set.Start of mandatory changes.Location of your photos.scanDirectory=./librephotos/pictures Internal data of LibrePhotosdata=./librephotos/data ------------------------------------------------------------------------------------------------Wow, we are at the optional now. Pretty easy so far. You do not have to change any of the below.#What port should Libre Photos be accessed at (Default 3000) What branch should we install the latest weekly build or the development branch (dev)tag=dev Number of workers, which take care of the request to the api. This setting can dramatically affect the ram usage.A positive integer generally in the 2-4 x $(NUM_CORES) range.You’ll want to vary this a bit to find the best for your particular workload.Each worker needs 800MB of RAM. Change at your own will. Default is 2.gunniWorkers=2 You can set the database name. Did you know Libre Photos was forked from OwnPhotos?dbName=librephotos Here you can change the user name for the database.dbUser=docker The password used by the database.dbPass=AaAa1234 Default minimum rating to interpret as favorited. This default value is used when creating a new user.Users can change this in their settings (Dashboards > Library).DEFAULT_FAVORITE_MIN_RATING=4 Database host. Only change this if you want to use your own existing Postgres server. If using your own server, you can remove the 'db' container from docker-compose.yml. If you're changing the name of the DB's container name (DB_CONT_NAME further down), you need to set this variable to match that name too.dbHost=db Set the names of the docker containers to your own entries. Or don't, I'm not your dad.Changing these will require you to
|
|
You say "The Unknown - Other faces appears in the labeled faces" I understand that you have a cluster of 1163 faces in labelled tab in your case? In my case I have none... |
|
I noted several inconsistencies in captures provided by Nerodacles which go in the same direction than me! |
|
To clear this issue report which mixes several things, I think i'm going to close and to open at least 2 new. |
having so many (small) problems with my current ops base (53k pictures) I decided to restart the scan from scratch.
After about 1h40 the whole process was terminated. For "scan photos" task I saw item number increasing normally till 53k but some seconds after it came back to 0. Here is the final picture :
But 0 photo in Photo tab...
In ownphotos.log where are a lot errors as you can see here:
What do you think?
The text was updated successfully, but these errors were encountered: