[Bug]: Instructions for the offline setup does not work

[jotatx]

What happened?

I really wanted to see a user case of a successful offline setup in ubuntu.
I've tried, and I feel frustrated. it does not work.

tried to build and this is all i get.

Relevant log output

sudo docker compose up --build
[sudo] password for jtx: 
[+] Building 3.1s (52/62)                                        docker:default
 => [worker internal] load build definition from Dockerfile                0.0s
 => => transferring dockerfile: 1.05kB                                     0.0s
 => [worker internal] load .dockerignore                                   0.0s
 => => transferring context: 185B                                          0.0s
 => [beat internal] load .dockerignore                                     0.0s
 => => transferring context: 185B                                          0.0s
 => [beat internal] load build definition from Dockerfile                  0.0s
 => => transferring dockerfile: 1.05kB                                     0.0s
 => [backend-core internal] load metadata for docker.io/library/python:3.  2.2s
 => [frontend internal] load .dockerignore                                 0.0s
 => => transferring context: 340B                                          0.0s
 => [frontend internal] load build definition from Dockerfile              0.0s
 => => transferring dockerfile: 2.20kB                                     0.0s
 => [frontend internal] load metadata for docker.io/library/node:18-alpin  1.9s
 => [flower 1/8] FROM docker.io/library/python:3.11.6-slim-bullseye@sha25  0.0s
 => [worker internal] load build context                                   0.1s
 => => transferring context: 14.03kB                                       0.0s
 => [beat internal] load build context                                     0.1s
 => => transferring context: 14.03kB                                       0.1s
 => [frontend internal] load build context                                 0.1s
 => => transferring context: 73.17kB                                       0.1s
 => [frontend base 1/1] FROM docker.io/library/node:18-alpine@sha256:3428  0.0s
 => CACHED [backend-core 2/8] RUN apt-get clean && apt-get update && apt-  0.0s
 => CACHED [backend-core 3/8] WORKDIR /code                                0.0s
 => CACHED [worker 4/8] COPY ./requirements.txt .                          0.0s
 => CACHED [worker 5/8] RUN pip install --upgrade pip                      0.0s
 => CACHED [worker 6/8] RUN pip install --no-cache-dir -r requirements.tx  0.0s
 => CACHED [worker 7/8] RUN if [ "" = "true" ]; then pip install --no-cac  0.0s
 => CACHED [worker 8/8] COPY . .                                           0.0s
 => [worker] exporting to image                                            0.0s
 => => exporting layers                                                    0.0s
 => => writing image sha256:10e71676976b01c71c93a5c7cb19c41bc696e91155305  0.0s
 => => naming to docker.io/library/backend-base                            0.0s
 => [beat] exporting to image                                              0.0s
 => => exporting layers                                                    0.0s
 => => writing image sha256:04cc9484a8d732505f4d890d6099323cff553916d2372  0.0s
 => => naming to docker.io/library/backend-base                            0.0s
 => CACHED [frontend runner 1/8] WORKDIR /app                              0.0s
 => CACHED [frontend runner 2/8] RUN addgroup --system --gid 1001 nodejs   0.0s
 => CACHED [frontend runner 3/8] RUN adduser --system --uid 1001 nextjs    0.0s
 => CACHED [frontend deps 1/5] RUN apk add --no-cache libc6-compat python  0.0s
 => CACHED [frontend deps 2/5] RUN yarn global add node-gyp                0.0s
 => CACHED [frontend deps 3/5] WORKDIR /app                                0.0s
 => CACHED [frontend deps 4/5] COPY package.json yarn.lock* package-lock.  0.0s
 => CACHED [frontend deps 5/5] RUN     if [ -f yarn.lock ]; then yarn --f  0.0s
 => CACHED [frontend builder 2/4] COPY --from=deps /app/node_modules ./no  0.0s
 => CACHED [frontend builder 3/4] COPY . .                                 0.0s
 => CACHED [frontend builder 4/4] RUN yarn build                           0.0s
 => CACHED [frontend runner 4/8] COPY --from=builder /app/public ./public  0.0s
 => CACHED [frontend runner 5/8] RUN mkdir .next                           0.0s
 => CACHED [frontend runner 6/8] RUN chown nextjs:nodejs .next             0.0s
 => CACHED [frontend runner 7/8] COPY --from=builder --chown=nextjs:nodej  0.0s
 => CACHED [frontend runner 8/8] COPY --from=builder --chown=nextjs:nodej  0.0s
 => [frontend] exporting to image                                          0.0s
 => => exporting layers                                                    0.0s
 => => writing image sha256:0b29bfb4b89e9ef4ac97aa1fcdd76973c0d27fe283b14  0.0s
 => => naming to docker.io/library/quivr-frontend                          0.0s
 => [flower internal] load build definition from Dockerfile                0.0s
 => => transferring dockerfile: 1.05kB                                     0.0s
 => [flower internal] load .dockerignore                                   0.0s
 => => transferring context: 185B                                          0.0s
 => [backend-core internal] load build definition from Dockerfile          0.0s
 => => transferring dockerfile: 1.05kB                                     0.0s
 => [backend-core internal] load .dockerignore                             0.0s
 => => transferring context: 185B                                          0.0s
 => [backend-core internal] load build context                             0.1s
 => => transferring context: 14.03kB                                       0.1s
 => [flower internal] load build context                                   0.1s
 => => transferring context: 14.03kB                                       0.1s
 => CACHED [backend-core 4/8] COPY ./requirements.txt .                    0.0s
 => CACHED [backend-core 5/8] RUN pip install --upgrade pip                0.0s
 => CACHED [backend-core 6/8] RUN pip install --no-cache-dir -r requireme  0.0s
 => CACHED [backend-core 7/8] RUN if [ "" = "true" ]; then pip install --  0.0s
 => CACHED [backend-core 8/8] COPY . .                                     0.0s
 => [backend-core] exporting to image                                      0.0s
 => => exporting layers                                                    0.0s
 => => writing image sha256:4672e4907ab362605e46b3265cb025224a7bf9f493da8  0.0s
 => => naming to docker.io/library/backend-base                            0.0s
 => [flower] exporting to image                                            0.0s
 => => exporting layers                                                    0.0s
 => => writing image sha256:9d09fda992c15c60794e0c08b0acf2cbf894033a8d283  0.0s
 => => naming to docker.io/library/backend-base                            0.0s
[+] Running 6/6
 ✔ Container redis         Running                                         0.0s 
 ✔ Container web           Running                                         0.0s 
 ✔ Container beat          Recreated                                       0.7s 
 ✔ Container worker        Recreated                                       2.4s 
 ✔ Container backend-core  Recreated                                       4.2s 
 ✔ Container flower        Recreated                                       0.7s 
Attaching to backend-core, beat, flower, redis, web, worker
backend-core  | INFO:     Uvicorn running on http://0.0.0.0:5050 (Press CTRL+C to quit)
backend-core  | INFO:     Started parent process [1]
worker        | /usr/local/lib/python3.11/site-packages/celery/platforms.py:840: SecurityWarning: You're running the worker with superuser privileges: this is
worker        | absolutely not recommended!
worker        | 
worker        | Please specify a different user using the --uid option.
worker        | 
worker        | User information: uid=0 euid=0 gid=0 egid=0
worker        | 
worker        |   warnings.warn(SecurityWarning(ROOT_DISCOURAGED.format(
beat          | [2023-11-27 22:54:38,385: INFO/MainProcess] beat: Starting...
worker        |  
worker        |  -------------- celery@56dfe58f746c v5.2.7 (dawn-chorus)
worker        | --- ***** ----- 
worker        | -- ******* ---- Linux-6.2.0-37-generic-x86_64-with-glibc2.31 2023-11-27 22:54:38
worker        | - *** --- * --- 
worker        | - ** ---------- [config]
worker        | - ** ---------- .> app:         celery_worker:0x7f47bf399090
worker        | - ** ---------- .> transport:   redis://redis:6379/0
worker        | - ** ---------- .> results:     redis://redis:6379/0
worker        | - *** --- * --- .> concurrency: 4 (prefork)
worker        | -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
worker        | --- ***** ----- 
worker        |  -------------- [queues]
worker        |                 .> celery           exchange=celery(direct) key=celery
worker        |                 
worker        | 
worker        | [tasks]
worker        |   . celery_task.create_embedding_for_document
worker        |   . celery_worker.remove_onboarding_more_than_x_days_task
worker        |   . process_crawl_and_notify
worker        |   . process_file_and_notify
worker        | 
flower        | 2023-11-27 22:54:38,965:INFO - Visit me at http://0.0.0.0:5555
flower        | 2023-11-27 22:54:38,974:INFO - Broker: redis://redis:6379/0
flower        | 2023-11-27 22:54:38,978:INFO - Registered tasks: 
flower        | ['celery.accumulate',
flower        |  'celery.backend_cleanup',
flower        |  'celery.chain',
flower        |  'celery.chord',
flower        |  'celery.chord_unlock',
flower        |  'celery.chunks',
flower        |  'celery.group',
flower        |  'celery.map',
flower        |  'celery.starmap',
flower        |  'celery_task.create_embedding_for_document',
flower        |  'celery_worker.remove_onboarding_more_than_x_days_task',
flower        |  'process_crawl_and_notify',
flower        |  'process_file_and_notify']
flower        | 2023-11-27 22:54:38,994:INFO - Connected to redis://redis:6379/0
worker        | [2023-11-27 22:54:39,024: INFO/MainProcess] Connected to redis://redis:6379/0
worker        | [2023-11-27 22:54:39,035: INFO/MainProcess] mingle: searching for neighbors
worker        | [2023-11-27 22:54:40,050: INFO/MainProcess] mingle: all alone
flower        | 2023-11-27 22:54:40,051:WARNING - Inspect method registered failed
flower        | 2023-11-27 22:54:40,059:WARNING - Inspect method active_queues failed
flower        | 2023-11-27 22:54:40,064:WARNING - Inspect method scheduled failed
flower        | 2023-11-27 22:54:40,068:WARNING - Inspect method revoked failed
flower        | 2023-11-27 22:54:40,069:WARNING - Inspect method active failed
worker        | [2023-11-27 22:54:40,069: INFO/MainProcess] celery@56dfe58f746c ready.
flower        | 2023-11-27 22:54:40,069:WARNING - Inspect method stats failed
flower        | 2023-11-27 22:54:40,070:WARNING - Inspect method reserved failed
flower        | 2023-11-27 22:54:40,071:WARNING - Inspect method conf failed
worker        | [2023-11-27 22:54:43,992: INFO/MainProcess] Events of group {task} enabled by remote.
backend-core  | INFO:     Started server process [12]
backend-core  | INFO:     Waiting for application startup.
backend-core  | INFO:     Application startup complete.
backend-core  | INFO:     Started server process [11]
backend-core  | INFO:     Waiting for application startup.
backend-core  | INFO:     Application startup complete.
backend-core  | INFO:     Started server process [13]
backend-core  | INFO:     Waiting for application startup.
backend-core  | INFO:     Application startup complete.
backend-core  | INFO:     Started server process [10]
backend-core  | INFO:     Waiting for application startup.
backend-core  | INFO:     Application startup complete.
backend-core  | INFO:     Started server process [8]
backend-core  | INFO:     Waiting for application startup.
backend-core  | INFO:     Application startup complete.
backend-core  | INFO:     Started server process [9]
backend-core  | INFO:     Waiting for application startup.
backend-core  | INFO:     Application startup complete.

Twitter / LinkedIn details

No response

[dosubot]

🤖

Hello @jotatx,

I'm Dosu, an AI bot here to assist you. I can help you troubleshoot issues, answer your queries, and guide you on how to contribute effectively to this project. While we wait for a human maintainer, feel free to share any additional details about your issue. Let's work together to get you up and running with Quivr!

Based on the information you've provided, it seems like the issue might be related to the startup order of your services. Docker Compose's depends_on option only ensures that the dependent services are started before the service that depends on them. It does not wait for the dependent services to be "ready" before starting the other services. This could be why the flower service is throwing warnings about failed inspect methods - it might be trying to communicate with the worker service before it's fully ready to accept connections.

To resolve this, you could add a script to the flower service that waits for the worker service to be ready before starting the flower service. This script could ping the worker service until it gets a successful response, indicating that the worker service is ready to accept connections.

As for the backend-core container, if it seems to start up successfully but you're still having issues, it could be that there's a problem with the application code itself, or perhaps a necessary service or dependency isn't being correctly initialized. Without more specific error messages or logs, it's difficult to diagnose this part of the issue.

Regarding the role of the flower container, it is a part of the backend services and is responsible for monitoring and administrating the Celery task queue system. It interacts with the 'redis', 'worker', and 'beat' services to do so.

If you could provide more specific error messages or logs, it would be easier to diagnose and resolve your issue.

Sources

• docker-compose.dev.yml
• docker-compose.yml

[StanGirard]

Fixed :) New readme