Start of docker image fails with error Cannot find module 'log4js' with DB_TYPE=sqlite

[cybertschunk]

Describe the bug
We upgraded to the latest etherpad release and now our etherpad installation refuses to start correctly. It fails with the following error:

etherpad    | Error: Cannot find module 'log4js'
etherpad    | Require stack:
etherpad    | - /opt/etherpad-lite/src/node/server.js
etherpad    |     at Module._resolveFilename (node:internal/modules/cjs/loader:1075:15)
etherpad    |     at Module._load (node:internal/modules/cjs/loader:920:27)
etherpad    |     at Module.require (node:internal/modules/cjs/loader:1141:19)
etherpad    |     at require (node:internal/modules/cjs/helpers:110:18)
etherpad    |     at Object.<anonymous> (/opt/etherpad-lite/src/node/server.js:27:16)
etherpad    |     at Module._compile (node:internal/modules/cjs/loader:1254:14)
etherpad    |     at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
etherpad    |     at Module.load (node:internal/modules/cjs/loader:1117:32)
etherpad    |     at Module._load (node:internal/modules/cjs/loader:958:12)
etherpad    |     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) {
etherpad    |   code: 'MODULE_NOT_FOUND',
etherpad    |   requireStack: [ '/opt/etherpad-lite/src/node/server.js' ]
etherpad    | }

To Reproduce
Steps to reproduce the behavior:

1. We use the following docker compose file

version: "3.7"

services:
  etherpad:
    image: etherpad/etherpad
    container_name: etherpad
    volumes:
      - type: bind
        source: TRUNCATED/etherpad/pads.db
        target: /opt/etherpad-lite/var/pads.db
    environment:
      - TITLE=TRUNCATED
      - DB_TYPE=sqlite
      - DB_FILENAME=var/pads.db
      - API_KEY=${ETHERPAD_API_KEY}
      - ADMIN_PASSWORD=${ETHERPAD_ADMIN_PASSWORD}
      - ADMIN_USER=TRUNCATED
    ports:
      - 127.0.0.1:13080:9001
    restart: unless-stopped
    command: bash -c "npm install sqlite3 && node src/node/server.js"

3. Run docker-compose up
4. Wait for the error

Expected behavior
The docker container starts up normally.

Screenshots

Server (please complete the following information):

• Etherpad version: Latest tag
• OS: Docker Image

[SamTV12345]

Thanks for the issue. Did we change anything log4js related yesterday? I can't remember that we did something in that corner @JohnMcLear ?

Could be related to the comment in the Dockerfile stating that npm is buggy and might remove log4js when installing sqlite. The npm logic could have changed again.

[JohnMcLear]

@SamTV12345 no, we did not.

the Log4JS exploit doesn't impact Etherpad, it is something we need to address in general though.. Afaik the TLDR is that log4js changed the format of their settings blob so anyone with an old settings blob for logs would be unable to use the latest versions..

[SamTV12345]

Ok. Thanks for the info. The latest ueber version didn't fix the problem. Maybe installing an older SQLite version might work.

[jibon57]

I'm facing almost similar problem with redis. Looks like ueberdb not accepting custom host name. It's always using default 127.0.0.1

  "dbType": "redis",
  "dbSettings": {
    "host":     "host.docker.internal",
    "port":     6379
  },

[SamTV12345]

Im working on it. Unfortunately I couldn't find a fix yet. Do you also install something at startup? Try to add log4js to the initial installation.

[jibon57]

For my case I've bypassed the issue of missing log4js. Looks like if you want to install any custom plugin then you'll require to run npm directly inside src directory like:

ARG ETHERPAD_PLUGINS="ep_sticky_attributes ep_themes"
RUN npm install --package-lock-only --no-save --legacy-peer-deps -C src ${ETHERPAD_PLUGINS} && \
    rm -rf ~/.npm

But now I'm having problem with ueberdb which is always using default host.

[cybertschunk]

I tried to install log4js directly as we install sqlite. Unfortunately it still refused to start due to unresolved functions.

[SamTV12345]

Is the method named something like setGlobalLogLevel or appenders?

[cybertschunk]

the startup fails with the following log output

etherpad    | /opt/etherpad-lite/src/node/server.js:28
etherpad    | log4js.replaceConsole();
etherpad    |        ^
etherpad    |
etherpad    | TypeError: log4js.replaceConsole is not a function
etherpad    |     at Object.<anonymous> (/opt/etherpad-lite/src/node/server.js:28:8)
etherpad    |     at Module._compile (node:internal/modules/cjs/loader:1256:14)
etherpad    |     at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
etherpad    |     at Module.load (node:internal/modules/cjs/loader:1119:32)
etherpad    |     at Module._load (node:internal/modules/cjs/loader:960:12)
etherpad    |     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
etherpad    |     at node:internal/main/run_main_module:23:47

[SamTV12345]

@cybertschunk I finally found the error. We forgot to cd into the source directory. Installing in the root directory caused npm to be confused and remove all the other dependencies causing the container to fail. log4js was installed as the latest version because there was no other requirements in the root directory.

services:
  etherpad:
    image: etherpad/etherpad
    container_name: etherpad
     volumes:
          - type: bind
            source: TRUNCATED/etherpad/pads.db
            target: /opt/etherpad-lite/var/pads.db
    environment:
      - TITLE=TRUNCATED
      - DB_TYPE=sqlite
      - API_KEY=${ETHERPAD_API_KEY}
      - ADMIN_PASSWORD=${ETHERPAD_ADMIN_PASSWORD}
      - ADMIN_USER=TRUNCATED
    ports:
      - 127.0.0.1:13080:9001
    restart: unless-stopped
    command: bash -c "cd src && npm install sqlite3 && node node/server.js"

[JohnMcLear]

The redis bug is not really related but due to an redis & ueberdb version bump and the docs being straight up wrong for ueberdb & redis.

The TLDR is you should use url for Redis and someone should probably modify the redis logic to create a url from hostname et al.

[cybertschunk]

doing so still causes the server to fail. Now it can't locate the database file anymore
[2023-06-30 12:02:17.780] [ERROR] server - Error occurred while starting Etherpad etherpad | [2023-06-30 12:02:17.781] [ERROR] server - Metrics at time of fatal error: etherpad | { etherpad | "httpStartTime": 0, etherpad | "memoryUsage": 76132352, etherpad | "memoryUsageHeap": 19557784 etherpad | } etherpad | [2023-06-30 12:02:17.781] [ERROR] server - Error: SQLITE_CANTOPEN: unable to open database file

[SamTV12345]

I finally found the solution. Seems like the default type is erroring out because no directory is set.

Given the following docker-compose

services:
  etherpad:
    image: etherpad/etherpad:develop
    container_name: etherpad
    volumes:
          - type: bind
            source: ./data
            target: /opt/etherpad-lite/var
          - type: bind
            source: ./settings.json
            target: /opt/etherpad-lite/settings.json
    environment:
      - TITLE=TRUNCATED
      - DB_TYPE=sqlite
      - API_KEY=${ETHERPAD_API_KEY}
      - ADMIN_PASSWORD=${ETHERPAD_ADMIN_PASSWORD}
      - ADMIN_USER=TRUNCATED
    ports:
      - 127.0.0.1:13080:9001
    restart: unless-stopped
    command:  bash -c "cd src && npm install sqlite3 && node node/server.js"

Download the attached settings.log and put it in the root directory of your etherpad directory.

Execute this in your bash

mv settings.log settings.json
nano docker-compose.yml # Take the source from above

You should have a data directory in your etherpad directory which contains the sqlite database once the server is up and you e.g. create a pad.

To circumvent the constant download of sqlite you can create a new Dockerfile based on the etherpad image and run the commands in that shell and export that image again. That way you have a completely local file.

settings.log

[SamTV12345]

Closing as the issue is resolved. If you need more help. Just reopen the issue or comment here.

[cybertschunk]

thanks for your help! The command we finally ended up using was /bin/sh -c "cd src && npm install sqlite3 && etherpad" Everything is working as intended now.