Tempest-EPG over Docker-Compose on Openmediavault 7.x - Permission problems and/or (sub-)folder and files not being generated

[AngusMacGyver-DE]

Issue Type

Please choose your issue type

• Request
• Bug
• Question

OS Details (Not required for Request)

Please choose your OS

• Windows
• Unix
• Docker

Please write your version details (Below not required for Docker)

• PHP Version: ...

• Tempest Version: ...

Please write your web server name(Apache/Nginx etc.) or tick shared host if you are running on a shared host service / server (VPS, Web Hosting etc.)

• Web Server Name: ...

• Shared Host

Expected Behavior (Not required for Request)

Would like to install / use Tempest-EPG over Docker-Compose on Openmediavault without problems.

Current Behavior (Not required for Request)

Currently I tried to install / use Tempest-EPG over Docker-Compose on Openmediavault.
Unfortunately it seems that there are problems with folder/file permissions and/or folder structure is not generated at all.

Tried / Used the following compose files:

version: "2.1"
services:
tempest_epg:
image: kvanc/tempest_epg:latest
container_name: tempest_epg
environment:
- PUID=${PUID}
- PGID=${PGID}
- TZ=${TZ}
volumes:
- ./config/:/var/www/html/tempest_config/
ports:
- 8095:8095
restart: unless-stopped

Global environment variables are set in Openmediavault as follows:
${PUID}=1000=appuser
${PGID}=100=users
${TZ}=Europe/Berlin

If the above are not used it will revert back to system standard:
Owner: root
Group: root
Permission: Administrator - read/write, Users - no access, Others - no access

I also tried it without using the variables for PUID, PGID - same result: (sub)-folders and files are not generated.

On the other hand:
If I use it wit Docker for Windows or throw up the standalone tempest.php to an uberspace webhosting account (https://uberspace.de/en/) it will generate the (sub-)folders and files just fine.

What did I do wrong?
Why are the (sub-)folder and files not generated at all?
Which folder and file permissions does Tempest-EPG need?

Steps to Reproduce (Not required for Request/Question)

1. Install Openmediavault 7.x
2. Set up according to the following Quick Configuration Guide: https://forum.openmediavault.org/index.php?thread/49357-omv-quick-configuration-guide/
3. Use Compose File mentioned above
4. Up Compose File and use Tempest-EPG

Attachments (Not required for Request/Question)

Please attach your error/grab logs, configuration.xml file

[K-vanc]

hi;

I never understood properly how that composer thing works. To be honest, I couldn't even manage to install a single file successfully by using composer yet.. so hard to tell you way of fix but at first sight, what I suspect is below 2;

${PUID}=1000=appuser
${PGID}=100=users

docker image set by default to user "nobody" in order to avoid possible clashes with different OS. Also php set into same user so %99, this is your problem. I always see around but dont know the meaning of that 1000/100 numbers. Can you try to install again by changing group and user as nobody ?

[AngusMacGyver-DE]

Tried it with
PUID=65534 and
PGID=65534

This should be "nobody".
Unfortunately no luck; files and folders still not generated below config folder.

config folder itself seems to be setup with Owner: root, Group: root, Permission: 755 no matter what I have specified through PUID, PGID according to what i see over SSH/SFTP MobaXTerm-SSH-Terminal.

When exactly should the folders and files be generated?
By uping the Container or by browsing to tempest.php?

[K-vanc]

Folder structure will be generated on your first call to tempest.php via browser or cli. But i am not sure it can create folders in a root owned folder.

I looked a bit and some are setting users as below;

app:
user: "nobody:nobody"

But i am not sure same will work or not. Normally this permissions will be set since they are fixed instructions in dockerfile while creation of container so i am also not sure that PUID things are necessary to set. Did you try deleting container and rebuild without that variables in compose file?

[K-vanc]

Or do you have chance to run it directly via docker instead of composer?

docker run -p 80:8095 -e TZ=Europe/Berlin -v "/your/config/folder:/var/www/html/tempest_config/" kvanc/tempest_epg

This should do the same thing

Note: not sure about "./config" directory, as far as i know it should be absolute path to mount

[K-vanc]

I checked that puid/gpid things and they are for setting internal users of containers but no need to set them since nobody user will be already set during creation of container so just delete that lines from compose file. I guess you didnt call tempest.php so it didnt generate folder structure. from localhost:8095 you can run it and it will generate folders. Tempest has a build in module to check folder/file permissions. If something is wrong about it, you will see the error message on screen telling you there is a problem with permissions

[AngusMacGyver-DE]

Did you try deleting container and rebuild without that variables in compose file?

Yes, I did that as well. Unfortunately with same results: (sub-)folders and files are not generated.
Quote from my posting: "I also tried it without using the variables for PUID, PGID - same result: (sub)-folders and files are not generated."

Note: not sure about "./config" directory, as far as i know it should be absolute path to mount

This setting lead to the full path of /srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/config.

/srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/ is set as stanard compose path; below that a folder with the name of the container will be created automatically.

Or do you have chance to run it directly via docker instead of composer?

Unfortunately I do not know; as I`m not really a linux expert and mainly use the Openmediavault-GUI. The Openmediavault-GUI seems to work only through compose.

I also posted on Openmediavaults-Forum:
https://forum.openmediavault.org/index.php?thread/52245-how-to-access-appdata-folder-and-subfolders-from-windows-through-smb/

Hope that something / someone will turn up.
I will keep you posted.

[K-vanc]

hmm, it is strange. I think permission issue is related with your system, not with docker container or tempest. Are you able to call tempest.php from browser? It will work no matter folders are created or not but will trigger a message if there is problem.

I think issue is related with mounting your folder. Are you able to mount any other folder instead of that standard folder just to test? create a new folder in your desktop (or main directory) and try to mount it into "/var/www/html/tempest_config/".

Also just for curiosity, are you able to install/run any other docker container on that setup?

for running it directly through docker, I already added code as below;

docker run -p 80:8095 -e TZ=Europe/Berlin -v "/your/config/folder:/var/www/html/tempest_config/" kvanc/tempest_epg

just replace "/your/config/folder" part with the path of folder that you want to mount and paste that line into your ssh terminal.

[K-vanc]

I found below link about problems with mounting folders;

https://forums.docker.com/t/volume-use-in-two-or-more-containers/139768/4

It says

You need to make sure the directory/file owner and the processes in all containers that try to access them have the same UID:GID.

I dont know openvault settings but if this true, then folder to be mount and container user/group pids have to be matched(for me this is abit strange need but..). In this case, setting container internal user/group to your system will be very difficult but you can create a new folder and chown it to nobody(you said id is 65534) so UID/GID's of that folder will be same with containers inside. Theoretically, this should work when you mount that folder during container creation

[AngusMacGyver-DE]

Are you able to call tempest.php from browser?

Yes I could open tempest in Browser.
This works also for the compose-way.

Also just for curiosity, are you able to install/run any other docker container on that setup?
Yes, I have "plex (lscr.io/linuxserver/plex:latest)" and "telerising (takealug/new-telerising-api:latest)" up and runnig through the Compose-GUI of Openmediavault.

Also tried runnig it with:
docker run -p 8095:8095 -e TZ=Europe/Berlin -v "/srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/config/:/var/www/html/tempest_config/" kvanc/tempest_epg

Browser tempest.php reachable; but there as well: No (sub-)folders and files created.

[K-vanc]

can you check also if folders are created inside container(not in your mounted folder)? If subfolders are generated inside "/var/www/html/tempest_config/" in the container, this means problem is related with mounting your folder as I thought. I think you also didnt see any complain in the browser by Tempest about permissions so this means container side works properly

[AngusMacGyver-DE]

If i manually create a site_config folder and upload the siteconfig this is shown in the dropdown "Siteconfig Loader" at "Channel Generator".

But if I then generate the channels (with ticked "Enable Channel List Downloader") nothing will happen and I will be immediately redirected to the main menu. If I perform the same within uberspace-Webspace than it will generate a channel file and starts download of that file.

If i manually create all (sub-)folders in the mounted folder, then I will receive messages about permission issues in browser.

[AngusMacGyver-DE]

I also checked "/srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/docker/overlay2/d3abb3e7c45a5de3afb7ce8914eec3a467ba6f62f88a6864e111bad88d9c2aa2/merged/var/www/html/tempest_config/" which should be the full path to the inside of the container.

Unfortunately in there there aren`t (sub-)folders as well.
Only file is tempest.php inside the html folder.

tempest.php in there is set to Owner: nobody, Group: nogroup

[K-vanc]

tempest.php permissions are correct but when you manually add /create files, they will be done by your system user and cause permission issue again. I think this is the issue mentioned on #107 (comment)

so lets try case by case to test;

1. stop and hard deleted generated tempest container
2. open your terminal and create a new folder as
   sudo mkdir /srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/test/
3. set user permissions of that new folder
   sudo chown nobody:nogroup /srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/test/
4. Now we created a test folder with same user/group with container to be created
5. edit your composer file as below;
   `version: "2.1"
   services:
   tempest_epg:
   image: kvanc/tempest_epg:latest
   container_name: tempest_epg
   environment:

• TZ=${TZ}
  volumes:
• /srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/test/:/var/www/html/tempest_config/
  ports:
• 8095:8095
  restart: unless-stopped`

6. build new container with modified composer file
7. Call tempest.php from your browser and check inside of "/srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/test/" to see subfolder generated

Edit: I couldnt escape (-) chars..

[AngusMacGyver-DE]

7. Call tempest.php from your browser and check inside of "/srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/test/" to see subfolder generated

This has worked; files are present now ;-)

But I don`t know if working with user and group "nobody" will match good with Openmediavault.
Because I assume that the final resulting epg-file will be generated in the epg-folder.

I assume that at some time I will have to access that epg-folder from within another software or through another channel (SMB/NFS).

Would there be any way to change the docker-container of Tempest-EPG so that it would use any defined / set UID, GID from the environment settings for alpine / php-fpm?

[K-vanc]

it is more complicated to change for container than you thought. Also it will probably broke many things for other users. But why do you think nobody user will be problem for you to reach? it is the lowest previleged user of system. your epg will be accessible inside;

"/srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/test/epg/"

since it is owned by nobody, any system user shall be easily accessed there to read/write. Also you dont have to do everything with nobody user, it is only needed for tempest container. You can make .sh file for cron to generate epg then copy generated epg file into your samba folder etc since lines will be managed by system as 1 by 1;

example
#!/bin/bash
docker exec -u root tempest_epg php /var/www/html/tempest.php engine=Generate
cp /srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/test/epg/yourepg.xml /your/samba/folder

above command will run docker container as root regardless of your internal setting so can be croned and copy operations will be managed by your system user so permissions shouldn't be an issue anymore

Would there be any way to change the docker-container of Tempest-EPG so that it would use any defined / set UID, GID from the environment settings for alpine / php-fpm?

to achieve this, you need to make your own container from scratch and set all nginx and php permissions according to your system then build. that container is actually only php + nginx libraries from alpine linux and tempest.php file. Only preset to run with each other without problems

[AngusMacGyver-DE]

Additional Question:
To which Owner / Group and Permission do I have to set a php-user.ini?
I need to set the timezone to Europe/Berlin.

[K-vanc]

I dont think you understand permission issues well for Linux. Everything in container is nobody. For your system(openvault) container and mount folder shall have same user/groups so as same procedure, create another folder in somewhere, copy your custom php.ini in it and chown this folder to nobody:nogroup. When you mount it to php.ini location(it is written on main page examples), it will work

[K-vanc]

This is the mount example

"/your/local/folder/php-user.ini:/etc/php82/conf.d/custom.ini"

[AngusMacGyver-DE]

All works well now. nobody:nogroup
(Container have been down accidentally as I have tested it which has led to a blank page...)

[naijaping]

Iam following this post with keen interest as I am in the same situation as @AngusMacGyver-DE. Tempest running fine on my php host via cpanel. been pulling my hair for days to make it work in Docker OMV. no folder generated and when I copied the folder manually via cyberduck, it will still not show up in the UI. @AngusMacGyver-DE, please share your steps in you have finally make it work.

Many thanks

[K-vanc]

Hi, steps are already explained above posts. Briefly, Vault wants same user/group for both docker container's internal setup and folder(s) to be mounted during container building. Tempest container uses "nobody" as user and group.

This is equavilent in Vault as user= nobody and group= nogroup

So since the target is aligning folder owners, just create a folder and set ownership of this folder as below;

sudo chown nobody:nogroup /your/new/folder/path/tobe/mounted

Now that folder belongs to user nobody and group nogroup so mounting it to docker, will not cause permission issue. If you already build container, just destroy it and build a new one by mounting this folder.

[AngusMacGyver-DE]

@naijaping
On my OMV 7.x-System the following paths are set for the Docker-Plugin of OMV:
Compose Files > Shared Folder
appdata [on /dev/md0, data/system/appdata]

This means that on my system the standard path under which the containers will be created according to the used Compose File will be /srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/

Additionally the following is set as the global environment variables:

PUID=1000
PGID=100
TZ=Europe/Berlin

This is the Compose File I finally used:

---
version: "2.1"
services:
  tempest_epg:
    image: kvanc/tempest_epg:latest
    container_name: tempest_epg
    environment:
      - TZ=${TZ}
    volumes:
      - ./config:/var/www/html/tempest_config/
      - ./config/php/php-user.ini:/etc/php82/conf.d/custom.ini
    ports:
      - 8095:8095
    restart: unless-stopped

1. You then "up" (= start) the container in Web-GUI

2. Call tempest.php trough webbrowser; in my case: 192.168.178.62:8095/tempest.php or openmediavault.local:8095/tempest.php

3. Check under Compose > Containers if the container is up.
   In column Mounts you will have 2 paths:
   a) for the personal php-user.ini file (In my case: /srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/config/php/php-user.ini )
   and
   b) the path to the local mount point (In my case: /srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/config )

Copy that path under b) [as i grouped the personal php-user.ini below that path it will be sufficient to just use this path].

4. SSH / Log in into your OMV through any terminal application like Putty on terminal level

5. Execute the following command:

chown nobody:nogroup /srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/config

(Please adopt the part /srv/dev-disk-by-uuid-8b6b6ae4-384d-49c0-9df0-18f8320ada0d/data/system/appdata/tempest_epg/config to the path you copied earlier before you actually execute that command! > Personally I prepare such commands with Notepad before I paste them to the terminal and execute on the respective linux maschine)

Hint:
Contents of my php-user.ini file:

[Date]
date.timezone=Europe/Berlin
[Memory]
memory_limit = 256M

Upload your php-user.ini file to the respective folder which you defined in your compose file.

After that it has worked.
Best EPG results for Germany I got with tvdigital siteconfig.

(Unfortunately with tvinfo - which I wished and got from K-vanc a few weeks ago - it had quite some problems with parsing the page; so i changed to tvdigital siteconfig!)

[K-vanc]

(Unfortunately with tvinfo - which I wished and got from K-vanc a few weeks ago - it had quite some problems with parsing the page; so i changed to tvdigital siteconfig!)

What is the parsing problems of tvinfo.de siteconfig, it seemed to me working normal