cannot import service module

[mvgarcia07]

My application containing several modules with several inits on it and file dependency.

I am having an issue is not finding my own modules, how should I tell in setup.py to include them?

Here are my logs, By the way, the logs are very helpful!

[19684] 2021/01/20 19:52:49.446 starting logging at level ERROR
[19684] 2021/01/20 19:52:52.181 Python exception encountered:
[19684] 2021/01/20 19:52:52.181     Internal Message: cannot import service module
[19684] 2021/01/20 19:52:52.181     Type => <class 'ModuleNotFoundError'>
[19684] 2021/01/20 19:52:52.181     Value => No module named 'app.api.functions.func_gral'
[19684] 2021/01/20 19:52:52.183     Traceback (most recent call last):

[19684] 2021/01/20 19:52:52.183       File "<frozen zipimport>", line 259, in load_module

[19684] 2021/01/20 19:52:52.183       File "mypath\Model\ServiceHandler.py", line 12, in <module>
    from init import app

[19684] 2021/01/20 19:52:52.183       File "<frozen zipimport>", line 259, in load_module

[19684] 2021/01/20 19:52:52.183       File "mypath\Model\init.py", line 4, in <module>
    app = create_app('default')

[19684] 2021/01/20 19:52:52.183       File "mypathP\Model\app\__init__.py", line 13, in create_app
    from .api import api as api_blueprint

[19684] 2021/01/20 19:52:52.183       File "mypath\Model\app\api\__init__.py", line 5, in <module>
    from . import model

[19684] 2021/01/20 19:52:52.183       File "C:mypath\Model\app\api\model.py", line 3, in <module>
    from .functions.func_gral import *

[19684] 2021/01/20 19:52:52.183     ModuleNotFoundError: No module named 'app.api.functions.func_gral'

[19684] 2021/01/20 19:52:52.189 ending logging

my setup.py

from cx_Freeze import setup, Executable
import sys, os

options = {
    "build_exe": {
        "includes": ["ServiceHandler","cx_Logging"],
        "excludes": ["tkinter"], 
        "path": sys.path,
        "include_files": ['.\\app\\Models\\instanties\\sklear_randomforestclassifierv1.pkl', '.\\app\\api\\functions']
        
    }
}

[marcelotduarte]

Use "packages": ["app.api"] or similar, and remove the '.\app\api\functions' in include_files.

[marcelotduarte]

It is like this?
executables = [ Executable("Config.py", base="Win32Service", target_name="FlaskService.exe") ]
And setup has it?

[mvgarcia07]

I have to renamed because I already have a config for my API, is that an issue?
executables = [ Executable( "Configws.py", base="Win32Service", target_name="FlaskService.exe" ) ]

[marcelotduarte]

The name of the base config can be anyone. n the base console, the init script starts the import of the packages, and cx_Freeze can find the packages and modules. In the service the config has only variables, then the finder of packages and modules starts in the service handler.
Maybe a relative import in this ServiceHandler can be the cause of the trouble.

[mvgarcia07]

my init.py

from flask import Flask
from app import create_app

app = create_app('default')

my ServiceHanlder.py notice I was trying to add the system path but didn't work either.

import threading
import os
import sys
import cx_Logging
from init import app

#sys.path.append(os.path.dirname(__name__))
#os.chdir(sys._MEIPASS) #change to the current directory

class Handler:

    # no parameters are permitted; all configuration should be placed in the
    # configuration file and handled in the Initialize() method
    def __init__(self):
        self.stopEvent = threading.Event()
        self.stopRequestedEvent = threading.Event()

    # called when the service is starting
    def initialize(self, configFileName):
        self.directory = os.path.dirname(sys.executable)
        cx_Logging.StartLogging(os.path.join(self.directory, "teste.log"), cx_Logging.DEBUG)
        #pass

    # called when the service is starting immediately after Initialize()
    # use this to perform the work of the service; don't forget to set or check
    # for the stop event or the service GUI will not respond to requests to
    # stop the service
    def run(self):
        cx_Logging.Debug("stdout=%r", sys.stdout)
        sys.stdout = open(os.path.join(self.directory, "stdout.log"), "a")
        sys.stderr = open(os.path.join(self.directory, "stderr.log"), "a")
        app.run(host="127.0.0.1", port=5120)
        self.stopRequestedEvent.wait()
        self.stopEvent.set()

    # called when the service is being stopped by the service manager GUI
    def stop(self):
        self.stopRequestedEvent.set()
        self.stopEvent.wait()

[mvgarcia07]

I was able to overpass the module issue by adding__init__.py in the functions directory. So I was able to compile and install.

[mvgarcia07]

@marcelotduarte I was able to overpass the module issue by adding __init__.py in the functions directory. So I was able to compile and install.

However, the application stays in starting mode and is not running, so I can not see the logs.

Any hints on that one? Is any way I test the .exe without installing just to make sure is working?

Thanks in advance!

Should I open a new Q&A?

[marcelotduarte]

I think that you are confused ;-)
You cannot mixup the console and the service. You can have the two in separate executables, in your setup:

executables = [
    Executable(
        "consoleapp.py",
        base="Console",
        target_name="FlaskConsole"
    ),
    Executable(
        "Config.py", # or other name, but not the ServiceHandler that has the class Handler
        base="Win32Service",
        target_name="FlaskService"
    )
]

#consoleapp.py

from init import app
if __name__ == "__main__":
    app.run(host="127.0.0.1",port=5123)

[mvgarcia07]

@marcelotduarte thanks for taking the time and the hint to compile both at the same time!, I fixed and it is working, I added the logs code to the consoleapp.py in my case the init.py in my case and I can see it, not a problem with that. But as I said there are no issues with the console. I know the console is working.

My problem is the window services not starting and not having logs to troubleshoot.
https://github.com/mvgarcia07/python-flask-as-window-service/blob/main/cx_Freeze_example_modules/setup.py
https://github.com/mvgarcia07/python-flask-as-window-service/blob/main/cx_Freeze_example_modules/ServiceHandler.py

I am in desperate mode. I feel I am close, Probably I am doing something stupid and I don't see it because as you can tell I am not an expert in windows services or creating .exe from python.

I am trying to understand why it is working with a simple flask and not with a flask app with submodules and dependencies that is a well as simple.

Any hint I kindly appreciated. I need to conclude if this is possible with cx_freeze by tomorrow.

[marcelotduarte]

I would love to help you more, but I don't know if I can, but I'll try, that we can't give up...
Ah, upgrade to cx_Freeze 6.5.3
It is probably a problem with some package or something not allowed in services. For example, in console mode, sys.stdout and sys.stderr are shown on the screen, but in service mode they are None, so we had to create a file to save them.
And if some package check for input in sys.stdin, the service breaks.
Another thing can be a socket timeout. You can try use socket.setdefaulttimeout(10) in the initialize method.

There are a possibility to work around some of these issues using multiprocessing, and that console app will be useful.
Are you familiar with multiprocessing? Take a look in docs before read the following thread:
#536
There are 2 cases of success in that.

[mvgarcia07]

Did the upgrade, and changed AUTO_START = False and added socket.setdefaulttimeout(10)
I start the service manually and I was able to see the timeout error;

Error 1053: The Service Did Not Respond to the Start... This is the same error I was having with pyinstaller.

I understand the Error 1053 is because the application is taking more than 30 sec to launch. What is strange is that keeps hanging there and no stopping, I have to restart the machine to delete the application because --uninstall test is not enough.

I am not familiar with multiprocessing, I can give it a try if, you think I reached a dead-end having the service handler as it is.

[mvgarcia07]

@marcelotduarte thanks for the support.
I think I am done, if you have the time or the team here is the code. I look all over the internet, not body seems to have a solution neither for cx_freeze or pyinstaller.

I narrowed it down to a simple example so is easy to troubleshoot.
https://github.com/mvgarcia07/python-flask-as-window-service/tree/main/cx_Freeze_example_modules