Initial Update of Windows Packaging (#438)

* Update run.py & .gitignore for windows

- Update run.py to execute on windows as SIGQUIT is not available.
- Update .gitignore to ignore the generate .pyd files
Ref #300, #393

* Update package.py for windows

Add package_windows back into package.py
- Using cx_freeze for freezing installation
- Will be using nsis for actual installer
Tested with python 3.5 64bit on windows 10
Ref #393

* Update makefile for windows (+2 misc)

- Update the makefile to support windows
- Use different bin path in virtualenv
- Use pyd instead of so files
- Tested with Msys2
- Add *.exe to .gitignore
- Fix minor format error in package.py
Ref #393

* Add requirements-windows

Add the requirements-windows.txt
- contains cx-Freeze for bundling
Ref #393

* Add initial setup.nsi

Initial Version of a NSIS installer script
- Multi-user install (install for just one or all)
- Registers uninstaller (more values need to finish up)
- Tested both single and all install / uninstall and works
- Still need to add parameters instead of hardcoded values in some spots
- Need to clean up vendor folders / keys if empty on uninstall
- Need to add the other dupeGuru languages to the language list
- Minor cleanup of script needed as well
Ref #393

* Update setup.nsi

Updates to setup.nsi including:
- Defines from CLI
- Version information (MAJOR, MINOR, PATCH)
- Bits (64 / 32)
- SourcePath (if we wanted something other than build)
- Added extra defines to move application specifics to one location
- Added extra defines for uninstall information
- Added calculation of install size
- Added switching between 64 and 32 bit contexts (need to verify
functionality)
- Updated output file naming
- Added NSIS supported languages which are also supported by dupeGuru
- Added rest of registry keys for uninstall information
- Added missing registry key for installType
- Added removeing Vendor folder and registry key if empty on uninstall

Should be very close to having this installer script ready to integrate
into the package.py script if desired.
Ref #393

* Update README & requirements-windows

Minor update to README to indicate windows is supported.  Add PyQt5 to
requirements-windows.txt to make installation easier.

* Update packaging for windows

- Update package.py to integrate NSIS for windows
- Update makefile to deal with a few additional windows issues
- Add Windows.md to contain specific windows instructions, if we want
this can be merged with README.md
- Minor formatting update to setup.nsi
Ref #393

* Update README & Windows Instructions

- Update the README to include a reference to the Windows instructions.
-  Add some additional notes into Windows Instructions and remove one
incorrect command.
- Update .gitignore to ignore all permutations of env* to allow for
multiple side by side virtual environments (used to build different
versions for windows)
Ref:  #393

* Update Window.md

Fix broken python link and move nsis link for consistency.

* More Details in Windows.md

Update Windows.md including:
- Information on compilier requirements for windows
- Notes about the windows 10 sdk
- Some clarification around some of the steps
- Addition of msys2 links

Going to review this a bit more to polish it up.

Ref #393.

.gitignore

@@ -9,7 +9,7 @@ __pycache__
 
 build
 dist
-env
+env*
 /deps
 cocoa/autogen
 
@@ -19,3 +19,6 @@ cocoa/autogen
 /qt/*_rc.py
 /help/*/conf.py
 /help/*/changelog.rst
+
+*.pyd
+*.exe

Makefile

@@ -1,15 +1,31 @@
 PYTHON ?= python3
+PYRCC5 ?= pyrcc5
 REQ_MINOR_VERSION = 4
 PREFIX ?= /usr/local
 
+# Window compatability via Msys2 
+# - venv creates Scripts instead of bin
+# - compile generates .pyd instead of .so
+# - venv with --sytem-site-packages has issues on windows as well...
+
+ifeq ($(shell uname -o), Msys)
+	BIN = Scripts
+	SO = pyd
+	VENV_OPTIONS = 
+else
+	BIN = bin
+	SO = so
+	VENV_OPTIONS = --system-site-packages
+endif
+
 # Set this variable if all dependencies are already met on the system. We will then avoid the
 # whole vitualenv creation and pip install dance.
 NO_VENV ?=
 
 ifdef NO_VENV
 	VENV_PYTHON = $(PYTHON)
 else
-	VENV_PYTHON = ./env/bin/python
+	VENV_PYTHON = ./env/$(BIN)/python
 endif
 
 # If you're installing into a path that is not going to be the final path prefix (such as a
@@ -61,33 +77,33 @@ ifndef NO_VENV
 	$(VENV_PYTHON) -m pip install -r requirements.txt
 # We can't use the "--system-site-packages" flag on creation because otherwise we end up with
 # the system's pip and that messes up things in some cases (notably in Gentoo).
-	${PYTHON} -m venv --upgrade --system-site-packages env
+	${PYTHON} -m venv --upgrade ${VENV_OPTIONS} env
 endif
 
 build/help : | env
 	$(VENV_PYTHON) build.py --doc
 
 qt/dg_rc.py : qt/dg.qrc
-	pyrcc5 qt/dg.qrc > qt/dg_rc.py
+	$(PYRCC5) qt/dg.qrc > qt/dg_rc.py
 
 i18n: $(mofiles)
 
 %.mo : %.po
 	msgfmt -o $@ $<	
 
-core/pe/_block.*.so : core/pe/modules/block.c core/pe/modules/common.c | env
+core/pe/_block.*.$(SO) : core/pe/modules/block.c core/pe/modules/common.c | env
 	$(VENV_PYTHON) hscommon/build_ext.py $^ _block
-	mv _block.*.so core/pe
+	mv _block.*.$(SO) core/pe
 
-core/pe/_cache.*.so : core/pe/modules/cache.c core/pe/modules/common.c | env
+core/pe/_cache.*.$(SO) : core/pe/modules/cache.c core/pe/modules/common.c | env
 	$(VENV_PYTHON) hscommon/build_ext.py $^ _cache
-	mv _cache.*.so core/pe
+	mv _cache.*.$(SO) core/pe
 
-qt/pe/_block_qt.*.so : qt/pe/modules/block.c | env
+qt/pe/_block_qt.*.$(SO) : qt/pe/modules/block.c | env
 	$(VENV_PYTHON) hscommon/build_ext.py $^ _block_qt
-	mv _block_qt.*.so qt/pe
+	mv _block_qt.*.$(SO) qt/pe
 
-modules : core/pe/_block.*.so core/pe/_cache.*.so qt/pe/_block_qt.*.so
+modules : core/pe/_block.*.$(SO) core/pe/_cache.*.$(SO) qt/pe/_block_qt.*.$(SO)
 
 mergepot :
 	$(VENV_PYTHON) build.py --mergepot
@@ -123,6 +139,6 @@ uninstall :
 clean:
 	-rm -rf build
 	-rm locale/*/LC_MESSAGES/*.mo
-	-rm core/pe/*.so qt/pe/*.so
+	-rm core/pe/*.$(SO) qt/pe/*.$(SO)
 
 .PHONY : clean srcpkg normpo mergepot modules i18n reqs run pyc install uninstall all

README.md

@@ -1,6 +1,6 @@
 # dupeGuru
 
-[dupeGuru][dupeguru] is a cross-platform (Linux and OS X) GUI tool to find duplicate files in
+[dupeGuru][dupeguru] is a cross-platform (Linux, OS X, Windows) GUI tool to find duplicate files in
 a system. It's written mostly in Python 3 and has the peculiarity of using
 [multiple GUI toolkits][cross-toolkit], all using the same core Python code. On OS X, the UI layer
 is written in Objective-C and uses Cocoa. On Linux, it's written in Python and uses Qt5.
@@ -66,6 +66,9 @@ git submodules:
 
 ## How to build dupeGuru from source
 
+### Windows
+For windows instructions see the [Windows Instructions](Windows.md).
+
 ### Prerequisites
 
 * [Python 3.4+][python]

Windows.md

@@ -0,0 +1,59 @@
+## How to build dupeGuru for Windows
+
+### Prerequisites
+
+- [Python 3.5+][python]
+- [Visual Studio 2017][vs] or [Visual Studio Build Tools 2017][vsBuildTools] with the Windows 10 SDK
+- [nsis][nsis] (for installer creation)
+- [msys2][msys2] (for using makefile method)
+
+When installing Visual Studio or the Visual Studio Build Tools with the Windows 10 SDK on versions of Windows below 10 be sure to make sure that the Universal CRT is installed before installing Visual studio as noted in the [Windows 10 SDK Notes][win10sdk] and found at [KB2999226][KB2999226].
+
+After installing python it is recommended to update setuptools before compiling packages.  To update run (example is for python launcher and 3.5):
+
+    $ py -3.5 -m pip install --upgrade setuptools
+
+More details on setting up python for compiling packages on windows can be found on the [python wiki][pythonWindowsCompilers]
+
+### With build.py (preferred)
+To build with a different python version 3.5 vs 3.6 or 32 bit vs 64 bit specify that version instead of -3.5 to the `py` command below.  If you want to build additional versions while keeping all virtual environments setup use a different location for each vritual environment.
+
+    $ cd <dupeGuru directory>
+    $ git submodule init
+    $ git submodule update
+    $ py -3.5 -m venv .\env
+    $ .\env\Scripts\activate
+    $ pip install -r requirements.txt -r requirements-windows.txt
+    $ python build.py
+    $ python run.py
+
+### With makefile
+It is possible to build dupeGuru with the makefile on windows using a compatable POSIX environment.  The following steps have been tested using [msys2][msys2]. Before running make:
+1. Install msys2 or other POSIX environment
+2. Install PyQt5 globally via pip
+3. Use the respective console for msys2 it is `msys2 msys` 
+
+Then the following execution of the makefile should work.  Pass the correct value for PYTHON to the makefile if not on the path as python3.
+
+    $ cd <dupeGuru directory>
+    $ make PYTHON='py -3.5'
+    $ make run
+
+NOTE: Install PyQt5 & cx-Freeze with requirements-windows.txt into the venv before runing the packaging scripts in the section below.
+
+### Generate Windows Installer Packages
+You need to use the respective x86 or x64 version of python to build the 32 bit and 64 bit versions.  The build scripts will automatically detect the python architecture for you. When using build.py make sure the resulting python works before continuing to package.py.  NOTE: package.py looks for the 'makensis' executable in the default location for a 64 bit windows system.  Run the following in the respective virtual environment.
+
+    $ python package.py
+
+### Running tests
+The complete test suite can be run with tox just like on linux.
+
+[python]: http://www.python.org/
+[nsis]: http://nsis.sourceforge.net/Main_Page
+[vs]: https://www.visualstudio.com/downloads/#visual-studio-community-2017
+[vsBuildTools]: https://www.visualstudio.com/downloads/#build-tools-for-visual-studio-2017
+[win10sdk]: https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk
+[KB2999226]: https://support.microsoft.com/en-us/help/2999226/update-for-universal-c-runtime-in-windows
+[pythonWindowsCompilers]: https://wiki.python.org/moin/WindowsCompilers
+[msys2]: http://www.msys2.org/

package.py

@@ -4,13 +4,15 @@
 # which should be included with this package. The terms are also available at
 # http://www.gnu.org/licenses/gpl-3.0.html
 
+import sys
 import os
 import os.path as op
 import compileall
 import shutil
 import json
 from argparse import ArgumentParser
 import platform
+import re
 
 from hscommon.build import (
     print_and_do, copy_packages, build_debian_changelog,
@@ -112,22 +114,88 @@ def package_source_tgz():
         print_and_do('tar -A {} -f {}'.format(archive_path, dest))
     print_and_do('gzip {}'.format(dest))
 
+def package_windows():
+    from cx_Freeze import setup, Executable
+    app_version = get_module_version('core')
+    arch = platform.architecture()[0]
+    buildpath = op.join('build', 'dupeguru-win{}'.format(arch))
+    # remove existing build directory
+    if op.exists(buildpath):
+        shutil.rmtree(buildpath)
+    include_files = []
+    # include locale files if they are built otherwise exit as it will break
+    # the localization
+    if op.exists('build/locale'):
+        include_files.append(('build/locale', 'locale'))
+    else:
+        print("Locale files not built, exiting...")
+        return
+    # include help files if they are built otherwise exit as they should be included?
+    if op.exists('build/help'):
+        include_files.append(('build/help', 'help'))
+    else:
+        print("Help files not built, exiting...")
+        return
+    # options for cx_Freeze
+    # if zip_include packages is not used, the cx_Freeze packager will include
+    # the whole PyQT5 directory
+    options = {
+        'build_exe': {
+            'build_exe': buildpath,
+            'excludes': [],
+            'includes': ['atexit'],
+            'include_files': include_files,
+            'include_msvcr': True,
+            'zip_include_packages': ['*'],
+            'zip_exclude_packages': []
+        },
+    }
+    # executables to build, uses se edition icon
+    executables = [
+        Executable(
+            script='run.py',
+            base='Win32GUI',
+            targetName='dupeguru.exe',
+            icon='images/dgse_logo.ico',
+            copyright='Copyright (C) 2017 Hardcoded Software'
+        )
+    ]
+    # call cx_freeze
+    setup(
+        name='dupeguru',
+        version=app_version,
+        description='Tool to find duplicate files on your computer.',
+        options=options,
+        executables=executables,
+        script_args=['build']
+    )
+    # Information to pass to NSIS
+    version_array = app_version.split('.')
+    match = re.search('[0-9]+', arch)
+    bits = match.group(0)
+    # Call NSIS (TODO update to not use hardcoded path)
+    cmd = ('"C:\\Program Files (x86)\\NSIS\\Bin\\makensis.exe" '
+           '/DVERSIONMAJOR={0} /DVERSIONMINOR={1} /DVERSIONPATCH={2} /DBITS={3} setup.nsi')
+    print_and_do(cmd.format(version_array[0], version_array[1], version_array[2], bits))
+
 def main():
     args = parse_args()
     if args.src_pkg:
         print("Creating source package for dupeGuru")
         package_source_tgz()
         return
     print("Packaging dupeGuru with UI qt")
-    if not args.arch_pkg:
-        distname, _, _ = platform.dist()
+    if sys.platform == 'win32':
+        package_windows()
     else:
-        distname = 'arch'
-    if distname == 'arch':
-        package_arch()
-    else:
-        package_debian()
+        if not args.arch_pkg:
+            distname, _, _ = platform.dist()
+        else:
+            distname = 'arch'
+        if distname == 'arch':
+            package_arch()
+        else:
+            package_debian()
 
 if __name__ == '__main__':
     main()
-

requirements-windows.txt

@@ -0,0 +1,2 @@
+PyQt5 >=5.4,<6.0
+cx-Freeze>=5.0.2,<6.0.0

run.py

@@ -20,7 +20,12 @@
 from qt.platform import BASE_PATH
 from core import __version__, __appname__
 
-from signal import signal, SIGINT, SIGTERM, SIGQUIT
+# SIGQUIT is not defined on Windows
+if sys.platform == 'win32':
+    from signal import signal, SIGINT, SIGTERM
+    SIGQUIT = SIGTERM
+else:
+    from signal import signal, SIGINT, SIGTERM, SIGQUIT
 
 global dgapp
 dgapp = None