Skip to content

Developer Installation

Jump to bottom
Chris Meyer edited this page Jul 23, 2024 · 45 revisions

This guide assumes you are familiar with the Python path and Python environments. See Python Primer for more info.

The Contributing guide highlights what is expected if you submit issues or pull requests.

There are a few installation techniques and file layouts to use when installing Nion Swift. This one assumes you will be installing for development.

Nion Swift Library Structure

The Nion Swift project is broken into several distinct libraries.

  • nionswift contains the Nion Swift application. It utilizes the libraries below.
  • nionutils contains utility functions that are used by other libraries, but not Nion Swift specific.
  • niondata contains the data algorithms and works in conjunctions with NumPy and Scipy.
  • nionui contains a general Pythonic user interface API that is used for Nion Swift, but not Nion Swift specific.
  • nionswift-io contains file handlers for importing/exporting data.
  • nionswift-instrumentation-kit contains acquisition related user interface and algorithms.
  • nionswift-tool contains a Python launcher based on Qt/C++ with high performance display (threaded, low latency, high throughput).

Namespaces and Packages

The nionswift project is split into various packages and namespaces.

The nion.utils namespace in the nionutils package contains utility functions and classes used by the other packages.

The nion.ui namespace in the nionui package contains a general Pythonic user interface API.

The nion.data namespace in the niondata package contains data processing algorithms and works in conjunctions with NumPy and Scipy.

The nion.swift.model namespace in nionswift package contains the non-UI portion of Nion Swift. Classes in this namespace do not utilize UI widgets or canvas items, although they may utilize a drawing context. They also do not reference anything in the nion.swift namespace.

The nion.swift namespace in nionswift package contains the UI portion of Nion Swift. Classes in this namespace may reference anything in any other namespace.

How Nion Swift is Launched

The "Nion Swift" program is a Python program running within a backend application launcher. There are a few different launchers available for different purposes.

  • nionswift-tool is the recommended launcher. It is a native C++ program that provides high performance display capabilities for everyday use.
  • The PySide6 launcher utilizes the official Qt Python bindings to launch Nion Swift. It is native C++ for the Qt part but does not provide high performance display capabilities. However, it is useful for launching Nion Swift with a Python debugger and also for prototyping Qt-based changes to eventually go into nionswift-tool. The PyQtProxy.py file implements the bulk of the underlying UI in Python when launching using this technique.
  • nionui-tool is the underlying C++ launcher on which nionswift-tool is based. It is nearly identical except for the executable name, icon, and default Python application (i.e. branding). This would be the place for development of the launcher tool itself.

To launch Nion Swift using the nionswift-tool from the command line, first ensure both nionswift and nionswift-tool are installed (see below) and then run either of these commands from the command line:

nionswift
python -m nionswift --ui tool --no-fallback

To launch Nion Swift using the nionui-tool from the command line, first ensure both nionswift and nionui-tool are installed (see below) and then run either of these commands from the command line (note: you have to specify the application to run when running nionui-tool, whereas nionswift-tool always uses nionui_app.nionswift as the application):

nionui nionui_app.nionswift
python -m nionui --ui tool --no-fallback nionswift

To launch Nion Swift using the PySide6 Qt package from the command line, first ensure both nionswift and pyside6 are installed (see below) and then run either of these commands from the command line:

python -m nionswift --ui qt --no-fallback
python -m nionui --ui qt --no-fallback nionswift

If you are making changes to nionui-tool itself, you can build it locally and run it directly with something like the following command:

/path/to/NionUILauncher.exe /path/to/python/environment nionui_app.nionswift

If you are making changes to nionui-tool that have been merged but not released, the build artifacts will be available in the Artifacts section of the appropriate workflow run for the GitHub action here.

After finding the appropriate workflow run and the artifact matching your environment, you will download the artifact, unzip it, then install it from the command line using conda install /path/to/conda-artifact.tar.gz or something similar. Then you can use the instructions above to run the installed version of nionui-tool.

Python Environments

Nion Swift can be installed and launched using several Python environments and flavors.

  • Using Miniconda or Miniforge and the conda-forge installation channel. This is useful when you need the latest versions of common libraries.
  • Using PyPI. This is usually installed into a Python virtual environment. This is useful when you do not want to use a conda based installation.
  • Using Anaconda/Miniconda and the nion installation channel. This is useful when you want to use an Anaconda official installation.
  • Using an installer (currently only available for Windows and macOS). The installer includes a full Python environment based on the official Python distribution and the PyPI packages. This is experimental and currently difficult to extend with custom packages.

Once you have decided on your environment, you should activate it and create a virtual environment in which to install Nion Swift. The instructions below assume a conda-forge installation using Miniconda.

On Windows, ensure that the Visual C++ Redistributables for VS 2015 is installed.

On Linux, if you are launching with nionui-tool or nionswift-tool, ensure that qt5-default is installed.

sudo apt-get install qt5-default.

Installation Notes

  • Install NotePad++
  • Install Git
  • Install conda (C:\NionApps\Miniconda3)
C:\NionApps\Miniconda3\Scripts\Activate.bat
conda create -n developer -c conda-forge numpy==1.* scipy pytz h5py imageio pillow tzlocal pip mypy -y
conda activate developer
conda config --env --add channels conda-forge
conda config --env --add channels nion
conda config --env --remove channels defaults
conda config --env --show channels

cd C:\NionApps
mkdir Developer\main
cd Developer\main

git clone https://github.com/nion-software/nionutils.git
git clone https://github.com/nion-software/niondata.git
git clone https://github.com/nion-software/nionui.git
git clone https://github.com/nion-software/nionswift.git
git clone https://github.com/nion-software/nionswift-io.git
git clone https://github.com/nion-software/nionswift-instrumentation-kit.git
git clone https://github.com/nion-software/nionswift-video-capture.git
git clone https://github.com/nion-software/eels-analysis.git nionswift-eels-analysis
git clone https://github.com/nion-software/experimental.git nionswift-experimental

# only if you are installing the microscope simulator.
git clone https://github.com/nion-software/nionswift-usim.git

# editable install type checking: see https://github.com/python/mypy/issues/13392
# use the following line on macos before running the commands below
export SETUPTOOLS_ENABLE_FEATURES="legacy-editable"
# use the following line on windows
set SETUPTOOLS_ENABLE_FEATURES=legacy-editable

python -m pip install --no-deps --editable nionutils
python -m pip install --no-deps --editable niondata
python -m pip install --no-deps --editable nionui
python -m pip install --no-deps --editable nionswift
python -m pip install --no-deps --editable nionswift-io
python -m pip install --no-deps --editable nionswift-instrumentation-kit
python -m pip install --no-deps --editable nionswift-video-capture
python -m pip install --no-deps --editable nionswift-eels-analysis
python -m pip install --no-deps --editable nionswift-experimental

# only if you are installing the microscope simulator.
python -m pip install --no-deps --editable nionswift-usim

conda install -c nion nionswift-tool
  • Copy the DLL files to Python environment (if installing for acquisition).

Updating Conda Environment

conda update --all "numpy<2"

Running with PyCharm CE

  • install PyCharm CE, run with defaults
  • configure the Python interpreter to point to your developer environment
  • create a new project using the directory Developer\main
  • add configuration (Python)
  • enter nionswift for module name and --ui tool --no-fallback for the parameters.
  • double check by running python -m nionswift --ui tool --no-fallback from the command line.

Running with PyCharm CE and Debugger

You can launch nionswift with PyCharm CE using the alternate slower front-end based on Qt for Python.

  • follow the instructions above
  • install the pyside6 package.
  • add configuration (Python)
  • enter nionswift for module name and --ui qt --no-fallback for the parameters.
  • launch the configuration using the debugger.

Running with Visual Studio

Running with the command line

  • Activate your Python environment with Swift installed
  • nionswift from command line

Running Type Checks

Most nionswift related packages enforce strict type checking. To run type checking locally using the command line:

  1. ensure your conda development environment is activated
  2. ensure mypy is installed in your conda development environment
  3. change directory to package to be type checked (i.e. nionswift)
  4. run type checking with mypy command, which will read the file mypy.ini for options specific to the package you're type checking.

You can also install the PyCharm MyPy Plug-in for live type checking, although the plug-in is somewhat less reliable (misses some failures; highlights some things that aren't failures) than running type checking from the command line.

Most type checks are run in strict mode.

In each project, refer to the .github/workflows/python-package.yml for specific commands to run type checking.

This script / gist may serve as a useful template for running mass type checks from the command line.

Running Tests

Each package contributing to nionswift has associated integration tests.

To run the tests for a specific package, add a Run/Debug Configuration (see PyCharm Run/debug configurations) with the Python Tests > Unittests type. Select your preferred Python interpreter. Select Custom. Then add additional arguments according to the specific module you are testing.

Module Target Additional Arguments for Test
nionswift Custom discover -s nionswift/nion/swift/test -p "*_test.py"
nionutils Custom discover -s nionutils/nion/utils/test -p "*_test.py"
niondata Custom discover -s niondata/nion/data/test -p "*_test.py"
nionui Custom discover -s nionui/nion/ui/test -p "*_test.py"
nionswift-io (DM) Module name dm3parser_test.TestDM3ImportExportClass
nionswift-instrumentation-kit Custom discover -s nion/instrumentation/test -p "*_test.py"
eels-analysis (lib) Custom discover nionswift-eels-analysis/nion -v -p '*_test.py'
eels-analysis (UI) Custom discover nionswift-eels-analysis/nionswift_plugin -v -p '*_test.py'

You can also write a script to run all tests at once. You can run scripts like this from PyCharm by creating a new Run/Debug Configuration as a Python configuration. Choose your interpreter and point the script to your test script.

Each project also runs tests for each commit and PR on GitHub actions. To see how these tests are run, refer to the GitHub actino file in each project .github/workflows/python-package.yml. These files contain specific commands to run the tests for their respective package.

Updating from Git

pushd nionutils
git pull origin master
popd
pushd niondata
git pull origin master
popd
pushd nionui
git pull origin master
popd
pushd nionswift
git pull origin master
popd
pushd nionswift-io
git pull origin master
popd
pushd nionswift-instrumentation-kit
git pull origin master
popd
pushd nionswift-video-capture
git pull origin master
popd
pushd nionswift-eels-analysis
git pull origin master
popd
pushd nionswift-experimental
git pull origin master
popd

Installing the Microscope Simulator

git clone https://github.com/nion-software/nionswift-usim.git

pushd nionswift-usim
python setup.py develop --no-deps
popd

pushd nionswift-usim
git pull origin master
popd
Clone this wiki locally