prep 0.6.0 release (#53)

* Use toml setup (#47)

* use modern packaging format, and update to opendssdirect.py v0.7.0

* update readme with additional guidance for linting/formatting/etc

* pre-commit configs

* bump ruff & black versions in pre-commit

* remove license from dynamic pyproject setup because illegal, apparently

* move opendssdirect.py upgrade to #48

* Run pre commit (#50)

* remove ruff from pre-commit

* formatting by black

* the ruff alerts in update_licenses are acceptable and should be ignored

* address ruff concerns in cli

* ignore more linting warnings in read.py

* address (in some cases ignore) linting in read.py

* remove "ub" parameter from _get_xfmr_overloads() in urbanopt_ditto_reader.py

* ignore complexity linting in urbanopt_ditto_reader.py as well

* address (in some cases ignore) linting in urbanopt_ditto_reader.py

* Tests (#48)

* new empty test files

* new empty github actions ci config file

* update release instructions for new package style

* add .ruff_cache to gitignore

* add newline at end of changelog file

* add rnm output for testing

* tiny word change in readme

* remove license from dynamic section in pyproject.toml

* remove example/test output from git cache/tracking

* different path for example/test output in gitignore file

* remove example/test transformers from git cache/tracking

* Revert "remove example/test transformers from git cache/tracking"

This reverts commit 0d21c1c.

* allow command line flags to override config file

* adjust line-length to 90 characters

* fix typo in cli help text

* remove unnecessary test file

* rename test file

* comment why reopt test is commented out

* set up github actions for ci

* set python >= 3.8 in ci. needed for pre-commitv3

* set python >= 3.8 in repo

* restore license file accidentally deleted

* fix local import statement to match some historical refactor

* change line-length to 120 chars to not have so many line-breaks

* raise SystemExit cleanly for elegant warning to user

* allow user args to override json config appropriately

* ignore a few more lint checks in the test suite

* add subprocess check to test calls

* remove test output from git

* run ci weeknights and on PR review request

* bump ditto-reader to v0.6.0 and black/pre-commit versions

* use opendssdirect.py v0.8

* make cli error message an f-string like it is supposed to be

* slightly better way of confirming rnm workflow ran correctly

* update ruff to 0.1.0

* use correct min Python version in readme

* update linting dependencies in pre-commit

* update checkout action version in CI, and add detail to step names

* use traitlets <5.10 due to breaking change found in ditto.py

* declare support for Python3.12

* rounding outputs to 5 decimal places

* run CI on push instead of schedule

* `black` formatting

* autoupdate pre-commit hook versions

* un-ignore opendss files

* regenerate example opendss files by running the test suite

* use 2 hour timestep in test

* regen opendss files at 2 hour timestep

---------

Co-authored-by: tarekelgindy <tarek.elgindy@nrel.gov>

---------

Co-authored-by: tarekelgindy <tarek.elgindy@nrel.gov>

---------

Co-authored-by: tarekelgindy <tarek.elgindy@nrel.gov>

* remove outdated opendssdirect.py installation recommendation

* Cleaner dependencies (#51)

* traitlets issue fixed in ditto.py

* use ruff formatter instead of black

* ignore a rule when using ruff formatter

* apply ruff formatter

* bump ruff, remove black, reinstate traitlets restriction until ditto releases

* depend on ditto@master instead of released version - TEMPORARY

* use ditto.py fork containing bugfix

* use released version of ditto.py, hooray!

* New workflow to publish to pypi when we release on github (#52)

* WIP: new workflow to publish to pypi when we release on github [skip ci]

* update readme with new release instructions

---------

Co-authored-by: tarekelgindy <tarek.elgindy@nrel.gov>

.github/workflows/ci.yml

@@ -0,0 +1,32 @@
+# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions
+name: ci
+on:
+  push:
+  # schedule:
+    # https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule
+    # 5:20 am UTC (11:20pm MDT the day before) every weekday night in MDT
+  #   - cron: '20 5 * * 2-6'
+  # pull_request:
+  #   types: [review_requested]
+
+jobs:
+  test:
+    strategy:
+      matrix:
+        os: [ubuntu-latest, windows-latest]
+        python-version: ["3.8", "3.12"]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install ditto-reader
+        run: pip install .[dev]
+      - name: Run pre-commit to confirm linting and formatting
+        # https://github.com/pre-commit/action
+        uses: pre-commit/action@v3.0.0
+        with:
+          extra_args: --all-files
+      - name: Run unit tests
+        run: python -Wonce::DeprecationWarning -Im pytest -vv

.github/workflows/release.yml

@@ -0,0 +1,55 @@
+name: PyPIRelease
+# https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: 3.12
+
+      - name: Install pypa/build
+        run: >-
+          python3 -m
+          pip install
+          build
+          --user
+
+      - name: Build a binary wheel and a source tarball
+        run: python3 -m build
+
+      - name: Store the distribution packages
+        uses: actions/upload-artifact@v3
+        with:
+          name: python-package-distributions
+          path: dist/
+
+  release:
+    name: >-
+      Publish Python 🐍 distribution 📦 to PyPI
+    needs:
+    - build
+    runs-on: ubuntu-latest
+    environment:
+      name: release
+      url: https://pypi.org/p/urbanopt-ditto-reader
+    permissions:
+      id-token: write  # IMPORTANT: this permission is mandatory for trusted publishing
+
+    steps:
+      - name: Download all the dists
+        uses: actions/download-artifact@v3
+        with:
+          name: python-package-distributions
+          path: dist/
+      - name: Publish distribution 📦 to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

.gitignore

@@ -112,6 +112,7 @@ venv.bak/
 
 # IDE settings
 .vscode/
+.ruff_cache
 
 # Spyder project settings
 .spyderproject
@@ -134,8 +135,5 @@ dmypy.json
 # DS_Store
 .DS_Store
 
-# opendss files
-./example/run/baseline_scenario/opendss/
-
 # ditto local checkout
 ditto/

.pre-commit-config.yaml

@@ -0,0 +1,39 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.5.0
+    hooks:
+      - id: check-ast
+      - id: check-added-large-files
+        args: ['--maxkb=2000']
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+      - id: check-toml
+      - id: check-yaml
+      - id: check-json
+      - id: name-tests-test
+        args: ["--pytest-test-first"]
+      - id: fix-byte-order-marker
+      - id: check-case-conflict
+      - id: check-docstring-first
+      - id: check-executables-have-shebangs
+      - id: check-merge-conflict
+      - id: debug-statements
+      # - id: detect-aws-credentials
+      - id: detect-private-key
+  # - repo: meta
+    # hooks:
+      # - id: check-useless-excludes # Ensure the exclude syntax is correct
+      # - id: check-hooks-apply # Fails if a hook doesn't apply to any file
+  # Run the Ruff linter
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.1.6
+    hooks:
+      - id: ruff
+        args: [--fix, --exit-non-zero-on-fix]
+        types_or: [python, pyi, jupyter]
+  # Run the Ruff formatter
+  # https://docs.astral.sh/ruff/integrations/#pre-commit
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.1.6
+    hooks:
+      - id: ruff-format

CHANGELOG.md

@@ -98,4 +98,4 @@ Date: 3/31/2020
 # Version 0.1.0
 Date: 3/31/2020
 
-- Initial Release of the URBANopt DiTTo Reader
+- Initial Release of the URBANopt DiTTo Reader

README.md

@@ -3,7 +3,7 @@ Enhancement of URBANopt™ GeoJSON that can be consumed by DiTTo reader \
 More detailed documentation is available on the [URBANopt documentation page](https://docs.urbanopt.net/opendss/opendss.html)
 
 # Installation Pre-requisites
-- Python >=3.7
+- Python >=3.8
 - Requires Python 3.10 if using via the URBANopt CLI.
 
 # Installation
@@ -35,7 +35,7 @@ You are expected to have an existing URBANopt project dir with successful simula
 1. "use_reopt": Required, Boolean (True/False) to analyze reopt data, if it has been provided
 1. "start_date": Optional, String, Indicates the start date of the simulation. Uses format "YYYY/MM/DD"
 1. "start_time": Optional, String, Indicates the start time of the simulation. Uses format
-   "HH:MM:SS". 
+   "HH:MM:SS".
 The start_date and
    start_time are concatenated to get the timestamp (using format "YYYY/MM/DD HH:MM:SS") for the config
    file that is cross referenced with the timestamps in the
@@ -57,26 +57,19 @@ If either start_time and end_time are invalid or set to None, the simulation wil
 
 # Developer installation
 
-Clone the repository:
-`git clone https://github.com/urbanopt/urbanopt-ditto-reader.git`
-
-Change directories into the repository:
-`cd urbanopt-ditto-reader`
-
-If you are using an ARM chip (Apple Silicon) computer, install this branch of OpenDSSDirect.py:
-`pip install git+https://github.com/dss-extensions/OpenDSSDirect.py@dss_python-0.12.0`
-
-Install the respository:
-
-`pip install -e .`
-
-
-## Publish Package
-
-- update version in setup.py
-- make a release on GitHub
-- make the package: `python setup.py sdist`
-- install twine `pip install twine`
-- upload to pypi: `twine upload dist/*`
-
-
+- Clone the repository: `git clone https://github.com/urbanopt/urbanopt-ditto-reader.git`
+- Change directories into the repository: `cd urbanopt-ditto-reader`
+- As general guidance, we recommend using virtual environments to avoid dependencies colliding between your Python projects. [venv](https://docs.python.org/3/library/venv.html) is the Python native solution that will work everywhere, though other options may be more user-friendly.
+    - Some popular alternatives are:
+        - [pyenv](https://github.com/pyenv/pyenv) and [the virtualenv plugin](https://github.com/pyenv/pyenv-virtualenv) work together nicely for Linux/Mac machines
+        - [virtualenv](https://virtualenv.pypa.io/en/latest/)
+        - [miniconda](https://docs.conda.io/projects/miniconda/en/latest/)
+    - Activate pre-commit (only once, after making a new venv): `pre-commit install`
+    - Runs automatically on your staged changes before every commit
+- Update pip and setuptools: `pip install -U pip setuptools`
+- Install the respository with developer dependencies: `pip install -e .[dev]`
+- To check the whole repo, run `pre-commit run --all-files`
+    - Settings and documentation links for pre-commit and ruff are in .pre-commit-config.yaml and pyproject.toml
+
+# Releasing
+Increment the version in pyproject.toml. Use [semantic versioning](https://semver.org/). When a new release is made in GitHub, a [workflow](https://github.com/marketplace/actions/pypi-publish) automatically publishes to PyPI.

example/example_project_with_electric_network.json

@@ -1801,7 +1801,7 @@
         ]
       }
     }
-    
+
 
   ]
 }

example/run/baseline_scenario/opendss/dss_files/CNData.dss

@@ -1,2 +1 @@
 New CNDATA.UG_AL_750kcmil_15_kV InsLayer=0.01 DiaIns=0.0420218 k=24 DiaStrand=0.0020499999999999997 Rstrand=0.0058592 Diam=0.022021799999999998 DiaCable=0.04572 Rac=7.549999999999999e-05 GmrStrand=0.0007994 GMRac=0.008575300000000001 Runits=m Radunits=m GMRunits=m
-

example/run/baseline_scenario/opendss/dss_files/LineGeometry.dss

@@ -1,4 +1,3 @@
 New LineGeometry.Geometry_1 Nconds=4 Nphases=3 Units=m Cond=1 Wire=OH_ACSR_477kcmil X=0.0 H=10.0 Normamps=660.0 Cond=2 Wire=OH_ACSR_477kcmil X=0.304 H=10.0 Normamps=660.0 Cond=3 Wire=OH_ACSR_477kcmil X=0.608 H=10.0 Normamps=660.0 Cond=4 Wire=OH_ACSR_477kcmil X=0.0 H=9.6952 Normamps=660.0 Reduce=y
 
 New LineGeometry.Geometry_2 Nconds=3 Nphases=3 Units=m Cond=1 CNCable=UG_AL_750kcmil_15_kV X=0.0 H=-2.0 Normamps=640.0 Cond=2 CNCable=UG_AL_750kcmil_15_kV X=0.152 H=-2.0 Normamps=640.0 Cond=3 CNCable=UG_AL_750kcmil_15_kV X=0.304 H=-2.0 Normamps=640.0 Reduce=n
-

example/run/baseline_scenario/opendss/dss_files/Lines.dss

@@ -43,4 +43,3 @@ New Line.68c17aed-1c16-4a75-b0a7-7cf2db5d605d Units=km Length=0.1798320000000000
 New Line.db1a87fb-d770-4939-b747-7cadf1f304f8 Units=km Length=0.039014400000000005 bus1=7bba9925-4f7c-4f0d-9b94-d0b8298962ea.1.2.3 bus2=194f2b59-5877-4485-85b9-bd0ac6f75e25.1.2.3 switch=n enabled=y phases=3 geometry=Geometry_1
 
 New Line.bfdf4e28-5baf-4d54-9d0a-5a7dc3fbd4f3 Units=km Length=0.0697992 bus1=258aad84-ed22-48ee-995a-4c5869ae9274.1.2.3 bus2=b9ac3056-629d-40cc-b520-cfe15eb7296c.1.2.3 switch=n enabled=y phases=3 geometry=Geometry_1
-

example/run/baseline_scenario/opendss/dss_files/LoadShapes.dss

@@ -1,26 +1,25 @@
-New Loadshape.feature_1 npts= 8759 interval=1.0 mult = (file=..\profiles\load_1_pu.csv)
+New Loadshape.feature_1 npts= 8759 interval=1.0 mult = (file=../profiles/load_1_pu.csv)
 
-New Loadshape.feature_2 npts= 8759 interval=1.0 mult = (file=..\profiles\load_2_pu.csv)
+New Loadshape.feature_2 npts= 8759 interval=1.0 mult = (file=../profiles/load_2_pu.csv)
 
-New Loadshape.feature_3 npts= 8759 interval=1.0 mult = (file=..\profiles\load_3_pu.csv)
+New Loadshape.feature_3 npts= 8759 interval=1.0 mult = (file=../profiles/load_3_pu.csv)
 
-New Loadshape.feature_4 npts= 8759 interval=1.0 mult = (file=..\profiles\load_4_pu.csv)
+New Loadshape.feature_4 npts= 8759 interval=1.0 mult = (file=../profiles/load_4_pu.csv)
 
-New Loadshape.feature_5 npts= 8759 interval=1.0 mult = (file=..\profiles\load_5_pu.csv)
+New Loadshape.feature_5 npts= 8759 interval=1.0 mult = (file=../profiles/load_5_pu.csv)
 
-New Loadshape.feature_6 npts= 8759 interval=1.0 mult = (file=..\profiles\load_6_pu.csv)
+New Loadshape.feature_6 npts= 8759 interval=1.0 mult = (file=../profiles/load_6_pu.csv)
 
-New Loadshape.feature_7 npts= 8759 interval=1.0 mult = (file=..\profiles\load_7_pu.csv)
+New Loadshape.feature_7 npts= 8759 interval=1.0 mult = (file=../profiles/load_7_pu.csv)
 
-New Loadshape.feature_8 npts= 8759 interval=1.0 mult = (file=..\profiles\load_8_pu.csv)
+New Loadshape.feature_8 npts= 8759 interval=1.0 mult = (file=../profiles/load_8_pu.csv)
 
-New Loadshape.feature_9 npts= 8759 interval=1.0 mult = (file=..\profiles\load_9_pu.csv)
+New Loadshape.feature_9 npts= 8759 interval=1.0 mult = (file=../profiles/load_9_pu.csv)
 
-New Loadshape.feature_10 npts= 8759 interval=1.0 mult = (file=..\profiles\load_10_pu.csv)
+New Loadshape.feature_10 npts= 8759 interval=1.0 mult = (file=../profiles/load_10_pu.csv)
 
-New Loadshape.feature_11 npts= 8759 interval=1.0 mult = (file=..\profiles\load_11_pu.csv)
+New Loadshape.feature_11 npts= 8759 interval=1.0 mult = (file=../profiles/load_11_pu.csv)
 
-New Loadshape.feature_12 npts= 8759 interval=1.0 mult = (file=..\profiles\load_12_pu.csv)
-
-New Loadshape.feature_13 npts= 8759 interval=1.0 mult = (file=..\profiles\load_13_pu.csv)
+New Loadshape.feature_12 npts= 8759 interval=1.0 mult = (file=../profiles/load_12_pu.csv)
 
+New Loadshape.feature_13 npts= 8759 interval=1.0 mult = (file=../profiles/load_13_pu.csv)

example/run/baseline_scenario/opendss/dss_files/Loads.dss

@@ -1,6 +1,6 @@
 New Load.1 bus1=e4bcf9d6-e0b7-421c-91d4-554d06c69364.1.2.3 kV=12.47 model=1 kW=4961.147721461635 kvar=1138.1653453593858 Phases=3 yearly=feature_1
 
-New Load.2 bus1=53c917d8-b04d-4110-bdd9-cbf044e6a1fa.1.2.3 kV=0.48 model=1 kW=226.26934267290764 kvar=51.909747301714695 Phases=3 yearly=feature_2
+New Load.2 bus1=53c917d8-b04d-4110-bdd9-cbf044e6a1fa.1.2.3 kV=0.48 model=1 kW=226.26934267290758 kvar=51.90974730171469 Phases=3 yearly=feature_2
 
 New Load.3 bus1=67b07fac-6666-4954-98b8-3e08b0896646.1.2.3 kV=12.47 model=1 kW=1277.8064698880355 kvar=293.1489090339154 Phases=3 yearly=feature_3
 
@@ -10,7 +10,7 @@ New Load.5 bus1=3d1aa47d-69dd-4200-ab34-9b6244ad82d7.1.2.3 kV=0.48 model=1 kW=12
 
 New Load.6 bus1=db52eea7-f395-4fb1-999d-47c82333a4a7.1.2.3 kV=0.48 model=1 kW=15.74178000285817 kvar=3.6114120117846387 Phases=3 yearly=feature_6
 
-New Load.7 bus1=9d037967-7ac1-4296-8f87-2d4a842d7a87.1.2.3 kV=0.48 model=1 kW=1010.9459946369252 kvar=231.9269172631351 Phases=3 yearly=feature_7
+New Load.7 bus1=9d037967-7ac1-4296-8f87-2d4a842d7a87.1.2.3 kV=0.48 model=1 kW=1010.9459946369254 kvar=231.92691726313512 Phases=3 yearly=feature_7
 
 New Load.8 bus1=18fe5a25-2547-4c32-a225-8bd455e93c9b.1.2.3 kV=12.47 model=1 kW=2185.323191585642 kvar=501.347523741978 Phases=3 yearly=feature_8
 
@@ -23,4 +23,3 @@ New Load.11 bus1=14072af7-3121-4e3d-b85f-c79868702ec9.1.2.3 kV=0.48 model=1 kW=3
 New Load.12 bus1=753b77b9-b88b-443f-9c3b-3cd977bb3558.1.2.3 kV=12.47 model=1 kW=1313.8416564929219 kvar=301.415947814038 Phases=3 yearly=feature_12
 
 New Load.13 bus1=38a42375-ff02-497e-90ef-c5d9eb78850e.1.2.3 kV=12.47 model=1 kW=1139.366097122639 kvar=261.3885093186278 Phases=3 yearly=feature_13
-

example/run/baseline_scenario/opendss/dss_files/Master.dss

@@ -16,4 +16,4 @@ Calcvoltagebases
 
 Buscoords Buscoords.dss
 
-Solve
+Solve

example/run/baseline_scenario/opendss/dss_files/Transformers.dss

@@ -3,4 +3,3 @@ New Transformer.b02e3fe9-77d0-47f9-86cb-e4db960afd62 phases=3 windings=2 wdg=1 c
 New Transformer.54cd0a5a-2ebe-4b39-936a-f238b503c221 phases=3 windings=2 wdg=1 conn=wye Kv=12.47 kva=300.0 %R=0.0105216 bus=3ad29eb4-60b2-4050-a87c-dcd858184622 wdg=2 conn=wye Kv=0.48 kva=300.0 %R=0.0105216 bus=b1003fd0-cd27-4de5-9851-f14382846fc4 XHL=0.0376
 
 New Transformer.502e96e3-853a-4b3d-b88d-fe59af4d7b65 phases=3 windings=2 wdg=1 conn=wye Kv=12.47 kva=1000.0 %R=0.0027648 bus=b9ac3056-629d-40cc-b520-cfe15eb7296c wdg=2 conn=wye Kv=0.48 kva=1000.0 %R=0.0027648 bus=ed866262-f11e-4ea7-b5f9-eecfea8ecbaf XHL=0.0562
-

example/run/baseline_scenario/opendss/dss_files/WireData.dss

@@ -1,2 +1 @@
 New WireData.OH_ACSR_477kcmil GMRac=0.0088 GMRunits=m Diam=0.0218 Radunits=m normamps=660.0 Rac=0.00011999999999999999
-