Version 6.0.0 (#216)

* Adding Cython support to greatly speed up normal Box operations on supported systems * Adding #161 support for access box dots with `get` and checking with `in` (thanks to scott-createplay) * Adding #183 support for all allowed character sets (thanks to Giulio Malventi) * Adding #196 support for sliceable boxes (thanks to Dias) * Adding #164 default_box_create_on_get toggle to disable setting box variable on get request (thanks to ipcoder) * Changing #208 __repr__ to produce `eval`-able text (thanks to Jeff Robbins) * Changing #215 support ruamel.yaml new syntax (thanks to Ivan Pepelnjak) * Changing `update` and `merge_update` to not use a keyword that could cause issues in rare circumstances * Changing internal `_safe_key` logic to be twice as fast * Removing support for ruamel.yaml < 0.17
cdgriffith · Mar 15, 2022 · 985df83 · 985df83
1 parent 4f9b3a6
commit 985df83
Show file tree

Hide file tree

Showing 26 changed files with 516 additions and 118 deletions.
diff --git a/.github/workflows/pythonpublish.yml b/.github/workflows/pythonpublish.yml
@@ -8,7 +8,7 @@ on:
     types: [created]
 
 jobs:
-  deploy:
+  deploy-generic:
 
     runs-on: ubuntu-latest
 
@@ -29,3 +29,54 @@ jobs:
       run: |
         python setup.py sdist bdist_wheel
         twine upload dist/*
+
+  deploy-cython:
+    strategy:
+      matrix:
+        os: [macos-latest, windows-latest]
+        python-version: ["3.7", "3.8", "3.9", "3.10"]
+    runs-on: ${{ matrix.os }}
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v1
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install setuptools wheel twine Cython --upgrade
+    - name: Build and publish
+      env:
+        TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
+        TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
+      run: |
+        python setup.py bdist_wheel
+        twine upload dist/*
+
+  deploy-cython-manylinux:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python 3.10
+      uses: actions/setup-python@v1
+      with:
+        python-version: "3.10"
+
+    - uses: RalfG/python-wheels-manylinux-build@v0.4.2-manylinux2014_x86_64
+      with:
+        python-versions: 'cp37-cp37m cp38-cp38 cp39-cp39 cp310-cp310'
+        build-requirements: 'cython'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install twine --upgrade
+
+    - name: Publish
+      env:
+        TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
+        TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
+      run: |
+        twine upload dist/*-manylinux*.whl
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
@@ -10,24 +10,28 @@ on:
     branches: [ master, development, develop, test, tests ]
 
 jobs:
-  package_checks:
-    runs-on: ubuntu-latest
+  package-checks:
     strategy:
       matrix:
-        python-version: [3.9]
-
+        python-version: ["3.6", "3.7", "3.8", "3.9", "3.10", "pypy-3.8"]
+        os: [ubuntu-latest, macos-latest, windows-latest]
+    runs-on: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v2
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v1
+        uses: actions/setup-python@v2
         with:
           python-version: ${{ matrix.python-version }}
+      - uses: actions/cache@v2
+        with:
+          path: ~/.cache/pip
+          key: package-check-${{ hashFiles('requirements.txt') }}-${{ hashFiles('requirements-test.txt') }}
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
           pip install -r requirements.txt
           pip install -r requirements-test.txt
-          pip install coveralls flake8 flake8-print mypy setuptools wheel twine
+          pip install coveralls flake8 flake8-print mypy setuptools wheel twine Cython
       - name: Lint with flake8
         run: |
           # stop the build if there are Python syntax errors, undefined names or print statements
@@ -36,28 +40,89 @@ jobs:
           flake8 . --count --exit-zero --max-complexity=20 --max-line-length=120 --statistics --extend-ignore E203
       - name: Run mypy
         run: mypy box
-      - name: Check distrubiton log description
+      - name: Build Wheel and check distrubiton log description
         run: |
           python setup.py sdist bdist_wheel
           twine check dist/*
+      - name: Test packaged wheel on *nix
+        if: matrix.os != 'windows-latest'
+        run: |
+          pip install dist/*.whl
+          rm -rf box
+          python -m pytest
+      - name: Test packaged wheel on Windows
+        if: matrix.os == 'windows-latest'
+        run: |
+          $wheel = (Get-ChildItem dist\*.whl | Sort lastWriteTime | Select-Object -last 1).Name
+          pip install dist\${wheel}
+          Remove-item box -recurse -force
+          python -m pytest
+      - name: Upload wheel artifact
+        uses: actions/upload-artifact@v2
+        with:
+          name: python_box
+          path: dist/*.whl
 
-  test:
+  package-manylinux-checks:
     runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v1
+        with:
+          python-version: "3.10"
+
+      - uses: actions/cache@v2
+        with:
+          path: ~/.cache/pip
+          key: package-manylinux-check-${{ hashFiles('requirements.txt') }}-${{ hashFiles('requirements-test.txt') }}
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+          pip install -r requirements-test.txt
+          pip install coveralls flake8 flake8-print mypy setuptools wheel twine Cython
+
+      - uses: RalfG/python-wheels-manylinux-build@v0.4.2-manylinux2014_x86_64
+        with:
+          python-versions: 'cp37-cp37m cp38-cp38 cp39-cp39 cp310-cp310'
+          build-requirements: 'cython'
+
+      - name: Test packaged wheel on linux
+        run: |
+          pip install dist/*cp310-manylinux*.whl
+          rm -rf box
+          python -m pytest
+
+      - name: Upload wheel artifact
+        uses: actions/upload-artifact@v2
+        with:
+          name: python_box
+          path: dist/*-manylinux*.whl
+
+  test:
     strategy:
       matrix:
-        python-version: [3.6, 3.7, 3.8, 3.9, 3.10-dev, pypy3]
-
+        python-version: ["3.7", "3.8", "3.9", "3.10"]
+        os: [ubuntu-latest, macos-latest, windows-latest]
+    runs-on: ${{ matrix.os }}
     steps:
     - uses: actions/checkout@v2
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v2
       with:
         python-version: ${{ matrix.python-version }}
+    - uses: actions/cache@v2
+      with:
+        path: ~/.cache/pip
+        key: test-${{ hashFiles('requirements.txt') }}-${{ hashFiles('requirements-test.txt') }}
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
         pip install -r requirements.txt
         pip install -r requirements-test.txt
+        pip install setuptools wheel Cython
+        python setup.py build_ext --inplace
     - name: Test with pytest
       env:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

diff --git a/.gitignore b/.gitignore
@@ -94,3 +94,6 @@ ENV/
 .pypirc
 release.bat
 coverage/
+# don't upload cython files
+box/*.c
+.pytest_cache/
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -1,7 +1,7 @@
 repos:
--   repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.0.1
-    hooks:
+- repo: https://github.com/pre-commit/pre-commit-hooks
+  rev: v4.1.0
+  hooks:
     # Identify invalid files
     - id: check-ast
     - id: check-yaml
@@ -27,13 +27,32 @@ repos:
     - id: check-executables-have-shebangs
     - id: end-of-file-fixer
       exclude: ^test/data/.+
--   repo: https://github.com/ambv/black
-    rev: 21.7b0
-    hooks:
+
+- repo: https://github.com/ambv/black
+  rev: 22.1.0
+  hooks:
     - id: black
       args: [--config=.black.toml]
--   repo: https://github.com/pre-commit/mirrors-mypy
-    rev: 'v0.910'
-    hooks:
+
+- repo: local
+  hooks:
+    - id: cythonize-check
+      name: Cythonize
+      entry: python setup.py build_ext --inplace
+      language: system
+      types: [python]
+      pass_filenames: false
+    # cythonize must come before the pytest to make sure we update all c code
+    - id: pytest-check
+      name: Check pytest
+      entry: pytest
+      language: system
+      pass_filenames: false
+      always_run: true
+
+- repo: https://github.com/pre-commit/mirrors-mypy
+  rev: 'v0.931'
+  hooks:
     - id: mypy
-      additional_dependencies: [ruamel.yaml,toml,msgpack]
+      types: [python]
+      additional_dependencies: [ruamel.yaml,toml,msgpack,types-PyYAML,types-toml]
diff --git a/CHANGES.rst b/CHANGES.rst
@@ -1,6 +1,20 @@
 Changelog
 =========
 
+Version 6.0.0
+-------------
+
+* Adding Cython support to greatly speed up normal Box operations on supported systems
+* Adding #161 support for access box dots with `get` and checking with `in` (thanks to scott-createplay)
+* Adding #183 support for all allowed character sets (thanks to Giulio Malventi)
+* Adding #196 support for sliceable boxes (thanks to Dias)
+* Adding #164 default_box_create_on_get toggle to disable setting box variable on get request (thanks to ipcoder)
+* Changing #208 __repr__ to produce `eval`-able text (thanks to Jeff Robbins)
+* Changing #215 support ruamel.yaml new syntax (thanks to Ivan Pepelnjak)
+* Changing `update` and `merge_update` to not use a keyword that could cause issues in rare circumstances
+* Changing internal `_safe_key` logic to be twice as fast
+* Removing support for ruamel.yaml < 0.17
+
 Version 5.4.1
 -------------
 

diff --git a/LICENSE b/LICENSE
@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2017-2020 Chris Griffith
+Copyright (c) 2017-2022 Chris Griffith
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

diff --git a/MANIFEST.in b/MANIFEST.in
@@ -2,3 +2,6 @@ include LICENSE
 include AUTHORS.rst
 include CHANGES.rst
 include box/py.typed
+include box/*.c
+include box/*.so
+include box/*.pyd
diff --git a/README.rst b/README.rst
@@ -23,24 +23,64 @@ Check out the new `Box github wiki <https://github.com/cdgriffith/Box/wiki>`_ fo
 Install
 =======
 
+**Version Pin Your Box!**
+
+If you aren't in the habit of version pinning your libraries, it will eventually bite you.
+Box has a `list of breaking change <https://github.com/cdgriffith/Box/wiki/Major-Version-Breaking-Changes>`_ between major versions you should always check out before updating.
+
+requirements.txt
+----------------
+
+.. code:: text
+
+        python-box[all]~=6.0
+
+As Box adheres to semantic versioning (aka API changes will only occur on between major version),
+it is best to use `Compatible release <https://www.python.org/dev/peps/pep-0440/#compatible-release>`_ matching using the `~=` clause.
+
+Install from command line
+-------------------------
+
 .. code:: bash
 
-        pip install --upgrade python-box[all]
+        pip install python-box[all]~=6.0 --upgrade
+
+Install with selected dependencies
+----------------------------------
 
-Box 5 is no longer forcing install of external dependencies such as yaml and toml. Instead you can specify which you want,
-for example, `all` is shorthand for:
+Box is no longer forcing install of external dependencies such as yaml and toml. Instead you can specify which you want,
+for example, `[all]` is shorthand for:
 
 .. code:: bash
 
-        pip install --upgrade python-box[ruamel.yaml,toml,msgpack]
+        pip install python-box[ruamel.yaml,toml,msgpack]~=6.0 --upgrade
 
-But you can also sub out "ruamel.yaml" for "PyYAML".
+But you can also sub out `ruamel.yaml` for `PyYAML`.
 
 Check out `more details <https://github.com/cdgriffith/Box/wiki/Installation>`_ on installation details.
 
-Box 5 is tested on python 3.6+ and pypy3, if you are upgrading from previous versions, please look through
-`any breaking changes and new features <https://github.com/cdgriffith/Box/wiki/Major-Version-Breaking-Changes-and-New-Features>`_.
+Box 6 is tested on python 3.6+, if you are upgrading from previous versions, please look through
+`any breaking changes and new features <https://github.com/cdgriffith/Box/wiki/Major-Version-Breaking-Changes>`_.
+
+Optimized Version
+-----------------
+
+Box 6 is introducing Cython optimizations for major platforms by default.
+Loading large data sets can be up to 10x faster!
+
+If you are **not** on a x86_64 supported system you will need to do some extra work to install the optimized version.
+There will be an warning of "WARNING: Cython not installed, could not optimize box" during install.
+You will need python development files, system compiler, and the python packages `Cython` and `wheel`.
+
+**Linux Example:**
+
+First make sure you have python development files installed (`python3-dev` or `python3-devel` in most repos).
+You will then need `Cython` and `wheel` installed and then install (or re-install with `--force`) `python-box`.
+
+.. code:: bash
 
+        pip install Cython wheel
+        pip install python-box[all]~=6.0 --upgrade --force
 
 If you have any issues please open a github issue with the error you are experiencing!
 
@@ -96,7 +136,7 @@ Also special shout-out to PythonBytes_, who featured Box on their podcast.
 License
 =======
 
-MIT License, Copyright (c) 2017-2020 Chris Griffith. See LICENSE_ file.
+MIT License, Copyright (c) 2017-2022 Chris Griffith. See LICENSE_ file.
 
 
 .. |BoxImage| image:: https://raw.githubusercontent.com/cdgriffith/Box/master/box_logo.png

diff --git a/box/__init__.py b/box/__init__.py
@@ -2,14 +2,15 @@
 # -*- coding: utf-8 -*-
 
 __author__ = "Chris Griffith"
-__version__ = "5.4.1"
+__version__ = "6.0.0"
 
 from box.box import Box
 from box.box_list import BoxList
 from box.config_box import ConfigBox
 from box.exceptions import BoxError, BoxKeyError
 from box.from_file import box_from_file
 from box.shorthand_box import SBox
+import box.converters
 
 __all__ = [
     "Box",