Initial Implementation (#1)

* Organize existing user and node-related models into modules.
Move existing unit tests to this repository

* Add missing requirements.txt
Refactor README.md into expected path

* Add `pytest` test dependency

* Fix copy/paste errors in unit test automation
Add "test" extras

* Fix missed import refactor in `node_v1` module

* Add `util` module with function to build JSON schema
Add JSON schema for all models

* Add `pytz` dependency for tz parsing

* Add new `BaseModel` with envvar to configure allowing extra params globally
Document envvar to allow extra params

* Fix syntax error in BaseModel definition

* Ensure license notices in all files
Remove bundled `schema` directory
Update `build_json_schema` util to be more flexible

* Implement `base` module with `Message` object definition (with context sepc)
Update `node_v1` to extend base module and define `__all__`
Normalize newer `lat`/`lon` refs to `latitude`/`longitude` for consistency with database
Remove `verified` config options as those should not be self-reported
Remove unused `brands` config spec
Fix `UserProfile.from_user_object` bugs
Remove top-level import helpers
Add unit tests

* Refactor MQRequest into `mq.UserDbRequest` for extensibility to include other MQ message specs
Define Data schemas for all node messages
Move enum classes related to data schema to this module
Outline repo organization in README.md

* Add test coverage for `node_v1` module
Refactor node_v1 to improve `msg_type` validation, specify `data` models for all objects, and remove models that are used only in one other model

* Refactor UserData enum to be more descriptive

* Add example value to `avatar_url`

* Update `neon_profile` annotations per review comments

* Annotate timestamp fields per review

* Reorganize `AlertData` fields per review

* Default local IP address to an empty string for consistency with other networking parameters

* Add note to `weekdays` enum that it is used for alerts

* Fix copy/paste error in publish release action

* Remove commented out unused model
Add api.mq module import for consistency

.github/workflows/license_tests.yml

@@ -0,0 +1,12 @@
+name: Run License Tests
+on:
+  push:
+  workflow_dispatch:
+  pull_request:
+    branches:
+      - master
+jobs:
+  license_tests:
+    uses: neongeckocom/.github/.github/workflows/license_tests.yml@master
+    with:
+      packages-exclude: '^(neon-data-models|dnspython).*'

.github/workflows/propose_release.yml

@@ -0,0 +1,28 @@
+name: Propose Stable Release
+on:
+  workflow_dispatch:
+    inputs:
+      release_type:
+        type: choice
+        description: Release Type
+        options:
+          - patch
+          - minor
+          - major
+jobs:
+  update_version:
+    uses: neongeckocom/.github/.github/workflows/propose_semver_release.yml@master
+    with:
+      branch: dev
+      release_type: ${{ inputs.release_type }}
+      update_changelog: True
+      version_file: "neon_data_models/version.py"
+  pull_changes:
+    uses: neongeckocom/.github/.github/workflows/pull_master.yml@master
+    needs: update_version
+    with:
+      pr_reviewer: neonreviewers
+      pr_assignee: ${{ github.actor }}
+      pr_draft: false
+      pr_title: ${{ needs.update_version.outputs.version }}
+      pr_body: ${{ needs.update_version.outputs.changelog }}

.github/workflows/publish_release.yml

@@ -0,0 +1,12 @@
+# This workflow will generate a release distribution and upload it to PyPI
+
+name: Publish Build and GitHub Release
+on:
+  push:
+    branches:
+      - master
+
+jobs:
+  build_and_publish_pypi_and_release:
+    uses: neongeckocom/.github/.github/workflows/publish_stable_release.yml@master
+    secrets: inherit

.github/workflows/publish_test_build.yml

@@ -0,0 +1,18 @@
+# This workflow will generate a distribution and upload it to PyPI
+
+name: Publish Alpha Build
+on:
+  push:
+    branches:
+      - dev
+    paths-ignore:
+      - 'neon_data_models/version.py'
+
+jobs:
+  publish_alpha_release:
+    uses: neongeckocom/.github/.github/workflows/publish_alpha_release.yml@master
+    secrets: inherit
+    with:
+      version_file: "neon_data_models/version.py"
+      publish_prerelease: true
+      publish_pypi: true

.github/workflows/unit_tests.yml

@@ -0,0 +1,33 @@
+name: Run Unit Tests
+on:
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  py_build_tests:
+    uses: neongeckocom/.github/.github/workflows/python_build_tests.yml@master
+    with:
+      python_version: "3.9"
+  unit_tests:
+    strategy:
+      matrix:
+        python-version: [ 3.9, "3.10", "3.11" ]
+    timeout-minutes: 15
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up python ${{ matrix.python-version }}
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install system dependencies
+        run: |
+          sudo apt update
+          sudo apt install -y swig gcc libpulse-dev portaudio19-dev
+      - name: Install package
+        run: |
+          python -m pip install --upgrade pip
+          pip install .[test]
+      - name: Run Tests
+        run: |
+          pytest tests

LICENSE.md

@@ -0,0 +1,21 @@
+# NEON AI (TM) SOFTWARE, Software Development Kit & Application Development System
+# All trademark and other rights reserved by their respective owners
+# Copyright 2008-2024 Neongecko.com Inc.
+# BSD-3
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the
+following conditions are met:
+1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following
+disclaimer.
+2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following
+disclaimer in the documentation and/or other materials provided with the distribution.
+3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products
+derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

@@ -0,0 +1,31 @@
+## Neon Data Models
+This repository contains Pydantic models and JSON schemas for common data
+structures. The `models` module contains Pydantic models, organized by application.
+
+## Configuration
+To allow passing or handling parameters that are not explicitly defined in the
+models provided by this package, the `NEON_DATA_MODELS_ALLOW_EXTRA` envvar may
+be set to `true`. This is generally not necessary and helps to prevent sending
+extraneous data, but may help in cases where the server and client are using
+different revisions of this package.
+
+## Organization
+Models are broadly organized into the following categories.
+
+### API
+These schemas are used in API requests and responses. They are grouped by the
+applicable API (node, HANA, mq). Use these schemas for sending requests and
+parsing responses.
+
+### Client
+These schemas are specific to client applications (i.e. Nodes). Use these
+schemas for client-specific configuration.
+
+### User
+These schemas define user-specific data structures. Use these schemas for 
+user-specific configuration.
+
+### Messagebus
+These schemas define messages sent on the messagebus. Historically, messagebus
+events have not used any validation, so there is greater risk of Message objects
+failing validation than other schemas defined here. 

neon_data_models/__init__.py

@@ -0,0 +1,25 @@
+# NEON AI (TM) SOFTWARE, Software Development Kit & Application Development System
+# All trademark and other rights reserved by their respective owners
+# Copyright 2008-2024 Neongecko.com Inc.
+# BSD-3
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+#    this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+#    this list of conditions and the following disclaimer in the documentation
+#    and/or other materials provided with the distribution.
+# 3. Neither the name of the copyright holder nor the names of its
+#    contributors may be used to endorse or promote products derived from this
+#    software without specific prior written permission.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS  BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
+# OR PROFITS;  OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE,  EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

neon_data_models/enum.py

@@ -0,0 +1,83 @@
+# NEON AI (TM) SOFTWARE, Software Development Kit & Application Development System
+# All trademark and other rights reserved by their respective owners
+# Copyright 2008-2024 Neongecko.com Inc.
+# BSD-3
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+#    this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+#    this list of conditions and the following disclaimer in the documentation
+#    and/or other materials provided with the distribution.
+# 3. Neither the name of the copyright holder nor the names of its
+#    contributors may be used to endorse or promote products derived from this
+#    software without specific prior written permission.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS  BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
+# OR PROFITS;  OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE,  EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+from enum import IntEnum
+
+
+class AccessRoles(IntEnum):
+    """
+    Defines access roles such that a larger value always corresponds to more
+    permissions. `0` equates to no permission, negative numbers correspond to
+    non-user roles. In this way, an activity can require, for example,
+    `permission > AccessRoles.GUEST` to grant access to all registered users,
+    admins, and owners.
+    """
+    NONE = 0
+    GUEST = 1
+    USER = 2
+    ADMIN = 3
+    OWNER = 4
+
+    NODE = -1
+
+
+class UserData(IntEnum):
+    """
+    Defines types of user data.
+    """
+    CACHES = 0
+    PROFILE = 1
+    TRANSCRIPTS = 2
+    LIKED_BRANDS = 3
+    DISLIKED_BRANDS = 4
+    ALL_DATA = 5
+    ALL_MEDIA = 6
+    UNITS_CONFIG = 7
+    LANGUAGE_CONFIG = 8
+
+
+class AlertType(IntEnum):
+    """
+    Defines kinds of alerts.
+    """
+    ALL = -1
+    ALARM = 0
+    TIMER = 1
+    REMINDER = 2
+    UNKNOWN = 99
+
+
+class Weekdays(IntEnum):
+    """
+    Defines weekdays as used in the Alerts skill.
+    """
+    MON = 0
+    TUE = 1
+    WED = 2
+    THU = 3
+    FRI = 4
+    SAT = 5
+    SUN = 6

neon_data_models/models/__init__.py

@@ -0,0 +1,25 @@
+# NEON AI (TM) SOFTWARE, Software Development Kit & Application Development System
+# All trademark and other rights reserved by their respective owners
+# Copyright 2008-2024 Neongecko.com Inc.
+# BSD-3
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+#    this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+#    this list of conditions and the following disclaimer in the documentation
+#    and/or other materials provided with the distribution.
+# 3. Neither the name of the copyright holder nor the names of its
+#    contributors may be used to endorse or promote products derived from this
+#    software without specific prior written permission.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS  BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
+# OR PROFITS;  OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE,  EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

neon_data_models/models/api/__init__.py

@@ -0,0 +1,28 @@
+# NEON AI (TM) SOFTWARE, Software Development Kit & Application Development System
+# All trademark and other rights reserved by their respective owners
+# Copyright 2008-2024 Neongecko.com Inc.
+# BSD-3
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+#    this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+#    this list of conditions and the following disclaimer in the documentation
+#    and/or other materials provided with the distribution.
+# 3. Neither the name of the copyright holder nor the names of its
+#    contributors may be used to endorse or promote products derived from this
+#    software without specific prior written permission.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS  BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
+# OR PROFITS;  OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE,  EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+from neon_data_models.models.api.node_v1 import *
+from neon_data_models.models.api.mq import *

neon_data_models/models/api/mq.py

@@ -0,0 +1,40 @@
+# NEON AI (TM) SOFTWARE, Software Development Kit & Application Development System
+# All trademark and other rights reserved by their respective owners
+# Copyright 2008-2024 Neongecko.com Inc.
+# BSD-3
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+#    this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+#    this list of conditions and the following disclaimer in the documentation
+#    and/or other materials provided with the distribution.
+# 3. Neither the name of the copyright holder nor the names of its
+#    contributors may be used to endorse or promote products derived from this
+#    software without specific prior written permission.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS  BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
+# OR PROFITS;  OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE,  EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+from typing import Literal, Optional
+from neon_data_models.models.base.contexts import MQContext
+from neon_data_models.models.user.database import User
+
+
+class UserDbRequest(MQContext):
+    operation: Literal["create", "read", "update", "delete"]
+    username: str
+    password: Optional[str] = None
+    access_token: Optional[str] = None
+    user: Optional[User] = None
+
+
+__all__ = [UserDbRequest.__name__]