Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: prepare release for v1.1.0 #1

Merged
merged 128 commits into from
May 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
128 commits
Select commit Hold shift + click to select a range
f1bc4a1
still move source code into src folder
Bodong-Yang Feb 13, 2024
b010ad5
minor fix
Bodong-Yang Feb 13, 2024
d3e4c18
add Dockerfile
Bodong-Yang Feb 13, 2024
0c02092
Dockerfile: add mount placeholder for /opt and /greengrass
Bodong-Yang Feb 13, 2024
8ea8013
add example systemd service file and aws_profile_info.yaml
Bodong-Yang Feb 13, 2024
42fe68b
config: credential_endpoint -> credential_endpoint_url; also validate…
Bodong-Yang Feb 13, 2024
4aba116
add .dockerignore
Bodong-Yang Feb 13, 2024
475d5e3
Dockerfile: add missing libcurl4 in the final image
Bodong-Yang Feb 13, 2024
21ddd7d
create a dedicated Dockerfile for logger-server with tpm2.0 support
Bodong-Yang Feb 13, 2024
f623dbd
fix Dockerfile missing libcurl4
Bodong-Yang Feb 13, 2024
ec8c83e
aws_iot_logger: fix logs dropped when log_stream is not created
Bodong-Yang Feb 13, 2024
452e841
remove Dockerfile_tpm2.0 as operating tpm2.0 in container mode is not…
Bodong-Yang Feb 14, 2024
8549f9c
configs: add UPLOAD_LOGGING_SERVER_LOGS and SERVER_LOGSTREAM_SUFFIX, …
Bodong-Yang Feb 14, 2024
60a6816
add support for upload server logging
Bodong-Yang Feb 14, 2024
03557fb
update otaclient-logger.service example
Bodong-Yang Feb 14, 2024
5bdbf21
log_proxy_server: properly typing launch_server function
Bodong-Yang Feb 14, 2024
dc01df6
add support for uploading iot_logging_server logs to cloud too
Bodong-Yang Feb 14, 2024
c475b49
add on_release action
Bodong-Yang Feb 14, 2024
48a144f
pyproject.toml: add awsiot_credentialhelper and pyopenssl deps, remov…
Bodong-Yang Feb 14, 2024
1980917
configs.py: credential_endpoint_url -> credential_endpoint
Bodong-Yang Feb 14, 2024
2652170
update aws_profile_info.yaml examples accordingly
Bodong-Yang Feb 14, 2024
9f7d54c
gg_cfg: add PKCS11 related logics
Bodong-Yang Feb 14, 2024
06154b8
_common: add PKCS11URI def
Bodong-Yang Feb 14, 2024
4c68b14
_utils: add parse_pkcs11_uri
Bodong-Yang Feb 14, 2024
96acb35
gg_config: complete_uri is not needed anymore
Bodong-Yang Feb 14, 2024
9e7917d
fully refactor boto3_session, dip pycurl
Bodong-Yang Feb 14, 2024
6fb5bee
minor update
Bodong-Yang Feb 14, 2024
9d45257
boto3_session: pass priv_key_label into the session
Bodong-Yang Feb 15, 2024
c8d9138
minor update
Bodong-Yang Feb 16, 2024
635187c
configs: use RootModel to define AWSProfileInfo
Bodong-Yang Feb 16, 2024
a5b65d0
aws_iot_logger: refinement
Bodong-Yang Feb 16, 2024
d231106
utils: fix parse_pkcs11_uri
Bodong-Yang Feb 16, 2024
b7d1689
minor cleanup
Bodong-Yang Feb 16, 2024
5ca3899
use github python.gitignore template
Bodong-Yang Feb 26, 2024
dec1d36
pyproject.toml: remove fixing to py3.11 for black and pyright
Bodong-Yang Feb 26, 2024
b9f3278
boto3_session: tear down Boto3Session class into multiple helper func…
Bodong-Yang Feb 26, 2024
763c260
(WIP) for passing strict type check
Bodong-Yang Feb 26, 2024
dfaf78f
boto3_session: pass strict type check
Bodong-Yang Feb 26, 2024
e274266
utils: properly type retry decorator
Bodong-Yang Feb 26, 2024
c0df655
finish up typing fix up, all modules now pass strict type check
Bodong-Yang Feb 26, 2024
d5aa9ae
tests: add sample cert, minor update to gg_v2_cfg.yaml_tpm2.0
Bodong-Yang Feb 26, 2024
57d2605
_common: update typing
Bodong-Yang Feb 27, 2024
0ae74c7
test: add test_utils
Bodong-Yang Feb 27, 2024
1013132
minor fix to test_utils
Bodong-Yang Feb 27, 2024
d8ec536
tests: introduce thing_name differences in gg_v1_cfg and gg_v2_cfg
Bodong-Yang Feb 27, 2024
1f266d1
implement test_greengrass_conf
Bodong-Yang Feb 27, 2024
4e3a6d2
add test_config, cleanup pyproject.toml
Bodong-Yang Feb 28, 2024
ff9ff54
add test__main__.py
Bodong-Yang Feb 28, 2024
3b9805c
minor update to pyproject.toml, add tests/conftest.py
Bodong-Yang Feb 28, 2024
3f2133a
add test_boto3_session
Bodong-Yang Feb 28, 2024
589d60f
common: define package scope type LogsQueue
Bodong-Yang Feb 28, 2024
d08923f
aws_iot_logger: start_sendind_msg_thread -> start_aws_iot_logger_thre…
Bodong-Yang Feb 28, 2024
db3e42b
common: fix LogsQueue as type alias
Bodong-Yang Feb 28, 2024
0af6d68
log_proxy_server: now launch_server only takes care of launching logg…
Bodong-Yang Feb 28, 2024
696f98d
aws_iot_logger: IoTSessionConfig is only used by aws_iot_logger, comp…
Bodong-Yang Feb 28, 2024
5f3518a
main: split logging related logics to a new module _log_setting
Bodong-Yang Feb 28, 2024
b772eb1
cleanup __main__
Bodong-Yang Feb 28, 2024
7e76fb3
split test__main
Bodong-Yang Feb 28, 2024
a80a566
log_proxy_server: minor type fix
Bodong-Yang Feb 28, 2024
8b8cca1
add test_log_proxy_server
Bodong-Yang Feb 28, 2024
dce9b8f
fix test__main__
Bodong-Yang Feb 28, 2024
3c21a11
minor update
Bodong-Yang Feb 28, 2024
d158f78
github_action: implement test CI
Bodong-Yang Feb 28, 2024
eaa6da2
test CI: minor updates
Bodong-Yang Feb 28, 2024
d6d285f
minor update to release CI
Bodong-Yang Feb 28, 2024
fc6d8ff
common: cleanup unused
Bodong-Yang Feb 29, 2024
18d67c7
aws_iot_logger: minor fix typing
Bodong-Yang Feb 29, 2024
67b75b5
add test_aws_iot_logger, minor update to test_log_proxy_server
Bodong-Yang Feb 29, 2024
35f96be
minor change to release CI
Bodong-Yang Feb 29, 2024
f471806
aws_iot_logger: fix main loop breaks on failed upload
Bodong-Yang Feb 29, 2024
9a558e4
log_proxy_server: filter out empty request
Bodong-Yang Feb 29, 2024
b02f80b
configs: add new options ALLOWED_ECUS
Bodong-Yang Feb 29, 2024
601aa2f
silently ignore requests from unknowned ECUs
Bodong-Yang Feb 29, 2024
9ef2bfc
fix configs ALLOWED_ECUS field parsing
Bodong-Yang Feb 29, 2024
4a78b35
update README.md
Bodong-Yang Feb 29, 2024
60f7a6a
fix test
Bodong-Yang Feb 29, 2024
82c22da
aws_iot_logger: add logic to detect failure caused by failed mtls con…
Bodong-Yang Mar 4, 2024
5b07231
aws_iot_logger: move create_log_group from __init__ to thread_main
Bodong-Yang Mar 4, 2024
6609e50
configs: AWS_PROFILE_INFO, iot_logger -> iot-logger
Bodong-Yang Mar 14, 2024
c0d478c
configs: ALLOWED_ECUS now is optional, only enforce checking if ALLOW…
Bodong-Yang Mar 14, 2024
30484b6
fix test_configs
Bodong-Yang Mar 14, 2024
5cd6950
update README.md about ALLOWED_ECUS
Bodong-Yang Mar 14, 2024
19c1c21
update gitignore to ignore local vscode settings
Bodong-Yang Apr 9, 2024
a17a4e0
add gitignore
Bodong-Yang Apr 9, 2024
bc7c257
ci: calculate checksum over built package
Bodong-Yang Apr 9, 2024
2b5ce1e
deps: bump versions
Bodong-Yang Apr 15, 2024
e2991f4
now we parse ecu_info.yaml to only accept logs from known ECUs
Bodong-Yang Apr 15, 2024
8398d25
add stripped version of ecu_info.py
Bodong-Yang Apr 15, 2024
c08df88
integrate stripped ecu_info into log_proxy_server
Bodong-Yang Apr 15, 2024
b0913e7
add test
Bodong-Yang Apr 15, 2024
bcb417c
fix tests
Bodong-Yang Apr 15, 2024
27635b8
log_proxy_server: log loaded allowed_ecu_set
Bodong-Yang Apr 15, 2024
1f9c046
update README.md
Bodong-Yang Apr 24, 2024
befbe9e
update according to sonarcloud recommends
Bodong-Yang May 8, 2024
2d9c44a
greengrass_config: check thing_name length before processing the rege…
Bodong-Yang May 8, 2024
211fb25
dockerfile: only copy src and pyproject.toml
Bodong-Yang May 8, 2024
71d04c8
add sonarcloud.properties
Bodong-Yang May 8, 2024
2c19c32
enable sonarcloud scanning on ci
Bodong-Yang May 8, 2024
0fe6a89
sonarcloud: limit the python version
Bodong-Yang May 8, 2024
36de4b1
fix test
Bodong-Yang May 8, 2024
ff55bcc
fix sonarcloud config
Bodong-Yang May 8, 2024
692dca6
fix sonarcloud projectkey
Bodong-Yang May 8, 2024
10f74b3
test_ci: mark sonarcloud scan as optional for now
Bodong-Yang May 8, 2024
f6c9fe5
pyproject: update deps
Bodong-Yang May 8, 2024
8682201
add pre-commit config files
Bodong-Yang May 8, 2024
284b664
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 8, 2024
c113c07
pre-commit: remove mypy, add black
Bodong-Yang May 8, 2024
f5063cb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 8, 2024
c130ae0
fix according to pre-commit result
Bodong-Yang May 8, 2024
05984bc
fix sonarcloud settings
Bodong-Yang May 8, 2024
ce5bad5
test_ci: fetch all git branches, this is needed for sonarcloud
Bodong-Yang May 8, 2024
80dd014
pyproject: set coverage.run.relative_file to true
Bodong-Yang May 8, 2024
adc4893
update test ci
Bodong-Yang May 8, 2024
38aeaa0
test ci: fix python version
Bodong-Yang May 8, 2024
62813bd
test_ci: only trigger test CI on pr or push to main branch with src o…
Bodong-Yang May 9, 2024
5c2c792
pre-commit: enable markdownlint
Bodong-Yang May 13, 2024
a2613d5
aws_iot_logger: put_log_events now doesn't need sequenceToken anymore
Bodong-Yang May 13, 2024
4914fe6
fix test
Bodong-Yang May 13, 2024
4811680
log_proxy_server: issue a warning when no ecu_info.yaml is presented …
Bodong-Yang May 13, 2024
7c4400e
configs: set the default value of UPLOAD_INTERVAL to a reasonable 3 s…
Bodong-Yang May 13, 2024
58001ce
update README.md
Bodong-Yang May 13, 2024
03a3e4d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 13, 2024
29f3f29
feat: implement config_file_monitor (#2)
Bodong-Yang May 14, 2024
3561785
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 14, 2024
994c60a
add tests for test_config_file_monitor, fix up other tests
Bodong-Yang May 14, 2024
1241472
fix test_log_proxy
Bodong-Yang May 14, 2024
de75613
test_main: disable config_file_monitor
Bodong-Yang May 14, 2024
a1cb170
config_file_monitor: if config_file is being deleted, just skip it
Bodong-Yang May 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.devcontainer/
build/
dist/
2 changes: 1 addition & 1 deletion .flake8
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[flake8]
extend-ignore = E266, E501, E203
max-line-length = 88
max-complexity = 16
max-complexity = 16
40 changes: 40 additions & 0 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: build wheel for release

on:
release:
types: [published]

permissions:
contents: write # upload artifacts requires this permission

jobs:
build_wheel:
runs-on: ubuntu-22.04
steps:
- name: Checkout source code
uses: actions/checkout@v4

- name: Setup python environment
uses: actions/setup-python@v5
with:
# use the minimum py ver we support to
# generate the wheel.
python-version: "3.8"

- name: Build wheel and calculate checksum
run: |
python3 -m pip install -q -U pip
pip install -q -U hatch
hatch build -t wheel
for WHL in dist/*.whl; \
do \
sha256sum ${WHL} | sed -E "s@(\w+)\s+.*@sha256:\1@" > \
${WHL}.checksum; \
done

- name: Upload built wheel as release asset
uses: softprops/action-gh-release@v1
with:
files: |
dist/*.whl
dist/*.checksum
121 changes: 121 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
name: test CI

on:
pull_request:
branches:
- main
# only trigger unit test CI when src or tests changed
paths:
- "src/**"
- "tests/**"
- ".github/workflows/test.yaml"
push:
branches:
- main
# only trigger unit test CI when src or tests changed
paths:
- "src/**"
- "tests/**"
- ".github/workflows/test.yaml"
# allow the test CI to be manually triggerred
workflow_dispatch:

jobs:
pytest_with_coverage_on_supported_os:
strategy:
fail-fast: true
matrix:
# currently we only need to ensure it is running on the following OS
# with OS-shipped python interpreter.
os: ["ubuntu-20.04", "ubuntu-22.04"]
include:
- os: ubuntu-22.04
python_version: "3.10"
- os: ubuntu-20.04
python_version: "3.8"
runs-on: ${{ matrix.os }}
steps:
- name: Checkout commit
uses: actions/checkout@v4
with:
# sonarcloud needs main branch's ref
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python_version }}
cache: "pip"

- name: Install package
run: |
python -m pip install -q -U pip
pip install -q .[dev]

- name: Execute pytest with coverage
run: |
coverage run -m pytest --junit-xml=test_result/pytest.xml
coverage xml -o test_result/coverage.xml

# export the coverage report to the comment!
- name: Add coverage report to PR comment
continue-on-error: true
uses: MishaKav/pytest-coverage-comment@v1.1.51
with:
pytest-xml-coverage-path: test_result/coverage.xml
junitxml-path: test_result/pytest.xml

- name: SonarCloud Scan
uses: SonarSource/sonarcloud-github-action@master
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

pytest_on_supported_python_vers:
runs-on: ubuntu-22.04
strategy:
fail-fast: true
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11"]
steps:
- name: Checkout commit
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"

- name: Install package
run: |
python -m pip install -q -U pip
pip install -q .[dev]

- name: Execute pytest
run: pytest

python_lint_check:
runs-on: ubuntu-22.04
timeout-minutes: 3
strategy:
fail-fast: true
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11"]
steps:
- name: Checkout commit
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install check dependencies
run: |
python -m pip install -q -U pip
pip install -q .[dev]

- name: Check code linting
run: |
black --check src
flake8 src
169 changes: 162 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,164 @@
.devcontainer/
cache
coverage
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
mypy_cache/
otaclient_iot_logging_server/_version.py
__pycache__/
venv/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/

# local vscode settings
.devcontainer
.vscode
4 changes: 4 additions & 0 deletions .markdownlint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
"MD013": false
"MD041": false
"MD024":
"siblings_only": true
46 changes: 46 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
hooks:
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.10.0
hooks:
- id: python-check-mock-methods
- id: python-use-type-annotations
# Using this mirror lets us use mypyc-compiled black, which is about 2x faster
- repo: https://github.com/psf/black-pre-commit-mirror
rev: 24.4.2
hooks:
- id: black
# It is recommended to specify the latest version of Python
# supported by your project here, or alternatively use
# pre-commit's default_language_version, see
# https://pre-commit.com/#top_level-default_language_version
language_version: python3.11
- repo: https://github.com/pycqa/isort
rev: 5.13.2
hooks:
- id: isort
- repo: https://github.com/pycqa/flake8
rev: 7.0.0
hooks:
- id: flake8
additional_dependencies:
- flake8-bugbear==24.2.6
- flake8-comprehensions
- flake8-simplify
- repo: https://github.com/tox-dev/pyproject-fmt
rev: "1.8.0"
hooks:
- id: pyproject-fmt
# https://pyproject-fmt.readthedocs.io/en/latest/#calculating-max-supported-python-version
additional_dependencies: ["tox>=4.9"]
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.40.0
hooks:
- id: markdownlint
args: ["-c", ".markdownlint.yaml", "--fix"]
ci:
autoupdate_schedule: monthly
Loading