Skip to content

Commit

Permalink
Updates for v1.2 (#68)
Browse files Browse the repository at this point in the history
* Bug fixes in client

* CI fixes

* More bug fixes + package updates

* Fixed pylint

* Fixed 2.7 pylint

* Fix memory leak problem in sending BatchEventData (#70)

* Missing application_properties in BatchMessage fixes

* Missing data bytes in BatchMessage fixes

* Updated pylint version

This should resolve Travis error: `AttributeError: 'Import' object has no attribute 'infer_name_module'`

* improve type checking to enable message-like objects in BatchMessage

* pylint error fixes

* remove Python 3.4 from the travis config as Python 3.4 won't be supported from the next release

* Fix memory leak in sending BatchEventData

* improve solution by typecasting NULL to AMQP types

* Remove explicit test directory

* Separate test passes

* Some test updates

* Turned off test debug

* Fixed proxy test

* Adding livetest yaml (#71)

* adding evenhub secrets to livetest for UAMQP

* Splitting client.yml into a livetest build with environment variables set + maxparallel set to 1. This will ensure that any livetests won't accidentally stomp on each other.

* Trying latest cython

* Reverting Cython version

* Conditionally Disable Py38 (#72)

* adding evenhub secrets to livetest for UAMQP

* trying an adjustment of the build to see if continueonError will work

* removing extra variables in client.yml

* missed a single addition

* regular variable access

* trying without quotes

* disabling python 3.8

* removing shenanigans with continueonerror

* fixing the ordering

* Event Hub Track 2 Support (#74)

* Missing application_properties in BatchMessage fixes

* Missing data bytes in BatchMessage fixes

* Updated pylint version

This should resolve Travis error: `AttributeError: 'Import' object has no attribute 'infer_name_module'`

* improve type checking to enable message-like objects in BatchMessage

* pylint error fixes

* remove Python 3.4 from the travis config as Python 3.4 won't be supported from the next release

* Fix memory leak in sending BatchEventData

* improve solution by typecasting NULL to AMQP types

* init commit for websocket support

* Websocket functionality done, but further design is needed

* Reorganzie WebSocket related code

* Remove unnecessary tls http_proxy setting

* Module import and proxy username password update

* Update comment

* Update code

* Fix pylint error

* OAuth support

* Add Message app_prop checking

* Remove scope

* Updated proxy test

* Event Hub Track 2 Support (#75)

* Missing application_properties in BatchMessage fixes

* Missing data bytes in BatchMessage fixes

* Updated pylint version

This should resolve Travis error: `AttributeError: 'Import' object has no attribute 'infer_name_module'`

* improve type checking to enable message-like objects in BatchMessage

* pylint error fixes

* remove Python 3.4 from the travis config as Python 3.4 won't be supported from the next release

* Fix memory leak in sending BatchEventData

* improve solution by typecasting NULL to AMQP types

* init commit for websocket support

* Websocket functionality done, but further design is needed

* Reorganzie WebSocket related code

* Remove unnecessary tls http_proxy setting

* Module import and proxy username password update

* Update comment

* Update code

* Fix pylint error

* OAuth support

* Add Message app_prop checking

* Remove scope

* Add async jwt token support

* Update max message size

* Fix pylint error

* Code improvement

* Fix mistake

* Update code
1. Add MessageContentTooLarge error and will be raised when BatchMessage content is too large
2. Remove useless raise in cbs_auth_async

* Update history

* Add missing raise and correct misspelling (#77)

* Fix livetest error and compatibility with EventHub Track1 (#78)

* Update prefetch count in test

* Fix getting loop bug

* Lower logging level as too many logs will cause memory error in test pipeline

* Update prefetch count for the live test

* Update pretech count

* Update docstring of timeout type from int to float as part of the guideline
  • Loading branch information
annatisch authored Jun 13, 2019
1 parent b67e4fc commit 2f60b23
Show file tree
Hide file tree
Showing 42 changed files with 1,096 additions and 158 deletions.
315 changes: 315 additions & 0 deletions .azure-pipelines/client.test.live.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,315 @@
trigger:
- master

variables:
PythonVersion27: '2.7'
PythonVersion35: '3.5'
PythonVersion36: '3.6'
PythonVersion37: '3.7'
PythonVersion38: '3.8.0a2'

jobs:
- job: 'Windows'

pool:
vmImage: 'vs2017-win2016'

timeoutInMinutes: 120

strategy:
maxParallel: 1
matrix:
x64 Python 2.7:
PythonArchitecture: 'x64'
PythonVersion: '$(PythonVersion27)'
x64 Python 3.5:
PythonArchitecture: 'x64'
PythonVersion: '$(PythonVersion35)'
x64 Python 3.6:
PythonArchitecture: 'x64'
PythonVersion: '$(PythonVersion36)'
x64 Python 3.7:
PythonArchitecture: 'x64'
PythonVersion: '$(PythonVersion37)'
# x64 Python 3.8:
# PythonArchitecture: 'x64'
# PythonVersion: '$(PythonVersion38)'
x86 Python 2.7:
PythonArchitecture: 'x86'
PythonVersion: '$(PythonVersion27)'
x86 Python 3.5:
PythonArchitecture: 'x86'
PythonVersion: '$(PythonVersion35)'
x86 Python 3.6:
PythonArchitecture: 'x86'
PythonVersion: '$(PythonVersion36)'
x86 Python 3.7:
PythonArchitecture: 'x86'
PythonVersion: '$(PythonVersion37)'
# x86 Python 3.8:
# PythonArchitecture: 'x86'
# PythonVersion: '$(PythonVersion38)'

steps:
- task: UsePythonVersion@0
displayName: 'Use Python Version'
condition: ne(variables['PythonVersion'], variables['PythonVersion38'])
inputs:
architecture: '$(PythonArchitecture)'
versionSpec: '$(PythonVersion)'

- powershell: |
choco install python3 --pre --$(PythonArchitecture) --yes --no-progress --params "/InstallDir:C:\PythonPre"
Write-Host "##vso[task.prependpath]C:\PythonPre\Scripts"
Write-Host "##vso[task.prependpath]C:\PythonPre"
displayName: 'Install Python Version'
condition: eq(variables['PythonVersion'], variables['PythonVersion38'])
- powershell: |
Invoke-WebRequest -UseBasicParsing -Uri https://download.microsoft.com/download/7/9/6/796EF2E4-801B-4FC4-AB28-B59FBF6D907B/VCForPython27.msi -OutFile VCForPython27.msi
Start-Process -Wait -NoNewWindow msiexec.exe -ArgumentList "/i VCForPython27.msi /quiet /qn /norestart /log VCForPython27.log ALLUSERS=1"
Get-Content VCForPython27.log
displayName: 'Install Microsoft Visual C++ Compiler for Python 2.7'
condition: eq(variables['PythonVersion'], variables['PythonVersion27'])
- powershell: |
python --version
Invoke-WebRequest -UseBasicParsing -Uri https://bootstrap.pypa.io/get-pip.py | Select-Object -ExpandProperty Content | python
python -m pip install -r dev_requirements.txt
displayName: 'Install dependencies'
- script: python setup.py bdist_wheel
displayName: 'Build uAMQP Wheel'
env:
UAMQP_REBUILD_PYX: 'True'

- script: dir dist
displayName: 'Check output'

- powershell: |
$whlfile = Get-ChildItem -Filter *.whl dist | Select-Object -First 1 -ExpandProperty Name
python -m pip install --ignore-installed ./dist/$whlfile
pytest tests --doctest-modules --junitxml=junit/test-results-c.xml
pytest samples --doctest-modules --junitxml=junit/test-results-live.xml
displayName: 'Run tests'
env:
EVENT_HUB_HOSTNAME: $(python-eh-livetest-event-hub-hostname)
EVENT_HUB_NAME: $(python-eh-livetest-event-hub-name)
EVENT_HUB_SAS_POLICY: $(python-eh-livetest-event-hub-sas-policy)
EVENT_HUB_SAS_KEY: $(python-eh-livetest-event-hub-sas-key)
IOTHUB_HOSTNAME: $(python-iothub-livetest-host-name)
IOTHUB_HUB_NAME: $(python-iothub-livetest-hub-name)
IOTHUB_DEVICE: $(python-eh-livetest-event-hub-iothub-device)
IOTHUB_ENDPOINT: $(python-iothub-livetest-endpoint)
IOTHUB_SAS_POLICY: $(python-iothub-livetest-sas-policy)
IOTHUB_SAS_KEY: $(python-iothub-livetest-sas-key)
- task: PublishTestResults@2
displayName: 'Publish test results'
condition: succeededOrFailed()
inputs:
testResultsFiles: '**/test-results-*.xml'
testResultsFormat: 'JUnit'
testRunTitle: 'Windows $(PythonArchitecture) Python $(PythonVersion)'

- task: PublishBuildArtifacts@1
displayName: 'Publish wheel artifact'
inputs:
artifactName: uamqp-win$(PythonArchitecture)-$(PythonVersion)-whl
pathToPublish: 'dist'

- job: 'MacOS'

dependsOn: 'Windows'

timeoutInMinutes: 120

pool:
vmImage: 'macOS-10.13'

strategy:
maxParallel: 1
matrix:
Python 2.7:
MacOSXDeploymentTarget: '10.6'
PythonBin: 'python2'
PythonVersion: '$(PythonVersion27)'
Python 3.5:
MacOSXDeploymentTarget: '10.6'
PythonBin: 'python3'
PythonVersion: '$(PythonVersion35)'
Python 3.6:
MacOSXDeploymentTarget: '10.6'
PythonBin: 'python3'
PythonVersion: '$(PythonVersion36)'
Python 3.7:
MacOSXDeploymentTarget: '10.6'
PythonBin: 'python3'
PythonVersion: '$(PythonVersion37)'
# Python 3.8:
# MacOSXDeploymentTarget: '10.9'
# PythonBin: 'python3'
# PythonVersion: '$(PythonVersion38)'

variables:
OpenSSLDir: $(Agent.BuildDirectory)/openssl-macosx$(MacOSXDeploymentTarget)
PythonVersion27: '2.7.15'
PythonVersion35: '3.5.4'
PythonVersion36: '3.6.5'
PythonVersion37: '3.7.0'

steps:
- task: DownloadPipelineArtifact@1
displayName: 'Download OpenSSL artifact'
inputs:
artifactName: openssl-macosx$(MacOSXDeploymentTarget)
buildType: specific
buildVersionToDownload: latest
downloadPath: $(Agent.BuildDirectory)
pipeline: 119 # azure-uamqp-python - openssl
project: '29ec6040-b234-4e31-b139-33dc4287b756' # public

- script: sudo xcode-select --switch /Applications/Xcode_9.4.1.app
displayName: 'Select Xcode 9.4.1'

- script: source ./install_python_osx.sh
displayName: 'Install Official Python'

- script: |
$(PythonBin) --version
curl -sS https://bootstrap.pypa.io/get-pip.py | $(PythonBin) - --user
$(PythonBin) -m pip install --user -r dev_requirements.txt
echo "##vso[task.prependpath]/Users/vsts/Library/Python/`ls /Users/vsts/Library/Python | head -n1`/bin"
displayName: 'Install dependencies'
- script: $(PythonBin) setup.py bdist_wheel
displayName: 'Build uAMQP Wheel'
env:
CFLAGS: -mmacosx-version-min=$(MacOSXDeploymentTarget) -I$(OpenSSLDir)/include
CMAKE_OSX_ARCHITECTURES: 'i386;x86_64'
CMAKE_OSX_DEPLOYMENT_TARGET: $(MacOSXDeploymentTarget)
LDFLAGS: -mmacosx-version-min=$(MacOSXDeploymentTarget) -L$(OpenSSLDir)/lib
MACOSX_DEPLOYMENT_TARGET: $(MacOSXDeploymentTarget)
OPENSSL_INCLUDE_DIR: $(OpenSSLDir)/include
OPENSSL_ROOT_DIR: $(OpenSSLDir)
UAMQP_REBUILD_PYX: 'True'
UAMQP_USE_OPENSSL: 'True'

- script: ls ./dist
displayName: 'Check output'

- script: |
$(PythonBin) -m pip install --user --ignore-installed ./dist/*.whl
pytest tests --doctest-modules --junitxml=junit/test-results-c.xml
pytest samples --doctest-modules --junitxml=junit/test-results-live.xml
displayName: 'Run tests'
env:
EVENT_HUB_HOSTNAME: $(python-eh-livetest-event-hub-hostname)
EVENT_HUB_NAME: $(python-eh-livetest-event-hub-name)
EVENT_HUB_SAS_POLICY: $(python-eh-livetest-event-hub-sas-policy)
EVENT_HUB_SAS_KEY: $(python-eh-livetest-event-hub-sas-key)
IOTHUB_HOSTNAME: $(python-iothub-livetest-host-name)
IOTHUB_HUB_NAME: $(python-iothub-livetest-hub-name)
IOTHUB_DEVICE: $(python-eh-livetest-event-hub-iothub-device)
IOTHUB_ENDPOINT: $(python-iothub-livetest-endpoint)
IOTHUB_SAS_POLICY: $(python-iothub-livetest-sas-policy)
IOTHUB_SAS_KEY: $(python-iothub-livetest-sas-key)
- task: PublishTestResults@2
displayName: 'Publish test results'
condition: succeededOrFailed()
inputs:
testResultsFiles: '**/test-results-*.xml'
testResultsFormat: 'JUnit'
testRunTitle: 'MacOS Python $(PythonVersion)'

- task: PublishBuildArtifacts@1
displayName: 'Publish wheel artifact'
inputs:
artifactName: uamqp-macosx$(MacOSXDeploymentTarget)-$(PythonVersion)-whl
pathToPublish: 'dist'


- job: 'Linux'

timeoutInMinutes: 120

dependsOn: 'MacOS'

pool:
vmImage: 'ubuntu-16.04'

strategy:
maxParallel: 1
matrix:
Python 2.7:
PythonVersion: '$(PythonVersion27)'
Python 3.5:
PythonVersion: '$(PythonVersion35)'
Python 3.6:
PythonVersion: '$(PythonVersion36)'
Python 3.7:
PythonVersion: '$(PythonVersion37)'
# Python 3.8:
# PythonVersion: '$(PythonVersion38)'

steps:
- task: UsePythonVersion@0
displayName: 'Use Python Version'
condition: ne(variables['PythonVersion'], variables['PythonVersion38'])
inputs:
versionSpec: '$(PythonVersion)'

- script: |
export DEBIAN_FRONTEND=noninteractive
sudo add-apt-repository -y ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get -q -y install python3.8 python3.8-distutils python3.8-venv python3.8-dev
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.8 1
displayName: 'Install Python Version'
condition: eq(variables['PythonVersion'], variables['PythonVersion38'])
- script: |
python --version
curl -sS https://bootstrap.pypa.io/get-pip.py | python - --user
python -m pip install --user -r dev_requirements.txt
echo "##vso[task.prependpath]/home/vsts/.local/bin"
displayName: 'Install dependencies'
- script: python setup.py bdist_wheel
displayName: 'Build uAMQP Wheel'
env:
UAMQP_REBUILD_PYX: 'True'

- script: ls ./dist
displayName: 'Check output'

- script: |
python -m pip install --user --ignore-installed ./dist/*.whl
pytest tests --doctest-modules --junitxml=junit/test-results-c.xml
pytest samples --doctest-modules --junitxml=junit/test-results-live.xml
displayName: 'Run tests'
env:
EVENT_HUB_HOSTNAME: $(python-eh-livetest-event-hub-hostname)
EVENT_HUB_NAME: $(python-eh-livetest-event-hub-name)
EVENT_HUB_SAS_POLICY: $(python-eh-livetest-event-hub-sas-policy)
EVENT_HUB_SAS_KEY: $(python-eh-livetest-event-hub-sas-key)
IOTHUB_HOSTNAME: $(python-iothub-livetest-host-name)
IOTHUB_HUB_NAME: $(python-iothub-livetest-hub-name)
IOTHUB_DEVICE: $(python-eh-livetest-event-hub-iothub-device)
IOTHUB_ENDPOINT: $(python-iothub-livetest-endpoint)
IOTHUB_SAS_POLICY: $(python-iothub-livetest-sas-policy)
IOTHUB_SAS_KEY: $(python-iothub-livetest-sas-key)
- task: PublishTestResults@2
displayName: 'Publish test results'
condition: succeededOrFailed()
inputs:
testResultsFiles: '**/test-results-*.xml'
testResultsFormat: 'JUnit'
testRunTitle: 'Linux Python $(PythonVersion)'

- task: PublishBuildArtifacts@1
displayName: 'Publish wheel artifact'
inputs:
artifactName: uamqp-linux-$(PythonVersion)-whl
pathToPublish: 'dist'
Loading

0 comments on commit 2f60b23

Please sign in to comment.