-
-
Notifications
You must be signed in to change notification settings - Fork 4.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CI] Enhance the pipeline with git diff method
#4737
Open
ueqri
wants to merge
8
commits into
PointCloudLibrary:master
Choose a base branch
from
ueqri:CI-git-diff-based
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+420
−154
Open
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
a6be357
added git diff together with main pipeline
ueqri e509526
Added build reason check in diff stage
ueqri 6ad3a8f
Added git-diff in tutorials stage
ueqri f6326c4
Enhanced the script of tutorial stage trigger
ueqri d322356
Merge branch 'master' into CI-git-diff-based
ueqri 0f51143
Apply master changes after resolving the conflicts
ueqri d36562b
Add description about the CI structure
ueqri 9fd3f0b
Add more comments for CI
ueqri File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,168 +1,103 @@ | ||
| # Exclude Rules: | ||
| # 1. path-filter in trigger and pr, | ||
| # this is NOT exclude rules for documentation, | ||
| # set this rule only to avoid any CI run. | ||
| # 2. DOCS_LISTS in diff script, | ||
| # the lists contain all documentation-related files, | ||
| # fill the list to help git-diff recognize docs changes. | ||
|
|
||
| trigger: | ||
| paths: | ||
| exclude: | ||
| - doc | ||
| - README.md | ||
| - CHANGES.md | ||
| - CONTRIBUTING.md | ||
| - .ci | ||
| - .dev | ||
| - .github | ||
| # add more exclude rules, e.g. fileName, directoryName | ||
| # which won't cause any CI to run. | ||
|
|
||
| pr: | ||
| paths: | ||
| exclude: | ||
| - doc | ||
| - README.md | ||
| - CHANGES.md | ||
| - CONTRIBUTING.md | ||
| - .ci | ||
| - .dev | ||
| - .github | ||
| # add more exclude rules, e.g. fileName, directoryName | ||
| # which won't cause any CI to run. | ||
|
|
||
| resources: | ||
| containers: | ||
| - container: winx86 | ||
| image: pointcloudlibrary/env:winx86 | ||
| - container: winx64 | ||
| image: pointcloudlibrary/env:winx64 | ||
| - container: fmt | ||
| - container: fmt # for formatting.yaml | ||
| image: pointcloudlibrary/fmt | ||
| - container: env1804 | ||
| image: pointcloudlibrary/env:18.04 | ||
| - container: env2004 | ||
| image: pointcloudlibrary/env:20.04 | ||
| - container: env2010 | ||
| image: pointcloudlibrary/env:20.10 | ||
|
|
||
| stages: | ||
| - stage: formatting | ||
| displayName: Formatting | ||
| jobs: | ||
| - template: formatting.yaml | ||
|
|
||
| - stage: build_gcc | ||
| displayName: Build GCC | ||
| dependsOn: formatting | ||
| - stage: diff | ||
| displayName: Code Changes Diff | ||
| jobs: | ||
| - job: ubuntu | ||
| displayName: Ubuntu | ||
| - job: CodeDiff | ||
| displayName: Code Diff | ||
| pool: | ||
| vmImage: 'Ubuntu 20.04' | ||
| strategy: | ||
| matrix: | ||
| 18.04 GCC: # oldest LTS | ||
| CONTAINER: env1804 | ||
| CC: gcc | ||
| CXX: g++ | ||
| BUILD_GPU: ON | ||
| CMAKE_ARGS: '-DPCL_WARNINGS_ARE_ERRORS=ON' | ||
| 20.10 GCC: # latest Ubuntu | ||
| CONTAINER: env2010 | ||
| CC: gcc | ||
| CXX: g++ | ||
| BUILD_GPU: OFF | ||
| container: $[ variables['CONTAINER'] ] | ||
| timeoutInMinutes: 0 | ||
| variables: | ||
| BUILD_DIR: '$(Agent.BuildDirectory)/build' | ||
| CMAKE_CXX_FLAGS: '-Wall -Wextra -Wnoexcept-type' | ||
| DISPLAY: :99.0 # Checked for in CMake | ||
| vmImage: 'ubuntu-latest' | ||
| steps: | ||
| - template: build/ubuntu.yaml | ||
| - checkout: self | ||
| - script: | | ||
| # Add all docs path to this array | ||
| DOCS_LISTS=("doc" "*.md") | ||
| for ENTRY in "${DOCS_LISTS[@]}"; do | ||
| DIFF_EXCLUDE="':(exclude)${ENTRY}' ${DIFF_EXCLUDE}" | ||
| done | ||
|
|
||
| # Check the CI build reason: Pull Request or Git Push | ||
| BUILD_REASON=$(Build.Reason) | ||
| if [ "${BUILD_REASON}" -eq "PullRequest" ]; then | ||
| MERGE_BASE_PAIR="HEAD origin/master" | ||
| else | ||
| MERGE_BASE_PAIR="HEAD HEAD~1" | ||
| fi | ||
|
|
||
| # Initiate HasCodeChanges variable | ||
| echo "##vso[task.setvariable variable=HasCodeChanges;isOutput=true]false" | ||
|
|
||
| # Show full git diff results | ||
| cd $(Build.SourcesDirectory) | ||
| git diff $(git merge-base ${MERGE_BASE_PAIR}) | ||
|
|
||
| # Set HasCodeChanges to true, if there's still file changed when excluding those docs | ||
| git diff --quiet $(git merge-base ${MERGE_BASE_PAIR}) -- ${DIFF_EXCLUDE} || \ | ||
| echo "##vso[task.setvariable variable=HasCodeChanges;isOutput=true]true" | ||
| name: GitResult | ||
| displayName: Show Git Diff Result | ||
|
|
||
| - stage: build_clang | ||
| displayName: Build Clang | ||
| dependsOn: formatting | ||
| - stage: documentation | ||
| displayName: Goto Documentation | ||
| dependsOn: diff | ||
| condition: and(succeeded(), eq(dependencies.diff.outputs['CodeDiff.GitResult.HasCodeChanges'], 'false')) | ||
kunaltyagi marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| jobs: | ||
| - job: osx | ||
| displayName: macOS | ||
| pool: | ||
| vmImage: '$(VMIMAGE)' | ||
| strategy: | ||
| matrix: | ||
| Catalina 10.15: | ||
| VMIMAGE: 'macOS-10.15' | ||
| OSX_VERSION: '10.15' | ||
| Mojave 10.14: | ||
| VMIMAGE: 'macOS-10.14' | ||
| OSX_VERSION: '10.14' | ||
| timeoutInMinutes: 0 | ||
| variables: | ||
| BUILD_DIR: '$(Agent.WorkFolder)/build' | ||
| GOOGLE_TEST_DIR: '$(Agent.WorkFolder)/googletest' | ||
| CMAKE_CXX_FLAGS: '-Wall -Wextra -Wabi -Werror -Wno-error=deprecated-declarations' | ||
| steps: | ||
| - template: build/macos.yaml | ||
| - job: ubuntu | ||
| displayName: Ubuntu | ||
| # Placement of Ubuntu Clang job after macOS ensures an extra parallel job doesn't need to be created. | ||
| # Total time per run remains same since macOS is quicker so it finishes earlier, and remaining time is used by this job | ||
| # Therefore, number of parallel jobs and total run time of entire pipeline remains unchanged even after addition of this job | ||
| # The version of Ubuntu is chosen to cover more versions than covered by GCC based CI | ||
| dependsOn: osx | ||
| condition: succeededOrFailed() | ||
| pool: | ||
| vmImage: 'Ubuntu 20.04' | ||
| strategy: | ||
| matrix: | ||
| 20.04 Clang: | ||
| CONTAINER: env2004 | ||
| CC: clang | ||
| CXX: clang++ | ||
| BUILD_GPU: ON | ||
| CMAKE_ARGS: '' | ||
| container: $[ variables['CONTAINER'] ] | ||
| timeoutInMinutes: 0 | ||
| variables: | ||
| BUILD_DIR: '$(Agent.BuildDirectory)/build' | ||
| CMAKE_CXX_FLAGS: '-Wall -Wextra' | ||
| DISPLAY: :99.0 # Checked for in CMake | ||
| steps: | ||
| - template: build/ubuntu.yaml | ||
| - job: ubuntu_indices | ||
| displayName: Ubuntu Indices | ||
| # Test 64 bit & unsigned indices | ||
| dependsOn: osx | ||
| condition: succeededOrFailed() | ||
| - job: prompt | ||
| displayName: Trigger Prompt | ||
| pool: | ||
| vmImage: 'Ubuntu 20.04' | ||
| strategy: | ||
| matrix: | ||
| 20.04 Clang: | ||
| CONTAINER: env2004 | ||
| CC: clang | ||
| CXX: clang++ | ||
| INDEX_SIGNED: OFF | ||
| INDEX_SIZE: 64 | ||
| CMAKE_ARGS: '' | ||
| container: $[ variables['CONTAINER'] ] | ||
| timeoutInMinutes: 0 | ||
| variables: | ||
| BUILD_DIR: '$(Agent.BuildDirectory)/build' | ||
| CMAKE_CXX_FLAGS: '-Wall -Wextra' | ||
| vmImage: 'ubuntu-latest' | ||
| steps: | ||
| - template: build/ubuntu_indices.yaml | ||
| - script: echo Trigger docs-pipeline.yaml... | ||
|
|
||
| - stage: build_msvc | ||
| displayName: Build MSVC | ||
| dependsOn: formatting | ||
| - stage: build | ||
| displayName: Goto Build | ||
| dependsOn: diff | ||
| condition: and(succeeded(), eq(dependencies.diff.outputs['CodeDiff.GitResult.HasCodeChanges'], 'true')) | ||
| jobs: | ||
| - job: Windows | ||
| displayName: Windows Build | ||
| - job: prompt | ||
| displayName: Trigger Prompt | ||
| pool: | ||
| vmImage: 'windows-2019' | ||
| strategy: | ||
| matrix: | ||
| x86: | ||
| CONTAINER: winx86 | ||
| PLATFORM: 'x86' | ||
| ARCHITECTURE: 'x86' | ||
| GENERATOR: '"Visual Studio 16 2019" -A Win32' | ||
| x64: | ||
| CONTAINER: winx64 | ||
| PLATFORM: 'x64' | ||
| ARCHITECTURE: 'x86_amd64' | ||
| GENERATOR: '"Visual Studio 16 2019" -A x64' | ||
| container: $[ variables['CONTAINER'] ] | ||
| timeoutInMinutes: 0 | ||
| variables: | ||
| BUILD_DIR: 'c:\build' | ||
| CONFIGURATION: 'Release' | ||
| VCPKG_ROOT: 'c:\vcpkg' | ||
| vmImage: 'ubuntu-latest' | ||
| steps: | ||
| - template: build/windows.yaml | ||
| - script: echo Trigger build-pipeline.yaml... | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| # Use pipeline trigger only, so set `trigger` and `pr` fields to none to avoid unexamined run. | ||
| # See here, https://docs.microsoft.com/en-us/azure/devops/pipelines/process/pipeline-triggers?tabs=yaml&view=azure-devops#stage-filters | ||
|
|
||
| trigger: none | ||
|
|
||
| pr: none | ||
|
|
||
| resources: | ||
| pipelines: | ||
| - pipeline: Main-CI | ||
| source: Main | ||
| trigger: | ||
| stages: | ||
| - build | ||
| containers: | ||
| - container: env1804 | ||
| image: pointcloudlibrary/env:18.04 | ||
| - container: env2004 | ||
| image: pointcloudlibrary/env:20.04 | ||
| - container: env2010 | ||
| image: pointcloudlibrary/env:20.10 | ||
| - container: winx86 | ||
| image: pointcloudlibrary/env:winx86 | ||
| - container: winx64 | ||
| image: pointcloudlibrary/env:winx64 | ||
|
|
||
| stages: | ||
| - stage: build_gcc | ||
| displayName: Build GCC | ||
| dependsOn: [] | ||
| jobs: | ||
| - job: ubuntu | ||
| displayName: Ubuntu | ||
| pool: | ||
| vmImage: 'Ubuntu 20.04' | ||
| strategy: | ||
| matrix: | ||
| 18.04 GCC: # oldest LTS | ||
| CONTAINER: env1804 | ||
| CC: gcc | ||
| CXX: g++ | ||
| BUILD_GPU: ON | ||
| CMAKE_ARGS: '-DPCL_WARNINGS_ARE_ERRORS=ON' | ||
| 20.10 GCC: # latest Ubuntu | ||
| CONTAINER: env2010 | ||
| CC: gcc | ||
| CXX: g++ | ||
| BUILD_GPU: OFF | ||
| container: $[ variables['CONTAINER'] ] | ||
| timeoutInMinutes: 0 | ||
| variables: | ||
| BUILD_DIR: '$(Agent.BuildDirectory)/build' | ||
| CMAKE_CXX_FLAGS: '-Wall -Wextra -Wnoexcept-type' | ||
| DISPLAY: :99.0 # Checked for in CMake | ||
| steps: | ||
| - template: build/ubuntu.yaml | ||
|
|
||
| - stage: build_clang | ||
| displayName: Build Clang | ||
| dependsOn: [] | ||
| jobs: | ||
| - job: osx | ||
| displayName: macOS | ||
| pool: | ||
| vmImage: '$(VMIMAGE)' | ||
| strategy: | ||
| matrix: | ||
| Catalina 10.15: | ||
| VMIMAGE: 'macOS-10.15' | ||
| OSX_VERSION: '10.15' | ||
| Mojave 10.14: | ||
| VMIMAGE: 'macOS-10.14' | ||
| OSX_VERSION: '10.14' | ||
| timeoutInMinutes: 0 | ||
| variables: | ||
| BUILD_DIR: '$(Agent.WorkFolder)/build' | ||
| GOOGLE_TEST_DIR: '$(Agent.WorkFolder)/googletest' | ||
| CMAKE_CXX_FLAGS: '-Wall -Wextra -Wabi -Werror -Wno-error=deprecated-declarations' | ||
| steps: | ||
| - template: build/macos.yaml | ||
| - job: ubuntu | ||
| displayName: Ubuntu | ||
| # Placement of Ubuntu Clang job after macOS ensures an extra parallel job doesn't need to be created. | ||
| # Total time per run remains same since macOS is quicker so it finishes earlier, and remaining time is used by this job | ||
| # Therefore, number of parallel jobs and total run time of entire pipeline remains unchanged even after addition of this job | ||
| # The version of Ubuntu is chosen to cover more versions than covered by GCC based CI | ||
| dependsOn: osx | ||
| condition: succeededOrFailed() | ||
| pool: | ||
| vmImage: 'Ubuntu 20.04' | ||
| strategy: | ||
| matrix: | ||
| 20.04 Clang: | ||
| CONTAINER: env2004 | ||
| CC: clang | ||
| CXX: clang++ | ||
| BUILD_GPU: ON | ||
| CMAKE_ARGS: '' | ||
| container: $[ variables['CONTAINER'] ] | ||
| timeoutInMinutes: 0 | ||
| variables: | ||
| BUILD_DIR: '$(Agent.BuildDirectory)/build' | ||
| CMAKE_CXX_FLAGS: '-Wall -Wextra' | ||
| DISPLAY: :99.0 # Checked for in CMake | ||
| steps: | ||
| - template: build/ubuntu.yaml | ||
| - job: ubuntu_indices | ||
| displayName: Ubuntu Indices | ||
| # Test 64 bit & unsigned indices | ||
| dependsOn: osx | ||
| condition: succeededOrFailed() | ||
| pool: | ||
| vmImage: 'Ubuntu 20.04' | ||
| strategy: | ||
| matrix: | ||
| 20.04 Clang: | ||
| CONTAINER: env2004 | ||
| CC: clang | ||
| CXX: clang++ | ||
| INDEX_SIGNED: OFF | ||
| INDEX_SIZE: 64 | ||
| CMAKE_ARGS: '' | ||
| container: $[ variables['CONTAINER'] ] | ||
| timeoutInMinutes: 0 | ||
| variables: | ||
| BUILD_DIR: '$(Agent.BuildDirectory)/build' | ||
| CMAKE_CXX_FLAGS: '-Wall -Wextra' | ||
| steps: | ||
| - template: build/ubuntu_indices.yaml | ||
|
|
||
| - stage: build_msvc | ||
| displayName: Build MSVC | ||
| dependsOn: [] | ||
| jobs: | ||
| - job: Windows | ||
| displayName: Windows Build | ||
| pool: | ||
| vmImage: 'windows-2019' | ||
| strategy: | ||
| matrix: | ||
| x86: | ||
| CONTAINER: winx86 | ||
| PLATFORM: 'x86' | ||
| ARCHITECTURE: 'x86' | ||
| GENERATOR: '"Visual Studio 16 2019" -A Win32' | ||
| x64: | ||
| CONTAINER: winx64 | ||
| PLATFORM: 'x64' | ||
| ARCHITECTURE: 'x86_amd64' | ||
| GENERATOR: '"Visual Studio 16 2019" -A x64' | ||
| container: $[ variables['CONTAINER'] ] | ||
| timeoutInMinutes: 0 | ||
| variables: | ||
| BUILD_DIR: 'c:\build' | ||
| CONFIGURATION: 'Release' | ||
| VCPKG_ROOT: 'c:\vcpkg' | ||
| steps: | ||
| - template: build/windows.yaml |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.ci and .github can mean change in the pipeline. We want to run pipelines in those cases
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the mistake, I would fix it.