Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added scripts for build edge-proxy and iotedged images #1212

Merged
merged 34 commits into from
May 31, 2019
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
1246a7f
Added scripts to build edge-proxy images
dmolokanov May 13, 2019
b7282b1
Added missed files
dmolokanov May 13, 2019
c057e59
Added missed path to executable script
dmolokanov May 13, 2019
8401a9d
Commented for tests only
dmolokanov May 13, 2019
2d4ade0
Reverted changes
dmolokanov May 13, 2019
c69b2d0
Updated build script for edgelet projects
dmolokanov May 16, 2019
634c64b
Fixed strip tool
dmolokanov May 16, 2019
57a233f
Refactored pipeline to avoid redundant builds
dmolokanov May 16, 2019
5c8740e
Added tasks to build image for iotedged
dmolokanov May 21, 2019
55d2e18
Made lowercase comparison
dmolokanov May 21, 2019
987746d
Copy *.so files only in release configuration
dmolokanov May 21, 2019
6290011
Added manifest to build multi-arch images
dmolokanov May 21, 2019
6b51b06
Fixed edge-proxy build steps
dmolokanov May 22, 2019
cb56c5a
Updated path to source files for buildImage task
dmolokanov May 22, 2019
1d14742
Made bash linter happy
dmolokanov May 22, 2019
4885944
Forced to use base imaged for arm
dmolokanov May 22, 2019
3f07609
Update build images on docker hub
dmolokanov May 23, 2019
f72f4d3
Added arm64 check-in step
dmolokanov May 23, 2019
a590fb9
Merge branch 'master' into dmolokanov/edge-proxy
dmolokanov May 23, 2019
7faeb66
Apply suggestions from code review
dmolokanov May 24, 2019
3e5d7c2
Fixed shellcheck warnings
dmolokanov May 24, 2019
1d29381
Fixed error in build script
dmolokanov May 24, 2019
205028b
Fixed misstypes
dmolokanov May 24, 2019
3a2b868
Fixed misstypes
dmolokanov May 25, 2019
c742098
Reverted some checks
dmolokanov May 25, 2019
6b74e99
Use debian:9-slim as a base image for iotedged
dmolokanov May 28, 2019
f949597
Update builds/misc/templates/build-edgelet-linux.yaml
dmolokanov May 28, 2019
cf342b2
Merge branch 'master' into dmolokanov/edge-proxy
dmolokanov May 28, 2019
fe49171
Exit if edge-proxy runs in non-kubernetes environment
dmolokanov May 29, 2019
600369a
Merge branch 'master' into dmolokanov/edge-proxy
dmolokanov May 30, 2019
1685ad1
s/goes/go/g
dmolokanov May 30, 2019
5371c97
Fixed shellcheck warnings for run.sh
dmolokanov May 30, 2019
6dbcc55
Merge branch 'master' into dmolokanov/edge-proxy
avranju May 31, 2019
a3584ed
Merge branch 'master' into dmolokanov/edge-proxy
dmolokanov May 31, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 46 additions & 24 deletions builds/misc/images.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ jobs:
- script: scripts/linux/buildDiagnostics.sh $(Build.Configuration)
displayName: Build iotedge-diagnostics

- script: scripts/linux/buildEdgeProxy.sh
displayName: Build Edge Proxy

- task: PublishBuildArtifacts@1
displayName: 'Publish Artifacts'
inputs:
Expand All @@ -47,79 +50,86 @@ jobs:
# azureiotedge-diagnostics - Not Using Template for ARM64 because we have 2 different .NET Core.
- task: Bash@3
displayName: Build Image azureiotedge-diagnostics - aarch64
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-diagnostics -n microsoft -P azureiotedge-diagnostics --target-arch aarch64
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-diagnostics -n microsoft -P azureiotedge-diagnostics --target-arch aarch64

# Edge Proxy - Not Using Template because it doesn't have .NET code
- task: Bash@3
displayName: Build Image - Edge Proxy - aarch64
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-proxy -n microsoft -P azureiotedge-proxy --target-arch aarch64

# Edge Agent - Not Using Template for ARM64 because we have 2 different .NET Core.
- task: Bash@3
displayName: Build Edge Agent Image - aarch64
inputs:
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-agent -n microsoft -P Microsoft.Azure.Devices.Edge.Agent.Service --target-arch aarch64

# Edge Hub - Not Using Template for ARM64 because we have 2 different .NET Core.
- task: Bash@3
displayName: Build Edge Hub Image - aarch64
inputs:
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-hub -n microsoft -P Microsoft.Azure.Devices.Edge.Hub.Service --target-arch aarch64

# Simulated Temperature Sensor - Not Using Template for ARM64 because we have 2 different .NET Core.
- task: Bash@3
displayName: Build Image Temperature Sensor - aarch64
inputs:
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-simulated-temperature-sensor -n microsoft -P SimulatedTemperatureSensor --target-arch aarch64
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-simulated-temperature-sensor -n microsoft -P SimulatedTemperatureSensor --target-arch aarch64

# Temperature Filter - Not Using Template for ARM64 because we have 2 different .NET Core.
- task: Bash@3
displayName: Build Image Temperature Filter - aarch64
inputs:
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-temperature-filter -n microsoft -P TemperatureFilter --target-arch aarch64
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-temperature-filter -n microsoft -P TemperatureFilter --target-arch aarch64

# Load Gen - Not Using Template for ARM64 because we have 2 different .NET Core.
- task: Bash@3
displayName: Build Image Load Gen - aarch64
inputs:
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-load-gen -n microsoft -P load-gen --target-arch aarch64
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-load-gen -n microsoft -P load-gen --target-arch aarch64

# Messages Analyzer - Not Using Template for ARM64 because we have 2 different .NET Core.
- task: Bash@3
displayName: Build Image Messages Analyzer - aarch64
inputs:
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-analyzer -n microsoft -P MessagesAnalyzer --target-arch aarch64
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-analyzer -n microsoft -P MessagesAnalyzer --target-arch aarch64

# Functions Sample - Not Using Template for ARM64 because we have 2 different .NET Core.//TODO: Enable this once Functions supports arm64v8. Right now they are not ready.
# Functions Sample - Not Using Template for ARM64 because we have 2 different .NET Core.//TODO: Enable this once Functions supports arm64v8. Right now they are not ready.
#- task: Bash@3
# displayName: Build Image Functions Sample - aarch64
# inputs:
# inputs:
# filePath: scripts/linux/buildImage.sh
# arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-functions-filter -n azureiotedge -P EdgeHubTriggerCSharp --target-arch aarch64
# arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-functions-filter -n azureiotedge -P EdgeHubTriggerCSharp --target-arch aarch64

# Direct Method Sender - Not Using Template for ARM64 because we have 2 different .NET Core.
- task: Bash@3
displayName: Build Image Direct Method Sender - aarch64
inputs:
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-direct-method-sender -n microsoft -P DirectMethodSender --target-arch aarch64
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-direct-method-sender -n microsoft -P DirectMethodSender --target-arch aarch64

# Direct Method Receiver - Not Using Template for ARM64 because we have 2 different .NET Core.
- task: Bash@3
displayName: Build Image Direct Method Receiver - aarch64
inputs:
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-direct-method-receiver -n microsoft -P DirectMethodReceiver --target-arch aarch64
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-direct-method-receiver -n microsoft -P DirectMethodReceiver --target-arch aarch64

# Direct Method Cloud Sender - Not Using Template for ARM64 because we have 2 different .NET Core.
- task: Bash@3
displayName: Build Image Direct Method Cloud Sender - aarch64
inputs:
inputs:
filePath: scripts/linux/buildImage.sh
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-direct-method-cloud-sender -n microsoft -P DirectMethodCloudSender --target-arch aarch64
arguments: -r $(registry.address) -u $(registry.user) -p $(registry.password) -i azureiotedge-direct-method-cloud-sender -n microsoft -P DirectMethodCloudSender --target-arch aarch64


################################################################################
Expand Down Expand Up @@ -154,6 +164,9 @@ jobs:
- script: scripts/linux/buildDiagnostics.sh $(Build.Configuration)
displayName: Build iotedge-diagnostics

- script: scripts/linux/buildEdgeProxy.sh
displayName: Build Edge Proxy

- script: scripts/linux/createArtifactInfo.sh --output-folder '$(Build.BinariesDirectory)/publish' --build-number $(Build.BuildNumber)
displayName: 'Create Artifact Info File'

Expand All @@ -170,6 +183,13 @@ jobs:
imageName: azureiotedge-diagnostics
project: azureiotedge-diagnostics

# Edge Proxy
- template: templates/image-linux.yaml
parameters:
name: Edge Proxy
imageName: azureiotedge-proxy
project: azureiotedge-proxy

# Edge Agent
- template: templates/image-linux.yaml
parameters:
Expand Down Expand Up @@ -377,3 +397,5 @@ jobs:
displayName: 'Publish Temperature Sensor Manifest'
- script: scripts/linux/buildManifest.sh -r $(registry.address) -u $(registry.user) -p $(registry.password) -v $(Build.BuildNumber) -t $(System.DefaultWorkingDirectory)/edgelet/iotedge-diagnostics/docker/manifest.yaml.template -n microsoft --tags "$(tags)"
displayName: 'Publish azureiotedge-diagnostics Manifest'
- script: scripts/linux/buildManifest.sh -r $(registry.address) -u $(registry.user) -p $(registry.password) -v $(Build.BuildNumber) -t $(System.DefaultWorkingDirectory)/edge-proxy/docker/manifest.yaml.template -n microsoft --tags "$(tags)"
displayName: 'Publish azureiotedge-proxy Manifest'
3 changes: 3 additions & 0 deletions edge-proxy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Edge Proxy

This project contains a proxy server for edge modules that runs in kubernetes environment.
3 changes: 3 additions & 0 deletions edge-proxy/docker/linux/amd64/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM traefik:v1.7.11-alpine
COPY ./docker/linux/amd64/run.sh /
CMD ["/run.sh"]
3 changes: 3 additions & 0 deletions edge-proxy/docker/linux/arm32v7/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM arm32v6/traefik:v1.7.11-alpine
COPY ./docker/linux/arm32v7/run.sh /
CMD ["/run.sh"]
3 changes: 3 additions & 0 deletions edge-proxy/docker/linux/arm64v8/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM arm64v8/traefik:v1.7.11-alpine
COPY ./docker/linux/arm64v8/run.sh /
CMD ["/run.sh"]
18 changes: 18 additions & 0 deletions edge-proxy/docker/manifest.yaml.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
image: __REGISTRY__/__NAMESPACE__/azureiotedge-proxy:__VERSION__
tags: __TAGS__
manifests:
-
image: __REGISTRY__/__NAMESPACE__/azureiotedge-proxy:__VERSION__-linux-amd64
platform:
architecture: amd64
os: linux
-
image: __REGISTRY__/__NAMESPACE__/azureiotedge-proxy:__VERSION__-linux-arm32v7
platform:
architecture: arm
os: linux
-
image: __REGISTRY__/__NAMESPACE__/azureiotedge-proxy:__VERSION__-linux-arm64v8
platform:
architecture: arm64
os: linux
31 changes: 31 additions & 0 deletions edge-proxy/src/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/bin/sh
avranju marked this conversation as resolved.
Show resolved Hide resolved
set -e

# read service account token from filesystem
file="/var/run/secrets/kubernetes.io/serviceaccount/token"
if [[ -e ${file} ]]
then
token=`cat ${file}`
else
echo "Could not find file ${file}"
fi
dmolokanov marked this conversation as resolved.
Show resolved Hide resolved

config_src="/etc/traefik/traefik.toml"
if [[ ! -e ${config_src} ]]
then
echo "Could not find config ${config_src}"
exit 1
fi

# move traefik config to a new place to allow modification
config_dst="/traefik.toml"
cp ${config_src} ${config_dst}

# find TOKEN placeholders and replace them with a token value
if [[ ! -z ${token} ]]
then
sed -i -e 's/__TOKEN__/'${token}'/g' ${config_dst}
fi

# call traefik entry point script with a new config
/entrypoint.sh -c /traefik.toml
17 changes: 17 additions & 0 deletions scripts/linux/buildEdgeProxy.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/bin/bash

# This script copies the edge-proxy executable files that goes into the azureiotedge-proxy image,
avranju marked this conversation as resolved.
Show resolved Hide resolved
# for each supported arch (x86_64, arm32v7, arm64v8).
# It then publishes executable files along with their corresponding dockerfiles to the publish directory,
# so that buildImage.sh can build the container image.

VERSIONINFO_FILE_PATH=$BUILD_REPOSITORY_LOCALPATH/versionInfo.json
PUBLISH_FOLDER=$BUILD_BINARIESDIRECTORY/publish
export VERSION="$(cat "$VERSIONINFO_FILE_PATH" | jq '.version' -r)"

mkdir -p $PUBLISH_FOLDER/azureiotedge-proxy/
cp -R $BUILD_REPOSITORY_LOCALPATH/edge-proxy/docker $PUBLISH_FOLDER/azureiotedge-proxy/docker

cp $BUILD_REPOSITORY_LOCALPATH/edge-proxy/src/run.sh $PUBLISH_FOLDER/azureiotedge-proxy/docker/linux/amd64/
cp $BUILD_REPOSITORY_LOCALPATH/edge-proxy/src/run.sh $PUBLISH_FOLDER/azureiotedge-proxy/docker/linux/arm32v7/
cp $BUILD_REPOSITORY_LOCALPATH/edge-proxy/src/run.sh $PUBLISH_FOLDER/azureiotedge-proxy/docker/linux/arm64v8/