Tanzu Tile implementation and documentation (#2726)

Tanzu Tile implementation and docs

This change implements the required code for creating the Tanzu Tile.
The Tanzu Tile is a way to deploy the collector to get metrics from
the Tanzu environment. In the tile directory, run the command
./make-latest-tile to generate the Tanzu tile. Refer to DEVELOPMENT.md
to know how to properly use the created tile.

.github/workflows/bosh-release.yml

@@ -25,7 +25,8 @@ jobs:
       - name: Check out the codebase.
         uses: actions/checkout@v3
       - name: Set up Homebrew
-        uses: Homebrew/actions/setup-homebrew@0c611abec1fc679030907286a2c084cc92fb8f2e
+        id: set-up-homebrew
+        uses: Homebrew/actions/setup-homebrew@251cc053a43110ce4dea3cdb6f038147c82d0257
       - name: Install BOSH CLI
         shell: bash
         run: |

.github/workflows/tanzu-tile.yml

@@ -0,0 +1,58 @@
+name: Tanzu Tile
+
+# The workflow triggered by any change in deployments/cloudfoundry/bosh/
+# or /deployments/cloudfoundry/tile
+
+on:
+  pull_request:
+    paths:
+      - 'deployments/cloudfoundry/bosh/**'
+      - 'deployments/cloudfoundry/tile/**'
+
+permissions:
+  contents: write
+
+defaults:
+  run:
+    working-directory: 'deployments/cloudfoundry/tile'
+
+jobs:
+
+  test:
+    name: Test Tanzu Tile creation
+    # Use 20.04.5 until https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/16450 is resolved
+    runs-on: ubuntu-20.04
+    steps:
+      - name: Check out the codebase.
+        uses: actions/checkout@v3
+      - name: Set up Homebrew
+        id: set-up-homebrew
+        uses: Homebrew/actions/setup-homebrew@master
+      - name: Install BOSH CLI
+        shell: bash
+        run: |
+          brew install cloudfoundry/tap/bosh-cli
+          bosh -v
+      - name: Install PCF CLI
+        shell: bash
+        run: |
+          pip install tile-generator
+      - name: Run release script
+        shell: bash
+        run: |
+          ./make-latest-tile
+          tanzu_tile_regex="product/splunk-otel-collector-*.pivotal"
+          size="$(stat -c '%s' $tanzu_tile_regex)"
+          if [[ $size -eq 0 ]]; then
+            echo "File is empty!" >&2
+            exit 1
+          fi
+
+          tanzu_tile_path=$(pwd)/$tanzu_tile_regex
+          echo "tanzu_tile_path=$tanzu_tile_path" >> $GITHUB_ENV
+
+      - name: Uploading artifacts
+        uses: actions/upload-artifact@v3
+        with:
+          name: tanzu-tile-latest.pivotal
+          path: ${{ env.tanzu_tile_path }}

deployments/cloudfoundry/README.md

@@ -1,11 +1,25 @@
 # Splunk OpenTelemetry Collector Pivotal Cloud Foundry (PCF) Integrations
 
+Supported Tanzu Application Service (TAS) versions: v2
+
+Unsupported TAS versions: v3
+
 ### Cloud Foundry Buildpack
 
 This integration can be used to install and run the Collector as a sidecar to your app.
 In this configuration, the Collector will run in the same container as the app.
 
 ### Bosh Release
 
-This is a Bosh release of the Splunk Collector. This deploys the Collector to the PCF
-environment as a standalone deployment.
+This is a Bosh release of the Collector. This deploys the Collector to the PCF
+environment as a standalone deployment.
+
+### Tanzu Tile
+
+This is a Tanzu Tile of the Collector, which is a packaged release of the collector
+that can be integrated into the Ops Manager. The Tanzu Tile enables users to download, install,
+run, configure, and update the collector all from the Ops Manager.
+
+[Tanzu Tile UI](./tile/resources/tanzu_tile_in_ops_mgr.png)
+
+[Tanzu Tile Configuration UI](./tile/resources/tanzu_tile_config_options.png)

deployments/cloudfoundry/bosh/.gitignore

@@ -1,6 +1,6 @@
 latest-release.tgz
-signalfx-agent.tar.gz
 splunk_otel_collector
 .dev_builds
 blobs
-dev_releases
+dev_releases
+bosh-env

deployments/cloudfoundry/bosh/DEVELOPMENT.md

@@ -2,35 +2,65 @@
 
 ## Getting Started
 
-[Bosh Release Documentation (Overview)](https://bosh.io/docs/create-release/)
+### Start the BOSH Director
 
-[Quick Start Guide](https://bosh.io/docs/bosh-lite/)
+[Local machine prerequisites](https://bosh.io/docs/quick-start/#prerequisites)
+- The latest VirtualBox environment v7 is incompatible with this
+  functionality. Downgrade VirtualBox to [v6](https://www.virtualbox.org/wiki/Download_Old_Builds_6_1) to
+  ensure it can work. Tested successfully on `v6.1.42`
+
+
+Automated start process:
+```shell
+# Sets up BOSH Director locally
+make
+# Makes sure local shell gets proper credentials to access director
+source bosh-env/virtualbox/.envrc
+```
 
-### Supplementary Getting Started Checklist
+Manual start process:
+[Follow the quick start guide to run a BOSH Director.](https://bosh.io/docs/quick-start/)
 
-- Make sure director is running
-    - Quick start guide to run a BOSH Director can be found [here.](https://bosh.io/docs/quick-start/)
-- Need to upload right Ubuntu/OS blob
-    - Guide can be found [here.](https://bosh.io/docs/uploading-stemcells/)
-    - The following is an example command to upload the Warden (BOSH Lite) Ubuntu Bionic (18.04.6 LTS) stemcell:
+Common Errors:
+- "Waiting for the agent on VM" timeouts
+
+  - Run ```make reinstall-director```
+
+Delete director:
+```shell
+make delete-director
+```
+### Upload Ubuntu/OS blob
 
-    ```shell
-    $ bosh upload-stemcell --sha1 d44dc2d1b3f8415b41160ad4f82bc9d30b8dfdce \
-      https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-bionic-go_agent?v=1.71
-    ```
+Note: If you ran ```make``` successfully, you can skip this step.
+
+- [Uploading stemcells guide](https://bosh.io/docs/uploading-stemcells/)
+- [Official BOSH stemcells references (including SHAs)](https://bosh.io/stemcells).
+- The following is an example command to upload the Warden (BOSH Lite) Ubuntu Bionic (18.04.6 LTS) stemcell:
+
+```shell
+bosh upload-stemcell --sha1 d44dc2d1b3f8415b41160ad4f82bc9d30b8dfdce \
+https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-bionic-go_agent?v=1.71
+````
+
+### Supplemental Documentation Links
+
+[BOSH Release Documentation (Overview)](https://bosh.io/docs/create-release/)
+
+[Quick Start Guide](https://bosh.io/docs/bosh-lite/)
 
 ## Create Local BOSH Release
 
 ```shell
-$ # Check release script for more environment variables that can be set.
-$ export IS_DEV_RELEASE=1
-$ ./release
+# Check release script for more environment variables that can be set.
+export IS_DEV_RELEASE=1
+./release
 ```
 
 ## BOSH Release Usage
 
 ```shell
-$ bosh -d splunk-otel-collector deploy deployment.yaml
+bosh -d splunk-otel-collector deploy deployment.yaml
 ```
 Further explanation of the `deployment.yaml` file is found [here.](#deployment-config)
 
@@ -106,6 +136,9 @@ $ bosh deployments
 # View all bosh VMs. This will show if a deployment's VM is running or failed.
 $ bosh vms
 
+# View debug logs for a task (e.g. upload or deploy) that failed
+$ bosh task task_number --debug
+
 # View logs for a given deployment. Downloads a TAR file from the deployment.
 $ bosh logs -d <deployment-name>
 

deployments/cloudfoundry/bosh/Makefile

@@ -0,0 +1,29 @@
+# Copyright Splunk Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Explicitly define the shell we will use for commands.
+SHELL?=/bin/bash
+
+.DEFAULT_GOAL=install-director
+
+.PHONY: delete-director
+delete-director:
+	./delete-director.sh
+
+.PHONY: install-director
+install-director:
+	./install-director.sh
+
+.PHONY: reinstall-director
+reinstall-director: delete-director install-director

deployments/cloudfoundry/bosh/delete-director.sh

@@ -0,0 +1,28 @@
+# Copyright Splunk Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#!/bin/bash
+
+set -euo pipefail
+
+BOSH_DIRECTOR_DIR="./bosh-env/virtualbox"
+BOSH_DIRECTOR_DEPLOYMENT_DIR="bosh-deployment"
+SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
+if [ -d $BOSH_DIRECTOR_DIR ] && [ -d $BOSH_DIRECTOR_DIR/$BOSH_DIRECTOR_DEPLOYMENT_DIR ]; then
+  cd $BOSH_DIRECTOR_DIR
+  ./$BOSH_DIRECTOR_DEPLOYMENT_DIR/virtualbox/delete-env.sh
+  cd $SCRIPT_DIR
+  rm -rf $BOSH_DIRECTOR_DIR
+fi

deployments/cloudfoundry/bosh/install-director.sh

@@ -0,0 +1,43 @@
+# Copyright Splunk Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#!/bin/bash
+
+set -euo pipefail
+
+BOSH_DIRECTOR_DIR="./bosh-env/virtualbox"
+BOSH_DIRECTOR_DEPLOYMENT_DIR="bosh-deployment"
+SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
+if [ -d $BOSH_DIRECTOR_DIR/$BOSH_DIRECTOR_DEPLOYMENT_DIR ]; then
+  echo "Bosh director already exists"
+  echo "If you want to build again, run make reinstall-director"
+  exit 0
+fi
+
+mkdir -p $BOSH_DIRECTOR_DIR
+cd $BOSH_DIRECTOR_DIR
+
+git clone --depth 1 https://github.com/cloudfoundry/bosh-deployment.git
+
+./$BOSH_DIRECTOR_DEPLOYMENT_DIR/virtualbox/create-env.sh
+
+source .envrc
+bosh -e vbox env
+
+bosh -e vbox update-cloud-config bosh-deployment/warden/cloud-config.yml
+bosh upload-stemcell --sha1 d44dc2d1b3f8415b41160ad4f82bc9d30b8dfdce \
+https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-bionic-go_agent?v=1.71
+
+cd $SCRIPT_DIR

deployments/cloudfoundry/bosh/jobs/splunk-otel-collector/spec

@@ -8,7 +8,6 @@ templates:
 
 packages:
 - splunk_otel_collector
-- signalfx_agent
 
 properties:
 

deployments/cloudfoundry/bosh/jobs/splunk-otel-collector/templates/ctl.erb

@@ -13,7 +13,9 @@ case $1 in
 
         echo $$ > $PIDFILE
 
-        exec /var/vcap/packages/splunk_otel_collector/splunk_otel_collector \
+        # Collector needs to be run as user that has permissions to get metrics from RLP
+        # gateway
+        exec chpst -u vcap:vcap /var/vcap/packages/splunk_otel_collector/splunk_otel_collector \
             --config /var/vcap/jobs/splunk-otel-collector/bin/config/otel-collector-config.yaml \
             >>  $LOG_DIR/splunk-otel-collector.stdout.log \
             2>> $LOG_DIR/splunk-otel-collector.stderr.log