provide CI protected automatic installation script for HEAD

[or-shachar]

In rules_scala(https://github.com/bazelbuild/rules_scala) and in rules_go(https://github.com/bazelbuild/rules_go) we wrote scripts for automatic installation of HEAD bazel build.
Those are based on jenkins URL which used to be:

"http://ci.bazel.io/job/Bazel/JAVA_VERSION=1.8,PLATFORM_NAME=${OS}-x86_64/"

But recent changes in the CI server messed up that order + the name of the installation file is not always the same. For example:

• bazel-0.5.3rc3-darwin-x86_64-installer.sh
• bazel-0.5.3rc3-installer-linux-x86_64.sh
• bazel--installer-linux-x86_64.sh
• bazel--darwin-x86-x64-installer.sh

I recently updated the script that builds the download URL to the following:

# OS can be either linux or darwin, we get OS from travis

# step 1 - download jenkins html page for last succesful artifacts
# the job name is "pipeline"
CI_INDEX_URL="http://ci.bazel.io/view/Bazel%20bootstrap%20and%20maintenance/job/Global/job/pipeline/lastSuccessfulBuild/artifact/node=${OS}-x86_64/variation="
wget -q -O build-index.html "${CI_INDEX_URL}"

# step 2 - find the artifact name inside that html
# The artifact may have different names, based on os and whether it's RC version or not
# We use complex regex to pull out the most suitable artifact
CI_ARTIFACT=$(cat build-index.html | sed -n  's/.*\(bazel\(-[^-]*\?\)\?\(-installer\)\?-'${OS}'-x86_64\(-installer\)\?.sh\).*/\1/p')

# step 3 - build the final download URL
URL="${CI_INDEX_URL}/${CI_ARTIFACT}"

# step 4 - clean up
rm build-index.html

echo "artifact = $CI_ARTIFACT"
echo "download url = $URL"

The thing is - it feels super hacky, and not very robust.
It would be great if we can have something like https://get.docker.com and https://experimental.docker.com/ which automatically installs latest / latest-stable version based on the current OS, preferably without rebuilding it.

Those scripts must be somehow tested in CI whenever they're updated, or whenever someone changes the jenkins job, to ensure the stability of the other tools CI.

CC: @damienmg @ittaiz @johnynek

[damienmg]

Wait why are you downloading release from ci.bazel.io?

We have bazelbuild/continuous-integration#95 for providing good HEAD binaries, random binary from HEAD can be totally broken, why are you doing that?

[damienmg]

Also you touched a bug on our CI: bazelbuild/continuous-integration#101

[or-shachar]

@damienmg Thanks for letting me know. That's how we worked until now in rules_scala, copied originally from rules_go. I'll follow bazelbuild/continuous-integration#95 but until it would be fixed, we'd keep using that hack.