Enable Bazel remote caching on more/most continuous/experimental builds by default

[jamiesnape]

A tracking issue for the above. More relevant when bionic support causes the number of jobs to increase more than our CI budget can handle.

FYI @EricCousineau-TRI.

[EricCousineau-TRI]

Per f2f, can close when all PR jobs can use caching (including debug / sanitizer):

Remaining bits:

• linux-xenial-gcc-bazel-experimental-debug
• linux-bionic-clang-bazel-experimental-leak-sanitizer

At present, we save some budget, but dev experience is still a little painful due to having to wait for presently non-cached builds (debug, sanitizer).

Will see how things go for the next month to see how budget + dev experience goes.

[jwnimmer-tri]

FYI At TRI I'm experimenting with turning on Fission for Anzu. Fission should work fine on Bionic at least, and it's on the order of 5x reduction in debug build footprints (80% smaller). I think upstream bazelbuild/bazel#6606 would enable Fission by default on all of our platforms, but it's stalled in review. I wonder if we could help nudge that along, or if there is a 30-line patch to Drake that could have the same effect (declare our own crosstool, but delegate most of it to upstream local_cc).

Fission may also help with #9952.

[jamiesnape]

If it's truly mac that is holding them up, I could probably run with it, since I already worked with that code, but not convinced it is. That PR is on the wrong side of bazelbuild/bazel#7320 so would need some work to bring it up to date in any case.

[jamiesnape]

(My understanding is that debug fission with -gsplit-dwarf would not work with the Apple compiler at present, but the compiler test in the Bazel PR may erroneously indicate that it is.)

[jamiesnape]

bazelbuild/bazel#327 is relevant for Mac.

[jwnimmer-tri]

Fair enough. Sounds like the low-hanging fruit might be a bionic debug build that is included in the pre-merge set, depending on the crosstool hijinks that we'd need. How about I take a bit more time to see how Anzu fares once I turn this on there, and then we can revisit the question for Drake?

[jamiesnape]

Sounds good.

[jwnimmer-tri]

Per f2f, can close when all PR jobs can use caching (including debug / sanitizer) ...

I just looked at the most recent PR, and all of the jobs appear to be using the remote cache.