Build failure: bazel-watcher

[kyleconroy]

Steps To Reproduce

Steps to reproduce the behavior:

1. nix-build -A bazel-watcher

Build log

https://gist.github.com/kyleconroy/ff7de358868ddc33e4803392c8f947d4

Additional context

The builds are also failing on Hydra

• https://hydra.nixos.org/job/nixpkgs/trunk/bazel-watcher.aarch64-darwin
• https://hydra.nixos.org/job/nixpkgs/trunk/bazel-watcher.x86_64-darwin

Notify maintainers

@kalbasit

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
warning: ignoring untrusted substituter 'https://devenv.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
this path will be fetched (0.00 MiB download, 0.00 MiB unpacked):
  /nix/store/x51zvbgb5ymr7bmpcc34x3rh7v949f4l-nix-info
copying path '/nix/store/x51zvbgb5ymr7bmpcc34x3rh7v949f4l-nix-info' from 'https://cache.nixos.org'...
 - system: `"aarch64-darwin"`
 - host os: `Darwin 22.6.0, macOS 13.5.1`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.17.0`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

[kolloch]

Several errors like this locally and in https://hydra.nixos.org/build/245701308/nixlog/7:

ERROR: An error occurred during the fetch of repository 'com_github_mattn_go_shellwords':
   Traceback (most recent call last):
        File "/private/tmp/nix-build-bazel-watcher-0.23.1-deps.tar.gz.drv-0/output/external/bazel_gazelle/internal/go_repository.bzl", line 247, column 17, in _go_repository_impl
                fail("failed to fetch %s: %s" % (ctx.name, result.stderr))
Error in fail: failed to fetch com_github_mattn_go_shellwords: fetch_repo: github.com/mattn/go-shellwords@v1.0.10: GOPROXY list is not the empty string, but contains no entries
ERROR: /private/tmp/nix-build-bazel-watcher-0.23.1-deps.tar.gz.drv-0/source/WORKSPACE:78:16: fetching go_repository rule //external:com_github_mattn_go_shellwords: Traceback (most recent call last):
        File "/private/tmp/nix-build-bazel-watcher-0.23.1-deps.tar.gz.drv-0/output/external/bazel_gazelle/internal/go_repository.bzl", line 247, column 17, in _go_repository_impl
                fail("failed to fetch %s: %s" % (ctx.name, result.stderr))
Error in fail: failed to fetch com_github_mattn_go_shellwords: fetch_repo: github.com/mattn/go-shellwords@v1.0.10: GOPROXY list is not the empty string, but contains no entries
ERROR: /private/tmp/nix-build-bazel-watcher-0.23.1-deps.tar.gz.drv-0/source/internal/ibazel/lifecycle_hooks/BUILD:3:11: no such package '@com_github_mattn_go_shellwords//': failed to fetch com_github_mattn_go_shellwords: fetch_repo: github.com/mattn/go-shellwords@v1.0.10: GOPROXY list is not the empty string, but contains no entries
 and referenced by '//internal/ibazel/lifecycle_hooks:lifecycle_hooks'
ERROR: /private/tmp/nix-build-bazel-watcher-0.23.1-deps.tar.gz.drv-0/source/internal/ibazel/live_reload/BUILD:3:11: no such package '@com_github_jaschaephraim_lrserver//': failed to fetch com_github_jaschaephraim_lrserver: fetch_repo: github.com/jaschaephraim/lrserver@v0.0.0-20171129202958-50d19f603f71: GOPROXY list is not the empty string, but contains no entries
 and referenced by '//internal/ibazel/live_reload:live_reload'
ERROR: /private/tmp/nix-build-bazel-watcher-0.23.1-deps.tar.gz.drv-0/source/internal/ibazel/fswatcher/fsnotify/BUILD:17:11: no such package '@com_github_fsnotify_fsnotify//': failed to fetch com_github_fsnotify_fsnotify: fetch_repo: github.com/fsnotify/fsnotify@v1.5.4: GOPROXY list is not the empty string, but contains no entries
 and referenced by '//internal/ibazel/fswatcher/fsnotify:fsnotify'

[kolloch]

GOPROXY is not set on the system outside the nix-shell and also does not seem to be set when the bazel command is executed within the nix-build.

On the same system, with bazelisk, bazel-watcher version 0.23.1 builds from source: bazelisk build //cli/ibazel

[kolloch]

merge 8c7c922/#231839 introduced a build failure (found via git bisect) which merged this change:

0d8594e

Reverting it on master makes it work but

• also reverts bazel-watcher to an older version.
• AND only works if the fetchAttrs.sha256 is set. I assume it will not actually call the updated fetch logic at all since it just uses the fetched dependencies from the cache.

Maybe divanorama can help out fixing it...

[kolloch]

@divanorama :)

[kolloch]

The fetchAttrs.sha256 makes it hard to bisect since it will "hide" all fetch affecting changes behind that cached results. Long live fixed output derivations!

[kolloch]

I was wrong. The fetchAttrs.sha256 hid that it was, in fact, broken before 8c7c922/https://github.com/NixOS/nixpkgs/pull/231839

$ git checkout  8c7c922^
$ nix build -L  .#bazel-watcher # works

diff --git a/pkgs/development/tools/bazel-watcher/default.nix b/pkgs/development/tools/bazel-watcher/default.nix
index 137e54b04c33..75a7d76a59db 100644
--- a/pkgs/development/tools/bazel-watcher/default.nix
+++ b/pkgs/development/tools/bazel-watcher/default.nix
@@ -82,7 +82,7 @@ buildBazelPackage rec {
       rm -rf $bazelOut/external/com_google_protobuf
     '';
 
-    sha256 = "sha256-lC9e5Z2cxLAeWXkiFGmcB6aOaurMvwrP/k5jl3gCfAc=";
+    sha256 = "";
   };
 
   buildAttrs = {

$ nix build -L  .#bazel-watcher # DOES NOT work anymore

[kolloch]

I bisected this to the following commit: 1bb67a9

with this bisect script:

#!/usr/bin/env bash

set -euf -o pipefail

set -x

git checkout -- pkgs/development/tools/bazel-watcher/

{ 
    nix build -L --impure --expr 'let pkgs = import ./.{}; in pkgs.bazel-watcher.deps.overrideAttrs (x: y: { outputHash="";}) ' 2>&1 \
    || true; 
} | tee build.log
cp build.log ..

if grep -q -e 'error: hash mismatch in fixed-output derivation.*bazel-watcher-.*-deps.tar.gz.drv'"'"':' build.log; then
    echo "SUCCESS: fixed-output derivation mismatch for dependencies -- fetching dependencies worked."
    exit 0
fi

if grep -q -e 'error: builder for .*bazel-watcher-.*-deps.tar.gz.drv'"'"' failed' build.log; then
    echo "FAILURE"
    exit 1
fi

echo "SKIP"
exit 125

Unfortunately, it is a huge merge.

[kolloch]

Okay fetching broke when go was unpinned: dc6ea0a

[kolloch]

Maybe related to golang/go#61928

[kolloch]

@kalbasit : Could you maybe merge #279898? It would be appreciated.