tvix: "path /bin/sh is not in the Nix store" when using tvix as a remote builder

#73
Opened by aspen at 2020-11-18T14·05+00

❯ nix-build test_drvs/basic.nix
these derivations will be built:
  /nix/store/yqpskxfy9z1dpy3f6s05kdg931fp5yy0-test.drv
building '/nix/store/yqpskxfy9z1dpy3f6s05kdg931fp5yy0-test.drv' on 'ssh://grfn@whitby.tvl.fyi'...
E20201118 14:02:13.077600 26467 shared.cc:306] error: Rpc call buildDerivation to unix:///nix/var/nix/daemon-socket/socket failed (INTERNAL): path '/bin/sh' is not in the Nix store
error: unexpected end-of-file
builder for '/nix/store/yqpskxfy9z1dpy3f6s05kdg931fp5yy0-test.drv' failed with exit code 1
error: build of '/nix/store/yqpskxfy9z1dpy3f6s05kdg931fp5yy0-test.drv' failed

  1. grfn changed the subject of this issue from "tvix: "path /bin/sh is not in the Nix store" when using tvix as a remote builder" to "tvix: "path /bin/sh is not in the Nix store" when building some derivations" at 2020-11-22T22·29+00

  2. I originally thought this was remote-builder related, but it turns out that this is caused by all derivations. here's the contents of basic.nix:

    derivation {
  name = "test";
  builder = "/bin/sh";
  args = [ "-c" "echo 2 > $out" ];
  system = "x86_64-linux";
}

    aspen at 2020-11-22T22·33+00

  3. s/all/some/

    aspen at 2020-11-22T22·42+00

  4. more is breaking here:

    E20201122 23:17:13.701364 16431 shared.cc:306] error: Rpc call buildDerivation to unix:///nix/var/nix/daemon-socket/socket failed (INTERNAL): path '/nix/store/y0sw8sxkr7kwlsndwixyf4s4snlfhjh1-bash-4.4-p23/bin/bash' is not in the Nix store

    aspen at 2020-11-22T23·18+00

  5. I can no longer reproduce this. Closing until I can.

    aspen at 2020-11-27T18·42+00

  6. grfn closed this issue at 2020-11-27T18·42+00
  7. grfn changed the subject of this issue from "tvix: "path /bin/sh is not in the Nix store" when building some derivations" to "tvix: "path /bin/sh is not in the Nix store" when using tvix as a remote builder" at 2020-11-27T20·30+00

  8. This does in fact appear to be limited to tvix-as-a-remote-builder. Also:

    ❯ nix-build -A ops.pipelines.depot
building path /nix/store/izqsfvk60q8z6n6a4abf2n5rp3cly6zm-golang.org_x_net-structure.json
building '/nix/store/n9f4m1yxb7hskarsi08bmfvn09mwnj9h-golang.org_x_net-structure.json.drv' on 'ssh://grfn@whitby.tvl.fyi'
E20201127 20:30:35.653002 49739 shared.cc:306] error: Rpc call buildDerivation to unix:///nix/var/nix/daemon-socket/socket failed (INTERNAL): path '/nix/store/y0sw8sxkr7kwlsndwixyf4s4snlfhjh1-bash-4.4-p23/bin/bash' is not in the Nix store
builder for '/nix/store/n9f4m1yxb7hskarsi08bmfvn09mwnj9h-golang.org_x_net-structure.json.drv' failed with exit code 1
E20201127 15:30:35.831517 32731 shared.cc:306] error: Operation failed: INTERNAL: build of '/nix/store/n9f4m1yxb7hskarsi08bmfvn09mwnj9h-golang.org_x_net-structure.json.drv' failed (exit code 1)
I20201127 15:30:35.833055 32731 shared.cc:308] (use '--show-trace' to show detailed location information)

    aspen at 2020-11-27T20·31+00

  9. note that that was even with tvix as the client.

    aspen at 2020-11-27T20·31+00

  10. I have checked and whitby does in fact have this path, as does my local machine.

    aspen at 2020-11-27T20·34+00

  11. grfn reopened this issue at 2020-11-27T21·05+00

  12. We never should've been calling assertStorePath on BasicDerivation.builder in the first place. https://cl.tvl.fyi/c/depot/+/2175 fixes this.

    aspen at 2020-11-27T21·29+00

  13. grfn closed this issue at 2020-11-27T21·29+00