3p/nix: Remote builder downloads all dependencies locally, then uploads them to the remote builder - even if the remote builder already has those dependencies.

Opened by grfn at 2020-08-01T02·41+00

this appears to occur even with builders-use-substitutes=true

  1. Is this based on code you read or behaviour you observed?

    tazjin at 2020-08-01T14·03+00

  2. The latter

    grfn at 2020-08-01T15·04+00

  3. understand what's going on here a little better now - it looks like builtin fetchers such as fetchTarball actually execute on the client, then upload their results to the daemon via AddToStoreNar - I can only imagine storing the result in memory along the way (ugh). What this really should be is a method in the store protocol to execute a fetcher on the target.

    grfn at 2020-08-06T03·01+00

  4. More interesting stuff here that I'm observing in upstream nix - if the path for a goal already exists on the remote builder but that remote builder is not configured as a substituter, the build will download that path from the remote builder, even though it's only necessary at the location where we're building the derivation

    grfn at 2020-08-21T13·44+00