tvix/eval: nested thunk memoization is broken

#246
Opened by tazjin at 2023-02-01T09·31+00

We've been seeing some sudden unexpected slowness and multiple imports of the same files, even in seemingly simple code. For example:

let lib = import <nixpkgs/lib>;
in lib.fix(self: {
  a = 40;
  b = 2;
  c = self.a + self.b;
})

Ends up actually calling builtins.import on lib/fixed-points.nix multiple times. This is unexpected and something seems to be wrong with thunking, might be a side-effect of the previous trampoline refactoring. Investigating ...

  1. This reproducer doesn't actually reproduce the real issue, as fixed-points.nix is literally imported twice.

    We're seeing full re-imports of lib (over 50x) when evaluating e.g. <nixpkgs>.stdenv.bootstrapTools.drvPath, however, and this is likely a truer representation of the problem:

    env RUST_BACKTRACE=1 NIX_PATH="nix="(nix-instantiate --eval -E '<nix>')":$NIX_PATH" cargo run --profile release-with-debug -- -E '(import <nixpkgs> {}).stdenv.bootstrapTools.drvPath'

    tazjin at 2023-02-01T18·03+00

  2. tazjin changed the subject of this issue from "tvix/eval: suspicious multiple imports" to "tvix/eval: nested thunk memoization is broken" at 2023-02-02T10·02+00

  3. This is fixed in cl/8012. The fix and the problem are quite complex, and are described in detail in the commit message there.

    tazjin at 2023-02-02T10·03+00

  4. tazjin closed this issue at 2023-02-18T11·19+00