nixery: builds the same expression many times in parallel
#459
Opened by sterni at
I checked nevsky just a moment ago to look into 🦙 OOM-ing and discovered that almost all memory on the machine was being consumed by the following command which seems to have been invoked dozens of times in parallel by nixery:
/nix/store/bb3ayrdrpifnqwvpxhfn14hnlvsdbwr0-lix-2.94.0/bin/nix-build --show-trace --no-out-link --timeout 60 --argstr packages ["cacert","iana-etc","python311","python311packages.scapy"] --argstr srcType path --argstr srcArgs /nix/store/y29x6c3n3a0adljc8s6kh778qjlmgfjy-source --argstr system x86_64-linux --argstr loadPkgs /nix/store/jhhy0c8vvrhnxk0yxr10yg85yhdc20b8-load-pkgs.nix /nix/store/zk2qpnzz9hcvwh46q1vcrbzasfiny2jh-prepare-image.nix
I restarted nixery for now. I'm not sure how nixery works internally, but it seems to me that it should internally acquire a lock on a specific list of packages so the Nix evaluation isn't invoked in parallel (even though the Nix daemon should prevent redundant builds, but also only if the build succeeds).