ci: Reduce amount of duplicate builds for CLs

Opened by tazjin at 2021-08-26T11·44+00

Quick (abbreviated) description from IRC:

[2:31:42 pm] it might be time to make our CI system a little more stateful [2:37:03 pm] in each target, touch a stateful thing somewhere with the derivation hash if the build for that target succeeded [2:37:17 pm] then in the pipeline generation, skip each derivation hash that already succeeded (except on canon, where everything should be built) [2:37:28 pm] this would mean that each CL build only includes things that changed [2:37:54 pm] and it doesn't make the state important: if we lose it, it just means there'll be a slightly slower build once [2:40:23 pm] also I'm not sure if managing a state dir is necessary if we can just check if the outPath exists?

I think this would be nice. We could explicitly set those steps to skip in Buildkite also, so that they show up as greyed out on the build page.

I think that on canon skipping should be disabled, so that we always get a full build there after submitting (where it's not time-critical because dev workflow is not impacted as much).

  1. cl/3427

    tazjin at 2021-08-26T16·11+00

  2. tazjin closed this issue at 2021-08-26T16·29+00