tvix: add criterion benchmark for store path ingestion

#300
Opened by flokli at 2023-09-10T10·17+00

While we don't want to do any too fine-grained performance tuning in the evaluator yet, it'd still be helpful to be able to easily reason about the impact of a change for evaluation.

https://github.com/bheisler/criterion.rs provides a nice way to create some small benchmarks.

We should write a little bench for eval.

For now, this can be run on-demand by developers, invoking cargo bench before and after a certain change locally, similar to the existing benches.

An example for ATerm parsing is in https://cl.tvl.fyi/c/depot/+/9733

  1. I noticed that Gerrit adds extra git footers to commits, noting things like who approved the commit.

    Looping it into CI somehow could be a later step.

    It would be nifty if we logged these values in a git footer, like

    X-Benchmark-Milliseconds: 1058

    instead of doing what OfBorg does with eval reports and just chucking it on some random webpage (that probably gets garbage-collected periodically).

    amjoseph at 2023-09-13T10·22+00

  2. I'm not sure pulling this into the critical path is a good idea.

    flokli at 2023-10-16T08·37+00

  3. Let's start with some criterion benchmarks first, once we have them we can still see how to expose them somewhere.

    flokli at 2023-10-16T08·38+00

  4. flokli updated the body of this issue at 2023-10-16T08·39+00
  5. flokli changed the subject of this issue from "tvix: add criterion benchmark for some store paths" to "tvix: add criterion benchmark for store path ingestion" at 2023-10-16T08·54+00
  6. @amjoseph Exposing it in CI should be discussed in another issue (please open one if you want to), we already have other criterion benches and it's independent of this specific testbench.

    flokli at 2023-10-16T08·58+00

  7. flokli updated the body of this issue at 2023-10-16T08·58+00