tvix-store import: exit nonzero in case of errors

#326
Opened by flokli at 2023-10-16T17·27+00

If a new path is uploaded, the client traverses the filesystem structure, then calls PathInfoService.CalculateNAR() to get the nar hashes and size.

However, it seems in some cases the CalculateNAR() request fails, as the Directory with the requested digest doesn't exist.

This seems to be some sort of race, either the upload is still in progress, or the thread responding to CalculateNAR() doesn't see it yet.

The client also doesn't return a nonzero exit code.

Both should be fixed.

  1. Steps to reproduce:

    Start a tvix-store daemon on one machine, run tvix-store import on another machine. The two machines were on a noisy wireless network.

    We tried with the docs folder, touching a bunch of files between every attempt, and had a 50/50 chance to reproduce it.

    flokli at 2023-10-16T17·29+00

  2. Update, not really 50/50, more like a 10% chance.

    flokli at 2023-10-16T17·31+00

  3. Fix: https://cl.tvl.fyi/c/depot/+/9761

    cbrewster at 2023-10-17T02·40+00

  4. cbrewster closed this issue at 2023-10-17T13·01+00
  5. cbrewster reopened this issue at 2023-10-17T13·02+00

  6. The ingestion race has been fixed, but the exit code still needs to be fixed.

    cbrewster at 2023-10-17T13·02+00

  7. flokli changed the subject of this issue from "tvix: race in path import " to "tvix-store import: exit nonzero in case of errors" at 2023-10-19T11·37+00

  8. I think this should be fixed with https://cl.tvl.fyi/c/depot/+/10331/3.

    flokli at 2023-12-12T21·04+00

  9. flokli closed this issue at 2023-12-12T21·06+00