3p/nix: Find a way to make Boehm and gRPC work nicely together

#30
Opened by grfn at 2020-08-02T01·47+00

Basically the current issue is that gRPC's server is forking its own threads, in a way that Boehm can't find out about - then when we try to build a derivation we're getting "Collecting from unknown thread" with an abort. The boehm docs on the matter say that what we have to do is include gc.h after we include thread.h, then that redefines pthread_create et all to be the gc-traced versions; obviously that doesn't work for grpc because it's already been compiled.

There are a couple of options that we've thought of here:

  1. define an overridden derivation for grpc that #includes gc.h and use that in nix
  2. use the "bring your own threading model" grpc feature so that we spin up and manage our own thread pool
  3. wrap the shared pointer to the store in a mutex (which we should do anyway) and have every single grpc request spin off a new thread (this is intriguing for other reasons!)

There may also be something we can do in boehm to wrap dlopen to make all of this go away.

  1. https://github.com/ivmai/bdwgc/blob/master/doc/README.linux#L35-L47

    lukegb at 2020-08-02T01·48+00

  2. One compelling idea is to change the gRPC server to the "bring your own threading model" API, described here: https://grpc.io/docs/languages/cpp/async/

    This would nicely dovetail with protecting the nix Store against concurrent access: simply only have one thread responsible for actually executing the requests.

    riking at 2020-08-02T01·51+00

  3. grfn changed the subject of this issue from "Find a way to make Boehm and gRPC work nicely together" to "3p/nix: Find a way to make Boehm and gRPC work nicely together" at 2020-08-02T02·16+00

  4. Completely different thing pointed out by edef: Is any RPC server call actually calling the evaluator? If so, which?

    If not, can we move the GC initialisation to the evaluator and just ... not have the GC run in the daemon at all? (Nothing outside of libexpr should use it, but we should double-check)

    tazjin at 2020-08-02T22·03+00

  5. https://cl.tvl.fyi/c/depot/+/1584

    tazjin at 2020-08-02T23·19+00

  6. grfn closed this issue at 2020-08-09T02·41+00