Stream: t-compiler/wg-parallel-rustc

Topic: implicit token at startup


Zoxc (Dec 18 2019 at 02:19, on Zulip):

We can avoid giving up the implicit token when creating the Rayon thread pool by changing the semantics of build_scoped so that it inherits the token from the caller and gives it to the first Rayon thread and also let the first Rayon thread execute the passed closure.

Zoxc (Dec 18 2019 at 03:23, on Zulip):

@simulacrum Are you planning to fix this by not letting rustc return all tokens to cargo when you have a channel to each rustc process?

simulacrum (Dec 18 2019 at 03:26, on Zulip):

It was our impression that a fix inside rayon is possibly non trivial or might point at deeper questions.

Current plan I believe is to keep track of the token count and just not release/acquire the first token, which should be relatively easy to implement hopefully.

simulacrum (Dec 18 2019 at 03:27, on Zulip):

In particular I think @nikomatsakis was not sure about whether this is pointing at some underlying problem with rayon or if it's something we should try to patch over ourselves.

simulacrum (Dec 18 2019 at 03:31, on Zulip):

What you're proposing loosely sounds like a more invasive but maybe also more principled change.

simulacrum (Dec 18 2019 at 03:32, on Zulip):

Though we do have the query latch which I think has the same problem as rayon

simulacrum (Dec 18 2019 at 03:33, on Zulip):

(but not exactly sure there, since it is semi-plausible that we can't run into trouble there as we don't have a situation where all threads latch, but I'm not confident)

Zoxc (Dec 18 2019 at 03:42, on Zulip):

It's not that principled. It only fixes one case where it could happen. It can also happen when the main thread is waiting on some other thread.

simulacrum (Dec 18 2019 at 12:29, on Zulip):

well, it's fine if the main thread releases its token

simulacrum (Dec 18 2019 at 12:29, on Zulip):

we just can't have a rustc without tokens

Last update: Jul 02 2020 at 18:45UTC