Stream: t-compiler/major changes

Topic: Decentralize queries compiler-team#277

nikomatsakis (May 05 2020 at 10:27, on Zulip):

I think @cjgillot we might want a bit more detail here in terms of the proposed approach. =) eddyb described their preferred approach. I don't have a strong opinion yet but I am in favor of the overarching goal. I'd actually like to see us reconciling a bit with salsa, longer term, but I'm not pushing on that yet, in part because I don't have enough energy to keep up with salsa anyway :P

cjgillot (May 05 2020 at 18:18, on Zulip):

I don't really have a converged design in mind. I would like to get to something like @eddyb described. Each crate would be able to only define a set of queries. The same crate, or a downstream crate would then be able to provide it.

However, I don't yet have a plan to get there. AFAIK, the main blocker is the definition of the DepKind type, and how to get rid of it / render it opaque.
From my experiments, all the query invocation system can be moved out of librustc_middle. I think the definition of the Providers struct would be movable. The issue is the DepKind enum, and the types in rustc_middle::dep_graph. They are intertwined into manual dependency handling, for instance with the WithDepNode type. I don't have a path forward for this.

nikomatsakis (May 05 2020 at 19:20, on Zulip):

Hmm, so, salsa's solution here is that the dep-kind winds up getting generated at the very last crate (e.g., the driver), but very little code needs to interact with it anyhow.

nikomatsakis (May 05 2020 at 19:20, on Zulip):

We use a trait instead of providers, so that each crate can get a &dyn Database with all the query methods -- this has some pros and some cons

cjgillot (May 05 2020 at 21:21, on Zulip):

The problem is not so much of the DepKind being used, but of the DepKind being required to instantiate the DepGraph. And the DepGraph is accessed directly. The first step would be to get rid of these direct accesses to the DepGraph, and replace them with some (query-based?) abstraction.

cjgillot (May 05 2020 at 21:28, on Zulip):

The largest use of the DepGraph in rustc_middle is in traits/ A first step may be to integrate the selection and evaluation caches into the query system. What are the constraints on the design for these caches?

Charles Lew (Jul 19 2020 at 05:22, on Zulip):

I think i have an idea about this.

The goal: to decouple the compiler crates and remove unnecessary dependency.

The plan:

Split Providers into many different smaller Providers, and store them in a TypeMap.

Instead of rustc_interface invoking each provide fn separately, each module declare them into a distributed_slice, and rustc_interface will execute these functions to populate the TypeMap with the provider.

When one crate want to execute a provider defined within another crate, it imports the definition of that provider from that crate, and retrieve an instance of it from the TypeMap, since everything is already initialized it can succeed in doing so.

In this way, i think we can decouple all the modules. No one needs to use or import a provider it neither provides nor uses.

Charles Lew (Jul 19 2020 at 05:24, on Zulip):


eddyb (Jul 21 2020 at 21:23, on Zulip):

@Charles Lew that's far too dynamic for my liking. not to mention inefficient by default

eddyb (Jul 21 2020 at 21:23, on Zulip):

A library for safe cross-platform linker shenanigans.

"Press X to doubt" comes to mind

eddyb (Jul 21 2020 at 21:24, on Zulip):

like that's top 10 famous last words of all time right there

eddyb (Jul 21 2020 at 21:25, on Zulip):

@Charles Lew also this just doesn't seem necessary? look at how salsa does it. and even salsa seems a bit too much for rustc for me (but arguably doable)

eddyb (Jul 21 2020 at 21:29, on Zulip):

The largest use of the DepGraph in rustc_middle is in traits/ A first step may be to integrate the selection and evaluation caches into the query system.

@cjgillot sorry, I just saw this - couldn't we just make that code be in a crate that depends on all the crates that define queries? (not necessary all the crates that implement them)

eddyb (Jul 21 2020 at 21:30, on Zulip):

@cjgillot the problem with the trait system not being querifiable is mostly to do with cycles, which the trait system handles in a sound way, but queries in general cannot soundly depend on themselves. there also used to be inference too but we got around that with canonicalization. not sure how Chalk would integrate with incremental tbh

Last update: May 07 2021 at 06:45UTC