I think @cjgillot we might want a bit more detail here in terms of the proposed approach. =) eddyb described their preferred approach. I don't have a strong opinion yet but I am in favor of the overarching goal. I'd actually like to see us reconciling a bit with salsa, longer term, but I'm not pushing on that yet, in part because I don't have enough energy to keep up with salsa anyway :P
I don't really have a converged design in mind. I would like to get to something like @eddyb described. Each crate would be able to only define a set of queries. The same crate, or a downstream crate would then be able to
However, I don't yet have a plan to get there. AFAIK, the main blocker is the definition of the
DepKind type, and how to get rid of it / render it opaque.
From my experiments, all the query invocation system can be moved out of librustc_middle. I think the definition of the
Providers struct would be movable. The issue is the
DepKind enum, and the types in
rustc_middle::dep_graph. They are intertwined into manual dependency handling, for instance with the
WithDepNode type. I don't have a path forward for this.
Hmm, so, salsa's solution here is that the dep-kind winds up getting generated at the very last crate (e.g., the driver), but very little code needs to interact with it anyhow.
We use a trait instead of providers, so that each crate can get a
&dyn Database with all the query methods -- this has some pros and some cons
The problem is not so much of the DepKind being used, but of the DepKind being required to instantiate the DepGraph. And the DepGraph is accessed directly. The first step would be to get rid of these direct accesses to the DepGraph, and replace them with some (query-based?) abstraction.
The largest use of the DepGraph in rustc_middle is in traits/select.rs. A first step may be to integrate the selection and evaluation caches into the query system. What are the constraints on the design for these caches?
I think i have an idea about this.
The goal: to decouple the compiler crates and remove unnecessary dependency.
Providers into many different smaller Providers, and store them in a
rustc_interface invoking each
provide fn separately, each module declare them into a
rustc_interface will execute these functions to populate the
TypeMap with the provider.
When one crate want to execute a provider defined within another crate, it imports the definition of that provider from that crate, and retrieve an instance of it from the
TypeMap, since everything is already initialized it can succeed in doing so.
In this way, i think we can decouple all the modules. No one needs to use or import a provider it neither provides nor uses.
@Charles Lew that's far too dynamic for my liking. not to mention inefficient by default
A library for safe cross-platform linker shenanigans.
"Press X to doubt" comes to mind
like that's top 10 famous last words of all time right there
@Charles Lew also this just doesn't seem necessary? look at how salsa does it. and even salsa seems a bit too much for rustc for me (but arguably doable)
The largest use of the DepGraph in rustc_middle is in traits/select.rs. A first step may be to integrate the selection and evaluation caches into the query system.
@cjgillot sorry, I just saw this - couldn't we just make that code be in a crate that depends on all the crates that define queries? (not necessary all the crates that implement them)
@cjgillot the problem with the trait system not being querifiable is mostly to do with cycles, which the trait system handles in a sound way, but queries in general cannot soundly depend on themselves. there also used to be inference too but we got around that with canonicalization. not sure how Chalk would integrate with incremental tbh