Stream: t-compiler

Topic: queries for RLS


nikomatsakis (Jan 03 2019 at 21:20, on Zulip):

@Igor Matuszewski was asking me what concrete steps we should take to start transitioning the RLS model over to use the query system.

nikomatsakis (Jan 03 2019 at 21:20, on Zulip):

cc @mw, @eddyb, @Zoxc and probably others

Igor Matuszewski (Jan 03 2019 at 21:23, on Zulip):

As a side note, I've been stalking and looking at https://github.com/salsa-rs/salsa from afar, seeing as @Aleksey Kladov have been working on it for the Rust analyzer

nikomatsakis (Jan 03 2019 at 21:23, on Zulip):

it seems to me that a first step might be to try to convert save analysis to use more queries internally

Igor Matuszewski (Jan 03 2019 at 21:23, on Zulip):

Do we see rustc transitioning to that model/implementation?

nikomatsakis (Jan 03 2019 at 21:23, on Zulip):

unclear at this time

nikomatsakis (Jan 03 2019 at 21:24, on Zulip):

it's obviously derived from rustc, it's missing some things that rustc uses; I would like to see rustc move towards a model that is separate from rustc as salsa is, since I think it's much easier to look at the code that way

nikomatsakis (Jan 03 2019 at 21:24, on Zulip):

it seems likely that we can evolve salsa to meet rustc's needs

Igor Matuszewski (Jan 03 2019 at 21:25, on Zulip):

makes sense, okay

nikomatsakis (Jan 03 2019 at 21:25, on Zulip):

but I think salsa as it is today is not quite there yet (notably, it doesn't implement persistence, but also some other things)

nikomatsakis (Jan 03 2019 at 21:25, on Zulip):

sadly @Igor Matuszewski i've got to run in a bit, but I'd like to dig more into the specifics here, I thik it'd be great to start taking some actual steps

Igor Matuszewski (Jan 03 2019 at 21:25, on Zulip):

sure thing

nikomatsakis (Jan 03 2019 at 21:25, on Zulip):

iirc the save analysis code today is not using a lot of queries, so I still think that's a logical place to start

Igor Matuszewski (Jan 03 2019 at 21:25, on Zulip):

so a good starting point would be to investigate save-analysis using queries internally?

nikomatsakis (Jan 03 2019 at 21:26, on Zulip):

right, and trying to move it towards something more incremental friendly

nikomatsakis (Jan 03 2019 at 21:26, on Zulip):

right now I think the RLS is not using incremental at all?

nikomatsakis (Jan 03 2019 at 21:26, on Zulip):

even if we still kept the "one big file of data per crate" model that I think save analysis has,

nikomatsakis (Jan 03 2019 at 21:26, on Zulip):

it can still use incremental internally to e.g. avoid redoing type-checking for things that haven't changed

nikomatsakis (Jan 03 2019 at 21:27, on Zulip):

( I should note that I've not looked closely at the code recently and I may be out of date, so if you see me saying things that seem wrong, tell me :) )

Igor Matuszewski (Jan 03 2019 at 21:28, on Zulip):

we do run rustc in-process just like Cargo would but iirc we don't set anything for the incremental, so I think by default we should be using incremental between every passes

Igor Matuszewski (Jan 03 2019 at 21:28, on Zulip):

the analysis json blobs is used for dependencies, which we then read into the 'index', which is basically the database for everything IDE can query

matklad (Jan 03 2019 at 23:28, on Zulip):

Hi!

This perhaps is a good place to ask “how the ideal interface between rustc and RLS should like?” Sure, we can querize save analysis, and that’s probably the easiest thing to do, but is it the right thing?

I’ve been pondering “object oriented code model” interface. That is, you have types like “crate”, “module”, “function», etc. module has parent, children, functions method which return apropriate objects. All objects look as if they are fully compiled: all references are resolved, all types are infered, etc.

This model will be convenient to use on the ide (and probably rustdoc) side to use. It will also give us an enormose amount of flexibility, if the “implementation” of the model is swappable. I see at least two long term back ends for OO code model.

The first one is lazy on demand query based backend (although the model looks like as if everything is computed, it certainly can just kick queries when methods are called).

The second one is backed by a data file. That is, a model which reads something like save analysis, where method calls deserialize various bits of data.

And, of course, in the meantime rust-analyzer can also produce such a model :-)

matklad (Jan 03 2019 at 23:30, on Zulip):

An action item here (if we want to go this route) is to create such a model with a pluggable backend in a separate repo, and then write the impl for rust

matklad (Jan 03 2019 at 23:31, on Zulip):

This crate will have a set of concrete types used by consumers, and a trait definition (not implementation) for producers

matklad (Jan 03 2019 at 23:32, on Zulip):

So ‘trait Backend’ and ‘struct Module { backend: Arc<dyn Backend>, id: u32 }’

matklad (Jan 03 2019 at 23:33, on Zulip):

I am fairly confident in this design, and I’d love to start working on it :)

Perhaps this needs an implementation RFC though?

matklad (Jan 03 2019 at 23:34, on Zulip):

cc @Steve Klabnik : that’s the stuff we’ve discussed at rust-analyzer Discord

Steve Klabnik (Jan 03 2019 at 23:43, on Zulip):

:+1:

matklad (Jan 04 2019 at 00:41, on Zulip):

To put code where my mouth is, here's a super-minimal code model (as a separate crate), and the corresponding rust-analyzer impl: https://github.com/rust-analyzer/rust-analyzer/commit/f6ee0a5e57645c7bd87c1f39f11908fb5c2deefa

Igor Matuszewski (Jan 04 2019 at 11:02, on Zulip):

As of now, I'd say that rls-analysis crate acts as this interface for the RLS - everything is resolved, we operate on items with ids, know their kind, know their parent/child hierarchy

Igor Matuszewski (Jan 04 2019 at 11:02, on Zulip):

and rls-data (emitted by librustc_save_analysis) is consumed by the rls-analysis and lowered to make multi-crate analysis work and for the IDs not to clash

Igor Matuszewski (Jan 04 2019 at 11:03, on Zulip):

so it's more of an implementation detail - one could imagine we could try and plug in query-like backend to the rls-analysis and see how far along can we go with this

matklad (Jan 04 2019 at 11:52, on Zulip):

rls-analysis is not an object model.

I'd love to see something more like this:

impl Function {
    fn name() -> String;
    fn return_type() -> Type;
    fn parameters() -> Vec<Type>;
}

In contrast, save-analysis more or less exposes the raw data, and it's API shape is dictated by the shape of these data.

There's a big difference in abstraction level between, for exmaple C# model and save analysis api.

matklad (Jan 04 2019 at 11:55, on Zulip):

that's this impl Function in C# that I am talking about: https://docs.microsoft.com/en-us/dotnet/api/microsoft.codeanalysis.imethodsymbol

Igor Matuszewski (Jan 04 2019 at 12:41, on Zulip):

Thanks for C# model, interesting!

Igor Matuszewski (Jan 04 2019 at 12:43, on Zulip):

Re save analysis api, it’s crude and not really polished; I just wanted to observe that with the exposed API can act as an interface for pluggable backends (query, data-backed)

Igor Matuszewski (Jan 04 2019 at 12:43, on Zulip):

and it should contain all the necessary information to expose the object model on top, I believe?

Igor Matuszewski (Jan 04 2019 at 12:44, on Zulip):

These are mostly implementation details, however; I’m not arguing what’s the better abstraction for the comsumers

matklad (Jan 04 2019 at 12:45, on Zulip):

and it should contain all the necessary information to expose the object model on top, I believe?

yep, rls-analysis could be a backend for the model I am proposing.

Igor Matuszewski (Jan 04 2019 at 13:02, on Zulip):

(by the way, is there a way to subscribe to this topic alone or add it to favorites or similar?)

nikomatsakis (Jan 07 2019 at 21:15, on Zulip):

@Igor Matuszewski I don't know of a way to subscribe to one topic, but maybe we should make a stream for this. I think in general we need more streams.

nikomatsakis (Jan 07 2019 at 21:15, on Zulip):

e.g., maybe a #t-compiler/RLS or something

blitzerr (Jan 07 2019 at 22:26, on Zulip):

@Igor Matuszewski
Unfortunately not at the moment. I created an issue for zulip

Igor Matuszewski (Jan 07 2019 at 22:27, on Zulip):

Thanks for the heads up!

Igor Matuszewski (Jan 07 2019 at 22:28, on Zulip):

@nikomatsakis I’m not a frequent (yet? :wink:) user of Zulip so I can’t say

Igor Matuszewski (Jan 07 2019 at 22:28, on Zulip):

But I imagine RLS or E2E queries should be big enough topic to cover multiple, er, topics

nikomatsakis (Jan 09 2019 at 17:00, on Zulip):

yeah, I suspect we should be creating more zulip streams

nikomatsakis (Jan 09 2019 at 17:01, on Zulip):

we had talked about a help stream, for example

Last update: Nov 16 2019 at 02:10UTC