Stream: t-compiler

Topic: design meeting 2019-12-13


nikomatsakis (Dec 13 2019 at 14:50, on Zulip):

Hey @T-compiler/meeting -- design meeting in 10 minutes.

nikomatsakis (Dec 13 2019 at 14:51, on Zulip):

As a reminder, the topic is compiler-team#222, or "IDE/library-ification" plans. We have a hackmd that was prepared in advance which has some notes, questions, etc.

nikomatsakis (Dec 13 2019 at 14:52, on Zulip):

Please feel free to review and add things to the end.

pnkfelix (Dec 13 2019 at 14:53, on Zulip):

oh hey i guess this is maybe a pre-announcement: There is a beta-nom that needs to be resolved (approved or denied) asap.

pnkfelix (Dec 13 2019 at 14:53, on Zulip):

beta-nom: "Stabilize attribute macros on inline modules" #64273

pnkfelix (Dec 13 2019 at 14:53, on Zulip):

I assume we can allocate a couple minutes at the start to discuss it. (I don't want to discuss it until the meeting starts.)

simulacrum (Dec 13 2019 at 14:54, on Zulip):

there are three I think?

nikomatsakis (Dec 13 2019 at 14:54, on Zulip):

I thought we had a general policy of not backporting stabilizations

simulacrum (Dec 13 2019 at 14:54, on Zulip):

https://github.com/rust-lang/rust/pull/64273 is one, but also https://github.com/rust-lang/rust/pull/66983 and https://github.com/rust-lang/rust/pull/67134

pnkfelix (Dec 13 2019 at 14:54, on Zulip):

I thought we had a general policy of not backporting stabilizations

I agree with this and was planning on saying so at the meeting.

nikomatsakis (Dec 13 2019 at 14:54, on Zulip):

OK, I see @pnkfelix has a comment explaining their reasoning for why to stabilize faster

pnkfelix (Dec 13 2019 at 14:54, on Zulip):

uh wait did I somehow just contradict myself ... ?

nikomatsakis (Dec 13 2019 at 14:55, on Zulip):

Nope, sorry, my bad. Edited my comment. :)

nikomatsakis (Dec 13 2019 at 14:57, on Zulip):

Welp, let's setup a place for people to leave responses. There are some last minute backport nominations, add emojis:

nikomatsakis (Dec 13 2019 at 14:57, on Zulip):
nikomatsakis (Dec 13 2019 at 14:57, on Zulip):
nikomatsakis (Dec 13 2019 at 14:58, on Zulip):
simulacrum (Dec 13 2019 at 15:00, on Zulip):

(I will do the next round of backports today, most likely, fwiw)

nikomatsakis (Dec 13 2019 at 15:00, on Zulip):

Hello @T-compiler/meeting -- design meeting starting now! As a special treat, we have some backporting nominations, see above.

nikomatsakis (Dec 13 2019 at 15:00, on Zulip):

Announcements or comments on backports

nikomatsakis (Dec 13 2019 at 15:00, on Zulip):
pnkfelix (Dec 13 2019 at 15:01, on Zulip):

you mean yesterday's meeting?

pnkfelix (Dec 13 2019 at 15:01, on Zulip):

or last week's?

nikomatsakis (Dec 13 2019 at 15:02, on Zulip):

I meant last week's design meeting

nikomatsakis (Dec 13 2019 at 15:02, on Zulip):

IIRC I felt there were a lot of interesting points made back and forth but I was unable to keep a hackmd up to date in the moment

matklad (Dec 13 2019 at 15:04, on Zulip):

@Florian Diebold are you around by any chance?

pnkfelix (Dec 13 2019 at 15:04, on Zulip):

so regarding the back ports: I personally don't find petrochenkov's arguments for back porting #64273 compelling. I know they had wanted all the macro enhancements to be released in sync, but I don't see a big reason to do so

nikomatsakis (Dec 13 2019 at 15:04, on Zulip):

I agree with that

nikomatsakis (Dec 13 2019 at 15:04, on Zulip):

At least, I think the value of having a general rule to make our live's easier is higher :)

Florian Diebold (Dec 13 2019 at 15:04, on Zulip):

Florian Diebold are you around by any chance?

yes, more or less

nikomatsakis (Dec 13 2019 at 15:04, on Zulip):

It seems like 1.41 can be that "flag release" instead that authors can cite and rely on

pnkfelix (Dec 13 2019 at 15:04, on Zulip):

to be clear, I really don't have anything against this particular stabilization. But I also don't see much compelling me to deviate from our rule

centril (Dec 13 2019 at 15:05, on Zulip):

I have another backport candidate: https://github.com/rust-lang/rust/pull/67278

Igor Matuszewski (Dec 13 2019 at 15:05, on Zulip):

(FWIW here's agenda for today's meeting https://hackmd.io/w8OIoc7iQsuTrBlRV1xKMQ)

pnkfelix (Dec 13 2019 at 15:05, on Zulip):

(especially since in this case, a beta backport is implicitly a stable backport, in the sense that the beta->stable transition is imminent)

Pietro Albini (Dec 13 2019 at 15:06, on Zulip):

I have another backport candidate: https://github.com/rust-lang/rust/pull/67278

that's just a stable-to-stable right? I feel like it's too late and risky to backport it now

pnkfelix (Dec 13 2019 at 15:06, on Zulip):

I think #67278 can ride the trains

pnkfelix (Dec 13 2019 at 15:07, on Zulip):

or at least wait until after the beta->stable transition

centril (Dec 13 2019 at 15:07, on Zulip):

The last time we backported a stabilization was uniform_paths in 1.32.0 and we had a damn good reason for it then

centril (Dec 13 2019 at 15:08, on Zulip):

that's just a stable-to-stable right? I feel like it's too late and risky to backport it now

it is yes, in error code; it's in an error path, so it's not risky, I'd say

nikomatsakis (Dec 13 2019 at 15:08, on Zulip):

Seems like we are :back: on #67134 and #66983 but not #64273

centril (Dec 13 2019 at 15:09, on Zulip):

(the actual diff is only expected => self.expected_ty(), literally)

simulacrum (Dec 13 2019 at 15:09, on Zulip):

I would prefer to avoid backporting PRs that have not yet landed unless they have a clear reason to be backported

nikomatsakis (Dec 13 2019 at 15:09, on Zulip):

And #67278 is a bit less clear, but there's at least some objection, I think based mostly on the fact that it hasn't landed yet (c?) and doesn't seem that urgent

centril (Dec 13 2019 at 15:09, on Zulip):

yeah not urgent

nikomatsakis (Dec 13 2019 at 15:11, on Zulip):

OK.

nikomatsakis (Dec 13 2019 at 15:11, on Zulip):

I adjusted the labels and left comments on proposed PRs

centril (Dec 13 2019 at 15:11, on Zulip):

I'd say that even with the ICE the output is fairly clear anyways

nikomatsakis (Dec 13 2019 at 15:12, on Zulip):

Shall we get started with the meeting proper?

nikomatsakis (Dec 13 2019 at 15:12, on Zulip):

As @Igor Matuszewski noted, there is a hackmd with notes that we prepared

nikomatsakis (Dec 13 2019 at 15:12, on Zulip):

I think my goal for this meeting would be:

nikomatsakis (Dec 13 2019 at 15:13, on Zulip):
nikomatsakis (Dec 13 2019 at 15:13, on Zulip):
nikomatsakis (Dec 13 2019 at 15:14, on Zulip):
nikomatsakis (Dec 13 2019 at 15:14, on Zulip):

Anyway in the doc we proposed to start by summarizing the plans and logic behind them, so maybe we should start there

nikomatsakis (Dec 13 2019 at 15:15, on Zulip):

The question at hand is what to do about the fact that we have:

nikomatsakis (Dec 13 2019 at 15:15, on Zulip):
nikomatsakis (Dec 13 2019 at 15:17, on Zulip):

We were considering I think two major options:

nikomatsakis (Dec 13 2019 at 15:17, on Zulip):
nikomatsakis (Dec 13 2019 at 15:17, on Zulip):

The conclusion we came to was that the first option was preferable, for a number of reasons. This is sort of the path we've been taking anyway, that we've sometimes called "library-ification"

nikomatsakis (Dec 13 2019 at 15:18, on Zulip):

The reasoning is (I think) outlined in the doc, I'm not sure how much time to spend on it

nikomatsakis (Dec 13 2019 at 15:18, on Zulip):

Yeah, it's in the alternatives considered section

pnkfelix (Dec 13 2019 at 15:19, on Zulip):

should we review the potential discussion topics (written post overview in agenda) and maybe let people vote on which are most important to talk about first?

nikomatsakis (Dec 13 2019 at 15:19, on Zulip):

(The short version is that while this version does have a kind of maintenance cost, it lets us deliver value faster and also makes it easier for us to iterate)

pnkfelix (Dec 13 2019 at 15:19, on Zulip):

or do you want to just decide, without overhead of voting?

centril (Dec 13 2019 at 15:19, on Zulip):

Agree with first option; second seems unrealistic as a practical matter

nikomatsakis (Dec 13 2019 at 15:19, on Zulip):

well, before we do that I wanted to say one other thing @pnkfelix

nikomatsakis (Dec 13 2019 at 15:20, on Zulip):

one of the questions that's not addressed above is how to integrate RLS + rust-analyzer

Igor Matuszewski (Dec 13 2019 at 15:20, on Zulip):

agreeing with @centril , was going to raise the high cost of 2nd approach

nikomatsakis (Dec 13 2019 at 15:20, on Zulip):

we probably don't have to go into the deatils but it's worth summarizing the current thoughts

pnkfelix (Dec 13 2019 at 15:20, on Zulip):

integrate save-analysis you mean?

nikomatsakis (Dec 13 2019 at 15:21, on Zulip):

in terms of feature set, RLS offers two main things over rust-analyzer right now:

both of these work by executing the compiler and using save-analysis data

Igor Matuszewski (Dec 13 2019 at 15:21, on Zulip):

I think it's worth rebranding RLS to LSP serving static index using save-analysis while focusing on RA to be the "IDE" (low latency, incremental)

nikomatsakis (Dec 13 2019 at 15:21, on Zulip):

the thought is to extend rust-analyzer so that (by default) it works in a similar way

nikomatsakis (Dec 13 2019 at 15:21, on Zulip):

while also having its own, from scratch versions of these things using the shared library-ification codebase

nikomatsakis (Dec 13 2019 at 15:21, on Zulip):

that users can opt-in to

nikomatsakis (Dec 13 2019 at 15:21, on Zulip):

there are some details of how this might be done, e.g. @matklad was advocating for standalone executable tools that rust-analyzer can invoke

nikomatsakis (Dec 13 2019 at 15:22, on Zulip):

I'm not sure if there's been much experimentation in that direction but anyway that's the big picture from my perspective

centril (Dec 13 2019 at 15:22, on Zulip):

would this new "save-analysis"-like thing be an AST or HIR based analysis?

nikomatsakis (Dec 13 2019 at 15:22, on Zulip):

I mean I think to start it would literally be save-analysis

Igor Matuszewski (Dec 13 2019 at 15:22, on Zulip):

I don't think we need to reinvent save-analysis as of now

nikomatsakis (Dec 13 2019 at 15:22, on Zulip):

the goal is to bring rust-analyzer up to "parity" as quickly as possible

Igor Matuszewski (Dec 13 2019 at 15:22, on Zulip):

(we could, but that might not give us much benefit initially)

eddyb (Dec 13 2019 at 15:22, on Zulip):

save-analysis is... suboptimal though

centril (Dec 13 2019 at 15:23, on Zulip):

^--- very much this

Igor Matuszewski (Dec 13 2019 at 15:23, on Zulip):

Agreed on this

nikomatsakis (Dec 13 2019 at 15:23, on Zulip):

I wouldn't argue with that

nikomatsakis (Dec 13 2019 at 15:23, on Zulip):

I just dont' know how it's relevant :P

Igor Matuszewski (Dec 13 2019 at 15:23, on Zulip):

However as @nikomatsakis outlined I think it's more about feature parity

Igor Matuszewski (Dec 13 2019 at 15:23, on Zulip):

and make one IDE tool "official"

Igor Matuszewski (Dec 13 2019 at 15:23, on Zulip):

or blessed or other

nikomatsakis (Dec 13 2019 at 15:23, on Zulip):

I'm joking of course, it's relevant, but I think in terms of priorities it doesn't make sense to be focusing on re-implementing save-analysis (over, say, extending the IDE part of things libraryification efforts)

Igor Matuszewski (Dec 13 2019 at 15:24, on Zulip):

and to converge the effort, so to speak (and not confuse the users about "competing" platforms)

eddyb (Dec 13 2019 at 15:24, on Zulip):

my two cents on this is that RLS would be simpler and smaller if it just used the compiler APIs on the fly

eddyb (Dec 13 2019 at 15:24, on Zulip):

and there was no save-analysis between in-memory compiler state and the IDE

eddyb (Dec 13 2019 at 15:25, on Zulip):

but maybe save-analysis serves some function that this can't work with?

Igor Matuszewski (Dec 13 2019 at 15:25, on Zulip):

offline code index format is a good use case

eddyb (Dec 13 2019 at 15:25, on Zulip):

(ignoring the MXR/DXR efforts that spawned it in the first place)

Igor Matuszewski (Dec 13 2019 at 15:25, on Zulip):

although technically RLS could invoke compiler queries right now, yes

nikomatsakis (Dec 13 2019 at 15:25, on Zulip):

This is basically what we're talking about -- another option, presumably, might be to extend rust-analyzer to not use save-analysis but rather to communicate directly with rustc

nikomatsakis (Dec 13 2019 at 15:26, on Zulip):

Well, I think there's some tricky bits there

mw (Dec 13 2019 at 15:26, on Zulip):

some kind of pre-computed database like the save-analysis data is probably a good idea

nikomatsakis (Dec 13 2019 at 15:26, on Zulip):

For one thing, I think that we really want to invoke cargo a lot of the times

nikomatsakis (Dec 13 2019 at 15:26, on Zulip):

i.e., you want to handle things like build.rs and friends

eddyb (Dec 13 2019 at 15:26, on Zulip):

since RLS already runs rustc in-process, why don't we experiment with short-circuiting the data collection process there?

nikomatsakis (Dec 13 2019 at 15:27, on Zulip):

For another, rustc is presently focused on a crate at a time, but we need to incorporate many crates into the workspace and so forth

centril (Dec 13 2019 at 15:27, on Zulip):

save-analysis is some ~4kloc; honestly not that crazy to rewrite the whole thing

eddyb (Dec 13 2019 at 15:27, on Zulip):

also, even today, rust-analyzer could just run RLS in the background and provide some of that functionality by using LSP, no? if we want one tool that can do multiple things

eddyb (Dec 13 2019 at 15:27, on Zulip):

@centril I don't think that code has any reason to exist

Igor Matuszewski (Dec 13 2019 at 15:27, on Zulip):

I'm worried that for doing references we need to do the entire lex -> parse -> expand -> name resolve pass

Igor Matuszewski (Dec 13 2019 at 15:27, on Zulip):

so I'm not sure if in terms of engineering effort it'd be easier to reuse save-analysis for now

nikomatsakis (Dec 13 2019 at 15:27, on Zulip):

I'm trying to decide whether this is a rathole or the question to be answered :)

eddyb (Dec 13 2019 at 15:27, on Zulip):

like, I don't think what it does is useful, let alone it does it right now

matklad (Dec 13 2019 at 15:28, on Zulip):

Running RLS in background is technically complicated, b/c it's rustc_private. Reading save-analysis from disk is technically trivial

eddyb (Dec 13 2019 at 15:28, on Zulip):

@matklad run it like VSCode does

eddyb (Dec 13 2019 at 15:28, on Zulip):

not some library

eddyb (Dec 13 2019 at 15:29, on Zulip):

I'm worried that for doing references we need to do the entire lex -> parse -> expand -> name resolve pass

can you explain a bit more?

mw (Dec 13 2019 at 15:29, on Zulip):

having data instead of a runtime API as the common interface seems like a good idea to me

Igor Matuszewski (Dec 13 2019 at 15:29, on Zulip):

@eddyb I was thinking that for rust-analyzer to do all that we'd need to set up basically the compilation ourselves, which I'm not sure it's easier than to spawn rustc -Zsave-analysis and read the JSON

mw (Dec 13 2019 at 15:29, on Zulip):

it would be good to make that data incrementally updateable though

nikomatsakis (Dec 13 2019 at 15:29, on Zulip):

I'm a bit confused @eddyb -- you're proposing that rust-analyzer runs RLS and (e.g.) forwards some LSP queries to it? We did discuss this, I think, it seems potentially plausible to me, though I think what you really want is probably a stripped down version of RLS. It's kind of the "competitor" to having command-line utilities that execute.

centril (Dec 13 2019 at 15:29, on Zulip):

I don't think that code has any reason to exist

@eddyb yea sure; -- would also add that I wouldn't trust the correctness of save-analysis in general at least given my past experience of tweaking it

Igor Matuszewski (Dec 13 2019 at 15:30, on Zulip):

but I was just saying that from engineering PoV and initial cost

Igor Matuszewski (Dec 13 2019 at 15:30, on Zulip):

incremental updates of index files would be a great benefit, that's true

nikomatsakis (Dec 13 2019 at 15:31, on Zulip):

Another factor that I recall @matklad raising was being able to handle "custom setups" that don't use cargo; i.e., the more our interface is "run this command with some extra rustflags", the easier that will be to handle.

nikomatsakis (Dec 13 2019 at 15:31, on Zulip):

(I believe this is currently a weakness of the RLS today, but I'm not sure of the details there)

Igor Matuszewski (Dec 13 2019 at 15:31, on Zulip):

we can feed the build plan to RLS and it can work in that environment, but by default we run cargo initially to fetch that plan

nikomatsakis (Dec 13 2019 at 15:32, on Zulip):

In any case, am I correct that at the moment we are not arguing with the "overall plan" but more about the details of how one might augment rust-analyzer to bring it up to parity?

eddyb (Dec 13 2019 at 15:32, on Zulip):
  1. I consider save-analysis to be a complete hack
  2. LSP forward is also a JSON hack, but presumably requires less work
eddyb (Dec 13 2019 at 15:32, on Zulip):

this is why I prefer LSP over spreading the save-analysis misdesign further

Igor Matuszewski (Dec 13 2019 at 15:32, on Zulip):

regarding lsp, there's an LSIF (https://github.com/microsoft/language-server-protocol/issues/623) floating around

eddyb (Dec 13 2019 at 15:33, on Zulip):

then parts of RLS can be rewritten to bypass save-analysis and just use the compiler state

pnkfelix (Dec 13 2019 at 15:33, on Zulip):

"LSP forward" ?

matklad (Dec 13 2019 at 15:33, on Zulip):

@eddyb agree with 1, disagree with 2: it means that we need to maintain RLS as a separate binary at all.

The idea is that rust-analyzer gets save analysis directly from compiler, by running cargo check -Zsave-analyzis

Igor Matuszewski (Dec 13 2019 at 15:33, on Zulip):

which is an LSP save-analysis

mw (Dec 13 2019 at 15:33, on Zulip):

to be clear, nobody is proposing to keep save-analysis exactly the way it is forever here, right?

Igor Matuszewski (Dec 13 2019 at 15:33, on Zulip):

which I'd probably vote to move save-analysis to (in terms of the format/serving)

eddyb (Dec 13 2019 at 15:33, on Zulip):

@matklad but that's not forward-compatible with in-memory resident rustc

eddyb (Dec 13 2019 at 15:33, on Zulip):

@mw I think @matklad is to some extent otherwise his suggestion can't work

mw (Dec 13 2019 at 15:34, on Zulip):

just the approach of "the compiler generates a per-crate "database" for consumption elsewhere"

nikomatsakis (Dec 13 2019 at 15:34, on Zulip):

I don't think that's true; the idea would be that save-analysis is a temporary crutch to be used while we improve rust-analyzer and make progress on library-ification, though I think it's a crutch that might last a year or two.

eddyb (Dec 13 2019 at 15:34, on Zulip):

if you want something temporary, forward LSP to RLS, which already exists and most people have installed

eddyb (Dec 13 2019 at 15:34, on Zulip):

if you want something permanent, an offline database is not it IMO

mw (Dec 13 2019 at 15:35, on Zulip):

if you want something permanent, an offline database is not it IMO

why not? that could be cached for crates.io stuff for example

eddyb (Dec 13 2019 at 15:35, on Zulip):

oh god please let's not have something like that wasm proc macro thing happen again

pnkfelix (Dec 13 2019 at 15:35, on Zulip):

ah: @eddyb , are you saying that instead of filling in the gaps in rust-analyzer with save-analysis in short-term, we should instead fill in gaps in rust-analyzer by having it call out to RLS ?

Igor Matuszewski (Dec 13 2019 at 15:35, on Zulip):

@eddyb yes and no, we'd have to initialize RLS on the side in order to 'forward LSP'

eddyb (Dec 13 2019 at 15:35, on Zulip):

if we want to cache something, it should be the incremental data

Igor Matuszewski (Dec 13 2019 at 15:35, on Zulip):

so this is the same approach as 'run both things at the same time' (which we did consider)

eddyb (Dec 13 2019 at 15:36, on Zulip):

@pnkfelix pretty much yeah

Igor Matuszewski (Dec 13 2019 at 15:36, on Zulip):

I'm not sure this is a right approach

Igor Matuszewski (Dec 13 2019 at 15:36, on Zulip):

rather, I wouldn't forward LSP

Igor Matuszewski (Dec 13 2019 at 15:36, on Zulip):

but use 'rls cli'

nikomatsakis (Dec 13 2019 at 15:37, on Zulip):

So, what @matklad specifically was proposing, I think, is to have some command-line tools that could execute and respond to queries. The fact that they used "save-analysis" to communicate was, I think, going to be an implementation detail of these tools. Is that right, @matklad?

Igor Matuszewski (Dec 13 2019 at 15:37, on Zulip):

to forward LSP we'd have to maintain state on both sides, we'd duplicate memory usage only to feed find usages data

eddyb (Dec 13 2019 at 15:37, on Zulip):

tbh I'm considering the fact that I should probably not be in this meeting, since I have strong feelings about this, feeling like a Cassandra and all (I had a demo before 1.0 which used rustc as a library to do things like show types of expressions on hover, and RLS still can't do anything like that, despite having had mountains of efforts just sank into it)

Igor Matuszewski (Dec 13 2019 at 15:37, on Zulip):

so in terms of engineering it makes more sense to temporarily spawn RLS to answer one-off queries (find all refs isn't that common)

Florian Diebold (Dec 13 2019 at 15:37, on Zulip):

with save-analysis, rust-analyzer could probably use that to supplement its analysis for other requests, instead of just blindly forwarding the LSP requests that RLS supports

Florian Diebold (Dec 13 2019 at 15:38, on Zulip):

(current example, implementing call hierarchy)

Igor Matuszewski (Dec 13 2019 at 15:38, on Zulip):

this addresses the main con of rust-analyzer, namely lack of rustc parity

Igor Matuszewski (Dec 13 2019 at 15:38, on Zulip):

with save-analysis we'd know what compiler knows

pnkfelix (Dec 13 2019 at 15:38, on Zulip):

@eddyb I'm not sure whether you want us to tease apart that last message

Igor Matuszewski (Dec 13 2019 at 15:38, on Zulip):

(which could only be generated for those queries like find all refs or call hierarchy)

matklad (Dec 13 2019 at 15:39, on Zulip):

@nikomatsakis sort-of. But I'd consider the tools themselves implementation detail, at least for the time being

nikomatsakis (Dec 13 2019 at 15:39, on Zulip):

Implementation details of rust-analyzer, in other words

nikomatsakis (Dec 13 2019 at 15:39, on Zulip):

Yes, I agree

nikomatsakis (Dec 13 2019 at 15:39, on Zulip):

There are many levels of "impl details"

pnkfelix (Dec 13 2019 at 15:40, on Zulip):

@eddyb it seems clear to me that your knowledge of rustc's architecture and potential future seems like a valuable part of the conversation here. But I also think that it can be hard to reach common ground for communication

eddyb (Dec 13 2019 at 15:40, on Zulip):

this is not about communication. this is about the history of the RLS and decisions taken in the past. I can explain more in PM if people are curious

pnkfelix (Dec 13 2019 at 15:41, on Zulip):

I'm talking about failure to communicate here and now in the meeting, @eddyb

pnkfelix (Dec 13 2019 at 15:42, on Zulip):

sometimes I think I'm the only one who doesn't understand what youi

centril (Dec 13 2019 at 15:42, on Zulip):

@eddyb perhaps it would be best to lay this out as a text

pnkfelix (Dec 13 2019 at 15:42, on Zulip):

you are sayingh

mw (Dec 13 2019 at 15:42, on Zulip):

so one thing that makes me feel positive about an offline database approach is that rustc is probably going to stay optimized for one-off invocations (as opposed to becoming a long-running process)

pnkfelix (Dec 13 2019 at 15:42, on Zulip):

but I'm willing to say i don

pnkfelix (Dec 13 2019 at 15:42, on Zulip):

t understand

eddyb (Dec 13 2019 at 15:42, on Zulip):

so one thing that makes me feel positive about an offline database approach is that rustc is probably going to stay optimized for one-off invocations (as opposed to becoming a long-running process)

this is different from what I thought we've all been agreeing for years

eddyb (Dec 13 2019 at 15:42, on Zulip):

isn't this what end-to-end querifyication was for?

Igor Matuszewski (Dec 13 2019 at 15:43, on Zulip):

I don't think it's a discussion either-or

eddyb (Dec 13 2019 at 15:43, on Zulip):

@pnkfelix okay I'll admit I'm confused by some of @mw's statements

Igor Matuszewski (Dec 13 2019 at 15:43, on Zulip):

I think offline database has very much its uses

Igor Matuszewski (Dec 13 2019 at 15:43, on Zulip):

but we shouldn't just use it as main driver for IDE/compilation purposes

Igor Matuszewski (Dec 13 2019 at 15:43, on Zulip):

and I don't think anyone here wants to keep save-analysis as-is

centril (Dec 13 2019 at 15:43, on Zulip):

I think offline database has very much its uses

can you summarize?

Igor Matuszewski (Dec 13 2019 at 15:44, on Zulip):

I love the GitHub pre-computed navigation feature

Igor Matuszewski (Dec 13 2019 at 15:44, on Zulip):

which could be powered using the same approach

matklad (Dec 13 2019 at 15:44, on Zulip):

I feel both offline database, and long-running rustc are in the far future, and it probably doesn't make a lot of sense to discuss thouse in details

nikomatsakis (Dec 13 2019 at 15:44, on Zulip):

I think what @mw is saying (which I'm sympathetic to) is that there may well be a need to have two "drivers", one for "batch compilation" and one for "IDE". I imagine they would both be based around the same set of core libaries and share 99% of the same code, but might make some choices differently (e.g., batch compilation might be based on arenas and use the disk to cache incremental state, IDEs would likely not)

mw (Dec 13 2019 at 15:44, on Zulip):

to clarify: I thought that the RLS + rust-analyzer approach included:

nikomatsakis (Dec 13 2019 at 15:45, on Zulip):

But I also agree with @matklad -- one thing that I feel maybe got lost is that I think we're all aiming to build a "responsive" IDE design, much like the one that @eddyb has been advocating for (which I found quite inspiring), and the question is more charting out the course to get there

mw (Dec 13 2019 at 15:45, on Zulip):

yes, what Niko said

mw (Dec 13 2019 at 15:45, on Zulip):

I don't consider any of this as set in stone

nikomatsakis (Dec 13 2019 at 15:46, on Zulip):

the only caveat I would give @mw is that I think it would be amazing if we can unify rustc+rust-analyzer, but I'm not sure it makes sense; luckily in the short term the question feels moot, since we have to kind of do the same work either way.

centril (Dec 13 2019 at 15:46, on Zulip):

time check, 14 min left

nikomatsakis (Dec 13 2019 at 15:46, on Zulip):

but I see us doing a lot of optimizations (e.g., streaming the dep-graph to disk) that seem to be useful, but don't make sense for IDEs, which is what gives me a bit of pause to think that the we may want to have two kinds of "implementations" of the same query system, with different trade-offs

centril (Dec 13 2019 at 15:46, on Zulip):

We haven't touched on:

Library-ification priorities

at all yet.

mw (Dec 13 2019 at 15:47, on Zulip):

moving end-to-end queries (or at least moving queries more towards the beginning of the pipeline) is useful for a batch compiler too

mw (Dec 13 2019 at 15:47, on Zulip):

just as a side note

nikomatsakis (Dec 13 2019 at 15:47, on Zulip):

Thanks @centril for the time check :)

nikomatsakis (Dec 13 2019 at 15:48, on Zulip):

I'm not really sure where we stand at the moment

centril (Dec 13 2019 at 15:48, on Zulip):

two kinds of "implementations" of the same query system, with different trade-offs

@nikomatsakis this does have the drawback of fragility? (e.g. wrt. the test suite and whatnot?)

centril (Dec 13 2019 at 15:48, on Zulip):

I think we need several meetings :P

Igor Matuszewski (Dec 13 2019 at 15:48, on Zulip):

these would differ in optimizations but the end 'result' would be the same from test pov, no?

Igor Matuszewski (Dec 13 2019 at 15:49, on Zulip):

(such as streaming I/O or caches)

nikomatsakis (Dec 13 2019 at 15:49, on Zulip):

OK, well, we're at 10 minutes left, why don't we discuss follow-up meetings?

centril (Dec 13 2019 at 15:49, on Zulip):

@Igor Matuszewski assuming opts don't change end result

nikomatsakis (Dec 13 2019 at 15:49, on Zulip):

One thing I was thinking is that I'd like to try to have a meeting to talk about the work we're pursuing on chalk and type-system "library-ification"

nikomatsakis (Dec 13 2019 at 15:49, on Zulip):

I imagine such a discussion migfht help to make some of this debate more concrete

mw (Dec 13 2019 at 15:49, on Zulip):

so, @eddyb, my comments from before were meant to explore a design branch that takes what Niko listed as given. they were not meant as, I consider other approaches, branching off earlier, unfeasible or something.

Igor Matuszewski (Dec 13 2019 at 15:50, on Zulip):

@centril oh you mean like "fuel/resource"-limited queries?

Igor Matuszewski (Dec 13 2019 at 15:50, on Zulip):

I'm sure chalk and macro would need separate meetings in terms of librarification

eddyb (Dec 13 2019 at 15:51, on Zulip):

so, eddyb, my comments from before were meant to explore a design branch that takes what Niko listed as given. they were not meant as, I consider other approaches, branching off earlier, unfeasible or something.

so did you mean that rustc will for now remain a batch compiler? I read it as "for the indefinite future"

nikomatsakis (Dec 13 2019 at 15:51, on Zulip):

So yeah that's kind of one set of meetings: on specific library-ification topics

centril (Dec 13 2019 at 15:51, on Zulip):

@Igor Matuszewski more like basic "the choices made for the infrastructure are not dependent upon implicitly for correctness purposes of tests"

mw (Dec 13 2019 at 15:52, on Zulip):

so, eddyb, my comments from before were meant to explore a design branch that takes what Niko listed as given. they were not meant as, I consider other approaches, branching off earlier, unfeasible or something.

so did you mean that rustc will for now remain a batch compiler? I read it as "for the indefinite future"

I meant, in the scenario that Niko described, it would remain a batch compiler

centril (Dec 13 2019 at 15:52, on Zulip):

For the parser stuff, me, @Esteban Küber, @Vadim Petrochenkov, @matklad, other people who are interested, should probably discuss

Igor Matuszewski (Dec 13 2019 at 15:52, on Zulip):

one thing that I think we're also not touching upon would be multi-crate compilation

centril (Dec 13 2019 at 15:52, on Zulip):

I'm in the process of doing large scale refactorings of the parser

Igor Matuszewski (Dec 13 2019 at 15:52, on Zulip):

which may or may not impact some decisions here

eddyb (Dec 13 2019 at 15:52, on Zulip):

huh why can't I find niko's message

Esteban Küber (Dec 13 2019 at 15:52, on Zulip):

I'll be happy as long as we have "parity" in observed behavior

mw (Dec 13 2019 at 15:52, on Zulip):

@eddyb https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/design.20meeting.202019-12-13/near/183371074

pnkfelix (Dec 13 2019 at 15:52, on Zulip):

niko's message: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/design.20meeting.202019-12-13/near/183371074

pnkfelix (Dec 13 2019 at 15:53, on Zulip):

oh yeah

eddyb (Dec 13 2019 at 15:53, on Zulip):

oh I could've sworn it had bullet points or numbers in it, whoops

nikomatsakis (Dec 13 2019 at 15:53, on Zulip):

I think what mw is saying (which I'm sympathetic to) is that there may well be a need to have two "drivers", one for "batch compilation" and one for "IDE". I imagine they would both be based around the same set of core libaries and share 99% of the same code, but might make some choices differently (e.g., batch compilation might be based on arenas and use the disk to cache incremental state, IDEs would likely not)

I think this is the message @mw was referring to, @eddyb

centril (Dec 13 2019 at 15:53, on Zulip):

I'll be happy as long as we have "parity" in observed behavior

centril (Dec 13 2019 at 15:53, on Zulip):

e.g. not having a formal grammar is sub-optimal if we want to re-architecture things

nikomatsakis (Dec 13 2019 at 15:54, on Zulip):

@centril I think what would make sense is to start with some private discussion to hammer out rough design and then a larger meeting to help fill the rest of us in

eddyb (Dec 13 2019 at 15:54, on Zulip):

I can see having multiple "policies" for how to approach compilation and how much RAM vs disk space to use

centril (Dec 13 2019 at 15:54, on Zulip):

Sure; I believe we have a thread somewhere in this stream about the parser

Esteban Küber (Dec 13 2019 at 15:54, on Zulip):

What we'll ideally have is someone familiar with both ra and libsyntax :)

eddyb (Dec 13 2019 at 15:54, on Zulip):

in fact I think I suggested a long time ago that RLS could interleave compiling the crate and querying

centril (Dec 13 2019 at 15:55, on Zulip):

What we'll ideally have is someone familiar with both ra and libsyntax :slight_smile:

not sure who that is

matklad (Dec 13 2019 at 15:55, on Zulip):

@Esteban Küber that might be me (only me :sob: ?)

eddyb (Dec 13 2019 at 15:55, on Zulip):

because you can more or less "just" stop from e.g. type-checking all the functions (between two functions, that is) and run some query

nikomatsakis (Dec 13 2019 at 15:56, on Zulip):

I can see having multiple "policies" for how to approach compilation and how much RAM vs disk space to use

yes, I think in my ideal world we probably land here; it may take us some time to get there

eddyb (Dec 13 2019 at 15:56, on Zulip):

so what you're talking about doesn't seem implausible or undesirable for me

eddyb (Dec 13 2019 at 15:56, on Zulip):

I just don't think there is a need for persisting something other than incremental caches

nikomatsakis (Dec 13 2019 at 15:57, on Zulip):

So let's put this another way

centril (Dec 13 2019 at 15:57, on Zulip):

@matklad I can't say I got a clear impression of what "event" based means; If you could provide an example & also point me towards whatever RA uses that would be good. And I need to read up on whatever Swift is doing

nikomatsakis (Dec 13 2019 at 15:58, on Zulip):

Actually, let's not put this another way, because we have 2 minutes left here.

Esteban Küber (Dec 13 2019 at 15:58, on Zulip):

Esteban Küber that might be me (only me :sob: ?)

we better do something to change that

nikomatsakis (Dec 13 2019 at 15:58, on Zulip):

But I think I would like to break out a separate topic to chat with @eddyb, @matklad, and @Igor Matuszewski (and others who wish) about the best way to have rust-analyzer leverage rustc for precise queries

matklad (Dec 13 2019 at 15:58, on Zulip):

@centril could you organize a parser meeting? :heart: ?

nikomatsakis (Dec 13 2019 at 15:58, on Zulip):

Or put another way, that's clearly a follow-up item

nikomatsakis (Dec 13 2019 at 15:59, on Zulip):

And it seems to have been the heart of what we discussed here, though perhaps we went round about?

Igor Matuszewski (Dec 13 2019 at 16:00, on Zulip):

I think we went somewhat off-topic on that to be honest

centril (Dec 13 2019 at 16:00, on Zulip):

@matklad I think it would make sense to start with collecting some resources and reading material before organizing a meeting

Igor Matuszewski (Dec 13 2019 at 16:00, on Zulip):

it'd be good to come back to that

mw (Dec 13 2019 at 16:00, on Zulip):

so this sounds like the RLS would slowly merge into rustc over time, and libraries would be extracted to be used by both rustc and RA

centril (Dec 13 2019 at 16:01, on Zulip):

Otherwise it's hard to make informed conversations in the meeting

eddyb (Dec 13 2019 at 16:01, on Zulip):

I could see half of RLS remaining external to drive Cargo

nikomatsakis (Dec 13 2019 at 16:01, on Zulip):

Well, I'm not sure about the RLS part -- at least, our hope was that the RLS gets deprecated fairly quickly.

eddyb (Dec 13 2019 at 16:01, on Zulip):

or merging into Cargo somehow?

nikomatsakis (Dec 13 2019 at 16:01, on Zulip):

But replaced I guess with some other more targeted bits of tooling

centril (Dec 13 2019 at 16:01, on Zulip):

@matklad since you seem to have ideas about what direction to go in it would be good to elaborate on that + provide some resources

nikomatsakis (Dec 13 2019 at 16:01, on Zulip):

That are used for the time being (but not externally "known")

Igor Matuszewski (Dec 13 2019 at 16:01, on Zulip):

so I think it'd make sense to make RA the primary investment in the IDE space

eddyb (Dec 13 2019 at 16:01, on Zulip):

I guess cargo rustc -- --lsp localhost:35487 would "just" work heh

Igor Matuszewski (Dec 13 2019 at 16:02, on Zulip):

which needs parity with RLS as of now

Igor Matuszewski (Dec 13 2019 at 16:02, on Zulip):

we can discuss how to do that separately (as an implementation detail)

nikomatsakis (Dec 13 2019 at 16:02, on Zulip):

OK, well, I have to go do some errands, it seems clear we'll need to do some follow-ups, but I'm debating what shape they take

nikomatsakis (Dec 13 2019 at 16:03, on Zulip):

this is what I noted in the hackmd at the end:

centril (Dec 13 2019 at 16:03, on Zulip):

Maybe some elaborations from various parties in the form of "opinion pieces" would be helpful

Igor Matuszewski (Dec 13 2019 at 16:03, on Zulip):

yeah

centril (Dec 13 2019 at 16:03, on Zulip):

Like @eddyb can give their take on things in a write-up

Igor Matuszewski (Dec 13 2019 at 16:03, on Zulip):

I'll write some notes and share them here later on

pnkfelix (Dec 13 2019 at 16:04, on Zulip):

also: please distinguish short-term vs long-term strategies

eddyb (Dec 13 2019 at 16:04, on Zulip):

Like eddyb can give their take on things in a write-up

every time someone says that, I wonder if they underestimate the effort/time involved :P

matklad (Dec 13 2019 at 16:05, on Zulip):

re shape, I must say that the present meeting was too high energy/velocity for me.

nikomatsakis (Dec 13 2019 at 16:05, on Zulip):

Yeah, it was very hard to keep up.

eddyb (Dec 13 2019 at 16:05, on Zulip):

there are a lot of things where if I can dedicate time to it I would rather spend time on implementing it

eddyb (Dec 13 2019 at 16:05, on Zulip):

@matklad same, and I didn't help, sorry about that :/

pnkfelix (Dec 13 2019 at 16:05, on Zulip):

but this is the problem @eddyb : you implement it but then noone else knows what it is?

nikomatsakis (Dec 13 2019 at 16:06, on Zulip):

OK, I have to step out. Thanks all :heart:

matklad (Dec 13 2019 at 16:06, on Zulip):

Thanks for leading the meeting @nikomatsakis !

eddyb (Dec 13 2019 at 16:06, on Zulip):

@pnkfelix that happens with design documents I've written too :P (but really, I wish I had that problem - of needing to document an implementation - more often)

qmx (Dec 13 2019 at 16:06, on Zulip):

@matklad I want to watch the parser discussion, where is this going to happen?

centril (Dec 13 2019 at 16:07, on Zulip):

I don't think we're ready for that discussion yet

qmx (Dec 13 2019 at 16:07, on Zulip):

I mean, even the research material pre-meeting

eddyb (Dec 13 2019 at 16:08, on Zulip):

oh yeah wg-grammar should probably make a working parser testing rig too, shouldn't it?

centril (Dec 13 2019 at 16:09, on Zulip):

@eddyb well we're very much no-where near that

centril (Dec 13 2019 at 16:09, on Zulip):

like we don't have a way to disambiguate grammars so... ^^

eddyb (Dec 13 2019 at 16:09, on Zulip):

idk why people keep thinking that matters

eddyb (Dec 13 2019 at 16:10, on Zulip):

first off, we could be comparing syn and libsyntax (and ra-syntax? not sure what that looks like yet), and we could've all along

centril (Dec 13 2019 at 16:10, on Zulip):

@eddyb ah you mean just comparing different impls

eddyb (Dec 13 2019 at 16:10, on Zulip):

secondly, once we have that we can just check that what everyone else agrees on, happens to be one of the possible parse trees

centril (Dec 13 2019 at 16:10, on Zulip):

not against a formal grammar

eddyb (Dec 13 2019 at 16:11, on Zulip):

we've been talking about this for a year, I'm starting to suspect we don't have the free time between us combined to move forward :P (and in the meanwhile we keep forgetting what we can/should do)

eddyb (Dec 13 2019 at 16:11, on Zulip):

but also this is not #wg-grammar, sorry for offtopic

matklad (Dec 13 2019 at 16:12, on Zulip):

@qmx , @eddyb @centril @Esteban Küber I've opened https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/Parser for discussing parser

centril (Dec 13 2019 at 16:13, on Zulip):

@matklad I would like to have that in #t-compiler; not subscribed to wg-rls-2.0

centril (Dec 13 2019 at 16:13, on Zulip):

(and we already have a topic in that)

matklad (Dec 13 2019 at 16:14, on Zulip):

Could you give a link?

centril (Dec 13 2019 at 16:15, on Zulip):

https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/frontend.20library-ification

Last update: May 26 2020 at 11:10UTC