Stream: t-compiler

Topic: RLS sync


nikomatsakis (Apr 12 2019 at 13:02, on Zulip):

Hey @Igor Matuszewski , @pnkfelix

nikomatsakis (Apr 12 2019 at 13:03, on Zulip):

Actually @Igor Matuszewski maybe you can lay out some of your thoughts / questions to start?

Igor Matuszewski (Apr 12 2019 at 13:04, on Zulip):

Yeah, will do

nikomatsakis (Apr 12 2019 at 13:04, on Zulip):

(Also, should we cc matklad (not cc'ing yet so as not to bother them if not))

Igor Matuszewski (Apr 12 2019 at 13:04, on Zulip):

it's mostly about RLS 1.0 but we could

nikomatsakis (Apr 12 2019 at 13:04, on Zulip):

let's wait until it makes sense :)

Igor Matuszewski (Apr 12 2019 at 13:05, on Zulip):

So I'm happy with what's been done so far - most of the work was focused on paying the tech debt, fixing tests, doing issue triage, rewriting the extension

Igor Matuszewski (Apr 12 2019 at 13:05, on Zulip):

and didn't yet focus on implementing much of the new features (except some minor things like rustfmt returning only modified lines for reformatting and so on)

Igor Matuszewski (Apr 12 2019 at 13:06, on Zulip):

I did some profiling and came to the conclusion that a very significant portion of our time we spend on (de)serializing save-analysis data and synchronizing multiple rustc instances and cargo in one process (whereas we could parallelize that somewhat easily)

nikomatsakis (Apr 12 2019 at 13:06, on Zulip):

is there a place to look to get a sense of what the "new features" are etc?

nikomatsakis (Apr 12 2019 at 13:07, on Zulip):

(also, should we .. make a wg-rls-1.0? should we maybe consider renaming wg-rls-2.0 to something a bit different? I'm not convinced we have the right naming here -- think on it)

Igor Matuszewski (Apr 12 2019 at 13:07, on Zulip):

yeah, one way to look for new features would be to support more LSP requests, like hierarchical symbol outline (which itself needs a bit of work in rls-analysis)

pnkfelix (Apr 12 2019 at 13:07, on Zulip):

Do we know if the save-analysis serialization overhead is due to the sheer amount of data being serialized?

pnkfelix (Apr 12 2019 at 13:07, on Zulip):

or is the format itself compact, and the overhead arises from the traversal of structures to build the analysis data ?

Igor Matuszewski (Apr 12 2019 at 13:07, on Zulip):

I'd think so, we now easily serialize 5-30 megs of JSON back and forth

Igor Matuszewski (Apr 12 2019 at 13:08, on Zulip):

I imagine binary format would help here

nikomatsakis (Apr 12 2019 at 13:08, on Zulip):

does serde have a binary encoder/decoder?

Igor Matuszewski (Apr 12 2019 at 13:08, on Zulip):

and possibly investigate using serde in the compiler itself instead of rustc_serialize

Igor Matuszewski (Apr 12 2019 at 13:08, on Zulip):

bincode supports serde, yeah

Igor Matuszewski (Apr 12 2019 at 13:09, on Zulip):

(re wg-rls-1.0 probably, it'd be good to clarify a bit what's the connection between 1.0 and 2.0)

Igor Matuszewski (Apr 12 2019 at 13:09, on Zulip):

so there's that, another area which would be good to explore is to support more project layouts than Cargo to provide a bit more fault-tolerancy there

nikomatsakis (Apr 12 2019 at 13:10, on Zulip):

and possibly investigate using serde in the compiler itself instead of rustc_serialize

oh right I forgot we're not using serde yet

Igor Matuszewski (Apr 12 2019 at 13:11, on Zulip):

one (long-term, I bet!) thing worth exploring would be to work towards daemonizing rustc

nikomatsakis (Apr 12 2019 at 13:11, on Zulip):

(re wg-rls-1.0 probably, it'd be good to clarify a bit what's the connection between 1.0 and 2.0)

yes, I'm thinking about this some. I feel we should really try not to have "two teams", but think in terms of a good experience for users and building next gen tooling.

nikomatsakis (Apr 12 2019 at 13:12, on Zulip):

one (long-term, I bet!) thing worth exploring would be to work towards daemonizing rustc

hmm. I mean the rust-analyzer approach kind of already does this, right?

Igor Matuszewski (Apr 12 2019 at 13:12, on Zulip):

on every change (after some dynamic timeout) we spin up a rustc to emit the save-analysis and update the index

nikomatsakis (Apr 12 2019 at 13:12, on Zulip):

are we using incremental yet?

Igor Matuszewski (Apr 12 2019 at 13:12, on Zulip):

which means reading/saving the depgraph and doing all the work, but that's effectively end-to-end queries

Igor Matuszewski (Apr 12 2019 at 13:12, on Zulip):

are we using incremental yet

yes

nikomatsakis (Apr 12 2019 at 13:12, on Zulip):

ok. do we know if we are very effective at it?

nikomatsakis (Apr 12 2019 at 13:12, on Zulip):

by which I mean:

nikomatsakis (Apr 12 2019 at 13:13, on Zulip):

I don't know how the save-analysis code is setup, but if it e.g. emits one huge blob for the crate, I suspect it is re-doing a lot of work

Igor Matuszewski (Apr 12 2019 at 13:13, on Zulip):

didn't measure what is the cache hit ratio

nikomatsakis (Apr 12 2019 at 13:13, on Zulip):

though perhaps that's not the significant part

Igor Matuszewski (Apr 12 2019 at 13:13, on Zulip):

I don't know how the save-analysis code is setup, but if it e.g. emits one huge blob for the crate, I suspect it is re-doing a lot of work

yeah :( that also means more work to (de)serialize data again and so forth

nikomatsakis (Apr 12 2019 at 13:13, on Zulip):

(i.e., maybe it's stuff like parsing, HIR lowering, that comes earlier in the pipeline)

nikomatsakis (Apr 12 2019 at 13:13, on Zulip):

this is an area where having self-profile would be very useful

Igor Matuszewski (Apr 12 2019 at 13:14, on Zulip):

it'd be great if we could maybe redo parsing/HIR lowering but not emitting all of the data for a given crate target with each change

nikomatsakis (Apr 12 2019 at 13:14, on Zulip):

so if you were going to rank the "major problems" that remain in RLS 1.0 (i.e., without trying to dramatically change its infrastructure), would perf be the main one?

nikomatsakis (Apr 12 2019 at 13:15, on Zulip):

it'd be great if we could maybe redo parsing/HIR lowering but not emitting all of the data for a given crate target with each change

I feel like this should be possible, but it'd require changing save-analysis presumably to be grouped differently (e.g., something like codegen-units)

nikomatsakis (Apr 12 2019 at 13:15, on Zulip):

alternatively, or as a mid point, if we structure save-analysis code right,

Igor Matuszewski (Apr 12 2019 at 13:15, on Zulip):

I think so - I believe we can cut corners without changing the arch that much while still getting the greatly improved latency

nikomatsakis (Apr 12 2019 at 13:15, on Zulip):

it should at least not have to redo the work to generate the data that is being serialized

nikomatsakis (Apr 12 2019 at 13:15, on Zulip):

I think so - I believe we can cut corners without changing the arch that much while still getting the greatly improved latency

this seems like it might make a huge difference in user perception

Igor Matuszewski (Apr 12 2019 at 13:16, on Zulip):

agreed

pnkfelix (Apr 12 2019 at 13:16, on Zulip):

I'm trying to make sure I understand what is being proposed

Igor Matuszewski (Apr 12 2019 at 13:17, on Zulip):

first of all, I think it'd be good to move the compilation pipeline out of rls process itself

nikomatsakis (Apr 12 2019 at 13:18, on Zulip):

can you explain what you mean by the "compilation pipeline"

Igor Matuszewski (Apr 12 2019 at 13:18, on Zulip):

which means increased overhead of passing the data over IPC but I believe we'd get more speedup with parallelization alone

pnkfelix (Apr 12 2019 at 13:18, on Zulip):

and instead the compilation pipeline would live in a rustc daemon?

pnkfelix (Apr 12 2019 at 13:18, on Zulip):

(i guess answer @nikomatsakis 's Q first. :smile: if they don't know then I surely don't.)

Igor Matuszewski (Apr 12 2019 at 13:18, on Zulip):

so right now we first shell out to Cargo to "understand" the project - then we reuse the 'crate dep graph' to orchestrate the compilation ourselves (but inside the one rls process)

Igor Matuszewski (Apr 12 2019 at 13:19, on Zulip):

we do this by matching which file was dirty (we collect information which file is associated with each crate target in the graph)

nikomatsakis (Apr 12 2019 at 13:19, on Zulip):

(ps, remind me @Igor Matuszewski, in the runup to the All Hands, did you make a video or writeup of RLS architecture? did it cover this?)

Igor Matuszewski (Apr 12 2019 at 13:19, on Zulip):

(yep! it's here - https://github.com/rust-lang/rls/blob/master/architecture.md)

Igor Matuszewski (Apr 12 2019 at 13:20, on Zulip):

so we queue the 'dirty' crate targets and then linearly compile them in-process

Igor Matuszewski (Apr 12 2019 at 13:20, on Zulip):

the 'optimization' here is that for the leaf crates we pass the resulting save-analysis in-memory, which is good

Igor Matuszewski (Apr 12 2019 at 13:21, on Zulip):

(for deps we still emit the save-analysis data in blobs)

Igor Matuszewski (Apr 12 2019 at 13:21, on Zulip):

however because we compile it once per process, we can't manage to quickly compile anything that's more complex than a single crate target

Igor Matuszewski (Apr 12 2019 at 13:21, on Zulip):

because we can't parallelize this

Igor Matuszewski (Apr 12 2019 at 13:22, on Zulip):

and unfortunately each instance reads/saves dep graph alone and redoes all of the analysis, which we replace as a whole in the rls-analysis (our save-analysis indexer)

nikomatsakis (Apr 12 2019 at 13:24, on Zulip):

the 'optimization' here is that for the leaf crates we pass the resulting save-analysis in-memory, which is good

this is something we already do?

nikomatsakis (Apr 12 2019 at 13:25, on Zulip):

however because we compile it once per process, we can't manage to quickly compile anything that's more complex than a single crate target

does this apply to deps too? just things within the workspace?

nikomatsakis (Apr 12 2019 at 13:25, on Zulip):

If you have only one crate in the workspace, presumably in the "steady state" we are typically compiling only some leaf crate that changed?

nikomatsakis (Apr 12 2019 at 13:25, on Zulip):

But if you have multiple crates, I guess you are compiling the crate that changed plus anything dependent on it?

Igor Matuszewski (Apr 12 2019 at 13:31, on Zulip):

so the current setup is we run cargo, let the inner dep nodes output the JSON blob (and we also don't intercept build.rs runs)

Igor Matuszewski (Apr 12 2019 at 13:31, on Zulip):

but we intercept the leaf notes, for which we pass the save-analysis data in-memory

Igor Matuszewski (Apr 12 2019 at 13:32, on Zulip):

in the cargo run, we use the Executor trait there, where Cargo is responsible for coordinating the jobs, however on its own it spawns the command whereas we execute the compilation in-process

Igor Matuszewski (Apr 12 2019 at 13:33, on Zulip):

and by leaf node I mean the 'primary' node inside a given workplace (incl. path-based dependencies) and not deps

Igor Matuszewski (Apr 12 2019 at 13:33, on Zulip):

If you have only one crate in the workspace, presumably in the "steady state" we are typically compiling only some leaf crate that changed?

yes, exactly

pnkfelix (Apr 12 2019 at 13:33, on Zulip):

it seems like some of the revision of coordination between cargo+rustc that the WG-pipelining group wants might overlap with desirata objectives here?

pnkfelix (Apr 12 2019 at 13:34, on Zulip):

at least, it sounds like RLS is jumping through hoops in order to take over for some of the rustc invocations that cargo would make

pnkfelix (Apr 12 2019 at 13:34, on Zulip):

(sorry, just thinking out loud.)

nikomatsakis (Apr 12 2019 at 13:35, on Zulip):

so the current setup is we run cargo, let the inner dep nodes output the JSON blob (and we also don't intercept build.rs runs)

what are "dep nodes" here?

nikomatsakis (Apr 12 2019 at 13:35, on Zulip):

is that referring to something in cargo?

nikomatsakis (Apr 12 2019 at 13:35, on Zulip):

i.e., each node represents a crate?

nikomatsakis (Apr 12 2019 at 13:35, on Zulip):

in the cargo run, we use the Executor trait there, where Cargo is responsible for coordinating the jobs, however on its own it spawns the command whereas we execute the compilation in-process

I presume this is also some kind of hook that cargo provides -- the ability to control execution so that it runs in-process?

nikomatsakis (Apr 12 2019 at 13:36, on Zulip):

if I am understanding this correctly, @Igor Matuszewski, then the parallelism you are hoping to exploit would be between crates? (i.e., what cargo normally gets you)

Igor Matuszewski (Apr 12 2019 at 13:36, on Zulip):

if I am understanding this correctly, Igor Matuszewski, then the parallelism you are hoping to exploit would be between crates? (i.e., what cargo normally gets you)

yup, which might be prominent in bigger workspaces where we don't have an obvious linear dep chain between primary packages but some of those may be spread out

nikomatsakis (Apr 12 2019 at 13:37, on Zulip):

yes, my experience with the RLS has often been that

Igor Matuszewski (Apr 12 2019 at 13:37, on Zulip):

so the current setup is we run cargo, let the inner dep nodes output the JSON blob (and we also don't intercept build.rs runs)

what are "dep nodes" here?

by dep node I mean a crate to be compiled (bin, lib, build.rs) in the crate dep graph

nikomatsakis (Apr 12 2019 at 13:37, on Zulip):

it works "ok" if you have a single crate

nikomatsakis (Apr 12 2019 at 13:37, on Zulip):

but workspaces .. not so well

nikomatsakis (Apr 12 2019 at 13:37, on Zulip):

and maybe this is why

nikomatsakis (Apr 12 2019 at 13:38, on Zulip):

it works "ok" if you have a single crate

to be clear, by this I mean "it could definitely still be snappier" :)

Igor Matuszewski (Apr 12 2019 at 13:38, on Zulip):

that definitely contributes - right now we use naive linear compilation queue

nikomatsakis (Apr 12 2019 at 13:38, on Zulip):

but I guess that the single crate scenario is not going to be helped here

pnkfelix (Apr 12 2019 at 13:39, on Zulip):

which case is the most important to address?

Igor Matuszewski (Apr 12 2019 at 13:39, on Zulip):

for a single crate scenario it'd be ideal to either be able to ask for a 'portion' of save-analysis to be incrementally updated (something that still needs to be implemented in the rls-analysis) or even better to use rustc as a deamon and ask for analysis directly

Igor Matuszewski (Apr 12 2019 at 13:40, on Zulip):

(but that's changing a lot of arch at this point)

pnkfelix (Apr 12 2019 at 13:40, on Zulip):

seems like if we cannot scale to workspaces, while single crate is "okay but not great", then we should prioritize workspaces

nikomatsakis (Apr 12 2019 at 13:40, on Zulip):

I think I agree with that

pnkfelix (Apr 12 2019 at 13:40, on Zulip):

because in the end, if we cannot scale, we fail, right?

pnkfelix (Apr 12 2019 at 13:40, on Zulip):

("fail" as in the tool won't be used)

Igor Matuszewski (Apr 12 2019 at 13:40, on Zulip):

yeah, I think we'll have more and more workspace users and the linear approach definitely holds us back here

nikomatsakis (Apr 12 2019 at 13:41, on Zulip):

(but that's changing a lot of arch at this point)

I'd like to drill into this a bit more -- I feel like it may be worth putting energy here, but it's probably correct that we should do this after we try to get parallelism for workspaces?

Igor Matuszewski (Apr 12 2019 at 13:41, on Zulip):

this is also visible with many Cargo integration tests, which are seen by Cargo as separate crates - these are also unfortunately compiled linearly

Igor Matuszewski (Apr 12 2019 at 13:42, on Zulip):

so the single crate scenario might be also somewhat affected if there are a lot of cfg(test) crates to be analyzed as well (right now, by default, we analyze every Cargo crate target)

nikomatsakis (Apr 12 2019 at 13:42, on Zulip):

ok so it sounds like we are saying that the rls 1.0 top priority right now is improving workspace performance, primarily through parallelization. maybe next priority would be improving response time for single crates, or maybe other feature work, that isn't decided yet.

Igor Matuszewski (Apr 12 2019 at 13:44, on Zulip):

Yup, sounds good

Igor Matuszewski (Apr 12 2019 at 13:44, on Zulip):

a functional feature that would be nice to tackle is to support more (non-Cargo) project layouts somehow to improve the tool usability

nikomatsakis (Apr 12 2019 at 13:44, on Zulip):

(I'd like to return a bit to the "RLS 1.0" and "RLS 2.0" questions, then, unless there is more we should discuss now.)

Igor Matuszewski (Apr 12 2019 at 13:44, on Zulip):

but that's not the top priority

nikomatsakis (Apr 12 2019 at 13:44, on Zulip):

a functional feature that would be nice to tackle is to support more (non-Cargo) project layouts somehow to improve the tool usability

yes, prime example perhaps being rustc itself?

nikomatsakis (Apr 12 2019 at 13:45, on Zulip):

but I've definitely heard from multiple production users that they cannot use RLS 1.0 for this reason -- otoh they are all using rust-analyzer

nikomatsakis (Apr 12 2019 at 13:45, on Zulip):

which, might be ok

Zoxc (Apr 12 2019 at 13:45, on Zulip):

Why doesn't RLS use rustc queries directly?

Igor Matuszewski (Apr 12 2019 at 13:45, on Zulip):

the support is surprisingly okay, but works better when initialized in the crate directly (e.g. librustc_save_analysis/)

pnkfelix (Apr 12 2019 at 13:45, on Zulip):

so the single crate scenario might be also somewhat affected if there are a lot of cfg(test) crates to be analyzed as well (right now, by default, we analyze every Cargo crate target)

regarding this point: I am curious about whether the user experience here is that, from the RLS point of view, all the tests are always part of the analysis?

nikomatsakis (Apr 12 2019 at 13:45, on Zulip):

Why doesn't RLS use rustc queries directly?

that was one of the things we were talking about as a possible refactoring

nikomatsakis (Apr 12 2019 at 13:46, on Zulip):

the support is surprisingly okay, but works better when initialized in the crate directly (e.g. librustc_save_analysis/)

support for what exactly? in rustc you mean?

pnkfelix (Apr 12 2019 at 13:46, on Zulip):

in other words, are they always paying for the overhead of analyzing and processing them? Is that overhead they want to pay for when they are trying to understand the non-test portin of their codebase?

Igor Matuszewski (Apr 12 2019 at 13:46, on Zulip):

Why doesn't RLS use rustc queries directly?

that's part of the end-to-end-queries as I understand it, which seems like the end-goal, especially if we could combine it with cargo/rustc pipelining effort

nikomatsakis (Apr 12 2019 at 13:46, on Zulip):

so one thing I did want to bring up @Igor Matuszewski, regarding prioritizing parallelization for workspaces, is whether it would impact using rustc queries directly

Igor Matuszewski (Apr 12 2019 at 13:46, on Zulip):

in other words, are they always paying for the overhead of analyzing and processing them? Is that overhead they want to pay for when they are trying to understand the non-test portin of their codebase?

not sure I get it, but if we always want them to be analyzed, then potentially on every lib change we need to reanalyze cfg tests because something might have changed upstream

Zoxc (Apr 12 2019 at 13:46, on Zulip):

Does RLS predate queries? =P

nikomatsakis (Apr 12 2019 at 13:47, on Zulip):

Yes

pnkfelix (Apr 12 2019 at 13:47, on Zulip):

if we always want them to be analyzed

I guess this "if" here is what I'm interested in

nikomatsakis (Apr 12 2019 at 13:48, on Zulip):

(I'd like to return a bit to the "RLS 1.0" and "RLS 2.0" questions, then, unless there is more we should discuss now.)

regarding this topic -- maybe we should schedule another time to chat, this time with @matklad and maybe @Florian Diebold -- and dig into this

pnkfelix (Apr 12 2019 at 13:48, on Zulip):

(I don't know enough about RLS usage to know how it interacts with cfg-flags)

Igor Matuszewski (Apr 12 2019 at 13:48, on Zulip):

if we always want them to be analyzed

I guess this "if" here is what I'm interested in

that's our default setting, we could disable it now but a lot of users are surprised because they're not

nikomatsakis (Apr 12 2019 at 13:48, on Zulip):

in particular @matklad was asking me about nomenclature already, I think we both agreed we don't love RLS 2.0

Igor Matuszewski (Apr 12 2019 at 13:48, on Zulip):

if we parallelize this here then the overhead of linear compilation of cfg(test) targets shouldn't matter that much

nikomatsakis (Apr 12 2019 at 13:48, on Zulip):

and I really feel like I want the RLS to be more "first class" in our team

nikomatsakis (Apr 12 2019 at 13:49, on Zulip):

and having a working group around it seems like an obvious way to do that

Igor Matuszewski (Apr 12 2019 at 13:49, on Zulip):

so one thing I did want to bring up Igor Matuszewski, regarding prioritizing parallelization for workspaces, is whether it would impact using rustc queries directly

it would unfortunately - by doing stuff out of process we technically lose the API access to rustc internals

Igor Matuszewski (Apr 12 2019 at 13:49, on Zulip):

but that might be sort of okay until the end-to-end queries story matures somehow?

nikomatsakis (Apr 12 2019 at 13:50, on Zulip):

I think a major question is

nikomatsakis (Apr 12 2019 at 13:50, on Zulip):

do we ever expect to transition RLS away from save-analysis to queries

Igor Matuszewski (Apr 12 2019 at 13:50, on Zulip):

maybe WG-RLS with "1.0" and "2.0" subgroups work? (although I'm afraid to overhierachize this, e.g. WG-RLS/2.0/chalk subsubgroup)

nikomatsakis (Apr 12 2019 at 13:51, on Zulip):

do we ever expect to transition RLS away from save-analysis to queries

like, I think at some point we need to pick a lane and stay in it

nikomatsakis (Apr 12 2019 at 13:51, on Zulip):

and maybe the answer is that we are going to push on the rust-analyzer approach

Igor Matuszewski (Apr 12 2019 at 13:51, on Zulip):

I think save-analysis as a format definitely has value

Igor Matuszewski (Apr 12 2019 at 13:51, on Zulip):

to be used in code indexers and such

nikomatsakis (Apr 12 2019 at 13:51, on Zulip):

it would unfortunately - by doing stuff out of process we technically lose the API access to rustc internals

I'm not entirely sure how much this matter btw

Igor Matuszewski (Apr 12 2019 at 13:51, on Zulip):

(e.g. Microsoft started LSIF standard to improve that, similar to LSP)

nikomatsakis (Apr 12 2019 at 13:51, on Zulip):

i.e., we could have the processes communicating using IPC or something

Igor Matuszewski (Apr 12 2019 at 13:52, on Zulip):

but the end-goal would be for IDEs to uses queries directly, I believe

nikomatsakis (Apr 12 2019 at 13:52, on Zulip):

I think the end-goal is for the IDEs to "use queries", but not necessarily in process

nikomatsakis (Apr 12 2019 at 13:52, on Zulip):

but definitely for rustc to be resident

nikomatsakis (Apr 12 2019 at 13:52, on Zulip):

i.e., daemon-like

Igor Matuszewski (Apr 12 2019 at 13:52, on Zulip):

yeah, that makes sense as well

nikomatsakis (Apr 12 2019 at 13:53, on Zulip):

I guess it seems like shifting to using rustc + save-analysis out of process is, in some sense, a step in the "wrong direction" architecturally, but still maybe the right move :)

nikomatsakis (Apr 12 2019 at 13:53, on Zulip):

I guess it depends how mcuh work it is

nikomatsakis (Apr 12 2019 at 13:53, on Zulip):

and whether it would make it much harder later on to try and adopt the daemon, query-like approach

nikomatsakis (Apr 12 2019 at 13:53, on Zulip):

I mean it seems clear we're going to want to compile crates in parallel regardless

nikomatsakis (Apr 12 2019 at 13:54, on Zulip):

or at least we'll want parallelization, probably also intra-crate

nikomatsakis (Apr 12 2019 at 13:54, on Zulip):

let me rephrase this

nikomatsakis (Apr 12 2019 at 13:54, on Zulip):

the crate boundary is not necessarily meaningful, but it's clear we want parallelization

nikomatsakis (Apr 12 2019 at 13:54, on Zulip):

(and I believe rust-analyzer, for example, has this today? it certainly has the ability to do it)

nikomatsakis (Apr 12 2019 at 13:55, on Zulip):

anyway, steering meeting starting soon, this was useful

Igor Matuszewski (Apr 12 2019 at 13:55, on Zulip):

one interesting case, if RLS is to still coordinate the editor side of things, is that we'd have to support some sort of a inter-process VFS

Igor Matuszewski (Apr 12 2019 at 13:55, on Zulip):

so that rustc can use the editor in-memory buffers instead of the files on disk

nikomatsakis (Apr 12 2019 at 13:55, on Zulip):

hmm

Igor Matuszewski (Apr 12 2019 at 13:56, on Zulip):

which I'm not sure how to solve it yet, it's something worth having in mind

pnkfelix (Apr 12 2019 at 13:56, on Zulip):

can we just steal rust-analyzer's?

pnkfelix (Apr 12 2019 at 13:56, on Zulip):

I suppose that's not addressing the real problem of adding the plumbing for it

pnkfelix (Apr 12 2019 at 13:56, on Zulip):

(oh, and that's not inter-process, is it...)

Igor Matuszewski (Apr 12 2019 at 13:57, on Zulip):

RLS has its VFS implementation - the real challenge is for spawned rustc instances to use it

matklad (Apr 12 2019 at 13:57, on Zulip):

it's not, but it should be possible to make it so

matklad (Apr 12 2019 at 13:57, on Zulip):

@Igor Matuszewski can't we just write a small FileLoader in the rustc which opens a unix-socket or something like that?

matklad (Apr 12 2019 at 13:58, on Zulip):

Or, rather, can we just have pass modified files as "input" to rustc?

matklad (Apr 12 2019 at 13:58, on Zulip):

like --memfile foo.rs= "fn main {}"...

matklad (Apr 12 2019 at 13:59, on Zulip):

Given that we should support only modified files, this doesn't seem like a perf bottlneck

Igor Matuszewski (Apr 12 2019 at 14:00, on Zulip):

@matklad we could, the real difficulty would be to have a robust cross-platform implementation

Igor Matuszewski (Apr 12 2019 at 14:00, on Zulip):

the args approach is interesting - what is the limit to shell invocation/args length?

nikomatsakis (Apr 12 2019 at 14:00, on Zulip):

@matklad what do you think about finding a time to talk abut the whole rls 1.0 vs 2.0 naming + branding + overall strategy?

matklad (Apr 12 2019 at 14:00, on Zulip):

I'd love to do that, yeah

nikomatsakis (Apr 12 2019 at 14:00, on Zulip):

somebody make a doodle :P

nikomatsakis (Apr 12 2019 at 14:01, on Zulip):

I'm very busy today, or i'd say let's just do later today

matklad (Apr 12 2019 at 14:01, on Zulip):

not today: on :fire: prepping the lecture :D

nikomatsakis (Apr 12 2019 at 14:01, on Zulip):

but early next week sems good

Igor Matuszewski (Apr 12 2019 at 14:02, on Zulip):

:thumbs_up:

matklad (Apr 12 2019 at 14:03, on Zulip):

@Igor Matuszewski could you make a doodle? I'll forget (offtopic: is it possible to mark 1-hr chunks en-masse in doodle?) by the end of the lecture

Igor Matuszewski (Apr 12 2019 at 14:04, on Zulip):

yup, will do

Igor Matuszewski (Apr 12 2019 at 14:04, on Zulip):

for the next week?

matklad (Apr 12 2019 at 14:05, on Zulip):

yep

matklad (Apr 12 2019 at 14:06, on Zulip):

Thanks!

nikomatsakis (Apr 12 2019 at 14:09, on Zulip):

Igor Matuszewski could you make a doodle? I'll forget (offtopic: is it possible to mark 1-hr chunks en-masse in doodle?) by the end of the lecture

it sort of is -- if you go to the weekly view, you can at least repeat the days very easily

Igor Matuszewski (Apr 14 2019 at 21:34, on Zulip):

Went away for the weekend so made the poll just now: https://doodle.com/poll/xmz2ckttftaw7d2y

Igor Matuszewski (Apr 14 2019 at 21:36, on Zulip):

Which means it might be hard to schedule early Monday morning but please write down your availability, the date will be picked by the end of tomorrow if there will be no strong preference for a single time slot

Igor Matuszewski (Apr 14 2019 at 21:36, on Zulip):

@matklad @nikomatsakis @pnkfelix :point_up:

Igor Matuszewski (Apr 14 2019 at 21:36, on Zulip):

should I cc rls 2.0 WG as well?

Igor Matuszewski (Apr 15 2019 at 21:36, on Zulip):

@nikomatsakis could you fill the poll today/tomorrow if you prefer early this week?

detrumi (Apr 16 2019 at 11:44, on Zulip):

The meeting will be on Zulip right? (and not in a Zoom call)

pnkfelix (Apr 16 2019 at 11:48, on Zulip):

(i have no strong preference for the medium of communication. i'm assuming Zulip. but i have no problem with Zoom.)

nikomatsakis (Apr 16 2019 at 13:32, on Zulip):

+1 to Zulip

nikomatsakis (Apr 16 2019 at 13:32, on Zulip):

sorry @Igor Matuszewski for not participating but the time looks good to me

detrumi (Apr 17 2019 at 12:07, on Zulip):

^ starting in about an hour

Igor Matuszewski (Apr 17 2019 at 12:08, on Zulip):

I'm sure you got the invite but just in case - https://paper.dropbox.com/doc/Meeting-notes-RLS-1.0-2.0-brandingstrategy--AbbU0brN20uW8EHVrWwq50MEAg-xBajGKDvnVoyzJZUGSzhZ (not sure how much should we document explicitly if there's a conversation history here)

Igor Matuszewski (Apr 17 2019 at 12:08, on Zulip):

Feel free to add items to the agenda

detrumi (Apr 17 2019 at 12:09, on Zulip):

Yes, just noticed it. :+1:

detrumi (Apr 17 2019 at 12:12, on Zulip):

We don't need to write down everything we talked about in detail, but a high-level overview of the main points is always nice

Igor Matuszewski (Apr 17 2019 at 13:00, on Zulip):

:wave:

Igor Matuszewski (Apr 17 2019 at 13:01, on Zulip):

since the info was posted here, I think we might continue here as well

Igor Matuszewski (Apr 17 2019 at 13:01, on Zulip):

unless you prefer otherwise

detrumi (Apr 17 2019 at 13:02, on Zulip):

@nikomatsakis Are you around?

detrumi (Apr 17 2019 at 13:03, on Zulip):

Let's start anyway :slight_smile:

Igor Matuszewski (Apr 17 2019 at 13:03, on Zulip):

On a high level it'd be good to discuss about the RLS 1.0 and 2.0, more specifically how we should talk about it and what's our strategy about it

pnkfelix (Apr 17 2019 at 13:03, on Zulip):

so

pnkfelix (Apr 17 2019 at 13:03, on Zulip):

I admit I haven't been following stuff too too closely

Igor Matuszewski (Apr 17 2019 at 13:03, on Zulip):

I know that @nikomatsakis briefly mentioned an idea about WG-RLS-1.0 - I'm not sure whether we should do it and if so what would be the benefits

Igor Matuszewski (Apr 17 2019 at 13:04, on Zulip):

no worries

pnkfelix (Apr 17 2019 at 13:04, on Zulip):

but my understanding was that RLS 2.0 was (and is) an experiment to determine if we can leverage the work of rust-analyzer to make a better foundation

pnkfelix (Apr 17 2019 at 13:04, on Zulip):

for a future RLS (and that might also tie into the future of rustc itself)

nikomatsakis (Apr 17 2019 at 13:05, on Zulip):

:wave:

pnkfelix (Apr 17 2019 at 13:05, on Zulip):

so this idea of a WG-RLS-1.0 ... I would assume would be dedicated to addressing issues in the RLS as it exists today ?

pnkfelix (Apr 17 2019 at 13:05, on Zulip):

ah well maybe @nikomatsakis can explain

Igor Matuszewski (Apr 17 2019 at 13:05, on Zulip):

Agreed - I think it's worth underlining that RLS is available now and shipped officially and it sort of works in most of the cases and keep the RLS 2.0 on the experimental/future side

Igor Matuszewski (Apr 17 2019 at 13:06, on Zulip):

Since it's a first item on the list and we're discussing it now - @nikomatsakis could you explain more what you have in mind with the WG RLS 1.0 idea?

nikomatsakis (Apr 17 2019 at 13:06, on Zulip):

Yes I mean roughly what I was thinking is (a) the RLS 1.0 exists and has users and is our official IDE :)

nikomatsakis (Apr 17 2019 at 13:06, on Zulip):

so we should definitely be supporting it

nikomatsakis (Apr 17 2019 at 13:06, on Zulip):

as in, tracking bugs, fixing them

nikomatsakis (Apr 17 2019 at 13:06, on Zulip):

we should also be considering improvements and extensions

Igor Matuszewski (Apr 17 2019 at 13:07, on Zulip):

One thing that I raised in the rust dev tools meething yday is that we struggle with contributors now

nikomatsakis (Apr 17 2019 at 13:07, on Zulip):

Yes, that was part of my thinking

Igor Matuszewski (Apr 17 2019 at 13:07, on Zulip):

we did an issue triage and cleaned things up a bit to improve the contributability factor but I think we should improve in that area

nikomatsakis (Apr 17 2019 at 13:07, on Zulip):

on the other hand, so long as we are pursuing a "new and exciting" RLS 2.0 this is possibly to be expected

Igor Matuszewski (Apr 17 2019 at 13:07, on Zulip):

(still need to talk to @Manish Goregaokar what might be a good thing to do)

nikomatsakis (Apr 17 2019 at 13:08, on Zulip):

basically I'm concerned that right now the only person really following the details there is @Igor Matuszewski

Igor Matuszewski (Apr 17 2019 at 13:09, on Zulip):

That is true right now, unfortunately

detrumi (Apr 17 2019 at 13:09, on Zulip):

Well, it feels like many people are moving to RLS2 already, as RLS2+cargo watch already works better for many people than the old RLS

matklad (Apr 17 2019 at 13:10, on Zulip):

This I think is the key question: when do we expect rls2 to be generally useful?

nikomatsakis (Apr 17 2019 at 13:10, on Zulip):

It definitely seems like one option is to just try and adopt RLS 2.0

Igor Matuszewski (Apr 17 2019 at 13:10, on Zulip):

the main pain point is latency for RLS1.0 I believe, and rightfully so

nikomatsakis (Apr 17 2019 at 13:10, on Zulip):

This I think is the key question: when do we expect rls2 to be generally useful?

I think part of the problem here is defining what generally useful means more precisely

nikomatsakis (Apr 17 2019 at 13:10, on Zulip):

It would be nice if we could have some kind of checklist of like "these are the things we would need to have working, and a set of crates they have to be working on"

nikomatsakis (Apr 17 2019 at 13:11, on Zulip):

the main pain point is latency for RLS1.0 I believe, and rightfully so

that makes sense to me -- but i'm not entirely sure if it's true

matklad (Apr 17 2019 at 13:11, on Zulip):

I think the better definition would be "when users are more satisfied with it"

nikomatsakis (Apr 17 2019 at 13:11, on Zulip):

maybe it all comes down to latency

nikomatsakis (Apr 17 2019 at 13:11, on Zulip):

I think the better definition would be "when users are more satisfied with it"

that..doesn't feel better to me :) it seems sort of unactionable.

Igor Matuszewski (Apr 17 2019 at 13:11, on Zulip):

there's one thing to keep in mind and that's parity with the compiler

matklad (Apr 17 2019 at 13:11, on Zulip):

as in, it's hard to predict which features are crucial for workdlow

nikomatsakis (Apr 17 2019 at 13:12, on Zulip):

there's one thing to keep in mind and that's parity with the compiler

yes, you mean, as we implement new features and the like

Igor Matuszewski (Apr 17 2019 at 13:12, on Zulip):

it's great to have the IDE do the right thing most of the time... I'm not sure about the times when it doesn't

detrumi (Apr 17 2019 at 13:12, on Zulip):

Latency is a big one, but the rls doesn't work for big codebases for other reasons than latency, AFAIK

Igor Matuszewski (Apr 17 2019 at 13:12, on Zulip):

in a way I believe we should converge to a model where we have rustc-as-a-language-server as a daemon we can throw requests at

Florian Diebold (Apr 17 2019 at 13:13, on Zulip):

latency isn't the only thing -- correct me if I'm wrong, but I think RLS's architecture makes it very hard to implement new assists and so on

matklad (Apr 17 2019 at 13:13, on Zulip):

it seems sort of unactionable.

Agree here. I currently have no idea about how much rls2 is actually used in the wild: we don't have any telemetry.

Igor Matuszewski (Apr 17 2019 at 13:13, on Zulip):

RLS 1.0 doesn't operate on AST directly

Igor Matuszewski (Apr 17 2019 at 13:14, on Zulip):

it asks the compiler to traverse it, dump all the data and the internal "index db" backend collects that and lets the RLS answer all the relevant language queries

matklad (Apr 17 2019 at 13:14, on Zulip):

in a way I believe we should converge to a

+1 to the idea of convergance. The approach with out of process rustc might actually allow us to get the best of both worlds in a single codebase

nikomatsakis (Apr 17 2019 at 13:14, on Zulip):

I think we should step back a hair here -- and try to identify the aims of this meeting a bit? I'm not sure for example whether I consider it 'in scope' for us right now to decide to focus purely on rust-analyzer, etc. Though we might think about trying to come up with "angles" on the discussion Basically looking at each of the plans and trying to identify the crucial questions involved.

Igor Matuszewski (Apr 17 2019 at 13:15, on Zulip):

so right now there is a split between 1.0 and 2.0 - it'd be good to talk how these relate to each other and how should we brand this

pnkfelix (Apr 17 2019 at 13:15, on Zulip):

Main agenda items: 1. What would be mission of a hypothetical WG-RLS-1.0 ?, and 2. Do we form such a WG ?

Igor Matuszewski (Apr 17 2019 at 13:15, on Zulip):

and also what to do with 1.0 now

Igor Matuszewski (Apr 17 2019 at 13:16, on Zulip):

As with every software project, list of stuff to do is potentially endless when it comes to improving the 1.0

Igor Matuszewski (Apr 17 2019 at 13:16, on Zulip):

it'd be good to tackle compiling out of process, improving projecy layout detection, incrementalizing and possibly parallelizing the rls-analysis (indexer) and so on

Igor Matuszewski (Apr 17 2019 at 13:17, on Zulip):

so in that sense I believe WG-RLS-1.0 could find work to do, so that's not a problem

nikomatsakis (Apr 17 2019 at 13:17, on Zulip):

Main agenda items: 1. What would be mission of a hypothetical WG-RLS-1.0 ?, and 2. Do we form such a WG ?

and perhaps a (3), what do we call "RLS 2.0" and how do we think about it? The name itself implies a certain... evolution that is perhaps not what we want to imply.

detrumi (Apr 17 2019 at 13:18, on Zulip):

Maybe there's also improvements that benefit both RLS1 and RLS2

Igor Matuszewski (Apr 17 2019 at 13:18, on Zulip):

I love the experiment and how it's capable right now, I'm mostly worried about the compiler parity and accuracy so as @nikomatsakis I'm not sure that's a clear evolution path and what RLS 1.0 "should become"

nikomatsakis (Apr 17 2019 at 13:19, on Zulip):

So the original plan as I understood it was roughly

1. We continue to support RLS as it exists today, but in something of a holding pattern.
2. We evolve rust-analyzer (I'm going to stick with this name for now), trying to get it to the point of general usability.
In particular, we aim to extend rust-analyzer by creating external crates that can be shared with rustc wherever possible, so as to improve rustc and rust-analyzer simultaneously.

Then, at some unspecified point, we try to figure out how to merge rustc and RA.

Igor Matuszewski (Apr 17 2019 at 13:20, on Zulip):

Right, that's how I understood it initially

pnkfelix (Apr 17 2019 at 13:20, on Zulip):

the main problem that seems to have arisen

nikomatsakis (Apr 17 2019 at 13:20, on Zulip):

The questions we're facing now are sort of obvious follow-ons from that I think:

What kinds of features do we want to support? How much refactoring will we do? Are things like a rustc daemon "out of scope"?

pnkfelix (Apr 17 2019 at 13:20, on Zulip):

is that we lack manpower/initiative to support RLS 1.0 ?

nikomatsakis (Apr 17 2019 at 13:20, on Zulip):

The latter hasn't become a big problem yet but I think it will be

nikomatsakis (Apr 17 2019 at 13:20, on Zulip):

I guess there is a third one

That process is hard =)

nikomatsakis (Apr 17 2019 at 13:21, on Zulip):

is that we lack manpower/initiative to support RLS 1.0 ?

And maybe RLS 2.0 =) I know that @matklad has wished for more involvement, for example

Igor Matuszewski (Apr 17 2019 at 13:21, on Zulip):

I wonder if we should put more emphasis into sharing and librarifying what we can to somehow amortize the cost of maintaining two compilers

pnkfelix (Apr 17 2019 at 13:22, on Zulip):

hmm. I guess I know that @matklad has wanted more involvement with RLS 2.0, but I had thought there at least were external contributors there. Am I wrong?

nikomatsakis (Apr 17 2019 at 13:22, on Zulip):

What kinds of features do we want to support? How much refactoring will we do? Are things like a rustc daemon "out of scope"?

Thinking about this, I feel like there is a similarity here to the idea of introducing shared crates and so forth. That is -- in this case, by pursuing a rustc daemon, the RLS codebase is kind of evolving. This seems like a concept that rust-analyzer too may wind up wanting? (Although the main benefits I think are parallelism, which rust-analyzer can get in other ways?)

pnkfelix (Apr 17 2019 at 13:22, on Zulip):

(I'm mentally trying to compare contributions to RLS 1.0 vs RLS 2.0 ...)

Igor Matuszewski (Apr 17 2019 at 13:22, on Zulip):

FWIW the save-analysis infrastructure should survive to support more code-indexer-oriented features (rather than lazy IDE ones)

nikomatsakis (Apr 17 2019 at 13:23, on Zulip):

The "elephant in the room" perhaps is @eddyb's thoughts on evolving rustc itself. I know they've been talking about some more ideas here and I'd like to learn more about them. But basically there isn't universal consensus on some details from rust-analyzer.

matklad (Apr 17 2019 at 13:24, on Zulip):

@pnkfelix there are awesome external contributors, but only two people who can do mentoring (me and @Florian Diebold ). Given that rls2.0 is 90% E-needs-design, just the volume of contributors does not solve all of the problmesn

Igor Matuszewski (Apr 17 2019 at 13:24, on Zulip):

@pnkfelix to measure manpower on the RLS 1.0, unfortunately right now it's me working on the project

nikomatsakis (Apr 17 2019 at 13:24, on Zulip):

you mentioned at one point some other person who was regularly contributing, right?

Igor Matuszewski (Apr 17 2019 at 13:25, on Zulip):

apart from an occasional contributor but retention seems to be low (is the project not accessible enough?)

nikomatsakis (Apr 17 2019 at 13:25, on Zulip):

@matklad part of my motivation for this design meeting concept I talked about at the last steering meeting, btw, was helping to drive some of that design and discussion, although maybe it won't work as well for RLS2.0-specific experiments

Igor Matuszewski (Apr 17 2019 at 13:25, on Zulip):

yes, @alexheretic but he's busy atm and mostly sporadically replies to issue reports now

nikomatsakis (Apr 17 2019 at 13:25, on Zulip):

apart from an occasional contributor but retention seems to be low (is the project not accessible enough?)

maybe, but this is partly what I was hoping to address via a RLS working group

nikomatsakis (Apr 17 2019 at 13:26, on Zulip):

i.e., maybe it's also visibility

Igor Matuszewski (Apr 17 2019 at 13:26, on Zulip):

I think the RLS 2.0 brand makes it so that RLS 1.0 is marked as obsolete - it'd be good to step back and to think if that's the case

Igor Matuszewski (Apr 17 2019 at 13:26, on Zulip):

do we want it to run parallel or to move towards 2.0

nikomatsakis (Apr 17 2019 at 13:26, on Zulip):

Yes, I worry about that.

nikomatsakis (Apr 17 2019 at 13:27, on Zulip):

I feel two conflicting things

matklad (Apr 17 2019 at 13:27, on Zulip):

I guess the main problem is that we came up with branding before we came up with actual plan of action :-)

Florian Diebold (Apr 17 2019 at 13:27, on Zulip):

it doesn't seem to me like the RLS 2.0 brand is very visible outside this zulip though?

nikomatsakis (Apr 17 2019 at 13:28, on Zulip):

On the one hand, I feel great about rust-analyzer, and I think that if we could just throw our energy behind such a design we could make fast progress.

On the other hand, I know that we in t-compiler have a habit of grand visions that take a long time to come to fruition, and our track record on doing the detailed, follow-on work is less good.

pnkfelix (Apr 17 2019 at 13:28, on Zulip):

so we want a label for RLS 2.0 that would simultaneously not scare away contributors from RLS 2.0 but also would not steal potential contributors to RLS 1.0 ?

pnkfelix (Apr 17 2019 at 13:29, on Zulip):

(that's assuming we do want to run in parallel; which is what @Igor Matuszewski asked above...)

nikomatsakis (Apr 17 2019 at 13:29, on Zulip):

Well, before we go there, let's ask this question:

Do we want to be doing maintenance and improvements to the existing RLS?

If so, I think we need to make good plans for doing that.

Igor Matuszewski (Apr 17 2019 at 13:29, on Zulip):

I believe there is also clear path to improve RLS-rustc by putting more work into pipelining and which can also benefit rustc (with our current incremental/on-demand setup)

matklad (Apr 17 2019 at 13:30, on Zulip):

@nikomatsakis let me ask another question :)

matklad (Apr 17 2019 at 13:30, on Zulip):

Can it be the case that, for average user, rls2 is already more useful than rls?

nikomatsakis (Apr 17 2019 at 13:31, on Zulip):

I think there are definitely users for which this is the case

matklad (Apr 17 2019 at 13:31, on Zulip):

I think the answer is no (actually, I think yes, but I am biased), but it could be pretty close

matklad (Apr 17 2019 at 13:31, on Zulip):

If we could measure this, that should inform the strategy better

nikomatsakis (Apr 17 2019 at 13:31, on Zulip):

Yes. So this is an interesting thought.

nikomatsakis (Apr 17 2019 at 13:32, on Zulip):

When we were writing up the roadmap, I remember struggling a bit with what to say about our RLS strategy

nikomatsakis (Apr 17 2019 at 13:32, on Zulip):

It felt to me at that time that we had made a kind of momentous decision in the all hands

nikomatsakis (Apr 17 2019 at 13:32, on Zulip):

without really admitting it to ourselves

nikomatsakis (Apr 17 2019 at 13:32, on Zulip):

or doing widespread consultation

nikomatsakis (Apr 17 2019 at 13:32, on Zulip):

in terms of calling something RLS 2.0

detrumi (Apr 17 2019 at 13:33, on Zulip):

I'm not sure lots of bikeshedding would've helped (in terms of speed/initiative)

nikomatsakis (Apr 17 2019 at 13:33, on Zulip):

I mean we talked about it :) but there is a kind of "whole product" aspect to this that feels a bit beyond the compiler team, in some sense

nikomatsakis (Apr 17 2019 at 13:33, on Zulip):

Well, it's not so much about the technical design or even the name

nikomatsakis (Apr 17 2019 at 13:33, on Zulip):

But IDE support is very important to the rust project

pnkfelix (Apr 17 2019 at 13:33, on Zulip):

Well for Rust, 2.0 can mean “is never going to arrive”

matklad (Apr 17 2019 at 13:33, on Zulip):

100% agree. Descisions on all-hands were very ad-hoc, and with zero consultation with wider commnunity

Igor Matuszewski (Apr 17 2019 at 13:34, on Zulip):

if we could tackle this from a slightly different angle - what are the pros/cons of commiting to either of the RLSes?

Igor Matuszewski (Apr 17 2019 at 13:34, on Zulip):

does running both in parallel bring us net positive?

Igor Matuszewski (Apr 17 2019 at 13:34, on Zulip):

I agree that immediate architecture of RLS 1.0 is limiting, whereas I imagine long-term investing a lot of work in the RLS 2.0 infra means we have two compilers

Igor Matuszewski (Apr 17 2019 at 13:35, on Zulip):

@matklad could we focus more on the sharing/librarification? Would that bring us some gains and limit the duplication of effort somehow?

Igor Matuszewski (Apr 17 2019 at 13:36, on Zulip):

I imagine working directly on RLS 1.0 would mean more benefit to the actual rustc in terms of making it more IDE-friendly

matklad (Apr 17 2019 at 13:36, on Zulip):

I guess, we already are focusing on sharing: chalk is shared, I am trying to extract rustc-lexer rigtht now, and, hopefully, tommorrow @pnkfelix will come up with plan of extracting the name resolution

detrumi (Apr 17 2019 at 13:38, on Zulip):

It doesn't feel like those efforts directly help RLS 1.0 though

nikomatsakis (Apr 17 2019 at 13:38, on Zulip):

100% agree. Descisions on all-hands were very ad-hoc, and with zero consultation with wider commnunity

I'm trying to think what this consultation might look like.

nikomatsakis (Apr 17 2019 at 13:39, on Zulip):

I think there are two angles here

nikomatsakis (Apr 17 2019 at 13:39, on Zulip):

One question is what @matklad raised -- what features do users want, and on what range of codebases?

nikomatsakis (Apr 17 2019 at 13:40, on Zulip):

(Sorry, not ignoring the library comments, just continuing to think aloud)

nikomatsakis (Apr 17 2019 at 13:42, on Zulip):

I'm not entirely sure what my second question was :)

Igor Matuszewski (Apr 17 2019 at 13:42, on Zulip):

well, it'd be good to think what benefit do users have from using the either

Igor Matuszewski (Apr 17 2019 at 13:43, on Zulip):

hm, I guess this duplicates the question what features do users want

matklad (Apr 17 2019 at 13:43, on Zulip):

Should we run some kind of survey among users who tried both rls and rust-analyzer?

nikomatsakis (Apr 17 2019 at 13:43, on Zulip):

I was thinking about that

Igor Matuszewski (Apr 17 2019 at 13:43, on Zulip):

Should we ask people on IRLO about all of that?

nikomatsakis (Apr 17 2019 at 13:44, on Zulip):

It's the obvious next step, if that's the question we're trying to answer

Igor Matuszewski (Apr 17 2019 at 13:44, on Zulip):

I'm pretty sure people will mention RLS 1.0 as being unbearably sluggish :laughter_tears:

nikomatsakis (Apr 17 2019 at 13:44, on Zulip):

If we were going to do that, we would want to work in consultation with a few others (e.g., on the community team) that have experience drawing up surveys

nikomatsakis (Apr 17 2019 at 13:44, on Zulip):

But it'd be good to try and brainstorm what info precisely we're looking for

detrumi (Apr 17 2019 at 13:45, on Zulip):

What would we do with those results? I think we'll still be in the same situation

nikomatsakis (Apr 17 2019 at 13:45, on Zulip):

e.g., trying to tease out -- perf is part of it, but also like "what kinds of codebases" (how many crates, workspaces, how much is it stable/nightly features), what IDE features do people use (completions, jump to def, not sure what else there is), what do they try to do and it doesn't work?

detrumi (Apr 17 2019 at 13:45, on Zulip):

Because RLS 2.0 is still far off from going all the way with reporting compiler errors

Igor Matuszewski (Apr 17 2019 at 13:45, on Zulip):

Well, if it happens that a feature X is the main reason 1.0 is used and not 2.0 then it might give us an idea where 2.0 needs to be improved

detrumi (Apr 17 2019 at 13:45, on Zulip):

(go to symbol is another big one)

Igor Matuszewski (Apr 17 2019 at 13:46, on Zulip):

it's great that 2.0 doesn't need Cargo project layout (something I wanted to work on for 1.0)

matklad (Apr 17 2019 at 13:46, on Zulip):

Well, a hypothetical fast path is that everyone says that rls2 works better right now, and, if that's the case, we can expedite shipping by implementing rustc-out-of-process in RLS1 and than merging it with RLS2

pnkfelix (Apr 17 2019 at 13:46, on Zulip):

what do they try to do and it doesn't work?

specifically asking: "What code base (or feature) did you stop using IDE X with" might be good for that.

nikomatsakis (Apr 17 2019 at 13:47, on Zulip):

What would we do with those results? I think we'll still be in the same situation

a good question :) it seems like useful data whatever we do, but it is good to think a bit further ahead. I thnk we are considering a few things:

nikomatsakis (Apr 17 2019 at 13:48, on Zulip):

Well, a hypothetical fast path is that everyone says that rls2 works better right now, and, if that's the case, we can expedite shipping by implementing rustc-out-of-process in RLS1 and than merging it with RLS2

I don't quite follow this

nikomatsakis (Apr 17 2019 at 13:48, on Zulip):

but I think it's kind of the same thing I just said

detrumi (Apr 17 2019 at 13:48, on Zulip):

If we're able to think of a fast path from where we're at right now, I think we should just follow that (but maybe I'm biased)

matklad (Apr 17 2019 at 13:49, on Zulip):

I don't quite follow this

There's two points:

Igor Matuszewski (Apr 17 2019 at 13:49, on Zulip):

what do we specifically mean by out of process rustc here?

nikomatsakis (Apr 17 2019 at 13:49, on Zulip):

I guess the second half is what I don't understand :)

Igor Matuszewski (Apr 17 2019 at 13:50, on Zulip):

in the RLS case, we do it to generate save-analysis artifacts which tie into the whole SA/rls-analysis architecture

Igor Matuszewski (Apr 17 2019 at 13:50, on Zulip):

so I'm not sure how that's applicable rn to rls 2.0

matklad (Apr 17 2019 at 13:51, on Zulip):

If we can generate save-analysis for current state of the world by just shelling out to Cargo, we can share the infra that transformes save-analysis into LSP between RLS 1 and 2.

nikomatsakis (Apr 17 2019 at 13:51, on Zulip):

(so fyi I have another call in 9 minutes (on the hour))

Florian Diebold (Apr 17 2019 at 13:51, on Zulip):

maybe we should just make a proper release of RA and see how many people are interested? so far, it's been kind of stealthy and has never had a release that said "this might actually be useful for you", right?

Florian Diebold (Apr 17 2019 at 13:52, on Zulip):

(maybe after Chalk integration lands ;)

matklad (Apr 17 2019 at 13:52, on Zulip):

That is, we can do sort-of racer/RLS setup, but with rls2/rls

Igor Matuszewski (Apr 17 2019 at 13:52, on Zulip):

I think I'm biased but I believe RA is very popular right now, no?

nikomatsakis (Apr 17 2019 at 13:52, on Zulip):

Maybe, yeah, but I think we have to be cautious about branding. (Regarding a release.)

nikomatsakis (Apr 17 2019 at 13:52, on Zulip):

People will want to know what the plan is

nikomatsakis (Apr 17 2019 at 13:52, on Zulip):

And we should have an answer :)

Igor Matuszewski (Apr 17 2019 at 13:52, on Zulip):

if it's not, I'm afraid 1.0 could be considered dead by now, even if it's used by a lot of the people thanks to rustup xd

Florian Diebold (Apr 17 2019 at 13:53, on Zulip):

yeah, I don't mean it should be branded as "RLS 2", it should be clear that it's still an experiment

Igor Matuszewski (Apr 17 2019 at 13:53, on Zulip):

so I'm curious about the 1.0-2.0 bridge and would love to talk more to @matklad about it

nikomatsakis (Apr 17 2019 at 13:53, on Zulip):

yes, that's an interesting thought I hadn't heard before

nikomatsakis (Apr 17 2019 at 13:53, on Zulip):

this was an interesting discussion, even if we didn't really get to the original agenda :)

Igor Matuszewski (Apr 17 2019 at 13:54, on Zulip):

it'd be good to also talk to @eddyb and @Zoxc at some point regarding the rustc arch and if they have a vision what to do next (sorry for not pinging you before!)

matklad (Apr 17 2019 at 13:54, on Zulip):

I guess I used the wrong words all the previos times I've tried to articulate the idea :)

nikomatsakis (Apr 17 2019 at 13:54, on Zulip):

I think my current feeling is roughly this:

towards the second point, I could see value in trying to articulate out the options in more detail.

Igor Matuszewski (Apr 17 2019 at 13:54, on Zulip):

so to slowly start summing it up, we'd love to have more data

Igor Matuszewski (Apr 17 2019 at 13:55, on Zulip):

=> we need to do a survey and consult with more people?

nikomatsakis (Apr 17 2019 at 13:55, on Zulip):

in particular, I think it'd be nice to be thinking beyond RLS + Rust-analyzer, and thinking a bit about the "incremental rustc" evolution

nikomatsakis (Apr 17 2019 at 13:55, on Zulip):

I also think we should think about how/when to involve the wider team + community (beyond compiler team)

detrumi (Apr 17 2019 at 13:55, on Zulip):

Doing a survey also gives people the expectation that we'll listen to the result, and that might turn out to be problematic implementation-wise

Igor Matuszewski (Apr 17 2019 at 13:56, on Zulip):

Well it's not a "what do you think we should do" type of survey here, so I think we're safe on that front

Igor Matuszewski (Apr 17 2019 at 13:56, on Zulip):

rather collect people's experiences and figure out what might be some options/problems we didn't see

nikomatsakis (Apr 17 2019 at 13:58, on Zulip):

Maybe the thing to do is to think about action items. I see a few things that might be useful to give overall context

?

nikomatsakis (Apr 17 2019 at 13:58, on Zulip):

I'm not imagining very long 'roadmaps', just some bullets

nikomatsakis (Apr 17 2019 at 13:59, on Zulip):

anyway, I gotta go, I'll try to follow on async

Igor Matuszewski (Apr 17 2019 at 13:59, on Zulip):

apart from roadmaps, it'd be good to maybe figure out a couple of approaches in more detail and what would it mean

pnkfelix (Apr 17 2019 at 13:59, on Zulip):

i'm torn about whether the roadmap(s) would be useful in constructing the survey

nikomatsakis (Apr 17 2019 at 13:59, on Zulip):

I guess a very simple action item is to summarize some of this conversaton :)

nikomatsakis (Apr 17 2019 at 13:59, on Zulip):

apart from roadmaps, it'd be good to maybe figure out a couple of approaches in more detail and what would it mean

yeah this is kind of what I was trying to do

pnkfelix (Apr 17 2019 at 13:59, on Zulip):

or if the survery results useful in constructing the roadmaps

nikomatsakis (Apr 17 2019 at 13:59, on Zulip):

i'm not sure if roadmap is the word I wanted

nikomatsakis (Apr 17 2019 at 14:00, on Zulip):

I was going to say "skech out various possible plans and how they would work"

nikomatsakis (Apr 17 2019 at 14:00, on Zulip):

but as I tried to enumerate what those plans were, I thought maybe they would be constructed by starting from information about the existing approaches? but maybe not

Igor Matuszewski (Apr 17 2019 at 14:01, on Zulip):

that looks reasonable

Igor Matuszewski (Apr 17 2019 at 14:01, on Zulip):

thanks everybody!

Igor Matuszewski (Apr 17 2019 at 14:01, on Zulip):

and please note down things at https://paper.dropbox.com/doc/Meeting-notes-RLS-1.0-2.0-brandingstrategy--AbbU0brN20uW8EHVrWwq50MEAg-xBajGKDvnVoyzJZUGSzhZ if you can, whether it's your idea what is a good action item from now on (I guess putting it to words is the biggest hurdle?)

nikomatsakis (Apr 17 2019 at 14:02, on Zulip):

seems clear we need to have another discussion or two like this :)

Igor Matuszewski (Apr 17 2019 at 14:02, on Zulip):

or if you'd like to sum it up or highlight a particular point of this convo

Igor Matuszewski (Apr 17 2019 at 14:02, on Zulip):

seems clear we need to have another discussion or two like this :)

should I create a doodle for next week or should we first do the survey?

nikomatsakis (Apr 17 2019 at 14:03, on Zulip):

I don't think we should block on doing the survey, it'll take some time to craft; I think it'd make sense to do a bit of prep work before we talk again of some kind though.

nikomatsakis (Apr 17 2019 at 14:03, on Zulip):

I think trying to elaborate out possible plans and what the challenges will be with each one is a good idea

nikomatsakis (Apr 17 2019 at 14:04, on Zulip):

maybe we can collaboratively do that

pnkfelix (Apr 17 2019 at 14:04, on Zulip):

or if you'd like to sum it up or highlight a particular point of this convo

if nothing else, making sure all the :point_up: -labelled messages are reflected in the notes is a good heuristic.

matklad (Apr 17 2019 at 14:04, on Zulip):

yeah, today's convo were unstructured.... Should we create a brainstorming paper docs or somehting like that?

Igor Matuszewski (Apr 17 2019 at 14:04, on Zulip):

My bad, I should've created a better agenda beforehand :(

nikomatsakis (Apr 17 2019 at 14:04, on Zulip):

I think we need a paper doc where we can start to sketch out the plans, and leave comments for one another, etc

nikomatsakis (Apr 17 2019 at 14:04, on Zulip):

My bad, I should've created a better agenda beforehand :(

no worries, it turned out quite productive I think :) it seems like we were all sort of unaware of how much uncertainty we had

matklad (Apr 17 2019 at 14:05, on Zulip):

@Igor Matuszewski I think we are all genuinely don't know what we are doing

matklad (Apr 17 2019 at 14:05, on Zulip):

:)

matklad (Apr 17 2019 at 14:06, on Zulip):

also, unstructured does not mean bad, it was productive!

detrumi (Apr 17 2019 at 14:06, on Zulip):

A short agenda is better than following the wrong agenda

matklad (Apr 17 2019 at 14:09, on Zulip):

@Igor Matuszewski I'd love to chat about 1-2 bridge later toady, if that's possible?

eddyb (Apr 17 2019 at 14:09, on Zulip):

aaaaaaaaaaah this is too much discussion to read :(

eddyb (Apr 17 2019 at 14:10, on Zulip):

I have to go right now but I just went through all the mentions over the past few months, in case anything important was forgotten

eddyb (Apr 17 2019 at 14:10, on Zulip):

(and saw I was pinged here)

pnkfelix (Apr 17 2019 at 14:20, on Zulip):

aaaaaaaaaaah this is too much discussion to read :(

Uses the notes @eddyb

eddyb (Apr 17 2019 at 14:20, on Zulip):

oh this is a meeting, not random discussion?

eddyb (Apr 17 2019 at 14:20, on Zulip):

the topic didn't make it clear :P

detrumi (Apr 17 2019 at 14:21, on Zulip):

We started the meeting in the same topic as the planning of said meeting, might not have been the best option for those reading back afterwards

eddyb (Apr 17 2019 at 14:26, on Zulip):

oh, for other meetings there are usually separate topics

eddyb (Apr 17 2019 at 14:27, on Zulip):

@nikomatsakis I promised you something so I opened inkscape, and, well, here's a sneak peek pasted image

eddyb (Apr 17 2019 at 14:27, on Zulip):

I'll either try to make it nice or do it in a Markdown table

pnkfelix (Apr 17 2019 at 14:28, on Zulip):

We started the meeting in the same topic as the planning of said meeting, might not have been the best option for those reading back afterwards

I knew there was some reason my spidey-sense was tingling when we didn't make a fresh topic

Igor Matuszewski (Apr 17 2019 at 16:07, on Zulip):

@matklad yes please (sorry, didn't see the message)

matklad (Apr 17 2019 at 16:08, on Zulip):

Let's make a stream for it ... somewhere? I guess rls2.0 might be a better fit?

Igor Matuszewski (Apr 17 2019 at 16:18, on Zulip):

@matklad sounds good!

matklad (Apr 17 2019 at 16:18, on Zulip):

moved to https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/RLS.201.2E0.20--.20RLS.202.2E0.20bridge

Last update: Nov 20 2019 at 01:45UTC