Stream: t-compiler

Topic: rls sync


nikomatsakis (Sep 18 2019 at 13:55, on Zulip):

OK @Igor Matuszewski :wave: -- so, right, out of process works now :)

nikomatsakis (Sep 18 2019 at 13:55, on Zulip):

(Separate question: Should we have a Zulip stream for the rls, btw?)

Igor Matuszewski (Sep 18 2019 at 13:55, on Zulip):

Yes :tada:

nikomatsakis (Sep 18 2019 at 13:55, on Zulip):

Anyway I didn't quite follow whether one can work on the RLS w/ (e.g.) a nightly rustc or not

Igor Matuszewski (Sep 18 2019 at 13:55, on Zulip):

uh, I feel like we come back to the old problems again of compiler/rls/WG rls 2.0 :laughter_tears:

Igor Matuszewski (Sep 18 2019 at 13:55, on Zulip):

either works for me, we can stay here

nikomatsakis (Sep 18 2019 at 13:56, on Zulip):

yeah, I think @matklad and I are both not super comfortable with that name

pnkfelix (Sep 18 2019 at 13:56, on Zulip):

yeah I was just going to say that (about compiler/rls/rls2.0)

nikomatsakis (Sep 18 2019 at 13:56, on Zulip):

but for now let's just chat and stop moving around :")

Igor Matuszewski (Sep 18 2019 at 13:56, on Zulip):

let's get back to that later :smile:

nikomatsakis (Sep 18 2019 at 13:56, on Zulip):

(though one of the things I would like us to talk about is trying to unify our "IDE story", which is related to the naming question)

Igor Matuszewski (Sep 18 2019 at 13:56, on Zulip):

out of process works right now - the support has to be compiled in and by default is turned off in Rust tree and in the separate repository

Igor Matuszewski (Sep 18 2019 at 13:57, on Zulip):

this is done because we're currently using Tokio and the async stuff for the communication

Igor Matuszewski (Sep 18 2019 at 13:57, on Zulip):

using JSON-RPC over IPC (UDS on Unix, named pipes on Windows)

nikomatsakis (Sep 18 2019 at 13:58, on Zulip):

ok

Igor Matuszewski (Sep 18 2019 at 13:58, on Zulip):

not sure if this is premature optimization - I wanted to avoid spawning a thread and blocking on each connection

Igor Matuszewski (Sep 18 2019 at 13:59, on Zulip):

and felt like latency would be a problem in this push to decrease latency as much as we can

nikomatsakis (Sep 18 2019 at 13:59, on Zulip):

each connection == running rustc?

Igor Matuszewski (Sep 18 2019 at 13:59, on Zulip):

yes

nikomatsakis (Sep 18 2019 at 13:59, on Zulip):

ok. seems like it may be premature to me in that case

nikomatsakis (Sep 18 2019 at 13:59, on Zulip):

just because the overhead of running rustc is high enough

Igor Matuszewski (Sep 18 2019 at 13:59, on Zulip):

but right now it's not really optimal as well, since we spawn the tokio runtime, run the thing, shutdown the runtime

Igor Matuszewski (Sep 18 2019 at 13:59, on Zulip):

so it's almost worst of two worlds :laughter_tears: but I imagined we can later set up one server for that

nikomatsakis (Sep 18 2019 at 14:00, on Zulip):

how much data goes over this connection?

Igor Matuszewski (Sep 18 2019 at 14:00, on Zulip):

or we can just spawn threads and block on the connection

nikomatsakis (Sep 18 2019 at 14:00, on Zulip):

I guess the VFS primarily?

nikomatsakis (Sep 18 2019 at 14:00, on Zulip):

also save-analysis output?

Igor Matuszewski (Sep 18 2019 at 14:00, on Zulip):

save-analysis would be the biggest one, yes

Igor Matuszewski (Sep 18 2019 at 14:00, on Zulip):

we serialize to a JSON - effectively send SA index file contents

Igor Matuszewski (Sep 18 2019 at 14:01, on Zulip):

and send that over the JSON-RPC

Igor Matuszewski (Sep 18 2019 at 14:01, on Zulip):

I think it's less than 5 MB in most cases

Igor Matuszewski (Sep 18 2019 at 14:01, on Zulip):

obviously depends on the crates - winapi is considerably bigger for example (don't have numbers rn)

nikomatsakis (Sep 18 2019 at 14:01, on Zulip):

OK. Well, the details of how it's setup don't seem like the most important thing.

nikomatsakis (Sep 18 2019 at 14:02, on Zulip):

I guess we'll just have to measure

nikomatsakis (Sep 18 2019 at 14:02, on Zulip):

I do suspect a simple setup would suffice

Igor Matuszewski (Sep 18 2019 at 14:02, on Zulip):

I measured the performance overhead of doing this in the out-of-process way

Igor Matuszewski (Sep 18 2019 at 14:02, on Zulip):

nothing big, just a couple warm runs on some medium sized projects

Igor Matuszewski (Sep 18 2019 at 14:03, on Zulip):

and it's <5-10% slower

pnkfelix (Sep 18 2019 at 14:03, on Zulip):

performance overhead of tokio-runtime + added rls process?

Igor Matuszewski (Sep 18 2019 at 14:03, on Zulip):

yes

nikomatsakis (Sep 18 2019 at 14:03, on Zulip):

I imagine that means compared to running rust in process, right?

Igor Matuszewski (Sep 18 2019 at 14:04, on Zulip):

start to finish for a couple of primary crate targets in a workspace

nikomatsakis (Sep 18 2019 at 14:04, on Zulip):

that is, you're not comparing the cost of different forms of out-of-process

Igor Matuszewski (Sep 18 2019 at 14:04, on Zulip):

yes

Igor Matuszewski (Sep 18 2019 at 14:04, on Zulip):

sorry

nikomatsakis (Sep 18 2019 at 14:04, on Zulip):

makes sense

Igor Matuszewski (Sep 18 2019 at 14:04, on Zulip):

that's <5-10% slower when running RLS out-of-process comparing to the current mode where we run things in-process

Igor Matuszewski (Sep 18 2019 at 14:05, on Zulip):

which sort of makes sense, but I'm mostly happy about potentially dropping the non-obvious locking environment setup

Igor Matuszewski (Sep 18 2019 at 14:05, on Zulip):

and there seems to be a memory leak - either in Cargo or rustc?

Igor Matuszewski (Sep 18 2019 at 14:05, on Zulip):

let me link an issue

Igor Matuszewski (Sep 18 2019 at 14:05, on Zulip):

(which running this out-of-process basically "fixes")

pnkfelix (Sep 18 2019 at 14:05, on Zulip):

and that 5-10% overhead includes the time to start up tokio runtime (as well as run rustc and serialize and transmit the save-analysis data), right?

Igor Matuszewski (Sep 18 2019 at 14:06, on Zulip):

https://github.com/rust-lang/rls/issues/1188

pnkfelix (Sep 18 2019 at 14:06, on Zulip):

I guess the running rustc was included in the baseline, of course...

Igor Matuszewski (Sep 18 2019 at 14:06, on Zulip):

yes, entirely start to finish

Igor Matuszewski (Sep 18 2019 at 14:06, on Zulip):

rn we spawn Tokio runtime before each rustc invocation and then tear it down which is less than ideal

pnkfelix (Sep 18 2019 at 14:06, on Zulip):

I didn't actually know that the tokio runtime was expensive

pnkfelix (Sep 18 2019 at 14:06, on Zulip):

you probably could measure that on its own, if you wanted

Igor Matuszewski (Sep 18 2019 at 14:06, on Zulip):

the factors important here is that save-analysis is serialized and deserialized over the wire

pnkfelix (Sep 18 2019 at 14:07, on Zulip):

(by adding the startup/shutdown of tokio to the in-process RLS benchmarking)

Igor Matuszewski (Sep 18 2019 at 14:07, on Zulip):

also lock contention on the environment (each rustc instance wants a dedicated access to the env)

Igor Matuszewski (Sep 18 2019 at 14:07, on Zulip):

that's a fair point!

Igor Matuszewski (Sep 18 2019 at 14:07, on Zulip):

we could try that

pnkfelix (Sep 18 2019 at 14:07, on Zulip):

but I'm also willing to believe that its expensive. :)

Igor Matuszewski (Sep 18 2019 at 14:08, on Zulip):

we could also spawn a dedicated IPC server thread and run it all there - that'd avoid the startup/teardown on each execution

nikomatsakis (Sep 18 2019 at 14:08, on Zulip):

(which running this out-of-process basically "fixes")

are you satisfied with this "fix"? any thoughts or ideas of what the problem was before? something internal to rls?

nikomatsakis (Sep 18 2019 at 14:08, on Zulip):

could it have been (e.g.) rustc interning pools or other static data?

Igor Matuszewski (Sep 18 2019 at 14:08, on Zulip):

but @matklad also had a point - we could just spawn a thread and do a blocking I/O with JSON separated with newlines to drop the async ecosystem

nikomatsakis (Sep 18 2019 at 14:09, on Zulip):

(personally, I'd prefer something like that, just because it seems simpler and has fewer deps, but I don't care very much)

nikomatsakis (Sep 18 2019 at 14:09, on Zulip):

(my personal experience has been that spawning threads is usually not a big thing, but I guess it will depend if we ever get to the point where we're doing these calls at a really fine-grained level)

Igor Matuszewski (Sep 18 2019 at 14:09, on Zulip):

@nikomatsakis I didn't dig deep enough (started though) - I know that Cargo interners came up as the leaked "source" but didn't investigate fully so don't want to say anything for certain

nikomatsakis (Sep 18 2019 at 14:10, on Zulip):

ok. Have you verified that this out-of-process work fixes the problem?

nikomatsakis (Sep 18 2019 at 14:10, on Zulip):

Or you are just hypothesizing?

nikomatsakis (Sep 18 2019 at 14:10, on Zulip):

It seems like a reasonable guess, to be clear, just wanted to be sure

Igor Matuszewski (Sep 18 2019 at 14:10, on Zulip):

hypothesizing unfortunately

pnkfelix (Sep 18 2019 at 14:10, on Zulip):

also lock contention on the environment (each rustc instance wants a dedicated access to the env)

Does this imply that you cannot even have the invocations of rustc running in parallel, even if you did implement it via distinct spawned threads?

nikomatsakis (Sep 18 2019 at 14:10, on Zulip):

OK.

Igor Matuszewski (Sep 18 2019 at 14:11, on Zulip):

@pnkfelix yes, that's one of the reasons we wanted to pursue out-of-process compilation

nikomatsakis (Sep 18 2019 at 14:11, on Zulip):

So now we can support multiple instances of rustc?

nikomatsakis (Sep 18 2019 at 14:11, on Zulip):

(Are we taking advantage of that?)

Igor Matuszewski (Sep 18 2019 at 14:11, on Zulip):

right now we cache the build plan by Cargo whenever something major happens (changed Cargo.{toml,lock} files, removed /target etc.) and executed these in serial fashion

nikomatsakis (Sep 18 2019 at 14:11, on Zulip):

OK. It seems like that might well offset the 5-10% overhead

Igor Matuszewski (Sep 18 2019 at 14:11, on Zulip):

We're not taking advantage of that yet

Igor Matuszewski (Sep 18 2019 at 14:12, on Zulip):

Was wondering if we could extract the graph processing logic that Cargo has out to a crate with @Alex Crichton

Igor Matuszewski (Sep 18 2019 at 14:12, on Zulip):

but didn't progress further

nikomatsakis (Sep 18 2019 at 14:13, on Zulip):

Interesting. Might be worth looping in @Taylor Cramer (on vacation this week) and/or jsgf

Igor Matuszewski (Sep 18 2019 at 14:13, on Zulip):

technically we could run IPC_ENDPOINT=... cargo check

nikomatsakis (Sep 18 2019 at 14:13, on Zulip):

because this seems to relate to the "cargo build plan" support that more complex build systems want (e.g. bazel or what have you)

Igor Matuszewski (Sep 18 2019 at 14:14, on Zulip):

I know that Cargo can emit build plans and we can leverage that as well

nikomatsakis (Sep 18 2019 at 14:14, on Zulip):

anyway, very exciting that out-of-process is working -- what is next on your plate? (I'm going to hvae to run in a few minutes, fyi)

Igor Matuszewski (Sep 18 2019 at 14:14, on Zulip):

I've been thinking about converging the IDE effort, finally

Igor Matuszewski (Sep 18 2019 at 14:15, on Zulip):

since the query infra is now in place I was wondering if it'd be feasible to wrap the query system somehow out-of-process

Igor Matuszewski (Sep 18 2019 at 14:15, on Zulip):

like - execute rustc but only really ask it to parse/nameres and then execute a single query and come back with results

Igor Matuszewski (Sep 18 2019 at 14:16, on Zulip):

to probably rely less on save-analysis itself

Igor Matuszewski (Sep 18 2019 at 14:16, on Zulip):

that sounds very experimental though

Igor Matuszewski (Sep 18 2019 at 14:17, on Zulip):

I think this could help us look a bit more on how rustc itself could be changed internally rather than redoing the frontend?

Igor Matuszewski (Sep 18 2019 at 14:17, on Zulip):

For example IIRC we can't change inputs in the rustc session from the query system PoV

Igor Matuszewski (Sep 18 2019 at 14:17, on Zulip):

I know that @Alexander Regueiro also wanted to work on something similar for Rust REPL

Igor Matuszewski (Sep 18 2019 at 14:18, on Zulip):

but apart from that I wanted to verify the memory leak issues, see if we can drop down Tokio and what would be the cost of that

nikomatsakis (Sep 18 2019 at 14:18, on Zulip):

I've been thinking about converging the IDE effort, finally

maybe @matklad (and/or @Florian Diebold), you, and I (and whomever else) should try to schedule some time to think about this specifically?

nikomatsakis (Sep 18 2019 at 14:19, on Zulip):

I think it would be great if we could start to move towards a single "framework"

nikomatsakis (Sep 18 2019 at 14:19, on Zulip):

though I'm not entirely sure what that means

Igor Matuszewski (Sep 18 2019 at 14:19, on Zulip):

also right now tools are not available on tier 2 platforms due to proc macros and cross-compilation - while tangentially related, it'd reduce some development friction (for example for rls-data to switch to serde it had to manually expand the serde_derive code because x.py couldn't cope with multiple serde_derive available)

Igor Matuszewski (Sep 18 2019 at 14:20, on Zulip):

well one crazy idea would be to investigate doing dot completion using the existing rustc query infra

Igor Matuszewski (Sep 18 2019 at 14:20, on Zulip):

and honestly this would fix a lot of outstanding issues such as 'can't complete anything that needs type inference because of Racer'

Igor Matuszewski (Sep 18 2019 at 14:21, on Zulip):

or type-checking expressions (which is really handy)

matklad (Sep 18 2019 at 14:21, on Zulip):

catching up with conversation, but cargo definitelly leaked tons of strings for interning at one point

Alexander Regueiro (Sep 18 2019 at 14:21, on Zulip):

@Igor Matuszewski I’m about to publish it in fact. :-)

Igor Matuszewski (Sep 18 2019 at 14:22, on Zulip):

So yeah, still a lot of things we can tackle at first =)

Igor Matuszewski (Sep 18 2019 at 14:23, on Zulip):

Maybe a bit (too?) specific question - when fixing save-analysis generation at https://github.com/rust-lang/rust/pull/64250#issuecomment-531521579 I was bit by missing typeck tables for a synthetic def

Igor Matuszewski (Sep 18 2019 at 14:24, on Zulip):

which is an impl trait in return position

matklad (Sep 18 2019 at 14:24, on Zulip):

technically we could run IPC_ENDPOINT=... cargo check

I am very pro this approach, as it allows to move cargo out of RLS, which seems like a huge source of complexity, and it also would work for rust-analyzer

Igor Matuszewski (Sep 18 2019 at 14:25, on Zulip):

(I was wondering if it's a bug that we don't have these? Clippy also hit some missing typeck tables where one would expect them IIRC at https://github.com/rust-lang/rust-clippy/pull/4545#issuecomment-532171307)

Igor Matuszewski (Sep 18 2019 at 14:26, on Zulip):

I promoted the debug assertion to a regular one when using hir id and typeck tables but I'm wondering if it's causing more harm now than good :frown:

Igor Matuszewski (Sep 18 2019 at 14:28, on Zulip):

@matklad that's fair! I think that an extracted 'executor' crate would be also helpful, since I imagine things like Buck would want to be supported in similar vein

Igor Matuszewski (Sep 18 2019 at 14:29, on Zulip):

but I agree, reducing coupling and code duplication would be great!

matklad (Sep 18 2019 at 14:34, on Zulip):

@Igor Matuszewski @Florian Diebold @nikomatsakis let's find time for "rls/rust-analyzer joint planning" meeting: https://doodle.com/poll/t42575t79pbxmasa

Florian Diebold (Sep 18 2019 at 14:36, on Zulip):

I'll be on vacation all of next week, but I'm probably not essential for the meeting :slight_smile:

Igor Matuszewski (Sep 18 2019 at 20:39, on Zulip):

@matklad sorry for the off hours, I'm in the US timezone for a week

Igor Matuszewski (Sep 18 2019 at 20:40, on Zulip):

filled

matklad (Sep 19 2019 at 12:38, on Zulip):

rls/rust-analyzer meeting will happen later today: https://calendar.google.com/event?action=TEMPLATE&tmeid=NTNxdXU4MDQ0ZDFsZ2lkamk1NTcwcTY4dHVfMjAxOTA5MTlUMTQwMDAwWiA2dTVycnRjZTZscnR2MDdwZmkzZGFtZ2p1c0Bn&tmsrc=6u5rrtce6lrtv07pfi3damgjus%40group.calendar.google.com&scp=ALL

let me pre-emptively create a stream...

matklad (Sep 19 2019 at 13:40, on Zulip):

Note, the link above is wrong, the right one is this:

https://calendar.google.com/event?action=TEMPLATE&tmeid=NzZmMnRoYzQ2MnYxcDZnY28zcHA5cGhyaWsgNnU1cnJ0Y2U2bHJ0djA3cGZpM2RhbWdqdXNAZw&tmsrc=6u5rrtce6lrtv07pfi3damgjus%40group.calendar.google.com

nikomatsakis (Oct 01 2019 at 13:45, on Zulip):

@Igor Matuszewski :wave:

Igor Matuszewski (Oct 01 2019 at 13:49, on Zulip):

@nikomatsakis :wave:

Igor Matuszewski (Oct 01 2019 at 13:49, on Zulip):

I'm here

nikomatsakis (Oct 01 2019 at 13:49, on Zulip):

So how goes

nikomatsakis (Oct 01 2019 at 13:49, on Zulip):

(also, cc @pnkfelix in case you're interested)

Igor Matuszewski (Oct 01 2019 at 13:50, on Zulip):

Nothing new on the RLS side in the recent days

Igor Matuszewski (Oct 01 2019 at 13:50, on Zulip):

was in a bit of a flux due to the Colorado Gold Rust conference and travel

nikomatsakis (Oct 01 2019 at 13:51, on Zulip):

Ah, yeah, ok

Igor Matuszewski (Oct 01 2019 at 13:51, on Zulip):

https://github.com/rust-lang/rust/issues/64659 is on my list for this week

nikomatsakis (Oct 01 2019 at 13:51, on Zulip):

Did you see the issue we tagged you with in the compiler meeting?

nikomatsakis (Oct 01 2019 at 13:51, on Zulip):

I think that's the one

nikomatsakis (Oct 01 2019 at 13:51, on Zulip):

ok

Igor Matuszewski (Oct 01 2019 at 13:51, on Zulip):

yeah

Igor Matuszewski (Oct 01 2019 at 13:52, on Zulip):

so that's planned now, I believe I made the mistake of promoting a debug assertion to regular one close to beta cut-off and just realized that this might be in the current beta

Igor Matuszewski (Oct 01 2019 at 13:52, on Zulip):

so the fix will probably need to be backported

nikomatsakis (Oct 01 2019 at 13:52, on Zulip):

ok

nikomatsakis (Oct 01 2019 at 13:52, on Zulip):

although in principle that should be ok?

nikomatsakis (Oct 01 2019 at 13:52, on Zulip):

like, the debug-assertion ought to always be true...?

Igor Matuszewski (Oct 01 2019 at 13:53, on Zulip):

the fix to the bug uncovered by the assertion promotion in itself should be backported

nikomatsakis (Oct 01 2019 at 13:53, on Zulip):

ok

Igor Matuszewski (Oct 01 2019 at 13:53, on Zulip):

and not the demotion to debug assertion :smile:

Igor Matuszewski (Oct 01 2019 at 13:54, on Zulip):

need to fix the VS Code extension since it waited for 1.38 to land to fix the false positive 'unknown RLS configuration' warning we've been giving due to stable not supporting certain new configuration

Igor Matuszewski (Oct 01 2019 at 13:54, on Zulip):

but 1.38 has landed so this is unblocked

Igor Matuszewski (Oct 01 2019 at 13:54, on Zulip):

also because of it it'd be good to think of how we can manage the settings 'deprecation' wrt different toolchains in the RLS

nikomatsakis (Oct 01 2019 at 13:55, on Zulip):

I don't quite know what you mean

Igor Matuszewski (Oct 01 2019 at 13:55, on Zulip):

since people will use one exact version of the extension and it comes with a set of 'supported' configurations, but the user can opt-in into different toolchains

Igor Matuszewski (Oct 01 2019 at 13:55, on Zulip):

but that's a detail and should be discussed separately I believe

Igor Matuszewski (Oct 01 2019 at 13:56, on Zulip):

briefly: since 1.38 we can now suppress a spurious warning in the VSCode

nikomatsakis (Oct 01 2019 at 13:56, on Zulip):

ok

Igor Matuszewski (Oct 01 2019 at 13:56, on Zulip):

apart from that, next thing up would be to probably try and get rid of Tokio deps in the out-of-process scheme

nikomatsakis (Oct 01 2019 at 13:57, on Zulip):

what do you plan to replace with?

Igor Matuszewski (Oct 01 2019 at 13:57, on Zulip):

or maybe just enable them and see if there's any meaningful cost in the CI/CD pipeline

Igor Matuszewski (Oct 01 2019 at 13:57, on Zulip):

threads with blocking I/O based on JSONs per line

nikomatsakis (Oct 01 2019 at 13:57, on Zulip):

ok ok

Igor Matuszewski (Oct 01 2019 at 13:57, on Zulip):

but I believe we also talked about it; just wanted to say what's up next

nikomatsakis (Oct 01 2019 at 13:57, on Zulip):

yeah, hmm, maybe worth measuring -- though I somewhat favor a simpler approach just in general, because fewer deps can be useful

nikomatsakis (Oct 01 2019 at 13:57, on Zulip):

yep, whichever way you find best I guess

Igor Matuszewski (Oct 01 2019 at 13:58, on Zulip):

yeah, will measure the overhead beforehand

Igor Matuszewski (Oct 01 2019 at 13:59, on Zulip):

so yeah, that's it for me - sorry for the list being so short now :bow:

nikomatsakis (Oct 01 2019 at 13:59, on Zulip):

no worries, I'm happy to have a bit of time to do reviews :)

nikomatsakis (Oct 01 2019 at 13:59, on Zulip):

I'd like us to make more progress on the "bigger IDE story" questions but I don't have much new to bring to the table there right now

Igor Matuszewski (Oct 01 2019 at 13:59, on Zulip):

and I'm also glad we have a dedicated time slot to talk about it :thumbs_up:

nikomatsakis (Oct 01 2019 at 14:00, on Zulip):

Yes

nikomatsakis (Oct 01 2019 at 14:00, on Zulip):

All set then?

Igor Matuszewski (Oct 01 2019 at 14:00, on Zulip):

yeah

nikomatsakis (Oct 01 2019 at 14:00, on Zulip):

:heart: sounds good

Igor Matuszewski (Oct 01 2019 at 14:00, on Zulip):

wrt bigger IDE story - that's a fair point, I believe it'd be good for me to focus on that as well

Igor Matuszewski (Oct 01 2019 at 14:00, on Zulip):

I'll report back with any progress :)

nikomatsakis (Oct 01 2019 at 14:03, on Zulip):

I didn't read the VFS discussion in detail but it seemed like there might be something there

Igor Matuszewski (Oct 01 2019 at 14:03, on Zulip):

that'd be a good first step, yes :)

Igor Matuszewski (Oct 15 2019 at 13:48, on Zulip):

@nikomatsakis :wave:

nikomatsakis (Oct 15 2019 at 13:48, on Zulip):

Hey @Igor Matuszewski

Igor Matuszewski (Oct 15 2019 at 13:49, on Zulip):

Hey there

nikomatsakis (Oct 15 2019 at 13:49, on Zulip):

How are things?

Igor Matuszewski (Oct 15 2019 at 13:49, on Zulip):

I think the recent theme was fixing the fallout from the assertion promotion

Igor Matuszewski (Oct 15 2019 at 13:49, on Zulip):

https://github.com/rust-lang/rust/pull/65353

Igor Matuszewski (Oct 15 2019 at 13:50, on Zulip):

I feel the current approach is somewhat hacky wrt the nesting and the def/hir ids

Igor Matuszewski (Oct 15 2019 at 13:50, on Zulip):

but things seems to be working

nikomatsakis (Oct 15 2019 at 13:50, on Zulip):

hmm ok

Igor Matuszewski (Oct 15 2019 at 13:50, on Zulip):

this awaits your approval and I'd like to nominate this for beta, since the breakage hit quite a bit of deps on beta still

nikomatsakis (Oct 15 2019 at 13:50, on Zulip):

I hadn't seen that PR yet

nikomatsakis (Oct 15 2019 at 13:51, on Zulip):

the first commit seems fine

nikomatsakis (Oct 15 2019 at 13:51, on Zulip):

I'm not sure what nest_tables does

nikomatsakis (Oct 15 2019 at 13:51, on Zulip):

but that also "seems fine", I just have to look into the code a bit

Igor Matuszewski (Oct 15 2019 at 13:52, on Zulip):

the visitor has a 'current/local' view on the typeck tables, that's used to resolve a qpath

Igor Matuszewski (Oct 15 2019 at 13:53, on Zulip):

originally we nested the tables when visiting nested bodies to update that

Igor Matuszewski (Oct 15 2019 at 13:53, on Zulip):

but in this particular case we have a struct which doesn't have a body per se, but its member type has a definition defined under that struct instead

Igor Matuszewski (Oct 15 2019 at 13:54, on Zulip):

because it's a qpath, we tried previously to resolve that using the old/stale tables

Igor Matuszewski (Oct 15 2019 at 13:54, on Zulip):

so the new approach is rather than leaving the old tables, whenever we nest into a definition/body and can't find tables, we just use empty ones instead

nikomatsakis (Oct 15 2019 at 13:54, on Zulip):

ah ok

Igor Matuszewski (Oct 15 2019 at 13:54, on Zulip):

if that makes sense

nikomatsakis (Oct 15 2019 at 13:55, on Zulip):

so the ICE was arising because T::Assoc was being resolved relative to the tables from main, you're saying

Igor Matuszewski (Oct 15 2019 at 13:55, on Zulip):

(speaking specifically about the bug that unearthed this interaction)

nikomatsakis (Oct 15 2019 at 13:55, on Zulip):

and I guess there is some logic that's like "is there a key for this id"

nikomatsakis (Oct 15 2019 at 13:55, on Zulip):

or someting

nikomatsakis (Oct 15 2019 at 13:55, on Zulip):

that was going wrong,..somehow

Igor Matuszewski (Oct 15 2019 at 13:55, on Zulip):

yes

Igor Matuszewski (Oct 15 2019 at 13:55, on Zulip):

specifically the T::Assoc had a def path under struct, but we had tables for main

Igor Matuszewski (Oct 15 2019 at 13:56, on Zulip):

and since the IDs have (root, local_key) layout, it can easily provide to a key lookup mismatch by using a wrong local key (using a wrong point of reference, wrong def root)

Igor Matuszewski (Oct 15 2019 at 13:56, on Zulip):

that's specifically what the debug assertion was about, which was promoted recently to a regular one

nikomatsakis (Oct 15 2019 at 13:56, on Zulip):

ok, r+'d

Igor Matuszewski (Oct 15 2019 at 13:57, on Zulip):

great!

nikomatsakis (Oct 15 2019 at 13:57, on Zulip):

@Igor Matuszewski will you be at RBR? I think so, right?

Igor Matuszewski (Oct 15 2019 at 13:57, on Zulip):

yes

Igor Matuszewski (Oct 15 2019 at 13:58, on Zulip):

flying tomorrow

nikomatsakis (Oct 15 2019 at 13:58, on Zulip):

ok great

nikomatsakis (Oct 15 2019 at 13:58, on Zulip):

I think I gotta run early today

Igor Matuszewski (Oct 15 2019 at 13:58, on Zulip):

no worries

nikomatsakis (Oct 15 2019 at 13:58, on Zulip):

but we can catch up a bit there, too :)

nikomatsakis (Oct 15 2019 at 13:58, on Zulip):

I'll be there

Igor Matuszewski (Oct 15 2019 at 13:58, on Zulip):

I know, I'm speaking just before you :sweat_smile:

Igor Matuszewski (Oct 15 2019 at 13:59, on Zulip):

let's definitely do that, then

Igor Matuszewski (Oct 15 2019 at 13:59, on Zulip):

apart from that we published a 0.7 version of the extension which fixed an annoying warning (fixed in RLS 1.38) and we introduced extension-side support for project layout

Igor Matuszewski (Oct 15 2019 at 13:59, on Zulip):

I was w ondering if it'd be feasible to create a single Rust IDE extension but that's a topic for another time

Igor Matuszewski (Oct 15 2019 at 14:00, on Zulip):

and for completeness sake we tried to flesh out a potential rustc (v)fs design so the only thing left is to create and submit a patch

nikomatsakis (Oct 15 2019 at 14:00, on Zulip):

:)

Igor Matuszewski (Oct 15 2019 at 14:00, on Zulip):

(in terms of unified IDE effort)

Igor Matuszewski (Oct 15 2019 at 14:02, on Zulip):

okay then, let's catch up during the event!

Igor Matuszewski (Oct 29 2019 at 13:54, on Zulip):

@nikomatsakis so

nikomatsakis (Oct 29 2019 at 13:54, on Zulip):

:wave:

Igor Matuszewski (Oct 29 2019 at 13:54, on Zulip):

I'm currently working on https://github.com/rust-lang/rust/issues/65590 and I think I'll need some help

nikomatsakis (Oct 29 2019 at 13:55, on Zulip):

OK

Igor Matuszewski (Oct 29 2019 at 13:55, on Zulip):

this is yet another ICE unearthed by the HIR ID validation when resolving qualified paths

Igor Matuszewski (Oct 29 2019 at 13:55, on Zulip):

hopefully the last one

Igor Matuszewski (Oct 29 2019 at 13:56, on Zulip):

We queued a crater run at https://github.com/rust-lang/rust/issues/65590#issuecomment-547391047 to see if there is any other fallout we should be looking at

Igor Matuszewski (Oct 29 2019 at 13:56, on Zulip):

per @pnkfelix's suggestion I also tried to run our internal test suite with -Zsave-analysis locally but didn't find anything

Igor Matuszewski (Oct 29 2019 at 13:56, on Zulip):

but let's come back to the ICE in a second

nikomatsakis (Oct 29 2019 at 13:57, on Zulip):

OK. I see the problem has to do with the more "advanced" lowerings done by things like async blocks?

Igor Matuszewski (Oct 29 2019 at 13:57, on Zulip):

it seems so, from what I understand

nikomatsakis (Oct 29 2019 at 13:57, on Zulip):

is save-analysis walking the AST, or HIR?

Igor Matuszewski (Oct 29 2019 at 13:58, on Zulip):

AST

Igor Matuszewski (Oct 29 2019 at 13:58, on Zulip):

a simple reproduction is:

impl ... {
    async fn foo<T: Trait>(&self) -> T::Assoc { ... }
}
nikomatsakis (Oct 29 2019 at 13:58, on Zulip):

it requires the T: Trait I guess

nikomatsakis (Oct 29 2019 at 13:58, on Zulip):

and T::Assoc

Igor Matuszewski (Oct 29 2019 at 13:58, on Zulip):

when we encounter T::Assoc we are looking it up at typeck tables for foo, so to speak

nikomatsakis (Oct 29 2019 at 13:59, on Zulip):

Does this ICE occur with just -Zsave-analysis?

Igor Matuszewski (Oct 29 2019 at 13:59, on Zulip):

yes

nikomatsakis (Oct 29 2019 at 13:59, on Zulip):

OK, so we can easily create a standalone test case then

Igor Matuszewski (Oct 29 2019 at 13:59, on Zulip):
trait Trait { type Assoc; }

I omitted that for brevity

nikomatsakis (Oct 29 2019 at 13:59, on Zulip):

sure

Igor Matuszewski (Oct 29 2019 at 13:59, on Zulip):

yep, working on a fix right now but not sure how best to tackle it tbh

pnkfelix (Oct 29 2019 at 13:59, on Zulip):

/me wonders if there would be value in adding a -Z save-analysis output target to play.rust-lang.org

nikomatsakis (Oct 29 2019 at 14:00, on Zulip):

seems plausible

Igor Matuszewski (Oct 29 2019 at 14:00, on Zulip):

there's a question whether we should consider this format for stabilization

Igor Matuszewski (Oct 29 2019 at 14:00, on Zulip):

(if so)

nikomatsakis (Oct 29 2019 at 14:00, on Zulip):

but also, it seems like we lack any tests targeting -> T::Assoc in the async fn test base, eh?

nikomatsakis (Oct 29 2019 at 14:00, on Zulip):

(since you said that running all tests with -Zsave-analysis didn't turn up anything)

nikomatsakis (Oct 29 2019 at 14:00, on Zulip):

there's a question whether we should consider this format for stabilization

well, that seems like a distinct question

nikomatsakis (Oct 29 2019 at 14:01, on Zulip):

like, we can test without stabilizing

pnkfelix (Oct 29 2019 at 14:01, on Zulip):

(play.rlo has other stuff that only works on nightly, like WASM)

nikomatsakis (Oct 29 2019 at 14:01, on Zulip):

/me wonders if there would be value in adding a -Z save-analysis output target to play.rust-lang.org

oh, I misread this anyway, I thought you meant CI

nikomatsakis (Oct 29 2019 at 14:01, on Zulip):

as for the bug itself

nikomatsakis (Oct 29 2019 at 14:02, on Zulip):

I guess I don't know what best approach is but

Igor Matuszewski (Oct 29 2019 at 14:02, on Zulip):

there's Language Server Index Format (https://github.com/microsoft/language-server-protocol/issues/623) to consider as a "stable" API

Igor Matuszewski (Oct 29 2019 at 14:02, on Zulip):

but we can continue about that later I imagine

nikomatsakis (Oct 29 2019 at 14:02, on Zulip):

now thats interesting

nikomatsakis (Oct 29 2019 at 14:02, on Zulip):

yes

pnkfelix (Oct 29 2019 at 14:02, on Zulip):

oh, I misread this anyway, I thought you meant CI

/me is scared to add more passes to CI after seeing the dev reactions to NLL's compare-mode

nikomatsakis (Oct 29 2019 at 14:03, on Zulip):

could imagine it being a non-blocking thing, sort of like toolstate

nikomatsakis (Oct 29 2019 at 14:03, on Zulip):

anyway, point is, ICEs in save-analysis pretty clearly affect the RLS

nikomatsakis (Oct 29 2019 at 14:03, on Zulip):

let's not get too distracted tho

Igor Matuszewski (Oct 29 2019 at 14:03, on Zulip):

right now I'm trying a weird approach of checking if a function is async and if so, somehow try to get a parent id of body but that's not the case

Igor Matuszewski (Oct 29 2019 at 14:03, on Zulip):

maybe I'll have a better luck with actual parameter parent

nikomatsakis (Oct 29 2019 at 14:03, on Zulip):

I don't really understand the problem yet

nikomatsakis (Oct 29 2019 at 14:03, on Zulip):

can you say just a bit more ?

Igor Matuszewski (Oct 29 2019 at 14:03, on Zulip):

yes

nikomatsakis (Oct 29 2019 at 14:04, on Zulip):

(or maybe we should find another time to do a deeper technical dive?)

Igor Matuszewski (Oct 29 2019 at 14:04, on Zulip):

as a disclaimer this is what I presume is happening

Igor Matuszewski (Oct 29 2019 at 14:04, on Zulip):

well that's a good idea, although hopefully the fix is not so engaging

nikomatsakis (Oct 29 2019 at 14:04, on Zulip):

depends what else you have in mind

nikomatsakis (Oct 29 2019 at 14:04, on Zulip):

I do want to talk some about the eternal "IDE strategy" question -- was just discussing that with @pnkfelix a tiny bit

Igor Matuszewski (Oct 29 2019 at 14:05, on Zulip):

okay then, let's schedule the problem for later then

nikomatsakis (Oct 29 2019 at 14:05, on Zulip):

the only real thing I wanted to talk about with regard to IDE strategy was which of these sounded best to you

Igor Matuszewski (Oct 29 2019 at 14:05, on Zulip):

hopefully a crater run will tell us if we're missing something else than that (I plan to land the fix before the next beta/stable promotion)

Igor Matuszewski (Oct 29 2019 at 14:06, on Zulip):

do you mean what tool should we focus on (to simplify)?

nikomatsakis (Oct 29 2019 at 14:06, on Zulip):

I think we need to try and "map out" the various "high level" options -- basically what you and I are started doing -- and their pros/cons and (I think) examples of their implications, and then commit to a course

nikomatsakis (Oct 29 2019 at 14:07, on Zulip):

I am not sure how best to do this, it is hard :)

Igor Matuszewski (Oct 29 2019 at 14:07, on Zulip):

right

nikomatsakis (Oct 29 2019 at 14:07, on Zulip):

I think the first step might be to spend some time on this in a very small gruop, e.g. I could imagine felix and I trying to do it, to figure out the overall shape.

Igor Matuszewski (Oct 29 2019 at 14:07, on Zulip):

do you have something specific in mind now, in addition to what we started doing?

nikomatsakis (Oct 29 2019 at 14:08, on Zulip):

I think the next step would be discussing with you + @matklad

nikomatsakis (Oct 29 2019 at 14:08, on Zulip):

and maybe after that a wider design meeting?

Igor Matuszewski (Oct 29 2019 at 14:08, on Zulip):

sounds reasonable =)

nikomatsakis (Oct 29 2019 at 14:08, on Zulip):

do you have something specific in mind now, in addition to what we started doing?

not exactly, I think that was roughly in the right direction -- I think just trying to organize that information in a hackmd, and one thing I would specifically want to do is to evaluate different proposed bits of work against these plans, and maybe look at a few scenarios: like "what kind of RLS support are we doing here" etc

nikomatsakis (Oct 29 2019 at 14:09, on Zulip):

basically I think the more we can make the plans concrete in terms of what we would do or not do, the better

Igor Matuszewski (Oct 29 2019 at 14:09, on Zulip):

I think it'd be also great to have a more specific set of deliverables

nikomatsakis (Oct 29 2019 at 14:09, on Zulip):

but I don't think this has to be that horribly complex

Igor Matuszewski (Oct 29 2019 at 14:10, on Zulip):

in the sense that it'd be good to say, e.g. the goal of working on X is to separate parser data structures in rustc, or fully integrate Chalk or something

nikomatsakis (Oct 29 2019 at 14:10, on Zulip):

I was telling Felix that I think we should be aiming to have some form of decision by 2020, or very early therein

nikomatsakis (Oct 29 2019 at 14:10, on Zulip):

so that 2020's roadmap is to "execute"

Igor Matuszewski (Oct 29 2019 at 14:10, on Zulip):

if that makes sense

nikomatsakis (Oct 29 2019 at 14:10, on Zulip):

in the sense that it'd be good to say, e.g. the goal of working on X is to separate parser data structures in rustc, or fully integrate Chalk or something

incidentally I think that #wg-traits has the same problem :)

Igor Matuszewski (Oct 29 2019 at 14:10, on Zulip):

not to trap ourselves a bit in the "continuous improvement" (which is obviously good but it'd be great to ship in that sense)

nikomatsakis (Oct 29 2019 at 14:10, on Zulip):

i.e., the question of "to chalk or not to chalk" has a lot of the same feeling

nikomatsakis (Oct 29 2019 at 14:10, on Zulip):

not to trap ourselves a bit in the "continuous improvement" (which is obviously good but it'd be great to ship in that sense)

can you say what you mean by "continuous improvement"?

Igor Matuszewski (Oct 29 2019 at 14:11, on Zulip):

sorry if I'm being overly general :sweat_smile:

Igor Matuszewski (Oct 29 2019 at 14:11, on Zulip):

constantly working something and improving is good, but if we come up with a plan it'd be good for it to have a well-defined 'end' so to speak

Igor Matuszewski (Oct 29 2019 at 14:11, on Zulip):

like NLL migration work

nikomatsakis (Oct 29 2019 at 14:12, on Zulip):

ok ok

Igor Matuszewski (Oct 29 2019 at 14:12, on Zulip):

or e.g. librarification until now sharing the lexer

nikomatsakis (Oct 29 2019 at 14:12, on Zulip):

I think probably a good ingredient here is

nikomatsakis (Oct 29 2019 at 14:12, on Zulip):

"where can we realistically expect to be in a year" (say)

Igor Matuszewski (Oct 29 2019 at 14:12, on Zulip):

(to Chalk or not to Chalk is actually a good summary)

nikomatsakis (Oct 29 2019 at 14:12, on Zulip):

which sort of fits in

Igor Matuszewski (Oct 29 2019 at 14:13, on Zulip):

that's a good question

nikomatsakis (Oct 29 2019 at 14:14, on Zulip):

I think it's both useful to know on its own, but it probably also helps to define the "goals"

Igor Matuszewski (Oct 29 2019 at 14:14, on Zulip):

so we should think about that before coming up with a more fleshed out strategy?

Igor Matuszewski (Oct 29 2019 at 14:15, on Zulip):

it'd be good to analyze the landscape a bit and see what's missing/not great

nikomatsakis (Oct 29 2019 at 14:15, on Zulip):

hmm let's just do this :)

Igor Matuszewski (Oct 29 2019 at 14:15, on Zulip):

I know that compiler-backed completion isn't really done the way users would expect it to in either of the tools/models

nikomatsakis (Oct 29 2019 at 14:15, on Zulip):

when is a good time to start talking this out

nikomatsakis (Oct 29 2019 at 14:15, on Zulip):

/me thinks about his own calendar

nikomatsakis (Oct 29 2019 at 14:16, on Zulip):

I know that compiler-backed completion isn't really done the way users would expect it to in either of the tools/models

say a bit more about that?

Igor Matuszewski (Oct 29 2019 at 14:16, on Zulip):

I have to admit I don't have exact knowledge on the RA side of the things

Igor Matuszewski (Oct 29 2019 at 14:16, on Zulip):

but Racer is lacking (simplistic type system), RLS doesn't do it

nikomatsakis (Oct 29 2019 at 14:17, on Zulip):

ok ok

Igor Matuszewski (Oct 29 2019 at 14:17, on Zulip):

and I think just having a bulletproof completion, following through type inference, derefs, trait resolution would be very useful for the end users

Igor Matuszewski (Oct 29 2019 at 14:18, on Zulip):

hm, so I wanted to lay out a couple of perspectives such as project support, "indexing" features but I'm not sure if that's a good idea

Igor Matuszewski (Oct 29 2019 at 14:18, on Zulip):

maybe another perspective would be how convenient is getting all the tooling for the end-user

Igor Matuszewski (Oct 29 2019 at 14:20, on Zulip):

sorry, in the weeds here

nikomatsakis (Oct 29 2019 at 14:20, on Zulip):

those are useful details

nikomatsakis (Oct 29 2019 at 14:21, on Zulip):

ok, so, I've created a hackmd -- I'm going to schedule some time to try and transcribe our notes from rustbeltrust and see how I feel.

Igor Matuszewski (Oct 29 2019 at 14:21, on Zulip):

sure thing

nikomatsakis (Oct 29 2019 at 14:21, on Zulip):

feel free to add some notes in there yourself

Igor Matuszewski (Oct 29 2019 at 14:21, on Zulip):

@David Tolnay also mentioned that it if a language feature is harder to get a proper IDE support it might be worth warning against

nikomatsakis (Oct 29 2019 at 14:21, on Zulip):

e.g. the above content

Igor Matuszewski (Oct 29 2019 at 14:22, on Zulip):

one such example might be a deeply nested mod file statements or impl blocks inside function bodies that affect global resolution

nikomatsakis (Oct 29 2019 at 14:22, on Zulip):

indeed

Igor Matuszewski (Oct 29 2019 at 14:22, on Zulip):

maybe it'd be good to lay down a tiered support model

Igor Matuszewski (Oct 29 2019 at 14:22, on Zulip):

e.g. "we will work on fully supporting regular Rust without X,Y,Z"

Igor Matuszewski (Oct 29 2019 at 14:23, on Zulip):

once that's somewhat done, we could move on to gradually work on and polish other features

Igor Matuszewski (Oct 29 2019 at 14:23, on Zulip):

maybe that's also connected in general to what people want to see from their tooling rather than just blindly trying to support everything or supporting fully a very specific subset

Igor Matuszewski (Oct 29 2019 at 14:24, on Zulip):

will take a look at the doc

Igor Matuszewski (Oct 29 2019 at 14:26, on Zulip):

I think the main idea was to split the data and logic so you could specialize over tool-specific structures to optimize for a specific use case? (batch compilation vs ide)

Igor Matuszewski (Oct 29 2019 at 14:26, on Zulip):

/me trying not to forget all of what we talked about at rustbeltrust

matklad (Oct 29 2019 at 14:31, on Zulip):

I am semi-here, will catch up on the discussion later!

matklad (Oct 29 2019 at 16:00, on Zulip):

Yep, answering the IDE question would be great! I feel like between various meeting we had we've covered the design space pretty thoroughly (in partciular, these notes: https://hackmd.io/S5wub4mfToSnnLGvK3I-ew?edit). So, we probably might want place emphasis on picking a concrete actionable tactical plan

Igor Matuszewski (Nov 12 2019 at 14:46, on Zulip):

@nikomatsakis :wave:

nikomatsakis (Nov 12 2019 at 14:47, on Zulip):

@Igor Matuszewski :wave:

Igor Matuszewski (Nov 12 2019 at 14:47, on Zulip):

So recently we fixed hopefully the last ICE that I mentioned during our last meeting

Igor Matuszewski (Nov 12 2019 at 14:48, on Zulip):

with the associated type in impl trait in async lowering

Igor Matuszewski (Nov 12 2019 at 14:48, on Zulip):

and it seems that was the last of it now :tada:

Igor Matuszewski (Nov 12 2019 at 14:48, on Zulip):

before the release cut, so that's good news

nikomatsakis (Nov 12 2019 at 14:49, on Zulip):

That's good :)

Igor Matuszewski (Nov 12 2019 at 14:49, on Zulip):

in terms of unifying IDE effort I was wondering if we can at least unify the VS Code extension for RLS/RA

Igor Matuszewski (Nov 12 2019 at 14:49, on Zulip):

and we talked briefly with @matklad about it

Igor Matuszewski (Nov 12 2019 at 14:49, on Zulip):

but the concern was whether RA is of "production quality" which I believe leans a bit more into the discussion you had

Igor Matuszewski (Nov 12 2019 at 14:49, on Zulip):

with Felix as well

Igor Matuszewski (Nov 12 2019 at 14:50, on Zulip):

so I'm interesting if it touches on that anyhow

nikomatsakis (Nov 12 2019 at 14:50, on Zulip):

Good question

nikomatsakis (Nov 12 2019 at 14:50, on Zulip):

I was just talking to @pnkfelix

nikomatsakis (Nov 12 2019 at 14:50, on Zulip):

I want to try and pull those notes together into something a bit more "approachable" but I think my main takeaway from the conversation with @matklad was

nikomatsakis (Nov 12 2019 at 14:50, on Zulip):

Well we kind of landed on two alternatives, and explored them somewhat:

nikomatsakis (Nov 12 2019 at 14:51, on Zulip):
nikomatsakis (Nov 12 2019 at 14:51, on Zulip):

I feel pretty persuaded that, given where we are now, the first path makes more sense

nikomatsakis (Nov 12 2019 at 14:51, on Zulip):

Among other things, rust-analyzer is delivering value to users now, and basically each successful library-ification will increase that value

nikomatsakis (Nov 12 2019 at 14:51, on Zulip):

However, there remain some open questions

nikomatsakis (Nov 12 2019 at 14:52, on Zulip):

and I think the biggest one is how rust-analyzer and RLS "relate"

nikomatsakis (Nov 12 2019 at 14:52, on Zulip):

I'm not sure I fully understand the range of choices here

nikomatsakis (Nov 12 2019 at 14:52, on Zulip):

I had in mind two options, but you just raised an (appealing) third

nikomatsakis (Nov 12 2019 at 14:52, on Zulip):

I was thinking of

nikomatsakis (Nov 12 2019 at 14:53, on Zulip):
nikomatsakis (Nov 12 2019 at 14:53, on Zulip):

or

nikomatsakis (Nov 12 2019 at 14:53, on Zulip):
nikomatsakis (Nov 12 2019 at 14:53, on Zulip):

I am not sure I understand what is required to do the second or under what conditions it makes sense

nikomatsakis (Nov 12 2019 at 14:53, on Zulip):

I do think it would be good to see if we can create some metrics to help drive our effort, whichever we do

nikomatsakis (Nov 12 2019 at 14:54, on Zulip):

I am also open that these two options are overly "rust-analyzer" centric

nikomatsakis (Nov 12 2019 at 14:54, on Zulip):

in any case you raised an interesting third option

Igor Matuszewski (Nov 12 2019 at 14:55, on Zulip):

what do you specifically refer to by a third option?

nikomatsakis (Nov 12 2019 at 14:55, on Zulip):

I do think it would be good to see if we can create some metrics to help drive our effort, whichever we do

(to circle back to this slightly, I remember that we had some controversy around whether RLS was "ready" to be 1.0 -- I think part of the problem there was we never did much work in establishing a set of projects and functionality that we deemed "sufficient"? maybe? I'm also worried about wasting time building up definitions that won't in reality be of much use)

nikomatsakis (Nov 12 2019 at 14:55, on Zulip):

what do you specifically refer to by a third option?

sorry, what I meant is

nikomatsakis (Nov 12 2019 at 14:55, on Zulip):
nikomatsakis (Nov 12 2019 at 14:55, on Zulip):

it's not really a third option on its own

Igor Matuszewski (Nov 12 2019 at 14:55, on Zulip):

just a separate part of it, rather, right

nikomatsakis (Nov 12 2019 at 14:55, on Zulip):

but it might be an aspect of the final choice I guess

nikomatsakis (Nov 12 2019 at 14:56, on Zulip):

my concern with trying to have rust-analyzer play the role of racer is that it's .. kind of too complex and heavy-weight for that role, maybe?

Igor Matuszewski (Nov 12 2019 at 14:56, on Zulip):

so anything in the long run that's beneficial for the user is the best for us

nikomatsakis (Nov 12 2019 at 14:56, on Zulip):

I worry about memory usage, etc, and basically just creating more complexity than we solve

Igor Matuszewski (Nov 12 2019 at 14:57, on Zulip):

Yes, memory usage was of a primary concern from what I understand that @matklad told about the performance overhead of the RA

Igor Matuszewski (Nov 12 2019 at 14:57, on Zulip):

having a unified front in some form, like the extension is good

Igor Matuszewski (Nov 12 2019 at 14:57, on Zulip):

but the problem is that it's only there for a single editor

nikomatsakis (Nov 12 2019 at 14:57, on Zulip):

what do you .. oh, like VSCode

nikomatsakis (Nov 12 2019 at 14:58, on Zulip):

I see

Igor Matuszewski (Nov 12 2019 at 14:58, on Zulip):

and doing this for two editor-agnostic tools seems somewhat counterintuitive (although it might be productive seeing as vscode is one of the most popular ones)

nikomatsakis (Nov 12 2019 at 14:58, on Zulip):

because the other integrations are not managed by us

nikomatsakis (Nov 12 2019 at 14:58, on Zulip):

is there some way to do it at the "LSP" level?

Igor Matuszewski (Nov 12 2019 at 14:58, on Zulip):

huh

Igor Matuszewski (Nov 12 2019 at 14:58, on Zulip):

we could in theory somewhat unify that

nikomatsakis (Nov 12 2019 at 14:58, on Zulip):

I don't really understand the division of responsibilities between the plugin and the LSP

Igor Matuszewski (Nov 12 2019 at 14:58, on Zulip):

but the deeper we go in this direction the quicker we approach the RA-in-RLS approach

Igor Matuszewski (Nov 12 2019 at 14:59, on Zulip):

right now our VSCode plugin does VS Code workspace detection

nikomatsakis (Nov 12 2019 at 14:59, on Zulip):

maybe the real question is: in what way would adopting rust-analyzer seem a "regression" for end-users vs RLS? One thing I know is precision of "Go To Definition"

Igor Matuszewski (Nov 12 2019 at 14:59, on Zulip):

and delegates where and how to spawn rls instances for the relevant folders, it synchronizes the configuration and it's capable of installing the Rust toolchains (and Rust as well)

nikomatsakis (Nov 12 2019 at 15:00, on Zulip):

Actually, let me back up, I want to ask another question

Igor Matuszewski (Nov 12 2019 at 15:00, on Zulip):

FWIU find all references does not work in RA

nikomatsakis (Nov 12 2019 at 15:00, on Zulip):

(I think that may have recently changed)

Igor Matuszewski (Nov 12 2019 at 15:00, on Zulip):

go to definition might be precise but sometimes we don't have total knowledge and that's there the heuristical approach works somewhat good (where we use Racer for it now)

nikomatsakis (Nov 12 2019 at 15:00, on Zulip):

How much work is it to "maintain" RLS in its current form, without making architectural changes?

nikomatsakis (Nov 12 2019 at 15:00, on Zulip):

It seems like we have had a reasonably steady stream of ICEs

nikomatsakis (Nov 12 2019 at 15:01, on Zulip):

But those may be linked to other refactorings?

nikomatsakis (Nov 12 2019 at 15:01, on Zulip):

I forget

Igor Matuszewski (Nov 12 2019 at 15:01, on Zulip):

oh that was due to the change of save-analysis - it seems that was caused by doing the wrong thing and having only a debug assertion

Igor Matuszewski (Nov 12 2019 at 15:01, on Zulip):

one bug report popped up, we fixed that and promoted the assertion but this in turn unearthed a couple of different bad interactions in the form of ICEs

Igor Matuszewski (Nov 12 2019 at 15:02, on Zulip):

so it was save-analysis refactoring purely

Igor Matuszewski (Nov 12 2019 at 15:02, on Zulip):

we could considerably do the other integration, so try to embed RLS in RA

Igor Matuszewski (Nov 12 2019 at 15:02, on Zulip):

in the sense that RA would accept a save-analysis files and use the existing infrastructure to lower all that into a single interface

nikomatsakis (Nov 12 2019 at 15:03, on Zulip):

( regarding find usages in rust-analyzer, see this comment from @matklad )

Igor Matuszewski (Nov 12 2019 at 15:03, on Zulip):

Oh, that's great to hear!

nikomatsakis (Nov 12 2019 at 15:03, on Zulip):

that's true. I am not sure how plausible that is (rust-analyzer integrating save analysis). It is conceivable at least.

Igor Matuszewski (Nov 12 2019 at 15:03, on Zulip):

it seemed like the most obvious "blocker"

nikomatsakis (Nov 12 2019 at 15:04, on Zulip):

I guess what I'm wondering is

Igor Matuszewski (Nov 12 2019 at 15:04, on Zulip):

by embedding save-analysis support in the RA we could potentially get the quick and alzy approach of RA but where we're concerned we could get the data dump as done by rustc exactly

nikomatsakis (Nov 12 2019 at 15:05, on Zulip):

if we opted to put "all our energies" into rust-analyzer, while keeping RLS as the deprecated, but usable option in the meantime, how much work would it be to maintain it. My hope would be that you would be able to instead to come up to speed on rust-analyzer and work on pushing that to the point where it can be used instead.

Igor Matuszewski (Nov 12 2019 at 15:05, on Zulip):

How much work is it to "maintain" RLS in its current form, without making architectural changes?

To come back to that, the save-analysis regressions were unrelated to the RLS itself and so maintaining what we have in the current form is not that time-consuming

nikomatsakis (Nov 12 2019 at 15:05, on Zulip):

yeah, it's an interesting thought

nikomatsakis (Nov 12 2019 at 15:05, on Zulip):

This seems like it would be worth trying to have a focused meeting with you, @matklad, plus maybe myself + @pnkfelix

Igor Matuszewski (Nov 12 2019 at 15:06, on Zulip):

definitely

matklad (Nov 12 2019 at 15:06, on Zulip):

+1

Igor Matuszewski (Nov 12 2019 at 15:06, on Zulip):

personally I'd feel comfortable if we try to polish and improve the shared library approach

Igor Matuszewski (Nov 12 2019 at 15:06, on Zulip):

I think that's the best way to improve two sides at the same time

Igor Matuszewski (Nov 12 2019 at 15:07, on Zulip):

rather than just writing another compiler, if that makes sense

nikomatsakis (Nov 12 2019 at 15:07, on Zulip):

personally I'd feel comfortable if we try to polish and improve the shared library approach

"libraryification"?

Igor Matuszewski (Nov 12 2019 at 15:07, on Zulip):

to do so, I think it'd be good to gauge the user experience

Igor Matuszewski (Nov 12 2019 at 15:07, on Zulip):

personally I'd feel comfortable if we try to polish and improve the shared library approach

"libraryification"?

Yes, as I understand it and what you proposed as the "1st approach" and what's been happening with lexer for example

nikomatsakis (Nov 12 2019 at 15:07, on Zulip):

in other words, that's what you meant by "shared library", right?

nikomatsakis (Nov 12 2019 at 15:08, on Zulip):

OK

Igor Matuszewski (Nov 12 2019 at 15:08, on Zulip):

so if we were to do the switch

Igor Matuszewski (Nov 12 2019 at 15:08, on Zulip):

it'd be good to see what RLS features RA might be lacking and the other way around

Igor Matuszewski (Nov 12 2019 at 15:09, on Zulip):

to promote around the better parts (quick, lazy?)

Igor Matuszewski (Nov 12 2019 at 15:09, on Zulip):

and to see what we might want to prioritize on the RA side

Igor Matuszewski (Nov 12 2019 at 15:09, on Zulip):

to provide parity

Igor Matuszewski (Nov 12 2019 at 15:09, on Zulip):

if that makes sense

Igor Matuszewski (Nov 12 2019 at 15:10, on Zulip):

or rather, to smoothen the transition

Igor Matuszewski (Nov 12 2019 at 15:10, on Zulip):

On that note, I agree with @matklad general sentiment that save-analysis is useful for the offline code indexing

Igor Matuszewski (Nov 12 2019 at 15:11, on Zulip):

so it'd be good to not fully deprecate/remove RLS as it's a convenient tool to drive the generation of SA

Igor Matuszewski (Nov 12 2019 at 15:11, on Zulip):

and from that we might consider following the LSIF and possibly power the code smartness in the GitHub

Igor Matuszewski (Nov 12 2019 at 15:11, on Zulip):

which I think would be valuable and very marketable

Igor Matuszewski (Nov 12 2019 at 15:11, on Zulip):

but I'm getting a little off-track :sweat_smile:

matklad (Nov 12 2019 at 15:12, on Zulip):

Hm, I guess, if we implement rename refactoring via save-analysis in rust-analyzer, than ra can keep sa from bitrotting?

nikomatsakis (Nov 12 2019 at 15:13, on Zulip):

well, I think the LSIF (this is the offline language server stuff?) is really interesting

Igor Matuszewski (Nov 12 2019 at 15:13, on Zulip):

yeah that's one usage/approach

nikomatsakis (Nov 12 2019 at 15:13, on Zulip):

one of my concerns with save-analysis as is is that it is just kind our own one-off thing

Igor Matuszewski (Nov 12 2019 at 15:13, on Zulip):

well, I think the LSIF (this is the offline language server stuff?) is really interesting

It is, it's basically save-analysis-by-microsoft

Igor Matuszewski (Nov 12 2019 at 15:13, on Zulip):

very oversimplifying

Igor Matuszewski (Nov 12 2019 at 15:13, on Zulip):

oh about that, it'd be good to also consult the Mozilla engineers working on searchfox

Igor Matuszewski (Nov 12 2019 at 15:13, on Zulip):

since they rely on SA for the indexer

nikomatsakis (Nov 12 2019 at 15:14, on Zulip):

yep

nikomatsakis (Nov 12 2019 at 15:14, on Zulip):

so

nikomatsakis (Nov 12 2019 at 15:14, on Zulip):

let's schedule a meeting but

nikomatsakis (Nov 12 2019 at 15:14, on Zulip):

I was wondering what would be a good "prep work" to do

nikomatsakis (Nov 12 2019 at 15:14, on Zulip):

is enumerating the set of "features" a useful thing to do, to aid in comparison?

Igor Matuszewski (Nov 12 2019 at 15:14, on Zulip):

and other parties possibly interested, like @est31 ? I know they consumed the save-analysis format so it'd be good to gauge how the data is useful in the current format

nikomatsakis (Nov 12 2019 at 15:14, on Zulip):

it'd be good to see what RLS features RA might be lacking and the other way around

(I'm thinking about this question that @Igor Matuszewski raised)

Igor Matuszewski (Nov 12 2019 at 15:15, on Zulip):

a simple side-by-side pros/cons would be good to do, wrt to above ^

Igor Matuszewski (Nov 12 2019 at 15:15, on Zulip):

as part of prep work

Igor Matuszewski (Nov 12 2019 at 15:16, on Zulip):

possibly first by going over the LSP methods to get a good idea and then filling with external stuff, like project detection/integration?

Igor Matuszewski (Nov 12 2019 at 15:16, on Zulip):

@nikomatsakis do you have a hackmd document handy?

nikomatsakis (Nov 12 2019 at 15:18, on Zulip):

I'm trying to find it

Igor Matuszewski (Nov 12 2019 at 15:18, on Zulip):

https://hackmd.io/9qrJYdSLTK2MWG4-pbmpWw

Igor Matuszewski (Nov 12 2019 at 15:18, on Zulip):

we could use this

nikomatsakis (Nov 12 2019 at 15:18, on Zulip):

too many documents :)

nikomatsakis (Nov 12 2019 at 15:19, on Zulip):

https://hackmd.io/66OlBzy-RyW0YdXuUyJkuQ

nikomatsakis (Nov 12 2019 at 15:19, on Zulip):

OK, let's link them all together :)

Igor Matuszewski (Nov 12 2019 at 15:19, on Zulip):

oh whoops

Igor Matuszewski (Nov 12 2019 at 15:19, on Zulip):

let's use the one you linked then

nikomatsakis (Nov 12 2019 at 15:20, on Zulip):

actually I disagree :)

nikomatsakis (Nov 12 2019 at 15:20, on Zulip):

https://hackmd.io/9qrJYdSLTK2MWG4-pbmpWw

I like the title of this one

nikomatsakis (Nov 12 2019 at 15:20, on Zulip):

let's use that as the "master doc"

nikomatsakis (Nov 12 2019 at 15:20, on Zulip):

and I'll link other things into it

Igor Matuszewski (Nov 12 2019 at 15:29, on Zulip):

These are great metics and notes, thanks @nikomatsakis and @matklad !

nikomatsakis (Nov 12 2019 at 15:31, on Zulip):

OK @Igor Matuszewski @matklad @pnkfelix I created a doodle poll to schedule the next discussion on this topic

nikomatsakis (Nov 12 2019 at 15:32, on Zulip):

I'm going to extract those notes, @Igor Matuszewski, into a fresh hackmd, and link from that main document

Igor Matuszewski (Nov 12 2019 at 15:37, on Zulip):

I'm on the move and still in Barcelona, so including prep work the next week would work best for me

nikomatsakis (Nov 12 2019 at 15:38, on Zulip):

OK, looks like Mon at 9am (Boston time) is maybe best for everyone?

Igor Matuszewski (Nov 12 2019 at 15:39, on Zulip):

@nikomatsakis there's a full green column on the 20th

Igor Matuszewski (Nov 12 2019 at 15:40, on Zulip):

(because I think the one you're referring to is "Yes, if need be" instead?)

nikomatsakis (Nov 12 2019 at 15:40, on Zulip):

True. I'm ok with either, but earlier might be better

Igor Matuszewski (Nov 12 2019 at 15:40, on Zulip):

either is fine by me, obviously

Igor Matuszewski (Nov 12 2019 at 15:45, on Zulip):

so finally Mon 9am (Boston time)?

nikomatsakis (Nov 12 2019 at 15:57, on Zulip):

let's do monday

nikomatsakis (Nov 12 2019 at 15:58, on Zulip):

I'll make a calendar invite (ps, I'd like to keep the meeting small for max productivity, but not trying to exclude anyone)

nikomatsakis (Nov 12 2019 at 15:59, on Zulip):

Done

matklad (Nov 18 2019 at 14:01, on Zulip):

@nikomatsakis @pnkfelix @Igor Matuszewski ping

pnkfelix (Nov 18 2019 at 14:02, on Zulip):

ah on my way

Igor Matuszewski (Nov 18 2019 at 14:03, on Zulip):

am here

Igor Matuszewski (Nov 18 2019 at 14:04, on Zulip):

https://hackmd.io/fAnj6pNqRRGIyDQ4el5tcQ

Last update: Nov 22 2019 at 04:50UTC