Stream: t-compiler/wg-rls-2.0

Topic: weekly sync-up


matklad (Sep 03 2019 at 12:20, on Zulip):

:wave: @nikomatsakis

nikomatsakis (Sep 03 2019 at 12:21, on Zulip):

So, yeah, last Friday I spent some time reading into Chalk and (somewhat) into the RLS

nikomatsakis (Sep 03 2019 at 12:21, on Zulip):

Trying to remember the lay of the land

nikomatsakis (Sep 03 2019 at 12:22, on Zulip):

I came up with a number of improvements and steps that we could take in/around Chalk, but they're kind of low-level specifics.

nikomatsakis (Sep 03 2019 at 12:23, on Zulip):

I guess it's worth stepping back a bit to review our overall goals over this week etc

matklad (Sep 03 2019 at 12:23, on Zulip):

I'd say the high order bit right now is fixing the "chalk spins CPU at 100%" problem

nikomatsakis (Sep 03 2019 at 12:24, on Zulip):

Well, depends I guess ;)

matklad (Sep 03 2019 at 12:24, on Zulip):

Yeah, reviewing goals seems right, but let me quickly note that happend on the ra side recently

nikomatsakis (Sep 03 2019 at 12:24, on Zulip):

go for it

matklad (Sep 03 2019 at 12:25, on Zulip):
matklad (Sep 03 2019 at 12:26, on Zulip):

The last bit is interesting, in that I think starts are aligned perfectly to make us reconsider file watching and general interractions with file system. I'd love to transition to lazy file loading

matklad (Sep 03 2019 at 12:26, on Zulip):

(hence that question in salsa's zulip)

nikomatsakis (Sep 03 2019 at 12:26, on Zulip):

Yeah, that is interesting

matklad (Sep 03 2019 at 12:27, on Zulip):

oh, and, on rustc side, I do first small steps to make rustc parser use the same token model as proc-macros use (with jointness)

matklad (Sep 03 2019 at 12:27, on Zulip):

that's all I can remember I guess?

nikomatsakis (Sep 03 2019 at 12:28, on Zulip):

So I guess one question is -- we had planned to have a compiler team meeting. One of my immediate goals was to .. figure out what to be discussing there.

nikomatsakis (Sep 03 2019 at 12:29, on Zulip):

I guess I can see a few things to do

matklad (Sep 03 2019 at 12:30, on Zulip):

Say more?

nikomatsakis (Sep 03 2019 at 12:30, on Zulip):

Well I don't think we're going to be in a position to propose a detailed technical propsal around chalk -- but we could focus on improving the experience in rust-analyzer and try to give an overview of where it is and discuss the idea of trying to extract chalk/type-checker into shared library, and maybe float out some of the challenges there.

nikomatsakis (Sep 03 2019 at 12:31, on Zulip):

I was vacillating about how much it makes sense to try and propose specific technical designs

nikomatsakis (Sep 03 2019 at 12:31, on Zulip):

But I think the answer is probably not at all relatively little, at least not yet

matklad (Sep 03 2019 at 12:33, on Zulip):

I am not sure: I think, at least in general terms, everyone is on board with the idea. So, I think it would be valuable to:

nikomatsakis (Sep 03 2019 at 12:33, on Zulip):

I'd say the high order bit right now is fixing the "chalk spins CPU at 100%" problem

On this topic, one thing I was thinking about was that it'd be nice to have a mode for RA (or chalk, perhaps) where it can basically dump out a self-contained foo.chalk file that tracks the goal(s) it is solving, so that we can easily reproduce these problems and isolate them

matklad (Sep 03 2019 at 12:34, on Zulip):

Would that really be more useful than dumping the rust source-code?

nikomatsakis (Sep 03 2019 at 12:35, on Zulip):

seems like yes

nikomatsakis (Sep 03 2019 at 12:35, on Zulip):

it could be minimzed more readily, I think, and it's smaller than the rust source code

matklad (Sep 03 2019 at 12:35, on Zulip):

Like, I think it doesn't really help with isolation: a small source file and a small goal look pretty equivalent to me? You can dbg!(goal) at the entry to chalk to see the actual goal.

nikomatsakis (Sep 03 2019 at 12:36, on Zulip):

it's more about the "environment" around the goal

matklad (Sep 03 2019 at 12:36, on Zulip):

And for minimization the biggest problem seems to the the envirionment with a lot of traits, and minimizing that seems not easy either way?

nikomatsakis (Sep 03 2019 at 12:36, on Zulip):

right now you have to debug that from within rust-analyzer

nikomatsakis (Sep 03 2019 at 12:37, on Zulip):

I'd like to be able to create stronger boundaries, so that you can debug and test chalk problems without really knowing much about rust-analyzer

nikomatsakis (Sep 03 2019 at 12:37, on Zulip):

I feel like that's some of the promise of library-ification is to enable these sorts of boundaries

nikomatsakis (Sep 03 2019 at 12:37, on Zulip):

(speaking as someone who feels "uncomfortable" in the r-a code base, I can attest it'd be nice if there were some standalone file I could load up into chalk)

matklad (Sep 03 2019 at 12:37, on Zulip):

that's true :)

nikomatsakis (Sep 03 2019 at 12:38, on Zulip):

that said I agree it's maybe more of a "side goal"

Florian Diebold (Sep 03 2019 at 12:38, on Zulip):

One thing to note about the Chalk-spinning-problem is that we're currently still producing a lot of 'broken' clauses (because of name resolution bugs or missing features), so in the cases I've looked into the problem seemed to me to be that it was enumerating solutions for some subgoal where in the end the final goal had no solution. My point being, maybe these problems aren't really representative of 'real' situations, where usually only a few clauses/impls are broken like that :)

nikomatsakis (Sep 03 2019 at 12:38, on Zulip):

I am not sure: I think, at least in general terms, everyone is on board with the idea. So, I think it would be valuable to:

in any case you're probably right about this

matklad (Sep 03 2019 at 12:38, on Zulip):

Actually, I think figuring out such things should be the primary goal !

matklad (Sep 03 2019 at 12:39, on Zulip):

It seems like retrofit such inspectability would be hard after the fact

matklad (Sep 03 2019 at 12:39, on Zulip):

the main problem I see here is that we want pull-based interface from chalk: there isn't a "world" that we can just serde_json::to_string, we have a bunch of callbacks...

nikomatsakis (Sep 03 2019 at 12:40, on Zulip):

well, chalk pulls info from r-a;

nikomatsakis (Sep 03 2019 at 12:40, on Zulip):

I was imagining that a given engine incarnation would gradually dump the info it pulls as it pulls it into a file

nikomatsakis (Sep 03 2019 at 12:40, on Zulip):

along with goals

nikomatsakis (Sep 03 2019 at 12:40, on Zulip):

so you could kind of load that up and "replay"

nikomatsakis (Sep 03 2019 at 12:40, on Zulip):

so, a very relevant question -- I think recently you were asking @matklad about integration vs unit tests

nikomatsakis (Sep 03 2019 at 12:41, on Zulip):

I guess a real question is

nikomatsakis (Sep 03 2019 at 12:41, on Zulip):

suppose we encountered some bug in chalk via rust-analyzer, would we want to (as I'm vagualy proposing) extract out a .chalk file and add that as a unit test to chalk,

nikomatsakis (Sep 03 2019 at 12:41, on Zulip):

or try to reduce it to Rust source

nikomatsakis (Sep 03 2019 at 12:41, on Zulip):

I think in general I tend towards the latter because it's more "canonical"

nikomatsakis (Sep 03 2019 at 12:42, on Zulip):

but I think in some ideal world we'd have relatively robust, Rust-like interfaces

nikomatsakis (Sep 03 2019 at 12:43, on Zulip):

anyway, that's kind of a bigger scope than I was initially imagining :) beacuse you wouldn't want like serialized chalk data structures, but some Rust-subset or something that captures the problem setup I guess?

nikomatsakis (Sep 03 2019 at 12:43, on Zulip):

plausible, just more complex

matklad (Sep 03 2019 at 12:44, on Zulip):

So, I think what we really want for tests is "data in, data out" setup

matklad (Sep 03 2019 at 12:44, on Zulip):

Boundary which is not data, but is an interface (callbacks, etc) is much harder to keep stable

nikomatsakis (Sep 03 2019 at 12:45, on Zulip):

One thing to note about the Chalk-spinning-problem is that we're currently still producing a lot of 'broken' clauses ...

interesting, btw, I hadn't realize that.

nikomatsakis (Sep 03 2019 at 12:45, on Zulip):

So, I think what we really want for tests is "data in, data out" setup

Yes for sure this

nikomatsakis (Sep 03 2019 at 12:45, on Zulip):

And I think also that the "data in" has to be pretty high-level

matklad (Sep 03 2019 at 12:45, on Zulip):

So, for thinkgs like lexing or parsing where the input and output data has simple format, unit-tests and this boundary seem OK

nikomatsakis (Sep 03 2019 at 12:45, on Zulip):

Something like a .chalk file is actually pretty plausible (i.e., some subset of Rust syntax)

matklad (Sep 03 2019 at 12:46, on Zulip):

Seems so

nikomatsakis (Sep 03 2019 at 12:47, on Zulip):

It seems like this should be part of the "things we figure out" when deciding on a good library boundary

matklad (Sep 03 2019 at 12:47, on Zulip):

(BTW, I like how Swift .SIL is a real well-defined intermediate langauge which you can read from string)

nikomatsakis (Sep 03 2019 at 12:47, on Zulip):

I'd like to see MIR becoming comparably stable (and spec'd)

nikomatsakis (Sep 03 2019 at 12:47, on Zulip):

In any case, to circle back a bit, re: chalk spinning and being slow

nikomatsakis (Sep 03 2019 at 12:48, on Zulip):

So one thing that @Florian Diebold pointed out to me was issues around normalization; I have specific thoughts on those, but I'll try to write them out in chalk issues / on that channel

nikomatsakis (Sep 03 2019 at 12:48, on Zulip):

But more deeply I am wondering about two options:

nikomatsakis (Sep 03 2019 at 12:48, on Zulip):
nikomatsakis (Sep 03 2019 at 12:49, on Zulip):

I think for fuel (as I wrote somewhere) what I'd want to do is to refactor chalk's loop to not do looping, so that fuel can be enforced more at the r-a level (stop pulling for answers after enough iterations)

nikomatsakis (Sep 03 2019 at 12:49, on Zulip):

(though @Florian Diebold had some concerns, we can discuss that a bit separately)

nikomatsakis (Sep 03 2019 at 12:50, on Zulip):

the main question is: prioritize the general mechanism first? or solve the specific bugs?

nikomatsakis (Sep 03 2019 at 12:50, on Zulip):

it feels like the general mechanism will be needed at the end of the day no matter what

matklad (Sep 03 2019 at 12:50, on Zulip):

Specific bugs should have priority

matklad (Sep 03 2019 at 12:50, on Zulip):

fuel can too easily mask bugs

nikomatsakis (Sep 03 2019 at 12:50, on Zulip):

Interesting

nikomatsakis (Sep 03 2019 at 12:51, on Zulip):

fuel can too easily mask bugs

yeah so this is what I was wrestling back and forth with :)

Florian Diebold (Sep 03 2019 at 12:54, on Zulip):

(though Florian Diebold had some concerns, we can discuss that a bit separately)

not so much concerns as, I couldn't figure out a good way to do it ;)

matklad (Sep 03 2019 at 12:54, on Zulip):

My point being, maybe these problems aren't really representative of 'real' situations, where usually only a few clauses/impls are broken like that :)

It still would be nice if chalk degraded gracefully in such cases. Like, if you mistyped a dependency name in Cargo.toml you still can get a lot of broken code even for "normal" projects

Florian Diebold (Sep 03 2019 at 12:55, on Zulip):

yeah, it would probably be fine if that was caught by fuel though

matklad (Sep 03 2019 at 12:58, on Zulip):

fine, yeah, but not perfect. I think ideally fuel should be required only for explicitelly adversary code, like "look at my type-level Turing machine!". For random garbage, I intuitively feel that arriving an "there's no answer" should be fast

nikomatsakis (Sep 03 2019 at 12:58, on Zulip):

I'm trying to summarize up some of our thoughts so far

1. For meeting, we should be thinking about identifying key technical challenges and maybe proposing specific boundaries to explore
- interesting question: what do unit tests look like? How can we enable people to be experts on one side but not the other?
2. For chalk and r-a etc, let's look at r-a pain points and try to diagnose them
- fuel is a somewhat lower priority
3. experience from the second point kind of feeds into the first

nikomatsakis (Sep 03 2019 at 12:59, on Zulip):

seems like zulip's markdown parser doesn't like my mixed bullet/numbered/indented lists but hopefully can you read that :)

matklad (Sep 03 2019 at 13:00, on Zulip):

4. file IO is another acute pain point at the moment, which needs attention

nikomatsakis (Sep 03 2019 at 13:01, on Zulip):

As far as specific boundaries for libraries, I think that there is an obvious one that encompasses:

and then there's another level up that includes the type checker. The distinction I am drawing is basically that the type checker requires us to model the HIR and the full set of Rust expressions.

matklad (Sep 03 2019 at 13:01, on Zulip):

How can we enable people to be experts on one side but not the other?

Unit tests should also help tremendously with rebuild times: i guess building only chalk is much faster than building ra + chalk (and we really should be splitting our ra_hir which does everything into crates )

nikomatsakis (Sep 03 2019 at 13:01, on Zulip):

4. file IO is another acute pain point at the moment, which needs attention

yes and this is partially a salsa problem, right? We can discuss a bit more about how to model "lazy inputs" on the salsa side, it doesn't seem all that hard to me

Florian Diebold (Sep 03 2019 at 13:02, on Zulip):

@nikomatsakis what do you mean by "type inference" separate from type checking there?

matklad (Sep 03 2019 at 13:02, on Zulip):

Yeah, salsa's part seems easy, what is hard is cross-platform file watching

nikomatsakis (Sep 03 2019 at 13:03, on Zulip):

@Florian Diebold I mean encompassing operations like T1 <: T2 (make T1 and subtype of T2) and "coerce T1 to T2"

nikomatsakis (Sep 03 2019 at 13:03, on Zulip):

basically, you can think of type-checking as "generating a bunch of constraints" and "solving a bunch of constraints" -- I am separating out the solving part

nikomatsakis (Sep 03 2019 at 13:03, on Zulip):

the generation comes from walking the HIR

Florian Diebold (Sep 03 2019 at 13:03, on Zulip):

ah ok

nikomatsakis (Sep 03 2019 at 13:04, on Zulip):

OK. So I think I'll have some more time to devote to this late this afternoon or tomorrow. I'm going to try to convert some of my current thoughts into a vague paper doc and send it to y'all for feedback -- and maybe hack a bit on chalk because I can't help myself :P

nikomatsakis (Sep 03 2019 at 13:05, on Zulip):

Yeah, salsa's part seems easy, what is hard is cross-platform file watching

I feel I personally don't have much to contribute to that latter bit :)

nikomatsakis (Sep 03 2019 at 13:05, on Zulip):

That's a good example of the kind of thing I endeavor not to learn :P

matklad (Sep 03 2019 at 13:06, on Zulip):

I'd rather not to as well, but I fear I had to :D

nikomatsakis (Sep 03 2019 at 13:06, on Zulip):

that and linkers

nikomatsakis (Sep 03 2019 at 13:06, on Zulip):

In all seriousness, though, and I guess we want to be sure to keep in mind use cases like FB, where there are too many files -- but I guess that's somewhat addressed just by laziness.

matklad (Sep 03 2019 at 13:07, on Zulip):

Yeah, that's another recent conversation that convinced me that the current eager model is no right

matklad (Sep 03 2019 at 13:07, on Zulip):

(I suggest we table file-watching discussion though, for the lack of relevant expertise among participants :) )

matklad (Sep 03 2019 at 13:11, on Zulip):

and maybe hack a bit on chalk because I can't help myself :P

@nikomatsakis I've noticed that "trivial" prs like "update deps" are not merged into chalk on a timely basis. Do you think we should try to actively seek chalks co-maintainer?

Florian Diebold (Sep 03 2019 at 13:11, on Zulip):

@nikomatsakis so concerning using Chalk's Ty from RA, there are a few points that come to mind:
- I think we need a Ty::Error in Chalk (I currently handle this in a not-great way that may actually be contributing to the hangs)
- it would be nice if we could get rid of the lalrpop_intern::InternedString in UnselectedProjectionTy, maybe we could just introduce a new Id type for associated type names? I'd be up for trying that out

nikomatsakis (Sep 03 2019 at 21:24, on Zulip):

we could and probably should remove the "unselected" stuff for now

matklad (Sep 07 2019 at 18:24, on Zulip):

In term of reproducing a problem, I've spend the last hour looking at the cycling error (in macro expansion) on CrateId(52) somewhere in rustc. I guess a nice first step would be to add crate.dbg(db) method, which prints actually useful information. I am somewhat proud by the fact that so little information is exposed to our HIR, but debugging is a pain in the back due to this

nikomatsakis (Sep 09 2019 at 17:24, on Zulip):

Hey @matklad so I'm around now-ish at last

nikomatsakis (Sep 09 2019 at 17:24, on Zulip):

You still around?

matklad (Sep 09 2019 at 17:24, on Zulip):

yep!

matklad (Sep 09 2019 at 17:25, on Zulip):

So, some things that has happend:

matklad (Sep 09 2019 at 17:26, on Zulip):
nikomatsakis (Sep 09 2019 at 17:26, on Zulip):

what is "watchman"?

nikomatsakis (Sep 09 2019 at 17:26, on Zulip):

i.e., is that some pre-existing library?

matklad (Sep 09 2019 at 17:27, on Zulip):
matklad (Sep 09 2019 at 17:27, on Zulip):

Watchman is Facebooks daemon for file watching

matklad (Sep 09 2019 at 17:28, on Zulip):

Ie, it is sort of a "library" for file watching, but it exists in a separate process, because OS APIs are horrible and it's hard to make a library

matklad (Sep 09 2019 at 17:29, on Zulip):
matklad (Sep 09 2019 at 17:30, on Zulip):
matklad (Sep 09 2019 at 17:31, on Zulip):
matklad (Sep 09 2019 at 17:31, on Zulip):

And I think that's all bigger things that have happend since last week

nikomatsakis (Sep 09 2019 at 17:31, on Zulip):

ok. On my part not much has happened :) but I'm thinking about this upcoming Fri meeting

nikomatsakis (Sep 09 2019 at 17:32, on Zulip):

On Friday I was mostly focused on looking into some soundness related problems unrelated to rust-analyzer :)

nikomatsakis (Sep 09 2019 at 17:32, on Zulip):

That and reviewing various PRs

nikomatsakis (Sep 09 2019 at 17:35, on Zulip):

Anyway I'm going to try and draw up a start of a plan today sketching out how the type checker could be separated out, and also give some feedback in/around rust-analyzer/chalk problems.

matklad (Sep 09 2019 at 17:36, on Zulip):

@nikomatsakis what's the prep I need to do for Friday meeting? Short update about rust-analyzer + rough plan for librarification (are doing/can do/should do/must do table)?

matklad (Sep 09 2019 at 17:36, on Zulip):

Anything else?

nikomatsakis (Sep 09 2019 at 17:36, on Zulip):

That's what I'm trying to figure out

nikomatsakis (Sep 09 2019 at 17:36, on Zulip):

Let's start creating some kind of rough sketch

nikomatsakis (Sep 09 2019 at 17:36, on Zulip):

https://hackmd.io/_ixQrAr5TKafV9m8eSU_VA

nikomatsakis (Sep 09 2019 at 17:36, on Zulip):

I think we've probably said a fair amount of things

nikomatsakis (Sep 09 2019 at 17:37, on Zulip):

in our overall approach

nikomatsakis (Sep 09 2019 at 17:37, on Zulip):

that would be good to document and try to get feedback on

matklad (Sep 09 2019 at 17:38, on Zulip):

ok, I'll jump to the hack.md document then!

nikomatsakis (Sep 09 2019 at 17:39, on Zulip):

great; I'm trying to figure out how to organize it, so feel free to edit

matklad (Sep 09 2019 at 17:42, on Zulip):

Oh, I've also noticed that there's a GSOC happening in Swift, whose goal is to move swift compiler over to Swift's libsyntax. Basically, the same which we'll need to do in rustc. I've contacted the person who does GSOC and asked them to blog about their experience :)

nikomatsakis (Sep 09 2019 at 17:44, on Zulip):

Nice

nikomatsakis (Sep 09 2019 at 18:02, on Zulip):

@matklad one of the things I am wondering about

nikomatsakis (Sep 09 2019 at 18:03, on Zulip):

to what extent does it make sense to try and code libraries "generically", so that they are not tied to a specific representation of types, HIR, etc

nikomatsakis (Sep 09 2019 at 18:03, on Zulip):

and to what extent should we just be sharing the definitions

matklad (Sep 09 2019 at 18:03, on Zulip):

I'd love to be concrete if we can

nikomatsakis (Sep 09 2019 at 18:03, on Zulip):

I agree

nikomatsakis (Sep 09 2019 at 18:03, on Zulip):

That suggests that an area of focus might be on figuring out how to share the actual definitions

matklad (Sep 09 2019 at 18:04, on Zulip):

And I think with index-based structure you can acutlally do that: you can always key additional info by index

nikomatsakis (Sep 09 2019 at 18:04, on Zulip):

I think this is also one of the harder problems though

matklad (Sep 09 2019 at 18:04, on Zulip):

But I am not sure how interning and lifetimes should work for indices

nikomatsakis (Sep 09 2019 at 18:04, on Zulip):

right

nikomatsakis (Sep 09 2019 at 18:04, on Zulip):

I was thinking of calling this out as a "challenge"

matklad (Sep 09 2019 at 18:05, on Zulip):

Especially, how index assignment works across modifications

matklad (Sep 09 2019 at 18:06, on Zulip):

I guess, we can just try to move Ty out of ra_hir to ra_ty, which doesn't have access to DB, and see what breaks?

nikomatsakis (Sep 09 2019 at 18:12, on Zulip):

seems like a first step

nikomatsakis (Sep 09 2019 at 18:12, on Zulip):

I added a few more thoughts in the "managing code" section

nikomatsakis (Sep 09 2019 at 18:12, on Zulip):

see if you think they make sense :)

matklad (Sep 09 2019 at 18:17, on Zulip):

Makes sense, though I feel like we miss the word we want to use instead of "unit tests". We want IR data-driven tests: input is a serialized IR, output is also a piece of data we match against, and the actual #[test] function just pipies input IR via component

matklad (Sep 09 2019 at 18:18, on Zulip):

THat is, thests are not really unit, they test subsytem as a whole, and critically, the are data based, though input data is not necessary rust syntax (to avoid dependency on the parser)

nikomatsakis (Sep 09 2019 at 18:19, on Zulip):

yeah, that sounds right

nikomatsakis (Sep 09 2019 at 18:19, on Zulip):

I think a lot of things are pointing to

nikomatsakis (Sep 09 2019 at 18:19, on Zulip):

the correct selection of the component boundary is key

nikomatsakis (Sep 09 2019 at 18:19, on Zulip):

I'm thinking about e.g. @centril's comments about wanting to audit correctness from a language perspective

nikomatsakis (Sep 09 2019 at 18:20, on Zulip):

I feel like if the component boundaries are well-chosen, that actually becomes potentially easier

nikomatsakis (Sep 09 2019 at 18:21, on Zulip):

the correct selection of the component boundary is key

an example of an ungreat boundary, I think, is probably "type unification"

nikomatsakis (Sep 09 2019 at 18:21, on Zulip):

like, it makes sense to collect the full set of things together I listed

nikomatsakis (Sep 09 2019 at 18:22, on Zulip):

because they are approaching the level of concepts that might appear in a language spec

nikomatsakis (Sep 09 2019 at 18:22, on Zulip):

(at least, to my mind.)

nikomatsakis (Sep 09 2019 at 18:24, on Zulip):

the more I think about it, though, the more I think it makes sense to dive a bit into what sharing types would mean

matklad (Sep 09 2019 at 18:24, on Zulip):

The open question is "do such boundaries exist?" :) Compiler as a whole is obv a great bounary: text on the input, binary on the output, a perfect black-box inside

matklad (Sep 09 2019 at 18:25, on Zulip):

I am not altogether sure that extracting bits out of the middle would be net positive. I hope it would be

nikomatsakis (Sep 09 2019 at 18:26, on Zulip):

Yes, my hypothesis is that such boundaries exist, but they are fairly high-level

nikomatsakis (Sep 09 2019 at 18:26, on Zulip):

And further that exploiting them will be super useful :)

nikomatsakis (Sep 09 2019 at 18:26, on Zulip):

But I suppose there is a degenerate plan in which we use such boundaries only internally, in the code

nikomatsakis (Sep 09 2019 at 18:26, on Zulip):

I think this plan argues for a mono-repo and for integration tests only

matklad (Sep 09 2019 at 18:27, on Zulip):

Specifically, there's a lot of truth to https://www.tedinski.com/2018/02/06/system-boundaries.html#boundaries-arent-free

nikomatsakis (Sep 09 2019 at 18:34, on Zulip):

@matklad ok I'm finding this to be a somewhat useful brain dump. Something I think would help -- maybe a list of questions we would like help to answer. Or maybe we should figure out the question and a proposed answer and look for feedback.

nikomatsakis (Sep 09 2019 at 18:35, on Zulip):

I think from the traits/types side of thing, I would like to try and lay out a "trail" of steps to take, maybe a dep graph. I'm going to try and sketch out what that means :)

matklad (Sep 09 2019 at 18:35, on Zulip):

"Who wants to help out? :)" is a good question

nikomatsakis (Sep 09 2019 at 18:35, on Zulip):

lol

nikomatsakis (Sep 09 2019 at 18:35, on Zulip):

fair enough!

nikomatsakis (Sep 09 2019 at 18:35, on Zulip):

one question I had in mind was something like

nikomatsakis (Sep 09 2019 at 18:36, on Zulip):

is there some .. "time limit"? Are there bits of work we can frontload that would help us to figure out if we're going in the right direction?

nikomatsakis (Sep 09 2019 at 18:36, on Zulip):

but that question is not "well formed" yet I think

nikomatsakis (Sep 09 2019 at 18:37, on Zulip):

I'm basically experiencing a measure of self doubt around "is this a good use of time vs trying to soup up rust's existing systems in place", I suspect.

matklad (Sep 09 2019 at 18:38, on Zulip):

soup up: stir the soup until it nicely separates itself into original components? :)

matklad (Sep 09 2019 at 18:39, on Zulip):

But yeah, I also has "if this even the right direction" feeling: adding boundaries has costs

nikomatsakis (Sep 09 2019 at 18:39, on Zulip):

I think part of the way to answer that question is choosing the order to attack things

nikomatsakis (Sep 09 2019 at 18:40, on Zulip):

and then re-evaluating

matklad (Sep 09 2019 at 18:40, on Zulip):

uguh. I think that means "go for the easiest boundary first"

nikomatsakis (Sep 09 2019 at 18:53, on Zulip):

that's what I'm wondering, indeed

matklad (Sep 16 2019 at 14:01, on Zulip):

@WG-rls2.0 I've realised that we've never actually announced this, but we are trying to have a weekly sync-up in this channel at this time

matklad (Sep 16 2019 at 14:01, on Zulip):

@nikomatsakis are you around?

nikomatsakis (Sep 16 2019 at 14:01, on Zulip):

yep

nikomatsakis (Sep 16 2019 at 14:01, on Zulip):

although it may be wise to make a fresh topic for each week :)

Florian Diebold (Sep 16 2019 at 14:01, on Zulip):

I'm kind of around

nikomatsakis (Sep 16 2019 at 14:02, on Zulip):

I'm a bit distracted because I'm trying to get to the bottom of an async-await thing right now

nikomatsakis (Sep 16 2019 at 14:02, on Zulip):

but I would like to follow up on the friday meeting

nikomatsakis (Sep 16 2019 at 14:02, on Zulip):

I guess makes sense to start with general updates, if there are any since last time we talked

matklad (Sep 16 2019 at 14:02, on Zulip):

@nikomatsakis I think that the same topic would actually be nicer, now? It's easier to caught up with prev meetings and do historical digging

nikomatsakis (Sep 16 2019 at 14:03, on Zulip):

no strong opinion

matklad (Sep 16 2019 at 14:03, on Zulip):

ok, let's stick with status quo, unless anyone objects

matklad (Sep 16 2019 at 14:03, on Zulip):

updates:

matklad (Sep 16 2019 at 14:03, on Zulip):
matklad (Sep 16 2019 at 14:04, on Zulip):
matklad (Sep 16 2019 at 14:04, on Zulip):
matklad (Sep 16 2019 at 14:05, on Zulip):
matklad (Sep 16 2019 at 14:05, on Zulip):
matklad (Sep 16 2019 at 14:05, on Zulip):

Specifically, we have an enum Adt { Struct(Struct), Enum(Enum), Union(Union) }

matklad (Sep 16 2019 at 14:06, on Zulip):

and enum ModuleDef {Adt(Adt), Function(Function), ...}

matklad (Sep 16 2019 at 14:06, on Zulip):

and transitive Intoimpls: Struct -> Adt, Adt -> ModuleDef, Struct -> ModuleDef.

Florian Diebold (Sep 16 2019 at 14:07, on Zulip):

but we also have e.g. TypableDef which includes a subset of those, so it's kind of multiple inheritance ;)

matklad (Sep 16 2019 at 14:07, on Zulip):

ohhh, https://github.com/uHOOCCOOHu added completion for macros, so there's no need to type out unimplemented any more :tada:

nikomatsakis (Sep 16 2019 at 14:08, on Zulip):

I was anticipating that this would happen, if we had "strongly typed ids"

nikomatsakis (Sep 16 2019 at 14:09, on Zulip):

So on my side, I started doing some work on chalk, specifically looking at what it would take to alter the interface to be more "selective" when it asks for impls

Florian Diebold (Sep 16 2019 at 14:09, on Zulip):

oh, I worked on closure types, but that's now a bit blocked on (at least a small subset of) https://github.com/rust-lang/chalk/issues/241 @nikomatsakis

nikomatsakis (Sep 16 2019 at 14:09, on Zulip):

Yeah, I was just going to say, I opened https://github.com/rust-lang/chalk/issues/241 -- and I see @Florian Diebold left some comments I've not yet read :)

nikomatsakis (Sep 16 2019 at 14:10, on Zulip):

I have to look more closely at my initial results. I made some preliminary changes in that direction and I'm getting some failing tests, but I don't know that the failures are significant.

nikomatsakis (Sep 16 2019 at 14:10, on Zulip):

(They have to do with how we model coherence specifically, and there is probably a way to circumvent them)

nikomatsakis (Sep 16 2019 at 14:12, on Zulip):

I think @Florian Diebold we should be able to land some changes that, while not perfect, help a lot in practice relatively soon. My hope for today was that I would sketch out a small series of steps around chalk, prioritizing (a) support for dyn and impl types, which is started, (b) the RustIrDatabase traits, (c) and steps to improve how we are handling lazy norm (although I think there are some open questions, as I wrote) -- but I'm not sure if those are the best priorities.

nikomatsakis (Sep 16 2019 at 14:12, on Zulip):

One thing I was wondering is whether my change to flounder when self type is unknown helped with the "chalk never terminates" bugs

Florian Diebold (Sep 16 2019 at 14:13, on Zulip):

I haven't seen any "chalk never terminates" bugs since the change that makes all traits non-enumerable

matklad (Sep 16 2019 at 14:13, on Zulip):

It did, yeah: at least, there's a clean improvement in runtime from oo to 10s for that PR

Florian Diebold (Sep 16 2019 at 14:14, on Zulip):

there are some reports of crashes inside Chalk, which I haven't looked into in detail yet

Florian Diebold (Sep 16 2019 at 14:14, on Zulip):

https://github.com/rust-analyzer/rust-analyzer/issues/1805

Florian Diebold (Sep 16 2019 at 14:15, on Zulip):

it's very possible that this is some error in the Chalk lowering code in RA ;)

nikomatsakis (Sep 16 2019 at 14:15, on Zulip):

ah, interesting

nikomatsakis (Sep 16 2019 at 14:15, on Zulip):

one thing i'd like to do for chalk that's been on my radar for some time

nikomatsakis (Sep 16 2019 at 14:15, on Zulip):

is to try and make some "Benchmarks"

nikomatsakis (Sep 16 2019 at 14:16, on Zulip):

the code was never really written with perf in mind

nikomatsakis (Sep 16 2019 at 14:16, on Zulip):

it'd be nice to have e.g. some realistic-ish setup with some number of traits and a real query that we can run perf on and use to measure changes

nikomatsakis (Sep 16 2019 at 14:16, on Zulip):

just something to keep in mind if you come across some queries that seem plausible

nikomatsakis (Sep 16 2019 at 14:17, on Zulip):

that said, this is exactly the sort of thing that I would think a "chalk extractor" could do a great job on (and I'd still like to add that at some point, on the chalk side)

nikomatsakis (Sep 16 2019 at 14:17, on Zulip):

anyway, do my priorities sound plausible?

(a) support for dyn and impl types, which is started, (b) the RustIrDatabase traits, (c) and steps to improve how we are handling lazy norm

matklad (Sep 16 2019 at 14:17, on Zulip):

I think ra_cli analys-bench is a pretty realistic example (but it is bound to rust-analyzer of course)

matklad (Sep 16 2019 at 14:18, on Zulip):

It would be interesting to tie "dumping chalk IR as we go" idea from prev meeting into analysis-bench, so that you can record a real sample, and then use it to bench chalk, without ra

nikomatsakis (Sep 16 2019 at 14:19, on Zulip):

yeah; I think the chalk side is the right side to do it on

nikomatsakis (Sep 16 2019 at 14:19, on Zulip):

it'd be nice to be able to do it for rustc too, for example

nikomatsakis (Sep 16 2019 at 14:20, on Zulip):

anyway, @matklad, what is status arond parser work?

nikomatsakis (Sep 16 2019 at 14:20, on Zulip):

I've yet to catch up with that discussion from friday

matklad (Sep 16 2019 at 14:20, on Zulip):

Remembered two more things:

nikomatsakis (Sep 16 2019 at 14:20, on Zulip):

Yes. In particular, I would expect to change this at a rust edition boundary

nikomatsakis (Sep 16 2019 at 14:21, on Zulip):

(I was thinking about that earlier, that we ought to be preparing for such a change well in advance)

matklad (Sep 16 2019 at 14:21, on Zulip):

re parser: nothing new on the rustc side, but rust-analyzer now handles composed tokens in the way I want rustc to handle them, so I know have the model to change rustc

nikomatsakis (Sep 16 2019 at 14:21, on Zulip):

I don't quite understand what you mean about Source<T> yet

matklad (Sep 16 2019 at 14:21, on Zulip):

Let me try to talk this through, as it seems important.

matklad (Sep 16 2019 at 14:22, on Zulip):

So, syntax trees in rust-analyzer are by designg value types. 1 + 1 in foo.rs looks exactly like 1 + 1 in bar.rs, and given that AST is reference counted, it's ok to intern syntax trees

matklad (Sep 16 2019 at 14:23, on Zulip):

(and this even happens within a single file already)

matklad (Sep 16 2019 at 14:24, on Zulip):

For analysis, you need to add some form of identity to the syntax trees.

matklad (Sep 16 2019 at 14:24, on Zulip):

for example, if you have #[path = "foo.rs"] mod x; #[path = "foo.rs"] mod y; , then the same syntax from foo.rs would represent two different modules

nikomatsakis (Sep 16 2019 at 14:25, on Zulip):

editor's note: responses about Source<T> broken out into a separate topic

matklad (Sep 23 2019 at 14:01, on Zulip):

@nikomatsakis, @WG-rls2.0 are you around for a weekly sync-up ? :)

nikomatsakis (Sep 23 2019 at 14:02, on Zulip):

I'm around

matklad (Sep 23 2019 at 14:03, on Zulip):

Stuff that happened this week:

nikomatsakis (Sep 23 2019 at 14:03, on Zulip):

Nice

nikomatsakis (Sep 23 2019 at 14:03, on Zulip):

"web version" of rust-analyzer?

matklad (Sep 23 2019 at 14:03, on Zulip):

https://rust-analyzer.github.io/wasm-demo.html

matklad (Sep 23 2019 at 14:04, on Zulip):

(warning: 5 MB download)

nikomatsakis (Sep 23 2019 at 14:04, on Zulip):

That's very cool :)

matklad (Sep 23 2019 at 14:05, on Zulip):
matklad (Sep 23 2019 at 14:06, on Zulip):
matklad (Sep 23 2019 at 14:07, on Zulip):
matklad (Sep 23 2019 at 14:07, on Zulip):

We also had a pretty interesting discussion about joint rls/rust-analyzer strategy. We didn't come up with a strategy, but made this nice markdown doc: https://hackmd.io/S5wub4mfToSnnLGvK3I-ew

matklad (Sep 23 2019 at 14:08, on Zulip):

I think that's it?

nikomatsakis (Sep 23 2019 at 14:08, on Zulip):

OK. My update is shorter, I guess. I landed a small update to chalk's IR, removing the all_structs callback, and I plan to work a bit more today to refine the "request impl" callback, and also to help push the dyn/impl integration forward.

matklad (Sep 23 2019 at 14:09, on Zulip):

Oh, one more thing: while working on mbe, I've realized that this could be another candidate for rather painless librarification. Maybe

nikomatsakis (Sep 23 2019 at 14:10, on Zulip):

Ah, interesting

nikomatsakis (Sep 23 2019 at 14:10, on Zulip):

I've thought about that before

matklad (Sep 23 2019 at 14:10, on Zulip):

In theory the interface for mbe are token trees, which is a relatively thing data-structure

nikomatsakis (Sep 23 2019 at 14:10, on Zulip):

But I had kind of forgotten it

nikomatsakis (Sep 23 2019 at 14:10, on Zulip):

Yeah; there is some interaction with name resolution, but there is also a part that can be factored out from that I suppose

matklad (Sep 23 2019 at 14:10, on Zulip):

In practice, the token trees in rustc contain AST, and that throws quite a wrench in the works

nikomatsakis (Sep 23 2019 at 14:10, on Zulip):

Indeed

nikomatsakis (Sep 23 2019 at 14:10, on Zulip):

I have also disliked that

nikomatsakis (Sep 23 2019 at 14:10, on Zulip):

Particularly since it's visible (in edge cases)

nikomatsakis (Sep 23 2019 at 14:11, on Zulip):

It's basically an efficiency optimization, which I think we could/should do in other ways

matklad (Sep 23 2019 at 14:11, on Zulip):

Is it visible in non-backwards compatible way?

nikomatsakis (Sep 23 2019 at 14:11, on Zulip):

Hmm so I think something else I should prioritize is doing a bit of work in/around rust-analyzer's type checker. I guess I can start by reviewing what @Florian Diebold did around associated types.

matklad (Sep 23 2019 at 14:12, on Zulip):

Like, we can't today re-parse $var:expr as something else, but it seems like if we allow reparsing, that would only increase the set of accepted programs

nikomatsakis (Sep 23 2019 at 14:12, on Zulip):

Is it visible in non-backwards compatible way?

depends what you mean by "non backawrds compatible" -- I guess the question is "compatible with what"

matklad (Sep 23 2019 at 14:12, on Zulip):

Compatible with current stable behavior

nikomatsakis (Sep 23 2019 at 14:12, on Zulip):

if you had some stream of tokens that could be parsed as either an expression or a type

nikomatsakis (Sep 23 2019 at 14:12, on Zulip):

no, I mean, what future behavior are you imagining

nikomatsakis (Sep 23 2019 at 14:13, on Zulip):

I think that you could observe e.g. macro_rules! foo { ($x:expr) => { .. } ($x:ty) => { .. } }

matklad (Sep 23 2019 at 14:13, on Zulip):

Ah. So, I imagine that we store $var:expr as a token tree with virtual delimiters

nikomatsakis (Sep 23 2019 at 14:13, on Zulip):

right, and I think that might not be enough to be fully compatible

nikomatsakis (Sep 23 2019 at 14:13, on Zulip):

though it is arguably a bug-fix

nikomatsakis (Sep 23 2019 at 14:14, on Zulip):

it's the kind of change where the behavior is sort of loosely documented :)

matklad (Sep 23 2019 at 14:14, on Zulip):

we definitelly should make sure that macro thing doesn't use the AST

matklad (Sep 23 2019 at 14:14, on Zulip):

macro 2.0 that is

nikomatsakis (Sep 23 2019 at 14:14, on Zulip):

Yeah.

nikomatsakis (Sep 23 2019 at 14:14, on Zulip):

( I have to run a bit early today btw -- but I'll be back around hacking on traits and rust-analyzer this afternoon )

nikomatsakis (Sep 23 2019 at 14:15, on Zulip):

So, are you still actively investigating parsing extraction?

nikomatsakis (Sep 23 2019 at 14:15, on Zulip):

I'm trying to understand the "status" of that

matklad (Sep 23 2019 at 14:15, on Zulip):

sort-of

matklad (Sep 23 2019 at 14:15, on Zulip):

I am actively trying to move the parser to the token-tree modelN

matklad (Sep 23 2019 at 14:15, on Zulip):

(which is a part of that)

matklad (Sep 23 2019 at 14:16, on Zulip):

I haven't looked further than that

nikomatsakis (Sep 23 2019 at 14:16, on Zulip):

the rustc parser?

nikomatsakis (Sep 23 2019 at 14:16, on Zulip):

or r-a parser?

nikomatsakis (Sep 23 2019 at 14:16, on Zulip):

that does seem like a good first step

matklad (Sep 23 2019 at 14:16, on Zulip):

rustc parser

nikomatsakis (Sep 23 2019 at 14:16, on Zulip):

Ah. So, I imagine that we store $var:expr as a token tree with virtual delimiters

to be clear: I think I'd prefer to try this model first,

matklad (Sep 23 2019 at 14:16, on Zulip):

r-a parser uses token tree model like the one is proc_macro

nikomatsakis (Sep 23 2019 at 14:17, on Zulip):

because I think it's more what users would expect

nikomatsakis (Sep 23 2019 at 14:17, on Zulip):

but I think the "fix" to match rustc is also not that terrible

nikomatsakis (Sep 23 2019 at 14:17, on Zulip):

it'd kind of be that the "virtual delimeters" also encode the fragment

nikomatsakis (Sep 23 2019 at 14:17, on Zulip):

i.e., there are virtual "type" delimeters, virtual "expr" delimeters

nikomatsakis (Sep 23 2019 at 14:17, on Zulip):

I should probably make some proof of concept test case for what I'm talking about

matklad (Sep 23 2019 at 14:18, on Zulip):

Yeah, I think I undestand what you are talking about

matklad (Sep 23 2019 at 14:18, on Zulip):

It wouldn't be hard to implement indeed

matklad (Sep 23 2019 at 14:19, on Zulip):

but it would mean that mbe model is not quite proc_macro2 tokens, which is a rather big problem, even if the difference is small

matklad (Sep 23 2019 at 14:19, on Zulip):

Though, perhaps we can store this info on the side

nikomatsakis (Sep 23 2019 at 14:20, on Zulip):

not very realistic, but here is an example

nikomatsakis (Sep 23 2019 at 14:21, on Zulip):

oh, maybe not, I guess that doesn't parse as an expr for us

nikomatsakis (Sep 23 2019 at 14:21, on Zulip):

well I think I can make one but it takes some work :P

matklad (Sep 23 2019 at 14:21, on Zulip):

yeah, I did the same: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=e9a29816fa71e6abb09fb349f4053d5a

nikomatsakis (Sep 23 2019 at 14:21, on Zulip):

ah yeah duh there you go

nikomatsakis (Sep 23 2019 at 14:21, on Zulip):

great

nikomatsakis (Sep 23 2019 at 14:21, on Zulip):

ok, I gotta run though, bbiab

matklad (Sep 23 2019 at 14:21, on Zulip):

heah, yeh, mine is better :D

matklad (Sep 23 2019 at 14:22, on Zulip):

:wave:

matklad (Sep 30 2019 at 14:01, on Zulip):

hey, @WG-rls2.0, it's time for another weekly sink up. I have to run really soon, so I'll give just a brife update, and try to follow up later

nikomatsakis (Sep 30 2019 at 14:04, on Zulip):

:wave:

matklad (Sep 30 2019 at 14:05, on Zulip):

Things that happened since last week:

nikomatsakis (Sep 30 2019 at 14:05, on Zulip):

Can I get some links to the closures/coercions PRs etc?

nikomatsakis (Sep 30 2019 at 14:05, on Zulip):

Or is it best to read the source

nikomatsakis (Sep 30 2019 at 14:05, on Zulip):

I guess I can find them on the repo

matklad (Sep 30 2019 at 14:05, on Zulip):

Also, I will be traveling this week: giving a "Rust as a High Level Langauge" talk in Toulouse, so I might be slow to repond.

matklad (Sep 30 2019 at 14:06, on Zulip):

@nikomatsakis commit hash is c7420ddaaa76741d1eebe393406b38ba5596e54a

matklad (Sep 30 2019 at 14:07, on Zulip):

a better link: https://github.com/rust-analyzer/rust-analyzer/commit/c7420ddaaa76741d1eebe393406b38ba5596e54a

nikomatsakis (Sep 30 2019 at 14:08, on Zulip):

thanks

nikomatsakis (Sep 30 2019 at 14:10, on Zulip):

on my side not a lot to tell, I plan to work more on the chalk changes that @Florian Diebold alluded to today, along with some other things. I also started thinking about what it might look like to extract types into a library that many could share.

nikomatsakis (Sep 30 2019 at 14:10, on Zulip):

seems like one of the hardest questions to answer will come up with interning and other such details, and how generic we want to be about that

nikomatsakis (Sep 30 2019 at 14:11, on Zulip):

rust doesn't make it super easy to extract over these sorts of things, though I think in this case it could plausibly be done.

nikomatsakis (Sep 30 2019 at 14:11, on Zulip):

though mostly I've been pondering how to cleanup chalk to address the normalization and "known self type" cases

matklad (Oct 07 2019 at 14:00, on Zulip):

Hello @WG-rls2.0 , time for another update.

matklad (Oct 07 2019 at 14:01, on Zulip):

Not a lot of stuff happened this week, but the highlight is the support for #[cfg()] attributes for conditional compilation by https://github.com/oxalica: https://github.com/rust-analyzer/rust-analyzer/pull/1928

matklad (Oct 07 2019 at 14:05, on Zulip):

Some other things:

matklad (Oct 07 2019 at 14:07, on Zulip):

I think I want to look at supporting dynamic reloading of the crate graph next:

matklad (Oct 07 2019 at 14:08, on Zulip):

@nikomatsakis , any updates on your side?

nikomatsakis (Oct 07 2019 at 14:09, on Zulip):

Hey @matklad sorry -- a neighbor came by with an emergency

nikomatsakis (Oct 07 2019 at 14:09, on Zulip):

was afk

nikomatsakis (Oct 07 2019 at 14:09, on Zulip):

just got back, let me catch up

nikomatsakis (Oct 07 2019 at 14:11, on Zulip):

OK, so, updates:

nikomatsakis (Oct 07 2019 at 14:11, on Zulip):

I spent most of my time last week exploring some non-RLS-related trait things (e.g., upcasts in rustc and what it takes to unblock const generics)

nikomatsakis (Oct 07 2019 at 14:11, on Zulip):

(The good news being that I think I see the path, but we'll see)

matklad (Oct 07 2019 at 14:13, on Zulip):

That reminds me... At the all hands, when we were discussing traits, there was an idea that, to unblock immediate progress on rustc side, we (well, you I presume :) ) should pause chalk work and instead augment the current trait solver.

I am curious what's the final plan here: are we doing GATs and stuff before chalkification?

nikomatsakis (Oct 07 2019 at 14:14, on Zulip):

I'm still kind of trying to figure that out. My current plan is to pursue two main things:

nikomatsakis (Oct 07 2019 at 14:15, on Zulip):

Once we have lazy norm going, the next step would probably be figuring out some details of the "universe story" on rustc side; I did some of that work already. That might then enable GATs on the rustc side, but I'm not entirely sure. I remember the last time I did work on this I felt like it would be hard

nikomatsakis (Oct 07 2019 at 14:16, on Zulip):

(I also opened a lot of issues around refactoring chalk last week, incidentally, but I need to prioritize them a bit)

matklad (Oct 14 2019 at 14:33, on Zulip):

Belated sync for this week:

matklad (Oct 21 2019 at 14:00, on Zulip):

Next week, next syncup!

nikomatsakis (Oct 21 2019 at 14:03, on Zulip):

:wave:

nikomatsakis (Oct 21 2019 at 14:05, on Zulip):

Update from last week:

matklad (Oct 21 2019 at 14:05, on Zulip):

I unfortunately don't have a lot of too report this week: I've been trying different thing, and they didn't quite work out.

In particular, I've made all preliminary refactoings for a lazy VFS, only to realise that we probably don't actually want a fully lazy VFS. We want to load file texts lazily, but the set of files should be loaded eagarly. This is how watchman works, and this I think is the only reliable way to get file watching right.

I've also looked into splitting hir database of rust-analyzer into several crates. The results are mixed:

nikomatsakis (Oct 21 2019 at 14:06, on Zulip):
nikomatsakis (Oct 21 2019 at 14:07, on Zulip):

Hmm, ok, I will try to catch up a bit on that discussion

nikomatsakis (Oct 21 2019 at 14:07, on Zulip):

(the splitting of the crate)

nikomatsakis (Oct 21 2019 at 14:07, on Zulip):

Oh, there is another thing, which is that we landed some support for dyn Trait and impl Trait, but it's quite broken

matklad (Oct 21 2019 at 14:07, on Zulip):

Indeed we ought to figure this particular elephant in the room out

nikomatsakis (Oct 21 2019 at 14:07, on Zulip):

Yeah. I was finding it useful to try and explore various "scenarios"

matklad (Oct 21 2019 at 14:08, on Zulip):

@nikomatsakis yeah, @Florian Diebold already has a WIP PR which integrates that with rust-analyzer, which doens't quite work yet

nikomatsakis (Oct 21 2019 at 14:08, on Zulip):

e.g., if we go "all in" on rust-analyzer, what does that look like? which of the things we might want to do on rustc conflict with that? What is the opposite extreme? What's in the middle?

nikomatsakis (Oct 21 2019 at 14:08, on Zulip):

OK, I see a topic that looks like I need to catch up on it

nikomatsakis (Oct 21 2019 at 14:08, on Zulip):

on this topic, not sure if you followed that at all @matklad?

matklad (Oct 21 2019 at 14:09, on Zulip):

Nope, I think I've missed that

nikomatsakis (Oct 21 2019 at 14:09, on Zulip):

the relevant thing to rust-analyzer is that -- at least to start and maybe permanently -- I want to have traits that let you interact with types generically, without regard to their concrete representation

matklad (Oct 21 2019 at 14:09, on Zulip):

Like, I think I wasn't pinged, do you have a link to the zulip thread?

nikomatsakis (Oct 21 2019 at 14:09, on Zulip):

one sec

nikomatsakis (Oct 21 2019 at 14:09, on Zulip):

here are the notes on the meeting

nikomatsakis (Oct 21 2019 at 14:10, on Zulip):

there are links to the topic but also significant points

nikomatsakis (Oct 21 2019 at 14:11, on Zulip):

the relevant thing to rust-analyzer is that -- at least to start and maybe permanently -- I want to have traits that let you interact with types generically, without regard to their concrete representation

there are two reasons to do this:

nikomatsakis (Oct 21 2019 at 14:11, on Zulip):

I'm exploring this

nikomatsakis (Oct 21 2019 at 14:11, on Zulip):

(I expect to put more time into that today)

nikomatsakis (Oct 21 2019 at 14:12, on Zulip):

If that makes more progress, I am definitely interested in trying to work with @Florian Diebold to factor out the type checker from RLS into something more independent (maybe even integrate it into the chalk repository, not sure -- it depends on what we decide the "scope" of the chalk repo should be)

matklad (Oct 21 2019 at 14:12, on Zulip):

Aha, the typefamaly pattern seems like a hammer big enough to handle every librarification

nikomatsakis (Oct 21 2019 at 14:12, on Zulip):

Yes, I've found it a very powerful pattern in the past

nikomatsakis (Oct 21 2019 at 14:12, on Zulip):

Right now though what I'm trying to do

nikomatsakis (Oct 21 2019 at 14:13, on Zulip):

is refactor some of the parts of chalk that are quite messy anyway

nikomatsakis (Oct 21 2019 at 14:13, on Zulip):

my goal is that the code reads nicer with the generics than without

nikomatsakis (Oct 21 2019 at 14:13, on Zulip):

and helps to avoid some kinds of bugs

nikomatsakis (Oct 21 2019 at 14:13, on Zulip):

we'll see if I can achieve that ;)

nikomatsakis (Oct 21 2019 at 14:13, on Zulip):

my main concern is undue complexity; otoh I think that people who hack on chalk should probably be pretty comfortable with traits :P

nikomatsakis (Oct 21 2019 at 14:13, on Zulip):

one of the nice things though of the "one true underlying type" thing

nikomatsakis (Oct 21 2019 at 14:13, on Zulip):

is that you can kind of "cut off" the generics at any point

nikomatsakis (Oct 21 2019 at 14:14, on Zulip):

i.e., in my chalk branch right now, the low-level code is generic, but at some point everybody just hardcodes on "type family"

nikomatsakis (Oct 21 2019 at 14:14, on Zulip):

(this lets me convert crate by crate)

matklad (Oct 21 2019 at 14:14, on Zulip):

I would also be concerned about separate compilation. But yeah, if, in the end, we have some concrete representation, that fixes this problem.

nikomatsakis (Oct 21 2019 at 14:15, on Zulip):

there is certainly the issue of compilation time and monomorphization, but -- yes -- keeping the number of concrete representations minimal means there aren't really multiple copies in the end

nikomatsakis (Oct 21 2019 at 14:15, on Zulip):

(though you do defer some of that compilation to the "leaf crate")

nikomatsakis (Oct 21 2019 at 14:16, on Zulip):

(hopefully incremental helps with this, if not, we should figure out why not)

nikomatsakis (Oct 21 2019 at 14:16, on Zulip):

this reminds me -- I've really failed to keep up with salsa, as you no doubt saw

nikomatsakis (Oct 21 2019 at 14:16, on Zulip):

but you had mentioned something about &dyn Foo not working for rust-analyzer?

matklad (Oct 21 2019 at 14:16, on Zulip):

&(dyn AstDatabase + DefDatabase) not working, yeah

nikomatsakis (Oct 21 2019 at 14:16, on Zulip):

yeah that certainly won't work

matklad (Oct 21 2019 at 14:17, on Zulip):

And there's a separate issue of creating mock databses for tests

nikomatsakis (Oct 21 2019 at 14:17, on Zulip):

actually the work that @Alexander Regueiro has been doing on trait upcast could be quite relevant here

nikomatsakis (Oct 21 2019 at 14:17, on Zulip):

I remember that when I was pushing &dyn changes through Lark

nikomatsakis (Oct 21 2019 at 14:17, on Zulip):

not having trait upcast was a horrible pain

matklad (Oct 21 2019 at 14:17, on Zulip):

basically, you need concrete db not only in the leaf trait, but also in the tests for intermediate crates, and creating that mock everytime is boilerplaty

nikomatsakis (Oct 21 2019 at 14:17, on Zulip):

maybe because we were using a large hierarchy of salsa traits

nikomatsakis (Oct 21 2019 at 14:18, on Zulip):

basically, you need concrete db not only in the leaf trait, but also in the tests for intermediate crates, and creating that mock everytime is boilerplaty

yeah I think in Lark I moved the tests by and large to the leaf crate

nikomatsakis (Oct 21 2019 at 14:18, on Zulip):

so as to have a single "mock db"

nikomatsakis (Oct 21 2019 at 14:18, on Zulip):

I certainly remember encountering this problem

matklad (Oct 21 2019 at 14:18, on Zulip):

Yeah, I also feel that salsa atm pushes me towards this direction, but I think that's horrible long-term

nikomatsakis (Oct 21 2019 at 14:19, on Zulip):

I'm not actually sure that's true

matklad (Oct 21 2019 at 14:19, on Zulip):

basically, if you fix a bug in a low-level crate, you need to recompile the whole tower to test it

nikomatsakis (Oct 21 2019 at 14:19, on Zulip):

But I guess I agree I'd prefer to make the choice for other reasons

nikomatsakis (Oct 21 2019 at 14:19, on Zulip):

Yeah, that part is not great I guess

matklad (Oct 21 2019 at 14:20, on Zulip):

I am actually not sure if it's a good idea to split databases over several crates.....

nikomatsakis (Oct 21 2019 at 14:20, on Zulip):

the other option

nikomatsakis (Oct 21 2019 at 14:20, on Zulip):

which I've experimented with in chalk

nikomatsakis (Oct 21 2019 at 14:20, on Zulip):

is just having regular traits basically

matklad (Oct 21 2019 at 14:20, on Zulip):

I can see a design where there's only a single database crate that does everytihng, and all other code lives in separate crates, and uses traits to interface to DB without knowning about salsa at all

nikomatsakis (Oct 21 2019 at 14:20, on Zulip):

yeah, that

nikomatsakis (Oct 21 2019 at 14:21, on Zulip):

I thnk this is a better fit (at minimum) for library-ification

matklad (Oct 21 2019 at 14:22, on Zulip):

For librarification, definitelly yes. For rust-analyzer itself, not sure

detrumi (Oct 21 2019 at 14:22, on Zulip):

What's the reason to use a DB above using traits directly in the first place?

matklad (Oct 21 2019 at 14:22, on Zulip):

currently, hir is pretty slow to compile and pretty big. I'd love to slice it into at least "types" and "not types"

nikomatsakis (Oct 21 2019 at 14:23, on Zulip):

What's the reason to use a DB above using traits directly in the first place?

there isn't really a very good one that I can see, except that it's slightly less repetitive maybe

nikomatsakis (Oct 21 2019 at 14:24, on Zulip):

I could definitely be persuaded that it makes more sense to just use plain traits all the way down

matklad (Oct 21 2019 at 14:24, on Zulip):

@detrumi

matklad (Oct 21 2019 at 14:24, on Zulip):

but yeah, maybe just using traits is the way to go....

nikomatsakis (Oct 21 2019 at 14:24, on Zulip):

@matklad regarding testing, I guess another way to go about reducing boilerplate would be to have macros or things that help you to generate the impls

nikomatsakis (Oct 21 2019 at 14:24, on Zulip):

I haven't thought much about it

nikomatsakis (Oct 21 2019 at 14:24, on Zulip):

it doesn't really feel like salsa's fault to me

nikomatsakis (Oct 21 2019 at 14:25, on Zulip):

I think Rust in general is not very good at this kind of re-use

nikomatsakis (Oct 21 2019 at 14:25, on Zulip):

It's kind of an OO strength and -- lacking specialization and a few other things -- we don't quite have the tools you want

nikomatsakis (Oct 21 2019 at 14:25, on Zulip):

But regardless it is a limitation

nikomatsakis (Oct 21 2019 at 14:25, on Zulip):

I think the main reason I was thinking that pulling tests to the "root" is ok is the old tension of integration vs unit testing.

nikomatsakis (Oct 21 2019 at 14:26, on Zulip):

To some extent it's nice for the tests to be kind of independent of the precise nature of your division into crates -- but to some extent that's also not nice, esp. for practical compilation time reasons.

matklad (Oct 21 2019 at 14:26, on Zulip):

Hm, I am not sure. I think, if salsa created a concrete DB (which are composed together in single inheritance style), I would be able to write

fn mock_db(fixture: &str) -> TheDbStruct

in the intermediate crate, and then use it in the final crate

nikomatsakis (Oct 21 2019 at 14:26, on Zulip):

if your crates are sufficiently coarse, and thus change infrequently, it's probably better to have the tests be linked to them

nikomatsakis (Oct 21 2019 at 14:27, on Zulip):

hmm, ok, I see, maybe there is some way for us to make the databases more composable in this way

nikomatsakis (Oct 21 2019 at 14:28, on Zulip):

this reminds me that you also had some other comments about state that I never fully understood

nikomatsakis (Oct 21 2019 at 14:28, on Zulip):

we probably need to create some kind of hackmd to help us track these concerns a bit

matklad (Oct 21 2019 at 14:28, on Zulip):

@nikomatsakis it seems like we need a meeting/design document for "physical architecture of salsa-based compilers"?

nikomatsakis (Oct 21 2019 at 14:28, on Zulip):

yes, exactly

matklad (Oct 21 2019 at 14:29, on Zulip):

Heh, great minds think alike

nikomatsakis (Oct 21 2019 at 14:29, on Zulip):

I'd like to go now and think about other things but I'd like to not forget this, and to maybe schedule an hour to talk it out

matklad (Oct 21 2019 at 14:29, on Zulip):

or, as they say in Russia, у дураков мысли сходятся

matklad (Oct 21 2019 at 14:31, on Zulip):

I think I'll open a thread on the salsa repository to just write down some thougts: that way, we won't forget this

nikomatsakis (Oct 21 2019 at 14:32, on Zulip):

good starting point

nikomatsakis (Oct 21 2019 at 14:32, on Zulip):

ok, great chat :) ttyl

matklad (Oct 28 2019 at 14:01, on Zulip):

Hey @WG-rls2.0 , it's time for weekly sync-up

nikomatsakis (Oct 28 2019 at 14:02, on Zulip):

Hey @matklad :wave:

matklad (Oct 28 2019 at 14:02, on Zulip):

A bunch off things happened recently

matklad (Oct 28 2019 at 14:03, on Zulip):

Most importantly, we've finally merged find usages, implemented by https://github.com/viorina

matklad (Oct 28 2019 at 14:03, on Zulip):

It does the classic IDE trick of using text-search to over-approximate the set of matchings, and then confirming the matches using the real analysis

matklad (Oct 28 2019 at 14:04, on Zulip):

We don't have a trigram index yet, so the text-search phase is brute force

matklad (Oct 28 2019 at 14:05, on Zulip):

As a consequence, it's now possible to invoke "rename" for functions, and it mostly works :)

matklad (Oct 28 2019 at 14:06, on Zulip):

Another biggish thing is that we now generate assists docs and tests from "doc comments". And in general I did some cleanup around assists. We have a whole bunch of them now:

https://github.com/rust-analyzer/rust-analyzer/blob/master/docs/user/assists.md

(and a couple new were added this week)

matklad (Oct 28 2019 at 14:08, on Zulip):

Find usages also was a stress-test for our type inference, which uncovered that infinite allocation in chalk problem. I don't think the root cause if fixed yet, but I've improved our profiling infrastructure a bit:

matklad (Oct 28 2019 at 14:10, on Zulip):

Finally, I've put the VFS work on pause without actually finishing it: it seems like I am stuck a bit, as I don't see a design that feels right, and, if we speak about various tradeoff-based designs, the one we have already seems OK. I want to come back to this later and just try to write direct bindings to watchman, but I have sunk way to many days into this, and I think I need to swticht to something else to make progress

matklad (Oct 28 2019 at 14:11, on Zulip):

Oh, and @nikomatsakis and I had a productive discussion about splitting salsa-database across the crates, I hope to do something concrete in this area this week

matklad (Oct 28 2019 at 14:11, on Zulip):

That's all for me I guess?

nikomatsakis (Oct 28 2019 at 14:13, on Zulip):

Cool --

matklad (Oct 28 2019 at 14:13, on Zulip):

Ahhh, one more thing: @detrumi implemented client-side shortening of type hints, which helps a lot with "as long as a screen" infered types :)

nikomatsakis (Oct 28 2019 at 14:13, on Zulip):

I've been working on some changes to chalk, some of which it seems like @Florian Diebold has integrated

nikomatsakis (Oct 28 2019 at 14:14, on Zulip):

the overall goal is to rework how we handle associated types so that we never ask "open-ended" requests for "all impls of a trait"

nikomatsakis (Oct 28 2019 at 14:14, on Zulip):

though we're not there yet

nikomatsakis (Oct 28 2019 at 14:14, on Zulip):

it's also I think laying the foundations for a "shared type" library that both rustc / rust-analyzer can employ

nikomatsakis (Oct 28 2019 at 14:14, on Zulip):

though I want to do a bit more digging in that direction

nikomatsakis (Oct 28 2019 at 14:14, on Zulip):

I also think it's related to that bug you uncovered

detrumi (Oct 28 2019 at 14:14, on Zulip):

Ahhh, one more thing: detrumi implemented client-side shortening of type hints, which helps a lot with "as long as a screen" infered types :slight_smile:

Good to hear it's useful :+1:

nikomatsakis (Oct 28 2019 at 14:14, on Zulip):

but I've not 100% proven that

nikomatsakis (Oct 28 2019 at 14:15, on Zulip):

(that is, I think that bug is caused by the current logic in unification -- which would be removed in the new scheme -- causing a kind of "infinite set of goals to prove")

nikomatsakis (Oct 28 2019 at 14:16, on Zulip):

there has been some good PRs in this direction, with @detrumi ( :tada:) implementing a derive for chalk's Fold trait, and @Jack Huey working on simplifying some of rust's context traits

nikomatsakis (Oct 28 2019 at 14:16, on Zulip):

(as an aside, I am thinking about trying to rip out the chalk integration that's in rustc now and rework it so that it hooks into the chalk-solve crate, like rust-analyzer does)

nikomatsakis (Oct 28 2019 at 14:17, on Zulip):

anyway, we still didn't fix the bugs around dyn Trait but hopefully soon

nikomatsakis (Oct 28 2019 at 14:17, on Zulip):

so that we can combine that logic with rust-analyzer

matklad (Oct 28 2019 at 14:18, on Zulip):

Exciting, given that we want to use dyn Trait inside rust-analyzer more, when splitting stuff across crates :)

nikomatsakis (Oct 28 2019 at 14:18, on Zulip):

the overall goal is to rework how we handle associated types so that we never ask "open-ended" requests for "all impls of a trait"

on this topic, I was thinking that it hopefully soon I can turn to looking more into rust-analyzer's type checker and how to best integrate it with check (and ultimately create a shared library), because I think it's going to take some experimentation to get that truly working

nikomatsakis (Oct 28 2019 at 14:18, on Zulip):

Exciting, given that we want to use dyn Trait inside rust-analyzer more, when splitting stuff across crates :)

Yay =) I think there's not that much work to do, I'm not sure if @Keith Yeung has had a chance to look into it yet

nikomatsakis (Oct 28 2019 at 14:19, on Zulip):

(I'll check if they're busy, because I think it wouldn't take me long to prep the PR anyhow, but I don't want to steal it from them -- and it might be more work than I think)

matklad (Oct 28 2019 at 14:20, on Zulip):

Oh, one more thing: we are receiving more "rust-analyzer eats 8GB of ram after two hours of usage" issues recently. I haven't seen this myself (as I pretty-much rebuild rust-analyzer once in ten minutes anyway), but it might be a symptom of something wrong with GC

nikomatsakis (Oct 28 2019 at 14:21, on Zulip):

hmm

matklad (Oct 28 2019 at 14:21, on Zulip):

Would be cool to be able "print, how much space each table occupies" to debug this issues, but I don't think we can esily implement this at the moment

nikomatsakis (Oct 28 2019 at 14:23, on Zulip):

What's blocking is the "heapsize" support?

matklad (Oct 28 2019 at 14:24, on Zulip):

the fact that heapsize on crates.io is pretty much dead, and the new one in Servo is not public

matklad (Oct 28 2019 at 14:24, on Zulip):

Although perhaps we should look at just the number of keys/values?

detrumi (Oct 28 2019 at 14:25, on Zulip):

the fact that heapsize on crates.io is pretty much dead, and the new one in Servo is not public

I'm guessing they can't just open-source it?

matklad (Oct 28 2019 at 14:26, on Zulip):

"just" means making sure that the API actually makes sense outside of Servo

detrumi (Oct 28 2019 at 14:26, on Zulip):

Ah, of course

matklad (Oct 28 2019 at 14:27, on Zulip):

Like, it seems like this is a pretty solvable problem, but someone needs to apply necessary labor (and a good API/macro design taste) to actually provide a production-ready solution

matklad (Oct 28 2019 at 14:27, on Zulip):

IIRC, HeapSize was just dumped to crates.io, and it's not maintained because it turned out to be not that great

detrumi (Oct 28 2019 at 14:28, on Zulip):

But yeah, approximating it should be fine too as long as you identify the cause

matklad (Oct 28 2019 at 14:28, on Zulip):

(for posterity, the new heapsizeof is here https://github.com/servo/servo/blob/master/components/malloc_size_of/lib.rs)

Jeremy Kolb (Oct 28 2019 at 14:31, on Zulip):

Is there a plan to defeat compile times? It's getting pretty bad with running tests.

matklad (Oct 28 2019 at 14:34, on Zulip):

Sort of. The immediate things I want to do are:

Long term, I'd love to see if we can make salsa-based API less generic, but I think it makes sense to look into that after by-reference values (@nikomatsakis wants to change salsa sot hat queries return &Thing and not Arc<Thing>).

detrumi (Oct 28 2019 at 14:39, on Zulip):

Large crates are really having a big impact on compile times, it seems. Hopefully that can be improved on rustc's side in the future, as splitting things up each time is a lot of work

detrumi (Oct 28 2019 at 14:41, on Zulip):

@matklad By less generic, do you mean the &impl Database parameters?

matklad (Oct 28 2019 at 14:42, on Zulip):

Yeah, and the fact that test and read database are different types

matklad (Nov 04 2019 at 15:06, on Zulip):

@WG-rls2.0, time for another sync up!

matklad (Nov 04 2019 at 15:07, on Zulip):

I think the two biggest changes this week are:

matklad (Nov 04 2019 at 15:07, on Zulip):

I can probably say a bit about the second one

matklad (Nov 04 2019 at 15:09, on Zulip):

First of all, the split is not complete, so the code is really not great at places. FOr example, we have DefDatabase and DefDatabase2...

matklad (Nov 04 2019 at 15:11, on Zulip):

What we do have, however, is three crates instead of one:

hir_expand deals with macro expansion. Specifically, it defines the core datatypes required to talk about all Rust source code (some of which might be generated from macros). I originally didn't intend to split this part out, but I really really like how it looks now.

It's interesting how macro expansion does not depend on name-resolution even in terms of dependencies between crates: this is beccause we encode resovled macro definition into the id of a macro call

matklad (Nov 04 2019 at 15:13, on Zulip):

hir_def is a work-in-progress -- it should contain everything srictly between macro expansion and types. At the moment, it only has module-level name resolution. I want to move expression lowering to hir_def as well. hir_def is written with ECS API, where one operates with raw IDs

matklad (Nov 04 2019 at 15:14, on Zulip):

hir currently contains everything else. I think that, after hir_def is done, we should split hir into hir_ty and hir, and make hir a strict facade over hir_x crates.

matklad (Nov 04 2019 at 15:15, on Zulip):

One thing we've lost from the split is AstDb / DefDb separation, which we used to enforce via require salsa clause. That is, we put some effort to make sure you can't accidently invoke a "brittle" query form a robust query previously, but this work is undone by the split.

matklad (Nov 04 2019 at 15:16, on Zulip):

For testing, the idea is to have a #[cfg(test)] MockDatabse for each hir crate (renamed TestDb, as it is not really a mock), which share most of the fixture-related code via a WithFixture trait, defined in the root ra_db crate

matklad (Nov 04 2019 at 15:17, on Zulip):

the observation here is that various hir layers never add new inputs, so you only need to define fixtures at the bottom layer

matklad (Nov 04 2019 at 15:20, on Zulip):

Some of the less groundshaking changes:

nikomatsakis (Nov 04 2019 at 15:21, on Zulip):

@matklad sorry I'm late, I should've written, I had to run at the last minute (I've also been heads down on something this morning)

nikomatsakis (Nov 04 2019 at 15:21, on Zulip):

I'm going to catch up in a bit though

matklad (Nov 04 2019 at 15:21, on Zulip):

oh, @Christopher Durham is also doing some crazy unsafe experiments with rowan, with an effort to cut down the number of allocations even more. I haven't had a chance to review it properly, but the latest approach with boxed and interned tokens seems really promising

matklad (Nov 04 2019 at 15:22, on Zulip):

And I am excited to see @Edwin Cheng back! Looks like we are about to see some progress on macro front soon :)

Jeremy Kolb (Nov 04 2019 at 15:31, on Zulip):

I think I can piggyback off @Edwin Cheng 's work for macro signature help

matklad (Nov 11 2019 at 14:55, on Zulip):

I need to run soon, so let me give a short async update:

matklad (Nov 11 2019 at 14:58, on Zulip):

The most interesting bit today is probably the first point. The main problem there was to identify tokens in the expended code as coming from either the call or the definition. My understanding that in rustc this is handled by each token having a global unique span.

In rust-analyzer, tokens are locally numbered with small interegers, and, when you substitute captures into a macro template, you shift the numbers of the tokens in captures, sort-of how you do subst in lambda-calculus with de-Bruijn indices

matklad (Nov 11 2019 at 15:03, on Zulip):

Ok, I am leaving, will catch up later :) :wave:

nikomatsakis (Nov 11 2019 at 15:03, on Zulip):

@matklad thanks for the update! On the chalk side, @Jack Huey has been doing great work refactoring and fixing various bugs. I just remembered that I need to investigate that infinite loop. One good thing is that the work he's been doing is helping to pave way for fuel.

matklad (Nov 18 2019 at 20:44, on Zulip):

@WG-rls2.0 I've forgot about today's sync-up, but let me make an async update

matklad (Nov 25 2019 at 15:04, on Zulip):

@WG-rls2.0 time for the next sync up!

matklad (Nov 25 2019 at 15:11, on Zulip):

Things that happened last week:

.. let me post this while I type some more...

Dirkjan Ochtman (Nov 25 2019 at 15:16, on Zulip):

Can I suggest that you post these somewhere outside Zulip, too? Like a TWiRA?

Dirkjan Ochtman (Nov 25 2019 at 15:16, on Zulip):

I bet it would get you more contributors

matklad (Nov 25 2019 at 15:18, on Zulip):
matklad (Nov 25 2019 at 15:21, on Zulip):

@Dirkjan Ochtman yeah, in general it definitely makes sense to talk more about what we do in rust-analyzer. We definitelly should start TWiRA once we have a release process, but maybe we can start it even before that....

Dirkjan Ochtman (Nov 25 2019 at 15:21, on Zulip):

I don't think it needs to be anything more than what you've got in this stream already :)

matklad (Nov 25 2019 at 15:22, on Zulip):

It needs at least some frontend work on https://github.com/rust-analyzer/rust-analyzer.github.io to separate long-form blog post from weakly updates

detrumi (Nov 25 2019 at 15:24, on Zulip):

Wow, that's a lot of work in a week's time :tada:

matklad (Nov 25 2019 at 15:36, on Zulip):

Note that one observation I did while refactorings is that, at the moment, the code is very prone to repeated climbings up the tree of defs.

For example, processing a record literal might call field.ty for each field, and each of those calls would fetch the parent struct, create name resolution ctx for it, etc... I feel this might become a problem, not due to any perf problems, but due to un-normalized data storage (basically, the same info is rederived several times).

Instead, I am trying to switch to a more Kotlin-like approach: when we need to learn a bit of info about thing, we look at its parent, process all the children and store the info into a map.

To get a bit of info about specific child, we load the map and do a lookup using a local ID.

nikomatsakis (Dec 02 2019 at 15:13, on Zulip):

Hey @matklad -- back from vacation, still catching up

nikomatsakis (Dec 02 2019 at 15:13, on Zulip):

guessing no sync this week?

matklad (Dec 02 2019 at 15:14, on Zulip):

@nikomatsakis I'll make an async sync later this week. I've moved to Berlin over the weeked, and still catching up with emails :)

ice1000 (Dec 04 2019 at 02:13, on Zulip):

ra_ide compiles extremely slow

ice1000 (Dec 04 2019 at 02:13, on Zulip):

I wish it can be split into even smaller ones

detrumi (Dec 04 2019 at 07:53, on Zulip):

Did you try these linking flags?

matklad (Dec 04 2019 at 14:22, on Zulip):

@WG-rls2.0

Hey, so here's belated update for this week.

First, as per @Dirkjan Ochtman advice, I've moved "the laundry list of things that changed since last week" to the rust-analyzer website. You can read it here: https://rust-analyzer.github.io/thisweek/2019/12/04/changelog-1.html. For some reason, the format is eerie similar to this week in IntelliJ Rust. I will make it a point to publish an update there each Monday before the sync-up (which should save everyone's time, as I routinely take like 15 minutes to just type this stuff).

With this in mind, I suggest re purposing this slot for a more free-form discussion of latest things. @nikomatsakis how would you feel about this?

In particular, the highlight of this week is the introduction of a separatehir_ty crate, which deals with all type-related things. It should make it easier to move more of the types to chalk.

Dirkjan Ochtman (Dec 04 2019 at 14:37, on Zulip):

reddit: https://www.reddit.com/r/rust/comments/e603vf/rustanalyzer_changelog_1/

Dirkjan Ochtman (Dec 05 2019 at 14:44, on Zulip):

@matklad I rest my case that this at least generates nice amounts of PR :)

nikomatsakis (Dec 06 2019 at 20:52, on Zulip):

@matklad repurposing for freeform discussion seems good to me

nikomatsakis (Dec 09 2019 at 15:01, on Zulip):

@matklad :wave:

matklad (Dec 09 2019 at 15:01, on Zulip):

hello @WG-rls2.0 !

matklad (Dec 09 2019 at 15:01, on Zulip):

this week changes are here: https://github.com/rust-analyzer/rust-analyzer.github.io/blob/3779a91eff76547ac1f347e8ac7e2ee933343ff7/thisweek/_posts/2019-12-09-changelog-2.md

matklad (Dec 09 2019 at 15:02, on Zulip):

(will be on the website once the build is finished -- forgot to push the changes, and forgot to check that I am looking at the deopolied site and not 127.0.0.1 one when checking)

nikomatsakis (Dec 09 2019 at 15:03, on Zulip):

So you were talking about repurposing the slot for "freeform design discussion"

nikomatsakis (Dec 09 2019 at 15:03, on Zulip):

I was noticing that we have this upcoming meeting on Friday to discuss IDE/library-ification

nikomatsakis (Dec 09 2019 at 15:03, on Zulip):

And I was thinking I should touch base with you, @pnkfelix and @Igor Matuszewski to try and figure out what we plan to say exactly

matklad (Dec 09 2019 at 15:03, on Zulip):

The highlights are:

matklad (Dec 09 2019 at 15:05, on Zulip):

@nikomatsakis right. I think we want to "pitch" the plan with integrating rust-analyzer with save-analysis, and continuing library-ification (as opposed to slowly boiling rustc itself)

nikomatsakis (Dec 09 2019 at 15:05, on Zulip):

Yeah. I'm wondering if you've done any more investigation into your idea of command-line tools

nikomatsakis (Dec 09 2019 at 15:05, on Zulip):

your blog post is of course part of the story here

nikomatsakis (Dec 09 2019 at 15:05, on Zulip):

We had a link to a "master" hackmd that linked to other notes and things

nikomatsakis (Dec 09 2019 at 15:05, on Zulip):

Do you happen to have it off hand?

Florian Diebold (Dec 09 2019 at 15:06, on Zulip):

Also go-to def works within println! etc now, which I'm pretty happy about :)

matklad (Dec 09 2019 at 15:06, on Zulip):

not much: the only thing I did was to briefly check why loading save-analysis from disk is slow.

matklad (Dec 09 2019 at 15:07, on Zulip):

And looks like the main reason is that JSON we currently use is just super verbose. zipping save-analysis gives 97% compression (the zip is 3% of original)

nikomatsakis (Dec 09 2019 at 15:07, on Zulip):

This is "master" hackmd I was talking about

matklad (Dec 09 2019 at 15:07, on Zulip):

don't have link to master hack md :(

nikomatsakis (Dec 09 2019 at 15:08, on Zulip):

And looks like the main reason is that JSON we currently use is just super verbose. zipping save-analysis gives 97% compression (the zip is 3% of original)

fascinating

nikomatsakis (Dec 09 2019 at 15:08, on Zulip):

can we zip/unzip in process :)

nikomatsakis (Dec 09 2019 at 15:08, on Zulip):

not sure if that would actually help

nikomatsakis (Dec 09 2019 at 15:08, on Zulip):

presumably not

matklad (Dec 09 2019 at 15:09, on Zulip):

I'd rather we not encode each span as { "start_column": 0, "end_column": 1, "start_line": 0, "end_line": 1}

nikomatsakis (Dec 09 2019 at 15:09, on Zulip):

heh you think we could use a few fewer characters? :)

nikomatsakis (Dec 09 2019 at 15:10, on Zulip):

still it seems to me like the real question is

nikomatsakis (Dec 09 2019 at 15:10, on Zulip):

can we move save-analysis to something more universal

nikomatsakis (Dec 09 2019 at 15:10, on Zulip):

e.g., that LSP-based proposal

nikomatsakis (Dec 09 2019 at 15:10, on Zulip):

I have no idea how they compare

nikomatsakis (Dec 09 2019 at 15:10, on Zulip):

anywaY I'm going to make a new hackmd, maybe we can briefly draft an agenda for discussion and key bullet points

matklad (Dec 09 2019 at 15:10, on Zulip):

I think we can, and we should, long-term

nikomatsakis (Dec 09 2019 at 15:11, on Zulip):

hackmd

detrumi (Dec 09 2019 at 15:29, on Zulip):

It's tricky how we want to adopt rust-analyzer as the "official" IDE (which requires some stability), while at the same time library-ifying some parts (which is as of yet easier because rust-analyzer is still experimental and can change easier)

nikomatsakis (Dec 09 2019 at 15:31, on Zulip):

Yes, there is some tension there,

nikomatsakis (Dec 09 2019 at 15:31, on Zulip):

and I think one question might be

nikomatsakis (Dec 09 2019 at 15:31, on Zulip):

how far can we get before doing so?

nikomatsakis (Dec 09 2019 at 15:32, on Zulip):

that is, how much can/must we "library-ify"?

nikomatsakis (Dec 09 2019 at 15:32, on Zulip):

at the same time, I don't know, I'm not that worried about it

nikomatsakis (Dec 09 2019 at 15:32, on Zulip):

I guess in part because I think rust-analyzer delivers a lot of value "as is"

nikomatsakis (Dec 09 2019 at 15:32, on Zulip):

and we can "gate" access to some parts of it conceivably

nikomatsakis (Dec 09 2019 at 15:32, on Zulip):

e.g., that is one reason to use save-analysis to start

nikomatsakis (Dec 09 2019 at 15:32, on Zulip):

still, I think we should identify this as a tension and thing to discuss maybe?

nikomatsakis (Dec 09 2019 at 15:33, on Zulip):

I'm debating the best use of the "agenda"

detrumi (Dec 09 2019 at 15:33, on Zulip):

Yeah, using gates can be a tool to get the best of both worlds

detrumi (Dec 09 2019 at 15:35, on Zulip):

btw, I'm not 100% sure what falls under save analysis

nikomatsakis (Dec 09 2019 at 15:37, on Zulip):

OK, this was pretty producitve @matklad -- I'm going to have to run, I have some errands to do locally

matklad (Dec 09 2019 at 15:38, on Zulip):

bye @nikomatsakis ! I too feel this was productive!

nikomatsakis (Dec 09 2019 at 15:38, on Zulip):

we can try to clarify the doc and/or pick a final discussion agenda at the end?

nikomatsakis (Dec 09 2019 at 15:38, on Zulip):

I think we should advertise this doc in a broader topic on #t-compiler and solicit people to add questions and things to the end

nikomatsakis (Dec 09 2019 at 15:38, on Zulip):

and/or ask for "points of information" over the week

nikomatsakis (Dec 09 2019 at 15:38, on Zulip):

sorry, those msgs sent a bit out of order

nikomatsakis (Dec 09 2019 at 15:38, on Zulip):

but maybe you get the idea :)

matklad (Dec 09 2019 at 15:38, on Zulip):

yeah

matklad (Dec 09 2019 at 15:39, on Zulip):

I'll do the advertising

nikomatsakis (Dec 09 2019 at 15:41, on Zulip):

OK -- I had this draft I was about to post, but I'll happily let you do it:

HiT-compiler/meeting -- a few of us from @WG-rls2.0 took a stab at creating a hackmd for the upcoming design meeting this Friday. The topic is our plans regarding rust-analyzer, the "official" choice of IDE, and rustc. The document summarizes our current thinking and also includes a section at the end for the discussion agenda. My hope is that people can review the doc now, before-hand. You can ask questions here if there are things that are unclear or could be calrified, and perhaps leave ideas for broader discussion topics in the document. Then we can prune the agenda on Thu or so before the actual meeting.

matklad (Dec 09 2019 at 15:44, on Zulip):

@Florian Diebold gosh, how could I have missed goto-def in println? Pushed a new commit to the changelog

detrumi (Dec 09 2019 at 15:51, on Zulip):

@matklad maybe remove the ?edit from the url?

matklad (Dec 09 2019 at 15:52, on Zulip):

I see ?edit as a bit of social engineering, actually encouriging folks to leave suggestions :)

Edwin Cheng (Dec 09 2019 at 16:21, on Zulip):

#2492 2492, #2494 goto definition and syntax highlighting work properly for type parameters

Typo, double 2492?

matklad (Dec 16 2019 at 15:02, on Zulip):

@WG-rls2.0 This week's changelog is up: https://rust-analyzer.github.io/thisweek/2019/12/16/changelog-3.html

matklad (Dec 16 2019 at 15:03, on Zulip):

I don't have anything specific to discuss I think, and I am moving between laptops, so I suggest to keep this week more async :)

nikomatsakis (Dec 16 2019 at 15:04, on Zulip):

One of the things on my mind is what follow-up we need from last week's meeting, @matklad

nikomatsakis (Dec 16 2019 at 15:05, on Zulip):

(By this I mean Friday)

matklad (Dec 16 2019 at 15:06, on Zulip):

Yeah, it's true that we need some follow up. Actually, I think there are two followup:

matklad (Dec 16 2019 at 15:06, on Zulip):

the second one is more important, but much less clear

nikomatsakis (Dec 16 2019 at 15:07, on Zulip):

Yes.

nikomatsakis (Dec 16 2019 at 15:07, on Zulip):

I think you can generalize the first bullet a bit

nikomatsakis (Dec 16 2019 at 15:07, on Zulip):

basically I think we should be laying out clearer plans around specific library-ification efforts

matklad (Dec 16 2019 at 15:07, on Zulip):

I feel there are might be two possible answers:

Maybe pre-RFC thread on internals?

nikomatsakis (Dec 16 2019 at 15:07, on Zulip):

For the second point, you mean?

matklad (Dec 16 2019 at 15:08, on Zulip):

yeah

matklad (Dec 16 2019 at 15:08, on Zulip):

first one doesn't make tooo much sense unless we have consensus on the general direction

nikomatsakis (Dec 16 2019 at 15:08, on Zulip):

I don't think RFC is right

nikomatsakis (Dec 16 2019 at 15:08, on Zulip):

Yes

nikomatsakis (Dec 16 2019 at 15:08, on Zulip):

I thikn there's probably room to discuss specifically with @eddyb as well

nikomatsakis (Dec 16 2019 at 15:08, on Zulip):

I guess I'm not exactly sure which parts of the plan are controversial

nikomatsakis (Dec 16 2019 at 15:10, on Zulip):

at the highest level I think the plan was

nikomatsakis (Dec 16 2019 at 15:10, on Zulip):

it seemed like a lot of the discussion was more about how to achieve the first bullet

nikomatsakis (Dec 16 2019 at 15:10, on Zulip):

but maybe I'm not correct there, I'm not entirely sure

matklad (Dec 16 2019 at 15:10, on Zulip):

uhu

matklad (Dec 16 2019 at 15:11, on Zulip):

I think eddyb was uncomfortable with not removing save-analysis completely

matklad (Dec 16 2019 at 15:12, on Zulip):

I thikn there's probably room to discuss specifically with @eddyb as well

Should we setup a video maybe?

nikomatsakis (Dec 16 2019 at 15:12, on Zulip):

I think that might make sense, yes

nikomatsakis (Dec 16 2019 at 15:12, on Zulip):

I would probably start there

nikomatsakis (Dec 16 2019 at 15:13, on Zulip):

I think eddyb was uncomfortable with not removing save-analysis completely

yes, I think this may be true

nikomatsakis (Dec 16 2019 at 15:14, on Zulip):

I'm pondering this

nikomatsakis (Dec 16 2019 at 15:15, on Zulip):

I guess I feel like the goal of using save-analysis was primarily that it exists and it lets us minimize the time until useful features

nikomatsakis (Dec 16 2019 at 15:15, on Zulip):

and then turn the focus more to library-ification and "cleaning up" the "native" approach

nikomatsakis (Dec 16 2019 at 15:15, on Zulip):

I think the latter is what eddyb really wants

nikomatsakis (Dec 16 2019 at 15:15, on Zulip):

(and I agree)

nikomatsakis (Dec 16 2019 at 15:19, on Zulip):

In other words, it seems like we share a goal, and we're arguing about the steps towards it

matklad (Dec 16 2019 at 15:19, on Zulip):

@nikomatsakis there's also one more aspect of sava-analysis which is bad long-term, but which we find very useful right now. It is data that can be used outside of rustc process

matklad (Dec 16 2019 at 15:19, on Zulip):

This is good right now, because it allows us to have some of the correctness of RLS without build complexities of RLS

matklad (Dec 16 2019 at 15:20, on Zulip):

but it is wrong long-term, as it moves us further away from in-process lazy-analysis

nikomatsakis (Dec 16 2019 at 15:20, on Zulip):

Still, even sharing a goal, there's room for dispute, which comes back to the basic question -- we're kind of taking the "long path" towards a unified world, to some extent, and maybe eddyb prefers to take the most direction path

matklad (Dec 16 2019 at 15:20, on Zulip):

Well, it looks like it moves us away, but I want to argue that it is exactly the quick and dirty hack we can use while we are building proper solution properly

nikomatsakis (Dec 16 2019 at 15:22, on Zulip):

Yes.

nikomatsakis (Dec 16 2019 at 15:22, on Zulip):

OK, so, let's try to talk to eddyb -- maybe we can arrange a time this week

Florian Diebold (Dec 16 2019 at 15:55, on Zulip):

but it is wrong long-term, as it moves us further away from in-process lazy-analysis

is there actually consensus on this? With librarification, the end state would never be that 'RLS2' calls in-process queries from rustc, would it? At least my understanding is that the idea there is that code-sharing between the compiler would be achieved by sharing libraries, not the whole compiler. So I don't know if this is really just a disagreement about how to get to the end state

matklad (Dec 16 2019 at 16:45, on Zulip):

With librarification, the end state would never be that 'RLS2' calls in-process queries from rustc, would it?

In my mind, in the end state RLS2 and rustc are indistinguishable. In particular, they do use the same query infrastructure (but maybe in a slightly different ways)

matklad (Dec 23 2019 at 15:00, on Zulip):

@nikomatsakis , @WG-rls2.0 it's that time again!

matklad (Dec 23 2019 at 15:01, on Zulip):

This week's changelog: https://rust-analyzer.github.io/thisweek/2019/12/23/changelog-4.html

matklad (Dec 23 2019 at 15:01, on Zulip):

(quite a bunch of changes there)

nikomatsakis (Dec 23 2019 at 15:01, on Zulip):

:wave:

nikomatsakis (Dec 23 2019 at 15:01, on Zulip):

I'm skimming the change-log

nikomatsakis (Dec 23 2019 at 15:02, on Zulip):

My main update is that I've been working on drafting the "what does a type library look like" docs

nikomatsakis (Dec 23 2019 at 15:02, on Zulip):

As well as various chalk updates

nikomatsakis (Dec 23 2019 at 15:02, on Zulip):

I see that @Florian Diebold already rebased over some of the various tweaks to chalk-ir, I expect more to come

matklad (Dec 23 2019 at 15:03, on Zulip):

Yeah, we've updated to chalk master yesterday

matklad (Dec 23 2019 at 15:03, on Zulip):

I think the hihglight of this week is super-preliminary support for items, nested in functions

nikomatsakis (Dec 23 2019 at 15:03, on Zulip):

I'm trying to figure out the best way to "get ahead" of these changes and to structure such a document; my ambition is of course that we also use this as the basis for a "portable" type-checking library

matklad (Dec 23 2019 at 15:03, on Zulip):

Ie, we actually create hir for such items, and do a suuuuper crude name resolution for them.

nikomatsakis (Dec 23 2019 at 15:04, on Zulip):

One thing that might be good for us to talk briefly about are the next steps after last week's meeting and the subsequent discussion

matklad (Dec 23 2019 at 15:04, on Zulip):

+1

matklad (Dec 23 2019 at 15:05, on Zulip):

Also, I'd love to check in with @Igor Matuszewski about rls-save-analysis crate refactorings. Did you get a chance to do the cleanups you wanted?

nikomatsakis (Dec 23 2019 at 15:05, on Zulip):

Yeah, I was wondering about that -- I think maybe the two of us aren't quite the right audience for the 'next steps' discussion, seems like we probably want at least @Igor Matuszewski and maybe a bit more

matklad (Dec 23 2019 at 15:05, on Zulip):

One thing that might be good for us to talk briefly about are the next steps

Is it correct to assume that t-compiler has a rough agreement on the overall plan?

nikomatsakis (Dec 23 2019 at 15:06, on Zulip):

I don't entirely know but I think so

matklad (Dec 23 2019 at 15:06, on Zulip):

no one explicitelly disagreed, and Esteban Küber explictelly agreed, modulo diagnostics, so I'd lean towards yes

nikomatsakis (Dec 23 2019 at 15:06, on Zulip):

By "the overall plan" I think you're referring to #t-compiler > design meeting 2019-12-13 follow-up

matklad (Dec 23 2019 at 15:06, on Zulip):

yup

nikomatsakis (Dec 23 2019 at 15:06, on Zulip):

Yeah

nikomatsakis (Dec 23 2019 at 15:06, on Zulip):

One of the questions I'm pondering is what to do with that plan

nikomatsakis (Dec 23 2019 at 15:06, on Zulip):

I feel like it's even a bit bigger than t-compiler, it feels like something that should be at least "raised" with core team and reflected on the roadmap, for example, but I'm kind of trying to do that

matklad (Dec 23 2019 at 15:07, on Zulip):

Agree

nikomatsakis (Dec 23 2019 at 15:07, on Zulip):

Anyway let's assume there is general agreement

nikomatsakis (Dec 23 2019 at 15:07, on Zulip):

For the time being

matklad (Dec 23 2019 at 15:07, on Zulip):

I still think that maybe this is [pre-]RFC worthy

nikomatsakis (Dec 23 2019 at 15:07, on Zulip):

I actually agree with that

nikomatsakis (Dec 23 2019 at 15:07, on Zulip):

and was debating about drafting one

nikomatsakis (Dec 23 2019 at 15:08, on Zulip):

Though one of the questions is just what it should cover

nikomatsakis (Dec 23 2019 at 15:08, on Zulip):

In particular I'm not sure how deep to go on the technical details --

nikomatsakis (Dec 23 2019 at 15:09, on Zulip):

Anyway the main unresolved technical question, I think, is how to actually manage the "transition" to rust-analyzer

matklad (Dec 23 2019 at 15:09, on Zulip):

I think we should cover architecture in broad strokes, but explicitelly avoid digging into technical details

nikomatsakis (Dec 23 2019 at 15:09, on Zulip):

I think there are some "branding" questions -- e.g., is this "RLS 2.0" etc?

nikomatsakis (Dec 23 2019 at 15:09, on Zulip):

That is something I expect the core team may have opinions and insight into

nikomatsakis (Dec 23 2019 at 15:09, on Zulip):

(And/or other folks)

nikomatsakis (Dec 23 2019 at 15:09, on Zulip):

I think we should cover architecture in broad strokes, but explicitelly avoid digging into technical details

I lean this way as well

nikomatsakis (Dec 23 2019 at 15:09, on Zulip):

maybe we should create a hackmd to start drafting? and/or a repo

matklad (Dec 23 2019 at 15:10, on Zulip):

Though, we should solve the immediate technical question of save-analysis libary vs rls process, and preferably sooner rather than later

nikomatsakis (Dec 23 2019 at 15:10, on Zulip):

Agreed as well :)

nikomatsakis (Dec 23 2019 at 15:10, on Zulip):

We did the starting point of enumerating some of the design space

matklad (Dec 23 2019 at 15:10, on Zulip):

maybe we should create a hackmd to start drafting? and/or a repo

Yes! I think hackmd might be easier?

nikomatsakis (Dec 23 2019 at 15:10, on Zulip):

BTW

nikomatsakis (Dec 23 2019 at 15:10, on Zulip):

we should link all these things into that "master" hackmd

nikomatsakis (Dec 23 2019 at 15:10, on Zulip):

/me goes to look for it

nikomatsakis (Dec 23 2019 at 15:11, on Zulip):

(we could also use hackmd's tagging metadata feature...)

matklad (Dec 23 2019 at 15:11, on Zulip):

and we should link the "master" hackmd from somewhere, because I, offhand, don't know how to find it :)

nikomatsakis (Dec 23 2019 at 15:11, on Zulip):

We did the starting point of enumerating some of the design space

I feel like this is where @Igor Matuszewski's take would be useful

matklad (Dec 23 2019 at 15:11, on Zulip):

I guess I'll just bookmark

nikomatsakis (Dec 23 2019 at 15:11, on Zulip):

hold on, i'm finding and linking things together :)

nikomatsakis (Dec 23 2019 at 15:12, on Zulip):

I think another key point re: the technical design is the question of whether rust-analyzer will be distributed via rustup etc

matklad (Dec 23 2019 at 15:12, on Zulip):

yep

nikomatsakis (Dec 23 2019 at 15:12, on Zulip):

in particular, if it's going to be "implicitly" tied to rustc via save-analysis ...

nikomatsakis (Dec 23 2019 at 15:12, on Zulip):

but I think in general we should consider "rustup distribution" a feature

nikomatsakis (Dec 23 2019 at 15:13, on Zulip):

i.e., that is a feature of RLS, and we want to maintain parity with it, if the goal is to minimally disrupt user experience

matklad (Dec 23 2019 at 15:13, on Zulip):

I plan to try publish some binary releases directlyt o GitHub, jut to experiment

nikomatsakis (Dec 23 2019 at 15:13, on Zulip):

ideally people would just do rustup udpate and get rust-analyzer

nikomatsakis (Dec 23 2019 at 15:13, on Zulip):

without knowing the difference

nikomatsakis (Dec 23 2019 at 15:13, on Zulip):

right?

matklad (Dec 23 2019 at 15:13, on Zulip):

THe interesting bit is that rust-analyzer doesn't have to be distributed via rustup, as we don't need nightly

nikomatsakis (Dec 23 2019 at 15:13, on Zulip):

well

matklad (Dec 23 2019 at 15:13, on Zulip):

right?

I am not sure actually

nikomatsakis (Dec 23 2019 at 15:13, on Zulip):

I feel like that is less true if it is consuming save-analysis data directly

nikomatsakis (Dec 23 2019 at 15:14, on Zulip):

it may not need nightly, but it will break if it is not using the version of rustc it expets

matklad (Dec 23 2019 at 15:14, on Zulip):

Like, the ideal workflow for VS Code is to install extension via a marketplace, and let the extension download stuff

nikomatsakis (Dec 23 2019 at 15:14, on Zulip):

well I guess I don't know how the RLS upgrades itself now etc

matklad (Dec 23 2019 at 15:14, on Zulip):

That is, what matter for user experience is not the server binary, but the editor plugin, and we can't distribute that via rustup

nikomatsakis (Dec 23 2019 at 15:15, on Zulip):

PS link to the master hackmd document

matklad (Dec 23 2019 at 15:16, on Zulip):

I think that we should try weekly releases outside of rustup first, and, if that's not convenient/breaks in practice, switch to rustup

nikomatsakis (Dec 23 2019 at 15:16, on Zulip):

would we expect people to manually upgrade?

nikomatsakis (Dec 23 2019 at 15:16, on Zulip):

anyway this seems like another detail to resolve for sure

matklad (Dec 23 2019 at 15:16, on Zulip):

We expect the extension to automatically upgrade

matklad (Dec 23 2019 at 15:17, on Zulip):

editor extension that is

nikomatsakis (Dec 23 2019 at 15:17, on Zulip):

maybe we can start to update that "master" document with the broad outlines of the plan and the "things to resolve"

nikomatsakis (Dec 23 2019 at 15:17, on Zulip):

/me tries to do that

matklad (Dec 23 2019 at 15:18, on Zulip):

or maybe that's "unresolved questions" section of the RFC draft? Maybe too technical for it though

nikomatsakis (Dec 23 2019 at 15:19, on Zulip):

I do think this point is potentially germane to the RFC

nikomatsakis (Dec 23 2019 at 15:20, on Zulip):

I think the RFC should be about what "end users of Rust have to know"

Darin Morrison (Dec 23 2019 at 15:21, on Zulip):

you could have the editor extension notify the user when the server needs an upgrade

nikomatsakis (Dec 23 2019 at 15:22, on Zulip):

Personally I think this should be as "minimally disruptive" for folks as possible

nikomatsakis (Dec 23 2019 at 15:22, on Zulip):

But I don't exactly know what that means

nikomatsakis (Dec 23 2019 at 15:24, on Zulip):

I'm trying to think about the best way to start broadening up the conversation on these points, and include the right folks

nikomatsakis (Dec 23 2019 at 15:24, on Zulip):

internals thread is an option

nikomatsakis (Dec 23 2019 at 15:24, on Zulip):

I'm not 100% sure who the "right folks" are

Darin Morrison (Dec 23 2019 at 15:25, on Zulip):

Yeah you don't want to spam them but you could, on first install, ask if they want updates. I've seen other extensions do that.

matklad (Dec 23 2019 at 15:26, on Zulip):

Pre-rfc thread seems good. I think the set of "right folks" is open hear: we've already did the technical discussion with t-compiler

nikomatsakis (Dec 23 2019 at 15:26, on Zulip):

BTW, created a hackmd for the RFC drafting

matklad (Dec 23 2019 at 15:26, on Zulip):

well, we might want to have a round of conversation with core as well, but, after that, internals seems like the right audience

nikomatsakis (Dec 23 2019 at 15:26, on Zulip):

yeah that sounds correct, but I think we should do some "pre-gaming" to select the questions where we most want feedback

matklad (Dec 23 2019 at 15:27, on Zulip):

Hm, yeah, now that I think of it, it might be a good idea to talk to core before pre-RFC

nikomatsakis (Dec 23 2019 at 15:27, on Zulip):

I can certainly send a quick e-mail

nikomatsakis (Dec 23 2019 at 15:27, on Zulip):

Given that it's the holidays I wouldn't expect a lot of responses

matklad (Dec 23 2019 at 15:28, on Zulip):

Do we want to expedite something before the holidays end though?

nikomatsakis (Dec 23 2019 at 15:28, on Zulip):

That said I think an internals thread is also an ok next step -- but prior to that it probably makes sense to have a bit more of a "RFC draft" to point people at -- not finishing, and we should draft the internals message itself

nikomatsakis (Dec 23 2019 at 15:28, on Zulip):

I don't think there's a reason to stress

nikomatsakis (Dec 23 2019 at 15:28, on Zulip):

For full context, I'm also putting in some effort onto the roadmap discussion, and I think it's ok to discuss this a bit in that light

nikomatsakis (Dec 23 2019 at 15:28, on Zulip):

I have already raised it a number of times there

matklad (Dec 23 2019 at 15:28, on Zulip):

Oh, right, this is very roadmap related

matklad (Dec 23 2019 at 15:29, on Zulip):

/me realizes that its that time of the year again

nikomatsakis (Dec 23 2019 at 15:30, on Zulip):

I'm debating the right way to describe this plan for RFC summary, seems like a very good exercise

matklad (Dec 23 2019 at 15:30, on Zulip):

So, what about this plan:

nikomatsakis (Dec 23 2019 at 15:31, on Zulip):

sounds like a plan; I don't know how much time I will have this week, plannin to mostly take vacation

nikomatsakis (Dec 23 2019 at 15:31, on Zulip):

I'm around today but maybe not after today

nikomatsakis (Dec 23 2019 at 15:31, on Zulip):

I can certainly send an e-mail to core tho

matklad (Dec 23 2019 at 15:32, on Zulip):

Yeah, I think we should target discussion after new year, there's no particular rush

nikomatsakis (Dec 23 2019 at 15:32, on Zulip):

right

nikomatsakis (Dec 23 2019 at 15:32, on Zulip):

I'm practicing being more "zen" :)

matklad (Dec 23 2019 at 15:33, on Zulip):

I am also not sure about my availabilyty for the next two weeks, and I also plan to implement that standard lazy types RFC...

nikomatsakis (Dec 23 2019 at 15:33, on Zulip):

ok. I will put in some time today, I don't think the RFC has to be "done" to start discussing

nikomatsakis (Dec 23 2019 at 15:33, on Zulip):

but it needs at least enough bullets that people don't "fill in the blanks"

matklad (Dec 23 2019 at 15:33, on Zulip):

:thumbs_up:

matklad (Dec 23 2019 at 15:34, on Zulip):

It might not be the best time to open RFC during hollidays though, as those folks who use Rust profeccionally might be on vacation from RUst things

matklad (Dec 23 2019 at 15:35, on Zulip):

or it might actually be the opposite vacation => more time for bikeshedding

nikomatsakis (Dec 23 2019 at 15:38, on Zulip):

I'm not planning on opening it until next week

nikomatsakis (Dec 23 2019 at 15:38, on Zulip):

if then :)

nikomatsakis (Dec 23 2019 at 15:39, on Zulip):

anyway not this week

nikomatsakis (Dec 30 2019 at 15:02, on Zulip):

@matklad not sure if you're around this week. I am but I'm a bit busy just now

matklad (Dec 30 2019 at 15:02, on Zulip):

It's time for the last update for this year :tada: :confetti:

Changelog is up, as usual: https://rust-analyzer.github.io/thisweek/2019/12/30/changelog-5.html

I don't have much to report personally, as I was traveling

matklad (Dec 30 2019 at 15:03, on Zulip):

But others were hard at work, so we landed a number of big features still!

matklad (Dec 30 2019 at 15:04, on Zulip):

I didn't get a change to fill-in the pre-RFC from the previous week sadly :(

nikomatsakis (Dec 30 2019 at 15:05, on Zulip):

Yeah, that's on my to-do list for today

nikomatsakis (Dec 30 2019 at 15:05, on Zulip):

Sounds like I should also upgrade my r-a installation

nikomatsakis (Dec 30 2019 at 15:05, on Zulip):

cargo watch would be nice :)

nikomatsakis (Dec 30 2019 at 15:06, on Zulip):

I did I think send an e-mail to core team as we discussed

nikomatsakis (Dec 30 2019 at 15:06, on Zulip):

but i've not checked my email to see if anyone responded :P

nikomatsakis (Dec 31 2019 at 21:14, on Zulip):

@matklad I put some more work into updating the draft RFC, though it's by no means complete

nikomatsakis (Jan 06 2020 at 15:04, on Zulip):

Hey @matklad -- you around this week?

matklad (Jan 06 2020 at 15:04, on Zulip):

So, let's have our first weekly update of the decade, @WG-rls2.0!

https://rust-analyzer.github.io/thisweek/2020/01/06/changelog-6.html

I sadly don't have a lot of to report this time: it was a mix of vacation and blogging about spinlocks for me :(

matklad (Jan 06 2020 at 15:04, on Zulip):

yup!

matklad (Jan 06 2020 at 15:04, on Zulip):

I haven't looked into the RFC yet, but I am going to do it today

nikomatsakis (Jan 06 2020 at 15:06, on Zulip):

OK. I don't have a ton to report. I had an epic conversation with @pnkfelix about traits, chalk, etc, and I am thinking a lot about next steps, but that's probably best discussed in #wg-traits.

nikomatsakis (Jan 06 2020 at 15:06, on Zulip):

I was wondering one thing, though

nikomatsakis (Jan 06 2020 at 15:07, on Zulip):

In general, you've several times relayed a wish to get more involvement from other t-compiler folks

matklad (Jan 06 2020 at 15:07, on Zulip):

yesssssss

nikomatsakis (Jan 06 2020 at 15:08, on Zulip):

I'm still feeling a bit unsure how to "engineer" that :) I guess one question I had was what you thought such involvement would look like. I am imagining feedback on various design questions ?

nikomatsakis (Jan 06 2020 at 15:09, on Zulip):

I guess also potentially directly hacking on rust-analyzer :)

nikomatsakis (Jan 06 2020 at 15:09, on Zulip):

It seems like getting aggressive about library-ification (as we are hoping) is the best way forward here

matklad (Jan 06 2020 at 15:10, on Zulip):

I think directly hacking is more important, as it is a way to get design feedback with teeth, so to say

nikomatsakis (Jan 06 2020 at 15:10, on Zulip):

I'm inclined to agree

matklad (Jan 06 2020 at 15:11, on Zulip):

It seems like getting aggressive about library-ification (as we are hoping) is the best way forward here

Yes. It helps tremendously that folks can work on chalk, for example

matklad (Jan 06 2020 at 15:11, on Zulip):

Though I do think that there's a lot of importatn design question about state management in rust-analyzer itself, and that won't be covered by librariification

nikomatsakis (Jan 06 2020 at 15:11, on Zulip):

one thing we've talked about is trying to spend more time planning out roadmaps, I wonder if it would be helpful to schedule some longer sessions (I think a lot of time is needed, e.g. potentially in the 2-4 hour region) to kind of "dump out" the thinking and plan out steps

nikomatsakis (Jan 06 2020 at 15:11, on Zulip):

felix and I did such a session in a cafe on friday for traits

nikomatsakis (Jan 06 2020 at 15:11, on Zulip):

and I'm still pondering it :)

nikomatsakis (Jan 06 2020 at 15:12, on Zulip):

side note: I feel like I owe you a lot of feedback re: salsa + encapsulation etc

nikomatsakis (Jan 06 2020 at 15:12, on Zulip):

not sure if you've talked much about the parser?

matklad (Jan 06 2020 at 15:12, on Zulip):

No, I didn't get to parser stuff still

nikomatsakis (Jan 06 2020 at 15:12, on Zulip):

Though I do think that there's a lot of importatn design question about state management in rust-analyzer itself, and that won't be covered by librariification

yeah; the initial steps are fairly modest somehow

matklad (Jan 06 2020 at 15:13, on Zulip):

I do think that planning roadmaps is important, but I feel that, for more technical roadmaps, some preliminary hacking is pre-requsite

nikomatsakis (Jan 06 2020 at 15:14, on Zulip):

yeah it's a difficult mix

nikomatsakis (Jan 06 2020 at 15:16, on Zulip):

OK, well, something to think about I guess. Anyway, I think we should try to finish up that RFC in any case, but I'm not sure how much it directly helps at getting people to hack some on rust-analyzer more.

matklad (Jan 06 2020 at 15:17, on Zulip):

FWIW, the general involvment with rust-analyzer is pretty great!

nikomatsakis (Jan 06 2020 at 15:17, on Zulip):

Yes! I shouldn't have written "people" so broadly

matklad (Jan 06 2020 at 15:17, on Zulip):

Probably bigger than what I can confortably review thoroughly already :)

nikomatsakis (Jan 06 2020 at 15:17, on Zulip):

(Did you see @pnkfelix's comments about writing up the steps to use rust-analyzer with rustc for various editors in the rustc-guide btw?)

nikomatsakis (Jan 06 2020 at 15:17, on Zulip):

Probably bigger than what I can confortably review thoroughly already :)

yeah, that's always a problem :)

matklad (Jan 06 2020 at 15:18, on Zulip):

It wasn't that bad with IntelliJ Rust -- probably fewer people know both kotlin & rust

matklad (Jan 06 2020 at 15:18, on Zulip):

Should have written rust-anlayzer in agda to solve this "problem" :D

detrumi (Jan 06 2020 at 15:18, on Zulip):

Heh, that wouldn't have stopped me

detrumi (Jan 06 2020 at 15:19, on Zulip):

But it seems like it's mostly you reviewing things, so that's a bottleneck

matklad (Jan 06 2020 at 15:19, on Zulip):

comments about writing up the steps to use rust-analyzer with rustc for various editors in the rustc-guide btw?

No, I only saw the poll

matklad (Jan 06 2020 at 15:20, on Zulip):

tbh, I haven't hacked on rustc in a while, so I am not sure I know the best tips for rust-anlayzer/rustc combo right now :)

pnkfelix (Jan 06 2020 at 15:42, on Zulip):

I'm thinking I might take up the task of writing up such docs

pnkfelix (Jan 06 2020 at 15:44, on Zulip):

if only to force myself to figure out what the options are for a "good" emacs setup. Up to this point, my experience with RA was with VSCode. (or rather, I started using VSCode just so I could see what RA was supposed to act like.)

Florian Diebold (Jan 06 2020 at 15:49, on Zulip):

I don't know anymore what the best emacs setup is currently... probably rustic, though it didn't play well with my spacemacs setup last time I tried. Our custom elisp code has actually mostly been integrated by emacs-lsp now, so the instructions in the repo are at least a bit outdated

nikomatsakis (Jan 13 2020 at 14:56, on Zulip):

@matklad (fyi, might be a bit late today)

matklad (Jan 13 2020 at 15:02, on Zulip):

@WG-rls2.0 this week's notes: https://rust-analyzer.github.io/thisweek/2020/01/13/changelog-7.html

matklad (Jan 13 2020 at 15:02, on Zulip):

I am super excited about today's release!

matklad (Jan 13 2020 at 15:04, on Zulip):

The most interesting bit, procedurally, is that we have binary releases now. Once we also add auto-update to the editor plugin, we'll be able to better answer the question about the best distribution strategy

matklad (Jan 13 2020 at 15:07, on Zulip):

I also opened a request for design meeting about parser library-ification: https://github.com/rust-lang/compiler-team/issues/237

In that proposal are links to the summary document about rust-analyzer's current approach to sytnax trees, as well as a plan I have in mind for sharing the parser

matklad (Jan 13 2020 at 15:08, on Zulip):

Also, @nikomatsakis I feel we should probably move forward with the rls/rust-analyzer RFC. I think I've added various bits of background info to the hackmd document

Joshua Nelson (Jan 13 2020 at 15:23, on Zulip):

I also opened a request for design meeting about parser library-ification: https://github.com/rust-lang/compiler-team/issues/237

That mentions merging the parsers for ra and rustc. Does that mean I should stop making PRs for the parser, since it's about to be in flux?

matklad (Jan 13 2020 at 15:25, on Zulip):

Does that mean I should stop making PRs for the parser, since it's about to be in flux?

Nope, I hope we can change everything incrementally and in small steps, without huge all-breaking PRs

nikomatsakis (Jan 13 2020 at 15:25, on Zulip):

On my side, I've been thinking a lot about the strategy for extracting a type library, and I'm beginning to question a bit the plan I had. In particular, I've been thinking it would make sense to start adjusting rustc to move it towards the proposed design -- as well perhaps as adding the bridge I was contemplating before -- and try to "meet in the middle" and then extract to a shared library.

nikomatsakis (Jan 13 2020 at 15:26, on Zulip):

The same sort of applies to the trait solver strategy and other things

nikomatsakis (Jan 13 2020 at 15:26, on Zulip):

I agree re: the RFC, @matklad, I guess I'll look over it -- I think it's probably ready to be posted soon? It seems to me that on the 'big picture' there aren't really many unknowns

nikomatsakis (Jan 13 2020 at 15:27, on Zulip):

I did a get few more e-mails from some core folks. @Nick Cameron in particular mentioned the fact that we ought to try and ensure a seamless transition, and I agree

matklad (Jan 13 2020 at 15:29, on Zulip):

Yeah, transition is a hard bit

matklad (Jan 13 2020 at 15:30, on Zulip):

I think we should have at least some transitionary period, when both are available, so that the users can switch when they are ready

matklad (Jan 13 2020 at 15:30, on Zulip):

but that makes automatic transition harder

Joshua Nelson (Jan 13 2020 at 15:30, on Zulip):

I'd be interested in benchmarking the parsers as well, I've noticed ra is usually faster than running cargo check

Joshua Nelson (Jan 13 2020 at 15:30, on Zulip):

not sure how much of that is due to semantic analysis

matklad (Jan 13 2020 at 15:31, on Zulip):

rust-analyzer parser was about 2X slower than rustc last time I've checked it (which was more than a year ago I guess?e)

detrumi (Jan 13 2020 at 15:32, on Zulip):

Makes sense, because rustc's parser probably has been optimized for a long time

matklad (Jan 13 2020 at 15:33, on Zulip):

For transition, I feel that accepting the RFC will take a non-trivial amount of time, during which we could figure out the details better.

I still have a hypothesis that maybe we want to do something different from a rustup component :)

matklad (Jan 13 2020 at 15:38, on Zulip):

Oh, one more chalk related thing: I've got an idea how to isolate chalk's panics and make them less painful: https://github.com/rust-analyzer/rust-analyzer/pull/2818

Basiacally, wrap catch_unwind around trait solver and return None on panics. This is tricker than expected, because chalk itself is not unwind safe, so we also have to reset the sate on panic

matklad (Jan 13 2020 at 15:58, on Zulip):

Oh, I totally forgot to mention that Emil Lauridsen heroically triaged all the issues on the repo, closed a bunch of obsolted ones, and worked with original issue reporters to get more info! This work sadly is invisible in git log --merges (that's why I missed it :( ), but is hugely important!

matklad (Jan 20 2020 at 15:08, on Zulip):

Note that thete's no sync up today: there's a holliday in USA, and Ferrous All Hands in Berlin

matklad (Jan 20 2020 at 15:08, on Zulip):

There is a release however, and I hope to get to writing release notes later today :)

std::Veetaha (Jan 20 2020 at 18:15, on Zulip):

Could you please elaborate on when and who is taking part in sync-ups and its process? Or if there is some reference info about them?

matklad (Jan 20 2020 at 18:17, on Zulip):

The only mandatory participants are matklad and nikomatsakis , but anyone else is free to join

matklad (Jan 20 2020 at 18:18, on Zulip):

In general, our development process is mostly chaotic and not super well defined, but it tends to work out OK in practice

detrumi (Jan 20 2020 at 18:18, on Zulip):

As for when, it's the same time every monday

std::Veetaha (Jan 20 2020 at 18:25, on Zulip):

okay, the same time, thank you!

matklad (Jan 27 2020 at 15:03, on Zulip):

And this week there are Mozilla all-hands, so we'll have only a short sync up I guess

matklad (Jan 27 2020 at 15:03, on Zulip):

The changelog is up here: https://rust-analyzer.github.io/thisweek/2020/01/27/changelog-9.html

matklad (Jan 27 2020 at 15:04, on Zulip):

As it was all-hands week for me last week, I didn't do many useful things. However, this release includes auto import, and semantic syntax highlighting inside macro calls.

matklad (Jan 27 2020 at 15:06, on Zulip):

It almost makes me think that I am not that useful for rust-analyzer anymore :laughter_tears:

detrumi (Jan 27 2020 at 15:10, on Zulip):

There's always room to do more :slight_smile:

detrumi (Jan 27 2020 at 15:10, on Zulip):

(also, shouldn't we ping people for the sync up?)

matklad (Jan 27 2020 at 15:12, on Zulip):

yeah, let's do this @WG-rls2.0

matklad (Jan 27 2020 at 15:26, on Zulip):

Actually, there's some thing I'd like to throw in the air...

It looks like we've accumulated a bunch of cases where rust-analyzer breaks. I think the primary case here is chalk, but I've also seen some utf8-boundary error somewhere, and we also ocasionaaly have panicy assists.

By itself, I think panics are OK: they are pretty isolated, and actually test, in practice, the resilience of rust-analyzer. What I don't really like though, is the fact that we've accumulated enougth panics to make it hard to notice new bugs/regressions

matklad (Jan 27 2020 at 15:26, on Zulip):

I wonder if we should maybe start moving towards zero bugs policy? Or at least zero panics?

detrumi (Jan 27 2020 at 15:29, on Zulip):

Isn't that kinda tricky because the errors are hard to track down?

std::Veetaha (Jan 27 2020 at 15:30, on Zulip):

Do you mean by moving towards zero panics that fixing bugs should be the priority?

matklad (Jan 27 2020 at 15:30, on Zulip):

Yeah

matklad (Jan 27 2020 at 15:30, on Zulip):

Basically, prioritizing fixes (even obscure ones, like "this needs to be fixed in chalk" over feature work)

detrumi (Jan 27 2020 at 15:31, on Zulip):

What if the fix requires a larger change, as often happens? Put a workaround in place to avoid crashing?

detrumi (Jan 27 2020 at 15:32, on Zulip):

(at first, while working on a real solution)

std::Veetaha (Jan 27 2020 at 15:34, on Zulip):

Well, removing bugs is always reasonable! I will try to finish with my changes to ra_syntax and take a look at adding sanititizers to better tackle issues with memory corruptions. Or does one think this wouldn't help/be possible?

std::Veetaha (Jan 27 2020 at 15:35, on Zulip):

issue

Jeremy Kolb (Jan 27 2020 at 15:37, on Zulip):

Could legitimate panics be hid by us using panics for cancellation?

matklad (Jan 27 2020 at 15:37, on Zulip):

@Jeremy Kolb nope, we specifically rethrough anything that is not the Canceled zst

matklad (Jan 27 2020 at 15:38, on Zulip):

@std::Veetaha i think that might help we have a repro. What confuses me though is that our fuzz tests (cargo xtask fuzz-tests) don't seem to catch anything....

Florian Diebold (Jan 27 2020 at 15:38, on Zulip):

concerning the Chalk problems, if you think it's the better trade-off, we could upgrade to current master and temporarily lose impl Trait support, but probably fix some panics and improve performance

Florian Diebold (Jan 27 2020 at 15:39, on Zulip):

I'm currently working on refactoring impl Trait handling on rust-analyzer side to prepare for the Chalk changes (and also to make it more correct), but it will take some more time

Florian Diebold (Jan 27 2020 at 15:40, on Zulip):

(I've realized that we need to handle argument-position impl trait by basically lowering to type variables anyway)

std::Veetaha (Jan 27 2020 at 15:41, on Zulip):

@matklad It seems that these fuzz tests generate overly-random data, I saw we could configure the fuzzer to emit something more like a rust source code.
It is called structure-aware fuzzing

detrumi (Jan 27 2020 at 15:42, on Zulip):

Ah, that's good to know for implementing impl trait on chalk's side

matklad (Jan 27 2020 at 15:43, on Zulip):

@Florian Diebold I think I prefer to move slowly with chalk in this case: upgrading to master might fix some, but uncover other bugs :)

Florian Diebold (Jan 27 2020 at 15:44, on Zulip):

@detrumi we might still use the impl Trait support for arguments when type-checking the function itself, I'm not sure... we could use the type param lowering there as well, I don't know yet what will work better

Florian Diebold (Jan 27 2020 at 15:45, on Zulip):

@matklad there's quite a bit to be gained in this case though, not just the panic fix but also fuel support

matklad (Jan 27 2020 at 15:46, on Zulip):

@Florian Diebold what would be overall less work?

matklad (Jan 27 2020 at 15:47, on Zulip):

To cut on round-trips, I think it makes sense to do whatever needs the less work overall, because optimizing for your time seems very reasonable :D

Florian Diebold (Jan 27 2020 at 15:48, on Zulip):

it's the same amount of work, I think

Florian Diebold (Jan 27 2020 at 15:49, on Zulip):

my refactorings are mostly independent, and upgrading now would mostly just involve deactivating a few tests

matklad (Jan 27 2020 at 15:49, on Zulip):

Hm, yeah, I don't have any opinion then :)

Florian Diebold (Jan 27 2020 at 15:55, on Zulip):

I guess we could at least see what effect the Chalk upgrade has on analysis-stats (performance and analysis wise)

matklad (Jan 27 2020 at 15:56, on Zulip):

Yeah, and also run it with --with-deps, to make sure nothing breaks horribly...

David Barsky (Jan 27 2020 at 16:16, on Zulip):

I guess we could at least see what effect the Chalk upgrade has on analysis-stats (performance and analysis wise)

@Florian Diebold I am not part of this wg, just a very big fan and user of rust-analyzer! That being said: your draft PR (https://github.com/rust-analyzer/rust-analyzer/pull/2872) would unblock me to use newer rust-analyzer on the main tracing repository, as ra crashes with that chalk bug. I don't even mind the lack of impl Trait, y'all have landed _so_ many amazing features in the last month and a half, and I'm really eager to try them out.

David Barsky (Jan 27 2020 at 16:18, on Zulip):

(I hope this does not come off as me asking you to do work! i'm just a customer/user who is voicing _one_ (hopefully easily ignorable!) opinion. y'all obviously have far more context than I on this project.

Florian Diebold (Jan 27 2020 at 16:25, on Zulip):

the work is already done anyway ;)

David Barsky (Jan 27 2020 at 16:28, on Zulip):

Understood! I have no clue how much effort it would be to disable the failing tests and I don't want to assume anything.

David Barsky (Jan 27 2020 at 16:29, on Zulip):

especially for a project that people contribute their own time to.

Jack Huey (Jan 27 2020 at 17:07, on Zulip):

I commented on the Chalk upgrade PR, but I personally recommend dropping impl Trait support for now in favor of upgrading

Jack Huey (Jan 27 2020 at 17:08, on Zulip):

Partly because the recent changes should fix a host of problems (mostly related to truncation) and allow limiting Chalk solving time.

Jack Huey (Jan 27 2020 at 17:08, on Zulip):

And partly because even though @detrumi is working on impl Trait, there still is some design work to do I think

Edwin Cheng (Jan 27 2020 at 18:28, on Zulip):

https://www.reddit.com/r/rust/comments/eus0cj/rustanalyzer_changelog_9/

Florian Diebold (Jan 27 2020 at 20:51, on Zulip):

current Chalk master does not crash (or hang) when analyzing RA, even if I set max size to 10. And setting fuel to 100 saves about 10% runtime on analysis, and doesn't seem to have much of an effect on unknown types (in fact it has slightly fewer unknown types than when I use 1000, which is probably because Chalk has less time to find ambiguities :thinking: )

Jack Huey (Jan 27 2020 at 21:50, on Zulip):

If you don't set any fuel, does it still crash or hang?

Jack Huey (Jan 27 2020 at 21:51, on Zulip):

This makes me super happy to hear

Florian Diebold (Jan 27 2020 at 22:34, on Zulip):

I haven't tried it without fuel

Joshua Nelson (Jan 30 2020 at 20:22, on Zulip):

my two cents as an outsider: I notice bugs in RA much more than I notice missing features. I'd much rather not have random errors pointing to imports instead of code than better type hints or anything like that

Joshua Nelson (Jan 30 2020 at 20:23, on Zulip):

also, I've put a fair amount of time into fuzzing https://github.com/jyn514/rcc/, so let me know if you want help with that :) Just giving AFL a bunch of valid [Rust] inputs usually gets good results

Joshua Nelson (Jan 30 2020 at 20:25, on Zulip):

AFAIK structure-aware fuzzing doesn't work very well with self-recursive data types, you can't #[derive(Arbitrary)] or anything like that

std::Veetaha (Jan 30 2020 at 21:00, on Zulip):

Take a star mate! Could you please elaborate on what AFL means?

Joshua Nelson (Jan 30 2020 at 21:01, on Zulip):

@std::Veetaha https://github.com/rust-fuzz/afl.rs/

Joshua Nelson (Jan 30 2020 at 21:02, on Zulip):

and this is what my fuzz setup looks like: https://github.com/jyn514/rcc/tree/master/fuzz

Joshua Nelson (Jan 30 2020 at 21:03, on Zulip):

although I've tried using hfuzz lately: https://github.com/jyn514/rcc/blob/hfuzz/fuzz/hfuzz.rs

std::Veetaha (Jan 30 2020 at 21:07, on Zulip):

What is the advantage of hfuzz?

Joshua Nelson (Jan 30 2020 at 21:08, on Zulip):

If you look at my issues page it's caught 12 _different_ panics in the last 2 weeks

Joshua Nelson (Jan 30 2020 at 21:08, on Zulip):

Honestly not much, a friend recommended it but it seems pretty similar to AFL

Joshua Nelson (Jan 30 2020 at 21:09, on Zulip):

I don't think he's on Zulip or I'd tag him

std::Veetaha (Jan 30 2020 at 21:10, on Zulip):

I wish I knew what was the secret...

matklad (Feb 03 2020 at 15:01, on Zulip):

@WG-rls2.0 weekly sync-up time!

We have the tenth changelog today: https://rust-analyzer.github.io/thisweek/2020/02/03/changelog-10.html

matklad (Feb 03 2020 at 15:02, on Zulip):

It is mostly stability&fixes release, a lot of bugs were fixed around cargo check (thanks @Emil Lauridsen!) and we've upgraded chalk to the fuel-enabled version!

matklad (Feb 03 2020 at 15:02, on Zulip):

@nikomatsakis are you around?

matklad (Feb 03 2020 at 15:03, on Zulip):

Another bigish thing is that I've finally get to learning more about sorbet, and it is really cool. I highly recommend reading the source code https://github.com/sorbet/sorbet

matklad (Feb 03 2020 at 15:04, on Zulip):

Their architecture is much simpler than ours, but doesn't handle LSP use-case too well.

std::Veetaha (Feb 03 2020 at 15:31, on Zulip):

FYI: Niko notified that he would not be available today

Florian Diebold (Feb 03 2020 at 15:36, on Zulip):

I'm still working on refactoring impl trait and type parameters in general, it's getting quite big, but in the end I think it'll get us closer to Chalk and fix a few bugs

std::Veetaha (Feb 03 2020 at 15:40, on Zulip):

I'm studying our VSCode extension codebase and VSCode api docs in parallel, hope to work on downloading github releases soon. Besides this I'd like to finish with this tech debt here after merging the linked PR

std::Veetaha (Feb 03 2020 at 15:42, on Zulip):

@matklad , I am surprised that you decided to remove seedrandom this is literally a package with 0 dependencies

matklad (Feb 03 2020 at 15:45, on Zulip):

it has zero dependnecies, but I've spend like 3x time figuring out how do you actually import seedrandom from seedrandom than it take me to write hashString and randomU32numbers :D

std::Veetaha (Feb 03 2020 at 15:46, on Zulip):

Yeah, the module system in TypeScript is very complicated and uneven, though that esModuleInterop was actually a good flag

std::Veetaha (Feb 03 2020 at 16:06, on Zulip):

@matklad how did you end up with studying sorbet and why this project in particular? Did you randomly saw that talk on YouTube?

Jeremy Kolb (Feb 03 2020 at 16:16, on Zulip):

If I have time this week I want to take a look at the latest semantic highlighting lsp bits

matklad (Feb 03 2020 at 16:24, on Zulip):

@std::Veetaha there was a link to sorbet in the top of lobste.rs recently, and then the author of olishell said that everyone must read sorbet's source code, and I trust their opinion

Emil Lauridsen (Feb 03 2020 at 16:27, on Zulip):

Now that it seems like cargo check is finally cooling down stability wise, I'd love to dive deeper into the infer infrastructure and chalk, and hopefully get custom BinOp inference working barring upstream issues.

Florian Diebold (Feb 03 2020 at 16:29, on Zulip):

the upstream issues (chalk#234) are still there, so it might work, but in many cases it might not give results :(

matklad (Feb 10 2020 at 15:05, on Zulip):

@WG-rls2.0 it's Monday again!

matklad (Feb 10 2020 at 15:05, on Zulip):

https://rust-analyzer.github.io/thisweek/2020/02/10/changelog-11.html

matklad (Feb 10 2020 at 15:06, on Zulip):

(Note that the actual release is still in progress; I feel like I should start cutting releases earlier, rust-analyzer moves fast and it takes a long time to catch up with all the stuff :-) )

Florian Diebold (Feb 10 2020 at 15:07, on Zulip):

I was just wondering whether we should mention in the change log that we kind of had impl trait support and removed it

matklad (Feb 10 2020 at 15:07, on Zulip):

I think we've mentioned it in the previous one?

detrumi (Feb 10 2020 at 15:07, on Zulip):

Not in Changelog 10, I think

Florian Diebold (Feb 10 2020 at 15:07, on Zulip):

I don't think we did

Florian Diebold (Feb 10 2020 at 15:08, on Zulip):

it just says "update Chalk. This should fix some cases of extremely slow trait checking."

Florian Diebold (Feb 10 2020 at 15:08, on Zulip):

I can send a PR

Florian Diebold (Feb 10 2020 at 15:08, on Zulip):

(for the current one)

matklad (Feb 10 2020 at 15:08, on Zulip):

sure, that would be nice!

std::Veetaha (Feb 10 2020 at 15:08, on Zulip):

Can we mention nodejs 10 -> nodejs 12 requirements change?

matklad (Feb 10 2020 at 15:09, on Zulip):

:thumbs_up:

detrumi (Feb 10 2020 at 15:09, on Zulip):

That's quite the requirement, isn't nodejs 10 the default on ubuntu and such?

std::Veetaha (Feb 10 2020 at 15:09, on Zulip):

And we need to update the documentation on installing via marketplace. Currently it says users to download .vsix file from the latest GitHub release

detrumi (Feb 10 2020 at 15:11, on Zulip):

(though maybe it's fine since it's only when you build from source)

std::Veetaha (Feb 10 2020 at 15:11, on Zulip):

I doubt that Ubuntu ships nodejs and npm at all

Jeremy Kolb (Feb 10 2020 at 15:11, on Zulip):

An apt-cache search nodejs on my ubuntu 18.04 is showing me 8?

Jeremy Kolb (Feb 10 2020 at 15:11, on Zulip):

Yes ubuntu ships nodejs and npm

matklad (Feb 10 2020 at 15:13, on Zulip):

So, highlights of the last week:

std::Veetaha (Feb 10 2020 at 15:13, on Zulip):

Well yes, this won't be such a breaking change for everyone, since this is to build from sources

Jeremy Kolb (Feb 10 2020 at 15:14, on Zulip):

Does the binary release work with the vscode remote extension?

matklad (Feb 10 2020 at 15:14, on Zulip):

This week I plan to focus on releases and docs, to try to make sure that by the next release the process is solid, and not as manual as today.

matklad (Feb 10 2020 at 15:14, on Zulip):

We also had a meeting with t-compiler about parser library-ification. Looks like everyone is on board in general terms, and that we should just do this work.

matklad (Feb 10 2020 at 15:16, on Zulip):

@nikomatsakis are you around? I'd love to sync up regarding the RFC :)

std::Veetaha (Feb 10 2020 at 15:29, on Zulip):

@matklad I would like discuss our validation logic in ra_syntax it you have time...

matklad (Feb 10 2020 at 15:31, on Zulip):

sure!

Florian Diebold (Feb 10 2020 at 15:41, on Zulip):

@matklad hmm the asciidoc seems to be broken in the first "new features" point?

Florian Diebold (Feb 10 2020 at 15:42, on Zulip):

also I assume "kbd::[Enter]" isn't supposed to look like that

Florian Diebold (Feb 10 2020 at 15:42, on Zulip):

and it'd probably be nicer to make "Marketplace" a link instead of having the full URL in the text

matklad (Feb 10 2020 at 15:45, on Zulip):

hmm the asciidoc seems to be broken in the first "new features" point?

Ah, so it's not the syntax highlighting in my editor that is broken :sweat_smile:

Florian Diebold (Feb 10 2020 at 15:46, on Zulip):

also, while I'm making suggestions anyway... maybe we could add the two sentences "rust-analyzer is an experimental modular compiler frontend for the Rust language. It is a part of a larger rls-2.0 effort to create excellent IDE support for Rust." to the beginning of each changelog and the home page? since it seems otherwise there's always someone on reddit who complains that the changelog doesn't say what RA is ;)

matklad (Feb 10 2020 at 15:46, on Zulip):

yeah, this is exactly something I want to fix this week

std::Veetaha (Feb 10 2020 at 15:47, on Zulip):

Also, on the home page of the website we can remove the sentence that rust-analyzer should be built from sources

matklad (Feb 10 2020 at 15:51, on Zulip):

Yeah, I plan to just rewrite all the docs from scratch

std::Veetaha (Feb 10 2020 at 15:53, on Zulip):

By the way, regarding rendering templates, there is much more Rust-y typesafe way of doing this: https://github.com/djc/askama

matklad (Feb 10 2020 at 15:56, on Zulip):

yeah, I know, I don't think though that templating engine (and, specifically, compile-time templating engine) makes sense for our use-case

Sander van Harmelen (Feb 10 2020 at 16:32, on Zulip):

Not sure where to ask, but I have a project that has 3 members in the [workspace] section of the top level Cargo file. Now it seems my editor isn't capable to handeling this properly. I'm using nvim+coc with the rust-analyzer plugin. When switching back to the RLS plugin it does work as expected. Is this a current limitation of the rust-analyzer (tested it with the release of a few days ago)?

matklad (Feb 10 2020 at 16:34, on Zulip):

That's roghtly the right place!

rust-analyzer itself is a workspace, so that use-case should be supported:.

matklad (Feb 10 2020 at 16:34, on Zulip):

What exactly does not work?

Sander van Harmelen (Feb 10 2020 at 16:36, on Zulip):

Hmm... Ok... Well it just stopped (when I changed my setup to a workspace configuration) giving we diagnostics. But let me revert back and see if I can see anything specific

matklad (Feb 10 2020 at 16:36, on Zulip):

We should provide diagnostics via cargo check in this situation, yeah

Sander van Harmelen (Feb 10 2020 at 16:43, on Zulip):

Yeah, as always it does seem to work now. I did close everything and reboot my mac since yesterday, so maybe that fixed something. Anyway, thanks for the confirmation that it should be supported. Good to know!

matklad (Feb 10 2020 at 16:44, on Zulip):

Glad to hear that it works now :)

std::Veetaha (Feb 10 2020 at 16:44, on Zulip):

Heh, easy bug

matklad (Feb 17 2020 at 15:06, on Zulip):

:wave: @WG-rls2.0, today's release is out (after four tries): https://rust-analyzer.github.io/thisweek/2020/02/17/changelog-12.html

matklad (Feb 17 2020 at 15:07, on Zulip):

I think the highlight of the week is the awesome work @std::Veetaha have been doing on the TypeScript extension. Specifically, it now downloads and upstes the server binary, which means that our release process is more or less finished!

We probably need to setup nightly builds as well some time, but that's lower priority

matklad (Feb 17 2020 at 15:08, on Zulip):

We've also got some refreshing of the docs: https://rust-analyzer.github.io/manual.html.

It currently includes only installation instructions, we should move feature and asssits docs to this new place as well

std::Veetaha (Feb 17 2020 at 15:12, on Zulip):

And we should probably use more recent screenshot of that download notification anyhow)

matklad (Feb 17 2020 at 15:12, on Zulip):

I personally would like to dedicate this week to paying down some technical debt:

std::Veetaha (Feb 17 2020 at 15:15, on Zulip):

And I would also want to raise a discussion on moving semantic errors reporting to ra_parser, though I always end up postponing this...

matklad (Feb 17 2020 at 15:16, on Zulip):

Yep, we at least need to bring that syntax error refactoring PR over the finish line

std::Veetaha (Feb 17 2020 at 15:17, on Zulip):

I think I will skip the part of adding tests for semantic validation and I'll just add some tests to that weird (to me) bug with merging errors on reprasing (which our current test don't test in anyway) and then I'll merge it.

Jeremy Kolb (Feb 17 2020 at 15:20, on Zulip):

Paying down tech debt is a really great idea

Jeremy Kolb (Feb 17 2020 at 15:24, on Zulip):

would implementing the lsp configuration bits help the config design? i feel like some stuff might fall out of that

matklad (Feb 17 2020 at 15:24, on Zulip):

Probably!

matklad (Feb 17 2020 at 15:24, on Zulip):

I still haven't looked at how LSP configuration should be handled :-(

Jeremy Kolb (Feb 17 2020 at 15:26, on Zulip):

@matklad https://github.com/microsoft/language-server-protocol/issues/676 gives some clarification

Jeremy Kolb (Feb 17 2020 at 15:27, on Zulip):

whereas things sent in the initialize request are more like command line arguments

std::Veetaha (Feb 17 2020 at 15:33, on Zulip):

#3139 bump VS Code requirenment to 1.41.

This is 1.42 actually @matklad

matklad (Feb 17 2020 at 15:34, on Zulip):

fixed, thanks!

std::Veetaha (Feb 17 2020 at 15:35, on Zulip):

#3174, #3174 improve debugging settings for VS Code extension.

Here is a typo

Laurențiu Nicola (Feb 17 2020 at 16:06, on Zulip):

Is it normal that some of the RA crates take ages to codegen?

Laurențiu Nicola (Feb 17 2020 at 16:07, on Zulip):

I guess they depend on generics-heavy code? I'm thinking of ra_ide, ra_assists, ra_lsp_server and ra_ide_db, mostly

matklad (Feb 17 2020 at 16:08, on Zulip):

Yeah, I was just looking at -Z timing earlier today

Laurențiu Nicola (Feb 17 2020 at 16:08, on Zulip):

ra_syntax builds in 8.6s, while ra_assists takes 74.8s

matklad (Feb 17 2020 at 16:09, on Zulip):

I think that's expected: ra_assists is the first crate to monomorphise the ton of db queries

Laurențiu Nicola (Feb 17 2020 at 16:17, on Zulip):

This is how it looks now:

pasted image

std::Veetaha (Feb 17 2020 at 16:17, on Zulip):

How did you build that tree?

Laurențiu Nicola (Feb 17 2020 at 16:18, on Zulip):

cargo build -Z timings --release

Jeremy Kolb (Feb 17 2020 at 16:19, on Zulip):

is that still nightly only?

Laurențiu Nicola (Feb 17 2020 at 16:19, on Zulip):

You could argue that ra_cli is not needed when building the LSP server, but..

Laurențiu Nicola (Feb 17 2020 at 16:19, on Zulip):

Uh, I don't know. I think every -Z flag is nightly-only.

Laurențiu Nicola (Feb 17 2020 at 16:21, on Zulip):

Maybe some generic code (the queries) is instantiated in every one of those crates? Could it be extracted into a separate crate that the others would depend upon instead?

matklad (Feb 17 2020 at 16:23, on Zulip):

Yeah, I think this is actually what might happen :thinking:

Without -Zshare-generics with --release, each crate does its own instantiatiosn

Laurențiu Nicola (Feb 17 2020 at 16:24, on Zulip):

Let me try that quickly

Jeremy Kolb (Feb 17 2020 at 16:26, on Zulip):

ugh this reminds me of the "instantiate all possibly C++ templates in one .cpp file"

Laurențiu Nicola (Feb 17 2020 at 16:27, on Zulip):

Yeah, bit better: pasted image

Laurențiu Nicola (Feb 17 2020 at 16:27, on Zulip):

They're more than twice as fast now and the full build is 1m 29s vs. 2m 33s.

Jeremy Kolb (Feb 17 2020 at 16:28, on Zulip):

wow is that from a clean build?

Laurențiu Nicola (Feb 17 2020 at 16:28, on Zulip):

Yeah

Laurențiu Nicola (Feb 17 2020 at 16:29, on Zulip):

(Sorry for hijacking the weekly sync thread, I tried to disjoin it, but I don't have permissions or I don't know how to)

matklad (Feb 17 2020 at 16:32, on Zulip):

@Laurențiu Nicola what was that? Just -Zshare-generics?

matklad (Feb 17 2020 at 16:32, on Zulip):

Are there any noticiable perf difference with analysis-stats?

Laurențiu Nicola (Feb 17 2020 at 16:45, on Zulip):

Yeah, it's much slower:

Database loaded, 191 roots, 158.113416ms
Crates in this dir: 30
Total modules found: 386
Total declarations: 6345
Total functions: 4740
Item Collection: 9.852548272s, 0b allocated 0b resident
Total expressions: 132346
Expressions of unknown type: 5431 (4%)
Expressions of partially unknown type: 3953 (2%)
Type mismatches: 266
Inference: 22.71735936s, 0b allocated 0b resident
Total: 32.569912625s, 0b allocated 0b resident

Database loaded, 191 roots, 1.343697027s
Crates in this dir: 30
Total modules found: 386
Total declarations: 6345
Total functions: 4740
Item Collection: 12.167735551s, 0b allocated 0b resident
Total expressions: 132346
Expressions of unknown type: 5423 (4%)
Expressions of partially unknown type: 3861 (2%)
Type mismatches: 266
Inference: 34.395031049s, 0b allocated 0b resident
Total: 46.562773422s, 0b allocated 0b resident
Laurențiu Nicola (Feb 17 2020 at 16:46, on Zulip):

Best of three runs. With -Z share-generics the first run was the fastest, which explains the larger "database loaded" time. The second run was:

Database loaded, 191 roots, 178.380958ms
Crates in this dir: 30
Total modules found: 386
Total declarations: 6345
Total functions: 4740
Item Collection: 12.137240132s, 0b allocated 0b resident
Total expressions: 132346
Expressions of unknown type: 5429 (4%)
Expressions of partially unknown type: 3944 (2%)
Type mismatches: 266
Inference: 34.593159173s, 0b allocated 0b resident
Total: 46.730405574s, 0b allocated 0b resident
matklad (Feb 24 2020 at 15:07, on Zulip):

Oups, @WG-rls2.0 almost missed today's sync up ;)

matklad (Feb 24 2020 at 15:07, on Zulip):

Today's changelog: https://rust-analyzer.github.io/thisweek/2020/02/24/changelog-13.html

matklad (Feb 24 2020 at 15:08, on Zulip):

It is a mostly quiet release with refactorings and bugfixes

matklad (Feb 24 2020 at 15:09, on Zulip):

Unfortunatelly, I wasn't able to complete many refactorings I've planed for the previous week

matklad (Feb 24 2020 at 15:09, on Zulip):

But the biggest one, https://github.com/rust-analyzer/rust-analyzer/pull/3222, shapes quite nicely so far

std::Veetaha (Feb 24 2020 at 15:16, on Zulip):

rust-analyzer.lspServerPath was actually raLspServerPath...

std::Veetaha (Feb 24 2020 at 15:22, on Zulip):

Do people really use such simple assits like remove mut? It's simple to just press Ctrl+Backspace to remove the mut symbol in one go. Such assists (even maybe useful for someone) clutter the assists selection menu. Should we provide feature flags for them?

matklad (Feb 24 2020 at 15:26, on Zulip):

I use it, because it deals with whitespace. This assist is only available only when the cursor is on the mut, so it dosn't clutters things that much

matklad (Feb 24 2020 at 15:26, on Zulip):

that's the most cool thing about asssist UX: they are only available in context where they make sense

Florian Diebold (Feb 24 2020 at 15:36, on Zulip):

@matklad if we could reliably detect whether the mut is unused, would you only show the assist in those cases?

Jeremy Kolb (Feb 24 2020 at 15:41, on Zulip):

The lsp allows for that. I think the next vscode release will allow for displaying assists disabled with a reason why it's not available.

std::Veetaha (Feb 24 2020 at 15:48, on Zulip):

Does it work when you place the caret on variable name symbol too? Otherwise for me this is just a shortcut which is not faster then Ctrl+Backspace

matklad (Feb 24 2020 at 15:48, on Zulip):

@Florian Diebold so I believe this needs to be both a quick fix for an error, but also an asssit/refactoring

matklad (Feb 24 2020 at 15:49, on Zulip):

The original usecase which prompted me was turning fn smth(&mut self) int fn smth(&self)

matklad (Feb 24 2020 at 15:49, on Zulip):

Otherwise for me this is just a shortcut which is not faster then Ctrl+Backspace

It is faster for me, because I don't have to fix up the whitespace

std::Veetaha (Feb 24 2020 at 15:50, on Zulip):

I press Ctrl+Backspace two times or Ctrl+Backspace .. Ctrl+Delete. For the assits its not faster, you press Alt+Enter and go through the list of assits to find remove mut

std::Veetaha (Feb 24 2020 at 15:52, on Zulip):

Well I won't argue if it's faster for you. You said that not all users want all features, that's why we implement feature flags

ice1000 (Mar 01 2020 at 08:18, on Zulip):

Hi, seems the vscode marketplace plugin is not updated since 24 Feb. Should we update it?

matklad (Mar 01 2020 at 09:21, on Zulip):

Updates are done weekly, on Monday, around 1600 CET

matklad (Mar 02 2020 at 15:02, on Zulip):

@WG-rls2.0 there's some annoying problems with release today, it'll be slightly late :(

std::Veetaha (Mar 02 2020 at 15:03, on Zulip):

What's wrong?

std::Veetaha (Mar 02 2020 at 15:04, on Zulip):

Oh, I see this is about the semantic highlighting

matklad (Mar 02 2020 at 15:05, on Zulip):

Many things are wrong:

Jeremy Kolb (Mar 02 2020 at 15:08, on Zulip):

I'm surprised that can't be commented out. I thought they supported comments

matklad (Mar 02 2020 at 15:08, on Zulip):

VS does, npm does not

std::Veetaha (Mar 02 2020 at 15:08, on Zulip):

Maybe we should not hurry? The world is not going nowhere without the release at 5 pm

matklad (Mar 02 2020 at 15:09, on Zulip):

Making releases on time is much easier for me than making releases "when it's ready"

matklad (Mar 02 2020 at 15:09, on Zulip):

anyway, that probably doesn't fir the weekly sync-up discussion

std::Veetaha (Mar 02 2020 at 15:10, on Zulip):

I mean, let's just give it a good thought to tackle the deployment bugs and not put hotfixes to release branch

matklad (Mar 02 2020 at 15:12, on Zulip):

So, the two biggest highlights of this week are:

matklad (Mar 02 2020 at 15:14, on Zulip):

I mean, let's just give it a good thought to tackle the deployment bugs and not put hotfixes to release branch

For me it's easier to get the releas out of the door and then fix stuff in between releases. I am also not feeling to comfortable arguing about philosophical issues regarding the release process :)

Laurențiu Nicola (Mar 02 2020 at 15:18, on Zulip):

But isn't the release already out?

matklad (Mar 02 2020 at 15:19, on Zulip):

Yup, it's out now! It just required some manual pushing to get to this state :D

std::Veetaha (Mar 02 2020 at 15:19, on Zulip):

Is hir::Semantics the implementation of SyntaxNode global identity?

Laurențiu Nicola (Mar 02 2020 at 15:19, on Zulip):

Ah :D

matklad (Mar 02 2020 at 15:20, on Zulip):

@std::Veetaha yeah, sort of. I really should blog about this pattern, as it is quite unusual, and is used in both Roslyn and Kotlin

std::Veetaha (Mar 02 2020 at 15:21, on Zulip):

Hmm, that didn't take so much to rewrite as you initially supposed, I guess

Edwin Cheng (Mar 02 2020 at 15:26, on Zulip):

@matklad you forgot to mention the change log :)

https://rust-analyzer.github.io/thisweek/2020/03/02/changelog-14.html

matklad (Mar 09 2020 at 14:04, on Zulip):

Hey @WG-rls2.0 time for another weekly sync up

matklad (Mar 09 2020 at 14:04, on Zulip):

(which might be an hour earlier, depending on DST in your timezone)

Florian Diebold (Mar 09 2020 at 14:04, on Zulip):

I was just wondering

matklad (Mar 09 2020 at 14:04, on Zulip):

Today's release: https://rust-analyzer.github.io/thisweek/2020/03/09/changelog-15.html

matklad (Mar 09 2020 at 14:05, on Zulip):

Thanks to work by @Florian Diebold and @Edwin Cheng (and many others, really) we now do code completion inside macro calls. This feels like a milestone to me :D

nikomatsakis (Mar 09 2020 at 14:06, on Zulip):

very cool indeed

matklad (Mar 09 2020 at 14:07, on Zulip):

Another curios thing is that we've disabled caching of Chalk, which improved perf and memory usage

matklad (Mar 09 2020 at 14:08, on Zulip):

I've also mostly finished the most pressing cleanups/refactorings, and am wondering what should I personally focus on next

nikomatsakis (Mar 09 2020 at 14:09, on Zulip):

Do you have a kind of "short list"?

matklad (Mar 09 2020 at 14:10, on Zulip):

Here are some things I want to do:

nikomatsakis (Mar 09 2020 at 14:11, on Zulip):

I'm skimming that 1st issue

matklad (Mar 09 2020 at 14:13, on Zulip):

I think I'd love to prioritize improving compile times (as it slows down everyone working on rust-analyzer), but the problem here is that I don't have any specific ideas here, except "make salsa database a concrete type"

nikomatsakis (Mar 09 2020 at 14:13, on Zulip):

I'm also wondering about -- more generally -- the idea of replacing RLS with rust-analyzer and what that really takes. We've kind of stalled out on that (sorry), but it seems like we're in a sort of weird spot where it's clear that rust-analyzer has the momentum but we've not talked about the overall transition story.

nikomatsakis (Mar 09 2020 at 14:14, on Zulip):

How much have you tried profiling compilation?

matklad (Mar 09 2020 at 14:14, on Zulip):

@nikomatsakis for the context, we've recently refactored rust-analyzer's IDE laye into ra_ide_db, which defines a concrete salsa database, and ra_assist, ra_ide crates, whcih provides features using that concrete DB

matklad (Mar 09 2020 at 14:14, on Zulip):

The problem with that is that now, I think, all three crates monomorphise type inference for the same types

matklad (Mar 09 2020 at 14:14, on Zulip):

(in `--release)

matklad (Mar 09 2020 at 14:15, on Zulip):

How much have you tried profiling compilation?

I've done cargo -Z timings, and it shows that each of the three crates I've mentioned above takes huge time to compile

Florian Diebold (Mar 09 2020 at 14:15, on Zulip):

TBH, I don't feel the compile times are a problem for me personally right now. Check is fast enough, I usually run single tests while developing, then all the tests for the crate, and a full cargo test once the PR is ready

matklad (Mar 09 2020 at 14:16, on Zulip):

I build in release pretty often, and it's easy 3-ish minutes after modifications to non-leaf crates.

matklad (Mar 09 2020 at 14:16, on Zulip):

I agree that it's not too bad at the moment, but it also won't get better unless we make it.

nikomatsakis (Mar 09 2020 at 14:19, on Zulip):

matklad said:

How much have you tried profiling compilation?

I've done cargo -Z timings, and it shows that each of the three crates I've mentioned above takes huge time to compile

I was thinking more of the self-profile

nikomatsakis (Mar 09 2020 at 14:19, on Zulip):

I believe it should be possible to drill in and get more details

nikomatsakis (Mar 09 2020 at 14:19, on Zulip):

and validate e.g. the hypothesis

nikomatsakis (Mar 09 2020 at 14:19, on Zulip):

that it is due to things being monomorphized multiple times

nikomatsakis (Mar 09 2020 at 14:20, on Zulip):

if it is, I'm not sure what I'd recommend but I wonder if we can come up with some kind of strategy that makes that less problematic

matklad (Mar 09 2020 at 14:20, on Zulip):

Yeah, it's a very good call to actually validate what's happening

nikomatsakis (Mar 09 2020 at 14:20, on Zulip):

side note that feel like -- perhaps because of my emacs setup -- I'm not experienced any of these improvements

nikomatsakis (Mar 09 2020 at 14:21, on Zulip):

when I use rust-analyzer on rustc, it's (honestly) a big step back from CTAGS in terms of successful jump-to-def, for example

nikomatsakis (Mar 09 2020 at 14:21, on Zulip):

part of this is that I can't jump-to-def from compilation errors and a few other places, because rust-analyzer isn't "hooked in" there, I could probably fix this

nikomatsakis (Mar 09 2020 at 14:21, on Zulip):

I'm wondering ifmy setup is wrong, but I'm also thinking that I will try to change my dev workflow for things like chalk to use vscode

nikomatsakis (Mar 09 2020 at 14:21, on Zulip):

because I feel like I have less of a "visceral" feel for what the problems are

nikomatsakis (Mar 09 2020 at 14:21, on Zulip):

(or lack of problems)

nikomatsakis (Mar 09 2020 at 14:22, on Zulip):

though it also reminds me of @pnkfelix's thoughts about documenting the "best practices" for setting up a rustc rust-analyzer experience

nikomatsakis (Mar 09 2020 at 14:22, on Zulip):

(in various editors)

matklad (Mar 09 2020 at 14:23, on Zulip):

Yeah, for goto def, we literally fall-back to ctags-like index, so it shouldn't be that bad

nikomatsakis (Mar 09 2020 at 14:24, on Zulip):

I'm also curious @matklad whether you see any value to trying to finish off that RFC we were working on. Specifically, I think the RFC would be about the plan to (1) embrace rust-analyzer as the editor of choice; (2) discuss how we will transition from RLS to rust-analyzer (e.g., rustup integration); (3) ratify perhaps the idea of not using save-analysis (and hence losing precise find-all-usages for the time being). At least (3) seemed to be the way you were leaning when we last discussed it and I think that I've come around to it.

nikomatsakis (Mar 09 2020 at 14:24, on Zulip):

matklad said:

Yeah, for goto def, we literally fall-back to ctags-like index, so it shouldn't be that bad

I'll try to take more careful note of where it doesn't work

nikomatsakis (Mar 09 2020 at 14:25, on Zulip):

If you are interested in trying to finish off the RFC, I think we could try to schedule some block of time to work on it. If you think it's not a good use of time, I'd like to briefly hear why =)

nikomatsakis (Mar 09 2020 at 14:25, on Zulip):

though I can imagine reasons

matklad (Mar 09 2020 at 14:25, on Zulip):

It might also make sense to chat directly with @Florian Diebold about emacs support. I've honestly haven't tried anything besides goto def

matklad (Mar 09 2020 at 14:26, on Zulip):

I do think it's a good idea to finish the RFC, for the purpose of informing community.

nikomatsakis (Mar 09 2020 at 14:26, on Zulip):

I'd like to put some time into that

matklad (Mar 09 2020 at 14:26, on Zulip):

I am not sure the RFC process will help us decide on any open questions, but it might

nikomatsakis (Mar 09 2020 at 14:27, on Zulip):

I'm curious if you think it'd be better for the two of us to e.g. schedule a 2h block or something to try and finish it off, or if it'd be better to work a bit independently

nikomatsakis (Mar 09 2020 at 14:27, on Zulip):

matklad said:

I am not sure the RFC process will help us decide on any open questions, but it might

maybe not the RFC process itself BUT

nikomatsakis (Mar 09 2020 at 14:27, on Zulip):

I think the process of writing out the questions

nikomatsakis (Mar 09 2020 at 14:27, on Zulip):

and perhaps touching base with others on tools team etc

nikomatsakis (Mar 09 2020 at 14:27, on Zulip):

very well could :)

nikomatsakis (Mar 09 2020 at 14:27, on Zulip):

that might hapen in RFC thread but might also happen before

matklad (Mar 09 2020 at 14:28, on Zulip):

e.g. schedule a 2h block or something to try and finish it off,

Yeah, I think some sync time would be helpful, especially if we literally agree to just publish the thing after two hours end :)

matklad (Mar 09 2020 at 14:29, on Zulip):

Ie, it is helpful to both once again survery the ground (we've sort-of changed our mind last time, we might do it once again) and to put a deadline

nikomatsakis (Mar 09 2020 at 14:31, on Zulip):

OK, let's block out some time.

nikomatsakis (Mar 09 2020 at 14:31, on Zulip):

I'm looking at my calendar :)

nikomatsakis (Mar 09 2020 at 14:34, on Zulip):

tomorrow starting at the same time as this sync could work

matklad (Mar 09 2020 at 14:34, on Zulip):

Works for me!

Laurențiu Nicola (Mar 09 2020 at 21:00, on Zulip):

nikomatsakis said:

I was thinking more of the self-profile
````

I tried that, but I'm not sure how to interpret the output. It looks like ThinLTO takes about half of the compile time (40s) of that/those crates and it has poor parallelism, but I might be looking at it the wrong way.
matklad (Mar 16 2020 at 14:02, on Zulip):

@WG-rls2.0 it's a sync-up time again

matklad (Mar 16 2020 at 14:02, on Zulip):

Today's release notes: https://rust-analyzer.github.io/thisweek/2020/03/16/changelog-16.html

matklad (Mar 16 2020 at 14:03, on Zulip):

There is a bunch of exciting stuff happening. So much, in fact, that I am not sure I am on top of everything (had a busy week last week)

matklad (Mar 16 2020 at 14:06, on Zulip):

@Edwin Cheng and @Emil Lauridsen are doing amazing work at supporting include!(concat!(env!)) macro.

@std::Veetaha implemented nightly updates

@Florian Diebold did some exciting work with making macro by example expansion resilient to errors (which helps completion)

And I've investigated compile times a bit, with moderate success.

matklad (Mar 16 2020 at 14:07, on Zulip):

@nikomatsakis and me also worked a bit on the RFC about further IDE support

Jeremy Kolb (Mar 16 2020 at 14:31, on Zulip):

FYI with vscode 1.43.1 it looks like you'll have to be on a default theme (or opt into it in a custom theme) to get semantic highlighting until they open it up: https://github.com/microsoft/vscode/pull/92737

std::Veetaha (Mar 16 2020 at 14:39, on Zulip):

Yeah, recently switched to vscode-insiders to test that. This is a killer feature for me, big thanks @Jeremy Kolb !

Florian Diebold (Mar 16 2020 at 14:42, on Zulip):

I also started work on resurrecting a recursive solver for Chalk, as a possible solution to the associated types problem, before getting side-tracked with the macros thing :sweat_smile: so I hope to finish the macros PR soon and get back to that

Edwin Cheng (Mar 16 2020 at 14:45, on Zulip):

I am working on a proc-macro support POC (custom-derive) , I think we finish in these 2 weeks :)

std::Veetaha (Mar 16 2020 at 14:48, on Zulip):

Also a note about nightlys. There is a slight problem that the weekly release is published from the fresh master branch, but the current nightly release doesn't contain last changes. This means that by switching to the nightly channel right now you are going to downgrade to the version without nightlys support ;D

matklad (Mar 23 2020 at 14:02, on Zulip):

:wave: @WG-rls2.0 ! It's time for another sync-up

matklad (Mar 23 2020 at 14:03, on Zulip):

The release is still in progress: have to do some last-minute yaml fixes

matklad (Mar 23 2020 at 14:05, on Zulip):

The three biggest things this week, I think, are:

matklad (Mar 23 2020 at 14:05, on Zulip):

Release failed :c

std::Veetaha (Mar 23 2020 at 14:06, on Zulip):

haha, classic

nikomatsakis (Mar 23 2020 at 14:07, on Zulip):

Was changing to &dyn DB difficult, @matklad ?

matklad (Mar 23 2020 at 14:09, on Zulip):

@nikomatsakis not really.

matklad (Mar 23 2020 at 14:09, on Zulip):

We were unable to do that originally because we had some cleverness around the requires

matklad (Mar 23 2020 at 14:10, on Zulip):

But, when we split our hir into many crates early this year, we removed requires in favor of simpler design

matklad (Mar 23 2020 at 14:10, on Zulip):

So now switch was possible, albeit it required some explicit upcasting of trait objetcs

matklad (Mar 23 2020 at 14:11, on Zulip):

https://github.com/rust-analyzer/rust-analyzer/blob/aaa6961aa6d19b405dd2e837a09ac96ed6ace995/crates/ra_hir_ty/src/db.rs#L23

matklad (Mar 23 2020 at 14:11, on Zulip):

(Upcast<dyn DefDB> is the upcasing bit)

nikomatsakis (Mar 23 2020 at 14:14, on Zulip):

interesting

matklad (Mar 23 2020 at 14:14, on Zulip):

@nikomatsakis the (not super rigorously measured) effects of the switch are:

nikomatsakis (Mar 23 2020 at 14:15, on Zulip):

really interesting

nikomatsakis (Mar 23 2020 at 14:15, on Zulip):

33% faster is a pretty big deal

std::Veetaha (Mar 23 2020 at 14:16, on Zulip):

Ideally, there could be a compiler option not to monomorphize stuff)

matklad (Mar 23 2020 at 14:16, on Zulip):

yup. cargo timing measures are here: https://github.com/rust-analyzer/rust-analyzer/issues/1987#issuecomment-598663651

matklad (Mar 23 2020 at 14:39, on Zulip):

belated changelog: https://rust-analyzer.github.io/thisweek/2020/03/23/changelog-17.html

std::Veetaha (Mar 23 2020 at 14:47, on Zulip):

@matklad, what do you think about the proposal from bjorn to use this bit of code from rustc. Its obvious that the read2() function should be extracted into a reusable module. How do you see doing it?

matklad (Mar 23 2020 at 14:55, on Zulip):

I think, ideally, this should be somehow exposed on std::process::Command directly, as that's needed for implementation of .output anyway. For the time being, I'm fine with either copy-pasting that bit of code from cargo, or publishing a read2 micro crate.

I'd also love to be able to completely bypass the need to read stdout and stderr in lockstep, but I don't know of that's possible

Laurențiu Nicola (Mar 23 2020 at 15:00, on Zulip):

See also https://github.com/joshtriplett/io-mux, which might be a bit more reliable wrt. ordering (but is Linux-only)

matklad (Mar 23 2020 at 15:01, on Zulip):

ohh, the API is kindof what we want, but the impl is overkill

Laurențiu Nicola (Mar 23 2020 at 15:02, on Zulip):

It's not so bad, except for the Linux-only part. That was a fun one to figure out.

std::Veetaha (Mar 23 2020 at 15:14, on Zulip):

I'd also love to be able to completely bypass the need to read stdout and stderr in lockstep, but I don't know of that's possible

What's bad in this read2() do you see?

Laurențiu Nicola (Mar 23 2020 at 15:20, on Zulip):

It's very hard to read like this while preserving the ordering. The code there reads from both pipes, but with no ordering guarantees.

matklad (Mar 30 2020 at 12:59, on Zulip):

(note, due to DST shift, the sync-up starts in an hour)

matklad (Mar 30 2020 at 14:01, on Zulip):

@WG-rls2.0 hey, it's time for another meeting!

matklad (Mar 30 2020 at 14:01, on Zulip):

Current changelog: https://rust-analyzer.github.io/thisweek/2020/03/30/changelog-18.html

matklad (Mar 30 2020 at 14:04, on Zulip):

Some highlights:

matklad (Mar 30 2020 at 14:05, on Zulip):

@Florian Diebold hows the recursive solver going? :)

Florian Diebold (Mar 30 2020 at 14:08, on Zulip):

it's making some kind of progress, there's still a bunch of test failures in Chalk, and performance isn't great, but it does seem to fix chalk#234 right now...

Jeremy Kolb (Mar 30 2020 at 14:13, on Zulip):

this is becoming ridiculously awesome

nikomatsakis (Mar 30 2020 at 14:13, on Zulip):

Minor note that we are beginning another chalk sprint

nikomatsakis (Mar 30 2020 at 14:14, on Zulip):

we posted a summary of last sprint and our current thoughts around goals, but I guess this is a chance to ask @Florian Diebold if there are particular things that would help with rust-analyzer -- it'd be good for you and I to sync up some more on the recursive solver, and I guess the other thing would be to put some energy into salsa in terms of extsending the model to support caching?

nikomatsakis (Mar 30 2020 at 14:15, on Zulip):

(I am reminded I've been hiding from the salsa zulip :eep:)

nikomatsakis (Mar 30 2020 at 14:15, on Zulip):

I wish :eep: was an emoji

Florian Diebold (Mar 30 2020 at 14:16, on Zulip):

nikomatsakis said:

it'd be good for you and I to sync up some more on the recursive solver, and I guess the other thing would be to put some energy into salsa in terms of extsending the model to support caching?

that pretty much, I think :slight_smile:

matklad (Mar 30 2020 at 14:16, on Zulip):

Yeah, at some point soon (tm), I'd like to maybe rethink salsa's abstractions...

matklad (Mar 30 2020 at 14:16, on Zulip):

Specifically, one think I stumble on repeatedly is that we store tree-shaped data in salsa

nikomatsakis (Mar 30 2020 at 14:17, on Zulip):

Say more?

nikomatsakis (Mar 30 2020 at 14:17, on Zulip):

(but yes I'm open to revisiting or at least repackaging the core ideas...)

matklad (Mar 30 2020 at 14:17, on Zulip):

So, our core crate_def_map query, which takes a crate id as an input, returns a tree of module

nikomatsakis (Mar 30 2020 at 14:17, on Zulip):

btw @matklad I was hoping that we could finish that RFC this week. I also talked some to @Igor Matuszewski about it.

matklad (Mar 30 2020 at 14:18, on Zulip):

or a lower-level query which takes a file id, and returns a tree of items

matklad (Mar 30 2020 at 14:18, on Zulip):

So, the layout in memory is a bunch of small trees

matklad (Mar 30 2020 at 14:18, on Zulip):

I'd love to see the layout as a bunch of arrays instead

matklad (Mar 30 2020 at 14:18, on Zulip):

Ie, all modules from all lcrates are stored in a single vec.

Laurențiu Nicola (Mar 30 2020 at 14:19, on Zulip):

ahem, SQLite :grimacing:

Laurențiu Nicola (Mar 30 2020 at 14:19, on Zulip):

/me runs away and hides

matklad (Mar 30 2020 at 14:20, on Zulip):

well, yes, persistence is obviously another huge missing wall in the whole rust-analyzer castle....

nikomatsakis (Mar 30 2020 at 14:20, on Zulip):

so, you could "easily enough" represent those trees as vectors

nikomatsakis (Mar 30 2020 at 14:21, on Zulip):

then you have a "bunch of vectors"

matklad (Mar 30 2020 at 14:21, on Zulip):

I was hoping that we could finish that RFC this week

@nikomatsakis are there any specific action items for me in this regard

nikomatsakis (Mar 30 2020 at 14:21, on Zulip):

(this is what we did in Lark for things like fn bodies)

matklad (Mar 30 2020 at 14:21, on Zulip):

so, you could "easily enough" represent those trees as vectors

yes, that's what I actually do

nikomatsakis (Mar 30 2020 at 14:21, on Zulip):

is that something you've considered and rejected? not satisfying enough..?

nikomatsakis (Mar 30 2020 at 14:21, on Zulip):

ok

nikomatsakis (Mar 30 2020 at 14:21, on Zulip):

this is purely a matter of performance?

matklad (Mar 30 2020 at 14:21, on Zulip):

But, still, a bunch of vectors seems somehow wrong

nikomatsakis (Mar 30 2020 at 14:21, on Zulip):

that's not obvious to me :)

nikomatsakis (Mar 30 2020 at 14:21, on Zulip):

but ok

nikomatsakis (Mar 30 2020 at 14:22, on Zulip):

I'd like to understand better what seems to be missing

matklad (Mar 30 2020 at 14:22, on Zulip):

One specific problem is that you'd then need to add explicit projections on top

nikomatsakis (Mar 30 2020 at 14:22, on Zulip):

matklad said:

are there any specific action items for me in this regard

maybe not, I think we may be just basically "ready" to post

matklad (Mar 30 2020 at 14:23, on Zulip):

One specific problem is that you'd then need to add explicit projections on top

I was thinking about a cool (but unsure if sound) scheme with reverse, push queries.

Basically, you run "lower_file" query, and that query calls a bunch of set_ queries to create children structs, function, modules

matklad (Mar 30 2020 at 14:24, on Zulip):

so, eg, a struct remebers that it was created as a side-effect of lowering file with specific ID.

nikomatsakis (Mar 30 2020 at 15:01, on Zulip):

Reminds me a bit of adapton. It sounds plausible but I'm not sure yet what those set queries would do

nikomatsakis (Mar 30 2020 at 15:01, on Zulip):

I still tend to favor a more uniform, pull-based setup, I guess one question I have is: is this primarily an ergonomics concern? An efficiency one?

nikomatsakis (Mar 30 2020 at 15:01, on Zulip):

Anyway, maybe we should split this conversation out to somewhere else I guess

nikomatsakis (Mar 30 2020 at 15:02, on Zulip):

/me has to run right now anyway, bbl

matklad (Mar 30 2020 at 15:04, on Zulip):

I guess it is primary ergonomics. Definining arenas like this seems like a busy-work. I'd love to be able to code and think in terms of entities, without defining storage explicitely.

matklad (Apr 06 2020 at 14:00, on Zulip):

Hello @WG-rls2.0 !

matklad (Apr 06 2020 at 14:00, on Zulip):

Today's changelog is here: https://rust-analyzer.github.io/thisweek/2020/04/06/changelog-19.html

matklad (Apr 06 2020 at 14:00, on Zulip):

Notable things:

matklad (Apr 06 2020 at 14:00, on Zulip):
matklad (Apr 06 2020 at 14:01, on Zulip):
matklad (Apr 06 2020 at 14:01, on Zulip):

And, most significant I think, https://github.com/rust-lang/rust/pull/70761 by @Luca Barbieri

matklad (Apr 06 2020 at 14:02, on Zulip):

This out-of-the blue PR ports rustc to use ra_parser and ra_syntax syntax trees.

nikomatsakis (Apr 06 2020 at 14:02, on Zulip):

fascinating

matklad (Apr 06 2020 at 14:03, on Zulip):

I've tried it out and indeed was able to compile a non-trivial crate with ra as a parser :D

matklad (Apr 06 2020 at 14:03, on Zulip):

Note that PR is rather a proof of concept that something on the road to merging I think

detrumi (Apr 06 2020 at 14:04, on Zulip):

So the main limitation is that it can't parse Rust 2015 syntax, because rust-analyzer only supports the 2018 edition currently

matklad (Apr 06 2020 at 14:04, on Zulip):

But it shows that the appoarch of lowering concrece syntax tree to ast (as opposed to even-based parsing approahc) is viable, and actually not that bad, diff wise

matklad (Apr 06 2020 at 14:04, on Zulip):

the PR adds about 3k likes to rustc

nikomatsakis (Apr 06 2020 at 14:04, on Zulip):

not that bad in the sense of "code complexity"?

detrumi (Apr 06 2020 at 14:04, on Zulip):

idk, merging it might avoid bitrot, though it also adds some cost to keep it up to date

nikomatsakis (Apr 06 2020 at 14:05, on Zulip):

one obvious concern I would have is build times

matklad (Apr 06 2020 at 14:05, on Zulip):

build times of rustc itself, or compile times of code compiled with rustc?

nikomatsakis (Apr 06 2020 at 14:05, on Zulip):

the latter

nikomatsakis (Apr 06 2020 at 14:05, on Zulip):

i.e., adding another IR to be converted requires more allocation, etc,

matklad (Apr 06 2020 at 14:05, on Zulip):

The latter is my main concern as well indeed.

nikomatsakis (Apr 06 2020 at 14:06, on Zulip):

so I would expect it to be slower

matklad (Apr 06 2020 at 14:06, on Zulip):

I think we can now answer "how much slower" actually

matklad (Apr 06 2020 at 14:07, on Zulip):

With the caveat that the PR is enables only for real files, as far as i understand. All code generated by macros is still parsed with the old parser

Jeremy Kolb (Apr 06 2020 at 14:07, on Zulip):

I think the changelog entry with the code-insiders invocation is missing --enable-proposed-api

Jeremy Kolb (Apr 06 2020 at 14:07, on Zulip):

though hopefully we can change that tomorrow when the new stable vscode is supposed to be out

matklad (Apr 06 2020 at 14:07, on Zulip):

@Jeremy Kolb fixed!

nikomatsakis (Apr 06 2020 at 14:07, on Zulip):

we can certainly test the perf of the current PR, if that's what you mean

matklad (Apr 06 2020 at 14:08, on Zulip):

yup, that's what I mean, but I am not sure if that would be useful, if we don't use this approach in the first place

matklad (Apr 06 2020 at 14:09, on Zulip):

Like, I do think that eventually we should remove ast and just use cst.

matklad (Apr 06 2020 at 14:09, on Zulip):

But for transition the event thing seems somewhat more promising.

nikomatsakis (Apr 06 2020 at 14:09, on Zulip):

So, one thing I was thinking about for this week is to try and get back into salsa a bit, and in particular to look into what it would take to support the recursive solver that @Florian Diebold has been working on "properly"

nikomatsakis (Apr 06 2020 at 14:09, on Zulip):

(On that note, I still don't fully understand what you were talking about last time, but I guess maybe we should revisit that over in the Salsa zulip)

matklad (Apr 06 2020 at 14:10, on Zulip):

Oh, that actually remins me about a bit which has seen little progress: I wasn't able to put a lot of time into the new item-tree ir unfortunately.

matklad (Apr 06 2020 at 14:11, on Zulip):

(On that note, I still don't fully understand what you were talking about last time, but I guess maybe we should revisit that over in the Salsa zulip)

+1

nikomatsakis (Apr 06 2020 at 14:14, on Zulip):

Oh and

nikomatsakis (Apr 06 2020 at 14:15, on Zulip):

we should mention that x.py in the compiler should now support things better

nikomatsakis (Apr 06 2020 at 14:15, on Zulip):

Is there a rustc-dev-guide chaper guiding folks in how to setup and use rust-analyezr with rusc?

nikomatsakis (Apr 06 2020 at 14:15, on Zulip):

I remember @pnkfelix talking about it; if so, it would want to include the .vscode/settings.toml etc.

matklad (Apr 06 2020 at 14:15, on Zulip):

@pnkfelix ^^ ?

nikomatsakis (Apr 06 2020 at 14:15, on Zulip):

It's not like "particularly hard" to do

nikomatsakis (Apr 06 2020 at 14:16, on Zulip):

But I do think having a list of steps would be really nice

pnkfelix (Apr 06 2020 at 14:16, on Zulip):

I haven't written anything on that yet

pnkfelix (Apr 06 2020 at 14:16, on Zulip):

I had wanted to do a survey of the common editors

pnkfelix (Apr 06 2020 at 14:16, on Zulip):

and I think that sent me down the rabbit hole of exploring neovim

pnkfelix (Apr 06 2020 at 14:17, on Zulip):

(I think part of the problem is that there are different instructions depending on which variant of vim one uses? Not sure if that is still true, but when I was looking it seemed to be the case, and so I wanted to figure out which variant of vim was the best one to document...)

matklad (Apr 06 2020 at 14:18, on Zulip):

@pnkfelix yeah, that's the biggest problem, every editor requires n, where n >= 1 instructions

matklad (Apr 06 2020 at 14:19, on Zulip):

For rust-analyzer, I just made the manual easy to edit, hoping that folks would crowd-write it.

nikomatsakis (Apr 06 2020 at 14:19, on Zulip):

I would definitelyl suggest we just get started with at least documenting vscode :)

nikomatsakis (Apr 06 2020 at 14:19, on Zulip):

That said

nikomatsakis (Apr 06 2020 at 14:20, on Zulip):

@matklad the cargo watch functionality, is that built into the VSCode plugin or is it also part of the LSP server itself in some way?

nikomatsakis (Apr 06 2020 at 14:20, on Zulip):

I remember us talking about moving it around

matklad (Apr 06 2020 at 14:20, on Zulip):

That's now LSP-level functionality

matklad (Apr 06 2020 at 14:20, on Zulip):

So, it should be supported in any editor

nikomatsakis (Apr 06 2020 at 14:21, on Zulip):

Presumably the way that that "Cargo watch" command gets customized would also depend on your editor, though?

nikomatsakis (Apr 06 2020 at 14:21, on Zulip):

e.g., the emacs plugin may just not support it?

matklad (Apr 06 2020 at 14:22, on Zulip):

So, the LSP has the concept of configuration built-in, and, if editor supports LSP, it should be able to set arbitrary options

matklad (Apr 06 2020 at 14:22, on Zulip):

but it is indeed true that this would be per-editor config. IE, we don't really have a rust-analyzer.toml config file.

matklad (Apr 06 2020 at 14:24, on Zulip):

Which is sort-of a limitation of the protocol:

I think we can bend the protocol here, and merge rust-analyzer.toml with whatever the editor sends us

std::Veetaha (Apr 06 2020 at 14:26, on Zulip):

What is the purpose of rust-analyzer.toml? To override the single flycheck command?

matklad (Apr 06 2020 at 14:27, on Zulip):

To have something you can put into a git repo, which would be picked up by all editors, and not just by vs code

nikomatsakis (Apr 06 2020 at 14:28, on Zulip):

we could, but it's not clear it's necessary, I guess we'd want to check if e.g. the emacs lsp-mode etc supports some way to set options

nikomatsakis (Apr 06 2020 at 14:28, on Zulip):

I bet it does

Florian Diebold (Apr 06 2020 at 14:30, on Zulip):

it does (via the normal emacs way, so you could have a .dir-locals.el)

std::Veetaha (Apr 06 2020 at 14:34, on Zulip):

I think configuring the editor is very personal thing. Not everyone would like to use that x.py script and she should not be forced to. Just documenting the recommended setup seems alright.

Jeremy Kolb (Apr 06 2020 at 14:38, on Zulip):

We might need to start collecting LSP bits we're missing in different editors too. We mostly ignore client caps right now and just assume things like "markdown is supported"

Florian Diebold (Apr 06 2020 at 14:39, on Zulip):

I think there are many settings where it would make sense to configure them per repo, like files to ignore

std::Veetaha (Apr 06 2020 at 14:40, on Zulip):

This also seems arrogant to put rust-analyzer.toml to the repo, whilst some folks may be using intelij

matklad (Apr 06 2020 at 14:43, on Zulip):

rust-analyzer.toml in no way precludes adding .idea. Honestly, I feel like the benefit of sharable config should be abundantly clear :)

std::Veetaha (Apr 06 2020 at 14:55, on Zulip):

Only if people don't save personal configs in this .idea folder ...

std::Veetaha (Apr 06 2020 at 15:02, on Zulip):

The fact that we commit .vscode folder already looks weird. So each time when we modify the debug configs for vscode we have to keep an eye on not committing this change.
The better approach would be to have a separate dir say recommended-editor-configs where people can look and copy the editor configs they want and be able to tweak them

matklad (Apr 06 2020 at 15:06, on Zulip):

I guess we can agree to disagree on the merits of sharing project config :)

Edwin Cheng (Apr 06 2020 at 15:09, on Zulip):

And I want to report there are around 2 more PRs to complete the proc-macro custom derive feature.

Florian Diebold (Apr 06 2020 at 15:09, on Zulip):

the whole point is that you only put the settings you do want to share into the rust-analyzer.toml ;)

Florian Diebold (Apr 06 2020 at 15:19, on Zulip):

Edwin Cheng said:

And I want to report there are around 2 more PRs to complete the proc-macro custom derive feature.

with this and all the Chalk work going on, I'm hopeful we'll have pretty complete type inference (modulo bugs) in a few months :)

matklad (Apr 06 2020 at 15:20, on Zulip):

I am scared about perf though :grimacing:

Edwin Cheng (Apr 06 2020 at 15:25, on Zulip):

@matklad I remember you mentioned you want some setup around --bench , e.g. auto-compare some source files. Do you remember which issue is it ? I would like to work on that task after these PRs.

matklad (Apr 06 2020 at 15:26, on Zulip):

https://github.com/rust-analyzer/rust-analyzer/issues/492 ?

Edwin Cheng (Apr 06 2020 at 15:28, on Zulip):

yes, thank you!

Laurențiu Nicola (Apr 07 2020 at 18:11, on Zulip):

"editor.semanticHighlighting.enabled": false

Is this right? (from the changelog)

nikomatsakis (Apr 13 2020 at 14:12, on Zulip):

Florian Diebold said:

it does (via the normal emacs way, so you could have a .dir-locals.el)

ps, @Florian Diebold, got any links or things to "apropos for" in order to find out how to set "custom settings" in emacs-lsp?

matklad (Apr 13 2020 at 14:16, on Zulip):

ouch, forgot to mention that there's no weekly today because there's a holiday here.

Florian Diebold (Apr 13 2020 at 14:16, on Zulip):

there's a bunch of customization variables, e.g. lsp-rust-analyzer-cargo-watch-command, you just have to make sure to set them before the server gets started. Also note that the setting names changed somewhat recently, there's been a commit in emacs-lsp to adapt to that but I haven't tested it since then. And I don't know whether the settings are complete

matklad (Apr 13 2020 at 14:16, on Zulip):

(but actually, because I feel as asleep after writing an article on the Pratt parsing)

nikomatsakis (Apr 13 2020 at 14:22, on Zulip):

Florian Diebold said:

there's a bunch of customization variables, e.g. lsp-rust-analyzer-cargo-watch-command, you just have to make sure to set them before the server gets started. Also note that the setting names changed somewhat recently, there's been a commit in emacs-lsp to adapt to that but I haven't tested it since then. And I don't know whether the settings are complete

ah wacky

nikomatsakis (Apr 13 2020 at 14:22, on Zulip):

I somehow expected a more generic mechanism

nikomatsakis (Apr 13 2020 at 14:23, on Zulip):

but ok, that makes sense

nikomatsakis (Apr 13 2020 at 14:23, on Zulip):

also, @matklad, seems ok, I'm still catching up on notifications anyway

nikomatsakis (Apr 13 2020 at 14:23, on Zulip):

always takes me more time than I leave for it

Florian Diebold (Apr 13 2020 at 14:33, on Zulip):

I've been continuing to test the recursive Chalk solver and implemented a bunch of fixes in RA, which I'm turning into PRs, hence the bunch of PRs right now :grimacing:

apiraino (Apr 13 2020 at 15:17, on Zulip):

Florian Diebold said:

there's a bunch of customization variables, ...

I'm always asking myself whether I should keep my LSP customizations in custom.el (therefore changing them with M-x customize-variable) or write them in my personal/mysettings.el. Currently I have custom settings a bit here and a bit there and it looks messy. Any suggestions?
This is more a generic emacs question ... anyway, thanks :-)

Florian Diebold (Apr 14 2020 at 09:14, on Zulip):

I don't use customize-variable and just put stuff into my user config function, but I use spacemacs so :shrug:

matklad (Apr 14 2020 at 09:41, on Zulip):

When I used to use emacs heavily (with prelude and vanilla config at different times) I also find M-x customize-variable to be more of a problem than a solution

matklad (Apr 20 2020 at 14:13, on Zulip):

Oups, almost missed weekly sync-up @WG-rls2.0 !

matklad (Apr 20 2020 at 14:14, on Zulip):

So, we have a humongous release today, https://rust-analyzer.github.io/thisweek/2020/04/20/changelog-21.html, thanks to work by @Edwin Cheng and @Florian Diebold over the past several weeks

matklad (Apr 20 2020 at 14:14, on Zulip):

release contains both new trait solver and procedural macros support.

Edwin Cheng (Apr 20 2020 at 14:14, on Zulip):

Only support Custom-derive for now :)

std::Veetaha (Apr 20 2020 at 14:15, on Zulip):

@Edwin Cheng how do we handle incorrect syntax in proc macros?

matklad (Apr 20 2020 at 14:15, on Zulip):

I am extremely happy to see custom derive working!

Like, for the past four years or something I was thinking "one day, we'll expand proc macros in an IDE", and today is that day.

nikomatsakis (Apr 20 2020 at 14:16, on Zulip):

oooh

nikomatsakis (Apr 20 2020 at 14:17, on Zulip):

the recursive solver is shipping, eh? Very exciting

matklad (Apr 20 2020 at 14:18, on Zulip):

On my side, I feel like I've done comparatively little :)

I am still in the process of solving the accidentally linear lookup in the sytnax trees, and the item-tree IR work is still on hold :(

nikomatsakis (Apr 20 2020 at 14:18, on Zulip):

So, I've been saying this for weeks, but I mean it now

nikomatsakis (Apr 20 2020 at 14:18, on Zulip):

(lol)

matklad (Apr 20 2020 at 14:18, on Zulip):

the recursive solver is shipping, eh?

Yes! And the impact is very visible.

nikomatsakis (Apr 20 2020 at 14:18, on Zulip):

I want to advance this IDE RFC

nikomatsakis (Apr 20 2020 at 14:18, on Zulip):

last week I got pretty distracted

nikomatsakis (Apr 20 2020 at 14:19, on Zulip):

I was going to read it over today and .. probably open it up? I'm not sure if there's any further consulting that makes sense, can't think what it would be right now

nikomatsakis (Apr 20 2020 at 14:19, on Zulip):

matklad said:

Yes! And the impact is very visible.

Very cool.

nikomatsakis (Apr 20 2020 at 14:19, on Zulip):

There's been a lot of activity in/around chalk

matklad (Apr 20 2020 at 14:19, on Zulip):

Even if there are, we can do this after opening the RFC

nikomatsakis (Apr 20 2020 at 14:19, on Zulip):

Right

matklad (Apr 20 2020 at 14:20, on Zulip):

I also want to write a "rust-analyzer is relased" blog post this week, and publish it next week.

nikomatsakis (Apr 20 2020 at 14:20, on Zulip):

What language features are most problematic for RLS in terms of chalk etc at this juncture?

nikomatsakis (Apr 20 2020 at 14:20, on Zulip):

Maybe in/around async I/O?

matklad (Apr 20 2020 at 14:20, on Zulip):

Like, we never had a proper announcement, with a short run-down of features, quick start, etc. I think combining that with RFC would be beneficial.

nikomatsakis (Apr 20 2020 at 14:20, on Zulip):

Ah, yeah, good idea.

Florian Diebold (Apr 20 2020 at 14:21, on Zulip):

the recursive solver is shipping, eh?

Yes! And the impact is very visible.

also in terms of performance though :grimacing: a timely review of rust-lang/chalk#404 would be helpful there

matklad (Apr 20 2020 at 14:21, on Zulip):

What language features are most problematic for RLS in terms of chalk etc at this juncture?

@Florian Diebold knows more, but, as a user, I see unresolved types with dyn Trait and impl Trait syntaxes

nikomatsakis (Apr 20 2020 at 14:22, on Zulip):

eek, yeah, will get on that today!

nikomatsakis (Apr 20 2020 at 14:22, on Zulip):

I was just going to say @Florian Diebold that maybe you and I can spend some time today discussing the caching/salsa story more generally

matklad (Apr 20 2020 at 14:23, on Zulip):

Does it make sense for me to hop onto that discussion?

nikomatsakis (Apr 20 2020 at 14:23, on Zulip):

Sure!

nikomatsakis (Apr 20 2020 at 14:23, on Zulip):

Maybe later today?

Florian Diebold (Apr 20 2020 at 14:24, on Zulip):

matklad said:

Florian Diebold knows more, but, as a user, I see unresolved types with dyn Trait and impl Trait syntaxes

yeah, impl Trait support should be ready soon though, for dyn Trait the problem is super traits, and there's nothing ongoing there... apart from that, the ongoing work on built-in impls will have a big impact I think, and then the only thing that comes to mind is support for integer/float variables

matklad (Apr 20 2020 at 14:24, on Zulip):

I think we also want the ability to get a specific impl from chalk, and not just the yes/no answer

Florian Diebold (Apr 20 2020 at 14:24, on Zulip):

nikomatsakis said:

I was just going to say Florian Diebold that maybe you and I can spend some time today discussing the caching/salsa story more generally

yeah, my other PR (rust-lang/chalk#403) is kind of related to that btw

matklad (Apr 20 2020 at 14:25, on Zulip):

Like, I belive for goto definition for method we want to land on the impl, if possible, and not on the trait.

nikomatsakis (Apr 20 2020 at 14:25, on Zulip):

matklad said:

I think we also want the ability to get a specific impl from chalk, and not just the yes/no answer

can you say more about why? I've specifically avoided giving that info and don't intend to change that, but it can be done in other ways

nikomatsakis (Apr 20 2020 at 14:25, on Zulip):

ok, so, if the question you want to answer is more like "what is the precise version of this method that will run"

nikomatsakis (Apr 20 2020 at 14:26, on Zulip):

that is something we can/should add to chalk, but not that way, it'd be modeled I think more like an associated type normalization

nikomatsakis (Apr 20 2020 at 14:26, on Zulip):

Florian Diebold said:

yeah, my other PR (rust-lang/chalk#403) is kind of related to that btw

yep I saw that

matklad (Apr 20 2020 at 14:26, on Zulip):

TO be clear, this is not something urgently needed, but I'd love to see it eventually

nikomatsakis (Apr 20 2020 at 14:26, on Zulip):

well, I do think it'll be needed as rustc integration proceeds too

nikomatsakis (Apr 20 2020 at 14:26, on Zulip):

so it makes sense to work on

nikomatsakis (Apr 20 2020 at 14:26, on Zulip):

in any case it's sort of "technicality", the distinction I'm drawing

nikomatsakis (Apr 20 2020 at 14:27, on Zulip):

but in rustc when you ask a question like T: Foo you get back an answer that is a specific impl, and I'd like to avoid making that the interface. But we obviously need the ability to figure out "what version of this method will actually run"

nikomatsakis (Apr 20 2020 at 14:27, on Zulip):

the idea is to leverage the fact that every function definition written by a user has a unique type associated with it

Jeremy Kolb (Apr 20 2020 at 14:27, on Zulip):

Like, I belive for goto definition for method we want to land on the impl, if possible, and not on the trait.

Why not both :)

Florian Diebold (Apr 20 2020 at 14:28, on Zulip):

there should be a command to go to the trait as well, but if you go to the impl, you can still easily go up to the trait, but not the other way around

matklad (Apr 20 2020 at 14:28, on Zulip):

Yeah.... somethimes I even think about exposing some sort of "chain of evidence" in the UI, which explains how a particular type implements a trait.

Florian Diebold (Apr 20 2020 at 14:29, on Zulip):

yeah, that would be cool

nikomatsakis (Apr 20 2020 at 14:29, on Zulip):

I have so many thoughts about this

matklad (Apr 20 2020 at 14:29, on Zulip):

Like how IntelliJ does this somewat implicitelly, remmebering the current subsitution when doing goto defitinition (waves hands furiously)

nikomatsakis (Apr 20 2020 at 14:29, on Zulip):

well maybe that many :P

nikomatsakis (Apr 20 2020 at 14:29, on Zulip):

mostly I have ideas of things I would like to find better ways to help users see, navigate, and understand

nikomatsakis (Apr 20 2020 at 14:30, on Zulip):

but I'm not sure how hard that is to do in contexts like VSCode

nikomatsakis (Apr 20 2020 at 14:30, on Zulip):

certainly the "chain of evidence" is on there, in any case.

matklad (Apr 20 2020 at 14:30, on Zulip):

/me can't wait until the basics are in place, and we can experiement with fancy visualisations and stuff

Florian Diebold (Apr 20 2020 at 14:30, on Zulip):

matklad said:

Like how IntelliJ does this somewat implicitelly, remmebering the current subsitution when doing goto defitinition (waves hands furiously)

that reminds me that I would also love to do a similar thing for macros -- basically if you go to definition on a macro call, remember the parameters, so you could expand it further

matklad (Apr 20 2020 at 14:32, on Zulip):

@nikomatsakis are there any action items for me with respect to the RFC?

nikomatsakis (Apr 20 2020 at 14:40, on Zulip):

Florian Diebold said:

that reminds me that I would also love to do a similar thing for macros -- basically if you go to definition on a macro call, remember the parameters, so you could expand it further

oh man it would be so great to see the series of expansions and recursions this way...

matklad (Apr 27 2020 at 14:04, on Zulip):

:wave: @WG-rls2.0, it's that time of Monday again!

matklad (Apr 27 2020 at 14:05, on Zulip):

There's a ton of technical things which were done this week, but I am mostly excited about more general project life things

matklad (Apr 27 2020 at 14:05, on Zulip):

We did a real release: https://rust-analyzer.github.io/blog/2020/04/20/first-release.html (long overdue)

matklad (Apr 27 2020 at 14:06, on Zulip):

Also, the rust-analyzer RFC seems to be extremely well received (we'v collected more :thumbs_up: than even nll :D )

matklad (Apr 27 2020 at 14:07, on Zulip):

The usual release notes are here:

https://rust-analyzer.github.io/thisweek/2020/04/27/changelog-22.html

std::Veetaha (Apr 27 2020 at 14:08, on Zulip):

How will our life change after this alpha release?

matklad (Apr 27 2020 at 14:08, on Zulip):

I wouldn't expect any changes

Jeremy Kolb (Apr 27 2020 at 14:09, on Zulip):

More github issues most likely

matklad (Apr 27 2020 at 14:09, on Zulip):

Though, because of the RFC, I think we need to spend more time making sure that rust-analyzer "just works" in other editors as well.

matklad (Apr 27 2020 at 14:09, on Zulip):

(I've already started this week)

matklad (Apr 27 2020 at 14:10, on Zulip):

oh, yeah, @Jeremy Kolb is 100% right. TBH, I don't look at every issue myself anymore, a lot of them get closed by the point I get to them in my mailbox

matklad (Apr 27 2020 at 14:11, on Zulip):

For that, I actually want to deflect some part of the torrent of questions to urlo: https://internals.rust-lang.org/t/discussion-forum-for-dev-tools-rust-analyzer/12160/5

matklad (Apr 27 2020 at 14:12, on Zulip):

Another bigish issue from this week is that proc-macros are broken on musl, because it can't dlopen

Laurențiu Nicola (Apr 27 2020 at 14:12, on Zulip):

But how does rustc work on musl?

matklad (Apr 27 2020 at 14:12, on Zulip):

@Laurențiu Nicola is asking the right questions...

Jeremy Kolb (Apr 27 2020 at 14:13, on Zulip):

matklad said:

Though, because of the RFC, I think we need to spend more time making sure that rust-analyzer "just works" in other editors as well.

If these are due to LSP weirdness assign to me and I can take a look.

matklad (Apr 27 2020 at 14:13, on Zulip):

wait, I think it doesn't? Like, rustc links glibc dynamically I think?

Laurențiu Nicola (Apr 27 2020 at 14:13, on Zulip):

I think we should try shipping glibc binaries and see if anyone complains. But maybe it's not the right moment

matklad (Apr 27 2020 at 14:14, on Zulip):

I think doing that next week would be OK! It might probably make sense to switch the nightly asap even

matklad (Apr 27 2020 at 14:14, on Zulip):

There's also this annoying slow allocator problem...

matklad (Apr 27 2020 at 14:14, on Zulip):

@Jeremy Kolb yup, I think that would be mostly it

Laurențiu Nicola (Apr 27 2020 at 14:14, on Zulip):

I don't think it's slow on musl, but rather that the glibc one is pretty fast these days

matklad (Apr 27 2020 at 14:15, on Zulip):

Also, kudos to @Jeremy Kolb for being on top of many small things, like version updates and what not. They rarely get into release notes, which is really unforutnate :(

Laurențiu Nicola (Apr 27 2020 at 14:15, on Zulip):

matklad said:

wait, I think it doesn't? Like, rustc links glibc dynamically I think?

There's a rust package on Alpine and I don't think it pulls in glibc

std::Veetaha (Apr 27 2020 at 14:16, on Zulip):

@matklad will you be able to look at ast docs pr some time this week?

matklad (Apr 27 2020 at 14:17, on Zulip):

That's not impossible :)

Edwin Cheng (Apr 27 2020 at 14:18, on Zulip):

IIUC, rustc musl host use dynamic-musl and glibc .

matklad (Apr 27 2020 at 14:19, on Zulip):

@Jeremy Kolb yup, I think that would be mostly it

The annoying thing would be our custom code-action format. I am warming up to the idea of using the text snippets as internal representation, but that would require some extensive changes to out assist infra. Which are necessary anyway, as our cursor placement in assists is pretty stupid in majority of cases

Jeremy Kolb (Apr 27 2020 at 14:21, on Zulip):

Yeah that plays into server-side command execution. I saw your proposal for cursor position... I hope that gets in somehow

Laurențiu Nicola (Apr 27 2020 at 15:05, on Zulip):

I wonder if we could also enable jemalloc for the releases

matklad (May 04 2020 at 14:02, on Zulip):

Hey @WG-rls2.0 , it's time for a weekly sync up again!

matklad (May 04 2020 at 14:03, on Zulip):

It was a quiet week (in comparision to couple of previous ones) with mostly feature polishing and internal refactorings

matklad (May 04 2020 at 14:04, on Zulip):

We now have a tracking issue for rust-analyzer LSP RFC: https://github.com/rust-analyzer/rust-analyzer/issues/4224#issue-610020082

matklad (May 04 2020 at 14:05, on Zulip):

This week, I'd like to tackle our use of custom rust-analyzer.applySourceChange comand, which hurts interoperability

matklad (May 04 2020 at 14:06, on Zulip):

I also want to refactor our internal representation of code changes -- it seems like using snippets as the data structure is a better approach than our separate caret_offset: TextSize field (which was cargo-culted from IntelliJ)

nikomatsakis (May 04 2020 at 14:11, on Zulip):

matklad said:

We now have a tracking issue for rust-analyzer LSP RFC: https://github.com/rust-analyzer/rust-analyzer/issues/4224#issue-610020082

great

nikomatsakis (May 04 2020 at 14:11, on Zulip):

I have a question to raise

nikomatsakis (May 04 2020 at 14:11, on Zulip):

How are we tracking the issues and feedback that arise in terms of potential blockers around the RLS -> rust-analyzer transition

nikomatsakis (May 04 2020 at 14:12, on Zulip):

I think you created an issue for that?

matklad (May 04 2020 at 14:12, on Zulip):

Yes, there's currently one issue which is sort of grab-bag for everything

matklad (May 04 2020 at 14:13, on Zulip):

https://github.com/rust-analyzer/rust-analyzer/issues/4224

matklad (May 04 2020 at 14:13, on Zulip):

(the same issue)

matklad (May 04 2020 at 14:14, on Zulip):

We currently don't do any kind of organized tracking. Should probably do that, though I kind of want to tackle the technical issue of custom edit command first.

matklad (May 04 2020 at 14:14, on Zulip):

Oh, we also briefly chatted with @Igor Matuszewski about merging the VS Code extensions.

matklad (May 04 2020 at 14:15, on Zulip):

One think we are debated in the moment is where this unified VS Code extension should live -- at the moment its in the separate repo, but rust-analyzer one is in-tree

Igor Matuszewski (May 04 2020 at 14:17, on Zulip):

There’s also WIP to support RA in the RLS extension, which should not require much work to be in an acceptable state

matklad (May 04 2020 at 14:17, on Zulip):

@Igor Matuszewski that's sweet! Is there a link?

Igor Matuszewski (May 04 2020 at 14:18, on Zulip):

Nothing shippable nor uploaded upstream, just hacking away locally

matklad (May 04 2020 at 14:18, on Zulip):

Also, one thing I want to note that, after switching Chalk to recursive solver, I enjoy using rust-analyzer so much more. Like, it doesn't feel like hit-and-miss anymore, it's mostly hit, and now, when, eg, I don't get completions I expect, I start to look for bug in my code and not a bug in rust-analyzer :D

Igor Matuszewski (May 04 2020 at 14:18, on Zulip):

I’ll link whenever I have something working, which will be this week

Jeremy Kolb (May 04 2020 at 14:21, on Zulip):

@matklad does using a snippet representation get you around the need for a custom LSP request?

matklad (May 04 2020 at 14:21, on Zulip):

sort of

matklad (May 04 2020 at 14:22, on Zulip):

we'll be able to reuse "standard" API, but, because snippets are not supported in edits yet, we'd have to hack our custom handler for a standard command

matklad (May 04 2020 at 14:22, on Zulip):

(natutally, if client advertises the capability)

Jeremy Kolb (May 04 2020 at 14:37, on Zulip):

@matklad semi related: we still need to be able to change server caps based on the client

matklad (May 11 2020 at 14:18, on Zulip):

Hey, I won't be able to make this sync up, though the release is up :)

matklad (May 11 2020 at 14:38, on Zulip):

More generally, I am doing some client work this week, so I won't be fully available :-)

matklad (May 18 2020 at 14:14, on Zulip):

:wave: @WG-rls2.0 it's sync-up time!

matklad (May 18 2020 at 14:15, on Zulip):

Today's changelog is here: https://rust-analyzer.github.io/thisweek/2020/05/18/changelog-25.html

matklad (May 18 2020 at 14:15, on Zulip):

I've also blogged about my vision for longer-term plans here: https://rust-analyzer.github.io/blog/2020/05/18/next-few-years.html

simulacrum (May 18 2020 at 14:16, on Zulip):

years?!

matklad (May 18 2020 at 14:16, on Zulip):

Well, I don't usually make plans, but when I do....

matklad (May 18 2020 at 14:17, on Zulip):

Though, realistically, I feel that we'll need a few years to get to a steady state with respect to IDE support.

nikomatsakis (May 18 2020 at 14:18, on Zulip):

@matklad is the TextMate plugin maintained in the r-a repo?

matklad (May 18 2020 at 14:18, on Zulip):

I guess, the lower bound is "time to merge chalk" :)

matklad (May 18 2020 at 14:18, on Zulip):

@nikomatsakis that's not plugin, it's a "grammar" file in a format, which is called TextMate

nikomatsakis (May 18 2020 at 14:18, on Zulip):

(have we merged the RFC yet? I think I have to ping people again)

nikomatsakis (May 18 2020 at 14:18, on Zulip):

I see

nikomatsakis (May 18 2020 at 14:18, on Zulip):

/me pours one out for the TextMate editor

matklad (May 18 2020 at 14:19, on Zulip):

Nope, the RFC is in FCP RN I think

nikomatsakis (May 18 2020 at 14:19, on Zulip):

https://github.com/rust-analyzer/rust-analyzer/pull/4431 is interesting

nikomatsakis (May 18 2020 at 14:19, on Zulip):

the "don't garbage collect" in particular

nikomatsakis (May 18 2020 at 14:20, on Zulip):

maybe I don't understand what you mean by that

matklad (May 18 2020 at 14:20, on Zulip):

So, used to store results of proc macro expansion in LRU

matklad (May 18 2020 at 14:21, on Zulip):

This breaks if proc-macro is non-deterministic (breaks as in, we get panics in rust-analyzer afterwards)

nikomatsakis (May 18 2020 at 14:21, on Zulip):

I'm skimming blog post btw, lots of interesting things to chew on there

nikomatsakis (May 18 2020 at 14:21, on Zulip):

it reminds me that I want to put in more time trying to create a "prioritization document" for compiler team

matklad (May 18 2020 at 14:24, on Zulip):

In terms of RFC progress:

nikomatsakis (May 18 2020 at 14:24, on Zulip):

matklad said:

This breaks if proc-macro is non-deterministic (breaks as in, we get panics in rust-analyzer afterwards)

actually can you say a bit more about this?

nikomatsakis (May 18 2020 at 14:24, on Zulip):

that surprises me mildly

matklad (May 18 2020 at 14:25, on Zulip):

SO, let's say proc-macro produces struct Foo; enum Bar;

nikomatsakis (May 18 2020 at 14:25, on Zulip):

I think I would expect that, if the procedural macro result is dropped, then we would re-execute it, and if it comes up with a different answer this time, it would be marked "red"

nikomatsakis (May 18 2020 at 14:25, on Zulip):

oh, hmm, maybe I see it

matklad (May 18 2020 at 14:25, on Zulip):

We intern id for Foo which says that it's the first thing in a file

matklad (May 18 2020 at 14:25, on Zulip):

we than LRU the tree out

nikomatsakis (May 18 2020 at 14:25, on Zulip):

the asserts perhaps come from us concluding that, because all inputs have not changed, the output can't have changed

matklad (May 18 2020 at 14:26, on Zulip):

inputs have not changed

nikomatsakis (May 18 2020 at 14:26, on Zulip):

sorry, I meant to write have not

matklad (May 18 2020 at 14:26, on Zulip):

Basically, the changed_at and validated_at are up to date, it's just that the value is missing.

nikomatsakis (May 18 2020 at 14:26, on Zulip):

ok, so, if you have a graph like this

Input -> ProcMacro -> Derived

nikomatsakis (May 18 2020 at 14:26, on Zulip):

and Derived is cached

nikomatsakis (May 18 2020 at 14:26, on Zulip):

we might reuse that cached result

nikomatsakis (May 18 2020 at 14:26, on Zulip):

then demand ProcMacro and re-execute

nikomatsakis (May 18 2020 at 14:27, on Zulip):

but the two are now inconsistent

nikomatsakis (May 18 2020 at 14:27, on Zulip):

sure, that makes sense

nikomatsakis (May 18 2020 at 14:27, on Zulip):

right so we are effectively treating ProcMacro as "volatile

nikomatsakis (May 18 2020 at 14:27, on Zulip):

ie., the salsa way should be to treat it as having an untracked input

nikomatsakis (May 18 2020 at 14:28, on Zulip):

(this is precisely why we don't collect volatile things until a new round)

nikomatsakis (May 18 2020 at 14:28, on Zulip):

correct?

matklad (May 18 2020 at 14:28, on Zulip):

Not sure: untracked input would force us to re-execute it on every change, right?

matklad (May 18 2020 at 14:29, on Zulip):

Ie, it would be sound, but it would be inefficient, in a pretty horrible way, as proc macros are the things which we really want to cache agressively.

nikomatsakis (May 18 2020 at 14:29, on Zulip):

yes correct

nikomatsakis (May 18 2020 at 14:30, on Zulip):

I think it's the sound thing

nikomatsakis (May 18 2020 at 14:30, on Zulip):

an alternative would be

nikomatsakis (May 18 2020 at 14:30, on Zulip):

given durabilities and all

nikomatsakis (May 18 2020 at 14:30, on Zulip):

to have them take an input from some "pseudo-input" with high durability, meant to represent randomness

nikomatsakis (May 18 2020 at 14:31, on Zulip):

anyway ok I getthe problem, it's an intersting one, and I think there is a kind of contract here that we should be trying to specify

nikomatsakis (May 18 2020 at 14:31, on Zulip):

i.e., do we expect proc macros to be deterministic, and how much do we "tolerate" them when they are not?

nikomatsakis (May 18 2020 at 14:31, on Zulip):

the batch compiler cares too after all

matklad (May 18 2020 at 14:32, on Zulip):

Yeah... I feel like the answer is "non deterministic proc macro is a bug", but that really needs a WM to enforce

nikomatsakis (May 18 2020 at 14:32, on Zulip):

it seems like we're saying:

nikomatsakis (May 18 2020 at 14:32, on Zulip):

which isn't really a concept that salsa quite has, I think, durabilities don't quite capture it

matklad (May 18 2020 at 14:32, on Zulip):

Yeah. To be clear, I think our current scheme where we just don't LRU is also sound.

nikomatsakis (May 18 2020 at 14:33, on Zulip):

I think it's sound. The question would be "what should you do if you did LRU"

matklad (May 18 2020 at 14:33, on Zulip):

(as a reminder, we don't run any other form of GC besides LRU, which seems super wrong, but also seems to work OK?)

nikomatsakis (May 18 2020 at 14:33, on Zulip):

and the answer "assume result will change"

nikomatsakis (May 18 2020 at 14:33, on Zulip):

( i.e., you have to re-execute and compare a hash or whatever, not that we have hashes, in order to re-use later results )

matklad (May 18 2020 at 14:34, on Zulip):

Hm, not sure

matklad (May 18 2020 at 14:34, on Zulip):

I would say you'd have to somehow proactively tear down depndent queries?

nikomatsakis (May 18 2020 at 14:34, on Zulip):

the alternative seems to be ICEing

Florian Diebold (May 18 2020 at 14:35, on Zulip):

should there be an RFC or at least some documentation in the reference that explicitly says that proc macros should be deterministic (even if we can't enforce it)? also relatedly, it's not defined anywhere whether proc macros are allowed to e.g. read files or the network, or is it?

nikomatsakis (May 18 2020 at 14:35, on Zulip):

I'm not saying it would have to be proactive

matklad (May 18 2020 at 14:35, on Zulip):

I guess, what I am trying to say is that we just can't LRU proc macros.

nikomatsakis (May 18 2020 at 14:35, on Zulip):

I don't think we're disagreeing in particular

nikomatsakis (May 18 2020 at 14:35, on Zulip):

I do feel an RFC is appropriate

nikomatsakis (May 18 2020 at 14:35, on Zulip):

I think what I'm saying is:

nikomatsakis (May 18 2020 at 14:36, on Zulip):

/me thinks how to phrase it

nikomatsakis (May 18 2020 at 14:37, on Zulip):

so we have a few options

nikomatsakis (May 18 2020 at 14:37, on Zulip):

we can say that

nikomatsakis (May 18 2020 at 14:37, on Zulip):

if that were really true, then you could LRU

nikomatsakis (May 18 2020 at 14:37, on Zulip):

but we know it's not true in practice, for a variety of reasons

nikomatsakis (May 18 2020 at 14:38, on Zulip):

we can still say it, but it seems like we're not willing to ICE and crash as a result of a buggy proc macro

nikomatsakis (May 18 2020 at 14:38, on Zulip):

so what we can say instead is:

This means we reserve the right not to re-run them, essentially, and to re-use the cached output.

nikomatsakis (May 18 2020 at 14:38, on Zulip):

However, if we don't have cached output (or we throw it away, or whatever), then we can't really assume they would produce precisely the same output the next time.

nikomatsakis (May 18 2020 at 14:39, on Zulip):

This is, if nothing else, a matter of empirical observation.

nikomatsakis (May 18 2020 at 14:40, on Zulip):

We might still say that it's wrong even if we tolerate it, but I guess that people will continue to write buggy proc macros that we have to cope with unless we die a horrible death when they do.

matklad (May 18 2020 at 14:40, on Zulip):

Yup, I agree with this characterisation.

matklad (May 18 2020 at 14:41, on Zulip):

And yeah, "dying a horrible death" is not really an option for IDE unfortunately

nikomatsakis (May 18 2020 at 14:41, on Zulip):

I do think this should be in an RFC, I think it'd be great to summarize the experiences, and perhaps also note some of the implications for use cases like network access :)

nikomatsakis (May 18 2020 at 14:41, on Zulip):

e.g., a procedural macro that does that sort of thing probably needs some way to "explicitly" change an input if they want to force a refresh

nikomatsakis (May 18 2020 at 14:42, on Zulip):

In particular, the batch compiler

nikomatsakis (May 18 2020 at 14:42, on Zulip):

ought to be able to re-use proc macro results without re-executing as well

nikomatsakis (May 18 2020 at 14:42, on Zulip):

and since it persists to disk, that could last a lot longer

matklad (May 18 2020 at 14:42, on Zulip):

I am not sure this needs to be the RLS/rust-analyzer RFC though....

nikomatsakis (May 18 2020 at 14:42, on Zulip):

I .. see it as a compiler team RFC?

matklad (May 18 2020 at 14:43, on Zulip):

yup. I believe @Jeremy Fitzhardinge was also interested in the topic of deterministic proc macros.

nikomatsakis (May 18 2020 at 14:43, on Zulip):

I guess I see rust-analyzer as becoming a compiler team project, and hence this sort of decision applies to the "unified architecture" we are ultimately envisioning

nikomatsakis (May 18 2020 at 14:43, on Zulip):

Yes, it obviously applies to questions of distribution as well

matklad (May 18 2020 at 14:44, on Zulip):

What do you mean by "distribution"?

nikomatsakis (May 18 2020 at 14:45, on Zulip):

Sorry, I meant distributing builds across a cluster or something

nikomatsakis (May 18 2020 at 14:45, on Zulip):

parallelizing

nikomatsakis (May 18 2020 at 14:45, on Zulip):

I think that's @Jeremy Fitzhardinge's "angle" here

nikomatsakis (May 18 2020 at 14:46, on Zulip):

it's worth pointing out actually that the rule we just gave above

nikomatsakis (May 18 2020 at 14:46, on Zulip):

is probably already embodied in cargo

matklad (May 18 2020 at 14:46, on Zulip):

Ahh, yeah, cloud build systems.

nikomatsakis (May 18 2020 at 14:46, on Zulip):

i.e., cargo won't rebuild your crate

nikomatsakis (May 18 2020 at 14:46, on Zulip):

unless it seems some diffs

simulacrum (May 18 2020 at 14:46, on Zulip):

proc macros do not have anyway to output dependencies to cargo today, yes

nikomatsakis (May 18 2020 at 14:47, on Zulip):

I remember having some problems with this actualy because I was abusing proc macros to generate tests by scraping files from the file system :)

matklad (May 18 2020 at 14:47, on Zulip):

(was reading the extended build systems à la carte on the weekend, it is still good read on salsa-like issues)

nikomatsakis (May 18 2020 at 14:47, on Zulip):

I haven't read the extended version yet...

nikomatsakis (May 18 2020 at 14:47, on Zulip):

So who is going to write the RFC? :P

nikomatsakis (May 18 2020 at 14:47, on Zulip):

it'd be good to at least take some notes on this conversation somewhere

matklad (May 18 2020 at 14:49, on Zulip):

I guess, I can draft some notes and maybe collaborate with @Jeremy Fitzhardinge ?

matklad (May 18 2020 at 14:49, on Zulip):

I am not sure we can really write an RFC which can be implemented today, but "declaration of intent" RFC seems good

nikomatsakis (May 18 2020 at 14:50, on Zulip):

In my view the RFC is clarifying the expectations, it's not something that can be "implemented" in any one place (multiple tools etc may come to depend onit)

simulacrum (May 18 2020 at 14:53, on Zulip):

Hm I would probably feel better about such an rfc if it suggested adding proc macro apis to explicitly indicate dependencies

matklad (May 18 2020 at 14:54, on Zulip):

I believe @Christopher Durham has done some work on adding file dependencies to proc-macros?

Christopher Durham (May 18 2020 at 15:48, on Zulip):

If you count reminding people that I want an API for it as work, then yes

Christopher Durham (May 18 2020 at 15:52, on Zulip):

The "minimum" API for proc-macro external dependencies would be just to bolt on the buildscript cargo:rerun-if-changed annotations

matklad (May 25 2020 at 14:01, on Zulip):

:wave: @WG-rls2.0 !

matklad (May 25 2020 at 14:01, on Zulip):

This week's changelog: https://rust-analyzer.github.io/thisweek/2020/05/25/changelog-26.html

matklad (May 25 2020 at 14:04, on Zulip):

This week we've mostly worked on intrnal stuff:

matklad (May 25 2020 at 14:11, on Zulip):

Oh, the FCP for the RFC has ended, so @nikomatsakis , do want to hit the merge button?

https://github.com/rust-lang/rfcs/pull/2912

matklad (May 25 2020 at 14:12, on Zulip):

There was an additional concern about singling out VS Code as the primary implementation, and an additional technical question about the location of VS Code plugin.

matklad (May 25 2020 at 14:13, on Zulip):

For location, I think moving rust-analyzer vs code plugin to the existing rls-vscode repo makes sense, at least until we deprecate RLS.

std::Veetaha (May 25 2020 at 14:13, on Zulip):

You haven't mentioned "toggle inlay hints" and inlayHinst in lsp extensions

matklad (May 25 2020 at 14:14, on Zulip):

@std::Veetaha I belive they are for the next release?

std::Veetaha (May 25 2020 at 14:14, on Zulip):

image.png ?

matklad (May 25 2020 at 14:14, on Zulip):

We now publish release from last night's nightly, so PRs merged today will be released next week

matklad (May 25 2020 at 14:15, on Zulip):

(and the tag is just wrong :D )

matklad (May 25 2020 at 14:15, on Zulip):

wait, no, if the tag is wrong, that means that changelogs are wrong :fear: ?

matklad (May 25 2020 at 14:16, on Zulip):

https://github.com/rust-analyzer/rust-analyzer/issues/4319

matklad (May 25 2020 at 14:16, on Zulip):

Should really look into fixing that....

Jeremy Kolb (May 25 2020 at 16:42, on Zulip):

As part of the "two step" LSP initialization I also snuck in server info so the server name and version are passed to the client. We should be able to use that for bug tracking

Jeremy Kolb (May 25 2020 at 16:43, on Zulip):

We also fall back properly on code actions if using an older client (not sure if that fixed anything in the wild but if someone's code actions mysteriously start showing up now....)

std::Veetaha (May 25 2020 at 17:18, on Zulip):

We should be able to use that for bug tracking

Don't we already have "Rust Analyzer: server version" command for that?

Jeremy Kolb (May 26 2020 at 12:36, on Zulip):

Definitely. But if someone dumps logs of LSP requests or something...

matklad (Jun 01 2020 at 14:10, on Zulip):

:wave: @WG-rls2.0 !

matklad (Jun 01 2020 at 14:10, on Zulip):

We have a new release: https://rust-analyzer.github.io/thisweek/2020/06/01/changelog-27.html

matklad (Jun 01 2020 at 14:11, on Zulip):

I can name two highlights of the week (but maybe I am missing some)

matklad (Jun 01 2020 at 14:11, on Zulip):
matklad (Jun 01 2020 at 14:12, on Zulip):
matklad (Jun 01 2020 at 14:13, on Zulip):

The RFC is now merged, so I think we should now tackle the next steps of merging RLS and rust-analyzer:

matklad (Jun 01 2020 at 14:14, on Zulip):

I think we can do 1. right now (I guess I need to ask @Pietro Albini what are specific steps here).

For 2., I'd love to clean up our inlay hints and runnables LSP extensions first, and then we should be ready to go

matklad (Jun 01 2020 at 14:15, on Zulip):

I also would like to opimize the meta process around rust-analyzer, which sounds hard:

matklad (Jun 01 2020 at 14:15, on Zulip):
matklad (Jun 01 2020 at 14:16, on Zulip):
nikomatsakis (Jun 01 2020 at 14:17, on Zulip):

hmm

nikomatsakis (Jun 01 2020 at 14:17, on Zulip):

(sorry, was reading manuals)

matklad (Jun 01 2020 at 14:17, on Zulip):

In general, i am interested in hearing how we can more effectively channel a lot of work people are willing to do on rust-analyzer :D

nikomatsakis (Jun 01 2020 at 14:17, on Zulip):

those are indeed hard problems :)

nikomatsakis (Jun 01 2020 at 14:18, on Zulip):

I feel like the "trick" with reviewer bandwidth is to have more reviewers

nikomatsakis (Jun 01 2020 at 14:18, on Zulip):

which... as you can tell by a glance at rustc's list of pending PRs... is sometimes not so easy

matklad (Jun 01 2020 at 14:19, on Zulip):

Yeah, and the trick to "more reviewers" is sharing "institutional knowledge" :)

nikomatsakis (Jun 01 2020 at 14:19, on Zulip):

lol, yes,

nikomatsakis (Jun 01 2020 at 14:19, on Zulip):

I feel like efforts like the rustc-dev-guide (and chalk book) are maybe helpful here but I'm not 100% sure how much

nikomatsakis (Jun 01 2020 at 14:20, on Zulip):

this specific example -- showing some kind of "structure" between models that is latent in the code -- is an interesting one that has sort of prodded me to want to use more crates than I would otherwise have used

nikomatsakis (Jun 01 2020 at 14:20, on Zulip):

(we're debating exactly this question around chalk)

matklad (Jun 01 2020 at 14:20, on Zulip):

I personally don't worry a lot about "code quality" aspect of review -- as long as code has tests, it should be fine to refactor it in subsequent PRs (but I am willing to change my optinion here, if people think otherwise).

What I worry more is drive-by "architecture" changes, which expand API of certain things, in ways which are not always appropriate. But figuring out what's appropriate is kind of hard...

nikomatsakis (Jun 01 2020 at 14:21, on Zulip):

/me nods

nikomatsakis (Jun 01 2020 at 14:22, on Zulip):

to what extent do you think the arch. can be written down or documented,

nikomatsakis (Jun 01 2020 at 14:22, on Zulip):

vs you not really knowing what those constraints are until you see them violated :)

matklad (Jun 01 2020 at 14:22, on Zulip):

We already have it documented, in architecture.md document

matklad (Jun 01 2020 at 14:22, on Zulip):

The problem is, I haven't updated it in a year or so...

matklad (Jun 01 2020 at 14:23, on Zulip):

My immediate action items are:

matklad (Jun 01 2020 at 14:23, on Zulip):

I am also entertaining the idea of generating architecture docs from code, the same way we now generate feature documentation

nikomatsakis (Jun 01 2020 at 14:24, on Zulip):

interesting

Laurențiu Nicola (Jun 01 2020 at 14:26, on Zulip):

(deleted)

matklad (Jun 01 2020 at 14:26, on Zulip):

Also, orthogonally, I again was thinking about how I'd love salsa to be more like ECS (with entitities in flat vectors), and didn't come to any conclusions about what exactly do I want

nikomatsakis (Jun 01 2020 at 14:27, on Zulip):

I've been thinking about salsa too, not sure if you saw some of my notes

matklad (Jun 01 2020 at 14:27, on Zulip):

I think not

nikomatsakis (Jun 01 2020 at 14:27, on Zulip):

I want to do a few things

matklad (Jun 01 2020 at 14:27, on Zulip):

/me listens

nikomatsakis (Jun 01 2020 at 14:27, on Zulip):

one of them is to simplify the thread handling so that by default we assume queries are idempotent

nikomatsakis (Jun 01 2020 at 14:28, on Zulip):

meaning that if two threads start executing same query at same time it just runs twice

nikomatsakis (Jun 01 2020 at 14:28, on Zulip):

with a side layer for declaring "synchronized" queries where that is inappropriate -- I think that rust-analyzer would need this around procedural macros

nikomatsakis (Jun 01 2020 at 14:28, on Zulip):

(does rust-analyzer actually run procedural macros? I remember us talking about it at some point)

matklad (Jun 01 2020 at 14:28, on Zulip):

Yup, we now run procedural macros!

matklad (Jun 01 2020 at 14:28, on Zulip):

(derive only)

nikomatsakis (Jun 01 2020 at 14:28, on Zulip):

I thought so

nikomatsakis (Jun 01 2020 at 14:29, on Zulip):

anyway that is partly just to simplify the code but also

matklad (Jun 01 2020 at 14:29, on Zulip):

need this around procedural macros

And also around the main crate_def_map query

nikomatsakis (Jun 01 2020 at 14:29, on Zulip):

the more important change would be added support for "fixed-point queries" that can accommodate cycles

nikomatsakis (Jun 01 2020 at 14:29, on Zulip):

which would allow us to move some of that logic out from chalk and into salsa

nikomatsakis (Jun 01 2020 at 14:29, on Zulip):

and start to cache chalk queries

nikomatsakis (Jun 01 2020 at 14:29, on Zulip):

I think it's "quite doable"

nikomatsakis (Jun 01 2020 at 14:30, on Zulip):

matklad said:

And also around the main crate_def_map query

say a bit more? what is this query?

matklad (Jun 01 2020 at 14:30, on Zulip):

This query computes the set of names visible inside each module

matklad (Jun 01 2020 at 14:30, on Zulip):

So, import resolution and macro expansion

matklad (Jun 01 2020 at 14:30, on Zulip):

It is clear, at this point, that this is basically the single thing that matter to make IDE work well

matklad (Jun 01 2020 at 14:31, on Zulip):

Like, everything else you could do somehow, and it would work, because you can always do bite-sized chunks of work

nikomatsakis (Jun 01 2020 at 14:31, on Zulip):

why would it require sychronization

nikomatsakis (Jun 01 2020 at 14:31, on Zulip):

because it's quite expensive?

matklad (Jun 01 2020 at 14:31, on Zulip):

Yup

matklad (Jun 01 2020 at 14:31, on Zulip):

I feel it would be just expensive CPU time.

nikomatsakis (Jun 01 2020 at 14:31, on Zulip):

yeah, those are basically the two reasons to want to synchronize: expensive and/or not truly deterministic

matklad (Jun 01 2020 at 14:32, on Zulip):

(I need to profile things, but my gut feeling is that the most expensive bit today is macro-by-example expansion, as that manipulates heavy data structures)

nikomatsakis (Jun 01 2020 at 14:32, on Zulip):

and it's not very incrementalizable, right?

matklad (Jun 01 2020 at 14:33, on Zulip):

Hm, no, each macro expansion incrementalizes well-enough

nikomatsakis (Jun 01 2020 at 14:33, on Zulip):

I wonder if it would make sense to talk through how it works again

nikomatsakis (Jun 01 2020 at 14:33, on Zulip):

no, sorry, I meant the query as a whole

matklad (Jun 01 2020 at 14:33, on Zulip):

I am thinking more about from-scrach perf, and the amount of stuff we need to store in memory

matklad (Jun 01 2020 at 14:34, on Zulip):

The query as a whole is incrementalizable well enough I would think

nikomatsakis (Jun 01 2020 at 14:34, on Zulip):

matklad said:

Like, everything else you could do somehow, and it would work, because you can always do bite-sized chunks of work

maybe I didn't understand this sentence

matklad (Jun 01 2020 at 14:34, on Zulip):

Everything else works on either per-file or per-function basis

matklad (Jun 01 2020 at 14:35, on Zulip):

And is embarassigbly parallel and easy to make lazy

nikomatsakis (Jun 01 2020 at 14:35, on Zulip):

right so that suggests that it's not as incrementalizable as you might like

nikomatsakis (Jun 01 2020 at 14:35, on Zulip):

maybe that is not the right word

matklad (Jun 01 2020 at 14:35, on Zulip):

For resolving imports and expanding macros, I feel we just have to do it fast

nikomatsakis (Jun 01 2020 at 14:35, on Zulip):

what I meant is, if you need to resolve "one name" or something, do you ultimately have to do the entire crate etc

matklad (Jun 01 2020 at 14:36, on Zulip):

I guess, the problem is "the input is big"

matklad (Jun 01 2020 at 14:37, on Zulip):

I don't think that this is an insurmontable problem -- processing the whole crate could be made fast enough, but it requires some careful coding

matklad (Jun 01 2020 at 14:37, on Zulip):

For example, we need to do something non-trivial to correctly parallelize the thing

nikomatsakis (Jun 01 2020 at 14:37, on Zulip):

that reminds me that

nikomatsakis (Jun 01 2020 at 14:38, on Zulip):

if we did the things I wanted to do above re: chalk and salsa integration

matklad (Jun 01 2020 at 14:38, on Zulip):

ideally, we should process all crates in parallel (which is constrained by DAG of deps), but we should also start parallely parsing each crate before fully resolving all deps

nikomatsakis (Jun 01 2020 at 14:38, on Zulip):

I think it would be parallelized as well and not require any state that salsa doesn't manage

nikomatsakis (Jun 01 2020 at 14:39, on Zulip):

(and salsa would be much less heavyweight in terms of how it manages threads in the common case, since it would basically just be a map that you first read and only write at the end)

matklad (Jun 01 2020 at 14:39, on Zulip):

More generally, I think the next big thing to tackle in rust-analyzer experiment is to make sure that we can saturate at least 4 cores :)

nikomatsakis (Jun 01 2020 at 14:39, on Zulip):

not particularly related to name resolution, although it'd be interesting to think whether the model can be stretched in some way to accommodate name resolution, since that is fundamentally also a fixed point process

nikomatsakis (Jun 01 2020 at 14:40, on Zulip):

I don't think (right now) you are re-using anything in between name resolutions apart from the expansions of macros, is that right?

matklad (Jun 01 2020 at 14:40, on Zulip):

Not really

matklad (Jun 01 2020 at 14:40, on Zulip):

There's an IR between syntax and name res

matklad (Jun 01 2020 at 14:40, on Zulip):

(the set of defs in the current file)

matklad (Jun 01 2020 at 14:40, on Zulip):

We re-use this IR

matklad (Jun 01 2020 at 14:41, on Zulip):

(And I have been itching to rewrite this IR for like three months already)

Coenen Benjamin (Jun 01 2020 at 15:59, on Zulip):

matklad said:

Yeah, and the trick to "more reviewers" is sharing "institutional knowledge" :)

Yes I try when I have free time to review some PRs to help. But I'm not always confident and don't want to make a bad review. So this kind of documentations could be very useful :)

Florian Diebold (Jun 01 2020 at 16:46, on Zulip):

matklad said:

Chalk is moving so fast at the moment that I'm not even keeping up with integrating all the new features -- we still have to integrate the integer/float variables support, and array types for example (and opaque types are in progress, of course)

matklad (Jun 01 2020 at 16:47, on Zulip):

Lol yes, everything moves so fast now :)

Jack Huey (Jun 01 2020 at 17:17, on Zulip):

Btw the integer/float variable support is great. (From my experience updating the rustc integration)

Jeremy Kolb (Jun 01 2020 at 17:20, on Zulip):

My bandwidth has certainly reduced

Jeremy Kolb (Jun 01 2020 at 17:20, on Zulip):

Maybe it will be the year of Rust IDE

vsrs (Jun 02 2020 at 12:23, on Zulip):

matklad said:

For 2., I'd love to clean up our inlay hints and runnables LSP extensions first, and then we should be ready to go

I'm a bit concerned with the current runnables status and usage. My gut feeling is that it is a cool feature currently used wrong.

At first glance, it fits great for running tests, but it is not (IMHO). We need not only to run a test but also to gather results and show them in the UI or start a debugger. With runnables there are more problems than profit, especially with debug sessions.

I think this concept is better suited for, for example, cargo asm or cargo expand integration. I.e. for any shell command that needs a context-specific function\module name.

Yes, this includes cargo test.... But it should not be the main way to run\debug tests.

Jeremy Kolb (Jun 02 2020 at 12:26, on Zulip):

@vsrs I think there's some truth to that. It would be nice if we could provide some sort of standard "debugger engine hooks" or conform to some standard "test adapter api" for those types of things.

matklad (Jun 02 2020 at 12:35, on Zulip):

I'd say "running" runnables should be left to the client

matklad (Jun 02 2020 at 12:35, on Zulip):

Which could draw test results in a smart way, or start a debug session or what not

matklad (Jun 08 2020 at 14:04, on Zulip):

Hello @WG-rls2.0 , it's time for another syncup

matklad (Jun 08 2020 at 14:05, on Zulip):

This week's changelog: https://rust-analyzer.github.io/thisweek/2020/06/08/changelog-28.html

matklad (Jun 08 2020 at 14:06, on Zulip):

There's a bunch of stuff in this release, my personal favorite one is support for -> impl Trait :)

matklad (Jun 08 2020 at 14:07, on Zulip):

Proceduraly, I did some work on documenting more of the review and contributing process, in hope to make the review process, ideally, embarrassingly parallel.

matklad (Jun 08 2020 at 14:08, on Zulip):

We also have an in-progress PR which adds rust-analyzer to rust-lang/rust project: https://github.com/rust-lang/rust/pull/72978#issuecomment-640163536

matklad (Jun 08 2020 at 14:08, on Zulip):

It fails with a completely bewildering issue, so if anyone knows whats going on, please speak up :)

nikomatsakis (Jun 08 2020 at 14:08, on Zulip):

This is so that we do builds/releases with rust-lang/rust infra or what?

matklad (Jun 08 2020 at 14:09, on Zulip):

Yep, and shipping with rustup as an end-goal

matklad (Jun 08 2020 at 14:09, on Zulip):

We also switched to using chalk from crates.io, and I think that maybe it makes sense to talk though our long-term plan here?

matklad (Jun 08 2020 at 14:10, on Zulip):

Ie, to have the dreaded "monorepo" meeting, instead of postponing it :)

nikomatsakis (Jun 08 2020 at 14:10, on Zulip):

lol yes perhaps

nikomatsakis (Jun 08 2020 at 14:10, on Zulip):

it's probably not "the meeting"

nikomatsakis (Jun 08 2020 at 14:11, on Zulip):

at minimum, the time slot we normally use is not one that @Vadim Petrochenkov is able to attend afaik

nikomatsakis (Jun 08 2020 at 14:11, on Zulip):

and I know that he's got strong opinions

nikomatsakis (Jun 08 2020 at 14:11, on Zulip):

and would probably like to take part in the discussion

matklad (Jun 08 2020 at 14:11, on Zulip):

I personally am focusing on non-cargo build systems and VFS right now -- it's about time we get these foundational bits right :)

nikomatsakis (Jun 08 2020 at 14:11, on Zulip):

that said, I personally think we can kick this can down the road a bit further

matklad (Jun 08 2020 at 14:12, on Zulip):

that said, I personally think we can kick this can down the road a bit further

Yeah, I think I agree :)

nikomatsakis (Jun 08 2020 at 14:12, on Zulip):

i.e., we're still gathering data on the new clippy submodule integration etc, though I think that seems to be working out fairly well

nikomatsakis (Jun 08 2020 at 14:12, on Zulip):

and I think that offers a tantalizing middle ground

nikomatsakis (Jun 08 2020 at 14:12, on Zulip):

plus the "ship a chalk every week" infra is quite new etc

nikomatsakis (Jun 08 2020 at 14:13, on Zulip):

matklad said:

I personally am focusing on non-cargo build systems and VFS right now -- it's about time we get these foundational bits right :)

interesting

nikomatsakis (Jun 08 2020 at 14:13, on Zulip):

say a bit more?

Laurențiu Nicola (Jun 08 2020 at 14:13, on Zulip):

(I think local imports and having to restart the server after project structure changes are the issues with most duplicates on the tracker.)

matklad (Jun 08 2020 at 14:13, on Zulip):

For non-cargo use-cases -- mostly documenting and ironing out bugs in our rust-project.json thing:

https://rust-analyzer.github.io/manual.html#non-cargo-based-projects

matklad (Jun 08 2020 at 14:14, on Zulip):

For VFS -- figuring out the right abstraction in terms of salsa queries and stored data, so that we can easily plug watchman and solve painful problems like dynamic changes to the set of crates.

matklad (Jun 08 2020 at 14:15, on Zulip):

@nikomatsakis there's actually something I want to chat about with respect to VFS and salsa

matklad (Jun 08 2020 at 14:16, on Zulip):

Question:

Suppose user creates a new file in src directory, foo.rs. How do we figure out that we don't have to recheck every mod m; declaration form a standard library?

matklad (Jun 08 2020 at 14:16, on Zulip):

(in theory, new file in user's src directory could affect module structure of creates from std or crates.io)

Florian Diebold (Jun 08 2020 at 14:18, on Zulip):

(I think local imports and having to restart the server after project structure changes are the issues with most duplicates on the tracker.)

yeah, it feels like we get one of those almost every day :grimacing: makes me almost want to look into local imports, but there's still some type system stuff I want to focus on

nikomatsakis (Jun 08 2020 at 14:18, on Zulip):

matklad said:

(in theory, new file in user's src directory could affect module structure of creates from std or crates.io)

how?

matklad (Jun 08 2020 at 14:19, on Zulip):

it could have had a #[path] attribute, for example

nikomatsakis (Jun 08 2020 at 14:19, on Zulip):

(maybe we should fork that out into its own topic but I feel like I'm a bit confused by the question so far) :)

nikomatsakis (Jun 08 2020 at 14:19, on Zulip):

well, having a path attribute would not alter the structure of the a crate in std..?

nikomatsakis (Jun 08 2020 at 14:19, on Zulip):

it might read a file from a crate in std

matklad (Jun 08 2020 at 14:19, on Zulip):

The module might go from unresolved to resolved state

nikomatsakis (Jun 08 2020 at 14:20, on Zulip):

I'm not sure what that means

matklad (Jun 08 2020 at 14:20, on Zulip):

yeah, let me rephrase this

matklad (Jun 08 2020 at 14:20, on Zulip):

So, when we create a new file, we need to recalculate some module declarations

matklad (Jun 08 2020 at 14:20, on Zulip):

Because this might be a file for a module which was unresolved before

nikomatsakis (Jun 08 2020 at 14:21, on Zulip):

OK, now I'm starting to see. You are saying:

matklad (Jun 08 2020 at 14:21, on Zulip):

How do we limit the set of module declrations we need to check?

nikomatsakis (Jun 08 2020 at 14:21, on Zulip):

Imagine that libstd had a #[path]

nikomatsakis (Jun 08 2020 at 14:21, on Zulip):

that somehow reached into the user's source directory

nikomatsakis (Jun 08 2020 at 14:21, on Zulip):

and they created a new file

nikomatsakis (Jun 08 2020 at 14:21, on Zulip):

obviously this is not the case

nikomatsakis (Jun 08 2020 at 14:21, on Zulip):

but how do we know this based on salsa?

matklad (Jun 08 2020 at 14:22, on Zulip):

Exactly!

nikomatsakis (Jun 08 2020 at 14:22, on Zulip):

Interesting question. I can imagine a few ways go about it, but it also reminds me of the question of "lazy inputs"

matklad (Jun 08 2020 at 14:22, on Zulip):

My current thinking (and current impl) is that we construct explict "sets of files" outside of salsa, and that each crates has a single "set of files"

matklad (Jun 08 2020 at 14:23, on Zulip):

Ie, we upfront sepcify the super-set of files which can comprise the crate

matklad (Jun 08 2020 at 14:23, on Zulip):

And yes, this is inherently in tension with the desire to be lazy

matklad (Jun 08 2020 at 14:24, on Zulip):

Ie, we upfront sepcify the super-set of files which can comprise the crate

And this is very much not how rustc works currently

nikomatsakis (Jun 08 2020 at 14:25, on Zulip):

right so ..

nikomatsakis (Jun 08 2020 at 14:25, on Zulip):

I guess that the not-lazy version I was imagining is basically the same or similar

nikomatsakis (Jun 08 2020 at 14:25, on Zulip):

that you might have an "input" which is a list of filenames per directory for example

nikomatsakis (Jun 08 2020 at 14:25, on Zulip):

and we can monitor what is referenced

nikomatsakis (Jun 08 2020 at 14:25, on Zulip):

but it seems like what we really would prefer is that

nikomatsakis (Jun 08 2020 at 14:26, on Zulip):

we have a "lazy" input where we ask if a file exists and read it only then, and then remember

nikomatsakis (Jun 08 2020 at 14:26, on Zulip):

and then when modifications occur, we check against the files that have been read thus far

nikomatsakis (Jun 08 2020 at 14:26, on Zulip):

I sort of remember hacking up something like this for lark

nikomatsakis (Jun 08 2020 at 14:26, on Zulip):

but I think it's a pattern that salsa probably ought to support natively

nikomatsakis (Jun 08 2020 at 14:26, on Zulip):

I think I hacked it up with a "dummy" input

nikomatsakis (Jun 08 2020 at 14:27, on Zulip):

i.e., we had a side cache that you could read from and when you did we registered a read against some dummy input that was always (), I forget.

matklad (Jun 08 2020 at 14:27, on Zulip):

I am not exactly sure that we need this. The problem with purely lazy model is that you can't diagnose the problem that user crated foo.rs, but forgot about mod foo;, becuse you just don't see foo.rs

nikomatsakis (Jun 08 2020 at 14:27, on Zulip):

but it seems easy enough for salsa to support this with some form of callback

nikomatsakis (Jun 08 2020 at 14:27, on Zulip):

matklad said:

I am not exactly sure that we need this. The problem with purely lazy model is that you can't diagnose the problem that user crated foo.rs, but forgot about mod foo;, becuse you just don't see foo.rs

indeed. this seems sort of inherent, no? but in any case I would imagine a kind of hybrid

nikomatsakis (Jun 08 2020 at 14:28, on Zulip):

where you can set value but you can also wait until they are demanded?

nikomatsakis (Jun 08 2020 at 14:28, on Zulip):

I guess a good question woul be to step back and ask what constraints we're trying to solve exactly

nikomatsakis (Jun 08 2020 at 14:28, on Zulip):

you mentioned one question, I'm not sure whether you also wanted to avoid scanning directories aggressively

matklad (Jun 08 2020 at 14:29, on Zulip):

Scanning I think should be OK

matklad (Jun 08 2020 at 14:29, on Zulip):

The problem is that sometimes you might not know what to scan.

matklad (Jun 08 2020 at 14:29, on Zulip):

Ie, if the project includes #[path = "/some/completely/unrelated/dir"] mod foo;

nikomatsakis (Jun 08 2020 at 14:30, on Zulip):

yeah so that maybe fits this hybrid model

nikomatsakis (Jun 08 2020 at 14:30, on Zulip):

though it makes the question of "Give me the keys" a bit trickier

matklad (Jun 08 2020 at 14:31, on Zulip):

I also was thinking about basically hacking this around....

When we try to read something which is ourside of the current set of dirs, we get back a None, but we also arrange to call .set on the next iteration of the event loop.

nikomatsakis (Jun 08 2020 at 14:33, on Zulip):

That's interesting

nikomatsakis (Jun 08 2020 at 14:33, on Zulip):

I guess there is also a legit question of

nikomatsakis (Jun 08 2020 at 14:33, on Zulip):

what range of things we want a path to be able to access anyway

nikomatsakis (Jun 08 2020 at 14:34, on Zulip):

I wonder if rustc imposes any limitations

nikomatsakis (Jun 08 2020 at 14:34, on Zulip):

I feel like I'd prefer that sources can't read files from random locations when building myself :)

nikomatsakis (Jun 08 2020 at 14:34, on Zulip):

I'm thinking what a lazy input really means

nikomatsakis (Jun 08 2020 at 14:34, on Zulip):

I guess it's basically a "volatile query"

nikomatsakis (Jun 08 2020 at 14:34, on Zulip):

(like, in its most pure form)

matklad (Jun 08 2020 at 14:35, on Zulip):

Yes, there's also build-system desire to know the set of files upfront. For example, rust+buck generally already list the set of files per crate.

nikomatsakis (Jun 08 2020 at 14:35, on Zulip):

that's probably not what we want because it implies that we have to 'poll' each file to know if they changed, vs being informed up front by vscode?

matklad (Jun 08 2020 at 14:35, on Zulip):

nikomatsakis: I'm thinking what a lazy input really means
nikomatsakis: I guess it's basically a "volatile query"

Not really I think

nikomatsakis (Jun 08 2020 at 14:36, on Zulip):

today, if you have an input, and you read an invalid key, you get a panic

nikomatsakis (Jun 08 2020 at 14:36, on Zulip):

the hybrid I could imagine would be that you get a callback

nikomatsakis (Jun 08 2020 at 14:36, on Zulip):

(but you still have to invoke set for any changes)

matklad (Jun 08 2020 at 14:36, on Zulip):

I think I've implemented that at some point already?

nikomatsakis (Jun 08 2020 at 14:36, on Zulip):

maybe it all seems familiar

nikomatsakis (Jun 08 2020 at 14:36, on Zulip):

matklad said:

Not really I think

in what way not?

matklad (Jun 08 2020 at 14:36, on Zulip):

https://salsa-rs.github.io/salsa/common_patterns/on_demand_inputs.html

matklad (Jun 08 2020 at 14:37, on Zulip):

in what way not?

We don't have to make the query volatile. We need a contract that the caller would start .seting the key after we've first looked at it.

nikomatsakis (Jun 08 2020 at 14:37, on Zulip):

yes, ok, so that is what I listed above

nikomatsakis (Jun 08 2020 at 14:38, on Zulip):

nikomatsakis said:

(but you still have to invoke set for any changes)

ie here

nikomatsakis (Jun 08 2020 at 14:38, on Zulip):

and yes I think you can model it today with some synthetic reads, much like that page suggests

nikomatsakis (Jun 08 2020 at 14:38, on Zulip):

so is that not what you want?

matklad (Jun 08 2020 at 14:38, on Zulip):

I don't know what I want, that the problem :)

matklad (Jun 08 2020 at 14:38, on Zulip):

Like, the eager model with explicit sets of files seems very compelling

matklad (Jun 08 2020 at 14:39, on Zulip):

But so does the lazy one , for complementary reasons :D

matklad (Jun 08 2020 at 14:39, on Zulip):

(and I think in lazy model the problem "how do we avoid rechecking std when new file is created in src" remains)

nikomatsakis (Jun 08 2020 at 14:40, on Zulip):

well, not entirely?

nikomatsakis (Jun 08 2020 at 14:40, on Zulip):

by that I mean

nikomatsakis (Jun 08 2020 at 14:41, on Zulip):

if there is a #[path], then there will be an edge to FileContents(/whatever/the/path/is)

nikomatsakis (Jun 08 2020 at 14:41, on Zulip):

and so if there is no set that effects this path, you're ok

matklad (Jun 08 2020 at 14:41, on Zulip):

Hm.... that makes sense I guess

nikomatsakis (Jun 08 2020 at 14:41, on Zulip):

or am I missing something?

matklad (Jun 08 2020 at 14:43, on Zulip):

Hm, yeah, it works if we do path normalization. The potential problem here is that foo/bar and foo/baz/../bar might be different things

nikomatsakis (Jun 08 2020 at 14:44, on Zulip):

yeah, ok, I hadn't considered that

nikomatsakis (Jun 08 2020 at 14:45, on Zulip):

you do need some notion of "truepath"

matklad (Jun 08 2020 at 14:46, on Zulip):

My current plant is to do both lazy and eager:

nikomatsakis (Jun 08 2020 at 14:48, on Zulip):

nikomatsakis said:

you do need some notion of "truepath"

this feels inherent

nikomatsakis (Jun 08 2020 at 14:48, on Zulip):

without canonicalization of some kind, I don't think you can have a "watcher" setup

nikomatsakis (Jun 08 2020 at 14:49, on Zulip):

ok I sort of take that back

nikomatsakis (Jun 08 2020 at 14:50, on Zulip):

I guess that the most generic watcher could be dealing with the canonicalization problem in its own way, sort of hiding it from you, but anyway if you have "set" operations coming in, you need a way to match up the path that is being set with the ones that have been read

matklad (Jun 15 2020 at 14:01, on Zulip):

:wave: @WG-rls2.0 !

This week's changelog is here: https://rust-analyzer.github.io/thisweek/2020/06/15/changelog-29.html

matklad (Jun 15 2020 at 14:01, on Zulip):

extract-enum.gif

matklad (Jun 15 2020 at 14:01, on Zulip):

I'd say this is the highlight of the week from the technical perspective -- it's the first refactoring which is non-local (it calls find usages inside)

matklad (Jun 15 2020 at 14:02, on Zulip):

It looks like there's a huge "semantic search & replace" infrastructure that should be developed around that :)

matklad (Jun 15 2020 at 14:03, on Zulip):

I've also made some progress with the VFS rewrite. The new API is here, and I am pretty happy with how it looks: https://github.com/rust-analyzer/rust-analyzer/pull/4891

matklad (Jun 15 2020 at 14:03, on Zulip):

Though there's still a lot of work to make rust-analyzer actually use it.

nikomatsakis (Jun 15 2020 at 14:04, on Zulip):

matklad said:

I'd say this is the highlight of the week from the technical perspective -- it's the first refactoring which is non-local (it calls find usages inside)

ah, really cool

matklad (Jun 15 2020 at 14:05, on Zulip):

The "ship rust-analyzer with rustup" work is currently blocked on implementation -- we need to make vendoring inside rust-lang/rust work with rust-analyzer (it is non-trivial, as rust-analyzer is a separate workspace). If someone wants to take over the PR adding rust-analyzer to rustc, please free to do this :) I think I'll be busy with VFS for at least several days yet

matklad (Jun 15 2020 at 14:05, on Zulip):

Ohhh, there's another cool thing I've almost forgot

nikomatsakis (Jun 15 2020 at 14:06, on Zulip):

(which PR is that?)

matklad (Jun 15 2020 at 14:06, on Zulip):

We now do syntax highlighting (with type inference and such) of code inside doc comments.

bjorn3 (Jun 15 2020 at 14:06, on Zulip):

nikomatsakis said:

(which PR is that?)

https://github.com/rust-lang/rust/pull/72978

matklad (Jun 15 2020 at 14:06, on Zulip):

PR; https://github.com/rust-lang/rust/pull/72978

matklad (Jun 15 2020 at 14:07, on Zulip):

We now do syntax highlighting (with type inference and such) of code inside doc comments.

We do that by creating an instance of a separate salsa DB for each doc comment

matklad (Jun 15 2020 at 14:08, on Zulip):

And there's an interesting problem here -- because this is a separate instance, this nested syntax highlighting is not really cancellable.

matklad (Jun 15 2020 at 14:08, on Zulip):

It seems like we might need some first-class salsa support for "ephemeral" inputs.

matklad (Jun 15 2020 at 14:09, on Zulip):

This is something we want during code completion as well: we'd love to type-check the code as if it had a fake identifier inserted at the position of the cursor.

matklad (Jun 15 2020 at 14:10, on Zulip):

We curetntly are able to "as if" parse the code (because parsing is a function and not a query), but we actually want to type-check it as well.

nikomatsakis (Jun 15 2020 at 14:12, on Zulip):

matklad said:

We now do syntax highlighting (with type inference and such) of code inside doc comments.

We do that by creating an instance of a separate salsa DB for each doc comment

woah

nikomatsakis (Jun 15 2020 at 14:12, on Zulip):

I guuss what I expected for that was that you would actually literally insert the token into the inputs

nikomatsakis (Jun 15 2020 at 14:12, on Zulip):

but I suppose that would have to happen "outside" of a query

Florian Diebold (Jun 15 2020 at 14:13, on Zulip):

I don't fully understand why we need to do that actually, instead of 'just' checking them as something like a separate file

nikomatsakis (Jun 15 2020 at 14:13, on Zulip):

this seems somewhat connected to "lazy" inputs

nikomatsakis (Jun 15 2020 at 14:13, on Zulip):

well, only somewhat

nikomatsakis (Jun 15 2020 at 14:14, on Zulip):

but yes I feel like for code comments at least you might have some kind of "code identifier" that you can give to that code block

matklad (Jun 15 2020 at 14:14, on Zulip):

Hm, actually we might be able to do what @Florian Diebold suggests at least for doc comments. Have another variant for HirFileId, which is "the text of this doc comment"

nikomatsakis (Jun 15 2020 at 14:14, on Zulip):

yeah, basically

matklad (Jun 15 2020 at 14:14, on Zulip):

Though I don't like how this is very invasive

nikomatsakis (Jun 15 2020 at 14:15, on Zulip):

in salsa, at least, we had some kind of identifier for each "Item" that we are processing, and it might be like "the method inside this impl block in side this file" or whatever

matklad (Jun 15 2020 at 14:15, on Zulip):

Like, HirFielId is a very core type inside the "compiler", and it seems wrong to have to change it just for the purposes of handing doc comments, which, strictly speaking, exist outside of the langauge.

nikomatsakis (Jun 15 2020 at 14:15, on Zulip):

it seems like it would be natural to extend that to doc comments

nikomatsakis (Jun 15 2020 at 14:15, on Zulip):

I can see that point, although to me it feels... "right" :)

nikomatsakis (Jun 15 2020 at 14:15, on Zulip):

like, you want to be able to say "where the code came from"..

matklad (Jun 15 2020 at 14:15, on Zulip):

How would we go about fake idents for completions?

Florian Diebold (Jun 15 2020 at 14:16, on Zulip):

yeah, it feels right to me as well -- similarly to how we handle all crates in one DB, even though the actual compiler has a single-crate view

nikomatsakis (Jun 15 2020 at 14:16, on Zulip):

fake idents I'm not so sure, I'm not 100% sure why you want/need them anyway yet I have to admit

nikomatsakis (Jun 15 2020 at 14:16, on Zulip):

can you remind me the motivation?

matklad (Jun 15 2020 at 14:17, on Zulip):

Like, we can add yet another variant to HirFileId which is "file, like foo.rs, but there's intellijRulez token inserted at offset 42", but feels even more in the wrong direction to me :)

nikomatsakis (Jun 15 2020 at 14:17, on Zulip):

can you just step back and explain to me how the cycle works for a sec? like, when do you decide to insert those tokens?

matklad (Jun 15 2020 at 14:17, on Zulip):

nikomatsakis: can you remind me the motivation?

imagine the following code:

foo == /* cursor here*/
nikomatsakis (Jun 15 2020 at 14:18, on Zulip):

/me imagines. "ok"

matklad (Jun 15 2020 at 14:18, on Zulip):

we want to get the type of the right hand side do suggest nice completions. It's much easier to do if there's a phisical rhs there

nikomatsakis (Jun 15 2020 at 14:18, on Zulip):

do we always want to do this

nikomatsakis (Jun 15 2020 at 14:18, on Zulip):

or did the user hit some key that prompted us to provide completions?

Florian Diebold (Jun 15 2020 at 14:18, on Zulip):

even more so when we're completing something like dbg!(/* cursor */) -- we need to macro-expand the macro as if it actually contained something

matklad (Jun 15 2020 at 14:18, on Zulip):

This happens here:

https://github.com/rust-analyzer/rust-analyzer/blob/5b013e5665a34fd757fd6c48dc912606c0915b2c/crates/ra_ide/src/completion/completion_context.rs#L92-L99

bjorn3 (Jun 15 2020 at 14:19, on Zulip):

The HirFileId could have a variant that represents the concatenation of multiple file parts. For example for

/// fn a() {
/// }

it would be Concat([file.rs:1:4-1:13, file.rs:2:4-2:13]) while for

a.<|>

it would be Concat([file.rs:1:1-1:2, <fake_ident>:1:1-1:1].

Florian Diebold (Jun 15 2020 at 14:20, on Zulip):

or did the user hit some key that prompted us to provide completions?

usually yes, the user hit some key

matklad (Jun 15 2020 at 14:21, on Zulip):

From our perspective, the LSP client asked us to provide completions at this point

Florian Diebold (Jun 15 2020 at 14:23, on Zulip):

@bjorn3 for completions, we don't want to put that into the DB though -- it's not worth caching. Also it's not a different file with different content like with doc comments, it's the same file 'in an alternative world' -- so it's in the same place in the module tree and so on

nikomatsakis (Jun 15 2020 at 14:24, on Zulip):

So what I imagined is that when we were asked to prove completions

nikomatsakis (Jun 15 2020 at 14:24, on Zulip):

we would insert some special token into the token stream

nikomatsakis (Jun 15 2020 at 14:24, on Zulip):

and then just compute completions

nikomatsakis (Jun 15 2020 at 14:24, on Zulip):

which would incrementally recompute type-check etc

nikomatsakis (Jun 15 2020 at 14:24, on Zulip):

in the same way as if the user had actually typed something

nikomatsakis (Jun 15 2020 at 14:25, on Zulip):

This works best to the extent that incremental re-parsing, etc works efficiently, of course

Florian Diebold (Jun 15 2020 at 14:25, on Zulip):

I think that's basically what we want, except that we can throw away all the changes to the DB afterwards?

matklad (Jun 15 2020 at 14:26, on Zulip):

And the "we would insert some special token into the token stream" works by calling the actual .set outside of the query?

nikomatsakis (Jun 15 2020 at 14:26, on Zulip):

well I think then you just re-modify the input to remove the token

nikomatsakis (Jun 15 2020 at 14:26, on Zulip):

right

nikomatsakis (Jun 15 2020 at 14:26, on Zulip):

it's just a normal edit

Florian Diebold (Jun 15 2020 at 14:26, on Zulip):

also, it seems... not thread-safe

nikomatsakis (Jun 15 2020 at 14:26, on Zulip):

nikomatsakis said:

well I think then you just re-modify the input to remove the token

note: you could imagine salsa doing a different caching mechanism to make this more efficient

matklad (Jun 15 2020 at 14:26, on Zulip):

I am not sure about literal modification --- it'll cancel in-progress higlighting and such

nikomatsakis (Jun 15 2020 at 14:26, on Zulip):

all set calls occur when no other queries are ongoing

nikomatsakis (Jun 15 2020 at 14:26, on Zulip):

so yeah it would cancel those sorts of things

nikomatsakis (Jun 15 2020 at 14:27, on Zulip):

I can believe it's not a good strategy :)

nikomatsakis (Jun 15 2020 at 14:27, on Zulip):

it's just the strategy I imagined

Florian Diebold (Jun 15 2020 at 14:27, on Zulip):

yeah, didn't mean literal data races, but things like that

matklad (Jun 15 2020 at 14:27, on Zulip):

I guess, I want exactly this: ability to call .set without advancing global revision counter, and, more generally, keep the DB itself logically immutable

nikomatsakis (Jun 15 2020 at 14:27, on Zulip):

I guess that another way to look at it would be

nikomatsakis (Jun 15 2020 at 14:27, on Zulip):

yes, a "diff" file-id

nikomatsakis (Jun 15 2020 at 14:27, on Zulip):

i.e., this file is the same as that one but with some modification (the "logical cursor") or whatever

nikomatsakis (Jun 15 2020 at 14:28, on Zulip):

but that wouldn't leverage any incremental re-computation necessarily

nikomatsakis (Jun 15 2020 at 14:28, on Zulip):

that I think is the approach that @matklad was kind of rejecting, though for a different reason

nikomatsakis (Jun 15 2020 at 14:28, on Zulip):

but also if you literally did it the wa I was just describing it's still a set in salsa terms

nikomatsakis (Jun 15 2020 at 14:28, on Zulip):

so it would still have all the bad side-effects

nikomatsakis (Jun 15 2020 at 14:29, on Zulip):

it sounds like what you want is the ability to create a "forked" database

matklad (Jun 15 2020 at 14:29, on Zulip):

(I can't help thniking that what we want here is "negative" durability for such transient inputs. Basically, everytying whcih depends on transient input can't be trusted in the next revision and should be teared down)

nikomatsakis (Jun 15 2020 at 14:29, on Zulip):

all of this is reminding me of some of my more ambituous ideas for reworking salsa to be more parallel friendly

nikomatsakis (Jun 15 2020 at 14:29, on Zulip):

where it took more of a "persistent data-structure" like approach

nikomatsakis (Jun 15 2020 at 14:30, on Zulip):

(which would make e.g. forking quite trivial, "just clone")

nikomatsakis (Jun 15 2020 at 14:30, on Zulip):

interesting

nikomatsakis (Jun 15 2020 at 14:30, on Zulip):

side note that

nikomatsakis (Jun 15 2020 at 14:31, on Zulip):

i am about to open a PR that fixes up the recursive solver's handling of negative goals

nikomatsakis (Jun 15 2020 at 14:31, on Zulip):

(in chalk)

nikomatsakis (Jun 15 2020 at 14:31, on Zulip):

at which point I wanted to return to the question of how to better integrate salsa / chalk, I realize now my old scheme didn't accommodate negative goals, but I think it can be fixed somehow :)

nikomatsakis (Jun 15 2020 at 14:31, on Zulip):

point being that it seems like we're due to kind of have some deeper talks about salsa and look again at the model with fresh eyes

matklad (Jun 15 2020 at 14:32, on Zulip):

Yesss

nikomatsakis (Jun 15 2020 at 14:32, on Zulip):

I know you've got an accruing list of "desiderata", matklad

matklad (Jun 15 2020 at 14:32, on Zulip):

I want more parallelism and less Arcs, which, admitedly, seem to be contradictory goals :D

nikomatsakis (Jun 15 2020 at 14:32, on Zulip):

maybe we should schedule some times to do some voice chats this week?

nikomatsakis (Jun 15 2020 at 14:32, on Zulip):

lol, maybe true

nikomatsakis (Jun 15 2020 at 14:33, on Zulip):

I have a few thoughts about parallelism

matklad (Jun 15 2020 at 14:33, on Zulip):

maybe we should schedule some times to do some voice chats this week?

Yup, I think some semi-structured chatting might be useful.

nikomatsakis (Jun 15 2020 at 14:33, on Zulip):

maybe we can start by just collecting ideas into a doc and things

nikomatsakis (Jun 15 2020 at 14:33, on Zulip):

to get a high-level view

matklad (Jun 15 2020 at 14:34, on Zulip):

@nikomatsakis could you just start the doc?

nikomatsakis (Jun 15 2020 at 14:34, on Zulip):

making it now

nikomatsakis (Jun 15 2020 at 14:34, on Zulip):

we can maybe chat a bit in salsa zulip too

nikomatsakis (Jun 15 2020 at 14:35, on Zulip):

not sure which is a better place ;)

nikomatsakis (Jun 15 2020 at 14:35, on Zulip):

hackmd "Salsa 2020"

matklad (Jun 15 2020 at 14:37, on Zulip):

Perfect name!

matklad (Jun 22 2020 at 14:02, on Zulip):

Hello @WG-rls2.0 , its Monday again!

matklad (Jun 22 2020 at 14:03, on Zulip):

Today's changelog is here: https://rust-analyzer.github.io/thisweek/2020/06/22/changelog-30.html

matklad (Jun 22 2020 at 14:04, on Zulip):

We have a bunch of fixes for type inference, cool new UI for exploring types by @vsrs , almost working VFS.

matklad (Jun 22 2020 at 14:05, on Zulip):

Another highlight is that https://github.com/davidlattimore, the author of rerast, is doing some awesome work merging rerast into rust-analyzer

matklad (Jun 22 2020 at 14:06, on Zulip):

Oh, @Jonas Schievink is also doing a bunch of perf work around memory usage and item tree refactoring.

matklad (Jun 22 2020 at 14:08, on Zulip):

No progress on shipping rust-analyzer via rustup -- no particular blockers, it's just that no one got to figuring the vendoring stuff.

nikomatsakis (Jun 22 2020 at 14:09, on Zulip):

@matklad would you like to discuss "Salsa 2020" a bit more today?

matklad (Jun 22 2020 at 14:10, on Zulip):

Yup!

matklad (Jun 22 2020 at 14:10, on Zulip):

I did fill the document. The biggest addition I have is "priorites"

matklad (Jun 22 2020 at 14:11, on Zulip):

I singled out only two really big things:

matklad (Jun 22 2020 at 14:11, on Zulip):

(the documet in question: https://hackmd.io/o07mw34ITEiXgRaouTb4Yw)

nikomatsakis (Jun 22 2020 at 14:14, on Zulip):

@matklad could we schedule a time to chat in a little bit maybe?

nikomatsakis (Jun 22 2020 at 14:14, on Zulip):

I'm trying to finish up my "morning prep"

nikomatsakis (Jun 22 2020 at 14:15, on Zulip):

although actually maybe now is not the worst time

nikomatsakis (Jun 22 2020 at 14:15, on Zulip):

I have to read what you wrote

matklad (Jun 22 2020 at 14:15, on Zulip):

I do think a chat/call would be useful.

nikomatsakis (Jun 22 2020 at 14:33, on Zulip):

@matklad how long are you around today?

matklad (Jun 22 2020 at 14:37, on Zulip):

for at least 4 hours more

nikomatsakis (Jun 22 2020 at 14:41, on Zulip):

OK, I'll ping you in a bit!

nikomatsakis (Jun 22 2020 at 15:38, on Zulip):

@matklad would you be up to have a quick zoom chat soon?

nikomatsakis (Jun 22 2020 at 15:38, on Zulip):

any time basically

matklad (Jun 22 2020 at 15:41, on Zulip):

yup, in 3 minutes!

matklad (Jun 22 2020 at 17:08, on Zulip):

Nope, audio is non-existig :(

matklad (Jun 29 2020 at 14:07, on Zulip):

:wave: @WG-rls2.0 it's sync-up time!

matklad (Jun 29 2020 at 14:07, on Zulip):

Changelog: https://rust-analyzer.github.io/thisweek/2020/06/29/changelog-31.html

matklad (Jun 29 2020 at 14:09, on Zulip):

Notable changes:

matklad (Jun 29 2020 at 14:11, on Zulip):

(concurrency nerds might want to chime in on https://trio.discourse.group/t/sizing-the-channel-deadlock-freedom-vs-back-pressure/311, there's still some unresolved questions left)

matklad (Jun 29 2020 at 14:13, on Zulip):

There are also some ongoing refactors for our test suite

vsrs (Jun 29 2020 at 14:13, on Zulip):

I've published cross-rust-analyzer as well. Perhaps it makes sense to update the repo link in the changelog to the extension link.

matklad (Jun 29 2020 at 14:14, on Zulip):

@vsrs could you PR?

vsrs (Jun 29 2020 at 14:15, on Zulip):

sure, but i don't know where are the changelog sources?

matklad (Jun 29 2020 at 14:15, on Zulip):

There are also some ongoing refactors for our test suite

matklad (Jun 29 2020 at 14:15, on Zulip):

@vsrs https://github.com/rust-analyzer/rust-analyzer.github.io

Jeremy Kolb (Jun 29 2020 at 14:18, on Zulip):

Yesterday I found some time to categorize assists and fix call hierarchy

matklad (Jun 29 2020 at 14:20, on Zulip):

Right, I saw the PR, did get to revieing it yet :)

vsrs (Jun 29 2020 at 14:29, on Zulip):

matklad said:

vsrs could you PR?

Done. https://github.com/rust-analyzer/rust-analyzer.github.io/pull/55

nikomatsakis (Jun 29 2020 at 15:53, on Zulip):

@matklad sorry I missed the sync today, this all sounds exciting though!

matklad (Jul 06 2020 at 13:56, on Zulip):

Today's sync will be an hour late (so, in ~1hr) -- @matklad is late with prepping relnotes :sweat_smile:

matklad (Jul 06 2020 at 15:05, on Zulip):

@WG-rls2.0 :wave:

matklad (Jul 06 2020 at 15:05, on Zulip):

This week's changelog is up: https://rust-analyzer.github.io/thisweek/2020/07/06/changelog-32.html

matklad (Jul 06 2020 at 15:07, on Zulip):
matklad (Jul 06 2020 at 15:07, on Zulip):

Oh, you can now grab rust-analyzer from https://static.rust-lang.org/dist/rust-nightly-x86_64-unknown-linux-gnu.tar.gz !

matklad (Jul 06 2020 at 15:07, on Zulip):

Hopefully, the next nightly would even contain rustup component

matklad (Jul 06 2020 at 15:08, on Zulip):

(which reminds me that I probably should setup automated rust-analzyer upgrade in rust-lang/rust repo

std::Veetaha (Jul 06 2020 at 17:13, on Zulip):

We should make a framework out of this (at some point)

matklad (Jul 13 2020 at 14:00, on Zulip):

:wave: @WG-rls2.0

matklad (Jul 13 2020 at 14:01, on Zulip):

It was a productive week, we have bunch of things in today's release:

https://rust-analyzer.github.io/thisweek/2020/07/13/changelog-33.html

matklad (Jul 13 2020 at 14:02, on Zulip):

The highlights are:

matklad (Jul 13 2020 at 14:03, on Zulip):

We are also moving vscode extension to the rust-lang/vscode-rust repo: https://github.com/rust-lang/vscode-rust/pull/816

Jack Huey (Jul 13 2020 at 14:07, on Zulip):

That Chalk upgrade looks awesome tbh

Jack Huey (Jul 13 2020 at 14:07, on Zulip):

The highlight of my week

matklad (Jul 13 2020 at 14:10, on Zulip):

Yup, it is just so awesome that chalk and rust-analyzer are developed independently, and both are moving fast!

Yay modularization!

Jeremy Kolb (Jul 13 2020 at 14:11, on Zulip):

I haven't looked closely at the PR but our development workflow will be to npm install the client separately? Will there be some sort of integration testing between server/client in the vscode-rust repo?

Jack Huey (Jul 13 2020 at 14:11, on Zulip):

I am curious just how much of a speedup https://github.com/rust-lang/chalk/pull/566 is going to bring

matklad (Jul 13 2020 at 14:15, on Zulip):

but our development workflow will be to npm install the client separately?

Good question, I haven't thought about that deeply. TBH, I almost never run the actual typesript extension myself, so I personally don't mind a lot that there's an extra step here

matklad (Jul 13 2020 at 14:15, on Zulip):

Will there be some sort of integration testing between server/client in the vscode-rust repo?

Not that we have such tests today :P

matklad (Jul 13 2020 at 14:16, on Zulip):

@Jack Huey I'll run analysis stats unless somebody beats me to it :D

Jack Huey (Jul 13 2020 at 14:16, on Zulip):

That would be awesome

Florian Diebold (Jul 13 2020 at 14:22, on Zulip):

I'm looking forward to that even just for cleaning up the logs :sweat_smile:

Jeremy Kolb (Jul 13 2020 at 14:22, on Zulip):

I'm expecting breakage with the recent change in the semantic tokens protocol but am waiting to see how it's handled in vscode first

matklad (Jul 13 2020 at 14:24, on Zulip):

"If you run less code, there's less code to debug" is a profound insight!

Jack Huey (Jul 13 2020 at 14:26, on Zulip):

Florian Diebold said:

I'm looking forward to that even just for cleaning up the logs :sweat_smile:

Oh for sure. I actually am using the branch to debug the cycle issue Matthew Jasper posted

Jack Huey (Jul 13 2020 at 14:26, on Zulip):

That being said, do you want to review/merge that PR @Florian Diebold?

Jack Huey (Jul 13 2020 at 14:26, on Zulip):

I'm not sure when Niko is going to be around and have time to review

matklad (Jul 13 2020 at 14:26, on Zulip):

Ah, so the signature of RustIrDatabase::impls_for_trait has changed, so it's not trivial for me to benchmark

Florian Diebold (Jul 13 2020 at 14:27, on Zulip):

@Jack Huey sure, I can probably take a look later today

matklad (Jul 13 2020 at 14:27, on Zulip):

I guess, @Florian Diebold might want to do this then :D

Florian Diebold (Jul 13 2020 at 14:27, on Zulip):

I guess @Jack Huey needs to rebase his PR

Jack Huey (Jul 13 2020 at 14:28, on Zulip):

whoops okay

Jack Huey (Jul 13 2020 at 14:29, on Zulip):

Okay rebased

matklad (Jul 13 2020 at 14:47, on Zulip):

@Jack Huey

Before:
Parallel Inference: 12.637288447s,

After:
Parallel Inference: 10.732962004s,
matklad (Jul 13 2020 at 14:47, on Zulip):

So, it is quite a bit faster, if we see 20% improvement in end-to-end bench

Jack Huey (Jul 13 2020 at 15:15, on Zulip):

That's not bad at all

detrumi (Jul 13 2020 at 16:49, on Zulip):

On to the next 20% then

Jeremy Kolb (Jul 13 2020 at 20:05, on Zulip):

Does anyone know how to match against a const cow?

matklad (Jul 20 2020 at 13:56, on Zulip):

Hey, I am not feeling great today, so I won’t participate in the sync up, but the release is out :)

Jeremy Kolb (Jul 20 2020 at 14:05, on Zulip):

Feel better!

matklad (Jul 27 2020 at 14:02, on Zulip):

:wave: @WG-rls2.0

matklad (Jul 27 2020 at 14:02, on Zulip):

Here's this week's changelog: https://rust-analyzer.github.io/thisweek/2020/07/27/changelog-35.html

matklad (Jul 27 2020 at 14:04, on Zulip):

Couple of highlights:

matklad (Jul 27 2020 at 14:05, on Zulip):

VS Code extension merging is still in progress

matklad (Jul 27 2020 at 14:06, on Zulip):

I've also started poking rustc's parser with a sharp stick today :) Trying to find a first-step towards parse library-ification

matklad (Jul 27 2020 at 14:38, on Zulip):

Also, it's a first time in a long while I am using rust-analyzer with rustc code base, and the progress is nice

matklad (Jul 27 2020 at 14:39, on Zulip):

Auto-importing stuff feels almost to good

matklad (Jul 27 2020 at 14:39, on Zulip):

And even auto-reload when adding new crate works!

detrumi (Jul 27 2020 at 14:53, on Zulip):

Metrics dashboard looks really cool! Would it be possible to split the build/analysis graphs into categories? (e.g. separate build/link times)

matklad (Jul 27 2020 at 14:54, on Zulip):

Yes, but I think that probably isn't required. Total is enough to catch regressions. For investigation, you'll get more info re-running stuff locally

matklad (Jul 27 2020 at 14:55, on Zulip):

And there's a benefit in having fewer graphs on the page

Laurențiu Nicola (Jul 27 2020 at 14:55, on Zulip):

Showing the commit hashes would be useful, because right now the horizontal axis doesn't say much

matklad (Jul 27 2020 at 14:56, on Zulip):

Well, it says 1, 2, 3, 4.... isn't that useful? :D

matklad (Jul 27 2020 at 14:56, on Zulip):

There's info about commit hashes, host machine, time, etc in the data

matklad (Jul 27 2020 at 14:57, on Zulip):

But I haven't looked into how to plug that into plotly.js

matklad (Jul 27 2020 at 14:58, on Zulip):

https://github.com/rust-analyzer/metrics/blob/f9d246925e87e6786228cfec09773a9fb46d8320/index.html#L52-L55

matklad (Jul 27 2020 at 14:58, on Zulip):

I think there should be API to add hover info to the labels

matklad (Aug 03 2020 at 14:03, on Zulip):

:wave: @WG-rls2.0

matklad (Aug 03 2020 at 14:04, on Zulip):

Here's this week chagelog: https://rust-analyzer.github.io/thisweek/2020/08/03/changelog-36.html

simulacrum (Aug 03 2020 at 14:05, on Zulip):

did we get the nightly update going again? I can take a look at disabling riscv for you if not

matklad (Aug 03 2020 at 14:05, on Zulip):

@simulacrum no, I haven't got to that yet

matklad (Aug 03 2020 at 14:05, on Zulip):

If yo could disable riscv, that would be helpful and I think more effective!

simulacrum (Aug 03 2020 at 14:06, on Zulip):

great! will do so in a bit

matklad (Aug 03 2020 at 14:06, on Zulip):

Though, until we flip the VS Code extension from matklad.rust-analyzer to rust-lang.rust, I wouldn't worry too much about frequently updating nightlies

simulacrum (Aug 03 2020 at 14:07, on Zulip):

ah okay

matklad (Aug 03 2020 at 14:07, on Zulip):

The highlight of this week is Rust ungrammar: https://github.com/rust-analyzer/ungrammar/blob/master/rust.ungram

matklad (Aug 03 2020 at 14:08, on Zulip):

We now fully specify structure of Rust's concrete syntax trees.

I hope that, using ungrammar, we'll be able to slowly bridge rustc's and rust-analyzer syntax trees. I want them both to be generated from ungrammar.

matklad (Aug 03 2020 at 14:09, on Zulip):

Though, it might be a bad IDE -- rustc's tree is a sufficiently abstract syntax tree, and generating it from CST grammar might be too annoying.

matklad (Aug 03 2020 at 14:10, on Zulip):

There's also an MCP in progress for estabilishing parse library-ification working group: https://github.com/rust-lang/compiler-team/issues/338

matklad (Aug 03 2020 at 14:11, on Zulip):

Finally, I am on vacation from Thursday to Tuesday. I wonder if @Jonas Schievink would want to be a release manager next Monday?

Jonas Schievink (Aug 03 2020 at 14:11, on Zulip):

I think I can do that, yeah. Just need some instructions.

matklad (Aug 03 2020 at 14:14, on Zulip):

I'll try to add them to dev/readme, but most things are automated

$ git clone rust-analyzer
$ git clone rust-analyzer.github.io
$ pushd rust-analyzer; cargo xtask release; popd
$ cd rust-analyzer.github.io;
$ emacs (ls thisweek/_posts | tail -n1)
$ git push
matklad (Aug 03 2020 at 14:15, on Zulip):

This releases latest nightly as stable, so there are no any kind of additional pre-flight checks

Jonas Schievink (Aug 10 2020 at 14:00, on Zulip):

Hi @WG-rls2.0 :wave:

Jonas Schievink (Aug 10 2020 at 14:00, on Zulip):

This week's changelog: https://rust-analyzer.github.io/thisweek/2020/08/10/changelog-37.html

Jonas Schievink (Aug 10 2020 at 14:00, on Zulip):

(I ran this week's release since Aleksey is on holidays)

Jonas Schievink (Aug 10 2020 at 14:02, on Zulip):

We got a bunch of nice improvements and bug fixes in this release. No major refactorings or anything fundamental though, as one might expect.

std::Veetaha (Aug 10 2020 at 15:12, on Zulip):

Hmm, have I missed some organizational changes in rust-analyzer?

Laurențiu Nicola (Aug 10 2020 at 15:13, on Zulip):

https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/weekly.20sync-up/near/205798254

matklad (Aug 17 2020 at 14:13, on Zulip):

Hey @WG-rls2.0 it's Monday again

matklad (Aug 17 2020 at 14:13, on Zulip):

I am back from :palm_tree: vacation :)

matklad (Aug 17 2020 at 14:14, on Zulip):

This week's changelog: https://rust-analyzer.github.io/thisweek/2020/08/17/changelog-38.html

matklad (Aug 17 2020 at 14:14, on Zulip):

/me remembers the need to update rust-analzyer in rust-lang/rust

Jonas Schievink (Aug 17 2020 at 14:15, on Zulip):

Let's see if this update sticks. The musl build should work, at least.

matklad (Aug 17 2020 at 14:15, on Zulip):

This was relatively quite week, I think the biggest change is that we've removed ra_ prefixes from crate names

detrumi (Aug 17 2020 at 14:18, on Zulip):

#5347 add rust-analyzer dump-chalk command.

(from the changelog) That ended up being changed to a CHALK_PRINT setting instead to print chalk programs in log output

Paul Faria (Aug 17 2020 at 14:22, on Zulip):

I'd like to work on improving support within macro_rules blocks. Is there anyone I could work with to ensure I'm going down the right path for the design? Re https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/Analysis.20within.20macro_rules/near/206934479

matklad (Aug 17 2020 at 14:23, on Zulip):

ouch, I should get back to this, still cleaning up vacation backlog, thanks for reminidng me!

Paul Faria (Aug 17 2020 at 14:25, on Zulip):

No worries, I also wasn't sure if I was pinging the right person, so I thought it might be better to ask during the sync-up.

Jack Huey (Aug 17 2020 at 14:34, on Zulip):

detrumi said:

#5347 add rust-analyzer dump-chalk command.

(from the changelog) That ended up being changed to a CHALK_PRINT setting instead to print chalk programs in log output

Ooh nice. I'm excited to see how this helps/changes debugging

detrumi (Aug 17 2020 at 14:36, on Zulip):

Some names aren't being shown in the output yet, but eventually you'll be able to take that output and drop it into the chalk cli directly

matklad (Aug 24 2020 at 14:08, on Zulip):

:wave: @WG-rls2.0

matklad (Aug 24 2020 at 14:08, on Zulip):

This week's changelog is up: https://rust-analyzer.github.io/thisweek/2020/08/24/changelog-39.html

matklad (Aug 24 2020 at 14:09, on Zulip):

Nothing to big happened in this release, although today, thanks to the awesome work by @pksunkara, we published a bunch of auto-published crates to crates.io

matklad (Aug 24 2020 at 14:10, on Zulip):

There's some progress on various parser/lexer bits in rust-lang/rustc, but it is suuuper slow

matklad (Aug 24 2020 at 14:11, on Zulip):

And I am also trying to sneak some snapshot tests into rustc :D

https://github.com/rust-lang/rust/pull/75773

matklad (Aug 24 2020 at 14:13, on Zulip):

Oh, I forgot the actual big thing

matklad (Aug 24 2020 at 14:13, on Zulip):

https://github.com/rust-lang/vscode-rust/pull/840

matklad (Aug 24 2020 at 14:13, on Zulip):

there's now a WIP PR merging typescript extensions!

Igor Matuszewski (Aug 24 2020 at 14:17, on Zulip):

I'm back from vacation so I'll resume the work this week =)

Laurențiu Nicola (Aug 31 2020 at 15:23, on Zulip):

This week's changelog is up: https://rust-analyzer.github.io/thisweek/2020/08/31/changelog-40.html :stuck_out_tongue:

matklad (Aug 31 2020 at 15:24, on Zulip):

OMG, @WG-rls2.0 I completely forgot about the sync-up

matklad (Aug 31 2020 at 15:24, on Zulip):

Thanks @Laurențiu Nicola :heart:

matklad (Aug 31 2020 at 15:25, on Zulip):

So, yeah, we have a bunch of stuff coming together this week

matklad (Aug 31 2020 at 15:26, on Zulip):
matklad (Aug 31 2020 at 15:28, on Zulip):

Oh, and I'll be on vacation the next two weeks :P

Jonas Schievink (Sep 07 2020 at 15:09, on Zulip):

Hey @WG-rls2.0! Sorry for the delay today, but the release should be happening now. As was mentioned last week, Aleksey is on holidays, so I'm doing the release.

Jonas Schievink (Sep 07 2020 at 15:11, on Zulip):

I'll also review some PRs to keep the queue clean, but probably only later this week.

Jonas Schievink (Sep 14 2020 at 14:04, on Zulip):

Hello @WG-rls2.0! :wave:

This week's changelog: https://rust-analyzer.github.io/thisweek/2020/09/14/changelog-42.html

Notably, we now support async blocks, a much requested feature, but there's a bunch of other cool stuff in there as well.

Jonas Schievink (Sep 14 2020 at 14:06, on Zulip):

Aleksey is still on holidays this week, but should be back for next week's release. I'll try to continue with the PR reviews until then.

veykril (Sep 14 2020 at 20:36, on Zulip):

If I'm not mistaken, the configuration for the MergeBehaviour isn't actually in this release? At least vscode doesnt seem to recognize the config option in the settings.json and it doesnt seem to be in the extensions package.json either. Did the release not actually pick that PR up or did I make a mistake in my PR :sweat_smile:? 5985

Jonas Schievink (Sep 14 2020 at 20:44, on Zulip):

Oh! I think releasing will just promote the already-built nightly binary, so the PR isn't included.

veykril (Sep 14 2020 at 20:45, on Zulip):

Ah okay that would make sense

Jonas Schievink (Sep 14 2020 at 20:46, on Zulip):

I'll remove it from the changelog

Laurențiu Nicola (Sep 15 2020 at 06:28, on Zulip):

@veykril since you're here, I didn't test your PR, but I'm not sure what the options mean. Which one is the "last layer", and can I use last to only merge one level of imports but without nesting them (e.g. use std::io::{self, Read} but not use std::{fs, io::{self, Read}}? I suspect that "last layer" can't really be ambiguous, though.

veykril (Sep 15 2020 at 07:36, on Zulip):

Yes thats what is basically meant with it, I knew the wording wasn't the best, but I couldn't think of a good concise way to describe that behaviour. It basically is meant to only merge imports if the resulting import only contains one level of nesting.

veykril (Sep 15 2020 at 07:38, on Zulip):

Basically it's supposed to do what was asked in this issue https://github.com/rust-analyzer/rust-analyzer/issues/3831

Laurențiu Nicola (Sep 15 2020 at 07:43, on Zulip):

So it's "merge, but don't nest". Cool, I've always wanted that.

matklad (Sep 21 2020 at 14:04, on Zulip):

:wave: @WG-rls2.0 it's the meeting time again!

matklad (Sep 21 2020 at 14:04, on Zulip):

I am back from vacation and trying to catch up

matklad (Sep 21 2020 at 14:05, on Zulip):

Looks like I am not really necessary to keep the pace of development of rust-analyzer, that's great!

I am especially impressed that we've shipped async blocks!

matklad (Sep 21 2020 at 14:05, on Zulip):

Thanks @Jonas Schievink for picking up the org stuff!

matklad (Sep 21 2020 at 14:07, on Zulip):

I guess, there's nothing too specific I can call out this week (still doing the catch up), but if I am needed to review or unblock something, feel free to agressively re-ping me!

Laurențiu Nicola (Sep 28 2020 at 15:35, on Zulip):

This week's changelog is up: https://rust-analyzer.github.io/thisweek/2020/09/28/changelog-44.html :smile:

matklad (Oct 12 2020 at 14:01, on Zulip):

Hello @WG-rls2.0, it's that time again!

Today's changelog is up!

https://rust-analyzer.github.io/thisweek/2020/10/12/changelog-46.html

No progress on parser library-ification sadly, I hope to get back to it next week.

nikomatsakis (Oct 12 2020 at 14:01, on Zulip):

@matklad wow congratulations! :ring:

Coenen Benjamin (Oct 12 2020 at 14:01, on Zulip):

Congrats @matklad

Igor Matuszewski (Oct 12 2020 at 14:02, on Zulip):

Congratulations!

Florian Diebold (Oct 12 2020 at 14:02, on Zulip):

congratulations :smiley:

Laurențiu Nicola (Oct 12 2020 at 14:20, on Zulip):

Congratulations!

bjorn3 (Oct 12 2020 at 14:24, on Zulip):

Congratulations!

Edwin Cheng (Oct 12 2020 at 15:07, on Zulip):

Congratulations!!

Kirill Bulatov (Oct 12 2020 at 15:50, on Zulip):

Oh, congrats!

Paul Faria (Oct 14 2020 at 13:54, on Zulip):

I'm a bit late, but Congratulations!

std::Veetaha (Oct 14 2020 at 14:28, on Zulip):

Oh my, congrats :D

matklad (Oct 19 2020 at 14:07, on Zulip):

@WG-rls2.0 it's Monday again!

matklad (Oct 19 2020 at 14:07, on Zulip):

Today's changelog: https://rust-analyzer.github.io/thisweek/2020/10/19/changelog-47.html

matklad (Oct 19 2020 at 14:07, on Zulip):

There's quite a bunch of "improve quality & polish" changes, I like that!

matklad (Oct 19 2020 at 14:10, on Zulip):

In the highlights, we have a new TextMate grammar, rewrote our build automation to use xshell (and I want to better automate all other builds: https://github.com/rust-analyzer/lsp-server/blob/master/xtask/src/main.rs) and introduces S-actionable and S-unactionable issue labels

Jeremy Kolb (Oct 19 2020 at 14:31, on Zulip):

I'm seeing some undefined responses coming back from inlay hints on master :(

matklad (Oct 19 2020 at 14:33, on Zulip):

Hm, is there an issue for that?

Jeremy Kolb (Oct 19 2020 at 14:33, on Zulip):

https://github.com/rust-analyzer/rust-analyzer/issues/6285

matklad (Oct 26 2020 at 14:13, on Zulip):

:wave: @WG-rls2.0 !

It's that time again (actually, it is different time this time, due to DST shenanigans)

matklad (Oct 26 2020 at 14:13, on Zulip):

Today's changelog: https://rust-analyzer.github.io/thisweek/2020/10/26/changelog-48.html

matklad (Oct 26 2020 at 14:14, on Zulip):

We have a whole bunch of features and bugfixes. Additionaly, @popzxc is working on refactoring our completion infrastructure

matklad (Oct 26 2020 at 14:15, on Zulip):

There's one notable bug-fix: cfg_if now works in std, which unlocks platform-specific modules.

The implemenation is ungreat -- we just pick random cfg-if from the crate graph and snap it onto std crates. It's not super clear what the proper fix should look like

matklad (Oct 26 2020 at 14:16, on Zulip):

See the writeup at https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Moving.20.2Flibrary.20to.20a.20new.20worksapce which proposes some direction for this issue

nikomatsakis (Oct 26 2020 at 14:26, on Zulip):

Hmm

nikomatsakis (Oct 26 2020 at 14:27, on Zulip):

I don't think I realized that #t-compiler > Moving /library to a new worksapce could help with IDEs

matklad (Oct 26 2020 at 14:29, on Zulip):

By itself, it would help a bit, it also needs "and vendor crates.io deps of stdlib into rust-src component" to be truly useufl

Laurențiu Nicola (Oct 26 2020 at 14:41, on Zulip):

It would also help with -Zbuild-std, I assume

Jeremy Kolb (Oct 26 2020 at 16:38, on Zulip):

There seem to be a lot of issues related to the new TM grammar. Having an up to date grammar is great but maybe we need to figure out how to make it more testable or disable it by default until it's a little more baked?

matklad (Oct 26 2020 at 16:40, on Zulip):

We fixed a bunch of issues in this release, so I think it would be better now, and more or less fine after a couple of releases more.

matklad (Oct 26 2020 at 16:41, on Zulip):

I don't think there's a way to simultaneously:

With our rapid release cycle and ability to easily not upgrade the extensions, I think we should favor "break things fast" approach

matklad (Oct 26 2020 at 16:42, on Zulip):

I wonder if all those issues were reported against stable or against nightly?

matklad (Oct 26 2020 at 16:43, on Zulip):

In theory, nightly should act as a buffer here

Jeremy Kolb (Oct 26 2020 at 16:43, on Zulip):

That's fair. We have a really good test framework for the rust side of things. TS needs more test love.

Laurențiu Nicola (Oct 26 2020 at 16:43, on Zulip):

https://github.com/rust-analyzer/rust-analyzer/pull/6137 was merged 14 days ago, so it was in.. one stable release?

Jeremy Kolb (Oct 26 2020 at 16:44, on Zulip):

Hm I'm not sure about stable vs nightly. I think if it was on stable we'd see more complaints on reddit

matklad (Oct 26 2020 at 16:44, on Zulip):

It's not just what we have, its what we could have. I belive most of the Rust stuff is unit-testable, while most of TS stuff is only testable by folks reported bugs

matklad (Oct 26 2020 at 16:46, on Zulip):

maybe its time to introduce beta:

When installing extension, opt-in 10% of the users into beta.

That should give us a much better coverage

matklad (Oct 26 2020 at 16:46, on Zulip):

However, to make this really deliver value, we would need beta backport process, and that's a lot of process!

Last update: Oct 28 2020 at 17:45UTC