Shall we begin? Since the doodle pool was not too popular, I expect the most interested parties are represented :-)
though I'll say up front: I don't have too much to report (mostly some questions about objectives/scope/requirements)
and given that I didn't finish pre-triage for the compiler meeting in an hour
I don't have any planed agenda for the meeting as well
I might want to just keep this pretty short?
Sure, seems fine!
I've only started a little bit with name resolution librarification stuff
let me start with a couple of ... claims? hypotheses.
sure, go ahead!
1. We want to create a name resolution library that can be used as a reference spec for how name resolution should behave, on correct programs
the reason I add that bit at the end, is because @Vadim Petrochenkov made a comment some time ago that a bulk of the complexity in nameres comes from supporting diagnostics for the error cases
but we want at least detect erroneous programs?
I imagine we don't want to claim that a given program is valid
when, based at least on nameres algorithm, it is not valid
yeah, that's what I had in mind, sounds good!
I have imagined a couple different ways to try to construct such an artifact
at this point, my intention is to make a name-resolver that runs in parallel with the existing one
and the two compare their results
for the correct cases
while the reference name-resolver makes no attempt to match diagnostic quality for the incorrect cases
by "runs in parallel"
I don't actually mean on parallel cores
Seems good, longish-term. Short-term, I expect plugging into rustc would be a ton of work. Another way to get something useful is by plugging into rust-analyzer first.
but rather than the code is just added to corresponding points in the control flow of the compiler
Yes, that is the place where I have been wondering about strategy here
namely, plugging into
rustc is a ton of work
Could we perhaps reuse save-analysis for comparison?
oh that's an interesting idea
Like, we don't plug code into rustc, we ask rustc to dump data, and then run test-suite against that data
We'll need a parser for rust, of course, but there are a couple available :)
I had been assuming that a reason it made sense to plug into rustc was that this would simultanously: 1. yield the testing infrastructure (since its just a mode you turn on via a rustc flag), and 2. yield a software artifact that we would be semi-confident in at the outset (since it was written to match the control-flow of rustc itself)
but I hadn't considered that save-analysis gives us an alternative path for testing
so, lets try the thought experiment then
Plugging direrctly into rustc runs the risk of replicating rustc's interface, which may not be ideal
right, that is the risk: How to make something general purpose enough that it can be reused elsewhere
do you think it makes sense to try to plan ahead for eventual integration with rustc (the aforementioned testing strategy, lets call it Just-in-Time Testing)
because I was semi-concerned about the opposite problem: If i start with Rust-analyzer's nameresolver
I may end up with something that I cannot effectively plug into
though I suppose this is somewhat silly
I think no, we shouldn't aim at integration from the outset: integration with rustc is a huge chunk of work, probably larger than just doing the thing from scratch.
i.e. anything that can work for an IDE (and that produces correct results), should be able to be hacked on in some manner to integrate with a batch compiler.
okay. I can go along with that
certainly sounds more palatable to me for the short term.
basically, the biggest value we can get here is wild experimentaion
another question that has arisen when I've posed the first claim above, that all we need to care about initially is replicating the behavior on correct programs
that has led some to point out "IDE's may want to produce diagnostic suggestions on incorrect programs"
suggesting the changes the user might want to make
so there's a question of how much of that to plan ahead for
Do you have any thoughts there? Or is that something you think we can worry about after we get something running that only worries about correct programs?
Good question! An interesting aspect of dagnostics in IDE is that they should be structural (like in structured logging)
So, you don't print spans on stderr, you produce some blob of data specific to the problem, and handle these data in the "fixes" layer, completely outside of resolver
the fixes layer isn't expected to need to extract information from the Error value it gets from the resolver?
I was assuming it might want e.g. a way to see nearby names ("near" in the sense of string-matching)
it is expected: it should downcast error to a specific variant
my current thinking, in any case, is that if the API is designed appropriately, then we should be able to swap in different name resolution libraries
where the library acting as the reference spec would produce Error objects that are practically
Yeah, seems resonable: create allowance for reporting errors, but report an ZST
actually, could we step back a bit
What do we mean that we say "name resolution"?
There's I think two rather orhogonal slices to the thing
the first one is "modules & imports" bit, with fixed point, etc
the second one is lexical scope stuff
which ... might have some overlap today?
I remember seeing some bug recently pointing out that you can write
impl blocks within a
Imports are also a "lexical scope stuff" with uniform paths.
But with fixed point, etc!
oh hi @Vadim Petrochenkov !
Hi @Vadim Petrochenkov !
Hm, but even local imports look only for items?
Like, you can have "modules and imports" inside functions, but it's still doesn't really intersect with lexical scopes?
In other words, would erasing expressions in function bodies change the meaning of items in function bodies?
Right now imports referring to local variables / generic parameters are actively reported as errors.
(Rather than skipping local variables and searching for items in outer scopes.)
With successful compilation imports can only refer to items, yes (+ preludes, built-in things, etc).
Yeah, that's... interesting
But I'd ideally want to unify the name search algorithm between early and late resolution, there are fewer differences than similarity now.
the reason why I seek this separation is that, in IDE, we really want to avoid name resolving function bodies
until we actually need to, eg, typecheck the function
isn't it enough to just delay such name resolution?
I suppose so. Like, they don't have to be two algoithms, they can be a single algorithm, operating on different sets of data
or is there some reason to avoid it, beyond latency?
Big-O complexity, basically
there are far more body code than top-level code
I'd ignore erroneous imports referring to locals for a start, it's an extremely corner case.
it seems to me like we should be able to delay such work until you want to e.g. type-check or resolve trait impls.
Yeah.... I guess this is all for my "stepping a step back": we should try to find ways to be lazy with nameres.
with regards to using RA as the basis for an initial nameres library
well, not as basis, as a client: I feel the lib should be independent, and have its own IR, like chalk
my understanding is that RA today does not attempt to produce results that are always correct
That's very true
do you think it would be better to have a nameres library that has multiple modes, in terms of how fast it gives results vs the correctness of those results
or do you think its better to focus on correctness alone
and assume that we'll figure out ways to keep the latency under control?
What would "fast" mode do?
it might fail to resolve
due to not expanding macros, for example
I feel it's crucial to aim for "100% correct if we fix all the bugs"
and it's very important to have lazyness, or, at least, incrementality
Basically, re-nameresolving after you modify function body should be fast
sure. I had said in the past that I figured one way to go here would be to have an asynchronous interface
If we can't do both "correct" and "lazy enough", that would be a pretty difficult position to be
with respect to the whole idea of using the same code base for compiler and ide, etc
I agree with that
I mostly was curious if, based on your work so far, if you thought that such a goal was not achievable in the first place.
(I suppose if you thought that, then we wouldn't be having this conversation at all)
I am now feeling more optimistic than initially :0)
okay. So I think the main action items I would like to look at
Salsa works wonders, and our (super-buggy) fixed-point iteration loop actually achives the required lazyness/latency
is to try to factor out code from RA that does name-resolution, and that validates its result against save-analysis
if I can get something running (and that presumably yields test failures on the appropirate cases exercising the bugs)
then that would be a proof-of-concept for what we're talking about here
though, I would add a step before that
(and the main issue might be if its too tightly tied to rust-analyzer API stuff)
(but being too tightly tied to rust-analyzer seems like a far better scenario than being too tightly tied to rustc API)
- scetch the IR required for name res
ah yes, okay
@pnkfelix if you feel like you need to run to that other thing, I feel we've discussed the most important bits :)
I think I do need to try to spend the next 10 minutes doing prep for the meeting that starts in 10 minutes.
thanks for this discussion, it has been super helpful food
I'll write notes and post them to rust-lang/compiler-team repo
Thanks for participating pnkfelix and Vadim Petrochenkov !