Stream: t-compiler/wg-rls-2.0

Topic: general


matklad (Mar 21 2019 at 19:12, on Zulip):

I am experimenting with providing some actual diagnostics!

matklad (Mar 21 2019 at 19:12, on Zulip):

pasted image

Cole Lawrence (Mar 26 2019 at 16:24, on Zulip):

How much overlap does RA and rustc have? I noticed that HIR is approaching stabilization (rustc) and would lead towards stabilizing incremental compilation. Is there overlap between rust-analyzer and that progress?

matklad (Mar 26 2019 at 16:27, on Zulip):

There's not that much overlap happening at the moment unfortunately. The most interesting bits, from IDE perspective, tend to happen before hir (syntax trees, parsing, macro expansion), so ra currently attacks those.

matklad (Mar 26 2019 at 16:29, on Zulip):

Though, producing HIR/MIR using ra frontend is one long-term path to merging the projects.

Another long-term path is librarification: for example, both rustc and ra could use chalk. We hope to produce similar libraries for name resolution, etc

Cole Lawrence (Mar 26 2019 at 16:29, on Zulip):

Is there an issue or documentation describing why "traits" are far into the future? https://github.com/rust-analyzer/rust-analyzer/issues/909#issuecomment-468366485

matklad (Mar 26 2019 at 16:30, on Zulip):

well, time flies fast in rust-analyzer, 26 days ago is a distant past :) We already started on chalk integration, see wg-rls-2.0/chalk subgroup :)

matklad (Mar 26 2019 at 16:39, on Zulip):

@Cole Lawrence this the paper doc from planing the traits: https://paper.dropbox.com/doc/Chalk-RLS-integration--AaBRVmOY2Qv~GNc~8re8bL_RAg-QCJelXDeaq7GV6jVQln4j, you might find it interesting

Cole Lawrence (Mar 26 2019 at 16:45, on Zulip):

Thanks @matklad I'll start reading into chalk and salsa and see where that takes me :-)

matklad (Mar 26 2019 at 16:46, on Zulip):

cc @Florian Diebold , do we have some good-first-bug issues for chalk integration? :)

Florian Diebold (Mar 26 2019 at 16:59, on Zulip):

hmm not really so far, I'll need to think a bit about the next steps, maybe then I'll come up with something...

Jeremy Kolb (Mar 27 2019 at 14:15, on Zulip):

@matklad Looks like the VM is stuck booting on Travis.

Jeremy Kolb (Mar 27 2019 at 14:29, on Zulip):

Looks like it finally went through.

Cole Lawrence (Mar 27 2019 at 18:12, on Zulip):

Kinda a weird idea, but has anyone considered that HIR could be used as a set of building blocks to create a structural editor? Think of like Scratch... But it translates into HIR->MIR->LLVM

Cole Lawrence (Mar 27 2019 at 18:13, on Zulip):

Can rust-analyzer HIR be translated to Rustc HIR? I imagined that this would have to be built on top of rustc's HIR

Cole Lawrence (Mar 27 2019 at 18:14, on Zulip):

Then, with chalk it seems like there would be a lot of heavy lifting with type support taken to build off of

Chris Czub (Mar 28 2019 at 04:10, on Zulip):

Is it possible to pass a feature flag in a project with rust-analyzer's VSCode integration? In the rls VSCode extension you can pass rust.features but I didn't see anything similar in rust-analyzer

matklad (Mar 28 2019 at 06:14, on Zulip):

@Chris Czub we don't hande cfg's yet, so we don't support feature in any specific way

matklad (Mar 28 2019 at 06:15, on Zulip):

@Cole Lawrence maybe? Though, I might be a wrong person to ask: based on what I know about JetBrains MPS, I am very strongly biased towards the plain text representation + good idea support (basically, extend selection gives you 90% of paredit for any language anyway) :)

Chris Czub (Mar 29 2019 at 00:23, on Zulip):

Is this being tracked in GitHub?

matklad (Mar 29 2019 at 05:39, on Zulip):

@Chris Czub no, might be a good idea to create one!

Edwin Cheng (Apr 16 2019 at 16:52, on Zulip):

Do anyone try to run cargo test for master in rustc 1.34 (Windows) , It seem to be dead lock in test_runnables_project.

cargo test test_runnables_project -- --nocapture
Edwin Cheng (Apr 16 2019 at 16:52, on Zulip):

I just wanna know whether it is my own machine problem, I tested it in macos and its works.

marcogroppo (Apr 16 2019 at 21:18, on Zulip):

Do anyone try to run cargo test for master in rustc 1.34 (Windows) , It seem to be dead lock in test_runnables_project.

Hi! I have the same problem... it started happening after a recent (< 7 days ago) commit, I have another (older) branch where the test runs fine

matklad (Apr 16 2019 at 21:28, on Zulip):

It might be multiple-workspaces support

Edwin Cheng (Apr 17 2019 at 02:11, on Zulip):

@Marco Groppo Thanks, i filed a issue in github for that. I will dig a little bit deeper to try understand whats happened.

Edwin Cheng (Apr 19 2019 at 06:25, on Zulip):

Hi, I am working on https://github.com/rust-analyzer/rust-analyzer/issues/1149 And try to replace alll as_source_file() to source_file(). However, after i change and build codes in navigation_target.rs. the following error occurs:

error[E0599]: no method named `parse` found for type `&&db::RootDatabase` in the current scope
  --> crates\ra_ide_api\src\references.rs:63:22
   |
63 |     let file = db.parse(position.file_id);
   |                      ^^^^^
   |
   = note: the method `parse` exists but the following trait bounds were not satisfied:
           `&&db::RootDatabase : ra_db::SourceDatabase`
           `&db::RootDatabase : ra_db::SourceDatabase`
           `db::RootDatabase : ra_db::SourceDatabase`
   = help: items from traits can only be used if the trait is implemented and in scope
   = note: the following trait defines an item `parse`, perhaps you need to implement it:
           candidate #1: `ra_db::SourceDatabase`

I am not familiar how salsa works, but i am so sure i imported SourceDataBase and RootDatabase in that file. (i didn't change any imports in that file), do anyone know why ? I can make it pass by change it to (db as SourceDataBase).parse , but just feel so weird ...

detrumi (Apr 19 2019 at 06:26, on Zulip):

That's a bug with incremental compilation

detrumi (Apr 19 2019 at 06:28, on Zulip):

You can fix it using something like rm -rf ./target/debug/incremental/ (sadly only temporarily, it can happen again)

detrumi (Apr 19 2019 at 06:28, on Zulip):

See this issue: https://github.com/rust-analyzer/rust-analyzer/issues/826

detrumi (Apr 19 2019 at 06:29, on Zulip):

Might be fixed on newer Rust versions, though

Edwin Cheng (Apr 19 2019 at 06:30, on Zulip):

Oh .. that's so weird. thanks a lot.

matklad (Apr 19 2019 at 08:33, on Zulip):

I think the current rustc-side issue is https://github.com/rust-lang/rust/issues/60010

pnkfelix (Apr 19 2019 at 11:17, on Zulip):

(technically the rustc-side issue is #58291 ; #60010 may or may not be related to the root cause)

L.apz (Apr 26 2019 at 18:23, on Zulip):

Recently the rls has been panicking for me and it seems to be casued by the type inference query. I got a backtrace here https://gist.github.com/Lapz/0589bb8fbcd8f631b034356b64983a11.

L.apz (Apr 26 2019 at 18:25, on Zulip):

It seems to be caused by the syntax highlighting

L.apz (Apr 26 2019 at 19:16, on Zulip):

Also i goto really slow for anyone else

L.apz (Apr 26 2019 at 19:30, on Zulip):

so the last working commit was 42c4e0f378faeabd425392d4a7a7839bd7e8ac2f

Jeremy Kolb (Apr 26 2019 at 19:57, on Zulip):

@L.apz Which commit are you on?

Jeremy Kolb (Apr 26 2019 at 19:59, on Zulip):

@L.apz if you disable highlighting in the client settings does that make it go away?

Jeremy Kolb (Apr 26 2019 at 20:00, on Zulip):

A test case would be useful or a github issue.

blitzerr (Apr 27 2019 at 02:08, on Zulip):

Hello guys ! I usually use vim for rust but wanted to checkout the rust-analyzer with vs-code.
How do I get my code to compile and debug with vs-code ?

blitzerr (Apr 27 2019 at 02:08, on Zulip):

Sorry for the ignorant question and sorry if geenral thread is not the right place to post it

Laurențiu Nicola (Apr 27 2019 at 08:18, on Zulip):

@blitzerr I usually run cargo build from a console, but you can set up some asks in VS Code for that. I'm not sure how, though.

Laurențiu Nicola (Apr 27 2019 at 08:18, on Zulip):

And there's also an integrated terminal

Laurențiu Nicola (Apr 27 2019 at 08:18, on Zulip):

For debugging there was a "native debug" extension IIRC

L.apz (Apr 27 2019 at 10:09, on Zulip):

@Jeremy Kolb It's caused by my local changes. So ill need to look into it

blitzerr (Apr 27 2019 at 11:37, on Zulip):

Thanks @Laurențiu Nicola

Jeremy Fitzhardinge (Jul 02 2019 at 22:19, on Zulip):

Hi all! I'm wondering how coupled the rust-analyzer is to cargo? We're interested in Rust LSP solutions which also work in a Buck-built codebase (as well as Cargo-built). RLS is currently very coupled to Cargo, and its not clear to me how to improve this. @Igor Matuszewski had done some work which allows us to make it work at all, but its still very partial. It looks like rust-analyzer is going in the right direction overall, and so we're thinking of directing all future attention to it.

Jeremy Fitzhardinge (Jul 02 2019 at 23:01, on Zulip):

Hm, from a quick poke around it looks like the intention is to mediate project interactions via ra_project_model, which already supports Cargo and Json project types. Though it looks like this is a pretty leaky abstraction.

matklad (Jul 03 2019 at 06:23, on Zulip):

@Jeremy Fitzhardinge the only thing that really depend on Cargo is cargo check integration. Everything else is completely Cargo-independent in theory, and, in practice, we have that rust-project.json thing

matklad (Jul 03 2019 at 06:24, on Zulip):

The problem with rust-project.json is that I don't know if it is used at all, so it doesn't get as much love as it deserves :-) If you could experiment with it in a real-world use-case, I would be delighted to improve it

Jeremy Fitzhardinge (Jul 03 2019 at 06:30, on Zulip):

Yeah, we have some dedicated resources for Rust stuff over the next 6 months, and improving the LSP experience is a big part of that. I'm also interested in the design around directory structure. For example, Cargo's structure means that a directory generally only contains the source for one crate, but that's not necessarily the case with Buck.

matklad (Jul 03 2019 at 06:30, on Zulip):

Though it looks like this is a pretty leaky abstraction.

Could you elaborate on this a bit more? The intention of rust-project.json is to be more-or less independent from any particular tool, and contain only language-level information. So, it talks about crates in rust-langauge-reference sense, and not about Cargo packages.

The only bit that is specific to rust-analyzer is that roots thing. Unlike compiler, IDE can't just traverse the crate from the root module, it needs to know the directory with sources to be able to watch files and provide features for files not yet included into module hierarchy.

matklad (Jul 03 2019 at 06:30, on Zulip):

. For example, Cargo's structure means that a directory generally only contains the source for one crate, b

That's not true: example, tests, and src/bin all contain many crates in one dir

matklad (Jul 03 2019 at 06:31, on Zulip):

Oh, if you are thinking about roots thing, it's totally OK to have many crates share a single root, and it's also ok to share files between the crates by adding mod foo to two crates.

Jeremy Fitzhardinge (Jul 03 2019 at 06:35, on Zulip):

Could you elaborate on this a bit more?

Well, that was just an impression from a quick poke around. But if it were a truly hermetic abstraction, I'd expect to only see references to Cargo* in ra_project_model, whereas ra_lsp_server also has cargo_target_spec. But as I say, I haven't really dug into it yet, or thought about how it maps to Buck's model.
We also need to be careful about directory watches. One problem we had with rls was that it traversed the entire directory tree to generate a watch LSP request. This means it ended up traversing many thousands of directories and millions of files, including dirs which contain only build artifacts and caches that it shouldn't be looking at. It would be much more efficient to pass a few **/*.rs style globs.

Jeremy Fitzhardinge (Jul 03 2019 at 06:40, on Zulip):

Our environment is a large monorepo filled with mostly non-Rust, with some Rust code dotted about in it, not rooted at any particular directory. The most general "project root" would be the top-level dir of the repo, but for that to work the tooling has to avoid anything like explicit directory scans or other operations which are O(repo size). Instead, it needs to look at what files are actually being used, query the buildsystem for dependent crates and their sources, and make efficient use of the editor's dir watching capabilities.

matklad (Jul 03 2019 at 06:49, on Zulip):

Those bits of Cargo elsewhere exist to enhance functionality if Cargo is available. For example, we draw :play: alongside each #[cfg(test)] function, clicking on which launches env RUST_BACKTRACE=short cargo test --nocapture $test_name. For non-cargo based projects, the button is not shown because we wouldn't know what "launching a test" means in buck

matklad (Jul 03 2019 at 06:52, on Zulip):

File watching with exclusion is something that is generally supported, though quality of implementation is not ideal yet.

matklad (Jul 03 2019 at 06:52, on Zulip):

Specifically, each SourceRoot comes with a filter that is used by our virtual file system to skip watching something: https://github.com/rust-analyzer/rust-analyzer/blob/f15f0d1ec006c6c628fb70414d311bd85c2a0469/crates/ra_project_model/src/lib.rs#L61-L77

matklad (Jul 03 2019 at 06:53, on Zulip):

Currently, this is hard-coded to some known offenders like the /target dir, but this totally should be configurable from rust-project.json or .gitingnore

matklad (Jul 03 2019 at 06:56, on Zulip):

The bigger problem I envision is that we use the notify crate for file watching, and we historically faced quite a few hard-to-track bugs with it, both because the code on our side is non-trivial, and because something inside notify was wrong.

I think you are already using watchman service though, so I am interested in exploring it as an alternative solution for file watching. Honestly, I'd just picked watchman for file watching, but I really really want to avoid non-rust dependencies in the default configuration.

Jeremy Fitzhardinge (Jul 03 2019 at 07:12, on Zulip):

Yeah, we use watchman extensively. Part of reason for our constraints is that we use a virtual filesystem which manifests things on demand, so any kind of scanning/walking operation is super-expensive because it forces the filesystem to manifest everything. But watchman interfaces with it directly so it gets a live feed of changes as they happen very cheaply.
But my understanding was that file watching was via LSP and mediated by the editor. Am I mistaken?

Jeremy Fitzhardinge (Jul 03 2019 at 07:13, on Zulip):

As far as things like tests go, we can query for them with buck query, and run them with buck test. So that seems like something that could be abstracted over.

matklad (Jul 03 2019 at 07:16, on Zulip):

But my understanding was that file watching was via LSP and mediated by the editor. Am I mistaken?

It is possible to use VS Code file watching services, but that's optional. We deliberately don't do this, for two reasons:

If you are curious, our impl of VFS/file watching is here: https://github.com/rust-analyzer/ra_vfs

Jeremy Fitzhardinge (Jul 03 2019 at 07:30, on Zulip):

Another consideration is that we have a split editor model, where the files are on a different machine from where the UI is displayed. But I guess vscode has formalized that model now.
I don't think the JsonProject would work for our use case. I could make it work by populating the json file from buck, but it would be pretty cumbersome and wouldn't update dynamically. I think it would be better to extract the same information from buck incrementally on the fly. That is, doing a single bulk query to get that info would be a O(repo size) operation. But given a foo/bar/lib.rs, I could query buck to get the target name(s) (ie, crate(s)) that own that file, and then get their transitive deps, and so on. That would be incremental with a cost proportional to the number of files open.

matklad (Jul 03 2019 at 08:25, on Zulip):

and wouldn't update dynamically

FWIW, we currently don't support dynamic updates of the project structure at all. There's no blockers for this, just some code that needs to be written, especially in the area of virtual file system where, due to root changes, you need to carefully and transactionally move files from one root to another.

The main idea behind rust-project-json (besides the fact that it's the simplest possible thing) is to make sure that the non-cargo build system and rust-analyzer could exist as separate processes. Queering buck from rust-analyzer is certainly possible, but I'd rather avoid rust-analyzer -> buck dependency edge. Surface layers of rust-analyzer know about Cargo, yes, but that is a compromise due to the fact that Cargo is a de-facto standard.

What would you think about LSP request like setRustProjectModel instead of rust-project.json file?

matklad (Jul 03 2019 at 08:32, on Zulip):

But given a foo/bar/lib.rs, I could query buck to get the target name(s) (ie, crate(s)) that own that file, and then get their transitive deps, and so on

Hm, this might be a problem. rust-analyzer is engineered to know the project information from the start. We maintain a consistent snapshot of all rust-files in memory, and use that as a single source of truth. I think that will always be the case for at least "current crate". For dependencies, we could and should, at some point, to switch to analysis of .rlib files (or rust-analyzer analogue) instead of analysis of the source code.

I need to think about this lazy-loading more. Currently I feel like this probably belongs to the editor layer. Like, if the editor notices that the user opens a file which is not a part of the current project-model, it asks buck for deps of this file, and feeds the new project model to rust-analyzer.

Jeremy Fitzhardinge (Jul 03 2019 at 17:55, on Zulip):

Yeah, from an overall system design point of view, we really want to avoid a "needs to know everything up front" model, mostly to improve startup latency and memory use. I don't have solid thoughts about how to achieve that goal, and offloading some of it to the editor might make sense. Another option is to have some way to serialize all the interesting state so that it can be bulk downloaded - it might not completely match the actual current state, but it should be close enough that making up the delta is pretty quick.

matklad (Jul 03 2019 at 18:53, on Zulip):

A good case to think about with regards to "up front" knowledge is "workspace symbol" query: given a fuzzy query like AnHost, the IDE should find a type whose name is something like AnalysisHost. To do so, it should know the set of type that exist in the project, it can't rely only on the files opened in the editor

Jeremy Fitzhardinge (Jul 03 2019 at 20:48, on Zulip):

Yeah, that's a tricky case. You could say something like "completion only works within the scope of the transitive deps of the current file", but that might miss out on useful cases, esp if you're trying to work out what crate you should be depending on.

Jeremy Kolb (Jul 04 2019 at 14:46, on Zulip):

Another option is to have some way to serialize all the interesting state so that it can be bulk downloaded - it might not completely match the actual current state, but it should be close enough that making up the delta is pretty quick.

Would LSIF fit that goal?

Jeremy Fitzhardinge (Jul 04 2019 at 18:21, on Zulip):

Good question, I'll need to look into it. The key feature would be that it can contain the state for multiple source control revisions at once, ideally sharing all the state that's common between revisions.

Jeremy Fitzhardinge (Jul 04 2019 at 18:33, on Zulip):

Hm, doesn't look like its rich enough. The goal would be to avoid having to do a local re-index, rather than offline use.

matklad (Jul 04 2019 at 20:34, on Zulip):

LSIF will a wrong solution here, it lacks language semantics. We basically need .rmeta files

Last update: Nov 19 2019 at 17:35UTC