Stream: t-compiler/rust-analyzer

Topic: Memory usage issue


Jonas Schievink [he/him] (Oct 22 2020 at 12:14, on Zulip):

I just ran the memory usage command and it only freed up something like 400 MB. r-a is still consuming almost 2 GB of RAM after that. Has anyone seen this happen?

matklad (Oct 22 2020 at 12:17, on Zulip):

No

matklad (Oct 22 2020 at 12:17, on Zulip):

I think I saw the report that recent release increased memory usage for someone from 2GB to 4GB

matklad (Oct 22 2020 at 12:18, on Zulip):

Could it be that freeing is run concurrently with prefetching, and it evens out to no-change?

matklad (Oct 22 2020 at 12:18, on Zulip):

Probably makes sense to add a config flag to disable priming, it would be useulf in the futur to debug all kinds of issues

Jonas Schievink [he/him] (Oct 22 2020 at 12:41, on Zulip):

Priming only happens once on startup

matklad (Oct 22 2020 at 12:42, on Zulip):

Hm, maybe&

matklad (Oct 22 2020 at 12:42, on Zulip):

I would think the opposite

matklad (Oct 22 2020 at 12:43, on Zulip):

We trigger it from update_file_notifications_on_threadpool

matklad (Oct 22 2020 at 12:43, on Zulip):

so, after every change

Jonas Schievink [he/him] (Oct 22 2020 at 12:48, on Zulip):

Oh, right, nevermind me

Jonas Schievink [he/him] (Oct 26 2020 at 01:38, on Zulip):

6.7 GB... this might be a problem

Jonas Schievink [he/him] (Oct 26 2020 at 13:02, on Zulip):

Jonas Schievink said:

I just ran the memory usage command and it only freed up something like 400 MB. r-a is still consuming almost 2 GB of RAM after that. Has anyone seen this happen?

I've checked this again and it doesn't seem to be caused by cache priming. Computing memory usage twice in a row correctly yields 0mb across the board for the second run. RSS stays high, so that might be a different issue, or just weird allocator behavior.

When modifying a file, "indexing..." starts, RSS immediately goes down by a lot, and then slowly grows back to where it was before.

matklad (Oct 26 2020 at 13:04, on Zulip):

:thinking: have zero idea about what might be the issue. The best I can suggest is dropping things one by one and noting when it floats :)

Jonas Schievink [he/him] (Oct 26 2020 at 13:06, on Zulip):

I'll try with mimalloc

Laurențiu (Oct 26 2020 at 13:07, on Zulip):

Does massif work?

Jonas Schievink [he/him] (Oct 26 2020 at 13:07, on Zulip):

Yes, but it's excruciatingly slow

Jonas Schievink [he/him] (Oct 26 2020 at 13:08, on Zulip):

so i'd rather avoid it

Jonas Schievink [he/him] (Oct 26 2020 at 13:12, on Zulip):

mimalloc behaves the same way, but is less memory efficient

Jonas Schievink [he/him] (Oct 26 2020 at 13:17, on Zulip):
Inference: 23.91s, 115428mi
Total: 36.84s, 206986mi

???

matklad (Oct 26 2020 at 13:18, on Zulip):

TBH, I don't remmeber if those are diffs or totals

Jonas Schievink [he/him] (Oct 26 2020 at 13:18, on Zulip):

Oh, that's million instructions, not megabytes of RAM

matklad (Oct 26 2020 at 13:18, on Zulip):

and yes, that as well

Laurențiu (Oct 26 2020 at 13:19, on Zulip):

We still use some 1.2-1.3 GB on RA itself, that's maybe a bit much?

Jonas Schievink [he/him] (Oct 26 2020 at 14:06, on Zulip):
Inference: 41.06ms, 96mi, -57kb
Total: 30.17s, 100352mi, 1225mb

-57kb????

Jonas Schievink [he/him] (Oct 26 2020 at 14:09, on Zulip):

Oh, it's just a diff, that seems confusing

Jonas Schievink [he/him] (Oct 26 2020 at 14:09, on Zulip):

Looks like we actually use around 1.4G

Last update: Jul 28 2021 at 05:00UTC