According to some memory profiles I took a while ago, attributes stored in
ItemTrees might take up significant space.
I suspect that this is in part due to token trees not using interning or well-known symbols (which would eliminate ~all copies of builtin attribute names).
What would be the right approach to improving this? It seems to me that using
hir_def::Name instead of
tt::Ident isn't semantically right. rustc just has
Symbols, which are interned strings without much semantic attached to them.
Also I'm wondering if we could do something about all those duplicated documentation strings here
AttrInput alone saves over 10 MB already, probably because there are a lot of similar-enough attributes like
One fun thing to try would be to replace
pub(crate) path: Interned<ModPath>, pub(crate) input: Option<AttrInput>,
pub(crate) raw_text: String,
and see how that affects memory usage. I don't have a good intuition about the difference in memory usage between a string and a token tree. I suspect strings might be significantly more memory-efficient.
@Jonas Schievink [he/him] what are you using for heap-profiling, mastiff? Do you think it makes sense to add an xtask for that? Or maybe some docs to the dev/README? I'd love to take a look at profiles myself, but I want to re-use your hard-earned knowledge if possible :)
new hack to quickly estimate the impact of sizer of things
Item Collection: 1.48s, 15ginstr, 73mb to
Item Collection: 1.45s, 15ginstr, 78mb for rowan, so RawAttrs themselves are 5-10% of total memory
which does sound like a lot
Jonas Schievink [he/him] what are you using for heap-profiling, mastiff? Do you think it makes sense to add an xtask for that? Or maybe some docs to the dev/README? I'd love to take a look at profiles myself, but I want to re-use your hard-earned knowledge if possible :)
nowadays I mostly use heaptrack since it's a lot faster than valgrind