Stream: t-compiler/help

Topic: rebuild llvm


davidtwco (Apr 29 2019 at 21:16, on Zulip):

I'm investigating a stack overflow in LLVM that I see on rust-lang/rust master for a specific test when debuginfo is enabled. How can I get LLVM to be rebuilt with x.py?

oli (Apr 29 2019 at 21:20, on Zulip):

I think there's some timestamp/lockfile in build/x86_...../llvm that you can remove

davidtwco (Apr 29 2019 at 21:21, on Zulip):

rm build/x86_64-unknown-linux-gnu/llvm/llvm-finished-building did the trick, thanks @oli.

pnkfelix (Jun 14 2019 at 08:47, on Zulip):

is there an option to x.py to expose this? We may want to add one. (That or I guess people can be like me and resort to rm -rf'ing build/ after tearing their hair out for a while.)

centril (Jun 14 2019 at 08:48, on Zulip):

(That or I guess people can be like me and resort to rm -rf'ing build/ after tearing their hair out for a while.)

I do this also and feel very sad about it every time...

oli (Jun 14 2019 at 08:50, on Zulip):

how exposed do you want it? have ./x.py clean always clear the llvm-finished-building file, but also have a flag to not do that?

centril (Jun 14 2019 at 08:50, on Zulip):

@oli ./x.py clean --nuclear-option

oli (Jun 14 2019 at 08:50, on Zulip):

the nuclear option is to rm build

centril (Jun 14 2019 at 08:51, on Zulip):

./x.py clean --llvm then

pnkfelix (Jun 14 2019 at 08:51, on Zulip):

wait, you're saying x.py clean does clear the llvm-finished-building file?

oli (Jun 14 2019 at 08:51, on Zulip):

no

oli (Jun 14 2019 at 08:51, on Zulip):

it could

pnkfelix (Jun 14 2019 at 08:51, on Zulip):

ah I see

pnkfelix (Jun 14 2019 at 08:52, on Zulip):

no I think we want to preserve the current behavior

oli (Jun 14 2019 at 08:52, on Zulip):

like I'm wondering if doing so by default would make more sense

pnkfelix (Jun 14 2019 at 08:52, on Zulip):

I think x.py clean --and-clean-llvm would be good

oli (Jun 14 2019 at 08:52, on Zulip):

ok, then I'll have ./x.py clean also emit a message about llvm not getting cleaned

oli (Jun 14 2019 at 08:52, on Zulip):

together with a suggestion for --and-clean-llvm

pnkfelix (Jun 14 2019 at 08:52, on Zulip):

(or at least, that would generalize to any other submodules that similarly leave products around post-clean)

centril (Jun 14 2019 at 08:53, on Zulip):

@pnkfelix clean --and-clean-llvm seems awfully wordy and repetitive

centril (Jun 14 2019 at 08:53, on Zulip):

is that on purpose? :P

pnkfelix (Jun 14 2019 at 08:53, on Zulip):

yeah but x.py clean --llvm is ambiguous to me

pnkfelix (Jun 14 2019 at 08:53, on Zulip):

does that only clean llvm?

centril (Jun 14 2019 at 08:54, on Zulip):

@pnkfelix fair; clean --and-llvm then

pnkfelix (Jun 14 2019 at 08:55, on Zulip):

/me also thinks redundancy ain't a bad thing when it comes to a destructive options like cleaning.

pnkfelix (Jun 14 2019 at 08:55, on Zulip):

like, I have my driver scripts set up to wait for five seconds if I issue the clean command

centril (Jun 14 2019 at 08:55, on Zulip):

o.O

pnkfelix (Jun 14 2019 at 08:55, on Zulip):

sure, waiting five seconds is annoying, right?

pnkfelix (Jun 14 2019 at 08:55, on Zulip):

but you know what's more annoying? accidentally issuing a clean command

centril (Jun 14 2019 at 08:55, on Zulip):

@pnkfelix yeah; I have a more YOLO approach to cleaning

pnkfelix (Jun 14 2019 at 08:56, on Zulip):

admittedly part of this workflow stems from my use of Emacs compilation mode, where just hitting g will repeat the last compile-command that was used in the buffer

pnkfelix (Jun 14 2019 at 08:57, on Zulip):

so that's a case where I want a reminder "oh, used a cleaning build the previous time, didn't mean to do that again"

centril (Jun 14 2019 at 08:57, on Zulip):

@pnkfelix I don't think I've ever done an accidental clean but I do frequently do other silly things like touch a file in the compiler and incremental is a bit daft so it causes a rebuild of the compiler...

pnkfelix (Jun 14 2019 at 08:57, on Zulip):

We definitely need to make incremental smarter

pnkfelix (Jun 14 2019 at 08:57, on Zulip):

and/or make x.py smarter

pnkfelix (Jun 14 2019 at 08:58, on Zulip):

I think making incremental smarter would help with a lot of existing problems we have

pnkfelix (Jun 14 2019 at 08:58, on Zulip):

(but we also need to improve the test suite for incremental; do more fuzzing and/or randomized test generation, in tandem with trying to ramp up effort on incremental.)

centril (Jun 14 2019 at 09:00, on Zulip):

@pnkfelix mmh; I've proposed to eddyb that we should add a (morally dubious) logic to incremental whereby we specifically hash post lexing so that whitespace changes, comment changes, and random saves don't cause rebuilds.

pnkfelix (Jun 14 2019 at 09:01, on Zulip):

and you just suffer with incorrect spans?

centril (Jun 14 2019 at 09:01, on Zulip):

@pnkfelix kinda :slight_smile:

pnkfelix (Jun 14 2019 at 09:01, on Zulip):

I was thinking more: Do rebuilds (so that span's get updated), but skip stuff like type-checking if the function only had changes of the form you mentioned

centril (Jun 14 2019 at 09:02, on Zulip):

well @eddyb has a principled solution for that I believe

centril (Jun 14 2019 at 09:02, on Zulip):

based on ranges or something

varkor (Jun 14 2019 at 10:03, on Zulip):

I've definitely accidentally cleaned

varkor (Jun 14 2019 at 10:04, on Zulip):

maybe adding a confirm y/n option would be helpful

varkor (Jun 14 2019 at 10:04, on Zulip):

it's not used frequently enough that that would be annoying, I think

varkor (Jun 14 2019 at 10:10, on Zulip):

x.py clean --and=llvm could be more extensible, though I'm not sure what else we might want to clean

eddyb (Jun 14 2019 at 13:51, on Zulip):

hang on

eddyb (Jun 14 2019 at 13:52, on Zulip):

@pnkfelix @centril just to check, regarding removing the llvm-finished-building, I think it only triggers a rebuild if one is actually needed, and only exists because cmake is slow to do that check. is that so?

eddyb (Jun 14 2019 at 13:53, on Zulip):

@centril the principled solution is either/both of 1. relative spans (the AST contains lengths instead of absolute positions) 2. token ranges instead of character ranges

eddyb (Jun 14 2019 at 13:54, on Zulip):

if only I had the time to do all this right away...

varkor (Jun 14 2019 at 13:56, on Zulip):

considering compile times are something we're trying to improve this year, it seems that it would be a good idea to prioritise things like this

varkor (Jun 14 2019 at 13:56, on Zulip):

(as a team, that is)

eddyb (Jun 14 2019 at 14:00, on Zulip):

I guess?

pnkfelix (Jun 14 2019 at 14:06, on Zulip):

okay well there might be enough sketched out in @eddyb 's note above that I could write up a few sentences proposal for the next rustc planning meeting

eddyb (Jun 14 2019 at 14:48, on Zulip):

@pnkfelix funny you say that...

eddyb (Jun 14 2019 at 14:49, on Zulip):

@pnkfelix https://hackmd.io/zrZhb94HS6KxW3sguwXNqA#B-Mapping-back-to-originsource-can-be-impreciselossy

mark-i-m (Jun 16 2019 at 00:00, on Zulip):

Perhaps we could just change the user's code ourselves in some non-trivial way? Then we know it should be rebuilt, and that would simplify incremental tremendously...

Zoxc (Jun 16 2019 at 00:05, on Zulip):

I want to fix up my build system so I won't need to do x.py clean or accidentally break build cache by switching between x.py build and x.py check

Last update: Nov 11 2019 at 23:05UTC