Stream: t-compiler

Topic: debug stage0 segfault on stage1


gnzlbg (Sep 18 2019 at 07:42, on Zulip):

Can someone help with https://github.com/rust-lang/rustc-guide/issues/443 ?

gnzlbg (Sep 18 2019 at 07:43, on Zulip):

A user is trying to debug a segfault in the stage0 compiler while building stage1

gnzlbg (Sep 18 2019 at 07:43, on Zulip):

just doing gdb --args ... with the rustc command that fails doesn't work, because environment variables that x.py sets are apparently missing

gnzlbg (Sep 18 2019 at 07:43, on Zulip):

How does one debug this ?

oli (Sep 18 2019 at 07:53, on Zulip):

ideally we'd add a --gdb command to x.py that reruns the last crashed command with gdb

oli (Sep 18 2019 at 07:53, on Zulip):

until then, I think it would suffice to dump the env vars, too

gnzlbg (Sep 18 2019 at 07:54, on Zulip):

maybe into a log file ?

gnzlbg (Sep 18 2019 at 07:54, on Zulip):

the envar dump can be huge

gnzlbg (Sep 18 2019 at 07:55, on Zulip):

we could then provide a command that uses a sub-shell to load the env vars from the log file, and then executes the command

oli (Sep 18 2019 at 07:57, on Zulip):

I just mean the necessary ones

oli (Sep 18 2019 at 07:57, on Zulip):

we know the rustc env vars needed

gnzlbg (Sep 18 2019 at 07:57, on Zulip):

Do we always know?

gnzlbg (Sep 18 2019 at 07:57, on Zulip):

ah ok

gnzlbg (Sep 18 2019 at 07:58, on Zulip):

otherwise we could only access env vars via a wrapper that caches all the ones accessed

gnzlbg (Sep 18 2019 at 07:58, on Zulip):

and dump those

gnzlbg (Sep 18 2019 at 07:59, on Zulip):

@oli or is it there a list somewhere ?

oli (Sep 18 2019 at 07:59, on Zulip):

It's just the three or four mentioned on the issue. I've added them manually many times

oli (Sep 18 2019 at 08:00, on Zulip):

basically one can test it out once to figure out the missing ones and then have the command printer print the env vars, too

gnzlbg (Sep 18 2019 at 08:03, on Zulip):

https://github.com/rust-lang/rust/blob/43a5ff4222e1f217ac14331afd59f82ec4204d12/src/bootstrap/bin/rustc.rs#L64

gnzlbg (Sep 18 2019 at 08:03, on Zulip):

So I think we could there cache all the variables accessed

gnzlbg (Sep 18 2019 at 08:32, on Zulip):

I've opened a PR for dumping the env-vars: #64574

pnkfelix (Sep 18 2019 at 09:00, on Zulip):

basically one can test it out once to figure out the missing ones and then have the command printer print the env vars, too

In my experience, you need to run it several times to learn all the missing ones, because it fails-fast on the first one that is missing. I am surprised @oli does not find this to be a big annoyance.

oli (Sep 18 2019 at 09:01, on Zulip):

I don't do this often enough to have been annoyed by it enough

gnzlbg (Sep 18 2019 at 09:06, on Zulip):

I'm not really annoyed by it, but the user that is cannot fix it, and I want them to do something for me :D

pnkfelix (Sep 18 2019 at 09:31, on Zulip):

by the way I think in the past, @nagisa has advised using x.py --on-fail as a way to deal with this (in terms of attempting to observe the state of the system at the time of failure.

gnzlbg (Sep 18 2019 at 09:31, on Zulip):

@oli so that place isn't enough to dump the env vars

pnkfelix (Sep 18 2019 at 09:31, on Zulip):

/me looks for context

gnzlbg (Sep 18 2019 at 09:31, on Zulip):

i was trying to see if i could add a test

pnkfelix (Sep 18 2019 at 09:32, on Zulip):

ah yes, context is #38686

gnzlbg (Sep 18 2019 at 09:33, on Zulip):

So I decided to make typeck abort, and see if I could see the env vars:

aused by:
  process didn't exit successfully: `/rust/build/bootstrap/debug/rustc --edition=2018 --crate-name core src/libcore/lib.rs   ...` (signal: 6, SIGABRT: process abort signal)
command did not execute successfully: "rust/build/x86_64-apple-darwin/stage0/bin/cargo" "build" ..."
expected success, got: exit code: 101
failed to run: rust/build/bootstrap/debug/bootstrap test
Build completed unsuccessfully in 0:00:31
gnzlbg (Sep 18 2019 at 09:33, on Zulip):

So apparently rustc is called by cargo here, and not by the bootstrap script

gnzlbg (Sep 18 2019 at 09:33, on Zulip):

When is it called by one or the other ?

gnzlbg (Sep 18 2019 at 09:34, on Zulip):

That error reporting comes from cargo, so I'm not sure how to fix this

gnzlbg (Sep 18 2019 at 09:34, on Zulip):

IIUC cargo would need to be able to query from rustc the environment variables that rustc reads

gnzlbg (Sep 18 2019 at 09:35, on Zulip):

Right now if the command cargo launches errors, cargo prints the command and the error code, but that's it

pnkfelix (Sep 18 2019 at 09:35, on Zulip):

@gnzlbg for your particular scenario/user, would the --on-fail approach work? I don't know whether that is agnostic to whether it is a cargo failure or a rustc one, but I would have assumed it is

gnzlbg (Sep 18 2019 at 09:36, on Zulip):

I'd have to look at what --on-fail does

pnkfelix (Sep 18 2019 at 09:36, on Zulip):

it is described in PR #39888

pnkfelix (Sep 18 2019 at 09:36, on Zulip):

it sounds like you can do x.py --on-fail=bash, and if a command fails, it will fire up an interactive shell

gnzlbg (Sep 18 2019 at 09:37, on Zulip):

sure, but does it let you attach a debugger ?

gnzlbg (Sep 18 2019 at 09:37, on Zulip):

ah the interactive shell allows you to launch a debugger with the same command preserving the environment ?

pnkfelix (Sep 18 2019 at 09:37, on Zulip):

exactly.

pnkfelix (Sep 18 2019 at 09:38, on Zulip):

Now, to be honest, I've never actually deployed this workflow myself ...

pnkfelix (Sep 18 2019 at 09:38, on Zulip):

but that doesn't mean it won't work for you

pnkfelix (Sep 18 2019 at 09:39, on Zulip):
gnzlbg (Sep 18 2019 at 09:39, on Zulip):

We should document that in some section about how to debug rustc

pnkfelix (Sep 18 2019 at 09:41, on Zulip):

indeed. #40902 had noted this in a long laundry list

pnkfelix (Sep 18 2019 at 09:41, on Zulip):

but then it was closed because it was "not clear that it was usefully tracking anything anymore"

gnzlbg (Sep 18 2019 at 09:43, on Zulip):

https://github.com/rust-lang/rustc-guide/issues/443#issuecomment-532608428

gnzlbg (Sep 18 2019 at 09:44, on Zulip):

The rustc guide doesn't have any section about how to debug a failure, so I think mentioning this would fit there.

pnkfelix (Sep 18 2019 at 09:44, on Zulip):

Just filed rustc-guide#448

pnkfelix (Sep 18 2019 at 09:44, on Zulip):

lets go add that as an issue linkifier

pnkfelix (Sep 18 2019 at 09:45, on Zulip):

rustc-guide#448

pnkfelix (Sep 18 2019 at 09:48, on Zulip):

hmm anyway --on-fail may not work, due to #47645

pnkfelix (Sep 18 2019 at 09:49, on Zulip):

(and that in turn may be due to cargo playing games with stdin/stdout.)

pnkfelix (Sep 18 2019 at 09:52, on Zulip):

but --on-fail=env should work, even in the presence of #47645

gnzlbg (Sep 18 2019 at 10:08, on Zulip):

thanks

Last update: Nov 22 2019 at 04:50UTC