Stream: t-compiler/help

Topic: build pipelining outside of cargo


tmandry (Jan 29 2020 at 23:45, on Zulip):

I'm experimenting with build pipelining in the fuchsia build

tmandry (Jan 29 2020 at 23:46, on Zulip):

is --emit=metadata output sufficient for this, or is there another magic flag that makes the rmeta "more complete"?

tmandry (Jan 29 2020 at 23:46, on Zulip):

I'm getting unexpected errors now that I produce rmeta files

simulacrum (Jan 29 2020 at 23:49, on Zulip):

I believe it should be, but I'm not sure that the compiler expects that exactly

simulacrum (Jan 29 2020 at 23:49, on Zulip):

it might help to tell the compiler to emit the "okay metadata is done now" message

simulacrum (Jan 29 2020 at 23:50, on Zulip):

(I forget how to do that, but either grepping Cargo source for pipelining or trying a cargo build should work). IIRC, it's something like --json artifact-notifications, but I could be wrong.

tmandry (Jan 29 2020 at 23:50, on Zulip):

so, the main thing I'm doing differently is I'm not emitting the rlib in the same invocation

tmandry (Jan 29 2020 at 23:50, on Zulip):

I suspect this is causing the rmeta file to be incomplete somehow

simulacrum (Jan 30 2020 at 00:00, on Zulip):

hm, perhaps, but I would expect no

simulacrum (Jan 30 2020 at 00:01, on Zulip):

(in theory you can check)

simulacrum (Jan 30 2020 at 00:02, on Zulip):

i.e., an rlib is I believe an ar archive so you can pull out the rmeta and diff them

simulacrum (Jan 30 2020 at 00:02, on Zulip):

What errors are you getting?

tmandry (Jan 30 2020 at 01:34, on Zulip):

@simulacrum sorry, got sidetracked

tmandry (Jan 30 2020 at 01:35, on Zulip):

there are multiple issues, and one of them that is preventing me from making progress

tmandry (Jan 30 2020 at 01:35, on Zulip):

is that --emit=metadata=<path> is not respected

tmandry (Jan 30 2020 at 01:36, on Zulip):

(the other issues I know of are my fault, but they muddy the waters on the original question)

tmandry (Jan 30 2020 at 01:36, on Zulip):

anyway, --emit=metadata seems hardcoded to emit ./<crate_name>.rmeta

tmandry (Jan 30 2020 at 01:37, on Zulip):

there might be ways of working around this on my end, but it's probably not worth it for an experiment

simulacrum (Jan 30 2020 at 01:47, on Zulip):

oh, I don't think that works for anything?

simulacrum (Jan 30 2020 at 01:47, on Zulip):

i.e., --emit=metdata=path is not a supported syntax?

simulacrum (Jan 30 2020 at 01:47, on Zulip):

maybe -o does, but I don't think

simulacrum (Jan 30 2020 at 01:53, on Zulip):

but if the hardcoding is a concern, then yeah, that might not be avoidable, not sure

nagisa (Jan 30 2020 at 13:11, on Zulip):

i.e., --emit=metdata=path is not a supported syntax?

Yeah, I don’t believe this is a syntax we support, although maybe something has changed? Definitely looks unusual.

nagisa (Jan 30 2020 at 13:12, on Zulip):

For pipelining I believe you also need to consume the json output as that has signals that indicate when something gets done

simulacrum (Jan 30 2020 at 13:24, on Zulip):

(I think @tmandry is trying "precompiled" pipelining, i.e., that the rmetas are already available way ahead of time)

simulacrum (Jan 30 2020 at 13:24, on Zulip):

I have wondered if we could make cargo build kick off all dependencies at the same time if you already had a cargo check previously completed (common case, realistically)

tmandry (Jan 30 2020 at 20:11, on Zulip):

we support it for dep-info: --emit=def-info=<path>

tmandry (Jan 30 2020 at 20:11, on Zulip):

and when I have that, plus --emit=link, plus -o, I get a warning

tmandry (Jan 30 2020 at 21:07, on Zulip):

okay I got it running, I think it had to do with the order of args

tmandry (Jan 30 2020 at 21:08, on Zulip):

hitting an ICE now :grimacing:

tmandry (Jan 30 2020 at 21:09, on Zulip):

(I think tmandry is trying "precompiled" pipelining, i.e., that the rmetas are already available way ahead of time)

I'm actually experimenting with doing the work twice, in a large build, in an environment with high parallelism

tmandry (Jan 30 2020 at 21:09, on Zulip):

i.e. spawning off --emit=metadata and --emit=link jobs simultaneously

tmandry (Jan 30 2020 at 21:09, on Zulip):

no idea if it will be better, though

simulacrum (Jan 30 2020 at 21:27, on Zulip):

I would expect that to be strictly worse, but who knows :)

Eric Huss (Feb 03 2020 at 16:14, on Zulip):

--emit THING=PATH is definitely a syntax that is supported. It looks like metadata is the only one that doesn't respect it, and I would say that is a bug (or at least it should be a warning/error if you specify it).

As for the original question, I'm pretty sure you need to include link to get a compatible rmeta file. Maybe nnethercote knows why?

tmandry (Feb 07 2020 at 21:52, on Zulip):

From the ICE I'm seeing, it seems like the rmeta does not include MIR for generic functions

tmandry (Feb 07 2020 at 21:52, on Zulip):

cc @Nicholas Nethercote

Last update: Feb 25 2020 at 03:45UTC