Stream: t-compiler

Topic: weekly meeting 2019-04-11 #54818


pnkfelix (Apr 11 2019 at 13:13, on Zulip):

Hi @T-compiler/meeting ; our weekly meeting, held in this topic area, will start in about 47 minutes.

pnkfelix (Apr 11 2019 at 13:14, on Zulip):

In the meantime, I will be doing pre-triage in a parallel topic

pnkfelix (Apr 11 2019 at 13:34, on Zulip):

:question: re: "regression: conflicting trait impls because Box<dyn Fn*> now implements Fn*." #59750 : was this expected fallout from some other recent change? And is this actually a T-libs problem rather than a T-compiler one?

pnkfelix (Apr 11 2019 at 13:40, on Zulip):

:construction_worker: seeking help on resolving "[firefox] error: relocation refers to local symbol "" [12], which is defined in a discarded section" #59652

centril (Apr 11 2019 at 13:51, on Zulip):

@pnkfelix I think Box<dyn Fn*> is a T-Lang/T-Libs issue; it happened due to standard coherence issues due the introduction of impls

centril (Apr 11 2019 at 13:51, on Zulip):

the teams that participated in FCP were T-Lang & T-Libs

pnkfelix (Apr 11 2019 at 13:57, on Zulip):

okay I'll go find the associated issues and link them and then close #59750

pnkfelix (Apr 11 2019 at 13:57, on Zulip):

:construction_worker: seeking help on resolving "Compiler panic with generic-typed nested closures" #59494

centril (Apr 11 2019 at 13:58, on Zulip):

@pnkfelix regression arose from: https://github.com/rust-lang/rust/pull/59500, FCP in https://github.com/rust-lang/rust/pull/55431.

nikomatsakis (Apr 11 2019 at 14:03, on Zulip):

Hi all =)

pnkfelix (Apr 11 2019 at 14:03, on Zulip):

hi @T-compiler/meeting

pnkfelix (Apr 11 2019 at 14:03, on Zulip):

you can see the results of pre-triage above

pnkfelix (Apr 11 2019 at 14:03, on Zulip):

the messages with a constructor worker ( :construction_worker:) are seeking volunteers

pnkfelix (Apr 11 2019 at 14:04, on Zulip):

I didn't finish all the pre-triage I wanted to

pnkfelix (Apr 11 2019 at 14:04, on Zulip):

but we can dive into what's left here

pnkfelix (Apr 11 2019 at 14:04, on Zulip):

lo and behold, there are zero beta-nominations. (Probably because we just cut the beta)

nikomatsakis (Apr 11 2019 at 14:04, on Zulip):

:construction_worker: I wanted to mention something. I think we should start posting "minimal minutes" from this meeting to an internals thread (and the compiler-team repository). I think they would have the form of this:

# Volunters wanted

# Backport decisions

# Working group sync

for each working group, link to the zulip comment where the sync started, and maybe include a few details, but maybe not even that.

Any volunteers to do that for this week? =)

nikomatsakis (Apr 11 2019 at 14:05, on Zulip):

I'm partially looking for a way to get more visibility for the volunteers wanted issues

pnkfelix (Apr 11 2019 at 14:05, on Zulip):

I'd offer to do it but I almost always have to leave immediately after these meetings end

Wesley Wiser (Apr 11 2019 at 14:05, on Zulip):

I can do it

oli (Apr 11 2019 at 14:05, on Zulip):

volunteers wanted seems like something that we should autogen from github tags

nikomatsakis (Apr 11 2019 at 14:05, on Zulip):

Thanks @Wesley Wiser! :hero:

oli (Apr 11 2019 at 14:06, on Zulip):

same with backport decisions

nikomatsakis (Apr 11 2019 at 14:06, on Zulip):

I think it's fine to have some github tags but I wouldn't want to block on having minutes for want of tooling

nikomatsakis (Apr 11 2019 at 14:07, on Zulip):

I also think we need to start looking into how to more widely advertise said minutes, but let's get some first

nikomatsakis (Apr 11 2019 at 14:07, on Zulip):

(I mean, probably just having a little blog and maybe a twitter account would suffice)

pnkfelix (Apr 11 2019 at 14:10, on Zulip):

so it seems like the set of triage topics is remarkably short today

pnkfelix (Apr 11 2019 at 14:11, on Zulip):

just two nominated issues, one of which is one of the "volunteers wanted" ones from above, #59652

nikomatsakis (Apr 11 2019 at 14:11, on Zulip):

there's an issue that's been on my radar for a bit

pnkfelix (Apr 11 2019 at 14:11, on Zulip):

and the other nominated issue is the PR about the performance impact of local interners, PR #57214

pnkfelix (Apr 11 2019 at 14:12, on Zulip):

where we can probably clear that nomination label

pnkfelix (Apr 11 2019 at 14:12, on Zulip):

since I'm pretty sure we've already discussed this a couple of times

pnkfelix (Apr 11 2019 at 14:12, on Zulip):

it is nonetheless waiting-on-team, in a sense

pnkfelix (Apr 11 2019 at 14:12, on Zulip):

(its marked do-not-merge, but we owe @Zoxc a decision about whether to move forward here.)

nikomatsakis (Apr 11 2019 at 14:12, on Zulip):

cargo check fails only in incremental mode #58291, which I think has been affecting the #t-compiler/wg-rls-2.0 folks (cc @matklad, @Florian Diebold) -- @mw have you seen this one?

mw (Apr 11 2019 at 14:13, on Zulip):

nope, haven't seen it yet

nikomatsakis (Apr 11 2019 at 14:13, on Zulip):

just two nominated issues, one of which is one of the "volunteers wanted" ones from above, #59652

so this seems semi-urgent, no?

pnkfelix (Apr 11 2019 at 14:13, on Zulip):

hmm unfortunately we probably need to start dealing with unprioiritzed/unnominated bugs like this one

pnkfelix (Apr 11 2019 at 14:13, on Zulip):

you're asking if #59652 seems urgent?

nikomatsakis (Apr 11 2019 at 14:13, on Zulip):

I am

pnkfelix (Apr 11 2019 at 14:14, on Zulip):

yes I think it is urgent.

nikomatsakis (Apr 11 2019 at 14:14, on Zulip):

Is there any reason not to just revert the offending PR? I guess because it seems so strange to imagine it having an affect?

pnkfelix (Apr 11 2019 at 14:14, on Zulip):

I would like to perhaps also get an isolated test case

pnkfelix (Apr 11 2019 at 14:15, on Zulip):

which vorner may be able to supply

nikomatsakis (Apr 11 2019 at 14:15, on Zulip):

(Or did we decide that this is not in fact the fault of #59401?)

nikomatsakis (Apr 11 2019 at 14:15, on Zulip):

ok

nikomatsakis (Apr 11 2019 at 14:15, on Zulip):

I guess we can wait to revert as a last resort

pnkfelix (Apr 11 2019 at 14:15, on Zulip):

I'll assign myself to follow up on this, I guess

pnkfelix (Apr 11 2019 at 14:15, on Zulip):

because ... I do such a stellar job at that on all the other issues I assign myself to ...

pnkfelix (Apr 11 2019 at 14:15, on Zulip):

</sarcasm>

pnkfelix (Apr 11 2019 at 14:17, on Zulip):

I'll un-nominate it

pnkfelix (Apr 11 2019 at 14:17, on Zulip):

regarding PR #57214

pnkfelix (Apr 11 2019 at 14:18, on Zulip):

should we consider making a WG-rustc-memory-usage ?

nikomatsakis (Apr 11 2019 at 14:18, on Zulip):

I am not inclined to do so unless we have people who really want to put energy into it

nikomatsakis (Apr 11 2019 at 14:18, on Zulip):

But I think it'd be good to have on a lsit of "things we would like to do"

nikomatsakis (Apr 11 2019 at 14:19, on Zulip):

i.e., not quite "volunteers wanted" but maybe "wg leaders wanted" :)

pnkfelix (Apr 11 2019 at 14:19, on Zulip):

okay

nikomatsakis (Apr 11 2019 at 14:19, on Zulip):

I guess it depends on how much @nnethercote thinks they would be able to do

nikomatsakis (Apr 11 2019 at 14:19, on Zulip):

(ps, @nnethercote, are you monitoring zulip now? That'd be awesome =)

pnkfelix (Apr 11 2019 at 14:19, on Zulip):

also, @nagisa and @nikomatsakis are the only ones with unchecked boxes on PR #57214

nikomatsakis (Apr 11 2019 at 14:19, on Zulip):

I thought I checked my box

nikomatsakis (Apr 11 2019 at 14:20, on Zulip):

whoops:)

nikomatsakis (Apr 11 2019 at 14:20, on Zulip):

/me checks

pnkfelix (Apr 11 2019 at 14:20, on Zulip):

eddyb also claimed their vote did not register

pnkfelix (Apr 11 2019 at 14:22, on Zulip):

okay well

pnkfelix (Apr 11 2019 at 14:22, on Zulip):

that might be it for triage. I mean, there were other P-high issues I didn't get to in pre-triage

pnkfelix (Apr 11 2019 at 14:22, on Zulip):

but I think we're all better off if we just jump to WG-checkins?

davidtwco (Apr 11 2019 at 14:23, on Zulip):

Is it the triage meeting we normally do small announcements in? (I don't have any myself, but some folks might)

nagisa (Apr 11 2019 at 14:23, on Zulip):

Call for somebody to benchmark runtime perf of https://github.com/rust-lang/rust/pull/59546

pnkfelix (Apr 11 2019 at 14:24, on Zulip):

oh I forgot to ask for any announcemnets at the start

pnkfelix (Apr 11 2019 at 14:24, on Zulip):

so yeah, lets maybe allow people to chime in with anything they want for a few minutes

nikomatsakis (Apr 11 2019 at 14:25, on Zulip):

Call for somebody to benchmark runtime perf of https://github.com/rust-lang/rust/pull/59546

cc @Adam Perry -- any chance we can run lolbench on this?

nikomatsakis (Apr 11 2019 at 14:25, on Zulip):

:horn: Announcement :horn:

The steering meeting is tomorrow. The theme is likely to be a proposal for a team design meeting

nikomatsakis (Apr 11 2019 at 14:26, on Zulip):

Which imo is also relevant to #57214 and #59546

oli (Apr 11 2019 at 14:26, on Zulip):

It's likely I'm not able to participate

nikomatsakis (Apr 11 2019 at 14:26, on Zulip):

In steering meeting you mean?

oli (Apr 11 2019 at 14:26, on Zulip):

yes

nikomatsakis (Apr 11 2019 at 14:27, on Zulip):

If you have thoughts you'd like to leave, please leave them in the pre-steering meeting topic then

nikomatsakis (Apr 11 2019 at 14:27, on Zulip):

And I'll try to echo them

pnkfelix (Apr 11 2019 at 14:29, on Zulip):

okay well while we wait another minute or so for any other announcements

pnkfelix (Apr 11 2019 at 14:30, on Zulip):

the check-in schedule lists today has being for wg-mir-opt and wg-pipelining

pnkfelix (Apr 11 2019 at 14:30, on Zulip):

(wasn't wg-pipelining just formed recently?)

pnkfelix (Apr 11 2019 at 14:31, on Zulip):

anyway, @oli, are you representing wg-mir-opt today?

oli (Apr 11 2019 at 14:31, on Zulip):

jup

pnkfelix (Apr 11 2019 at 14:32, on Zulip):

okay. and is either @Alex Crichton or @nnethercote around to represent wg-pipelining ?

pnkfelix (Apr 11 2019 at 14:32, on Zulip):

(I'm going to assume "no" for now, but I figured I'd ask)

pnkfelix (Apr 11 2019 at 14:32, on Zulip):

in the meantime: @oli , why don't you go ahead and let us know whats up in mir-optimization land.

nikomatsakis (Apr 11 2019 at 14:33, on Zulip):

okay. and is either Alex Crichton or nnethercote around to represent wg-pipelining ?

I can if needed

pnkfelix (Apr 11 2019 at 14:34, on Zulip):

@nikomatsakis that'd be great; thanks

Alex Crichton (Apr 11 2019 at 14:35, on Zulip):

I'm here now!

Alex Crichton (Apr 11 2019 at 14:35, on Zulip):

sorry for the delay

oli (Apr 11 2019 at 14:35, on Zulip):

So, our big next goal is making Place not recursive

oli (Apr 11 2019 at 14:36, on Zulip):

we've been idle on that for a few weeks, but @Santiago Pastorino has picked it up again and is proceeding very fast

oli (Apr 11 2019 at 14:36, on Zulip):

We will regress max-rss in an intermediate step, but the final structure will use less memory in total, be easier to work with and probably even faster

Santiago Pastorino (Apr 11 2019 at 14:37, on Zulip):

:+1:

Santiago Pastorino (Apr 11 2019 at 14:37, on Zulip):

sorry about the delay, Rust Latam stuff got all my time :)

oli (Apr 11 2019 at 14:37, on Zulip):

Once we have that, there are further refactorings of Place (like getting rid of the Deref projection), that are supposed to make borrow checking simpler. We have quite a list of things to do left from the all-hands, but are picking our battles one by one

eddyb (Apr 11 2019 at 14:38, on Zulip):

(I'm here btw)

Santiago Pastorino (Apr 11 2019 at 14:38, on Zulip):

thanks @oli for your help with the Place work :heart:

oli (Apr 11 2019 at 14:38, on Zulip):

That's mostly it from us, no big things, but progress

pnkfelix (Apr 11 2019 at 14:39, on Zulip):

for reference, the relevant Place bug was #52708, right?

csmoe (Apr 11 2019 at 14:40, on Zulip):

for reference, the relevant Place bug was #52708, right?

yes

pnkfelix (Apr 11 2019 at 14:40, on Zulip):

out of curiousity, you said the final structure will use less memory in total. nnethercote did voice concern in a recent comment about Place itself increasing from 16- to 24-bytes

pnkfelix (Apr 11 2019 at 14:40, on Zulip):

and that causing Statement to grow

pnkfelix (Apr 11 2019 at 14:40, on Zulip):

@oli ^

nikomatsakis (Apr 11 2019 at 14:41, on Zulip):

That's mostly it from us, no big things, but progress

ps, this sounds awesome y'all

pnkfelix (Apr 11 2019 at 14:41, on Zulip):

would you mind either here or in the issue, making a note explaining why the overall memory usage should not increase?

oli (Apr 11 2019 at 14:41, on Zulip):

yes, we'll get back down to 16 bits, but the interned memory will be less

oli (Apr 11 2019 at 14:42, on Zulip):

so no regression on Statement, but an improvement in interned memory

pnkfelix (Apr 11 2019 at 14:42, on Zulip):

surely not 16-bits :wink:

pnkfelix (Apr 11 2019 at 14:42, on Zulip):

are you going to decrease it from 24-bytes down to 16-bytes by using indices rather than pointers?

pnkfelix (Apr 11 2019 at 14:44, on Zulip):

well, we don't have to have this discussion synchronously here

oli (Apr 11 2019 at 14:44, on Zulip):

bytes -.- xD

pnkfelix (Apr 11 2019 at 14:44, on Zulip):

especially since the person most worried about it is not here in this room

pnkfelix (Apr 11 2019 at 14:45, on Zulip):

okay, so that's the update from WG-mir-opt

Santiago Pastorino (Apr 11 2019 at 14:45, on Zulip):

I'm also interested on what's the idea there :)

pnkfelix (Apr 11 2019 at 14:46, on Zulip):

lets hear from @Alex Crichton about WG-pipelining next

Santiago Pastorino (Apr 11 2019 at 14:46, on Zulip):

but yeah I guess I can open a topic on wg-mit-opt

Alex Crichton (Apr 11 2019 at 14:46, on Zulip):

Sure! So the pipelining WG was created a few days ago at the request of @nikomatsakis, so it's all very new and I'm not sure I have a ton to report other than that it exists

nikomatsakis (Apr 11 2019 at 14:47, on Zulip):

well it's worth reporting that you did a great writeup

eddyb (Apr 11 2019 at 14:47, on Zulip):

I should look at that and maybe help

Alex Crichton (Apr 11 2019 at 14:47, on Zulip):

indeed :)

nikomatsakis (Apr 11 2019 at 14:47, on Zulip):

and just conveying the general idea of what's going on

Alex Crichton (Apr 11 2019 at 14:47, on Zulip):

so @nnethercote and I met over video last Friday

eddyb (Apr 11 2019 at 14:47, on Zulip):

do we have someone who wants to work on that?

Alex Crichton (Apr 11 2019 at 14:47, on Zulip):

and I jotted down a ton of notes at https://github.com/rust-lang/compiler-team/blob/master/working-groups/pipelining/NOTES.md

Alex Crichton (Apr 11 2019 at 14:47, on Zulip):

currently the members of the WG are me and @nnethercote

Alex Crichton (Apr 11 2019 at 14:47, on Zulip):

and the plan, on the rustc side of things, is for this to likely be a relatively short-lived WG

Alex Crichton (Apr 11 2019 at 14:48, on Zulip):

the goal is to enable Cargo to invoke rustc in a pipelined fashion

Alex Crichton (Apr 11 2019 at 14:48, on Zulip):

but the WG isn't currently going to cover the Cargo integration

Alex Crichton (Apr 11 2019 at 14:48, on Zulip):

so it's just ensuring that all the compiler support is in place for Cargo to best take advantage of that

Alex Crichton (Apr 11 2019 at 14:48, on Zulip):

and by "pipelining" what I mean is that Cargo can start dependant rustc processes sooner

Alex Crichton (Apr 11 2019 at 14:48, on Zulip):

overlapping their execution with the parent processes slightly

nikomatsakis (Apr 11 2019 at 14:48, on Zulip):

basically today we compile all your dependencies one after the other:

         meta                meta
[-libA----|--------][-libB----|--------][-binary-----------]
0s        5s       10s       15s       20s                30s
Alex Crichton (Apr 11 2019 at 14:48, on Zulip):

(sort of like CPU pipeliens)

nikomatsakis (Apr 11 2019 at 14:49, on Zulip):

but we should be able to overlap them, letting the downstream crates start compiling while the upstream ones do LLVM work:

[-libA----|--------]
          [-libB----|--------]
                    [-binary-----------]
0s        5s       10s       15s       20s
Alex Crichton (Apr 11 2019 at 14:49, on Zulip):

heh that's a much better idea, and in that chart "meta" is where metadata is produced, where after rustc has metadata it shoudl be able to start down stream compiles immediately

nikomatsakis (Apr 11 2019 at 14:49, on Zulip):

I'm stealing from @Alex Crichton's writeup beacuse I don't think they would think to copy their own pretty pictures in here :P

Alex Crichton (Apr 11 2019 at 14:49, on Zulip):

(you're not wrong)

Alex Crichton (Apr 11 2019 at 14:49, on Zulip):

And... I think that's roughly everything for a checkin I can think of, but if others have questions I can try to answer

Alex Crichton (Apr 11 2019 at 14:50, on Zulip):

(a ton more technical details are in the NOTES.md file and we can probably discuss technical issues on the tracking issue)

Santiago Pastorino (Apr 11 2019 at 14:50, on Zulip):

do we have someone who wants to work on that?

I'm interested, but unsure what @nikomatsakis's plan is :)

Alex Crichton (Apr 11 2019 at 14:50, on Zulip):

which is located at https://github.com/rust-lang/rust/issues/58465

nikomatsakis (Apr 11 2019 at 14:50, on Zulip):

One non-obvious thing was the signaling mechanism

nikomatsakis (Apr 11 2019 at 14:51, on Zulip):

That is, the plan is for rustc to compile the metadata and then send a signal to cargo when that is ready

nikomatsakis (Apr 11 2019 at 14:51, on Zulip):

So cargo can know to kick off the dependent compilations

nikomatsakis (Apr 11 2019 at 14:51, on Zulip):

right?

Alex Crichton (Apr 11 2019 at 14:51, on Zulip):

Right yeah, so we're assuming that this is all going to be in-process for rustc

eddyb (Apr 11 2019 at 14:51, on Zulip):

oh, btw, I was talking to @mw this morning and thinking maybe we should make cross-crate metadata more like incremental even without moving to full "cross-crate incremental state usage"

Alex Crichton (Apr 11 2019 at 14:51, on Zulip):

so Cargo will not invoke rustc twice (once producing metadata and once producing the final linkable artifact)

Alex Crichton (Apr 11 2019 at 14:51, on Zulip):

so this necessitates rustc sending Cargo some sort of signal that metadata is ready

nikomatsakis (Apr 11 2019 at 14:51, on Zulip):

That is, the plan is for rustc to compile the metadata and then send a signal to cargo when that is ready

reviewing the doc, I guess that's just dumping some stuff to stdout

eddyb (Apr 11 2019 at 14:52, on Zulip):

I wonder if we could expose partially-written ar files or something, heh

Alex Crichton (Apr 11 2019 at 14:52, on Zulip):

right yes, so that's the current plan of action

nikomatsakis (Apr 11 2019 at 14:52, on Zulip):

oh, btw, I was talking to mw this morning and thinking maybe we should make cross-crate metadata more like incremental even without moving to full "cross-crate incremental state usage"

that would be an independent change, right @eddyb? (i.e., cargo wouldn't notice)

Alex Crichton (Apr 11 2019 at 14:52, on Zulip):

is to have something on the rustc CLI control whether rustc prints a JSON message (or somethign like that) when an output is ready

eddyb (Apr 11 2019 at 14:52, on Zulip):

so we keep appending object files to an .rlib and only update the file list at the end. but idk if that is supported by anything

Alex Crichton (Apr 11 2019 at 14:52, on Zulip):

(my initial thinking was to just reuse --error-format=json as the CLI flag)

pnkfelix (Apr 11 2019 at 14:53, on Zulip):

is to have something on the rustc CLI control whether rustc prints a JSON message (or somethign like that) when an output is ready

i assume we also need to be careful,when in this mode, not to emit anything to stdout that isn't a json message?

eddyb (Apr 11 2019 at 14:53, on Zulip):

so how would metadata be used without huge changes? rmeta + rlib output?

pnkfelix (Apr 11 2019 at 14:53, on Zulip):

or I guess cargo can just throw away any content that isn't already json formatted?

eddyb (Apr 11 2019 at 14:53, on Zulip):

@pnkfelix IIRC that is already a concern

Alex Crichton (Apr 11 2019 at 14:53, on Zulip):

@pnkfelix nah not necessarily, Cargo would just eat any lines that start with {

Alex Crichton (Apr 11 2019 at 14:53, on Zulip):

(which I think is what Cargo already does today)

eddyb (Apr 11 2019 at 14:54, on Zulip):

does --emit=rmeta,rlib work today?

eddyb (Apr 11 2019 at 14:55, on Zulip):

can cargo check reuse up-to-date crates that weren't built by cargo check? seems relevant here

Alex Crichton (Apr 11 2019 at 14:55, on Zulip):

Yes that was one thing I thought we would have to get working (rustc to only consume metadata files, not rlibs as inputs), but it turns out that all already works in rustc

eddyb (Apr 11 2019 at 14:55, on Zulip):

fascinating :D

Alex Crichton (Apr 11 2019 at 14:55, on Zulip):

There are no plans to integrate this with cargo check right now

eddyb (Apr 11 2019 at 14:56, on Zulip):

a lot of stuff might be coincidental, FWIW, I'd double-check with @mw and whoever else worked in those areas

eddyb (Apr 11 2019 at 14:56, on Zulip):

one thing I was looking at today was a list of symbol name exports in crate metadata, I hope that's serialized properly even with rmeta output

pnkfelix (Apr 11 2019 at 14:56, on Zulip):

can cargo check reuse up-to-date crates that weren't built by cargo check? seems relevant here

i got the impression from other bugs I've seen go by that cargo check does not play well with cargo build, in terms of either destroying or failing to reuse intermediate results

Alex Crichton (Apr 11 2019 at 14:57, on Zulip):

Correct, cargo check and cargo build cannot reuse the same caches

eddyb (Apr 11 2019 at 14:57, on Zulip):

(I was thinking that if Cargo starts doing --emit=rmeta,rlib, the rmeta is already what cargo check needs, but it's a bit offtopic)

Alex Crichton (Apr 11 2019 at 14:57, on Zulip):

there's actually behavior in the compiler today where --emit metadata produces different metadata than --emit metadata,link

eddyb (Apr 11 2019 at 14:57, on Zulip):

ahh! that's what I was wondering

eddyb (Apr 11 2019 at 14:57, on Zulip):

makes sense, even if not ideal :P

nikomatsakis (Apr 11 2019 at 14:58, on Zulip):

can you elaborate a bit?

Alex Crichton (Apr 11 2019 at 14:58, on Zulip):

there's other werid Cargo reasons they're not using the same cache, but yeah that's somewhat offtopic I think :)

nikomatsakis (Apr 11 2019 at 14:58, on Zulip):

is it because doing linking requires us to generate .. more info or something?

Alex Crichton (Apr 11 2019 at 14:58, on Zulip):

@nikomatsakis about how the metadata is different?

eddyb (Apr 11 2019 at 14:58, on Zulip):

I wonder if in rmeta output mode, Cargo could just watch for the file and start dependent rustc right away without any rustc changes

nikomatsakis (Apr 11 2019 at 14:58, on Zulip):

yeah

Alex Crichton (Apr 11 2019 at 14:58, on Zulip):

I believe so, what @eddyb was mentioning about symbols definitely comes to mind

Taylor Cramer (Apr 11 2019 at 14:58, on Zulip):

@Alex Crichton have you given any more consideration to having these two steps be available as separate rustc invocations? I'm currently working on goma (distributed compiler+cache) support for Rust and that's something that has come up repeatedly

Alex Crichton (Apr 11 2019 at 14:58, on Zulip):

where we use the crate types to guide symbol visibility tables and whanot

nikomatsakis (Apr 11 2019 at 14:58, on Zulip):

I've definitely encountered bugs and ICEs that only reproduce in very specific modes

Taylor Cramer (Apr 11 2019 at 14:59, on Zulip):

(we can talk on a separate thread later)

eddyb (Apr 11 2019 at 14:59, on Zulip):

since we create the file elsewhere and "atomically" move it into place

eddyb (Apr 11 2019 at 14:59, on Zulip):

so when Cargo might see the file, it's already complete

eddyb (Apr 11 2019 at 14:59, on Zulip):

assuming it has a predictable file name

nikomatsakis (Apr 11 2019 at 14:59, on Zulip):

PS, I'm so happy to see this "sync" doing exactly what I had hoped it would do, and getting feedback from the rest of the team on the ideas so far :)

Alex Crichton (Apr 11 2019 at 14:59, on Zulip):

@Taylor Cramer we have, yeah, but decided to not pursue that at this time because it seems unlikely to be a win for the local compile build case (all on the same machine). That being said I suspect for the distributed use case it would have clear wins. I also think, though, that it's probably just a change in Cargo to invoke rustc twice rather than changing rustc itself, so that's probably something we can tack on later if necessary.

nikomatsakis (Apr 11 2019 at 15:00, on Zulip):

( but yeah seems like we could move the detailed discussions to #t-compiler/wg-pipelining, seeing as meeting time is almost up )

eddyb (Apr 11 2019 at 15:00, on Zulip):

@Taylor Cramer we should talk about that but I think not integrating such a distributed cache with incremental would be a mistake

eddyb (Apr 11 2019 at 15:00, on Zulip):

and cross-crate incremental is different than rmeta (which could be considered "legacy crate metadata")

Taylor Cramer (Apr 11 2019 at 15:01, on Zulip):

@eddyb I'm not sure how we would integrate it with incremental

eddyb (Apr 11 2019 at 15:01, on Zulip):

basically there is more or less no reason incremental caches can't be read cross-crate to replace rmeta

eddyb (Apr 11 2019 at 15:01, on Zulip):

almost all the infrastructure is in place anyway

Taylor Cramer (Apr 11 2019 at 15:01, on Zulip):

There is-- they would dirty the inputs to the cache

eddyb (Apr 11 2019 at 15:01, on Zulip):

rmeta could just be an incremental cache archive at this point

pnkfelix (Apr 11 2019 at 15:02, on Zulip):

okay I have to go and I agree with @nikomatsakis that this conversation can move into #t-compiler/wg-pipelining

pnkfelix (Apr 11 2019 at 15:02, on Zulip):

thanks to everyone in @T-compiler/meeting for attending!

eddyb (Apr 11 2019 at 15:03, on Zulip):

@Alex Crichton what do you think of the file watching alternative to rustc signalling?

eddyb (Apr 11 2019 at 15:03, on Zulip):

the crazy thing is that it could've been implemented in any Cargo that supports a rustc that supports rmeta

Alex Crichton (Apr 11 2019 at 15:03, on Zulip):

Heh, sounds like a cross-platform nightmare

Alex Crichton (Apr 11 2019 at 15:04, on Zulip):

(also very unlikely to work on all kinds of filesystems like nfs)

eddyb (Apr 11 2019 at 15:04, on Zulip):

idk how I didn't think of it before - ofc, maybe I did and also heard about the nightmare

eddyb (Apr 11 2019 at 15:04, on Zulip):

Cargo could poll every 10ms or so, I wonder how sucky that would be

eddyb (Apr 11 2019 at 15:04, on Zulip):

it only needs to do it for the output of running rustc processes

pnkfelix (Apr 11 2019 at 15:04, on Zulip):

(To me an explicit print to stdout sounds less fragile.)

eddyb (Apr 11 2019 at 15:05, on Zulip):

yes, I agree

pnkfelix (Apr 11 2019 at 15:05, on Zulip):

if we didn't control the full stack here, then I'd be more open to @eddyb 's alternative approach.

eddyb (Apr 11 2019 at 15:05, on Zulip):

we could also use it for Cargo build progress updates :P

eddyb (Apr 11 2019 at 15:06, on Zulip):

I think I'm fine with the rustc thing if it's not very pipeline-centric, but more like "artifact emitted"

nikomatsakis (Apr 11 2019 at 15:07, on Zulip):

PS, @Wesley Wiser -- you will prepare some minutes? Maybe make a pull request to https://github.com/rust-lang/compiler-team creating a directory minutes/triage-meeting, similar to the steering-meeting directory?

nikomatsakis (Apr 11 2019 at 15:07, on Zulip):

If so, ping me when that's available :)

Wesley Wiser (Apr 11 2019 at 15:14, on Zulip):

@nikomatsakis It's available :slight_smile: https://github.com/rust-lang/compiler-team/pull/51

nikomatsakis (Apr 11 2019 at 15:16, on Zulip):

@Wesley Wiser I created this thread on IRLO, if you want to post there: https://internals.rust-lang.org/t/compiler-team-triage-meeting/9803

Wesley Wiser (Apr 11 2019 at 15:17, on Zulip):

Thanks @nikomatsakis! x-posted

eddyb (Apr 11 2019 at 15:17, on Zulip):

@Taylor Cramer btw, where should we discuss the distributed compilation matter further?

eddyb (Apr 11 2019 at 15:18, on Zulip):

I think we could do some really cool things, even though Rust is not a pure language :P

nikomatsakis (Apr 11 2019 at 15:20, on Zulip):

@eddyb and @Taylor Cramer -- I'd say just make a topic here in #t-compiler, if you think it's out of scope for the pipelining WG (which it sounds like it might be).

Last update: Nov 16 2019 at 01:05UTC