Stream: t-compiler/wg-polonius

Topic: meeting 2019.04.16


nikomatsakis (Apr 16 2019 at 18:34, on Zulip):

Hello @WG-polonius :wave:

nikomatsakis (Apr 16 2019 at 18:37, on Zulip):

Er, wait, I'm early :P

Albin Stjerna (Apr 16 2019 at 18:37, on Zulip):

Yep

nikomatsakis (Apr 16 2019 at 18:37, on Zulip):

Never mind :)

Albin Stjerna (Apr 16 2019 at 18:37, on Zulip):

I'm here though!

nikomatsakis (Apr 16 2019 at 18:37, on Zulip):

Either way, we can proceed now, or I can investigate some other stuff. I guess it's mostly just you at this point

nikomatsakis (Apr 16 2019 at 18:38, on Zulip):

How goes, @Albin Stjerna =)

Albin Stjerna (Apr 16 2019 at 18:38, on Zulip):

I guess so, in particular since @lqd couldn't attend

nikomatsakis (Apr 16 2019 at 18:38, on Zulip):

The other person was @Matthew Jasper I think but I think they're more following along for now

Albin Stjerna (Apr 16 2019 at 18:38, on Zulip):

ah, yeah probably

nikomatsakis (Apr 16 2019 at 18:39, on Zulip):

We had talked about trying to make a more detailed schedule this week, but maybe we can start by checking in on whether you've had much luck with the liveness and other things we talked about from last week

Albin Stjerna (Apr 16 2019 at 18:40, on Zulip):

I can report that a) the new computer I built builds rustc in half the time of my laptop (surprise), and b) I have finally finished the paperwork to get registered on the master's course, after camping out outside Tobias office for roughly an hour

nikomatsakis (Apr 16 2019 at 18:40, on Zulip):

Say more about (a)?

nikomatsakis (Apr 16 2019 at 18:40, on Zulip):

Oh, new computer

nikomatsakis (Apr 16 2019 at 18:40, on Zulip):

I misread that a few times as "new compiler"

Albin Stjerna (Apr 16 2019 at 18:40, on Zulip):

Haha I realised

Albin Stjerna (Apr 16 2019 at 18:41, on Zulip):

It's not super interesting really, except that I now have a much faster machine to do benchmarks, if there are any, on

nikomatsakis (Apr 16 2019 at 18:41, on Zulip):

Well, that seems positive =)

nikomatsakis (Apr 16 2019 at 18:41, on Zulip):

You opened that PR to extend rustc with the new hybrid algorithm

nikomatsakis (Apr 16 2019 at 18:41, on Zulip):

I haven't followed what happened to it since :)

nikomatsakis (Apr 16 2019 at 18:41, on Zulip):

Oh, wow, it landed

nikomatsakis (Apr 16 2019 at 18:42, on Zulip):

I'm not accustomed to things that touch Cargo.lock ever landing

Albin Stjerna (Apr 16 2019 at 18:42, on Zulip):

Ah, yes, but that's just version-bumping, that's not liveness

nikomatsakis (Apr 16 2019 at 18:42, on Zulip):

I know

Albin Stjerna (Apr 16 2019 at 18:42, on Zulip):

Haha

nikomatsakis (Apr 16 2019 at 18:42, on Zulip):

But we can now test the hybrid algorithm, at least :)

Albin Stjerna (Apr 16 2019 at 18:42, on Zulip):

Well I didn't know that and so wasn't impressed

nikomatsakis (Apr 16 2019 at 18:42, on Zulip):

Not that it's so important

nikomatsakis (Apr 16 2019 at 18:42, on Zulip):

But as far as liveness, you didn't get a chance to do much, is that correct?

Albin Stjerna (Apr 16 2019 at 18:42, on Zulip):

Oh, I did use Criterion to implement some micro-benchmarks that compared hybrid to the built-in

Albin Stjerna (Apr 16 2019 at 18:43, on Zulip):

That's correct, I'm still blocking on "how in the world does even rustc work"

nikomatsakis (Apr 16 2019 at 18:44, on Zulip):

Well wait

nikomatsakis (Apr 16 2019 at 18:44, on Zulip):

I'm remembering now

nikomatsakis (Apr 16 2019 at 18:44, on Zulip):

I had written up some simple instructons to extend polonius with liveness analysis, and you did those

nikomatsakis (Apr 16 2019 at 18:44, on Zulip):

(Correct?)

Albin Stjerna (Apr 16 2019 at 18:44, on Zulip):

Yes, and then you helped me fix the off-by-one error

nikomatsakis (Apr 16 2019 at 18:44, on Zulip):

So I guess we need some new instructions to cover the part about extending rustc

Albin Stjerna (Apr 16 2019 at 18:44, on Zulip):

Exactly, and I tried that on my own first

nikomatsakis (Apr 16 2019 at 18:44, on Zulip):

But -- we can probably land that first set of changes, in theory

nikomatsakis (Apr 16 2019 at 18:45, on Zulip):

or at least open a draft PR

nikomatsakis (Apr 16 2019 at 18:45, on Zulip):

( Not clear that there's a ton of value in landing them as they are )

Albin Stjerna (Apr 16 2019 at 18:45, on Zulip):

Yes, and no

nikomatsakis (Apr 16 2019 at 18:45, on Zulip):

We may want to, though, to make it easier to modify rustc

nikomatsakis (Apr 16 2019 at 18:45, on Zulip):

e.g., to add a new set of facts that rustc can export

Albin Stjerna (Apr 16 2019 at 18:45, on Zulip):

One of the problems, one that I ran into as I was trying this on my own, is that Output is now generic to variables

Albin Stjerna (Apr 16 2019 at 18:46, on Zulip):

That is, the output from Polonius

Albin Stjerna (Apr 16 2019 at 18:46, on Zulip):

And I have no idea what to use for the variable type, because I found nothing in the NLL code that looks even a bit like a unique variable identifier type

nikomatsakis (Apr 16 2019 at 18:47, on Zulip):

Ah. The answer is rustc::mir::Local

nikomatsakis (Apr 16 2019 at 18:47, on Zulip):

I'm wondering if it makes sense to do another (recorded) Zoom dive

Albin Stjerna (Apr 16 2019 at 18:47, on Zulip):

Great, that's progress already!

nikomatsakis (Apr 16 2019 at 18:47, on Zulip):

And, if so, exactly what to cover :)

nikomatsakis (Apr 16 2019 at 18:47, on Zulip):

Maybe not

nikomatsakis (Apr 16 2019 at 18:47, on Zulip):

I guess we could start with me just attempting to write up some notes

nikomatsakis (Apr 16 2019 at 18:47, on Zulip):

e.g., with some info like this :)

nikomatsakis (Apr 16 2019 at 18:48, on Zulip):

basically there would be two options

nikomatsakis (Apr 16 2019 at 18:48, on Zulip):

either I try to write up some notes

nikomatsakis (Apr 16 2019 at 18:48, on Zulip):

or we do a zoom call and try to write up the notes together

nikomatsakis (Apr 16 2019 at 18:49, on Zulip):

explaining here and there as we go

Albin Stjerna (Apr 16 2019 at 18:49, on Zulip):

As in right now you mean, or later?

nikomatsakis (Apr 16 2019 at 18:50, on Zulip):

I didn't mean this second

nikomatsakis (Apr 16 2019 at 18:50, on Zulip):

Probably friday

nikomatsakis (Apr 16 2019 at 18:51, on Zulip):

I guess it depends which you would prefer, in part

nikomatsakis (Apr 16 2019 at 18:51, on Zulip):

I can at least try to leave some notes today and maybe we can decide over next couple of days whether to also do something on friday or later

Albin Stjerna (Apr 16 2019 at 18:51, on Zulip):

I'm at my parent's on Friday evening, which isn't necessarily a problem

nikomatsakis (Apr 16 2019 at 18:51, on Zulip):

(I could also do Friday afternoon, if that's preferred)

Albin Stjerna (Apr 16 2019 at 18:51, on Zulip):

Yes, I think that's a good idea

nikomatsakis (Apr 16 2019 at 18:52, on Zulip):

I can at least try to leave some notes today and maybe we can decide over next couple of days whether to also do something on friday or later

you mean this?

Albin Stjerna (Apr 16 2019 at 18:52, on Zulip):

I can at least try to leave some notes today and maybe we can decide over next couple of days whether to also do something on friday or later

you mean this?

Err, yes (I need to learn the Zulip hotkeys)

nikomatsakis (Apr 16 2019 at 18:52, on Zulip):

(I wish Zulip made replying to messages a bit more first-class -- need threading within a topic, too...ah well :)

nikomatsakis (Apr 16 2019 at 18:53, on Zulip):

OK. So in terms of trying to come up with a schedule

nikomatsakis (Apr 16 2019 at 18:53, on Zulip):

I was looking over the polonius roadmap paper doc...

Albin Stjerna (Apr 16 2019 at 18:53, on Zulip):

I can at least try to leave some notes today and maybe we can decide over next couple of days whether to also do something on friday or later

you mean this?

Anyway, I think this is a good solution because it a) unblocks me and has me continuing so that I b) have relevant questions for the Zoom session

nikomatsakis (Apr 16 2019 at 18:54, on Zulip):

Yep

nikomatsakis (Apr 16 2019 at 18:54, on Zulip):

there is this list:

nikomatsakis (Apr 16 2019 at 18:54, on Zulip):

I think this still more or less covers all the major steps

nikomatsakis (Apr 16 2019 at 18:54, on Zulip):

but it's a bit high-level :)

Albin Stjerna (Apr 16 2019 at 18:54, on Zulip):

It is

nikomatsakis (Apr 16 2019 at 18:55, on Zulip):

and these things are not all alike

nikomatsakis (Apr 16 2019 at 18:55, on Zulip):

so I think the liveness stuff is ultimately fairly straightforward, although there is a good question as to whether i'm overlooking something

nikomatsakis (Apr 16 2019 at 18:55, on Zulip):

I'm going to reorder the steps in roughly the order I think we should be working on them

nikomatsakis (Apr 16 2019 at 18:57, on Zulip):

So this is I think the full set of steps for liveness:

nikomatsakis (Apr 16 2019 at 18:57, on Zulip):

I don't expect you to understand everything I wrote there

nikomatsakis (Apr 16 2019 at 18:57, on Zulip):

But you more or less did the first step, though there is this use-live and drop-live twist

nikomatsakis (Apr 16 2019 at 18:57, on Zulip):

By the way, have you seen the NLL RFC?

Albin Stjerna (Apr 16 2019 at 18:57, on Zulip):

Seen yes, read closely: no

nikomatsakis (Apr 16 2019 at 18:58, on Zulip):

In particular, it has a section on liveness

nikomatsakis (Apr 16 2019 at 18:58, on Zulip):

and a second section on accommodating dropck

nikomatsakis (Apr 16 2019 at 18:58, on Zulip):

these bits are all quite relevant

nikomatsakis (Apr 16 2019 at 18:59, on Zulip):

(the latter bit introduces this use-live vs drop-live distinction)

Albin Stjerna (Apr 16 2019 at 18:59, on Zulip):

Ah, ok so there is a difference between drop uses and live uses, basically

nikomatsakis (Apr 16 2019 at 19:01, on Zulip):

Yes

nikomatsakis (Apr 16 2019 at 19:01, on Zulip):

The basic idea of NLL is this

Albin Stjerna (Apr 16 2019 at 19:01, on Zulip):

Well, I guess that makes sense given that drop isn't a real "use" that needs the data

nikomatsakis (Apr 16 2019 at 19:01, on Zulip):

Any place where a variable X is live, then any regions that appear in its type T are "live" --

nikomatsakis (Apr 16 2019 at 19:01, on Zulip):

but if the only reason that X is live is because it will be dropped, we're a bit more precise, and we only include those regions that the drop may examine

nikomatsakis (Apr 16 2019 at 19:02, on Zulip):

I guess the most naive thing we can do

nikomatsakis (Apr 16 2019 at 19:02, on Zulip):

and probably the thing we should do to start

nikomatsakis (Apr 16 2019 at 19:02, on Zulip):

is to compute two entirely separate notions of liveness

nikomatsakis (Apr 16 2019 at 19:02, on Zulip):

this is inefficient because the places that a variable is a "drop-live" are a superset of those places where it is "use-live", in some sense

nikomatsakis (Apr 16 2019 at 19:03, on Zulip):

although I guess that this is not strictly true, depending on how you define things, because -- for some variables -- we have no drop at all, since we can trivially see the drop is a no-op

Albin Stjerna (Apr 16 2019 at 19:05, on Zulip):

Hmm, ok

Albin Stjerna (Apr 16 2019 at 19:05, on Zulip):

(I'm digesting this)

nikomatsakis (Apr 16 2019 at 19:06, on Zulip):

Anyway, it's honestly a bit hard for me to tell how long this will take you :)

Albin Stjerna (Apr 16 2019 at 19:06, on Zulip):

To begin with the real basics, the reason we need to compute regions at all is that they may be referred to by types ('a etc), right? If they were only calculated implicitly, we could only trace each variable on its own?

nikomatsakis (Apr 16 2019 at 19:06, on Zulip):

But it seems like it'd be good to try and elaborate the other steps to a similar level of detail

Albin Stjerna (Apr 16 2019 at 19:06, on Zulip):

Yes, I agree

nikomatsakis (Apr 16 2019 at 19:07, on Zulip):

Anyway, it's honestly a bit hard for me to tell how long this will take you :)

and maybe we can calibrate how long to expect for the other changes based partially on this data :)

nikomatsakis (Apr 16 2019 at 19:07, on Zulip):

(that said, it's the hardest right now, where you are still building up context)

nikomatsakis (Apr 16 2019 at 19:07, on Zulip):

To begin with the real basics, the reason we need to compute regions at all is that they may be referred to by types ('a etc), right? If they were only calculated implicitly, we could only trace each variable on its own?

I..guess? I'm not sure 100% what this means

Albin Stjerna (Apr 16 2019 at 19:08, on Zulip):

To begin with the real basics, the reason we need to compute regions at all is that they may be referred to by types ('a etc), right? If they were only calculated implicitly, we could only trace each variable on its own?

I..guess? I'm not sure 100% what this means

I'll see if I have more questions once I have re-read the NLL RFC and/or the Rust manual, but let's drop it for now (harr harr) and I'll add it to my backlog of things to think about

nikomatsakis (Apr 16 2019 at 19:09, on Zulip):

yes, ok

nikomatsakis (Apr 16 2019 at 19:10, on Zulip):

so it seems like we're ok for now, but I'll put in some energy to trying to enumerate the other points

nikomatsakis (Apr 16 2019 at 19:11, on Zulip):

basically I have two work items:

nikomatsakis (Apr 16 2019 at 19:11, on Zulip):

I have a question re: rustc, do you know how to build a copy of rustc that (locally) uses a distinct polonius-engine?

nikomatsakis (Apr 16 2019 at 19:11, on Zulip):

That is, you can configure rustc to use a polonius-engine from a local directory instead of crates.io

nikomatsakis (Apr 16 2019 at 19:11, on Zulip):

This would allow you to (for example) modify polonius-engine with new facts and things

nikomatsakis (Apr 16 2019 at 19:12, on Zulip):

and then build against that

nikomatsakis (Apr 16 2019 at 19:12, on Zulip):

and co-develop

Albin Stjerna (Apr 16 2019 at 19:12, on Zulip):

That is, you can configure rustc to use a polonius-engine from a local directory instead of crates.io

Yes, that's the thing I did figure out when trying generate the facts on my own

Albin Stjerna (Apr 16 2019 at 19:12, on Zulip):

Basically I stared from that and tried to see what broke in the compilation

Albin Stjerna (Apr 16 2019 at 19:12, on Zulip):

I am also fairly sure I have managed to build and use my own versions of rustc, but it's a bit hard to tell

Albin Stjerna (Apr 16 2019 at 19:13, on Zulip):

(I'll try to make one that crashes, that's usually a way to be sure)

nikomatsakis (Apr 16 2019 at 19:13, on Zulip):

heh :) you can tell with rustc -vV as well, I think

nikomatsakis (Apr 16 2019 at 19:13, on Zulip):

the output is pretty different

nikomatsakis (Apr 16 2019 at 19:14, on Zulip):

e.g.,

> rustc +nightly -vV
rustc 1.35.0-nightly (82e2f3ec2 2019-03-20)
binary: rustc
commit-hash: 82e2f3ec25a316c7536c33a6b6704366b14cbf2a
commit-date: 2019-03-20
host: x86_64-unknown-linux-gnu
release: 1.35.0-nightly
LLVM version: 8.0
> rustc +rust-0-stage2 -vV
rustc 1.33.0-dev
binary: rustc
commit-hash: unknown
commit-date: unknown
host: x86_64-unknown-linux-gnu
release: 1.33.0-dev
LLVM version: 8.0
Albin Stjerna (Apr 16 2019 at 19:15, on Zulip):

Well, this looks promising:

➜  Masterprojekt rustc +stage1 -vV
rustc 1.35.0-dev
binary: rustc
commit-hash: unknown
commit-date: unknown
host: x86_64-apple-darwin
release: 1.35.0-dev
LLVM version: 8.0

➜  Masterprojekt rustc +nightly -vV
rustc 1.35.0-nightly (3de010678 2019-04-11)
binary: rustc
commit-hash: 3de0106789468b211bcc3a25c09c0cf07119186d
commit-date: 2019-04-11
host: x86_64-apple-darwin
release: 1.35.0-nightly
LLVM version: 8.0
Albin Stjerna (Apr 16 2019 at 19:17, on Zulip):

But I guess the relevant question for me is what my next step is

Albin Stjerna (Apr 16 2019 at 19:23, on Zulip):

Also, is there anyone besides you I can/should ask about NLL/facts generation stuff when you are not around? Except yelling at the entire stream handle and hoping someone's responding

Albin Stjerna (Apr 16 2019 at 19:31, on Zulip):

err, I just realised I never actually changed the default. Should I make a new PR with only that change, or piggy-back it on lifetimes when they land?

lqd (Apr 16 2019 at 19:39, on Zulip):

(On my phone) IIRC you did change the default in your rustc PR (a string changing from DatafrogOpt to Hybrid)

Albin Stjerna (Apr 16 2019 at 19:41, on Zulip):

(On my phone) IIRC you did change the default in your rustc PR (a string changing from DatafrogOpt to Hybrid)

Nevermind, I did, and then I forgot about it

Last update: Nov 15 2019 at 20:00UTC