Stream: t-compiler

Topic: learning WG


nikomatsakis (Apr 18 2019 at 15:51, on Zulip):

So I've been talking to @Santiago Pastorino about the idea of creating a "Learning WG". I can think of a few others to cc here, but I'm sure I'm overlooking people: @Amanjeev Sethi @Iñaki Garay @Federico Carrone @mark-i-m.

I think the charter of this WG should be to make rustc easy to learn. This is perhaps too open-ended, so let me try to make it more specific:

I don't envision this being an overnight process, it's going to take some time, but I think it's one we can make steady progress on. I am also open to narrowing down our focus -- probably to just the first item, since I have a clearer plan there.

I have some concerns: primarily, I know my own time is way too finely sliced right now, so I would like to ensure that others are doing the organizational heavy lifting. I imagined that we would proceed by:

This results in (a) a video and (b) a chapter.

The plan for API docs would presumably be similar. We just try to work out way through.

I imagine my own participation would be limited to helping to craft the topics and doing some of those videos. The remaining organizational work would have to be done by others.

What do people think?

Nick Lawrence (Apr 18 2019 at 15:54, on Zulip):

I think after the baby steps post it's a good idea to have a room / wg to funnel the new folk to

nikomatsakis (Apr 18 2019 at 15:56, on Zulip):

If we want to do this, we should talk about the concrete steps to take. I would like to help in planning those but would like to encourage other people to take them. =) I imagine it mostly means setting up a meeting and some agenda (of the kind I outlined above), but also creating the WG and making an announcement etc.

nikomatsakis (Apr 18 2019 at 15:56, on Zulip):

One nice thing about API docs, I guess, is that they are relatively easy to "parallelize" in terms of having enough work for everyone

davidtwco (Apr 18 2019 at 15:56, on Zulip):

I don't think I've got any bandwidth to contribute to it, but I'm all for the idea.

nikomatsakis (Apr 18 2019 at 15:56, on Zulip):

if a lot of people show up

nikomatsakis (Apr 18 2019 at 15:56, on Zulip):

I'm also curious whether, in the process of doing this sort of thing, we'll come up with new ideas (I thnk yes)

nikomatsakis (Apr 18 2019 at 15:57, on Zulip):

e.g., I've thought about trying to have a "course" to learn rustc,

nikomatsakis (Apr 18 2019 at 15:57, on Zulip):

where we take an example program (say) and step through how it is compiled bit by bit

nikomatsakis (Apr 18 2019 at 15:57, on Zulip):

(actually, maybe that's how we should structure the rustc-guide process, not sure)

Federico Carrone (Apr 18 2019 at 15:57, on Zulip):

great idea. i think that in the process of working on this new ideas will appear

Federico Carrone (Apr 18 2019 at 16:03, on Zulip):

The course idea is awesome. I think that we should start doing small concrete contributions and within a few months we should be able to start working on something like that. I don't want to commit to something that we can't do yet :)! As soon as we have enough information of the different parts of the compilers we should be able to work on something like a course without needed. Right now I am already studying the ty module for the call we will have and I am having a rough time understanding the code. It is the first time I read about the internals and of a compiler as big as rustc and I am a little bit overwhelmed by the information and code.

Federico Carrone (Apr 18 2019 at 16:05, on Zulip):

@Juan Bono is part of our team and he will also help with this too.

Federico Carrone (Apr 18 2019 at 16:09, on Zulip):

Is there any reason why the expert map (https://github.com/rust-lang/compiler-team/blob/master/experts/MAP.md) is in hackmd instead of github (https://hackmd.io/Izvor8KZRiqUgcUyd2CYTw)? @Santiago Pastorino any idea? I am asking since I am trying to understand what each part of the compiler does and if I don't understand something I want to know who to ask what.

Amanjeev Sethi (Apr 18 2019 at 16:20, on Zulip):

I like this idea. I am pretty new to Rust and new to any such big compiler project (or any compiler project) but I want to help with this. I see a lot of value in this. It is daunting for say, someone like me, to even take a look at the source. This would not only help the future contributors but help me as well. I believe that we can start with smaller, low hanging fruit (if there is any?). As @Federico Carrone said that we can start with small concrete contributions.

davidtwco (Apr 18 2019 at 16:25, on Zulip):

Is there any reason why the expert map (https://github.com/rust-lang/compiler-team/blob/master/experts/MAP.md) is in hackmd instead of github (https://hackmd.io/Izvor8KZRiqUgcUyd2CYTw)? Santiago Pastorino any idea? I am asking since I am trying to understand what each part of the compiler does and if I don't understand something I want to know who to ask what.

We wanted it to be somewhere that was easy to edit while it was still being changed regularly.

Santiago Pastorino (Apr 18 2019 at 17:41, on Zulip):

hi everyone, I'm on vacations will be back on monday

Santiago Pastorino (Apr 18 2019 at 17:42, on Zulip):

so I may be very unresponsive :)

Santiago Pastorino (Apr 18 2019 at 17:44, on Zulip):

@nikomatsakis in general I guess I could take care of most (or all) the organizational part of this

Santiago Pastorino (Apr 18 2019 at 17:45, on Zulip):

and also, of course, contribute myself to this :)

Santiago Pastorino (Apr 18 2019 at 17:47, on Zulip):

e.g., I've thought about trying to have a "course" to learn rustc,

this is also one of the things I believe are great if we can accomplish

Santiago Pastorino (Apr 18 2019 at 17:48, on Zulip):

(actually, maybe that's how we should structure the rustc-guide process, not sure)

we can have parts of the general theory and explanations but also a (or some) programs that we use as an example and show what's going on with internally

Santiago Pastorino (Apr 18 2019 at 17:50, on Zulip):

Is there any reason why the expert map (https://github.com/rust-lang/compiler-team/blob/master/experts/MAP.md) is in hackmd instead of github (https://hackmd.io/Izvor8KZRiqUgcUyd2CYTw)? Santiago Pastorino any idea? I am asking since I am trying to understand what each part of the compiler does and if I don't understand something I want to know who to ask what.

this is meanwhile we stabilize it a bit more because it's easier to modify there. At some point, maybe now?, we should migrate to github /cc @nikomatsakis @davidtwco

Federico Carrone (Apr 18 2019 at 18:30, on Zulip):

who knows or is in change of libs? and what really is the libs area?

varkor (Apr 18 2019 at 18:31, on Zulip):

@Federico Carrone: https://www.rust-lang.org/governance/teams/library

varkor (Apr 18 2019 at 18:32, on Zulip):

the libs team is responsible for the standard library as a whole

Federico Carrone (Apr 18 2019 at 18:33, on Zulip):

oh cool. i will add that to the hackmd info and send a PR with the experts map

varkor (Apr 18 2019 at 18:38, on Zulip):

hmm, I'm not sure about the expert map with regards to the standard library — many of the APIs are quite straightforward

varkor (Apr 18 2019 at 18:38, on Zulip):

so if they're to be added to the expert map at all, it should probably be for things like the data structures

varkor (Apr 18 2019 at 18:38, on Zulip):

(e.g. people like @Gankro)

centril (Apr 18 2019 at 18:39, on Zulip):

I'd love an experts map for the standard library

centril (Apr 18 2019 at 18:39, on Zulip):

would be really helpful for t-release

varkor (Apr 18 2019 at 18:40, on Zulip):

what I mean is that we want individual experts for aspects of the standard library, rather than a T-libs "catch-all"

Federico Carrone (Apr 18 2019 at 18:43, on Zulip):

i can ask each of the standard library team what he/she is doing and do an expert map for the standard library too

varkor (Apr 18 2019 at 18:44, on Zulip):

@Federico Carrone: if you're looking for libs team members, I think they mostly hang out on IRC (the #rust-libs channel)

Iñaki Garay (Apr 18 2019 at 21:07, on Zulip):

@Federico Carrone https://github.com/rust-lang/compiler-team/issues/63 tracks this

Santiago Pastorino (Apr 18 2019 at 21:20, on Zulip):

if you find out something useful please let me know in the issue

Amanjeev Sethi (Apr 18 2019 at 21:24, on Zulip):

should the issue link to https://hackmd.io/Izvor8KZRiqUgcUyd2CYTw ?

Santiago Pastorino (Apr 19 2019 at 11:38, on Zulip):

@Amanjeev Sethi feel free to add that link as a comment on the issue

Iñaki Garay (Apr 19 2019 at 13:56, on Zulip):

Here is my list so far of things to do which could be a basis for the learning-wg goals:

A) Rust Compiler Lecture Series Transcription
    - transcribe debugging support in rustc video
        https://www.youtube.com/watch?v=elBxMRSNY
    - transcribe cranelift video 
        https://www.youtube.com/watch?v=9OIA7DTFQWU
    - transcribe rust analyzer video
        https://www.youtube.com/watch?v=ANKBNiSWyfc
    - transcribe mir and const eval video
        https://www.youtube.com/watch?v=5Pm2C1YXrvM
    - transcribe polonius walkthrough video
        https://www.youtube.com/watch?v=i5KdU0ieb_A
    - transcribe rustc-chalk integration video
        https://www.youtube.com/watch?v=MBWtbDifPeU
    - transcribe coherence in chalk video
        https://www.youtube.com/watch?v=rZqS4bLPL24
    - transcribe the ty overview video
B) Issues: 
    - Move relevant content from the forge
        https://github.com/rust-lang/rustc-guide/issues/133
    - document where various bits of rust are desugared
        https://github.com/rust-lang/rustc-guide/issues/94
    - "Diagnostics: reporting errors and lints"
        talk w/ esteban about this?
        https://github.com/rust-lang/rustc-guide/issues/14
C) Compiler walkthrough / tour
    (More or less an expansion of the "High-level overview of the compiler source" section)
    Content (or links to):
    - compiler high level structure
    - system requirements & dependencies
    - compiler module structure
    - crate dependencies
    - compiler configuration
    - compiler inputs
    - compiler outputs
    - compiler state
    - Walkthrough of compiling a meaningful example
D) Course
Santiago Pastorino (Apr 19 2019 at 16:38, on Zulip):

@Iñaki Garay that’s great, on monday I will try to start a plan for this where we can all collaborate /cc @nikomatsakis

Amanjeev Sethi (Apr 19 2019 at 17:55, on Zulip):

@Santiago Pastorino is this the narrow where you will post the updates on Monday?

Santiago Pastorino (Apr 19 2019 at 18:04, on Zulip):

what do you mean by narrow?, sorry but didn't get what you meant

Amanjeev Sethi (Apr 19 2019 at 19:11, on Zulip):

Sorry I am new to Zulip. I believe "learning WG" is called a "narrow", if I am not mistaken.

davidtwco (Apr 19 2019 at 19:44, on Zulip):

Zulip documentation calls it a topic, within the #t-compiler stream. Narrowing is the terminology they use to describe when you’ve filtered what you see down to just a single topic or stream.

Amanjeev Sethi (Apr 19 2019 at 19:48, on Zulip):

Thank you @davidtwco . :sad: sorry about the confusion.

davidtwco (Apr 19 2019 at 19:48, on Zulip):

All good, we all get tripped up with Zulip from time to time.

Santiago Pastorino (Apr 19 2019 at 19:55, on Zulip):

ahh yes, don’t worry I’ll let everyone know as soon as I can build something

nikomatsakis (Apr 19 2019 at 20:17, on Zulip):

@Iñaki Garay thanks for the list! That's a good starting point. I think the main thing we should do is to try and start by taking one specific goal. I'd like to try the "sprint-like" structure that many WGs are trying, basically taking some time to do planning and then the next 6 weeks or so to execute on that plan, and then take some more time to plan.

nikomatsakis (Apr 19 2019 at 20:18, on Zulip):

It seems like a good starting point would be for us to draft up a kind of "working group proposal"

nikomatsakis (Apr 19 2019 at 20:21, on Zulip):

I think starting with a focus on rustc-guide / explaining parts of the code makes total sense, it seems relatively clear

Iñaki Garay (Apr 19 2019 at 20:44, on Zulip):

Iñaki Garay thanks for the list! That's a good starting point. I think the main thing we should do is to try and start by taking one specific goal. I'd like to try the "sprint-like" structure that many WGs are trying, basically taking some time to do planning and then the next 6 weeks or so to execute on that plan, and then take some more time to plan.

this sounds reasonably and adapts to my (at least) current lifestyle well.
still getting up to speed on meta-wg but it seems like if we follow through with a wg proposal, it gets approved and created and we then hold a planning meeting we can agree on persisting these items in issues, selecting them for immediate work and distributing them.

Santiago Pastorino (Apr 20 2019 at 13:07, on Zulip):

It seems like a good starting point would be for us to draft up a kind of "working group proposal"

yes, talk to you on monday

nikomatsakis (Apr 22 2019 at 20:40, on Zulip):

ping @Santiago Pastorino

nikomatsakis (Apr 22 2019 at 20:41, on Zulip):

So I think that we want to do a few things

nikomatsakis (Apr 22 2019 at 20:41, on Zulip):

Probably the most important is to define our focus

nikomatsakis (Apr 22 2019 at 20:42, on Zulip):

But I guess we largely covered this already in this topic

nikomatsakis (Apr 22 2019 at 20:42, on Zulip):

(I'm skimming back over the comments)

Santiago Pastorino (Apr 22 2019 at 20:43, on Zulip):

hey @nikomatsakis

nikomatsakis (Apr 22 2019 at 20:43, on Zulip):

I think the concrete steps we want to take then are:

nikomatsakis (Apr 22 2019 at 20:43, on Zulip):

I guess that the selecting of initial goals can take place in a first meeting

Santiago Pastorino (Apr 22 2019 at 20:44, on Zulip):

:+1:

Santiago Pastorino (Apr 22 2019 at 20:44, on Zulip):

agree with what you've said

nikomatsakis (Apr 22 2019 at 20:44, on Zulip):

In that PR, we should define our "charter". I think our goal should be to ensure that the rustc-guide is "complete" (for some reasonable approximation thereof).

To be clear, I am debating here basically about how "focused" we want to be -- I guess I think it's fine to adopt a more wide-ranging goal ("to make rustc easier to learn"), but of course we'll never be "done" with that.

Santiago Pastorino (Apr 22 2019 at 20:44, on Zulip):

I can start working on the PR to add the WG directory

Santiago Pastorino (Apr 22 2019 at 20:45, on Zulip):

In that PR, we should define our "charter". I think our goal should be to ensure that the rustc-guide is "complete" (for some reasonable approximation thereof).

To be clear, I am debating here basically about how "focused" we want to be -- I guess I think it's fine to adopt a more wide-ranging goal ("to make rustc easier to learn"), but of course we'll never be "done" with that.

hehe, yes

nikomatsakis (Apr 22 2019 at 20:45, on Zulip):

I would be happy with something like

Santiago Pastorino (Apr 22 2019 at 20:45, on Zulip):

so ... I guess that's kind of a vision but I agree that the project is more specific than that

nikomatsakis (Apr 22 2019 at 20:45, on Zulip):

But I also think this is the kind fo thing where -- if we want to grow our scope, we can do that

nikomatsakis (Apr 22 2019 at 20:45, on Zulip):

i.e., maybe we start with a specific goal, and we can add new specific goals later :)

Santiago Pastorino (Apr 22 2019 at 20:45, on Zulip):

sure

nikomatsakis (Apr 22 2019 at 20:46, on Zulip):

my only concern about rustc-guide is that it's going to be hard to parallelize

Santiago Pastorino (Apr 22 2019 at 20:46, on Zulip):

why do you think that?

nikomatsakis (Apr 22 2019 at 20:46, on Zulip):

mm I mean that if we have e.g. 3 or 4 people who want to participaet

nikomatsakis (Apr 22 2019 at 20:46, on Zulip):

they're going to be somewhat gated on others being table to take the time to explain 3 or 4 parts of rustc

Santiago Pastorino (Apr 22 2019 at 20:47, on Zulip):

ahh that's true

nikomatsakis (Apr 22 2019 at 20:47, on Zulip):

that said, well, consider this Friday -- I was planning on talking about the ty module

nikomatsakis (Apr 22 2019 at 20:47, on Zulip):

I could imagine there being two people working on writing that up, for example, as a pair

nikomatsakis (Apr 22 2019 at 20:47, on Zulip):

or just one

Santiago Pastorino (Apr 22 2019 at 20:47, on Zulip):

yes

nikomatsakis (Apr 22 2019 at 20:47, on Zulip):

although, if there are more folks who are interested, maybe they get started by documenting some API stuff -- possilby int hat same area

nikomatsakis (Apr 22 2019 at 20:48, on Zulip):

i.e., I doubt that there are good rustdoc comments on this stuff :)

Santiago Pastorino (Apr 22 2019 at 20:48, on Zulip):

if there are bolder volunteers do you consider that reading what's currently in some parts of rustc plus investigating a bit would be a task to hand

Santiago Pastorino (Apr 22 2019 at 20:48, on Zulip):

or that's too hard? :)

nikomatsakis (Apr 22 2019 at 20:48, on Zulip):

not sure, might work

Santiago Pastorino (Apr 22 2019 at 20:49, on Zulip):

it may be a good idea to start gathering all the information that is around about certain topics

Santiago Pastorino (Apr 22 2019 at 20:49, on Zulip):

so ... we have rustc-guide

Santiago Pastorino (Apr 22 2019 at 20:49, on Zulip):

but there are already some lectures

Santiago Pastorino (Apr 22 2019 at 20:49, on Zulip):

like @oli about Miri, where there are some concepts that could be documented

Santiago Pastorino (Apr 22 2019 at 20:49, on Zulip):

and probably there are other lectures

Santiago Pastorino (Apr 22 2019 at 20:49, on Zulip):

and blog posts

Santiago Pastorino (Apr 22 2019 at 20:49, on Zulip):

and things that I may not know (?)

Santiago Pastorino (Apr 22 2019 at 20:50, on Zulip):

maybe a first thing could be having rustc-guide as the main source of knowledge

Santiago Pastorino (Apr 22 2019 at 20:50, on Zulip):

it probably already is, but what I mean is be sure that what's around it's already on rustc-guide

Santiago Pastorino (Apr 22 2019 at 20:50, on Zulip):

I'm not sure if that's true, you may know better :)

nikomatsakis (Apr 22 2019 at 20:51, on Zulip):

like oli about Miri, where there are some concepts that could be documented

yes, a good point

nikomatsakis (Apr 22 2019 at 20:51, on Zulip):

we can figure this out in the 'sprint planning meeting', I suspect

nikomatsakis (Apr 22 2019 at 20:52, on Zulip):

if I recall, @Iñaki Garay had a pretty thorough list of ideas here

Santiago Pastorino (Apr 22 2019 at 20:54, on Zulip):

yes

Iñaki Garay (Apr 22 2019 at 20:55, on Zulip):

There's already a significant backlog of tasks which can be parallelized. I don't think the wg will be blocked on expert bandwidth for some weeks. Eventually it might be an issue.

Santiago Pastorino (Apr 22 2019 at 20:55, on Zulip):

great :)

Iñaki Garay (Apr 22 2019 at 20:56, on Zulip):

I recently found more videos from other wg (e.g. RLS 2.0) I believe to be includable in the task list

Santiago Pastorino (Apr 22 2019 at 20:56, on Zulip):

@nikomatsakis I guess I can start adding more ideas to that list and working on a PR for the opening learning WG, then?

nikomatsakis (Apr 22 2019 at 20:57, on Zulip):

that sounds great @Santiago Pastorino

nikomatsakis (Apr 22 2019 at 20:57, on Zulip):

Do we think we need some kind of "first meeting"? Or can we get ourselves organized async?

nikomatsakis (Apr 22 2019 at 20:58, on Zulip):

I could imagine that the latter would work

nikomatsakis (Apr 22 2019 at 20:58, on Zulip):

(Having a stream etc would definitely help)

Santiago Pastorino (Apr 22 2019 at 21:00, on Zulip):

let's start async

Santiago Pastorino (Apr 22 2019 at 21:01, on Zulip):

let me go over the process of proposing this WG and after that let's see :)

Iñaki Garay (Apr 22 2019 at 21:02, on Zulip):

Let me know if there's anything I can help with. I'll wrap up the stages PR in the meantime

nikomatsakis (Apr 22 2019 at 21:14, on Zulip):

let's start async

I have a feeling this will work

Santiago Pastorino (Apr 23 2019 at 20:50, on Zulip):

@nikomatsakis @davidtwco I'm writing learning-WG proposal PR

Santiago Pastorino (Apr 23 2019 at 20:50, on Zulip):

I need to add a stream for it

Santiago Pastorino (Apr 23 2019 at 20:50, on Zulip):

what's the idea in general?, I guess in the PR I should leave a placeholder for the stream url until it's approved, right?

Santiago Pastorino (Apr 23 2019 at 20:52, on Zulip):

well, we have a diagnostics stream and the PR is still unmerged, so I guess it should be ok to start a stream and then merge the PR

Santiago Pastorino (Apr 23 2019 at 20:52, on Zulip):

(deleted)

davidtwco (Apr 23 2019 at 21:00, on Zulip):

The diagnostics stream existed before we did working groups.

davidtwco (Apr 23 2019 at 21:00, on Zulip):

It was just renamed.

nikomatsakis (Apr 23 2019 at 21:43, on Zulip):

@Santiago Pastorino I think we can create the stream before we merge the PR

Santiago Pastorino (Apr 23 2019 at 23:46, on Zulip):

@nikomatsakis I don't think I have rights to do so

Santiago Pastorino (Apr 24 2019 at 03:51, on Zulip):

btw https://github.com/rust-lang/compiler-team/pull/70

Amanjeev Sethi (Apr 24 2019 at 12:34, on Zulip):

let's create the stream already :grinning:

davidtwco (Apr 24 2019 at 12:51, on Zulip):

#t-compiler/wg-learning

Last update: Nov 21 2019 at 13:05UTC