Stream: t-compiler/help

Topic: cloning all the submodules?


mark-i-m (Jun 05 2020 at 15:57, on Zulip):

Is there a reason that everyone should have to clone all of the submodules when building? It takes a rather long time and wastes space, but I think it is only needed for building a distribution right?

simulacrum (Jun 05 2020 at 16:00, on Zulip):

kinda, cargo's lockfile resolution requires that you have Cargo.toml's around at least

simulacrum (Jun 05 2020 at 16:00, on Zulip):

(for the crates in the workspace anyway)

simulacrum (Jun 05 2020 at 16:01, on Zulip):

I do think that improvements here can and should be made, but it's not obvious what the right workflow is for some things

simulacrum (Jun 05 2020 at 16:01, on Zulip):

I have wanted to just entirely not have submodules in the past, and instead have a list of SHAs and repositories in some file

simulacrum (Jun 05 2020 at 16:01, on Zulip):

personally I think a good 70-80% of the problems with submodules are just because of the UI/UX that git imposes

mark-i-m (Jun 05 2020 at 16:33, on Zulip):

But why are they needed at all? Or at least, why can't we just list them in Cargo.toml as foo = { git = "something on github" }?

bjorn3 (Jun 05 2020 at 16:35, on Zulip):

Doing so would make it harder to change both rustc and a submodule at the same time. Also it will cause Cargo to clone the repo anyway.

mark-i-m (Jun 05 2020 at 16:36, on Zulip):

Oh, hmm... that's true

mark-i-m (Jun 05 2020 at 16:37, on Zulip):

But it still seems odd, e.g. that I need to clone rust-by-example to build the compiler... or the book or the rustc-dev-guide or llvm if I'm using system llvm

mark-i-m (Jun 05 2020 at 16:38, on Zulip):

Would it be hard to make these not cloned by default?

bjorn3 (Jun 05 2020 at 16:39, on Zulip):

For llvm x.py could skip the submodule update if you set config.toml to use the system LLVM. I believe it has to parse config.toml anyway.

mark-i-m (Jun 05 2020 at 17:00, on Zulip):

Also, I just noticed that one can set submodules = false in config.toml!

mark-i-m (Jun 05 2020 at 17:01, on Zulip):

Is there any reason we shouldn't recommend doing this?

mark-i-m (Jun 05 2020 at 17:01, on Zulip):

bjorn3 said:

For llvm x.py could skip the submodule update if you set config.toml to use the system LLVM. I believe it has to parse config.toml anyway.

Looking through the code, it appears this is already the case

Wesley Wiser (Jun 05 2020 at 17:02, on Zulip):

I think that would be ok if someone just wants to work on std or something but if they ever decide to do something with the compiler, they're probably going to run into weird errors

Wesley Wiser (Jun 05 2020 at 17:02, on Zulip):

Which can be frustrating as a newcomer

Wesley Wiser (Jun 05 2020 at 17:05, on Zulip):

In some respects, I think it's better to have a long-ish process that consistently works than to have a short-cut process that doesn't always work. You can start that long process and then go to bed or something while it works.

Wesley Wiser (Jun 05 2020 at 17:06, on Zulip):

I wonder if it would be worth while to investigate using shallow clones for some or all of those submodules though...

mark-i-m (Jun 05 2020 at 17:11, on Zulip):

I'm actually kind of conflicted here... on one hand I agree that having something that consistently work is best, but on the other hand long build times is one of the top things that people have complained about in the survey

mark-i-m (Jun 05 2020 at 17:12, on Zulip):

moreover, as I've been writing the "Getting started" PR, I'm finding that most of our recommended ways of doing things are "probably going to work, but could lead to weird issues"

Wesley Wiser (Jun 05 2020 at 17:16, on Zulip):

Yeah, that's totally fair! I don't want to stand in the way of progress, just providing a different perspective :slight_smile:

mark-i-m (Jun 05 2020 at 19:04, on Zulip):

Not at all :)

mark-i-m (Jun 05 2020 at 19:13, on Zulip):

I think the major difficulty is that the only really correct way to compile the compiler is _really_ slow, and so we have all of these hacky shortcuts that usually work, but sometimes not... and that's what we actually have to teach people

simulacrum (Jun 05 2020 at 20:06, on Zulip):

I think getting a document put together which lays out the "right way" to do a bunch of workflows would be a great start, because right now it's really hard (for me at least) to even know what to optimize

simulacrum (Jun 05 2020 at 20:06, on Zulip):

like I know I have things that sometimes annoy me, but I doubt that they're the primary concern of most newcomers

simulacrum (Jun 05 2020 at 20:07, on Zulip):

and when we do update things it'd be much better if we could get the word out by updating a central document and pinging compiler team rather than very slowly that knowledge percolating through the compiler team and eventually reaching new contributors

Last update: Sep 28 2020 at 15:45UTC