Stream: t-compiler

Topic: bootstrapping in windows


pnkfelix (Apr 08 2019 at 09:49, on Zulip):

Hey @Zoxc , do you mind if I ask you some newbie questions about trying to bootstrap rustc in windows?

Zoxc (Apr 08 2019 at 09:50, on Zulip):

Fire away. msvc or gnu? =P

pnkfelix (Apr 08 2019 at 09:51, on Zulip):

I'm trying to do msvc

pnkfelix (Apr 08 2019 at 09:52, on Zulip):

currently when I run C:\Python27\python.exe x.py build, ...

pnkfelix (Apr 08 2019 at 09:52, on Zulip):

I get a bunch of errors

pnkfelix (Apr 08 2019 at 09:52, on Zulip):

saying "SecurityProtocol: Cannot convert null to type System.Net.SecurityProtocolType" due to invalid enumeration values"

pnkfelix (Apr 08 2019 at 09:53, on Zulip):

and then that seems like its showing me a pseudo-call stack saying that is arising due to a call to DownloadFile

pnkfelix (Apr 08 2019 at 09:53, on Zulip):

where its trying to download the beta build for bootstrapping

pnkfelix (Apr 08 2019 at 09:53, on Zulip):

(static.rust-lang.org/dist/2019-03-20/rust-std-beta-x86_64-pc-windows-msvc.tar.gz.sha256)

pnkfelix (Apr 08 2019 at 09:54, on Zulip):

I cannot easily cut-and-paste the text itself, so I've been trying to transcribe it. Maybe I should post a screen grab of the window ...)

Zoxc (Apr 08 2019 at 09:55, on Zulip):

Are you on the latest Wimdows 10? Might be a PowerShell version mismatch

pnkfelix (Apr 08 2019 at 09:56, on Zulip):

Windows 7 i think

pnkfelix (Apr 08 2019 at 09:56, on Zulip):

yeah Windows 7 Sevice Pack 1

pnkfelix (Apr 08 2019 at 09:56, on Zulip):

You recommend using Windows 10 instead?

pnkfelix (Apr 08 2019 at 09:57, on Zulip):

I'm not wedded to any particular Windows version; I just had thought from past experience than when using a VirtualBox or VMWare like setup, you're usually better off going with an older version of Windows due to the performance overhead of virtualization

Zoxc (Apr 08 2019 at 09:59, on Zulip):

Try installing a newer PowerShell or .NET version

pnkfelix (Apr 08 2019 at 10:01, on Zulip):

okay, but don't worry about upgrading my Windows 7 yet? I'll see, thanks!

Zoxc (Apr 08 2019 at 10:02, on Zulip):

Windows 7 should work fine, but I haven't tried it

pnkfelix (Apr 08 2019 at 10:04, on Zulip):

hmm. there's this table with four different links depending on whether one wants PS (PowerShell) 3.0, 4.0, 5.0, or 5.1

pnkfelix (Apr 08 2019 at 10:04, on Zulip):

which one do you have? I guessI could jump right to 5.1

pnkfelix (Apr 08 2019 at 10:04, on Zulip):

(technically I guess its the version of the Window Management Framework (WMF)

Zoxc (Apr 08 2019 at 10:05, on Zulip):

5.1

pnkfelix (Apr 08 2019 at 10:31, on Zulip):

thanks @Zoxc ! Now I've gotten past the download step. (And have hit an entirely new error, of course)

Zoxc (Apr 08 2019 at 10:45, on Zulip):

Did you fix it too? =P

pnkfelix (Apr 08 2019 at 10:48, on Zulip):

no

pnkfelix (Apr 08 2019 at 10:48, on Zulip):

at this point I'm downloading a Windows 10 VM

pnkfelix (Apr 08 2019 at 10:49, on Zulip):

rather than keep trying to dissect every thing that pops up

Zoxc (Apr 08 2019 at 10:49, on Zulip):

Which error did you get? You should get a VM with copy and paste =P

pnkfelix (Apr 08 2019 at 10:50, on Zulip):

I couldn't exactly tell what the error was. the message was pretty malformed

pnkfelix (Apr 08 2019 at 10:50, on Zulip):

it was complaining that the command I was trying to run was not compatible with my version of windows

pnkfelix (Apr 08 2019 at 10:50, on Zulip):

but it wrote "%1" rather than the command name

pnkfelix (Apr 08 2019 at 10:50, on Zulip):

in the diagnostic

pnkfelix (Apr 08 2019 at 10:50, on Zulip):

which I assume is some sort of interpolation bug

Zoxc (Apr 08 2019 at 10:51, on Zulip):

:/

pnkfelix (Apr 08 2019 at 10:51, on Zulip):

e.g. someone writing a message assuming .bat semantics, maybe?

pnkfelix (Apr 08 2019 at 10:53, on Zulip):

Screen-Shot-2019-04-08-at-12.52.43.png

pnkfelix (Apr 08 2019 at 10:54, on Zulip):

I will admit that my problem may stem from using VS 2019

pnkfelix (Apr 08 2019 at 10:54, on Zulip):

I did see the note saying that bootstrapping only works in certain (older) versions of Visual Studio

Zoxc (Apr 08 2019 at 10:55, on Zulip):

Seems like something is trying to run x64-64 binaries on x86-32 Windows

pnkfelix (Apr 08 2019 at 10:55, on Zulip):

but the path here (https://github.com/rust-lang/rust#msvc) is not present on my (admittedly, free aka no-frills) Visual Studio install

pnkfelix (Apr 08 2019 at 10:55, on Zulip):

hmm maybe it is indeed a 32-bit version, let me see

pnkfelix (Apr 08 2019 at 10:56, on Zulip):

oh yeah this is absolutely 32-bit

pnkfelix (Apr 08 2019 at 10:56, on Zulip):

so running vcvars64.bat is certainly not going to work then

Zoxc (Apr 08 2019 at 10:56, on Zulip):

You'll want https://github.com/Zoxc/rust/tree/vs2019 for VS 2019 support =P

pnkfelix (Apr 08 2019 at 10:57, on Zulip):

is there a free version of VS that doesn't require using a branch off of master?

Zoxc (Apr 08 2019 at 10:58, on Zulip):

VS 2017 Community

Zoxc (Apr 08 2019 at 10:59, on Zulip):

Got to wait for @Alex Crichton to wake up for VS 2019 support on master

pnkfelix (Apr 08 2019 at 10:59, on Zulip):

let me see if I can grab that...

pnkfelix (Apr 08 2019 at 11:04, on Zulip):

I can't find an installer for VS 2017 Community. Microsoft seems to be pushing 2019 hard. :sad:

Zoxc (Apr 08 2019 at 11:04, on Zulip):

https://visualstudio.microsoft.com/downloads/

pnkfelix (Apr 08 2019 at 11:06, on Zulip):

I tried that. even though it has a separate section for 2017, ... the Installer app it then gives you does a search for "Available" versions

pnkfelix (Apr 08 2019 at 11:06, on Zulip):

and only shows 2019 for me. :sad:

pnkfelix (Apr 08 2019 at 11:07, on Zulip):

(looking)

pnkfelix (Apr 08 2019 at 11:08, on Zulip):

going to "older downlolads" takes me to a "My Benefits" page under my.visualstudio.com that says "Sorry, we couldn't find any downloads for you."

pnkfelix (Apr 08 2019 at 11:09, on Zulip):

/me should probably set a timer to limit how much more time he invests in this today

pnkfelix (Apr 08 2019 at 11:18, on Zulip):

weird, I just saw a glimpse of 2017 under a different run of the installer.

pnkfelix (Apr 08 2019 at 11:18, on Zulip):

okay so let me see if I can get that working

pnkfelix (Apr 08 2019 at 11:24, on Zulip):

(oh wow, moving my VirtualBox VM's window off of my laptop screen and onto the external monitor's screen seems to have made the UX way way way more responsive...)

Daniel Carosone (Apr 08 2019 at 11:37, on Zulip):

is the laptop high-dpi and the monitor not? maybe it's doing some really inefficient scaling?

pnkfelix (Apr 08 2019 at 11:46, on Zulip):

downgrading from VirtualBox 6.0 to VirtualBox 5.2 seems like it may be a workaround

nagisa (Apr 08 2019 at 16:05, on Zulip):

yeah Windows 7 Sevice Pack 1

I have successfully built Rust on Windows 7

nagisa (Apr 08 2019 at 16:05, on Zulip):

nothing special happened

nagisa (Apr 08 2019 at 16:06, on Zulip):

That was a longish time ago tho

pnkfelix (Apr 09 2019 at 13:51, on Zulip):

I am having a more productive experience with my attempt atop Windows 10

Zoxc (Apr 09 2019 at 13:52, on Zulip):

Are you producing more data for Microsoft?

pnkfelix (Apr 09 2019 at 13:52, on Zulip):

but this may be in part because Microsoft's set of free 90-day VirtualBox VM's only has 32-bit x86 for Windows 7. If you want/need x86_64, then you need to grab their Windows 10 VM

pnkfelix (Apr 09 2019 at 13:53, on Zulip):

my attempt atop Windows 10 is currently building LLVM. I never got that far with my 32-bit Windows 7 VM

Zoxc (Apr 11 2019 at 14:01, on Zulip):

VS 2019 now work on master =P
Have you tried VMware?

pnkfelix (Apr 11 2019 at 14:02, on Zulip):

in the past I used VMware

pnkfelix (Apr 11 2019 at 14:02, on Zulip):

and maybe I should indeed use it now, since I suspect its performance is superior to VirtualBox

pnkfelix (Apr 11 2019 at 14:03, on Zulip):

but various points in Mozilla's and/or Microsoft's infrastructure were pushing me to VirtualBox here

pnkfelix (May 03 2019 at 10:10, on Zulip):

okay so at this point I've gotten a used Windows laptop

pnkfelix (May 03 2019 at 10:10, on Zulip):

and I've installed both VS Studio 2017 Express edition, and VS Studio 2019 Community edition

pnkfelix (May 03 2019 at 10:10, on Zulip):

I'm trying to figure out how to do this without purchasing anything

pnkfelix (May 03 2019 at 10:11, on Zulip):

(to best emulate what some of our contributors need to do)

pnkfelix (May 03 2019 at 10:11, on Zulip):

and I still have not managed to bootstrap Rust

pnkfelix (May 03 2019 at 10:11, on Zulip):

mostly because I keep hitting what I assume are environment setup issues

Zoxc (May 03 2019 at 10:15, on Zulip):

Which issues? =P

pnkfelix (May 03 2019 at 10:16, on Zulip):

well

pnkfelix (May 03 2019 at 10:16, on Zulip):

my "most successful" runs tend to end with a link step problem

pnkfelix (May 03 2019 at 10:16, on Zulip):

with a mismatch between x86 and x64

pnkfelix (May 03 2019 at 10:16, on Zulip):

I'm doing another run right now, in order to transcribe the actual message

pnkfelix (May 03 2019 at 10:17, on Zulip):

but my main question, for anyone who like me is trying to bootstrap atop windows using free Visual Studio downloads, is: What command prompt are you starting with?

pnkfelix (May 03 2019 at 10:18, on Zulip):

my current run is using "x64 Native Tools Command Prompt for VS 2019"

Zoxc (May 03 2019 at 10:18, on Zulip):

I use plain powershell with python, cmake and git in PATH

pnkfelix (May 03 2019 at 10:19, on Zulip):

I assume after running powershell, you need to invoke some other script to set up environment variables?

Zoxc (May 03 2019 at 10:19, on Zulip):

cmake, the cc crate and rustc will detect and use VS tools on their own

pnkfelix (May 03 2019 at 10:19, on Zulip):

something analogous to vcvarsall.bat?

pnkfelix (May 03 2019 at 10:19, on Zulip):

hmm

pnkfelix (May 03 2019 at 10:19, on Zulip):

okay let me try powershell then

Zoxc (May 03 2019 at 10:24, on Zulip):

I'll use a msys2 shell if I need to run run-make tests, as those need bash

pnkfelix (May 03 2019 at 10:24, on Zulip):

just to be clear ...

pnkfelix (May 03 2019 at 10:25, on Zulip):

the detection that the cmake, cc, rustc crates do ... you're saying that works only in powershell?

pnkfelix (May 03 2019 at 10:25, on Zulip):

or it supposedly works in any terminal?

pnkfelix (May 03 2019 at 10:25, on Zulip):

because ... if its any terminal ... then I would imagine I'm going to run into the same essential problem.

Zoxc (May 03 2019 at 10:25, on Zulip):

It works in any terminal

Zoxc (May 03 2019 at 10:25, on Zulip):

But vcvarsall.bat might do something that overrides them

Zoxc (May 03 2019 at 10:26, on Zulip):

Are you on x64 now?

pnkfelix (May 03 2019 at 10:26, on Zulip):

Are you using Visual Studio as your compiler? Or are you using ... what's it called ...

pnkfelix (May 03 2019 at 10:26, on Zulip):

... whatever the way to get gcc and/or clang atop windows is ...?

Zoxc (May 03 2019 at 10:27, on Zulip):

I use msvc not mingw

pnkfelix (May 03 2019 at 10:27, on Zulip):

okay.

pnkfelix (May 03 2019 at 10:28, on Zulip):

did you do any customization of your environment that you can remember, in terms of what environment variables are set by default in your shell?

pnkfelix (May 03 2019 at 10:28, on Zulip):

or are you using whatever the vanilla install of MSVC?

pnkfelix (May 03 2019 at 10:28, on Zulip):

(and which MSVC is it? Are you on 2019 Community?)

pnkfelix (May 03 2019 at 10:29, on Zulip):

(sorry for all the questions; I'm trying to gather as much info as I can up front so I can replicate a supposedly known-to-work environment)

Zoxc (May 03 2019 at 10:29, on Zulip):

clang-cl is msvc-like just for the record

Zoxc (May 03 2019 at 10:29, on Zulip):

I haven't done any setup for environment variables specific to Rust

Zoxc (May 03 2019 at 10:30, on Zulip):

I have VS 2017 and VS 2019 now, not sure which is used, but it worked when only VS 2019 was installed

pnkfelix (May 03 2019 at 10:33, on Zulip):

okay

pnkfelix (May 03 2019 at 10:34, on Zulip):

I've been meaning to look more at Powershell anyway, especially after wycats' tweets raving about it

Zoxc (May 03 2019 at 10:34, on Zulip):

It's terrible. I just use it because it's VS Code's default, and I just run x.py in it =P

pnkfelix (May 03 2019 at 11:57, on Zulip):

so you run as a subshell of a VS Code editor instance?

pnkfelix (May 03 2019 at 11:58, on Zulip):

(that may be an important detail; its possible VS Code sets up envionment variables. I've just been trying to run in vanilla terminals, either cmd.exe, or the various bat scripts added to the Start Menu by Visual Studio install(s)

Zoxc (May 03 2019 at 12:03, on Zulip):

I use the VS Code terminal yes. That should not be important

Zoxc (May 03 2019 at 12:05, on Zulip):

A build from a clean slate (no build folder) with cmd.exe or powershell should not fail. File a bug if it's not from missing tools

pnkfelix (May 03 2019 at 12:11, on Zulip):

are you saying that the current readme is wrong, here: https://github.com/rust-lang/rust/blob/master/README.md#msvc

pnkfelix (May 03 2019 at 12:12, on Zulip):

where it says you may need to call the appropriate vcvars file before running the bootstrap?

Zoxc (May 03 2019 at 12:12, on Zulip):

Visual Studio 2013 is a bit of a clue =P

pnkfelix (May 03 2019 at 12:12, on Zulip):

or are you just saying that VS 2019 is one of the Visual Studio versions that should (now) work out of the box

pnkfelix (May 03 2019 at 12:12, on Zulip):

yeah well there is that "(or later)" next to that. :)

Zoxc (May 03 2019 at 12:13, on Zulip):

Probably dates the instructions to 2012

pnkfelix (May 03 2019 at 12:13, on Zulip):

it'd be nice if we actually listed the versions that we know work

pnkfelix (May 03 2019 at 12:13, on Zulip):

or maybe just the ones that we use for the CI testing

Zoxc (May 03 2019 at 12:14, on Zulip):

Our MSVC CI testing is a bit strange, it runs inside msys2 :/

Zoxc (May 03 2019 at 12:14, on Zulip):

I do not recommend that

pnkfelix (May 03 2019 at 12:15, on Zulip):

hmm. and the CI builds might be inheriting environment state from the outer msys2, right?

Zoxc (May 03 2019 at 12:17, on Zulip):

Not might. They are

pnkfelix (May 03 2019 at 12:17, on Zulip):

yeah that's unfortunate

pnkfelix (May 03 2019 at 12:18, on Zulip):

I can understand having to fire off a build as some sort of subshell of msys2

pnkfelix (May 03 2019 at 12:18, on Zulip):

just as an artifact of travis

pnkfelix (May 03 2019 at 12:18, on Zulip):

but it would be good to find some way to isolate the inner subshell's environment state

pnkfelix (May 03 2019 at 12:47, on Zulip):

(i am now wondering if my earlier environmental link issues are in fact because i kept running vcvarsall.bat or friends while trying to use VS 2019, based on earlier evidence that doing so was necessary for VS 2017)

pnkfelix (May 03 2019 at 12:48, on Zulip):

I still haven't seen a successful bootstrap, but my most recent problems may well originate from my attempts to run x.py outside of the root of the git checkout of rust.git

pnkfelix (May 03 2019 at 13:42, on Zulip):

Giving up for today; filed "cl.exe handling of llvm\include broken for VS 2019?" #60507

Zoxc (May 03 2019 at 13:44, on Zulip):

Looks like something doesn't handle spaces correctly

Zoxc (May 03 2019 at 13:50, on Zulip):

So probably don't put the repo in a path with spaces

eddyb (May 03 2019 at 13:55, on Zulip):

does anything work correctly with paths in spaces?

eddyb (May 03 2019 at 13:55, on Zulip):

I keep hearing about this in various situations

eddyb (May 03 2019 at 13:55, on Zulip):

(the sad answer is probably "malware, if that")

Zoxc (May 03 2019 at 13:57, on Zulip):

Plenty of things works correctly with spaces in path. Unix build systems and shell scripts always fail horribly though

Zoxc (May 03 2019 at 13:59, on Zulip):

They're pretty much perfectly designed to induce bugs

eddyb (May 03 2019 at 14:00, on Zulip):

there are ways to make it work, but, yeah, using spaces to separate things was a mistake

eddyb (May 03 2019 at 14:00, on Zulip):

you have to do silly things like "$foo"

pnkfelix (May 03 2019 at 14:04, on Zulip):

What do people do here, put their git checkout into C:\rust.git\ ?

eddyb (May 03 2019 at 14:04, on Zulip):

well, C:\Users\eddy\Projects doesn't have any spaces in it :P

Wesley Wiser (May 03 2019 at 14:05, on Zulip):

I used C:\Users\wwiser\Code\rust

eddyb (May 03 2019 at 14:05, on Zulip):

but, yeah, especially if you have two partitions, I wouldn't be surprised by D:\Projects\rust

Zoxc (May 03 2019 at 14:05, on Zulip):

I store all my data on a separate partition from C:

Zoxc (May 03 2019 at 14:06, on Zulip):

I also don't have a username with a space =P

pnkfelix (May 03 2019 at 15:00, on Zulip):

I got my machine with the account already set up with a space in my user name :(

pnkfelix (May 03 2019 at 15:00, on Zulip):

and I decided it would not be worth the effort to attempt to revise thaet

pnkfelix (May 03 2019 at 15:01, on Zulip):

AFAICT one can not easily rename themselves in Windows.

eddyb (May 03 2019 at 15:01, on Zulip):

maybe you can make a directory hard link?

eddyb (May 03 2019 at 15:01, on Zulip):

in NTFS?

pnkfelix (May 03 2019 at 15:01, on Zulip):

are we really not going to support this use case?

eddyb (May 03 2019 at 15:01, on Zulip):

so you run stuff from C:\Users\felix\... etc.

pnkfelix (May 03 2019 at 15:02, on Zulip):

i guess this maybe worthy of a meta-policy discussion at a higher level.

pnkfelix (May 03 2019 at 15:02, on Zulip):

at the very least, something to mention at the compiler team meeting. Maybe directory hard links are indeed an appropriate work around

pnkfelix (May 03 2019 at 15:02, on Zulip):

but ... what about a developer who does not have Admin rights to their workstation?

eddyb (May 03 2019 at 15:02, on Zulip):

I agree with fixing spaces in paths, but this is assuming we can do anything about it

eddyb (May 03 2019 at 15:03, on Zulip):

I think anyone can create a directory under C: btw, but I could be wrong

pnkfelix (May 03 2019 at 15:03, on Zulip):

I've done it before, at Adobe, i think by striving to switch to relative paths

pnkfelix (May 03 2019 at 15:03, on Zulip):

but maybe that is not always an option

pnkfelix (May 03 2019 at 15:03, on Zulip):

(e.g. when the relative/absolute path is embedded in the final product)

Wesley Wiser (May 03 2019 at 15:04, on Zulip):

I agree that we should fix this to support spaces in paths. However, it seems like there should be another way to create a folder without any spaces in the path.

Wesley Wiser (May 03 2019 at 15:05, on Zulip):

At least on my pc, I'm not prompted for elevated privileges to create a folder at C:\

pnkfelix (May 03 2019 at 15:07, on Zulip):

(This also is probably going to prompt me to file a bug with Mozilla IT on how they set up Windows Laptops; they really should have asked me for a user name)

Wesley Wiser (May 03 2019 at 15:07, on Zulip):

I am prompted if I try to create a new folder under C:\Users. So at least on Win 10, creating a folder at the root of the drive might be a workaround?

eddyb (May 03 2019 at 15:08, on Zulip):

I remember being surprised I can just do this in XP/Win7

eddyb (May 03 2019 at 15:08, on Zulip):

so this should work in every version, assuming we're not confused about it

Wesley Wiser (May 03 2019 at 15:08, on Zulip):

Creating a folder at the root is also "nice" because it helps you avoid path length limitations

Wesley Wiser (May 03 2019 at 15:11, on Zulip):

(MAX_PATH = 260 characters unless you use "long paths" which are the \\?\ style paths the win32 api supports)

eddyb (May 03 2019 at 15:15, on Zulip):

yeah I think the compiler uses those long paths a lot

Wesley Wiser (May 03 2019 at 15:20, on Zulip):

It should be using \\?\ style paths at the appropriate places just like it should handle paths with spaces in them. ¯\_(ツ)_/¯

nagisa (May 03 2019 at 21:21, on Zulip):

I remember being surprised I can just do this in XP/Win7

You cannot "just" create a directory under C: without being an administrative account under W10

nagisa (May 03 2019 at 21:22, on Zulip):

(in general W10 is now sane-by-default as far as things you can do to system-like directories go)

nagisa (May 03 2019 at 21:22, on Zulip):

@pnkfelix I usually build rustc in a shell that is specifically launched by finding a "x64 shell" somewhere in the start menu

nagisa (May 03 2019 at 21:23, on Zulip):

if I cannot find it there, I go and dig around Program Files for it.

eddyb (May 03 2019 at 21:44, on Zulip):

that's the Visual C++ Tools thing, right?

nagisa (May 03 2019 at 23:54, on Zulip):

Yeah, something like that.

Last update: Nov 16 2019 at 01:15UTC