Stream: t-compiler/wg-rls-2.0

Topic: ARM build?


qmx (Feb 13 2020 at 00:13, on Zulip):

Hey all! With the new behavior of the extensions of trying to always download the latest from the releases, would y'all be willing to run some ARM builds too? I can do the legwork of setting up the extra CI tasks for it using docker buildx multiarch support if that's the case

matklad (Feb 13 2020 at 09:33, on Zulip):

Yep, providing arm builds will be sweet. I'd love to try to avoid docker though, and rely on cross-compilation. rust-analyzer does not depend on C code, so that in theory should be not too hard

qmx (Feb 13 2020 at 14:21, on Zulip):

cool, I'll give it a try and report back

qmx (Feb 19 2020 at 14:53, on Zulip):

Should I add the arm build as another "os" on the matrix?

qmx (Feb 19 2020 at 14:54, on Zulip):

I'm leaning towards this option because you have to install the cross compilation gcc toolchain

Laurențiu Nicola (Feb 19 2020 at 15:02, on Zulip):

How does that work? GitHub Actions has no ARM support and I don't think they'll invoke the workflow with an unsupported OS.

Ah, runs-on vs. os, got it.

std::Veetaha (Feb 19 2020 at 15:04, on Zulip):

The thing that matters is the time to build and deploy. I guess xcompiling should be run in parallel with other jobs, or do we compile x64 and arm in the same process?

Laurențiu Nicola (Feb 19 2020 at 15:06, on Zulip):

I think it can be either an extra set of steps gated on os == 'ubuntu-latest', but I'm not sure how to run it in parallel via matrix. Well, don't mind me.

std::Veetaha (Feb 19 2020 at 15:07, on Zulip):

Matrix item combinations are run in parallel by itself, but I am not sure

Laurențiu Nicola (Feb 19 2020 at 15:08, on Zulip):

But we'd like "runs-on: ${{ matrix.os }} if not ARM, else ubuntu-latest", which..

Laurențiu Nicola (Feb 19 2020 at 15:09, on Zulip):

Make it a different job, maybe? Those also run in parallel

Laurențiu Nicola (Feb 19 2020 at 15:10, on Zulip):

The current release workflow builds the server binaries and the Code extension in parallel, then pushes a release when both are finished.

std::Veetaha (Feb 19 2020 at 15:14, on Zulip):

I guess we could just have

 strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
       arch: [x64, arm]

      exclude:
          - os: macos-latest
            arch: arm
std::Veetaha (Feb 19 2020 at 15:16, on Zulip):

https://github.community/t5/GitHub-Actions/How-to-not-Allow-Some-Matrices-Together/td-p/40625

matklad (Feb 19 2020 at 15:17, on Zulip):

I'm leaning towards this option because you have to install the cross compilation gcc toolchain

Why is that? We shouldn't have any C code?

qmx (Feb 19 2020 at 15:19, on Zulip):

libbacktrace

matklad (Feb 19 2020 at 15:21, on Zulip):

Ahhhhhhh.... Can we use clang instead of gcc?

matklad (Feb 19 2020 at 15:21, on Zulip):

Clang is a native cross compiler

qmx (Feb 19 2020 at 15:21, on Zulip):

I can try!

matklad (Feb 19 2020 at 15:31, on Zulip):

Ok, we depend on libbacktarce by accident! I'll send a PR to disable it as soon as GitHub stops throwing 500s at me

std::Veetaha (Feb 19 2020 at 15:36, on Zulip):

Aha, I do have the same problems

std::Veetaha (Feb 19 2020 at 15:36, on Zulip):

500 when I try to merge one PR

std::Veetaha (Feb 19 2020 at 15:39, on Zulip):

https://www.githubstatus.com/ pasted image

qmx (Feb 19 2020 at 15:55, on Zulip):

also the github action we use for installing the musl target only supports a single target, I'm working on a PR to allow for targets instead

matklad (Feb 19 2020 at 15:55, on Zulip):

@qmx can't we just call rustup target add manually?

matklad (Feb 19 2020 at 15:55, on Zulip):

I've send a PR which disables libbacktrace

matklad (Feb 19 2020 at 15:56, on Zulip):

I think that means that we should just be able to do something like RUSTFLAGS="-C linker=rust-lld -C linker-flavor=ld.lld" cargo build -p rust-analyzer --target aarch64-unknown-linux-musl

Laurențiu Nicola (Feb 19 2020 at 15:59, on Zulip):

I think it can't find pthread, rt and friends Ah, nevermind.

Laurențiu Nicola (Feb 19 2020 at 16:00, on Zulip):

Just __addtf3, __multf3 and __subtf3

Laurențiu Nicola (Feb 19 2020 at 16:01, on Zulip):

https://github.com/rust-lang/compiler-builtins/issues/201

qmx (Feb 19 2020 at 16:01, on Zulip):

sure we can, I'm just trying to minimize the amount of custom configurations we have to have for this extra architecture

qmx (Feb 19 2020 at 16:03, on Zulip):

@matklad cool, using rust-lld as the linker works!

qmx (Feb 19 2020 at 16:04, on Zulip):

it'd be really cool if cargo already did this linker choice by default on a cross compilation scenario

qmx (Feb 19 2020 at 16:06, on Zulip):

also verified that your backtrace branch / PR was the last standing native bits on rust-analyzer

qmx (Feb 19 2020 at 16:10, on Zulip):

@matklad hmm, the PR is green but bors didn't pick it up - do you need to @ the bot?

matklad (Feb 19 2020 at 16:13, on Zulip):

seems like there's still problems with delivering hooks. merged manally

matklad (Feb 19 2020 at 17:34, on Zulip):

Should I add the arm build as another "os" on the matrix?

it is important that cross build runs in parallel with our other builds, such that it doesn't slow down the release

qmx (Feb 19 2020 at 17:40, on Zulip):

I'll take this as the constraint to guide the solution then

matklad (Feb 19 2020 at 17:42, on Zulip):

It's not really a constraint, just an important nice-to-have

matklad (Feb 19 2020 at 17:45, on Zulip):

Hm, this actually makes me think....

Should we cross-compile to windows from ubuntu as well? The ubuntu build is faster, so cross compilation might actually be a net win?

matklad (Feb 19 2020 at 17:47, on Zulip):

Ohhhh, If we can xcompile from ubuntu to all our targets, we can move most of the release process to xtask...

But I think at least cross-compiling to osx requires installing some extra stuff?

matklad (Feb 19 2020 at 21:46, on Zulip):

Added a check to CI to make sure we don't regress no c state: https://github.com/rust-analyzer/rust-analyzer/pull/3242

was quite surprised to learn that we build libbacktrace by default :D

Laurențiu Nicola (Feb 19 2020 at 21:58, on Zulip):

It's still not used, right?

matklad (Feb 19 2020 at 22:02, on Zulip):

right, it was used in a debug-only fn which was never called

Vincent Rouillé (Feb 20 2020 at 14:56, on Zulip):

Windows user here: I'm using lld-link to link everything rust produce and have done it without a hitch for two years now :) It really speed up global compile time (link.exe is slooooww).

Laurențiu Nicola (Feb 20 2020 at 15:17, on Zulip):

Well, RA is too much even for lld, some of the crates take some 60-80 seconds for ThinLTO and linking

Jeremy Kolb (Feb 20 2020 at 18:04, on Zulip):

@Vincent Rouillé I haven't tried that on windows. Is t hat just for arm of x64 too?

Vincent Rouillé (Feb 20 2020 at 22:35, on Zulip):

For example, FreeBSD is currently using lld as default ELF linker on amd64, arm*, aarch64 and powerpc64
In lld 7 (we are at 9/10 now) release note it says: "lld 7 for ELF, COFF and MinGW are production-ready."
In lld 9 they state: "The Linux kernel for arm32_7, arm64, ppc64le and x86_64 can now be linked by lld."
So it should go quite well.

Vincent Rouillé (Feb 20 2020 at 22:37, on Zulip):

If I recall correctly it took them some times to implements most of ld scripts, while the ELF part was quite complete.

qmx (Mar 13 2020 at 23:59, on Zulip):

this should make the github action a little bit cleaner: https://github.com/actions-rs/toolchain/pull/62

qmx (Mar 13 2020 at 23:59, on Zulip):

let's hope the maintainers accept it

qmx (Mar 18 2020 at 11:38, on Zulip):

in the mean time, another question - how should the file naming strategy be? currently we have rust-analyzer-linux - would we just append a new binary rust-analyzer-linux-armv7 to the release?

qmx (Mar 18 2020 at 11:39, on Zulip):

on the same note, is the auto-update feature within vscode architecture-aware?

matklad (Mar 18 2020 at 11:39, on Zulip):

Yeah, I think rust-analyzer-linux-armv7 would do. We probably should change the others to -x64, but that can be done separatelly

matklad (Mar 18 2020 at 11:39, on Zulip):

Yup, auto-update is arch-aware

qmx (Mar 18 2020 at 11:45, on Zulip):

I was just looking here and the last update for actions-rs/toolchain was two months ago :(

qmx (Mar 19 2020 at 01:09, on Zulip):

any thoughts on just naming the binaries with their equivalent rust target-triple?

qmx (Mar 19 2020 at 01:10, on Zulip):

that could remove a bunch of "hardcoding" on build scripts and allow for new targets to be added later with minimal changes

matklad (Mar 19 2020 at 06:55, on Zulip):

that could remove a bunch of "hardcoding" on build scripts and allow for new targets to be added later with minimal changes

This will make the TypeScript side more complicated though. Also, binaries are public facing (downloading them manually from github is supported), and target tripples look scary for not already familiar with them

matklad (Mar 19 2020 at 06:56, on Zulip):

So I would probably stick with minimal naming. It's not that we are going to add a whole lot of targets

Last update: Jun 07 2020 at 10:25UTC