Somebody managed to bootstrap rustc from mrustc. It's not fully reproducible yet, but definitely a step in the right direction. https://www.gnu.org/software/guix/blog/2018/bootstrapping-rust/
also re: the overall topic, if anyone knows the state of
rustc in that regard, I'm curious
They discuss it briefly in the blog post
"Rust takes reproducible builds seriously, but there are some reproducibility problems left in earlier compilers that pop up very sporadically (mostly because of LLVM, and some because of Rust hashtable poisoning). Help wanted, especially from LLVM people!"
more specifically I'd be curious about a QEMU (or otherwise) recipe which, given a cargo project, always produces the same binary, byte-for-byte
Anyone familiar with
it looks very interesting but I can't find any information about it
here's something using it https://github.com/kpcyrd/sniffglue/blob/master/ci/reprotest.sh
I guess it's this? https://pypi.org/project/reprotest/
Question: if I make a binary with a Cargo.lock in the repo, and then publish that binary to crates.io, does that Cargo.lock propagate to users, or whenever someone types
cargo install my-program they actually get latest semver-compatible versions?
You can't publish binaries to crates.io at all, can you? I'd assume any
Cargo.lock you have around is ignored.
Right. I meant executables that are built from source on the user's machine. So Cargo.lock is not included in the upload to crates.io?
no, it can't be because Cargo.lock's resolution depends on the constraints of all of the other dependencies
You can definitely publish binary components in crates, without source code. It is done by some crates. The only requirement is that your Cargo.toml must generate something; you can include whatever else you want.
I don't know about that. Probably your build.rs has to build an executable. But you can have another executable included that it copies to the output and executes, I think.
I know you can have binary libraries (*.a) that you ship in your crate, for example.
on the other hand, I don't know what I expected
My understanding is that the new golang build/module system will require that what is in github matches what you have.
But, even then, I don't know if it lets you put your *.a and *.exe files in github and then says "OK". I'm guessing not.
I will bypass any heuristics they make up
the only thing that could sorta-reliably stop it is manual review
there are like
two concurrent threads I was trying to get to cross streams, heh
@Shnatsel's embedding Cargo.lock and such in binaries
and this one: https://github.com/rust-lang/cargo/issues/5654
because I think you can take @Shnatsel's thing and put it in a crate and solve ^^^ problem
anyone familiar with
guix challenge? https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-challenge.html
Well, I made a repo/crate for this: https://github.com/rust-secure-code/cargo-repro
it doesn't do anything yet :wink:
would love to gather input here: https://github.com/rust-secure-code/cargo-repro/issues/3
Nice! I just came back from vacation. Just some questions to get a good feeling for the goal of this tool (I might have missed some prior conversations here and there :))
Is it used to check if packages are the same (as in: project code and .zip archives downloaded from crates.io) or to check if binaries are the same? Or even mixes between them? (Like to check if an archive outputs the same binary?)
Of course I guess checking binaries really depends on the compiler that is used right?
to me it's two parts:
1) building binaries which are reproducible, and gathering the necessary environmental information to reproduce them. the former part is largely just
cargo build --locked but I think such a tool should try to collect any potential sources of nondeterminism and potentially encode them into the resulting binary
2) verifying a build is actually reproducible, given the source tree it was built from as a cargo project. there's
reprotest for this, but it's a bit cumbersome to use and also trying to be a general purpose reproducible build utility
mostly the problem to me is reproducible builds in Rust right now seem pretty cargo cult and I haven't even gotten one to work myself yet, so I'd like a tool that at least encodes the "recipe" for doing so somewhere other than a bunch of forum posts and GitHub issues
"cargo cult" yeesh unintentional pun :sweat_smile:
some very, very preliminary functionality in this PR: https://github.com/rust-secure-code/cargo-repro/pull/5
so we're working on something... mildly ambitious in this department https://github.com/iqlusioninc/synchronicity
tl;dr: Rustwide + Libra = cryptographic proofs of BFT consensus around reproducible builds
some interesting stuff here re: reproducing rustc https://github.com/rust-lang/rust/issues/34902#issuecomment-562698593
The README for Synchronicity does not describe what problem does it solve
@Shnatsel the general notion of the property is "binary transparency", but I think it provides something a bit stronger
I can perhaps make a note of it
When I read that README I don't understand what use cases that enables. I guess with this tool a few companies can get together, build all crates on crates.io and then tell if the build is reproducible or not, and if yes what the hash is? And ideally I'd be able to get a build environment these guys use and obtain a binary with the same hash myself? This I get. But, what problem does that solve?
@Shnatsel aah, the main use would be binary releases of crates that could be validated in some way, or a global build cache
I should probably write a blog post
or a few bullet points in the readme on what use cases that enables
our real goal is to have an excuse to play with HotStuffBFT at the moment, so it’s a bit of a hammer in search of nails 😅
and yeah sure
or a “Why does this help?” section
@Shnatsel how's this https://github.com/iqlusioninc/synchronicity/pull/18/files
Probably too technical to actually get people excited about it, but good as a first pass