Stream: wg-secure-code

Topic: cargo-audit


Alex Gaynor (Oct 13 2019 at 14:18, on Zulip):

@Tony Arcieri can you do an absicca release and then bump the absiscca and failure versions in cargo-audit? I think that'll reduce the number of copies of syn/quote/proc-macro2 that get compiled, which should help with build speed

Alex Gaynor (Oct 13 2019 at 14:19, on Zulip):

I'm happy to do the PRs for cargo-audit, but obviously I can't do an absicca release

Tony Arcieri (Oct 13 2019 at 14:19, on Zulip):

yeah I've been planning on it. there are a few other things I want to get in there

Tony Arcieri (Oct 13 2019 at 14:19, on Zulip):

just been updating all of my other pre-1.0 custom derive stuff

Alex Gaynor (Oct 13 2019 at 14:21, on Zulip):

Just realized we also need gumpdrop-derive, which doesn't have a release that uses the 1.0 family. Left a comment asking them for it.

Tony Arcieri (Oct 13 2019 at 14:25, on Zulip):

yeah that just got a PR to bump it

Tony Arcieri (Oct 13 2019 at 14:26, on Zulip):

which I've also been waiting on

Alex Gaynor (Oct 13 2019 at 14:27, on Zulip):

The PR was merged, so I left a comment asking about a release.

Alex Gaynor (Oct 13 2019 at 16:50, on Zulip):

gumdrop release out!

Tony Arcieri (Oct 13 2019 at 17:04, on Zulip):

woop

Tony Arcieri (Oct 13 2019 at 18:52, on Zulip):

ok, this gets rid of all of the pre-1.0 proc macro crates: https://github.com/iqlusioninc/abscissa/pull/141

Alex Gaynor (Oct 13 2019 at 19:30, on Zulip):

Screenshot-2019-10-13-at-3.28.58-PM.png

I did a cargo +nightly build -Z timings --release on cargo-audit master. That's the build graph, filtered to only crates that took >15 seconds to compile. It looks like getting down to only a single version of the proc macro crates will help a lot, but there's some other crates that still take a ton of time to compile.

Tony Arcieri (Oct 13 2019 at 19:41, on Zulip):

heh, wow @ darling_core taking longer than serde_derive

Tony Arcieri (Oct 13 2019 at 19:41, on Zulip):

wonder why

Alex Gaynor (Oct 13 2019 at 19:42, on Zulip):

I feel like both of them are at 10x what I'd like :-)

Tony Arcieri (Oct 13 2019 at 20:11, on Zulip):

https://github.com/RustSec/cargo-audit/pull/154

Tony Arcieri (Oct 14 2019 at 00:38, on Zulip):

ok, v0.10.0 is out w\ the upgrades https://crates.io/crates/cargo-audit

Shnatsel (Nov 03 2019 at 17:27, on Zulip):

At long last I've turned https://github.com/Shnatsel/rust-audit into an RFC against Cargo: https://github.com/rust-lang/rfcs/pull/2801

Alex Gaynor (Nov 03 2019 at 17:45, on Zulip):

Cool! I left a few comments

Shnatsel (Nov 03 2019 at 17:54, on Zulip):

Great comments, thanks a lot!

Shnatsel (Nov 03 2019 at 17:54, on Zulip):

Hmm, I wonder if I can abuse my access at Google to get Go developers to chime in on what has and hasn't worked for them with this metadata

Tony Arcieri (Nov 03 2019 at 18:14, on Zulip):

awesome

Tony Arcieri (Nov 03 2019 at 19:55, on Zulip):

@Shnatsel when I was at Square we had a tool like this for Ruby applications. It found all Gemfile.lock files on production servers/containers, audited them against RubySec, and then auto-filed VULN tickets against the app owners

Tony Arcieri (Nov 03 2019 at 19:56, on Zulip):

tickets also auto-closed when the apps were updated

Shnatsel (Nov 03 2019 at 19:59, on Zulip):

I am not familiar with the Ruby software distribution model. Is that file necessary to run the program?

Tony Arcieri (Nov 03 2019 at 20:01, on Zulip):

it's the most common way of managing Ruby apps. not completely necessary but definitely the most popular

Shnatsel (Nov 03 2019 at 20:02, on Zulip):

So it was a distinct file you had to ship, but it was typical for it to be shipped. Cool! Could you add this as a comment on the PR?

Alex Gaynor (Nov 03 2019 at 20:03, on Zulip):

If Go had a vuln db, and I had a lot more free time, I'd probably write something at $work that scanned docker images for binaries with embedded go.mod info, did a vuln check on them, and then filed tickets if either a) that image was :latest, or b) there was anything in the container orchestration system using that image.

Tony Arcieri (Nov 03 2019 at 20:08, on Zulip):

@Shnatsel yeah sure. you can imagine it sort of like a runtime Cargo.lock

Shnatsel (Nov 03 2019 at 22:54, on Zulip):

@Tony Arcieri could you chime in on the reproducible builds angle in https://github.com/rust-lang/rfcs/pull/2801 ?

Tony Arcieri (Nov 03 2019 at 22:55, on Zulip):

Cargo.lock should be deterministic for a given toolchain and index state, I think...

Tony Arcieri (Nov 03 2019 at 22:56, on Zulip):

I'm playing around with trying to make a reproducible build tool based on Rustwide

Shnatsel (Nov 03 2019 at 22:56, on Zulip):

Eh, I don't think I can vouch for Cargo.lock having stable sort order

Shnatsel (Nov 03 2019 at 22:57, on Zulip):

For reproducible builds this seems to be both a blessing (the info is right there in the binary) and a curse (the info itself is hard to make reproducible) - and there are concerns around the crate coming from one registry or another and that evaluating to different metadata

Tony Arcieri (Nov 03 2019 at 22:57, on Zulip):

I mean, if anything...

Tony Arcieri (Nov 03 2019 at 22:57, on Zulip):

reproducing a build needs Cargo.lock

Tony Arcieri (Nov 03 2019 at 22:57, on Zulip):

you can't reproduce a build without it

Tony Arcieri (Nov 03 2019 at 22:58, on Zulip):

as an input

Tony Arcieri (Nov 03 2019 at 22:58, on Zulip):

so if anything it's a helpful forensic artifact for reproducing builds! :smiley:

Shnatsel (Nov 03 2019 at 23:00, on Zulip):

Some are calling for including (crate-name, version, hash) only without the source URL of any kind. Do you think including registry URL or git repo url is helpful, or hinders reproducibility?

Tony Arcieri (Nov 03 2019 at 23:03, on Zulip):

the git repo stuff should allow you to reproduce the build still, if you have access to the repo and the relevant commits are still there

Tony Arcieri (Nov 03 2019 at 23:04, on Zulip):

Cargo.lock encodes all of the commit hashes for each package

Shnatsel (Nov 03 2019 at 23:04, on Zulip):

I guess this is about commit hash only vs repo url as well

Tony Arcieri (Nov 03 2019 at 23:05, on Zulip):

I think they might be a bit... entangled

Shnatsel (Nov 03 2019 at 23:08, on Zulip):

uuh, repo url is not included in the hash

Tony Arcieri (Nov 03 2019 at 23:10, on Zulip):

here's an example:

Tony Arcieri (Nov 03 2019 at 23:10, on Zulip):
[[package]]
name = "libra-config"
version = "0.1.0"
source = "git+https://github.com/libra/libra.git?rev=66734424#667344248287a1647f42a793e92414853a5fa335"
Tony Arcieri (Nov 03 2019 at 23:11, on Zulip):

also trying to redact Cargo.lock all gets very tricky with v1 vs v2, heh

Shnatsel (Nov 03 2019 at 23:14, on Zulip):

What's v1 vs v2?

Shnatsel (Nov 03 2019 at 23:16, on Zulip):

Darn, embedding paths is getting really thorny really quickly. Maybe I should just make it an optional, off-by default thing. This way enterprise can enforce it and use it, and everybody else enjoys no-info-leaks.

Tony Arcieri (Nov 03 2019 at 23:19, on Zulip):

https://github.com/rust-lang/cargo/pull/7070

Tony Arcieri (Nov 03 2019 at 23:20, on Zulip):

shipped in 1.38 I think?

Shnatsel (Nov 04 2019 at 16:50, on Zulip):

OIC. Thanks for the input on the thread!
We got a response from a Rust team member too, encouraging on the basic direction but with a lot of valid points about the weaknesses of the current proposal.
They all seem to be actionable, so I'll try to iterate on the RFC and request another round of review. Any help is appreciated, just brainstorming solutions for those points would be great.

Last update: Nov 11 2019 at 22:45UTC