Stream: t-compiler

Topic: when to meet regarding licensing policy/problems


Florian Gilcher (Nov 26 2019 at 13:40, on Zulip):

But currently, we're in the state of the whole compiler-rt being weirdly licensed, which will both cost a ton of effort + education to fix.

pnkfelix (Nov 26 2019 at 13:40, on Zulip):

I don't think anyone is saying there isn't a problem here

pnkfelix (Nov 26 2019 at 13:42, on Zulip):

Here are two possible directions to take, and I want to state them explicitly to try to clarify the pros/cons of dedicating a Friday meeting to this topic now

Florian Gilcher (Nov 26 2019 at 13:42, on Zulip):

I don't want to imply that. I just don't know if anyone has a good reading of the current state of their components and the project in general.

pnkfelix (Nov 26 2019 at 13:42, on Zulip):

(actually, let me first alpha-rename this fork of the planning meeting thread.)

pnkfelix (Nov 26 2019 at 13:43, on Zulip):

So, two possible directions:

pnkfelix (Nov 26 2019 at 13:44, on Zulip):
  1. We meet ASAP, and discuss the current state of affairs. We discuss past problems/concerns. We present whatever draft policy we have, to get feedback on it. We maybe talk about how to review the current code base and also how to impose rules in future.
pnkfelix (Nov 26 2019 at 13:46, on Zulip):

Or: 2. Find a group of self-identified "experts" who are interested in writing up the policy here; maybe call it WG-licensing. They do the review of past problems and current state of affairs. They write a draft policy and circulate it to anyone else who is interested, with goal of iteratively developing the "official policy". Only after the official policy is blessed by this WG, only then do we have a Friday meeting dedicating to explaining it to the contributors.

pnkfelix (Nov 26 2019 at 13:47, on Zulip):

The initial comments that you (@Florian Gilcher ) wrote, regarding not wanting to push it back, imply that you would prefer us to take a direction like (1.) above.

Florian Gilcher (Nov 26 2019 at 13:48, on Zulip):

I think there's 3 problems here:

  1. We have a number of ongoing issues that need to be fixed, ASAP (or they linger and get harder and harder to fix)
  2. The whole licensing situation is unclear
  3. We need to make sure that doesn't happen again
Florian Gilcher (Nov 26 2019 at 13:48, on Zulip):

Only 3. needs a policy

Florian Gilcher (Nov 26 2019 at 13:49, on Zulip):
  1. is definitely something that only needs an interesting circle and probably works better the smaller and more interested the group is
Florian Gilcher (Nov 26 2019 at 13:50, on Zulip):

That would be a bad pick for a friday meeting

pnkfelix (Nov 26 2019 at 13:50, on Zulip):

okay then, I think we agree about 3.

pnkfelix (Nov 26 2019 at 13:51, on Zulip):

For 1. and 2., we've known about issues here for a while. #63232 for example has brson complaining to the core team back in August.

Florian Gilcher (Nov 26 2019 at 13:51, on Zulip):
  1. and 2. might fit better into a planning meeting, as those problems are undeniably there. Could also fit a small group, but, TBH, component licensing is IMHO the job of the component maintainers. I have no good insight there.
Florian Gilcher (Nov 26 2019 at 13:52, on Zulip):

We've known at least since 2017 that our file is out of date: https://github.com/rust-lang/rust/issues/39897

pnkfelix (Nov 26 2019 at 13:52, on Zulip):

The fact that brson nominated #63232 for the core team points out that this, strictly speaking, is broader than just a compiler team concern

pnkfelix (Nov 26 2019 at 13:53, on Zulip):

We've known at least since 2017 that our file is out of date: https://github.com/rust-lang/rust/issues/39897

(also filed by brson. :smile: )

pnkfelix (Nov 26 2019 at 13:53, on Zulip):

the comments on #39897 made it seem like the core team was going to actually engage with a lawyer. But I have no idea what came of that.

Florian Gilcher (Nov 26 2019 at 13:55, on Zulip):

We've been in touch, but the lawyer can't work much without knowing what's even there.

pnkfelix (Nov 26 2019 at 13:56, on Zulip):

It is good that you split 1 and 2 into separate items, I suppose, since it points out: If we actually have concrete complaints/issues with specific modules/subcomponents, then we can prioritize addressing those

Florian Gilcher (Nov 26 2019 at 13:57, on Zulip):

Especially that some problems, like the apfloat port and the compiler-rt situation being pretty clear, we mainly need a path out of that.

pnkfelix (Nov 26 2019 at 13:57, on Zulip):

that is: Dealing with figuring out licensing for the whole tree is a big issue.

Florian Gilcher (Nov 26 2019 at 13:57, on Zulip):

Yep, 1. is also vastly more pressing then a general sense of insecurity

pnkfelix (Nov 26 2019 at 13:57, on Zulip):

but if there are fires we need to put out, then lets file issues about them? If we haven't already?

Florian Gilcher (Nov 26 2019 at 13:57, on Zulip):

Also, the compiler tree is much less of an issue than runtime components

Florian Gilcher (Nov 26 2019 at 13:57, on Zulip):

All of them have issues.

Florian Gilcher (Nov 26 2019 at 13:59, on Zulip):

Any dependency of the compiler only changes what we need to document when shipping the compiler. Even if we have GPL source in tree, we'd end up in a situation where we might block some commercial uses, but are still allowed to ship.

Florian Gilcher (Nov 26 2019 at 13:59, on Zulip):

e.g. a company that wants to ship a closed fork of the compiler would have to provide the source of their compiler fork - that's currently the worst case and I know of no company intending to do so.

Florian Gilcher (Nov 26 2019 at 14:00, on Zulip):

runtime components being mislicensed exposes users to a hazard on their produced binaries, though

Florian Gilcher (Nov 26 2019 at 14:00, on Zulip):

so, even 1. has different shades of ugh

Florian Gilcher (Nov 26 2019 at 14:02, on Zulip):

let me pull in some issues here.

Florian Gilcher (Nov 26 2019 at 14:06, on Zulip):

ap_float is discussed here: https://github.com/rust-lang/rust/issues/55993 (it was ported before the relicensing of LLVM, so the license should be University of Berkeley)
compiler-buildins is a port and declares the wrong license: https://github.com/rust-lang/compiler-builtins/issues/307
compiler-buildins currently has GPL-v3 code: https://github.com/rust-lang/compiler-builtins/issues/319 (I'm currently working on a fix there with some of my time I have)
backtrace has a misleading license: https://github.com/rust-lang/backtrace-rs/issues/234 (though, I'm not sure how much of this is just hiding the proper license from analysis, the glue layer having a different license does not seem a problem to me)

Florian Gilcher (Nov 26 2019 at 14:06, on Zulip):

(From https://github.com/rust-lang/rust/issues/63232#issuecomment-517918295)

Florian Gilcher (Nov 26 2019 at 14:07, on Zulip):

given that especially low-level pieces are often ported from LLVM and GCC though, I'm not sure how many of those cases will be found

pnkfelix (Nov 26 2019 at 14:07, on Zulip):

okay then.

Florian Gilcher (Nov 26 2019 at 14:07, on Zulip):

There was definitely a practice at some point where people just slapped MIT/Apache-2.0 on things... I hate to say it, I see no way past a full review.

pnkfelix (Nov 26 2019 at 14:08, on Zulip):

Okay. So what is your goal here? To solicit volunteers to help with a review?

pnkfelix (Nov 26 2019 at 14:09, on Zulip):

or just raise general attention to the flames engulfing us?

pnkfelix (Nov 26 2019 at 14:11, on Zulip):

Or to prioritize items from the specific list of issues that we have identified so far?

Florian Gilcher (Nov 26 2019 at 14:12, on Zulip):

I took on the task in core to care about that situation :). I'm happy to assist, but I believe that all teams are responsible for the soundness of their stack. I'm not sure if we can find _volunteers_ for a licensing review of a codebase of that size. So, currently, it's awareness work.

Florian Gilcher (Nov 26 2019 at 14:14, on Zulip):

That might sound a bit negative via chat, but it's a case of problem ignored for too long and we somehow need to get out of that. I also don't see a high chance of volunteering, we can't expect to onboard people to clean up after us :).

pnkfelix (Nov 26 2019 at 14:16, on Zulip):

I'm still trying to gauge the severity of the matter. I agree that the problem has laid dormant for too long.

pnkfelix (Nov 26 2019 at 14:17, on Zulip):

But then again, a lot of problems have laid dormant for a long time. (e.g. we have people screaming in our github comments about unsoundness issues that have been open for six years)

pnkfelix (Nov 26 2019 at 14:18, on Zulip):

So when I read a phrase like "needs to be fixed ASAP", I personally still need a bit of guidance about what that's supposed to mean, in terms of whether I and/or other people paid to work on Rust should be expected to drop everything to address this.

Florian Gilcher (Nov 26 2019 at 14:19, on Zulip):

The problem is that none of these issues can be fixed by upgrading and might taint whole past and future releases.

Florian Gilcher (Nov 26 2019 at 14:19, on Zulip):

Also, if we figure out that something is incompatibly licensed and the original author doesn't agree to a relicensing, we need to rewrite, potentially along with everything on top, at least in the same library.

Florian Gilcher (Nov 26 2019 at 14:20, on Zulip):

on the subject of compiler-buildins, we expose every user to shipping wrongly licensed software, which might lead them to losing usage rights

Florian Gilcher (Nov 26 2019 at 14:21, on Zulip):

I'm not sure how high the risk is of someone _actually_ pulling this move, but if someone does, we have a severe problem.

pnkfelix (Nov 26 2019 at 14:21, on Zulip):

The compiler-builtins issue gives me the impression that the problem shouldn't be quite so severe in that case

pnkfelix (Nov 26 2019 at 14:22, on Zulip):

but I agree its not good to be in a state where we are even asking the question

Florian Gilcher (Nov 26 2019 at 14:22, on Zulip):

The second compiler-builtins is worse.

Florian Gilcher (Nov 26 2019 at 14:23, on Zulip):

The first we might escape with a relicensing and updating of notices.

pnkfelix (Nov 26 2019 at 14:23, on Zulip):

ah yes the second one is worse

pnkfelix (Nov 26 2019 at 14:25, on Zulip):

/me hates the fact that issues like this are likely to lead to people just trying to cut-and-paste without attribution.

Florian Gilcher (Nov 26 2019 at 14:27, on Zulip):

TBH, with the algorithm being a standard algorithm, probably below the threshold of originality, that would have been better ;)

Florian Gilcher (Nov 26 2019 at 14:28, on Zulip):

And, put the other way, it's the GCC teams explicit wish that we don't derive from them if we don't want to use GPL code in our codebase

pnkfelix (Nov 26 2019 at 14:28, on Zulip):

yes I understand that

pnkfelix (Nov 26 2019 at 14:28, on Zulip):

I don't want people to cut-and-paste code in the first place

Florian Gilcher (Nov 26 2019 at 14:32, on Zulip):

Yep, so... to sum it up: this has become a large and multi-faceted problem to work through - pushing it off does worsen things. Given that we are moving into industry adoption, this is becoming worse and worse and a potential deal-breaker.

Florian Gilcher (Nov 26 2019 at 14:33, on Zulip):

I would like to avoid a situation where this gets fixed the moment someone makes a news item out of this - that impacts team planning _and_ is completely uncontrollable.

pnkfelix (Nov 26 2019 at 14:33, on Zulip):

Nonetheless, I'm still not sure what a Friday meeting here will do

pnkfelix (Nov 26 2019 at 14:34, on Zulip):

or lets see, it would:

pnkfelix (Nov 26 2019 at 14:34, on Zulip):

Draw attention to the problem

Florian Gilcher (Nov 26 2019 at 14:34, on Zulip):

Potentially coming up with a plan. But then again, I never attended one, so it might not be the right space to do that.

pnkfelix (Nov 26 2019 at 14:34, on Zulip):

and maybe garner volunteer support to address it

pnkfelix (Nov 26 2019 at 14:36, on Zulip):

here's my current thinking: Instead of trying to rejigger the Friday schedule that was previously set ...

pnkfelix (Nov 26 2019 at 14:36, on Zulip):

... maybe lets instead try to discuss this on Thurdsay

pnkfelix (Nov 26 2019 at 14:36, on Zulip):

during the triage meeting

pnkfelix (Nov 26 2019 at 14:37, on Zulip):

Obviously that's a shorter time slot (since it would need to share its time with normal triage activity and Working Group check-ins). But it would also serve to raise attention to the problem and maybe also garner volunteer effort.

Florian Gilcher (Nov 26 2019 at 14:49, on Zulip):

Sounds like a good plan, I'm not sure I can arrange the triage meeting, though.

Florian Gilcher (Nov 26 2019 at 14:49, on Zulip):

/me wants to skip time until after christmas

pnkfelix (Nov 26 2019 at 14:49, on Zulip):

That's okay, I believe I can handle the conversation there

Florian Gilcher (Nov 26 2019 at 14:52, on Zulip):

:+1:

Florian Gilcher (Nov 26 2019 at 14:53, on Zulip):

In general, I think it's a problem we should start tackling at a consistent pace rather then stop the world. Also, I'm handling the GPL issue.

pnkfelix (Nov 26 2019 at 14:53, on Zulip):

The GPL issue being that mulsi thing?

pnkfelix (Nov 26 2019 at 14:54, on Zulip):

I was struck by how different that code is from other presentations of "Booth Multiplication"

Florian Gilcher (Nov 26 2019 at 14:54, on Zulip):

Yep

pnkfelix (Nov 26 2019 at 14:55, on Zulip):

I freely admit I have not thought hard at all about the problem. I just noticed that pretty much every presentation of Booth Multiplication I could find, involved inspecting either the lowest 2 or 3 bits each time through the loop.

Florian Gilcher (Nov 26 2019 at 14:55, on Zulip):

TBH, my first go is just addressing the author of the code (which is a services company) to give us a license against a nice mention.

pnkfelix (Nov 26 2019 at 14:55, on Zulip):

while that code only inspects 1 bit. :)

pnkfelix (Nov 26 2019 at 14:55, on Zulip):

Sure, I can understand taking that route first

pnkfelix (Nov 26 2019 at 14:55, on Zulip):

Another route would be to clean-room implement the Booth code as documented on Wikipedia

Florian Gilcher (Nov 26 2019 at 14:55, on Zulip):

I also already have someone to do a cleanroom implementation of it. :)

pnkfelix (Nov 26 2019 at 14:56, on Zulip):

I'm working so hard to avoid being nerd-sniped into trying to find where the one-bit-inspecting approach deviates

pnkfelix (Nov 26 2019 at 14:58, on Zulip):

ah maybe this line from wikipedia is an answer: "This scheme can be extended to any number of blocks of 1s in a multiplier (including the case of a single 1 in a block)."

pnkfelix (Nov 26 2019 at 14:58, on Zulip):

okay that's enough time I've spent reading without thinking

pnkfelix (Nov 26 2019 at 15:24, on Zulip):

(Actually, the more I look at this, the more I think that this is not Booth's algorithm. It may just be a more naive multiplication?)

pnkfelix (Nov 26 2019 at 15:24, on Zulip):

in particular, Booth's algorithm is designed to avoid intermediate additions when you have a long block of 1's

pnkfelix (Nov 26 2019 at 15:24, on Zulip):

this code ... just does them.

pnkfelix (Nov 26 2019 at 15:39, on Zulip):

okay, nerd-sniping completed: https://en.wikipedia.org/wiki/Ancient_Egyptian_multiplication#Russian_peasant_multiplication

centril (Nov 26 2019 at 15:44, on Zulip):

Yeah, I definitely think we should have a meeting to broadcast the policy

@pnkfelix maybe just send out an email?

pnkfelix (Nov 26 2019 at 15:44, on Zulip):

I think the concern is that emails will be ignored

centril (Nov 26 2019 at 15:44, on Zulip):

other than that, the best thing for licensing is automated checks

centril (Nov 26 2019 at 15:45, on Zulip):

fwiw, last time I sent out an email it wasn't ignored

pnkfelix (Nov 26 2019 at 15:45, on Zulip):

other than that, the best thing for licensing is automated checks

see also #62618

Florian Gilcher (Nov 26 2019 at 15:57, on Zulip):

@centril automated checks don't help if people port over code and attach the wrong license or ported code gets merged.

centril (Nov 26 2019 at 15:59, on Zulip):

@Florian Gilcher well informing everyone with an email does; for straight up ports of some library it seems easier, but I don't think people with r+ can be expected to ensure that some random snippet of code wasn't copied from somewhere with a GPL license

Florian Gilcher (Nov 26 2019 at 15:59, on Zulip):

But yes, what can be automated should be, especially when we're talking about all added dependencies

Florian Gilcher (Nov 26 2019 at 16:00, on Zulip):

The code mentioned above was merged even in the presence of a comment that it was ported from GCC.

Florian Gilcher (Nov 26 2019 at 16:01, on Zulip):

Other cases like apfloat are derived ports and put under a new license.

centril (Nov 26 2019 at 16:01, on Zulip):

you can insert a tidy check for license comment strings or something

centril (Nov 26 2019 at 16:02, on Zulip):

the runtime repos have fewer reviewers

nagisa (Nov 26 2019 at 16:02, on Zulip):

email is definitely a better way to disseminate broad knowledge than a meeting.

centril (Nov 26 2019 at 16:02, on Zulip):

seems less likely this would happen in rust-lang/rust

nagisa (Nov 26 2019 at 16:02, on Zulip):

you’re much likely to have a contributor to read a mail eventually than have all of us in for a meeting.

Florian Gilcher (Nov 26 2019 at 16:03, on Zulip):

@nagisa right, personal meetings tend to have higher impact on leads, though, so "both" is probably the answer?

Florian Gilcher (Nov 26 2019 at 16:04, on Zulip):

Also, see policy discussion and issue above, we're currently writing guidelines and help for people doing reviews.

centril (Nov 26 2019 at 16:04, on Zulip):

Personally I'm concerned about meeting time that could go towards other design meetings when there seems to be broad agreement with the policy

Florian Gilcher (Nov 26 2019 at 16:04, on Zulip):

But as long as that is not done, we need to make sure things don't get worse

Florian Gilcher (Nov 26 2019 at 16:08, on Zulip):

I'm practical there, as long as the issue is improved and time spent on it. I believe there's a design component, especially around processes to it.

Florian Gilcher (Nov 26 2019 at 16:08, on Zulip):

But YMMV, I don't have enough insight into day-to-day processes of the relevant teams there.

Florian Gilcher (Nov 26 2019 at 16:09, on Zulip):

I agree with @pnkfelix though that I wouldn't initially spend an hour on it.

Last update: Dec 12 2019 at 00:50UTC