Stream: t-compiler/wg-nll

Topic: datafrog-ownership-etc


nikomatsakis (May 23 2018 at 12:24, on Zulip):

@Frank McSherry I wanted to raise a question with you. I've been thinking about moving more and more of rustc out into crates (polonius and chalk are examples of this, but also lesser things like ena, our hashtables, etc) that can live in the rust-lang-nurseryorg (or rust-lang, not important) as well as being on crates.io. In general, my preference is that if rustc depends on something, we have some people in the rust org (hopefully teams!) with write permissions, so we can patch bugs, issues crates.io releases, etc. I'm not sure where I think datafrog ought to live though =) my guess is that maybe it deserves its own Github org (sort of like LALRPOP) and we can try to ensure that members of the compiler team are actively involved in maintenance (my other guess is that, like ena, it will settle eventually and not need much). I definitely think datafrog is a tool that can be useful in many parts of the compiler and likely outside the compiler as well. Anyway, I don't have a firm idea, just trying to think things through a bit.

Frank McSherry (May 23 2018 at 12:41, on Zulip):

My guess is that you can have :datafrog:, for whatever that means. I suspect you'll want to re-license it dual MIT/Apache, for example, and it would be great (from my pov) if you all had ownership so that I don't have to maintain it. :)

Frank McSherry (May 23 2018 at 12:42, on Zulip):

oops, :frog: not :datafrog:, which doesn't have an emoji YET.

nikomatsakis (May 23 2018 at 12:43, on Zulip):

@Frank McSherry I think the maximal version would mean:

Frank McSherry (May 23 2018 at 12:44, on Zulip):

Yup, up for whatever. The repo was created one week ago, not significant emotional investment in it or anything. :)

nikomatsakis (May 23 2018 at 12:44, on Zulip):

I wasn't sure if you had further plans to use it for other things =)

nikomatsakis (May 23 2018 at 12:45, on Zulip):

/me secretly wants @Frank McSherry to keep improving it with nifty ideas ;)

nikomatsakis (May 23 2018 at 12:46, on Zulip):

In all seriousness, I do wonder how many diverse applications we might find. It'd be really cool if we can turn this into a nice toolbox for people who want to do numeric analysis. I suspect growing more of an independent community might help a lot also for things like shaving down ergonomic hits (e.g., proc-macro front-ends and the like)

nikomatsakis (May 23 2018 at 12:46, on Zulip):

but at minimum we know rustc needs it :)

Frank McSherry (May 23 2018 at 12:46, on Zulip):

Not really other plans, no. It was interesting to try and learn about the minimal (or small-ish) amount of stuff you would need to effect these sorts of computations, which helps me understand a bit more about what overhead or complexity is optional vs not.

Frank McSherry (May 23 2018 at 12:47, on Zulip):

It does highlight a bunch of pain points with EDSLs in Rust; perhaps there is something to learn there (e.g. what sort of macro pain is required to make this more useable).

nikomatsakis (May 23 2018 at 12:47, on Zulip):

yeah

nikomatsakis (May 23 2018 at 12:48, on Zulip):

ok, well, this is not super urgent, just a question I wanted to raise

Frank McSherry (May 23 2018 at 12:48, on Zulip):

On the one hand, great that all of these types could totally be reference types rather than interned integers, and Rust's type system makes that good to not screw up. On the other hand, ... urk.

Frank McSherry (May 23 2018 at 12:48, on Zulip):

Coffee break going on here. bbiab.

nikomatsakis (May 23 2018 at 12:48, on Zulip):

though if we're going to land the "in tree polonius support" (which I would like to do) we will have to at least solve the license question sooner rather than later

Frank McSherry (May 23 2018 at 12:49, on Zulip):

license question is easy. I'll relicense and we should just need sign off from the two contributors (you and Reed).

nikomatsakis (May 23 2018 at 12:49, on Zulip):

On the one hand, great that all of these types could totally be reference types rather than interned integers, and Rust's type system makes that good to not screw up. On the other hand, ... urk.

by this do you mean specifically the matter of passing around &T ? (or something else)

Frank McSherry (May 23 2018 at 12:50, on Zulip):

That you don't need to intern all the strings that come in, nor clone them etc., each of the relations could in principle be Relation<&'a String> and .. maybe it all works out. Meant to be more of "a Rust EDSL has value, because Rust brings something valuable other than a harness in which to run code".

Frank McSherry (May 23 2018 at 12:53, on Zulip):

Anyhow, plan at the moment is not to do anything that might complicate the process (e.g. crate push, PR blitz, etc) and just chill out waiting to figure out how to hand this off to you all.

nikomatsakis (May 23 2018 at 12:53, on Zulip):

ok.

Reed Koser (May 23 2018 at 14:56, on Zulip):

Just let me know what needs to be done to sign off on the licensing issues :smile:

nikomatsakis (May 23 2018 at 15:04, on Zulip):

@Frank McSherry after looking into it a bit, if you want to transfer datafrog to the rust-lang-nursery org, then I think the easiest thing to do is to transfer it to me, and I will add it to the org (you have to have certain permissions to add a repo to an org).

nikomatsakis (May 23 2018 at 15:05, on Zulip):

I don't think we have to get super formal about the licensing =) in any case, opening a PR making the change and having each of us comment is the usual way to go about it

Reed Koser (May 23 2018 at 15:26, on Zulip):

SGTM

Frank McSherry (May 24 2018 at 17:02, on Zulip):

Hi @nikomatsakis. I can't transfer :frog: to you as long as you have a datafrog repository!

nikomatsakis (May 24 2018 at 17:11, on Zulip):

oh dear :)

nikomatsakis (May 24 2018 at 17:11, on Zulip):

@Frank McSherry renamed

nikomatsakis (May 24 2018 at 17:11, on Zulip):

hopefully that suffices

Frank McSherry (May 24 2018 at 17:12, on Zulip):

"nikomatsakis already has a repository in the frankmcsherry/datafrog network"

nikomatsakis (May 24 2018 at 17:16, on Zulip):

@Frank McSherry deleted my fork

nikomatsakis (May 24 2018 at 17:16, on Zulip):

maybe that works?

nikomatsakis (May 24 2018 at 17:16, on Zulip):

if not, I'll write to github suppor :)

nikomatsakis (May 24 2018 at 17:16, on Zulip):

I don't think I had anything in there anyway but some edit I made to the readme

Frank McSherry (May 24 2018 at 17:16, on Zulip):

"transfer requested"

Frank McSherry (May 24 2018 at 17:18, on Zulip):

Not sure how formal you want to be, but if you do the PR for the license (to whatever you need) I suspect @qmx and i will ok it. :)

nikomatsakis (May 24 2018 at 17:19, on Zulip):

heh ok :) might as well cya coa

nikomatsakis (May 24 2018 at 17:20, on Zulip):

https://github.com/rust-lang-nursery/datafrog is live! :)

Frank McSherry (May 24 2018 at 17:25, on Zulip):

I just forked a copy to avoid link breakage from elsewhere, but you all have the conn.

lqd (May 24 2018 at 17:26, on Zulip):

@Frank McSherry thank you very much for all your awesome work on this btw :D

nikomatsakis (May 24 2018 at 17:26, on Zulip):

@Frank McSherry @Reed Koser if you could please check the box and comment on this PR, would be appreciated

nikomatsakis (May 24 2018 at 17:26, on Zulip):

those were all the authors I saw in the git log anyway

nikomatsakis (May 24 2018 at 17:26, on Zulip):

mostly @Frank McSherry :)

Frank McSherry (May 24 2018 at 17:29, on Zulip):

No worries y'all. I've gotten plenty out of Rust; happy to give back.

Reed Koser (May 24 2018 at 18:08, on Zulip):

I can't actually check the box, but I commented

nikomatsakis (May 24 2018 at 18:11, on Zulip):

even better

Frank McSherry (May 28 2018 at 11:57, on Zulip):

@Andrea Lattuada just observed that the treefrog stuff didn't make it when the repo got moved. It seems like I just never pushed them at the repo, I guess? I added them to my fork, and once I tidy them a bit I can PR it at the nursery version. Is there a sane place to PR the datafrog_opt.rs code using it? I can just sit on it (@lqd mentioned it might be a good project for someone else) or I can wait until the new :frog: lands and then PR it in as treefrog_opt.rs or something; don't want to increase the maintenance burden w/o someone excited to receive it though.

lqd (May 28 2018 at 12:10, on Zulip):

Yeah I didn’t make a PR with it yet since leapfrog was still in a branch and not on datafrog master (you were mentioning you wanted to iterate a bit on the types/API I think) ? If anyone makes a PR before TFLJ is finalized (and before I get back home tomorrow) there’s an additional index which can be removed in « treefrog_opt.rs » :) I am very excited about it, it’s really awesome :)

lqd (May 28 2018 at 12:22, on Zulip):

btw the leapjoin is in the repo https://github.com/rust-lang-nursery/datafrog/tree/leapfrog just on master

lqd (May 28 2018 at 12:22, on Zulip):

not on master sorry

Andrea Lattuada (May 28 2018 at 12:24, on Zulip):

A good place to PR the datafrog_opt.rs code is probably against https://github.com/rust-lang-nursery/polonius/blob/master/polonius-engine/src/output/datafrog_opt.rs on the nursery polonius repo, right?

Andrea Lattuada (May 28 2018 at 12:25, on Zulip):

btw the leapjoin is in the repo https://github.com/rust-lang-nursery/datafrog/tree/leapfrog just on master

Yup, @Frank McSherry lmk if I can help with the git-foo to tidy this up.

Frank McSherry (May 28 2018 at 12:36, on Zulip):

Oh, good that that branch made the transition. Somehow I wasn't sure that it did.

nikomatsakis (May 28 2018 at 13:22, on Zulip):

@Frank McSherry is the leapfrog stuff available on datafrog master?

nikomatsakis (May 28 2018 at 13:22, on Zulip):

if so, we should probably open an issue on polonius about integrating that into datafrog-opt

lqd (May 28 2018 at 13:44, on Zulip):

leapfrog is in the branch I linked not on master

lqd (May 28 2018 at 13:46, on Zulip):

and there’s the gist of the « leapfrog datafrog_opt » (w/ an index we can remove) for when it lands on master :)

Andrea Lattuada (May 28 2018 at 14:53, on Zulip):

Because people here are asking: Frank’s port of datafrog_opt.rs to TFLJ is here: https://gist.github.com/frankmcsherry/4f0769b085d2e8f39ae1cc8199f39d41

Last update: Nov 22 2019 at 00:50UTC