Stream: t-compiler/wg-prioritization

Topic: Automation of the meeting agenda


LeSeulArtichaut (Mar 25 2020 at 21:44, on Zulip):

@pnkfelix proposed to automate the addition of the PR's to the meeting agenda. We should decide how we want to do this. Would it be some script that gives text that you can directly paste in the hackmd? Or is it possible to automate even more?

pnkfelix (Mar 26 2020 at 02:06, on Zulip):

to be honest, a big thing that I had wanted to automate was just the process of going through the list of issues and posting them to the Zulip channel in the format that I thought was most readable/usable for the audience

pnkfelix (Mar 26 2020 at 02:07, on Zulip):

this is something that could be applied to both pre-triage (going through each unprioritized I-nominated, etc), and to triage itself (going through the beta-noms). Though if the hackmd agenda has them in the right format already, then it may make less sense to mechanize doing it for the triage meeting.

pnkfelix (Mar 26 2020 at 02:08, on Zulip):

but the point is, in my head, I figured a single interactive (probably command line) tool could hook into both the Zulip API and the github API

pnkfelix (Mar 26 2020 at 02:09, on Zulip):

reading the issues from github, posting each one to zulip in the right format, and then, after a conclusion is reached for each one, updating the github issue (or pull request) with an appropriate comment , including a link to the publicly-accessible zulip archive, and also updating the labels as appropriate for the issue/pr

Santiago Pastorino (Mar 26 2020 at 04:11, on Zulip):

as the final picture would be great to interact with a Zulip bot that you say something like ...

Santiago Pastorino (Mar 26 2020 at 04:15, on Zulip):

@bot prioritization next and it pastes the next prioritization step

Santiago Pastorino (Mar 26 2020 at 04:15, on Zulip):

and at the end

Santiago Pastorino (Mar 26 2020 at 04:15, on Zulip):

@bot prioritization agenda and it build the agenda

Santiago Pastorino (Mar 26 2020 at 04:15, on Zulip):

:)

pnkfelix (Mar 26 2020 at 18:40, on Zulip):

/me wasn't picturing a Zulip bot, but rather a command line tool. But whatever is easiest to get off the ground, I guess.

pnkfelix (Mar 26 2020 at 18:41, on Zulip):

A command line tool has the advantage that it can build in the prompting for what the next steps/options are. that, as far as I know, is a little trickier with a chat bot.

LeSeulArtichaut (Mar 26 2020 at 18:52, on Zulip):

I can dive into that. I don’t mind spending time adding the functionality for a chat bot, but would we have the required infrastructure (servers etc...)

simulacrum (Mar 26 2020 at 18:55, on Zulip):

Triagebot already has some support for talking to Zulip, I would be unopposed to extending it. I suspect the terminal interface would be far easier, though.

Santiago Pastorino (Mar 26 2020 at 19:16, on Zulip):

@LeSeulArtichaut oh that's great, I was planning at some point to start with that

Santiago Pastorino (Mar 26 2020 at 19:17, on Zulip):

maybe we can write some requirements down or just start with something and keep improving it meanwhile we use it

LeSeulArtichaut (Mar 26 2020 at 21:15, on Zulip):

@Santiago Pastorino @pnkfelix Does a HackMD API exist though? I couldn't find it

pnkfelix (Mar 27 2020 at 14:10, on Zulip):

hmm, I did find this (codimd) which is an open-sourced stripped down version of hackmd from the same developers, and I think that has an API

pnkfelix (Mar 27 2020 at 14:10, on Zulip):

but that would only be useful if we were interested in hosting our own instance, and I don't think we are at that point

pnkfelix (Mar 27 2020 at 14:13, on Zulip):

There is the ability to sync with github, which we might be able to use in tandem with the github API ...?

pnkfelix (Mar 27 2020 at 14:16, on Zulip):

Having said all that, the tool doesn't need to sync with hackmd

pnkfelix (Mar 27 2020 at 14:17, on Zulip):

Specifically, If we're talking about a command line tool, then it can just have a command that prints out the agenda to the screen (or to a file, etc)

pnkfelix (Mar 27 2020 at 14:17, on Zulip):

To me the big wins here are interactions with github and zulip, not hackmd.

Wesley Wiser (Mar 27 2020 at 14:18, on Zulip):

If we're using the GH api, it could also just post the agenda as a gist

Wesley Wiser (Mar 27 2020 at 14:19, on Zulip):

But I agree, dumping the agenda to a file or stdout seems good enough for now

Santiago Pastorino (Mar 27 2020 at 15:37, on Zulip):

LeSeulArtichaut said:

Santiago Pastorino pnkfelix Does a HackMD API exist though? I couldn't find it

in my opinion this can be solved simpler

Santiago Pastorino (Mar 27 2020 at 15:38, on Zulip):

if we were having a zulip bot, when you say @prioritization_bot build-agenda it could just send the whole agenda as a message here

Santiago Pastorino (Mar 27 2020 at 15:38, on Zulip):

we just copy the whole thing and paste it on hackmd

Santiago Pastorino (Mar 27 2020 at 15:38, on Zulip):

that takes 10 seconds to do at max

Santiago Pastorino (Mar 27 2020 at 15:39, on Zulip):

but yeah, we could also place the agenda on github or something if we want more automation

Santiago Pastorino (Mar 27 2020 at 15:39, on Zulip):

the problem with the agenda in github in my opinion is that then you want to quickly modify it and make it accessible

Santiago Pastorino (Mar 27 2020 at 15:39, on Zulip):

Wesley Wiser said:

But I agree, dumping the agenda to a file or stdout seems good enough for now

as a zulip message or as Wesley have said, to stdout if it's a command like program

pnkfelix (Mar 27 2020 at 16:51, on Zulip):

can we talk about the pros/cons of zulip bot vs command line tool? I feel like @Santiago Pastorino keeps going back to that as a default, and I'm struggling to understand why

Santiago Pastorino (Mar 27 2020 at 16:51, on Zulip):

@pnkfelix seems like a good idea yeah

pnkfelix (Mar 27 2020 at 16:51, on Zulip):

The main reason to have a zulip bot that I can immediately grasp is that it would allow anyone to participate in driving the meeting

Santiago Pastorino (Mar 27 2020 at 16:52, on Zulip):

any automation will be better

pnkfelix (Mar 27 2020 at 16:52, on Zulip):

without having to download a tool, learn how to run it properly and about the idiosyncrasies of its interface

Santiago Pastorino (Mar 27 2020 at 16:52, on Zulip):

I think a zulip bot will be better because we would be running the meeting in zulip

Santiago Pastorino (Mar 27 2020 at 16:52, on Zulip):

so one can just ask for things here and the bot will reply

Santiago Pastorino (Mar 27 2020 at 16:52, on Zulip):

like

Santiago Pastorino (Mar 27 2020 at 16:52, on Zulip):

spastorino: @prioritization-bot next

Santiago Pastorino (Mar 27 2020 at 16:53, on Zulip):

prioritization-bot: next is Unprioritized T-compiler, you need to prioritize issues and remove nomination of the ones not worth discussing

pnkfelix (Mar 27 2020 at 16:53, on Zulip):

what's the typical model for a bot to maintain state like that?

Santiago Pastorino (Mar 27 2020 at 16:54, on Zulip):

or something like that

pnkfelix (Mar 27 2020 at 16:54, on Zulip):

does it e.g. maintain internal state per channel? Per (channel, user)?

Santiago Pastorino (Mar 27 2020 at 16:54, on Zulip):

well, I wanted to note that the bot would need to keep state, unsure how easy is to do that

pnkfelix (Mar 27 2020 at 16:54, on Zulip):

Right. If its not easy to make a bot keep state, then it seems like a no-go to me.

pnkfelix (Mar 27 2020 at 16:54, on Zulip):

If we cannot have the bot maintain state, then I don't think it will add much convenience

pnkfelix (Mar 27 2020 at 16:55, on Zulip):

since we'd have to encode the current state in the command we issue

Santiago Pastorino (Mar 27 2020 at 16:55, on Zulip):

pnkfelix said:

does it e.g. maintain internal state per channel? Per (channel, user)?

I guess it could be a global state, we could use things like @prioritization-bot start and the end

pnkfelix (Mar 27 2020 at 16:55, on Zulip):

(though I suppose the bot could, heh, prompt you with the next command you should send to it.)

Santiago Pastorino (Mar 27 2020 at 16:55, on Zulip):

@simulacrum is something like that possible?

Santiago Pastorino (Mar 27 2020 at 16:55, on Zulip):

pnkfelix said:

Right. If its not easy to make a bot keep state, then it seems like a no-go to me.

agreed

pnkfelix (Mar 27 2020 at 16:56, on Zulip):

The main problem with a command line tool is that we'd have to figure out how to make such a tool interface with the Zulip API

Santiago Pastorino (Mar 27 2020 at 16:56, on Zulip):

to be clear, I wasn't trying to discuss what's better or not. I was thinking that is an option to consider and mainly just dreaming a bit about possibilities :)

pnkfelix (Mar 27 2020 at 16:57, on Zulip):

it took me a little while to work out how to communicate via the Github API, but I've been hoping that the stuff I learned in doing that task would be at least semi-applicable to interfacing to the Zulip API

simulacrum (Mar 27 2020 at 16:57, on Zulip):

Global state is easy

Santiago Pastorino (Mar 27 2020 at 16:57, on Zulip):

cool

simulacrum (Mar 27 2020 at 16:57, on Zulip):

(and even e.g. per channel/per user), just stick it in the database

Santiago Pastorino (Mar 27 2020 at 16:57, on Zulip):

well @triagebot handle state per user actually

simulacrum (Mar 27 2020 at 16:58, on Zulip):

Hm, well, no, currently there's no per user state associated with the bot

Santiago Pastorino (Mar 27 2020 at 16:58, on Zulip):

but in this case I think it's overkill, global state is enough

simulacrum (Mar 27 2020 at 16:58, on Zulip):

(just with the triagebot server in general)

Santiago Pastorino (Mar 27 2020 at 16:58, on Zulip):

how does triagebot knows then my pending stuff?

simulacrum (Mar 27 2020 at 16:59, on Zulip):

Sorry, I meant that it's a single db for the web and Zulip interfaces

Santiago Pastorino (Mar 27 2020 at 16:59, on Zulip):

ahh ok ok

simulacrum (Mar 27 2020 at 16:59, on Zulip):

i.e. no zulip-specific state

simulacrum (Mar 27 2020 at 16:59, on Zulip):

But adding a new table is super cheap

Santiago Pastorino (Mar 27 2020 at 17:00, on Zulip):

anyway, if we were going with global state our problem would be solved because won't be zulip specific neither

Santiago Pastorino (Mar 27 2020 at 17:00, on Zulip):

maybe I can try something out today

Santiago Pastorino (Mar 27 2020 at 17:01, on Zulip):

@simulacrum is there some repo I should take a look at? to use as an example

simulacrum (Mar 27 2020 at 17:01, on Zulip):

This is all in triagebot

simulacrum (Mar 27 2020 at 17:01, on Zulip):

zulip.rs iirc

Santiago Pastorino (Mar 27 2020 at 17:04, on Zulip):

and how do you deploy this?

Santiago Pastorino (Mar 27 2020 at 17:04, on Zulip):

I guess this is where the Docker setup I see on the repo enters

simulacrum (Mar 27 2020 at 17:29, on Zulip):

yeah it'll auto deploy on merge to master

LeSeulArtichaut (Mar 27 2020 at 18:51, on Zulip):

I’ve started to dive into the source code for @triagebot

triagebot (Mar 27 2020 at 18:51, on Zulip):

Unknown command.

LeSeulArtichaut (Mar 27 2020 at 18:51, on Zulip):

Oops, didn’t mean to wake you up :D

pnkfelix (Mar 27 2020 at 18:53, on Zulip):

/me sits and wonders if it is just inevitable that people are going to make this work via @triagebot, by hook or by crook

pnkfelix (Mar 27 2020 at 18:54, on Zulip):

I guess it shouldn't matter to me, as long as the automation gets done, one way or another

simulacrum (Mar 27 2020 at 18:58, on Zulip):

@pnkfelix I personally think that the zulip bot is probably the right interface precisely because it has the natural "you just need to do what someone else did (publicly)" vs. the CLI that is sort of everyone-discovers-themselves. I also think that the underlying logic is likely to be pretty similar, so I imagine that if we end up deciding the bot is not working well we can change the frontend to a CLI vs. the bot fairly easily

pnkfelix (Mar 27 2020 at 18:59, on Zulip):

The interfacing with Zulip itself and with Github are likely to be reused?

pnkfelix (Mar 27 2020 at 18:59, on Zulip):

to me those seem like the biggest hurdles

pnkfelix (Mar 27 2020 at 18:59, on Zulip):

for a CLI

pnkfelix (Mar 27 2020 at 18:59, on Zulip):

but maybe I'm over-simplifying the custom logic we need that is specific to the pre-triage and triage processes.

simulacrum (Mar 27 2020 at 19:06, on Zulip):

oh well the zulip interface is pretty much ready to go

simulacrum (Mar 27 2020 at 19:06, on Zulip):

i.e. the API etc is already done

simulacrum (Mar 27 2020 at 19:06, on Zulip):

and github (and team api) etc are also available mostly in triagebot

simulacrum (Mar 27 2020 at 19:07, on Zulip):

which is part of why I'd personally think that for something interactive zulip is probably easier

Santiago Pastorino (Mar 27 2020 at 20:51, on Zulip):

LeSeulArtichaut said:

I’ve started to dive into the source code for triagebot

hey, were you able to do something?

Santiago Pastorino (Mar 27 2020 at 20:51, on Zulip):

I've just finished some stuff and could dive, pretty late though :)

Santiago Pastorino (Mar 27 2020 at 20:52, on Zulip):

but anyway was interested in hearing from you

LeSeulArtichaut (Mar 27 2020 at 21:47, on Zulip):

I mostly looked up the code to try to understand the structure of the bot.

LeSeulArtichaut (Mar 27 2020 at 21:49, on Zulip):

I just need to find a design for how to handle a meeting (steps, interactions with the Zulip and GitHub APIs) so that it could be flexible and extensible, so that if the meeting procedure changes, implementing the changes into the bot wouldn’t be hard

LeSeulArtichaut (Mar 27 2020 at 21:50, on Zulip):

I’m assuming a meeting/pre-meeting will always be a sequence of steps of some sort, does that look right? @Santiago Pastorino @pnkfelix

Santiago Pastorino (Mar 27 2020 at 22:09, on Zulip):

unsure I got your question

LeSeulArtichaut (Mar 27 2020 at 22:49, on Zulip):

I meant, is it safe to assume that any (future) meeting procedure will always follow a step-by-step construction?

Santiago Pastorino (Mar 27 2020 at 22:51, on Zulip):

yes

Santiago Pastorino (Mar 27 2020 at 22:51, on Zulip):

if you take a look at the procedure document, the meeting should follow the order layed out there

LeSeulArtichaut (Apr 22 2020 at 22:04, on Zulip):

@pnkfelix @Santiago Pastorino I think it is time to revisit this?

LeSeulArtichaut (Apr 22 2020 at 22:04, on Zulip):

I mainly thought about the design, so I wanted to have your feedback

LeSeulArtichaut (Apr 22 2020 at 22:11, on Zulip):
LeSeulArtichaut (Apr 22 2020 at 22:13, on Zulip):
Santiago Pastorino (Apr 22 2020 at 22:34, on Zulip):

I'd do it as stateless as possible, I guess all the state you would need to have is where are we in the meeting, like having a next that goes to the next step

Santiago Pastorino (Apr 22 2020 at 22:35, on Zulip):

but about issues and all that could be stateless we just mention the issue number and what we want to do

Santiago Pastorino (Apr 22 2020 at 22:36, on Zulip):

LeSeulArtichaut said:

those commands already exist on github, so we would need to figure those from Zulip

Santiago Pastorino (Apr 22 2020 at 22:36, on Zulip):

we may want some way to have only a restricted set of Zulip users able to figure that

Santiago Pastorino (Apr 22 2020 at 22:36, on Zulip):

just in case :)

pnkfelix (Apr 22 2020 at 23:35, on Zulip):

is it difficult to figure out the zulip-archive URL to link to from the bot itself? I.e. are the message ID's over there predictable?

pnkfelix (Apr 22 2020 at 23:36, on Zulip):

(I'm imagining having the githubs comments injected by the bot link back to the relevant point in the meeting on zulip-archive. Which may be tricky for various reasons; e.g. the zulip-archive can have a bit of lag before it updates.)

simulacrum (Apr 22 2020 at 23:37, on Zulip):

Zulip archive should match Zulip I think; but I was unable to identify a nice way in the bot to identify Zulip urls at all. Probably just didn't look hard enough...

Santiago Pastorino (Apr 29 2020 at 14:08, on Zulip):

I'm thinking a bit if I should do something very tiny and also very bad that at least help with a couple of things we do during pre-triage

Santiago Pastorino (Apr 29 2020 at 14:08, on Zulip):

cc @LeSeulArtichaut

Santiago Pastorino (Apr 29 2020 at 14:08, on Zulip):

unsure if you've tried out something

LeSeulArtichaut (Apr 29 2020 at 14:08, on Zulip):

I didn't, if you want to do it go ahead

Santiago Pastorino (Apr 29 2020 at 14:08, on Zulip):

but for now, I think I'd do something for console, that I can run quickly and gives me the searches

Santiago Pastorino (Apr 29 2020 at 14:09, on Zulip):

ok, will see if I have time and can do something

Santiago Pastorino (Apr 29 2020 at 14:09, on Zulip):

I remember that we've talked about reusing code from rustbot was it?

LeSeulArtichaut (Apr 29 2020 at 14:14, on Zulip):

I need to implement pagination for a PR I have open

Santiago Pastorino (Apr 29 2020 at 14:15, on Zulip):

a PR about?

Santiago Pastorino (Apr 29 2020 at 14:15, on Zulip):

link?

LeSeulArtichaut (Apr 29 2020 at 14:15, on Zulip):

rust-lang/triagebot#477

Santiago Pastorino (Apr 29 2020 at 14:17, on Zulip):

ohh, I see

Santiago Pastorino (Apr 29 2020 at 14:17, on Zulip):

I'll potentially have the same problem

LeSeulArtichaut (Apr 29 2020 at 14:18, on Zulip):

I didn't implement pagination yet (to fetch things from GH)

LeSeulArtichaut (Apr 29 2020 at 14:18, on Zulip):

But I'll need to do it, and so will you

Santiago Pastorino (Apr 29 2020 at 14:18, on Zulip):

what would happen if you don't? I guess you would get just the first page?

LeSeulArtichaut (Apr 29 2020 at 14:18, on Zulip):

You get 30 issues

LeSeulArtichaut (Apr 29 2020 at 14:18, on Zulip):

You can request up to 100 at a time

Santiago Pastorino (Apr 29 2020 at 14:18, on Zulip):

ok for my use case that's enough :)

Santiago Pastorino (Apr 29 2020 at 14:18, on Zulip):

I mean, I told you that I was going to do something quick and dirty :joy:

Santiago Pastorino (Apr 29 2020 at 14:19, on Zulip):

I want to do something that takes me less time that the time it takes me do this manually

Santiago Pastorino (Apr 29 2020 at 14:19, on Zulip):

we can improve later

Santiago Pastorino (Apr 29 2020 at 14:19, on Zulip):

we never have more than 30/100 issues to prioritize or nominations or anything like that

LeSeulArtichaut (Apr 29 2020 at 14:20, on Zulip):

However if you need to count P-high or P-medium issues...

LeSeulArtichaut (Apr 29 2020 at 14:21, on Zulip):

I mean, there is a point where you'll be limited

LeSeulArtichaut (Apr 29 2020 at 14:21, on Zulip):

Another thing that you can reuse from triagebot is the response structs

Santiago Pastorino (Apr 29 2020 at 14:36, on Zulip):

I think in the end, if @simulacrum allows us to do so, we would want this to be integrated in rustbot

LeSeulArtichaut (Apr 29 2020 at 14:36, on Zulip):

I guess he already said he wouldn't mind

Santiago Pastorino (Apr 29 2020 at 14:36, on Zulip):

yeah

LeSeulArtichaut (Apr 29 2020 at 14:37, on Zulip):

simulacrum said:

Triagebot already has some support for talking to Zulip, I would be unopposed to extending it. I suspect the terminal interface would be far easier, though.

Santiago Pastorino (Apr 29 2020 at 14:37, on Zulip):

yeah, going to start with terminal interface so we have something :)

Santiago Pastorino (Apr 29 2020 at 17:03, on Zulip):

was trying some of this stuff out

Santiago Pastorino (Apr 29 2020 at 17:03, on Zulip):

is there a way in github api to filter issues that do not contain certain label

Santiago Pastorino (Apr 29 2020 at 17:03, on Zulip):

what you do with "-label" in the ui

Santiago Pastorino (Apr 29 2020 at 17:04, on Zulip):

from what I can see here https://developer.github.com/v3/issues/#parameters I don't see a way to do it

Santiago Pastorino (Apr 29 2020 at 17:04, on Zulip):

cc @LeSeulArtichaut @simulacrum

Santiago Pastorino (Apr 29 2020 at 17:05, on Zulip):

I guess I could manually do it but ... :)

DPC (Apr 29 2020 at 17:09, on Zulip):

not sure if we should use this but we could consider it: https://github.com/google/triage-party

simulacrum (Apr 29 2020 at 17:13, on Zulip):

Seems like a big commitment to use it, maybe misunderstanding though.

@Santiago Pastorino why do you want to ignore some label?

Santiago Pastorino (Apr 29 2020 at 17:13, on Zulip):

this is the kind of searches we have in our process

Santiago Pastorino (Apr 29 2020 at 17:14, on Zulip):

want to look for things that have not been prioritized

Santiago Pastorino (Apr 29 2020 at 17:14, on Zulip):

to the search is -label:P-high ... and all that chain :)

Santiago Pastorino (Apr 29 2020 at 17:14, on Zulip):

for instance

Santiago Pastorino (Apr 29 2020 at 17:15, on Zulip):

this is the search of unprioritized I-prioritize https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3AT-compiler+-label%3AP-critical+-label%3AP-high+-label%3AP-medium+-label%3AP-low+label%3AI-prioritize

Santiago Pastorino (Apr 29 2020 at 17:15, on Zulip):

which I'd claim we can just search for I-prioritize, if they are already prioritized we should remove I-prioritize

Santiago Pastorino (Apr 29 2020 at 17:15, on Zulip):

but there are others

Santiago Pastorino (Apr 29 2020 at 17:15, on Zulip):

Beta regressions without P-label: https://github.com/rust-lang/rust/issues?q=is%3Aopen+label%3Aregression-from-stable-to-beta+-label%3AP-critical+-label%3AP-high+-label%3AP-medium+-label%3AP-low+-label%3AT-libs+-label%3AT-infra+-label%3AT-rustdoc

simulacrum (Apr 29 2020 at 17:15, on Zulip):

Ah yeah, I think there's not really great support... hm

Santiago Pastorino (Apr 29 2020 at 17:16, on Zulip):

I think I need to search and filter manually

Santiago Pastorino (Apr 29 2020 at 17:16, on Zulip):

:(

simulacrum (Apr 29 2020 at 17:16, on Zulip):

Well if you're already limiting to just beta regressions for example that shouldn't be a problem

Santiago Pastorino (Apr 29 2020 at 17:18, on Zulip):

what do you mean? that we can just get them all and it's manageable?

DPC (Apr 29 2020 at 17:29, on Zulip):

Seems like a big commitment to use it, maybe misunderstanding though.

yeah definitely..

Santiago Pastorino (Apr 29 2020 at 21:26, on Zulip):

@LeSeulArtichaut @simulacrum was checking again things about this and prs are even more problematic

Santiago Pastorino (Apr 29 2020 at 21:26, on Zulip):

https://developer.github.com/v3/pulls/#parameters

Santiago Pastorino (Apr 29 2020 at 21:26, on Zulip):

doesn't seem to be a way to filter with labels

Santiago Pastorino (Apr 29 2020 at 21:27, on Zulip):

so it's completely useless for what we want

Santiago Pastorino (Apr 29 2020 at 21:28, on Zulip):

or we should retrieve all the PRs and filter manually, which ugh :)

Last update: Jun 05 2020 at 23:05UTC