Stream: rust-survey-2019

Topic: Publishing Results


XAMPPRocky (Feb 07 2020 at 12:28, on Zulip):

Hey everyone, does anyone know what the current status of publishing the results is? The roadmap said that we intended to publish in January, and well it's past that :stuck_out_tongue: I know there is a draft somewhere, but I don't know how close it is to being ready? cc @Jasun Carr @nikomatsakis

apiraino (Feb 07 2020 at 14:12, on Zulip):

yeah I'll add that if some help is needed, I am available to help (provided I can actually help on something)

nikomatsakis (Feb 07 2020 at 14:20, on Zulip):

I know the status :)

nikomatsakis (Feb 07 2020 at 14:20, on Zulip):

Thanks for raising it

nikomatsakis (Feb 07 2020 at 14:21, on Zulip):

I've been wondering how to distribute some of this work from my shoulders :)

nikomatsakis (Feb 07 2020 at 14:22, on Zulip):

So basically, there is a Dropbox paper doc that we've been using to look at the measurements. I've been communicating with @Jasun Carr This is not private but it contains "raw and unfiltered" thoughts, and could potentially expose data that we should keep private, so I've made it require invitation to view.

nikomatsakis (Feb 07 2020 at 14:23, on Zulip):

Er, I mixed up some sentences there

nikomatsakis (Feb 07 2020 at 14:23, on Zulip):

I've been communicating with @Jasun Carr in the form of dropbox "todo items" and comments, is what I meant to say.

nikomatsakis (Feb 07 2020 at 14:23, on Zulip):

@pnkfelix and I spent some time going through the data, we got about half-way, leaving comments or other thoughts.

nikomatsakis (Feb 07 2020 at 14:23, on Zulip):

We did indeed identify some interesting things

nikomatsakis (Feb 07 2020 at 14:23, on Zulip):

I was hoping to schedule some time next week to try and finish up

nikomatsakis (Feb 07 2020 at 14:24, on Zulip):

The paper doc might be a bit confusing, we tried a few different formats, but the context at the top is the structure that seems to work best -- it basically mirrors the structure of the survey, with notes and annotations on the results from each question

nikomatsakis (Feb 07 2020 at 14:25, on Zulip):

I was hoping to schedule some time next week to try and finish up

I'm very interested to get help here, either in the form of sitting together and working through things, or in a more async fashion

nikomatsakis (Feb 07 2020 at 14:25, on Zulip):

I do have the goal of getting some kind of blog post out ASAP, but I think another goal would be identifying things that we ought to try and dig into further

nikomatsakis (Feb 07 2020 at 14:26, on Zulip):

In particular, one result that really leapt out at @pnkfelix and I was that a significant percentage (21%) of active Rust users say they don't feel productive with Rust. Moreover, when we cross-indexed that with how long people had been using Rust, this turned out to be true of people at all levels of experience. There could be many interpretations of this -- e.g., @matklad commented that they don't feel productive due to lack of IDE support -- but it's still a result that bears introspection.

nikomatsakis (Feb 07 2020 at 14:27, on Zulip):

I've been trying to compare against previous year results, I don't think we found any real significant deviations yet.

nikomatsakis (Feb 07 2020 at 14:27, on Zulip):

To do those comparisons, I was just looking at the Google Forms data for English -- one thing we found is that the results from other languages and the results from English are generally compatible, so that's good enough to get a rough sense of last year's data. For this year, I've been pinging @Jasun Carr who has supplied results that include all languages.

nikomatsakis (Feb 07 2020 at 14:28, on Zulip):

I think that's about it :)

apiraino (Feb 07 2020 at 16:56, on Zulip):

thank you @nikomatsakis for the summary :-)
Looks to me that the data is there but we need to ponder on the results in order to extract meaningful and actionable thoughts, correct?
Do you think I can have a look at the Dropbox doc? Clearly, if you're comfortable sharing it (I've issued a request, but feel free to ignore it).
Else, we can maybe hook up at some point in the future and you can distill some points you feel that could be useful for me.
My point is: understand whether i can help with something or rather leave it to analysts better suited for the task.

nikomatsakis (Feb 10 2020 at 21:42, on Zulip):

I'm wondering if it would make sense to try and schedule a time to do some more discussion

nikomatsakis (Feb 10 2020 at 21:42, on Zulip):

e.g. I could imagine setting up a zoom call and trying to go through more of the results together

nikomatsakis (Feb 10 2020 at 21:43, on Zulip):

otoh I could try to get myself out of the process, as I think I'm a bottleneck :/

nikomatsakis (Feb 10 2020 at 21:44, on Zulip):

@apiraino I'll add you to the dropbox doc, if I can figure out how :P but yeah I'd ask that you try to keep the info to yourself for now, though I don't really think there's anything particularly shocking in there.

apiraino (Feb 10 2020 at 23:17, on Zulip):

@nikomatsakis thanks I'll have a look at the doc

apiraino (Feb 16 2020 at 18:46, on Zulip):

@nikomatsakis I managed to have a look at the Dropbox document with the 2019 survey results. There's a lot of info there and (imo) some work to compare those number to those of 2018 and extract a human readable summary. Yeah, I know, "welcome, Mr. Obvious" :-)
What I mean is, if you feel that we are stuck with the data extraction, we can try drafting a document and build it point by point, trying to attack the questions that are not clear, one by one.
On the other hand, if this document (that eventually will become the blog post) is never started, I feel that the process of data extraction is blocked because the task looks daunting.
Maybe all this is already happening, but who's actively working of the survey results?

apiraino (Feb 16 2020 at 18:47, on Zulip):

I do have the goal of getting some kind of blog post out ASAP, but I think another goal would be identifying things that we ought to try and dig into further

Do you (or anyone) have some draft of items to be put into this blog post?

nikomatsakis (Feb 19 2020 at 12:36, on Zulip):

I have created an outline document and I am populating it with interesting results

nikomatsakis (Feb 19 2020 at 13:12, on Zulip):

@Jasun Carr I've been posting a few more questions, as you no doubt saw, but it'd be good if you took a look at my draft blog post outline and maybe prioritize things that appear in there

XAMPPRocky (Feb 19 2020 at 13:20, on Zulip):

@nikomatsakis I don't know if you plan to reuse the graphs from the original documents or not, if not I can spend some time creating some for the blog post if not.

nikomatsakis (Feb 19 2020 at 13:21, on Zulip):

@XAMPPRocky I did not plan to re-use the graphs from the original paper -- I had a feeling we'd be better off making a fresh google spreadsheet with cherry-picked results and custom charts

nikomatsakis (Feb 19 2020 at 13:21, on Zulip):

for one thing, I would want percentages labeled

nikomatsakis (Feb 19 2020 at 13:21, on Zulip):

and perhaps fewer pie charts:)

XAMPPRocky (Feb 19 2020 at 13:22, on Zulip):

Alright, once the outline is a bit more settled I can get started on a google sheet for it.

nikomatsakis (Feb 19 2020 at 13:22, on Zulip):

(actually, I wonder if we could even make that particular spreadsheet world readable, to let people dive in more easily)

nikomatsakis (Feb 19 2020 at 13:22, on Zulip):

i.e., it'd be kind of neat if you could click on a chart and get taken to the underlying data

nikomatsakis (Feb 19 2020 at 13:22, on Zulip):

since we'd only be exposing aggregates that we're publishing anyway it doesn't seem like a privacy violation

nikomatsakis (Feb 19 2020 at 13:23, on Zulip):

Regarding the outline, i'm starting to see a kind of narrative flow

nikomatsakis (Feb 19 2020 at 13:23, on Zulip):

basically, "Why not use Rust" suggests the following major blockers (roughly in order):

nikomatsakis (Feb 19 2020 at 13:24, on Zulip):

adoption is the biggest one, of course, but that's not directly actionable, and presumably the secondary things cited are part of what is blocking adoption

nikomatsakis (Feb 19 2020 at 13:24, on Zulip):

so then I am organizing the results a bit by those themes

nikomatsakis (Feb 19 2020 at 13:25, on Zulip):

also:learning point for next year, dropbox paper, despite being so nice, really doesn't scale to docs of this size :)

nikomatsakis (Feb 19 2020 at 13:25, on Zulip):

I wonder if google docs would do better

XAMPPRocky (Feb 19 2020 at 13:25, on Zulip):

For the point about providing translated documentation I'll reach out @Manish Goregaokar I remember they said they were in process of starting up the localisation WG.

nikomatsakis (Feb 19 2020 at 13:26, on Zulip):

yes, they've been doing things there, not sure of current status (but they do monitor Zulip, so I imagine they'll weigh in)

nikomatsakis (Feb 19 2020 at 13:27, on Zulip):

@Jasun Carr to help in tracking whta to prioritize, I'm going to add "to do" items in the outline that correspond also to "to do" items in the main survey doc. IT'd be good to add the data into the main survey doc though, and we'll bring things over to the outline.

nikomatsakis (Feb 19 2020 at 13:29, on Zulip):

Oh I guess the correlary to what we were saying above is that we would want to also have the raw data for all charts that are added

nikomatsakis (Feb 19 2020 at 13:31, on Zulip):

OK, I'm pretty happy with that outline

nikomatsakis (Feb 19 2020 at 13:31, on Zulip):

I should go do something else now :)

XAMPPRocky (Feb 19 2020 at 13:43, on Zulip):

@nikomatsakis On dropbox paper alternatives: It might be worth trying hackmd, they just added support for rendering visualisations in documents in JSON using Vega, which can use Google Sheets as a data source. https://vega.github.io/vega-lite/

nikomatsakis (Feb 19 2020 at 13:43, on Zulip):

hmm, maybe

nikomatsakis (Feb 19 2020 at 13:43, on Zulip):

my main concern with hackmd has been their comment system is very ungood

nikomatsakis (Feb 19 2020 at 13:44, on Zulip):

to be more specific, it's not obvious to most people how to add comments, replying is sort of awkward, and there's no "Resolve" button to clear away comments en masse (only way I found to remove them is manually deleting them)

nikomatsakis (Feb 19 2020 at 13:44, on Zulip):

in general, the collaboration workflow feels pretty limited

nikomatsakis (Feb 19 2020 at 13:45, on Zulip):

oh and

Manish Goregaokar (Feb 19 2020 at 17:09, on Zulip):

@XAMPPRocky @Jasun Carr So basically the current status of translations stuff is that:

For docs specifically it's mostly blocked on rustdoc. I can write down a fleshed out version of the plan as a localization-wg issue and perhaps contributors could pick it up

Manish Goregaokar (Feb 19 2020 at 20:09, on Zulip):

https://github.com/rust-lang/project-localisation/issues/5

Manish Goregaokar (Feb 19 2020 at 20:09, on Zulip):

filed that issue for the stdlib ocs

Jasun Carr (Feb 19 2020 at 21:18, on Zulip):

nikomatsakis said:

Jasun Carr I've been posting a few more questions, as you no doubt saw, but it'd be good if you took a look at my draft blog post outline and maybe prioritize things that appear in there

Working through that now. Will focus on that - and apologies for the really long comment reply I just left.

nikomatsakis (Feb 27 2020 at 14:43, on Zulip):

BTW, I was just talking to @Nell Shamrell-Harrington and she might be game to work on turning the blog post outline into an actual blog post

nikomatsakis (Feb 27 2020 at 14:43, on Zulip):

cc'ing her into this stream :)

Nell Shamrell-Harrington (Feb 27 2020 at 14:43, on Zulip):

Yay!

nikomatsakis (Feb 27 2020 at 14:44, on Zulip):

@Nell Shamrell-Harrington some links for you:

Jasun Carr (Feb 28 2020 at 21:44, on Zulip):

@Nell Shamrell-Harrington - If there's anything unclear, feel free to ping me.

Nell Shamrell-Harrington (Mar 06 2020 at 21:17, on Zulip):

Finally have some time to work on this - drafting the blog post now, should be ready for review early next week

nikomatsakis (Mar 07 2020 at 12:43, on Zulip):

@Nell Shamrell-Harrington great!

Nell Shamrell-Harrington (Mar 11 2020 at 21:54, on Zulip):

While many of the results featured in https://paper.dropbox.com/doc/Rust-Survey-2019--Av_GidY5RWZa3uBaKH24XqHLAg-z6ZJYlpf1kBQp3uFNxa2q have graphs generated, not all of them do. There are a few without graphs that I think we still want to feature - like "What platform are you developing on?" How are these graphs generated?

nikomatsakis (Mar 12 2020 at 11:39, on Zulip):

@Nell Shamrell-Harrington I dont' think we decided how best to do this -- @Jasun Carr was generating them, but I was kind of thinking it might be better to get the raw data and generate the graphs via google spreadsheets or something, since I thought the format of the graphs that were pasted in was often not ideal

nikomatsakis (Mar 12 2020 at 11:39, on Zulip):

e.g., they didn't include labels like percentages

nikomatsakis (Mar 12 2020 at 11:40, on Zulip):

or were structured in pie charts or formats that I personally find a bit harder to read

Jasun Carr (Mar 12 2020 at 11:45, on Zulip):

All of the data used for the graphs is included in the Excel spreadsheet
that was uploaded to the Rust 2019 Survey folder on Google Drive. Should be
easy to import into Google sheets.

XAMPPRocky (Mar 12 2020 at 11:50, on Zulip):

Has the folder been shared? I think I've only had access to the sheet and the dropbox paper.

Nell Shamrell-Harrington (Mar 12 2020 at 20:52, on Zulip):

I also don't believe I have access to that folder - could someone with access share it with nell@mozilla.com?

I will have a draft that does not include all graphs by tomorrow, but the final post is going to stretch into next week at least if I need to generate more graphs for it.

Nell Shamrell-Harrington (Mar 12 2020 at 22:34, on Zulip):

Alright - very early draft of blog post (with some things missing) is up

Nell Shamrell-Harrington (Mar 12 2020 at 22:35, on Zulip):

What I need now

XAMPPRocky (Mar 13 2020 at 00:02, on Zulip):

@Nell Shamrell-Harrington I can help with making some of the graphs if need be, and I'll write up the "Feeling Welcome" section since I have that kind of written already.

Nell Shamrell-Harrington (Mar 13 2020 at 00:03, on Zulip):

@XAMPPRocky I would greatly appreciate you making some of the graphs and writing up the "Feeling Welcome" section!

nikomatsakis (Mar 13 2020 at 14:54, on Zulip):

Nell Shamrell-Harrington said:

What I need now

@Nell Shamrell-Harrington access granted, I'll check out the PR, thanks!

Nell Shamrell-Harrington (Mar 13 2020 at 16:58, on Zulip):

thank you!

nikomatsakis (Mar 13 2020 at 19:52, on Zulip):

So @Nell Shamrell-Harrington I was skimming the post -- obviously still some bits to fill in. I couldnt' quite tell if it matched what I had in mind from the outline -- for one thing, I think it included a lot less data, which may be good (I probably went overboard). But I remember I was trying to structure a bit of a "narrative", I think based around this question

the idea being that "adoption" is the biggest reason, but it's not "actionable", so what blocks adoption? probably those next items, and then structure the remainder of the post around those themes

Nell Shamrell-Harrington (Mar 13 2020 at 20:03, on Zulip):

The first draft was, indeed, meant to be just a draft to give us something to work with :). I incorporated most of the data in the outline, but not all of it as there was quite a bit. I was tailoring the narrative to last year's post, but am happy to take another stab at it using that narrative (which I do think is important). Are you thinking of starting off the post with the question "Why did you stop using Rust?" and structuring the whole post around that question? Or just one subsection of the post?

Nell Shamrell-Harrington (Mar 24 2020 at 22:56, on Zulip):

Hello all - Erin and I made some updates to https://github.com/rust-lang/blog.rust-lang.org/pull/544

Nell Shamrell-Harrington (Mar 24 2020 at 22:56, on Zulip):

There are currently four graphs missing, which I have listed in a comment on the PR

Nell Shamrell-Harrington (Mar 24 2020 at 22:56, on Zulip):

I would LOVE help generating those graphs

Nell Shamrell-Harrington (Mar 24 2020 at 22:57, on Zulip):

Although there is more data that we collected which we can also graph and analyze...I'm leaning toward posting this as it is (once we get the missing graphs) to get the survey info out there. We can of course do follow on posts with more data if we like.

XAMPPRocky (Mar 25 2020 at 08:17, on Zulip):

@Nell Shamrell-Harrington This won't help you now that you've put already put the graphs in there, but a handy thing I just learned is you can publish indivudual charts from a google sheet, and use them as images on other sites. So then if you need to change something about the chart, you just have to change the chart on google sheets and don't have to reupload them.

I just did this for publishing benchmarks on Tokei, so you can see how it looks. https://github.com/XAMPPRocky/tokei/releases

simulacrum (Mar 31 2020 at 15:39, on Zulip):

(Deleted a message above, which was causing Zulip to fail to load. Content:

Jasun Carr to help in tracking whta to prioritize, I'm going to add "to do" items in the outline that correspond also to "to do" items in the main survey doc What libraries do you consider critical in the Rust ecosystem?. IT'd be good to add the data into the main survey doc though, and we'll bring things over to the outline.

nikomatsakis (Mar 31 2020 at 16:25, on Zulip):

??

simulacrum (Mar 31 2020 at 16:49, on Zulip):

see https://rust-lang.zulipchat.com/#narrow/stream/122653-zulip/topic/invalid.20url.20bug

Nell Shamrell-Harrington (Apr 01 2020 at 20:18, on Zulip):

New draft of the survey blog post (this one based on the outline doc) is up - I indicated a few places where I could use help developing more copy/analysis https://github.com/rust-lang/blog.rust-lang.org/pull/555

nikomatsakis (Apr 02 2020 at 15:00, on Zulip):

Yay!

Nell Shamrell-Harrington (Apr 03 2020 at 20:09, on Zulip):

Hello all -

Nell Shamrell-Harrington (Apr 03 2020 at 20:09, on Zulip):

Glad to see the comments on https://github.com/rust-lang/blog.rust-lang.org/pull/555!

Nell Shamrell-Harrington (Apr 03 2020 at 20:10, on Zulip):

A lot of them are around the display/colors of the graphs - which I did not generate, I pulled them from https://github.com/rust-lang/blog.rust-lang.org/pull/555

Nell Shamrell-Harrington (Apr 03 2020 at 20:11, on Zulip):

I do have access to the raw excel spreadsheets with the data on them, and we have reached out to @Jasun Carr asking how he generated the graphs (he said he would get back to us as soon as he can)

Nell Shamrell-Harrington (Apr 03 2020 at 20:12, on Zulip):

Once we get those instructions, would anyone be willing to volunteer to generate edited graphs or new graphs? I have quite a bit on my plate now and don't know that I will have the time to generate the graphs myself

Nell Shamrell-Harrington (Apr 03 2020 at 20:18, on Zulip):

@XAMPPRocky I think you expressed some interest in the graphs - once we get guidance on how to generate them, would you be interested in helping?

XAMPPRocky (Apr 03 2020 at 20:19, on Zulip):

Yeah I can do that.

Nell Shamrell-Harrington (Apr 03 2020 at 20:23, on Zulip):

ty!

Nell Shamrell-Harrington (Apr 03 2020 at 20:23, on Zulip):

what would be a good email to share the spreadsheets with?

Last update: Jul 02 2020 at 19:50UTC