Stream: rust-survey-2019

Topic: Publishing Results


XAMPPRocky (Feb 07 2020 at 12:28, on Zulip):

Hey everyone, does anyone know what the current status of publishing the results is? The roadmap said that we intended to publish in January, and well it's past that :stuck_out_tongue: I know there is a draft somewhere, but I don't know how close it is to being ready? cc @Jasun Carr @nikomatsakis

apiraino (Feb 07 2020 at 14:12, on Zulip):

yeah I'll add that if some help is needed, I am available to help (provided I can actually help on something)

nikomatsakis (Feb 07 2020 at 14:20, on Zulip):

I know the status :)

nikomatsakis (Feb 07 2020 at 14:20, on Zulip):

Thanks for raising it

nikomatsakis (Feb 07 2020 at 14:21, on Zulip):

I've been wondering how to distribute some of this work from my shoulders :)

nikomatsakis (Feb 07 2020 at 14:22, on Zulip):

So basically, there is a Dropbox paper doc that we've been using to look at the measurements. I've been communicating with @Jasun Carr This is not private but it contains "raw and unfiltered" thoughts, and could potentially expose data that we should keep private, so I've made it require invitation to view.

nikomatsakis (Feb 07 2020 at 14:23, on Zulip):

Er, I mixed up some sentences there

nikomatsakis (Feb 07 2020 at 14:23, on Zulip):

I've been communicating with @Jasun Carr in the form of dropbox "todo items" and comments, is what I meant to say.

nikomatsakis (Feb 07 2020 at 14:23, on Zulip):

@pnkfelix and I spent some time going through the data, we got about half-way, leaving comments or other thoughts.

nikomatsakis (Feb 07 2020 at 14:23, on Zulip):

We did indeed identify some interesting things

nikomatsakis (Feb 07 2020 at 14:23, on Zulip):

I was hoping to schedule some time next week to try and finish up

nikomatsakis (Feb 07 2020 at 14:24, on Zulip):

The paper doc might be a bit confusing, we tried a few different formats, but the context at the top is the structure that seems to work best -- it basically mirrors the structure of the survey, with notes and annotations on the results from each question

nikomatsakis (Feb 07 2020 at 14:25, on Zulip):

I was hoping to schedule some time next week to try and finish up

I'm very interested to get help here, either in the form of sitting together and working through things, or in a more async fashion

nikomatsakis (Feb 07 2020 at 14:25, on Zulip):

I do have the goal of getting some kind of blog post out ASAP, but I think another goal would be identifying things that we ought to try and dig into further

nikomatsakis (Feb 07 2020 at 14:26, on Zulip):

In particular, one result that really leapt out at @pnkfelix and I was that a significant percentage (21%) of active Rust users say they don't feel productive with Rust. Moreover, when we cross-indexed that with how long people had been using Rust, this turned out to be true of people at all levels of experience. There could be many interpretations of this -- e.g., @matklad commented that they don't feel productive due to lack of IDE support -- but it's still a result that bears introspection.

nikomatsakis (Feb 07 2020 at 14:27, on Zulip):

I've been trying to compare against previous year results, I don't think we found any real significant deviations yet.

nikomatsakis (Feb 07 2020 at 14:27, on Zulip):

To do those comparisons, I was just looking at the Google Forms data for English -- one thing we found is that the results from other languages and the results from English are generally compatible, so that's good enough to get a rough sense of last year's data. For this year, I've been pinging @Jasun Carr who has supplied results that include all languages.

nikomatsakis (Feb 07 2020 at 14:28, on Zulip):

I think that's about it :)

apiraino (Feb 07 2020 at 16:56, on Zulip):

thank you @nikomatsakis for the summary :-)
Looks to me that the data is there but we need to ponder on the results in order to extract meaningful and actionable thoughts, correct?
Do you think I can have a look at the Dropbox doc? Clearly, if you're comfortable sharing it (I've issued a request, but feel free to ignore it).
Else, we can maybe hook up at some point in the future and you can distill some points you feel that could be useful for me.
My point is: understand whether i can help with something or rather leave it to analysts better suited for the task.

nikomatsakis (Feb 10 2020 at 21:42, on Zulip):

I'm wondering if it would make sense to try and schedule a time to do some more discussion

nikomatsakis (Feb 10 2020 at 21:42, on Zulip):

e.g. I could imagine setting up a zoom call and trying to go through more of the results together

nikomatsakis (Feb 10 2020 at 21:43, on Zulip):

otoh I could try to get myself out of the process, as I think I'm a bottleneck :/

nikomatsakis (Feb 10 2020 at 21:44, on Zulip):

@apiraino I'll add you to the dropbox doc, if I can figure out how :P but yeah I'd ask that you try to keep the info to yourself for now, though I don't really think there's anything particularly shocking in there.

apiraino (Feb 10 2020 at 23:17, on Zulip):

@nikomatsakis thanks I'll have a look at the doc

apiraino (Feb 16 2020 at 18:46, on Zulip):

@nikomatsakis I managed to have a look at the Dropbox document with the 2019 survey results. There's a lot of info there and (imo) some work to compare those number to those of 2018 and extract a human readable summary. Yeah, I know, "welcome, Mr. Obvious" :-)
What I mean is, if you feel that we are stuck with the data extraction, we can try drafting a document and build it point by point, trying to attack the questions that are not clear, one by one.
On the other hand, if this document (that eventually will become the blog post) is never started, I feel that the process of data extraction is blocked because the task looks daunting.
Maybe all this is already happening, but who's actively working of the survey results?

apiraino (Feb 16 2020 at 18:47, on Zulip):

I do have the goal of getting some kind of blog post out ASAP, but I think another goal would be identifying things that we ought to try and dig into further

Do you (or anyone) have some draft of items to be put into this blog post?

nikomatsakis (Feb 19 2020 at 12:36, on Zulip):

I have created an outline document and I am populating it with interesting results

nikomatsakis (Feb 19 2020 at 13:12, on Zulip):

@Jasun Carr I've been posting a few more questions, as you no doubt saw, but it'd be good if you took a look at my draft blog post outline and maybe prioritize things that appear in there

XAMPPRocky (Feb 19 2020 at 13:20, on Zulip):

@nikomatsakis I don't know if you plan to reuse the graphs from the original documents or not, if not I can spend some time creating some for the blog post if not.

nikomatsakis (Feb 19 2020 at 13:21, on Zulip):

@XAMPPRocky I did not plan to re-use the graphs from the original paper -- I had a feeling we'd be better off making a fresh google spreadsheet with cherry-picked results and custom charts

nikomatsakis (Feb 19 2020 at 13:21, on Zulip):

for one thing, I would want percentages labeled

nikomatsakis (Feb 19 2020 at 13:21, on Zulip):

and perhaps fewer pie charts:)

XAMPPRocky (Feb 19 2020 at 13:22, on Zulip):

Alright, once the outline is a bit more settled I can get started on a google sheet for it.

nikomatsakis (Feb 19 2020 at 13:22, on Zulip):

(actually, I wonder if we could even make that particular spreadsheet world readable, to let people dive in more easily)

nikomatsakis (Feb 19 2020 at 13:22, on Zulip):

i.e., it'd be kind of neat if you could click on a chart and get taken to the underlying data

nikomatsakis (Feb 19 2020 at 13:22, on Zulip):

since we'd only be exposing aggregates that we're publishing anyway it doesn't seem like a privacy violation

nikomatsakis (Feb 19 2020 at 13:23, on Zulip):

Regarding the outline, i'm starting to see a kind of narrative flow

nikomatsakis (Feb 19 2020 at 13:23, on Zulip):

basically, "Why not use Rust" suggests the following major blockers (roughly in order):

nikomatsakis (Feb 19 2020 at 13:24, on Zulip):

adoption is the biggest one, of course, but that's not directly actionable, and presumably the secondary things cited are part of what is blocking adoption

nikomatsakis (Feb 19 2020 at 13:24, on Zulip):

so then I am organizing the results a bit by those themes

nikomatsakis (Feb 19 2020 at 13:25, on Zulip):

also:learning point for next year, dropbox paper, despite being so nice, really doesn't scale to docs of this size :)

nikomatsakis (Feb 19 2020 at 13:25, on Zulip):

I wonder if google docs would do better

XAMPPRocky (Feb 19 2020 at 13:25, on Zulip):

For the point about providing translated documentation I'll reach out @Manish Goregaokar I remember they said they were in process of starting up the localisation WG.

nikomatsakis (Feb 19 2020 at 13:26, on Zulip):

yes, they've been doing things there, not sure of current status (but they do monitor Zulip, so I imagine they'll weigh in)

nikomatsakis (Feb 19 2020 at 13:27, on Zulip):

@Jasun Carr to help in tracking whta to prioritize, I'm going to add "to do" items in the outline that correspond also to "to do" items in the main survey doc. IT'd be good to add the data into the main survey doc though, and we'll bring things over to the outline.

nikomatsakis (Feb 19 2020 at 13:29, on Zulip):

Oh I guess the correlary to what we were saying above is that we would want to also have the raw data for all charts that are added

nikomatsakis (Feb 19 2020 at 13:31, on Zulip):

OK, I'm pretty happy with that outline

nikomatsakis (Feb 19 2020 at 13:31, on Zulip):

I should go do something else now :)

XAMPPRocky (Feb 19 2020 at 13:43, on Zulip):

@nikomatsakis On dropbox paper alternatives: It might be worth trying hackmd, they just added support for rendering visualisations in documents in JSON using Vega, which can use Google Sheets as a data source. https://vega.github.io/vega-lite/

nikomatsakis (Feb 19 2020 at 13:43, on Zulip):

hmm, maybe

nikomatsakis (Feb 19 2020 at 13:43, on Zulip):

my main concern with hackmd has been their comment system is very ungood

nikomatsakis (Feb 19 2020 at 13:44, on Zulip):

to be more specific, it's not obvious to most people how to add comments, replying is sort of awkward, and there's no "Resolve" button to clear away comments en masse (only way I found to remove them is manually deleting them)

nikomatsakis (Feb 19 2020 at 13:44, on Zulip):

in general, the collaboration workflow feels pretty limited

nikomatsakis (Feb 19 2020 at 13:45, on Zulip):

oh and

Manish Goregaokar (Feb 19 2020 at 17:09, on Zulip):

@XAMPPRocky @Jasun Carr So basically the current status of translations stuff is that:

For docs specifically it's mostly blocked on rustdoc. I can write down a fleshed out version of the plan as a localization-wg issue and perhaps contributors could pick it up

Manish Goregaokar (Feb 19 2020 at 20:09, on Zulip):

https://github.com/rust-lang/project-localisation/issues/5

Manish Goregaokar (Feb 19 2020 at 20:09, on Zulip):

filed that issue for the stdlib ocs

Jasun Carr (Feb 19 2020 at 21:18, on Zulip):

nikomatsakis said:

Jasun Carr I've been posting a few more questions, as you no doubt saw, but it'd be good if you took a look at my draft blog post outline and maybe prioritize things that appear in there

Working through that now. Will focus on that - and apologies for the really long comment reply I just left.

Last update: Feb 25 2020 at 04:15UTC