Building a sustainable and healthy collaborator base

[marco-ippolito]

I think we should create a survey for collaborators to see what are the pain points of being a Node.js collaborator and how to improve the environment.
It would also be interesting to gather some data on github to see:

• What is the average "lifespan" of a collaborators on the project
• How many collaborators are involved with working groups
• How many users keep contributing after their first commit (TBD)

Feel free to add your ideas and discuss

[UlisesGascon]

As discussed in #277, I wanted to invest some time in generating reports based on the GitHub activity for the project using Cauldron.

You can access commit information and view graphs for the following metrics here:

• Active Authors
• Author Onboarding/Last Active
• Author Retention Ratio
• Author Aging

You can get similar information for PR submissions here.

In this section, you can explore CHAOSS-related metrics (context), such as:

• Drive-through and Repeat Contributor Counts

You can also explore this data directly using Kibana with this link.

Note: I am trying to group all the repositories from the org into a single report, but I am encountering some errors (likely due to the amount of data). However, it is working fine for all the repos in multiple orgs that we have in Express (example). If you want to combine data from multiple Node.js repos, I can do so if you provide a list.

Side note (if you want to build reports): To generate these reports, I use a secondary GitHub account that does not have any access to private repositories. This is to avoid leaking sensitive information in the reports, as this tool requires GitHub tokens to pull data.

[marco-ippolito]

amazing job ❤️ I guess we can analyze the outcome in the next meeting

[mhdawson]

Questions from the brainstorm today:

• Can start by choosing the questions we want to ask
  • How long have you been contributing to the project?
  • Which working groups or teams do you participate in?
  • What makes you want to continue contributing to the project?
  • What makes you likely to choose to contribute less likely to continue
    contributing to the project?
    • Code reviews are to strict
    • Code reviews are too slow
    • CI is flaky
    • My work is not valued
  • Are you able to contribute as much as you would like?
  • If you are not able to contribute as much as you would like, if not why?
    • May company actively interferers with Open Source contributions
    • My company does not support my contributions

[marco-ippolito]

As follow up we need to setup a dedicated meeting to discuss the survey question

[marco-ippolito]

https://doodle.com/meeting/participate/id/ejLYJQlb
doodle for next week

[voxpelli]

I think we should create a survey for collaborators to see what are the pain points of being a Node.js collaborator and how to improve the environment.

To build a sustainable and healthy collaborator base – are you also looking into what may cause people to not contribute as well as what made people to take the leap to contribute?

Lowering the threshold for new contributors can be as key to sustainability and health as retaining current ones can be, and when improving things one should be careful that an improvement for one of the two doesn't come at the expense of another.

---

Also: Have you defined what a collaborator is in this context? Is it a committer? Someone contributing a patch? People helping out in issues, chat and/or documentation? In what projects? Node.js core or eg. undici and other auxiliary projects as well?

[marco-ippolito]

For the definition of collaborator see https://github.com/nodejs/node/blob/main/GOVERNANCE.md#collaborators.

The scope imho is more into improving the retention of existing collaborators rather than making new ones as the latter has always been a complex topic

[voxpelli]

Node.js core collaborators maintain the nodejs/node GitHub repository. The GitHub team for Node.js core collaborators is @nodejs/collaborators. Collaborators have:

• Commit access to the nodejs/node repository
• Access to the Node.js continuous integration (CI) jobs

The scope imho is more into improving the retention of existing collaborators rather than making new ones as the latter has always been a complex topic

@marco-ippolito I think that follow up survey's / investigations into non-collaborator contributors and into the journey from contributor to collaborator needs to be done as well then – especially if considering that next-10 is to set the strategic directions for next 10 years of Node.js, and something the first 10 years have shown is that the collaborators will not remain the same in 10 years time as of now, so focusing too much on the preferences needs of the current collaborators at the possible expense of attracting new collaborators will be supoptimal.

Either that or clarify in this issue why its of special interest to survey collaborators and not others.

[marco-ippolito]

@voxpelli retaining collaborators is as important as making new one. I dont understand what you mean by "the preferences of current collaborators", and at "the expenses of new collaborators". We had the deep dive on how to attract new collaborators recently (start of 2024 if I remember correctly).
If you would like to champion for a next deep dive, feel free to open a new issue.

[voxpelli]

retaining collaborators is as important as making new one

Exactly, that's why I was interested in the reasoning to limit this deep dive to existing collaborators :)

We had the deep dive on how to attract new collaborators recently (start of 2024 if I remember correctly)

That's good to know in this context, since eg. that wasn't mentioned I couldn't know, it kind of answers my question / concerns.

Could be great to link the two deep dives together then and check if some of the conclusions of that previous deep dive would be of interest to bring up in this new deep dive, to find possible common pain points but also possible needs that may be at odds to one another.

If you would like to champion for a next deep dive, feel free to open a new issue

I don't, but as a contributor that's not a collaborator I want to contribute with my perspective when I run across something like this.

---

I'll leave it at this, to avoid derailing the issue further, I'm in the OpenJSF Slack if someone wants to chat more on these things.

[marco-ippolito]

@nodejs/next-10 time set for Aug 29, 2024, 5:30 PM - 6:30 PM Europe/Rome (GMT+02:00)

[UlisesGascon]

Do we have a meeting link, @marco-ippolito ?

[mhdawson]

@UlisesGascon I just started the next-10 zoom, is there a different one ?

[mhdawson]

@marco-ippolito, @UlisesGascon and I spent some time working on the questions and this is what we came up with so far:

A bit about you

• How long have you been contributing to the project?
• Are you paid to contribute to Node.js
• Which working groups or teams do you participate in?
  • (add the list of the teams/working groups here)

How did you get involved

• How did you first get involved ?
  • Attended Node.js Conference
  • Attended collaborator summit
  • Code and learn
  • Joined working group
  • Issues/PR(s) to address problems I was having
  • PR(s) to add new features I wanted
  • PR(s)/Issue(s) because I wanted to get involved in the project
  • PR(s) to improve code coverage
  • PR(s) to improve tests
  • PR(s) to fix flaky tests
  • Pulled in by other collaborators
  • Other (specify)
• Why did you get involved ?
  • Hobby, wanted to contributed to a key open source project
  • Using Node.js personally wanted to contribute
  • Work for a company that uses Node.js, encouraged to contribute
  • Joined a team at the company I work for that contributes
  • Other (specify)
• What would have made it easier to get involved
  • More documentation
  • Easier ways to contact existing collaborators
  • Easier ways to get guidance/direction from existing collaborators
  • Faster build time
  • Mentoring
  • Better identification of work that needs to be done across the project
  • Other (specify)

What keeps you around?

• What makes you want to continue contributing to the project? (multi-select question)
  • Working Groups
  • Features development
  • Bug fixes
  • Other (specify)
• What makes you less likely to continue contributing to the project?
  • Code reviews are to strict
  • Code reviews are too slow
  • Code reviews are too lax
  • CI is flaky
  • CI takes a long time
  • Builds take a long time
  • My work is not valued
  • Required process to get a PR landed
  • PRs take too long to land
  • Long discussions on PRs
  • Other (specify)

Level of contribution

• How many hours do you dedicate to the project in an average month?
  • Free text.
• How many issues would you participate in each month?
• How many PRs would you submit each month ?
• How many meetings would you join each month ?

• Are you able to contribute as much as you would like?
  • Yes
  • Almost
  • A bit
  • No
• If you are not able to contribute as much as you would like, if not why?
  • May company actively interferers with Open Source contributions
  • My company does not support my contributions
• What are your major pain points in contributing ?

Plans going forward

• Are you planning to keep contributing this year?
  • Yes
  • No (specify)
• Are you planning to keep contributing for the next 5 years?
  • Yes
  • No (specify)

[mhdawson]

@marco-ippolito we might want to schedule another time to finish up the list unless you think what we came up with is pretty close to what we need.

[marco-ippolito]

Sorry everyone I have been in hospital due to a motorcycle accident. Ill try to catch up tomorrow

[marco-ippolito]

left few comments on the doc, overall LGTM

[mhdawson]

@marco-ippolito if we think the list is pretty much ready, we should plan how how/when we should send it out. I think we discussed just using survey monkey without asking for help from the Foundation since its internal. Do you have time to look into that?

[marco-ippolito]

@marco-ippolito if we think the list is pretty much ready, we should plan how how/when we should send it out. I think we discussed just using survey monkey without asking for help from the Foundation since its internal. Do you have time to look into that?

We should probably PR the question list first, but yeah I can look into it

So surveymonkey is free, I can create the survey with my account, we can create a discussion on the nodejs/collaborators repo. We can keep it open for 3 weeks.
Starting the 9th and finish 30th of September.

[marco-ippolito]

Survey: https://github.com/nodejs/collaborators/discussions/203

[bnb]

Is there any effort to combat survivorship bias as y'all look at this?

[marco-ippolito]

Is there any effort to combat survivorship bias as y'all look at this?

Can you elaborate?

[mhdawson]

I think @bnb means that we are only polling contributors who have "survived" and are still currently contributors. We won't get feedback from those who have already moved on.

I understand the concern, but I think starting with current collaborators makes sense as this first step.

I could see us tweaking the survey a bit and sending out an updated version to collaborators who have left the project over the last few years to capture additional info/insight.