infra-statistics has no new data since July 1st 2024

[krisstern]

Service(s)

Helpdesk, stats.jenkins.io

Summary

The branch at https://github.com/jenkins-infra/infra-statistics/tree/gh-pages has not been fetching new data for 3 months now it appears.

Reproduction steps

No response

[krisstern]

Thanks @shlomomdahan for reporting this earlier today ET!

c.c. @gounthar, @Vandit1604

[dduportal]

Ping @kohsuke : it looks like we will need you to create VPN access so your machine can send the statistics with succes

[dduportal]

Ping @kohsuke : it looks like we will need you to create VPN access so your machine can send the statistics with succes

Thanks @timja @kohsuke for #4299

=> first milestone achieved to solve the infra stats update issue!

[dduportal]

Hello @abayer @daniel-beck @olblak , it looks like the pipeline block at https://github.com/jenkins-infra/infra-statistics/blob/8209b583ee5be409d2dc8cdf780bd4358579b8de/Jenkinsfile#L88-L102 is expected to be run on trusted.ci.jenkins.io.

I cannot be sure if it ever existed of if it has been removed (if removed, then most probably by me in the past 2 years, mistakenly).

I'm failing to understand what "process" is filling the branch gh-pages of https://github.com/jenkins-infra/infra-statistics once a month.

Could you help me understand better the components at stake here?

[daniel-beck]

I think heidelberg is @abayer's, I poked him about it out of band.

[daniel-beck]

I cannot be sure if it ever existed of if it has been removed (if removed, then most probably by me in the past 2 years, mistakenly).

I remember an (always offline?) agent called census on trusted.ci (https://github.com/jenkins-infra/infra-statistics/blob/8209b583ee5be409d2dc8cdf780bd4358579b8de/Jenkinsfile#L20-L23), but that was cleaned up a while ago, possibly around the time the agents were standardized?

[dduportal]

I cannot be sure if it ever existed of if it has been removed (if removed, then most probably by me in the past 2 years, mistakenly).

I remember an (always offline?) agent called census on trusted.ci (https://github.com/jenkins-infra/infra-statistics/blob/8209b583ee5be409d2dc8cdf780bd4358579b8de/Jenkinsfile#L20-L23), but that was cleaned up a while ago, possibly around the time the agents were standardized?

🤔 this label still exists on trusted.ci.jenkins.io (ref. https://github.com/jenkins-infra/jenkins-infra/blob/a0db24773372ec6b1fdc0ee0c7186a9ef12ceaab/hieradata/clients/controller.trusted.ci.jenkins.io.yaml#L68-L75) but not anymore on ci.jenkins.io (ref. https://github.com/jenkins-infra/jenkins-infra/blob/production/hieradata/clients/controller.sponsorship.ci.jenkins.io.yaml).

[dduportal]

Update: I'm in contact with @kohsuke and @abayer to finalize the statistics update. There were unexpected technical issues which should be solved soon.

I plan to work with @abayer end of year, to pass knowledge and tools under the Jenkins infra for the KK -> census -> stats processes of the anonimyzed data.

[dduportal]

Deferring to mid- November (unless we have news)

[dduportal]

Quoting @MarkEWaite in https://matrix.to/#/!JLUOInpEYmxJIYXlzs:matrix.org/$AooefCZO2zDfeDBkUaKY0IfG5QeuatofTmIIqrcpboA?via=g4v.dev&via=gitter.im&via=matrix.org

Thanks very much to Andrew Bayer and Kohsuke Kawaguchi for the recent updates to plugin statistics. Installation data for JVM's is now showing Java installation measures through 1 Oct 2024. Those numbers look very good. 115k Java 17 installations, 27k Java 21 installations, with 82k Java 11 and 50k Java 8. So great to have the plugin installation count data current as well.

Many thanks @kohsuke @abayer for unblocking this topic!

Data available in jenkins-infra/infra-statistics@33620e3

[krisstern]

Can confirm that new data is being fetched on https://stats.jenkins.io/. Hooray!

[abayer]

I should note that the October data is a little wonky at the moment, because the anonymized data used to generate the reports only was through October 29, but I decided that since the October data will get regenerated when we run the report for November, and it's only two days, we'll be fine for now.

[krisstern]

Thanks @abayer for the follow-up!