Skip to content

Commit

Permalink
Merge branch 'main' into updatecli_56fa12a335dc39431bab0f5bfe1aecb47f…
Browse files Browse the repository at this point in the history
…b1cde7c6804fe8e8e04efe36058005
  • Loading branch information
MarkEWaite authored Jul 3, 2023
2 parents 049c025 + 6d6748c commit a37226a
Show file tree
Hide file tree
Showing 14 changed files with 252 additions and 2 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/updatecli.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
app_id: ${{ secrets.JENKINS_ADMIN_APP_ID }}
private_key: ${{ secrets.JENKINS_ADMIN_APP_PRIVKEY }}
- name: Setup updatecli
uses: updatecli/updatecli-action@009d365ecbd788de3f0dd996ca2a0c115474a476 # v2
uses: updatecli/updatecli-action@e5e6863d954b7495848bb43d693704934439d678 # v2

- name: Diff
continue-on-error: true
Expand Down
19 changes: 19 additions & 0 deletions content/issues/2023-06-05-ci-jenkins-io-scans-outage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: ci.jenkins.io scans outage
date: 2023-06-05T14:00:00-00:00
resolved: true
resolvedWhen: 2023-06-05T23:59:59-00:00
# Possible severity levels: down, disrupted, notice
severity: disrupted
affected:
- ci.jenkins.io
section: issue
---

A change in the GitHub REST API is responsible for a bug in the integration used by Jenkins to retrieve repository maintainers.
Repository scans on [ci.jenkins.io](https://ci.jenkins.io) fail to retrieve pull requests.

The issue will require a new release of the [GitHub API library](https://github.com/hub4j/github-api/) and the [GitHub API plugin](https://plugins.jenkins.io/github-api/) that packages the library as a Jenkins plugin.
A patched GitHub API has been installed on ci.jenkins.io while waiting for the release of the library and the plugin.

See the [help desk issue](https://github.com/jenkins-infra/helpdesk/issues/3617) to track progress.
19 changes: 19 additions & 0 deletions content/issues/2023-06-06-ldap-outage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: ldap outage
date: 2023-06-06T12:45:00-00:00
resolved: true
resolvedWhen: 2023-06-06T14:50:00-00:00
# Possible severity levels: down, disrupted, notice
severity: down
affected:
- infra.ci.jenkins.io
- login repo.jenkins-ci.org (UI)
- login accounts.jenkins.io
section: issue
---

[Final message]
The service has been restored, a misconfiguration was responsible for its outage.

[Initial message]
ldap service is down so no authentification working on jira or controllers
22 changes: 22 additions & 0 deletions content/issues/2023-06-08-artifactory-operation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
title: Brownout of the 'jgit' mirrored repository in JFrog Artifactory (repo.jenkins-ci.org)
date: 2023-06-08T12:30:00-00:00
resolved: true
resolvedWhen: 2023-06-08T13:00:00-00:00
# Possible severity levels: down, disrupted, notice
severity: disrupted
affected:
- repo.jenkins-ci.org
- ci.jenkins.io
section: issue
---

[Final message]
Our experiment is now finished. Projects using the Eclipse `jgit` had errors downloading artefacts during the operation but it is back to normal

[Initial message]
The mirror repository `jgit` in the JFrog Artifactory service <repo.jenkins-ci.org> will be restricted to only authenticated users the 8 June 2023 at 12:30UTC for 30 min.

This [planned brownout](https://en.wikipedia.org/wiki/Brownout_(software_engineering)) is a step of <https://github.com/jenkins-infra/helpdesk/issues/3599> to study the behavior of our builds.

There should be no impact, as the artifacts should be consumed through the `public` virtual repository but we cannot be 100% sure, hence this operation.
32 changes: 32 additions & 0 deletions content/issues/2023-06-08-ldap-migration-to-publick8s.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
title: ldap.jenkins.io migration
date: 2023-06-08T12:00:00-00:00
resolved: true
resolvedWhen: 2023-06-08T13:05:00-00:00
# Possible severity levels: down, disrupted, notice
severity: disrupted
affected:
- ldap.jenkins.io
- accounts.jenkins.io
section: issue
---

[Final message]
Although the intended redirections from accounts.jenkins.io & accounts.jenkins-ci.org to status.jenkins.io didn't work as expected, the LDAP migration has been successfully completed, no service interruption.

[Initial message]
A migration of the LDAP service (`ldap.jenkins.io` - used for Jenkins user authentication) from `prodpublick8s` cluster to `publick8s` cluster is planned today on Thursday 8th of June 2023 at 12h00 UTC.

In order to avoid any user base modification while migrating this service, [accounts.jenkins.io](https://accounts.jenkins.io) access will be disabled for less than an hour.

Although no downtime is expected, the following services might be impacted:
- JFrog Artifactory (repo.jenkins-ci.org)
- ci.jenkins.io
- infra.ci.jenkins.io
- release.ci.jenkins.io
- trusted.ci.jenkins.io
- cert.ci.jenkins.io
- vpn.jenkins.io
- issues.jenkins.io

More details on https://github.com/jenkins-infra/helpdesk/issues/3351#issuecomment-1576741071
20 changes: 20 additions & 0 deletions content/issues/2023-06-09-ci-outage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: Outage on VM agents on ci.jenkins.io
date: 2023-06-09T12:37:00-00:00
resolved: true
resolvedWhen: 2023-06-09T12:42:00-00:00
# Possible severity levels: down, disrupted, notice
severity: disrupted
affected:
- ci.jenkins.io
section: issue
---

[Final message]
revert disabled plugin and restart ci.jenkins.io

[Initial message]
There is an outage on ci.jenkins.io since 12h35 UTC.
following plugin removal there is a configuration as code error message.

https://github.com/jenkins-infra/helpdesk/issues/3573#issuecomment-1584503523
23 changes: 23 additions & 0 deletions content/issues/2023-06-12-mirrorbits-migration-to-publick8s.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
title: mirrorbits migration
date: 2023-06-12T12:00:00-00:00
resolved: true
resolvedWhen: 2023-06-12T13:30:00-00:00
# Possible severity levels: down, disrupted, notice
severity: disrupted
affected:
- get.jenkins.io
- mirrors.jenkins.io
- mirrors.jenkins-ci.org
- fallback.get.jenkins.io
section: issue
---

[Final message]
Migration completed.

[Initial message]
A migration of the "mirrorbits" service from `prodpublick8s` cluster to `publick8s` cluster is planned today on Monday 12st of June 2023 at 12h00 UTC.
While there should not be any interruption, the corresponding service may be down for less than one hour.

More details on https://github.com/jenkins-infra/helpdesk/issues/3351#issuecomment-1586969797.
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
title: plugin-site-issues migration
date: 2023-06-12T13:30:00-00:00
resolved: true
resolvedWhen: 2023-06-12T14:00:00-00:00
# Possible severity levels: down, disrupted, notice
severity: disrupted
affected:
- plugin-site-issues.jenkins.io
- plugins.jenkins.io
section: issue
---

[Final message]
Migration completed.

[Initial message]
A migration of the "plugin-site-issue" service (`plugin-site-issues.jenkins.io` - used by plugins.jenkins.io to retrieve release and issue information about a plugin) from `prodpublick8s` cluster to `publick8s` cluster is planned today on Monday 12st of June 2023 at 13h00 UTC.
While there should not be any interruption, the corresponding service may be down for less than one hour, and plugins.jenkins.io may be impacted.

More details on https://github.com/jenkins-infra/helpdesk/issues/3351#issuecomment-1586972368.
21 changes: 21 additions & 0 deletions content/issues/2023-06-13-jenkins-io-migration-to-publick8s.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
title: jenkins.io migration
date: 2023-06-13T09:25:00-00:00
resolved: true
resolvedWhen: 2023-06-13T10:10:00-00:00
# Possible severity levels: down, disrupted, notice
severity: disrupted
affected:
- www.jenkins.io
section: issue
---

[Final message]
Migration successfully completed, no service interruption.

[Initial message]
A migration of [Jenkins website](https://www.jenkins.io) from `prodpublick8s` cluster to `publick8s` cluster is planned on Tuesday 13th of June 2023 at 12h00 UTC.

In order to avoid any concurrent updates, jenkins-infra/jenkins.io builds will be disabled for less than an hour.

More details on https://github.com/jenkins-infra/helpdesk/issues/3351#issuecomment-1588040182.
21 changes: 21 additions & 0 deletions content/issues/2023-06-13-plugin-site-migration-to-publick8s.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
title: plugins.jenkins.io migration
date: 2023-06-13T08:00:00-00:00
resolved: true
resolvedWhen: 2023-06-13T09:15:00-00:00
# Possible severity levels: down, disrupted, notice
severity: disrupted
affected:
- plugins.jenkins.io
section: issue
---

[Final message]
Migration successfully completed, no service interruption.

[Initial message]
A migration of [the plugins site website](https://plugins.jenkins.io) from `prodpublick8s` cluster to `publick8s` cluster is planned on Tuesday 13th of June 2023 at 08h00 UTC.

In order to avoid any concurrent updates, jenkins-infra/plugin-site infra builds will be disabled for less than an hour.

More details on https://github.com/jenkins-infra/helpdesk/issues/3351#issuecomment-1588038925.
17 changes: 17 additions & 0 deletions content/issues/2023-06-14-ci-maintenance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
title: Maintenance on ci.jenkins.io
date: 2023-06-14T12:00:00-00:00
resolved: true
resolvedWhen: 2023-06-14T13:00:00-00:00
# Possible severity levels: down, disrupted, notice
severity: down
affected:
- ci.jenkins.io
section: issue
---

[Final message]
The maintenance is ended. Advisory published: <https://www.jenkins.io/security/advisory/2023-06-14/>.

[Initial message]
Maintenance on ci.jenkins.io, down for security advisory. (ref. <https://groups.google.com/g/jenkinsci-advisories/c/CD7La5U-dIQ>).
23 changes: 23 additions & 0 deletions content/issues/2023-06-21-digitalocean-incident.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
title: DigitalOcean outage
date: 2023-06-21T11:45:00-00:00
resolved: true
resolvedWhen: 2023-06-22T16:45:00-00:00
# Possible severity levels: down, disrupted, notice
severity: disrupted
affected:
- ci.jenkins.io
section: issue
---

[Final message]
Incident is reported officially by Digital Ocean and is finished: <https://status.digitalocean.com/incidents/bclmy32d12p0>.

We've restored all the nominal configuration after health-checking the clusters.

[Initial message]
DigitalOcean infrastructure is having a network outage.
We see a high rate of TCP errors on both Kubernetes clusters `doks` (agents for ci.jenkins.io) and `doks-public` (Artifact Caching Proxy for these agents).
We are disabling temporarily agent scheduling in DigitalOcean: all ci.jenkins.io containers workload will run in AWS or Azure instead.

More details in <https://github.com/jenkins-infra/helpdesk/issues/3633>.
12 changes: 12 additions & 0 deletions content/issues/2023-07-06-jira-outage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
title: Jira (issues.jenkins.io) outage
date: 2023-07-06T21:00:00-00:00
resolved: false
resolvedWhen: 2023-07-06T23:00:00-00:00
# Possible severity levels: down, disrupted, notice
severity: notice
affected:
- issues.jenkins.io
section: issue
---
The JIRA instance at issues.jenkins.io, managed by the Linux Foundation team, will be down for up to two hours beginning at 9:00 PM (UTC) Thursday July 6, 2023 so that it can be upgraded.
2 changes: 1 addition & 1 deletion content/monitoring/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ title: "Monitoring"

The project relies on Datadog for monitoring Jenkins infrastructure.
Here is a list of popular dashboard used.
Feel free to request additional ones.
Feel free to request additional ones by opening an issue in https://github.com/jenkins-infra/helpdesk

[%header,cols=3*]
|===
Expand Down

0 comments on commit a37226a

Please sign in to comment.