GitHub Community
[2025-02-25] Incident Thread #152308
-
|
❗ An incident has been declared:Incident with Packages Subscribe to this Discussion for updates on this incident. Please upvote or emoji react instead of commenting +1 on the Discussion to avoid overwhelming the thread. Any account guidance specific to this incident will be shared in thread and on the Incident Status Page. |
Beta Was this translation helpful? Give feedback.
Replies: 4 comments
-
UpdateWe have identified the issue impacting packages and have rolled out a fix. We are seeing signs of recovery and continue to monitor the situation. |
Beta Was this translation helpful? Give feedback.
-
UpdateWe have confirmed recovery for the majority of our systems. Some systems may still experience higher than normal latency as they catch up. |
Beta Was this translation helpful? Give feedback.
-
Incident ResolvedThis incident has been resolved. |
Beta Was this translation helpful? Give feedback.
-
Incident SummaryOn February 25, 2025, between 00:17 UTC and 01:08 UTC, GitHub Packages experienced a service degradation, leading to failures uploading and downloading packages, along with increased latency for all requests to GitHub Packages registry. At peak impact, about 14% of uploads and downloads failed, and all Packages requests were delayed by an average of 7 seconds. The incident was caused by the rollout of a database configuration change that resulted in a degradation in database performance. We mitigated the incident by rolling back the contributing change and failing over the database. In response to this incident, we are tuning database configurations and resolving a source of deadlocks. We are also redistributing certain workloads to read replicas to reduce latency and enhance overall database performance. |
Beta Was this translation helpful? Give feedback.
Incident Summary
On February 25, 2025, between 00:17 UTC and 01:08 UTC, GitHub Packages experienced a service degradation, leading to failures uploading and downloading packages, along with increased latency for all requests to GitHub Packages registry. At peak impact, about 14% of uploads and downloads failed, and all Packages requests were delayed by an average of 7 seconds. The incident was caused by the rollout of a database configuration change that resulted in a degradation in database performance. We mitigated the incident by rolling back the contributing change and failing over the database. In response to this incident, we are tuning database configurations and resolving a source…