Depot - Notice history

The authentication failure was caused by an expired OIDC signing certificate that has been renewed. We have deployed a fix to prevent this in the future. All builds are operational.

We needed extra capacity and are now monitoring.

Container build service is restored.

GitHub Actions jobs are processing normally. We are continuing to monitor recovery of container builds in us-east-1.

We are seeing service begin to restore for both container builds and GitHub Actions jobs as capacity rebalances. We're continuing to monitor.

This incident has been resolved and queue times for -latest labels are in the normal ranges again.

We have identified the problem and implemented a fix for those runner labels. We are monitoring queue times to verify they are in the expected range.

We have identified additional latency in starting depot-ubuntu-latest and depot-ubuntu-24.04 GitHub Actions runners. We are investigating whether the same latency exists in container builds.

We have resolved the latency issue that was impacting jobs and container builds from starting. We will continue to monitor the system.

We are continuing to monitor build service. We are aware of container builds being impacted as well and have rolled out a fix for those as well.

Build service is restored, we are currently monitoring the result.

We are currently investigating this incident.

The fix is fully deployed across the container builder regions, and we're observing full recovery. We will continue to monitor.

We have deployed a fix and are monitoring the rollout across the container builder regions.

We've identified a potential fix for amd64 builds and are working on deploying it.

We've isolated the problem to Intel container builds (i.e. linux/amd64) in both regions. We are working to identify the root cause.