Skip to content
Tech News
← Back to articles

A Server Lost Power at 00:32. We Found Out at 08:18

read original more articles
Why This Matters

This incident highlights the importance of robust monitoring and rapid response systems in the tech industry, as well as the resilience of cloud infrastructure that prevented data loss despite hardware failure. It underscores the need for effective incident management to minimize service disruption for consumers. The event also emphasizes the significance of transparency and communication during outages to maintain user trust.

Key Takeaways

Resolved. This incident ran from 00:32 to 08:47 UTC on 17 August 2026 and is closed. No customer data was lost, corrupted or exposed. If a deployment of yours failed during that window it will not have retried on its own — trigger it once more and it will go through. Everything below is the full account of what happened and what we changed.

On Sunday 17 August 2026, one of our storage servers lost power at 00:32 UTC. It did not reboot. It did not crash. It simply stopped, and it stayed off until an engineer pressed the power button by hand at 08:22 UTC.

For eight hours and fifteen minutes, S3-compatible object storage, the container registry, serverless deployments and static-site deployments were unavailable. Running applications kept serving. No data was lost.

Our status page detected the failure and opened an incident at 00:35 UTC — three minutes in. It was correct, it was public, and it stayed up for the entire outage.

Nobody saw it. We found out at 08:18 UTC, when an engineer opened the status page for an unrelated reason and asked why so many services were red.

That gap — between a system knowing and a person knowing — is the actual incident. Everything else is detail.

What Our Customers Saw

The failure had one cause and four symptoms, which is why the status page looked worse than a single dead machine should look.

Object storage returned connection refused, not slow responses. S3 was fully unavailable rather than degraded.

returned connection refused, not slow responses. S3 was fully unavailable rather than degraded. The container registry went down with it, because registry blobs are stored in object storage.

... continue reading