Cloudflare has confirmed that the massive service outage yesterday was not caused by a security incident and no data has been lost.
The issue has been largely mitigated. It started 17:52 UTC yesterday when the Workers KV (Key-Value) system went completely offline, causing widespread service losses across multiple edge computing and AI services.
Workers KV is a globally distributed, consistent key-value store used by Cloudflare Workers, the company’s serverless computing platform. It is a fundamental piece in many Cloudflare services and a failure can cause cascading issues across many components.
The disruption also impacted other services used by millions, most notably the Google Cloud Platform.
Workers KV error rate during the incident
Source: Cloudflare
In a post mortem, Cloudflare explains that the outage lasted almost 2.5 hours and the root cause was a failure in the Workers KV underlying storage infrastructure due to a third-party cloud provider outage.
“The cause of this outage was due to a failure in the underlying storage infrastructure used by our Workers KV service, which is a critical dependency for many Cloudflare products and relied upon for configuration, authentication, and asset delivery across the affected services,” Cloudflare says.
“Part of this infrastructure is backed by a third-party cloud provider, which experienced an outage today and directly impacted the availability of our KV service.”
Cloudflare has determined the impact of the incident on each service:
... continue reading