THIS IS A SCHEDULED EVENT Nov 20, 13:00 - 21:00 UTC Oct 28, 12:38 UTC Scheduled - We will be performing a maintenance to replace a faulty hardware part in our network that will temporarily affect access to files stored in non-replicated storage zones, Magic Container and Database workloads running in Sydney. Expected Impact During the maintenance window, requests for files in non-replicated storage zones may result in temporary 5xx errors. Replicated files will remain fully accessible. For Magic Containers and Bunny Database, single-location workloads in Sydney will experience a period of unavailability. Maintenance Window Date: November 20th Time: 13:00 - 21:00 (UTC +0) Expected Impact Duration: Up to 60 minutes
Nov 5, 16:24 UTC Resolved - This incident is now resolved Nov 5, 16:03 UTC Monitoring - We have resolved the API degradation issue, and are monitoring the services. Nov 5, 15:23 UTC Investigating - We are currently investigating an issue with degraded API response times on our platform. Dashboard services may have delayed response times, as well as some of our API endpoint responses. Active platform services are not affected by degradation.
Nov 4, 13:38 UTC Update - We can confirm that the revocation of all affected certificates is now complete. Let's Encrypt has published every revocation in its Certificate Revocation Lists (CRLs). Every Let's Encrypt certificate that was on the affected server has been revoked with the reason "key compromise". Browsers and clients that check revocation will now reject the old certificates. The replacement certificates now in use are not affected, and customers don't need to do anything. Our provider's investigation is still ongoing. We will continue to follow up and will share the final outcome once it is complete. Oct 31, 12:50 UTC Update - We've completed the reissuance of managed TLS certificates for custom domains. All managed certificates have been reissued with new private keys, and the previous certificates have been revoked. No action is required from customers using managed certificates. If you manage your own TLS certificates and haven't done so yet, we continue to recommend revoking and reissuing them with new private keys as a precaution. - What we're doing to prevent this in the future We have stopped using the affected provider while their investigation continues. We recognise that we can't rely on any single layer of security, so we're taking additional steps to reduce the potential impact of similar incidents that could occur in the future. We're reducing what each server holds: we're changing how configurations are distributed so that each PoP only receives the configuration and TLS certificates for the sites it is actively serving, limiting the data stored on any single server. Neither our investigation nor the provider's has found any evidence of unauthorized access to our server. Should anything change, we will provide an update via email and on our status page. Thank you for your patience while we completed this work. At bunny.net, we understand the trust you place in us to keep your content secure, and we take that responsibility seriously. If y