Update 3: November 14th incident
- Continuously improving our platform's reliability and scalability is our top priority. In this blog, you can find an overview of our ongoing efforts and what is coming up.
We confirmed the root cause of last week's platform availability issue.
First, we again apologize for last week's platform availability issue and the inconvenience this may have caused our users. We understand that making crypto trading accessible and reliable is fundamental to our service, and, last week, we fell short in meeting expectations.
Understanding the incident
Our 3rd party cloud infrastructure provider confirmed that the root cause of the incident was due the infrastructure problems. In this specific case the incident was exacerbated by the fact that monitoring system entered a never-ending loop while trying to activate a backup solution and not being able to do it successfully. This impacted Bitvavo’s system availability and caused the problem with access to the platform for our customers. The manual intervention of our infrastructure provider’s engineers stopped the loop and helped to bring systems to a healthy state.
We continue to work on increasing the resilience of our systems. A significant upgrade to our architecture, one of our key ongoing projects, will help us to contain impact and ensure the availability of the majority of remaining systems. Last week's maintenance window represented a major milestone in this project. We're committed to improving the reliability of our platform for our customers and will continue with several upgrades over the next weeks and months.
Funds are safe and secure
As stated previously, we want to emphasize that last week’s platform availability issue is not related to asset safety and security. The security of our users’ investments is safeguarded by top-grade custody providers, including Fireblocks and Copper custody for both our cold and hot wallets.
Future-proof tech setup
Continuously improving our platform's reliability and scalability has always been our top priority. Here’s an overview of our ongoing efforts and what is coming up:
We are accelerating our ongoing transition towards a microservices architecture, enhancing system resilience and scalability. This means that the system - over time - will be composed of hundreds of smaller parts (instead of several bigger parts), minimizing the impact of individual component issues on overall system performance.
We are developing a low latency high throughput state-of-the-art order management system. This can handle peak loads, whilst ensuring a continued high quality of service.
Much progress has already been made this year. Some major additional improvements are currently under development to be shipped in mid-January.
Our commitment
Making crypto trading accessible and reliable is fundamental to our service. Our top-notch engineering team, composed of specialists with previous experience at other leading tech companies, is fully committed to delivering on this. We are also actively investing and continuously expanding our team to further expedite ongoing improvements.
We will be back with more updates soon.
Your feedback is valuable to us. We encourage you to share any concerns or suggestions through our customer support via [email protected].