Degraded performance API requests
Incident Report for Betty Blocks
Resolved
We have been notified that the outage that affected the Microsoft servers has been resolved. We have received the root cause analysis and we will share a post mortem of this incident with the customers who have put in a request. If you would like to receive a post mortem of this outage, please contact us via support@bettyblocks.com. You will receive the post mortem report within 30 business days.

Betty Blocks will look at the possibilities within our power on how to mitigate similar situations in the future.

Our sincerest apologies for the inconvenience this outage has caused.
Posted Mar 28, 2023 - 15:51 CEST
Monitoring
All our services are operational since the implementation of the temporary solution by Microsoft. We will continue to actively monitor our traffic. We are still awaiting a root cause analysis from Microsoft.
Posted Mar 23, 2023 - 17:09 CET
Update
As of this moment we have not experienced connection errors since the implementation of the temporary solution by Microsoft. We are noticing that traffic is slower than before the connection issues, this is caused by the temporary solution. At the moment we are awaiting Root cause analysis from Microsoft. We will keep actively monitoring our traffic.
Posted Mar 21, 2023 - 10:45 CET
Update
At the moment we see a significant improvement with regard to connectivity issues. Microsoft is continuing investigating the root cause of the outage. We are actively monitoring our traffic and are awaiting the ongoing investigation from Microsoft.
Posted Mar 20, 2023 - 10:43 CET
Update
Microsoft have isolated the impacted devices and rerouted traffic to help bring the services back to a healthy state. This is not a 100% fix, but a temporarily solution to keep as much healthy as possible while they work on the underlying issues and roll out a fix for the outage.
Posted Mar 18, 2023 - 19:46 CET
Update
We are still experiencing connection issues, we see a slight improvement. We are currently monitoring our traffic and are awaiting the ongoing investigation from Microsoft.
Posted Mar 18, 2023 - 14:13 CET
Update
Unfortunately the suggested options from Microsoft have not resolved the connection issues. Microsoft has acknowledged issues throughout the western Europe region. They are currently continuing conducting further investigation. We will provide an update about the situation when an update from Microsoft is given. Our sincerest apologies for the connection issues we are currently experiencing on our platform
Posted Mar 17, 2023 - 21:02 CET
Update
Currently, we are implementing the suggested options from Microsoft within a test cluster to see if this will solve the issues we are encountering on the platform. If this is successful, this will also be implemented on our Production cluster.
Posted Mar 17, 2023 - 13:03 CET
Update
We have been provided with options by Microsoft, that in theory should resolve the symptoms of the problem. Together with Microsoft we are still investigating what the root cause is. We hope to inform you tomorrow with steps we can do to alleviate the symptoms.
Posted Mar 16, 2023 - 17:12 CET
Update
We wanted to provide an update on the ongoing issue with HTTP API requests that some of you have been experiencing for some time now. We understand that this has been frustrating for you, and we apologize for any inconvenience this may have caused.

Our engineers have been working diligently to investigate the root cause of the issue and find a solution. We're currently collaborating with our third-party hosting provider, Microsoft Azure, to address the issue.

Please know that we're taking this issue seriously, and we're doing everything we can to resolve it as soon as possible. We're committed to providing you with the best possible service, and we apologize for any frustration this may have caused.

If you have any questions or concerns, please don't hesitate to reach out to us. We value your feedback and appreciate your patience as we work to address this issue.
Posted Mar 14, 2023 - 00:42 CET
Update
We are still experiencing issues with HTTP API requests to the Microsoft Azure server. We are currently investigating this issue together with the Microsoft team.

We will provide an update about the situation around 17:00 CET
Posted Mar 13, 2023 - 14:14 CET
Update
We are still experiencing issues with HTTP API requests to the Microsoft Azure server. We are currently investigating this issue together with the Microsoft team.

We will provide an update about the situation around 14:00 CET
Posted Mar 13, 2023 - 10:26 CET
Update
We are still experiencing issues with HTTP API requests to the Microsoft Azure server. We are currently investigating this issue together with the Microsoft team. We will keep on working on the issues during the weekend.
We will provide new updates as soon as they are available.
Posted Mar 10, 2023 - 16:58 CET
Update
We are still experiencing issues with HTTP API requests. You may encounter the following errors when performing HTTP API requests:

- SSL_connect returned=1 errno=0 state=error: bad signature
- SSL_connect returned=1 errno=0 state=error: certificate verify failed
- SSL_connect returned=1 errno=0 state=error: tlsv1 alert decrypt error
- SSL_connect returned=1 errno=0 state=error: bad ecpoint

We will provide an update about the situation around 16:00 CET
Posted Mar 10, 2023 - 13:09 CET
Update
We are still experiencing issues with HTTP API requests. You may encounter the following errors when performing HTTP API requests:

- SSL_connect returned=1 errno=0 state=error: bad signature
- SSL_connect returned=1 errno=0 state=error: certificate verify failed
- SSL_connect returned=1 errno=0 state=error: tlsv1 alert decrypt error
- SSL_connect returned=1 errno=0 state=error: bad ecpoint

We will provide an update about the situation around 12:00 CET
Posted Mar 10, 2023 - 09:10 CET
Update
Issues Background actions and ActionJS on public zone NL4 are resolved.

We are still experiencing issues with HTTP API requests. You may encounter the following errors when performing HTTP API requests:

- SSL_connect returned=1 errno=0 state=error: bad signature
- SSL_connect returned=1 errno=0 state=error: certificate verify failed
- SSL_connect returned=1 errno=0 state=error: tlsv1 alert decrypt error
- SSL_connect returned=1 errno=0 state=error: bad ecpoint

We will provide an update about the situation around 09:00 CET
Posted Mar 09, 2023 - 19:46 CET
Update
We are still experiencing issues with HTTP API requests. The impact is major outage on public zone NL4. We will provide an update about the situation around 21:00 CET.
Posted Mar 09, 2023 - 16:35 CET
Update
We are still experiencing issues with HTTP API requests. The impact is major outage on public zone NL4. We will provide an update about the situation around 16:30 CET.
Posted Mar 09, 2023 - 15:35 CET
Update
We are still experiencing issues with HTTP API requests. The impact is major outage on public zone NL4. We will provide an update about the situation around 15:30 CET.
Posted Mar 09, 2023 - 14:34 CET
Update
We are still experiencing issues HTTP API requests. The impact has been increased to a major outage on public zone NL4. We will provide an update about the situation around 14:30 CET
Posted Mar 09, 2023 - 13:27 CET
Investigating
You may encounter the following errors when performing HTTP API requests:

- SSL_connect returned=1 errno=0 state=error: bad signature
- SSL_connect returned=1 errno=0 state=error: certificate verify failed
- SSL_connect returned=1 errno=0 state=error: tlsv1 alert decrypt error
- SSL_connect returned=1 errno=0 state=error: bad ecpoint

This issue has been occurring since Saturday the 4th of March. Our Product team has been investigating since the first report on Monday the 6th of March. We are in contact with Microsoft Azure Kubernetes services to understand and resolve the issue. We will provide an update about the situation around 14:00 CEST.
Posted Mar 09, 2023 - 11:12 CET
This incident affected: Betty Blocks private cloud zone 10 (Classic Background Actions), Betty Blocks private cloud zone 11 (Classic Background Actions), Betty Blocks zone NL1 (Classic Background Actions), Betty Blocks zone NL3 (Classic Background Actions), Betty Blocks zone CA3 (Classic Background Actions), Betty Blocks private cloud zone 1 (Classic Background Actions), Betty Blocks private cloud zone 3 (Classic Background Actions), Betty Blocks private cloud zone 4 (Classic Background Actions), Betty Blocks private cloud zone 6 (Classic Background Actions), Betty Blocks private cloud zone 7 (Classic Background Actions), Betty Blocks private cloud zone 8 (Classic Background Actions), Betty Blocks zone US2 (Classic Background Actions), Betty Blocks zone NL4 (Classic Background Actions, DataAPI, ActionsAPI), Betty Blocks private cloud zone 9 (Classic Background Actions), and Classic Background Actions, Classic Background Actions.