At 10:45 UTC 12 June we received alerts that some customers' newsletters were delayed.
Our team responded, made adjustments and reviewed our systems. Reporting confirmed that newsletters were sending as normal.
At 16:00 UTC we were again alerted to delays in newsletter processing. Our team re-investigated and found that one of our database clusters was no longer performant. This issue affected our ability to calculate segments and schedule newsletters, leading to delays for the majority of newsletters scheduled between 10:45 UTC and 17:30 UTC.
Our database was affected as a result of regular maintenance processes we run. Unfortunately this issue both affected scheduling and the regular health checks that report on newsletter delays.
Since 18:00 UTC, newsletters have been sending as normal.
Data collectiong, workflows and automated messages were not affected by this incident.
Based on what we have observed from this incident we will make changes to our maintenance processes and health checks.
If you have any questions, please conact us at firstname.lastname@example.org. Thank you.