Between 7:40am UTC 7 June and 1:45pm UTC today we experienced periodic delays in sending transactional and behavioral emails.
All API requests were captured during this period, however transactional and behavioural emails triggered by these API requests were delayed during this time. Delays ranged in frequency and lengthy but, critically, emails were not processed at their regular speeds.
We want to confirm that, during this period:
• All API requests were collected successfully. There was no API downtime.
• Newsletter emails were processed as usual, without any downtime.
Any delay in email processing is a serious outage for our team. This delay was the result of an issue with the workers that process our email queues. We have isolated the root cause of the delay and are currently investigating ways we can mitigate this automatically in the future, however we do not believe this issue should be recurring in the future and are comfortable things are operating normally.
We apologise for the inconvenience. If you have questions, please let us know. We will continue to monitor this for the next few days to ensure all is in order.
Thanks and have a superb Wednesday!
Posted Jun 07, 2017 - 23:08 AEST
Identified
The issue has been identified and a fix is being implemented.