All Systems Operational
API   ? Operational
90 days ago
99.99 % uptime
Today
Email Processing Operational
90 days ago
99.99 % uptime
Today
Transactional emails   Operational
90 days ago
99.99 % uptime
Today
Behavioral emails   Operational
90 days ago
99.99 % uptime
Today
Newsletter emails   Operational
90 days ago
99.99 % uptime
Today
Web Interface Operational
Email reports and statistics   ? Operational
Activity logs   ? Operational
Backend Processing Operational
Imports and Exports   ? Operational
Segment calcluation   Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
System Metrics Month Week Day
API Response Times
Fetching
Past Incidents
Sep 24, 2017

No incidents reported today.

Sep 23, 2017

No incidents reported.

Sep 22, 2017

No incidents reported.

Sep 21, 2017

No incidents reported.

Sep 20, 2017

No incidents reported.

Sep 19, 2017

No incidents reported.

Sep 18, 2017
Completed - All work related to this scheduled maintenance has now been completed.
Sep 18, 00:13 AEST
Verifying - Our maintenance tasks and capacity upgrades have successfully completed, we are running some further verification tests over the next day.
Sep 17, 03:12 AEST
In progress - Our maintenance work is starting now, we will post updates periodically during this time.

If you have any questions or concerns, please email us at support@getvero.com.
Sep 16, 10:02 AEST
Scheduled - We will be performing a series of important maintenance tasks and capacity upgrades on our databases (commencing from Friday 8pm ET) across the weekend.

The aim of this maintenance is to improve our systems performance and will also help prevent any recurrence of an incident earlier this week that resulted in delays in segment calculation.

This maintenance work will not delay any email sends or impact system performance, however there may be some intermittent delays for events or user properties tracked during the maintenance period being made available for inclusion in segment calculations.

If you have any questions or concerns, please email us at support@getvero.com.
Sep 15, 17:48 AEST
Sep 15, 2017
Resolved - Hi all, we're going to resolve this incident at this time.

For the last 10 hours we have observed our segment queries operating at their regular speeds. The changes we made worked effectively and rapidly.

We will continue to monitor proactively for the coming 24 hours.

This issue was hard to detect and, having found the cause, we have learnt from this case. In response to this incident we are setting up further monitoring to help alert us to another incident of this nature ahead of time.

Again, we apologise for the inconvenience causes. Have a great day.
Sep 15, 06:56 AEST
Identified - We have been investigating this issue over the past several hours.

We have not provided updates as we have only recently had concrete leads in this investigation.

Firstly, to clarify, this issue has not affected **every** segment. This has made our investigation difficult, as our core metrics suggest that things are operating effectively. After much work, we've isolated that the segments affected are any segments that contain the "user has property" condition.

Over the last 72 hours, the calculation time on these segments has been steadily increasing and has now reached an average calculation time of approximately four times the normal average. There are also significant outliers leading to long delays for some customers.

We believe we've found the root cause and are in the process of correcting this. We estimate it may take 12+ hours for these segments to be back to their normal calculation times.

In the interim, if you are sending **large** newsletters that use segments with these conditions we recommend that you:

• Refresh the segments manually yourself prior to sending.
• Email us at support@getvero.com and let us know which campaign this is / these are.
• We will update our system so that the send goes out on time, as the segments have been pre-calculated.

We will provide another update once we start seeing good improvements in calculation speed. This may take several hours. We will update earlier if we have more information of course. We're going to leave this incident open until full resolution.

Thank you.
Sep 14, 14:00 AEST
Investigating - We are currently experiencing delays in calculating some segments. We're investigating the root cause for this at this time.

Segments are still calculating, but are taking significantly longer to do so. For some customers, this will cause delays in newsletter sends as newsletters force recalculations on segments.

We will keep you updated here. We have not previously seen such an acute occurrence of this issue and have added a new component above to reflect Vero's current status. Thanks.
Sep 14, 07:24 AEST
Sep 13, 2017

No incidents reported.

Sep 12, 2017

No incidents reported.

Sep 11, 2017

No incidents reported.

Sep 10, 2017

No incidents reported.