Task queues are not processed in some tenancies.

Incident Report for Xen Education

Postmortem

What was the incident’s duration?

The incident lasted 0.5 hours from discovery to remediation.

Were there any consequences or damages?

No customers were impacted.

How was the incident addressed?

The instance was re-started.

What will prevent further similar incidents?

Implemented strict monitoring for the impacted services, alerting the DevOps & Development teams immediately, with automatic re-start of the impacted service,

In addition, before triggering any background jobs, a verification is done to test if the service is running, and pause the process until the service has re-started.

Posted Jun 25, 2024 - 17:47 AEST

Resolved

An issue with the task queues services prevented background actions to be processed, impacting all background jobs running on 1 instance ( enrolments, un-enrolments, registrations, password resets, drag & drop, exports/imports and email notifications).
Posted Jun 09, 2022 - 09:00 AEST