Delay in processing database jobs
Incident Report for Tribal Habits
Resolved
All queued jobs were successfully processed after a delay which occurred from approximately 10:00 am to 10:35 am. We apologise for this delay in processing, but this situation is now resolved.
Posted Aug 28, 2023 - 10:45 AEST
Update
All outstanding jobs are now processed through the queue and all jobs are now being processed in real-time. We will continue to monitor our database jobs for the next 30 mins to be sure.
Posted Aug 28, 2023 - 10:37 AEST
Update
Please note that this delay will cause delays in other jobs such as enrolments, updating users and setting modules to complete. IMPORTANT - No data is lost. Those jobs are just queued and awaiting completion.
Posted Aug 28, 2023 - 10:34 AEST
Monitoring
We have identified a delay in processing database jobs which started at approximately 10:00 am AEST this morning. The cause is a large scheduled job which has inadvertently triggered too many tasks. As a result, it has created a delay in our database processing while those jobs were processed. We have quickly moved to delete those jobs and our database is now processing the outstanding queue. We will update more as things settle back to normal.
Posted Aug 28, 2023 - 10:33 AEST
This incident affected: Back End Database Servers.