20/05/2011 07:24
 Venus Outage 20/05/2011 - 05:11AM to 06:58AM
As you may have noticed your web hosting service on Venus went down between 05:11AM and 06:58AM. Unfortunatley this did go un-detected by our monitoring and it was tickets from client's that alerted us to the problem. The reason for this is that our monitoring check's the accessibility of the port over the internet rather than the data being received from that port. We are now looking to extend our monitoring checks to add data checks to retrieve data from FTP, SMTP, HTTP and MySQL etc to better monitor the servers and hopefully catch problems like today.
The actual problem with the server was down to the CPU load going extremley high that meant we were unable to even remotley logon to the server. At this time we are un-sure exactly what caused this as we had to hard reboot the server in order to get access to it however we will be looking for traces of what caused it and also setting up some CPU load monitoring checks. Due to the hard reboot we then found that a number of MySQL databases had gone into the crashed state as they were in use at the time of the reboot. We quickly worked to find which databases these were and run a repair on the data to correct them. During this process however the repairs then filled up the /tmp partition meaning the repairs did not fully complete. To combat this we have extended the /tmp partition by almost 10 times to prevent any future problems with database repairs. After this we ran the repairs again and believe we have now corrected all databases that were marked as crashed and everything should be back to normal.
We appologise for any inconvenience caused and we can ensure you that we will be implementing new checks to both catch and more importantly prevent outages like these from happening. If you still have any websites or databases that are not functional please contact support by creating a ticket at https://secure.vooclients.com or by e-mailing support@vooservers.com and we will investigate the issue straight away.
Best Regards,
Vooservers Support.
<< Back
 View RSS Feed
Powered by WHMCompleteSolution |
