Weekend Site Outages

[peter]

We had two site outages over the weekend.

The first was due to the PHP eAccelerator crashing on Friday morning after a php version upgrade after running yum. I've learned my lesson, and plan for it better next time.

The second, most of the day Saturday, was due to a failed disk drive. I now have a backup server online to take over. (The server on which I archive the DB was fine, but unsuitable to web hosting).

I'm investigating a system with HW RAID.

On a related topic, I'm looking into methods of backing up the DB with a call to suspend the forums and then reactivate them after the backup is complete. At the moment I just stop apache and run the mysql backup.

Any suggestions?

[Schotty]

Setup a SQL cluster of a few machines and have the rendundancy act as your first line of defense and have it structured something like this

Server 1: Apache, Mysql
Server 2: Mysql
Server 3: Mysql, somebackup to another media or offsite

This gives you a machine that can be dedicated to the main job, one for backingup and another that has just the db stuff just in case.

I am sure there is a brighter solution, but not being a DBA I am a little slow on this kind of stuff, plus not adminning in a few years, I am sure the tech got better since my foray into IT.

HTH.

[peter]

Thanks, I'm going to think about this on the weekend.

[Crouse]

On a related topic, I'm looking into methods of backing up the DB with a call to suspend the forums and then reactivate them after the backup is complete. At the moment I just stop apache and run the mysql backup.

Any suggestions?

You could use a cron job and call this script.... I call this script 4 times a day for my forums and then use rsync to move copies of the database backups to several servers several times a day as well. The "--opt " locks the tables while backing up. So I never turn off apache at all.

-------------------------------------------------------------------------

#!/bin/sh 
for i in `mysql -B -e "show databases" | tail +2`; do
mysqldump --opt $i | gzip -9 > /var/backups/mysql/$i.`date --iso-8601`.gz
ls -t /var/backups/mysql/$i.* | tail +7 | xargs rm -f
done