Results 1 to 4 of 4

Thread: Weekend Site Outages

  1. #1
    Administrator Advisor peter's Avatar
    Join Date
    Apr 2004
    Posts
    882

    Weekend Site Outages

    We had two site outages over the weekend.

    The first was due to the PHP eAccelerator crashing on Friday morning after a php version upgrade after running yum. I've learned my lesson, and plan for it better next time.

    The second, most of the day Saturday, was due to a failed disk drive. I now have a backup server online to take over. (The server on which I archive the DB was fine, but unsuitable to web hosting).

    I'm investigating a system with HW RAID.

    On a related topic, I'm looking into methods of backing up the DB with a call to suspend the forums and then reactivate them after the backup is complete. At the moment I just stop apache and run the mysql backup.

    Any suggestions?

  2. #2
    Moderator
    Good Guru
    Schotty's Avatar
    Join Date
    Jul 2001
    Location
    Milwaukee, WI
    Posts
    5,760
    Setup a SQL cluster of a few machines and have the rendundancy act as your first line of defense and have it structured something like this

    Server 1: Apache, Mysql
    Server 2: Mysql
    Server 3: Mysql, somebackup to another media or offsite

    This gives you a machine that can be dedicated to the main job, one for backingup and another that has just the db stuff just in case.

    I am sure there is a brighter solution, but not being a DBA I am a little slow on this kind of stuff, plus not adminning in a few years, I am sure the tech got better since my foray into IT.

    HTH.

  3. #3
    Administrator Advisor peter's Avatar
    Join Date
    Apr 2004
    Posts
    882
    Thanks, I'm going to think about this on the weekend.

  4. #4
    Quote Originally Posted by peter View Post

    On a related topic, I'm looking into methods of backing up the DB with a call to suspend the forums and then reactivate them after the backup is complete. At the moment I just stop apache and run the mysql backup.

    Any suggestions?

    You could use a cron job and call this script.... I call this script 4 times a day for my forums and then use rsync to move copies of the database backups to several servers several times a day as well. The "--opt " locks the tables while backing up. So I never turn off apache at all.

    Code:
    -------------------------------------------------------------------------
    
    #!/bin/sh 
    for i in `mysql -B -e "show databases" | tail +2`; do
    mysqldump --opt $i | gzip -9 > /var/backups/mysql/$i.`date --iso-8601`.gz
    ls -t /var/backups/mysql/$i.* | tail +7 | xargs rm -f
    done

Similar Threads

  1. Recent site outages
    By peter in forum Announcements and Suggestions
    Replies: 3
    Last Post: 11-03-2014, 07:10 AM
  2. GMail server outages !
    By Compunuts in forum General Chat
    Replies: 7
    Last Post: 08-31-2004, 01:53 AM
  3. what a weekend!!
    By jme in forum General Chat
    Replies: 3
    Last Post: 05-15-2004, 07:50 AM
  4. There goes my weekend
    By ursidae in forum General Chat
    Replies: 11
    Last Post: 06-01-2002, 02:17 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •