MYSTcommunity: Server was down due to harddrive failure - MYSTcommunity

Jump to content

Page 1 of 1

Server was down due to harddrive failure All back on

#1 User is offline   benb 

  • glotahn (beginner)
  • Group: Admin
  • Posts: 239
  • Joined: 07-March 92
  • Location:Germany

Posted 26 August 2005 - 01:42 PM

First, for those who don't know me: I am not a Myst gamer, but I administrate (and share) the server that this forum is hosted on. The server itself is running under Linux and in operation since almost 3 years now, with an uptime above 99.99%.

Last week, we started to have problems with the harddrive, the partition with all the data just stopped working. A reboot fixed it, but that's not a strategy you use on servers, and the problem kept coming back a few times. I asked Chucker to make backups (and I think he did). A look at the harddrive's self-diagnostic (SMART) showed lots of problems. Not too long, and the server didn't boot anymore (I still don't know the exact symptom, some kernel panic about not finding something). That was a few days ago.

We asked to have the harddrive replaced, but I needed to install the previous operating system on the new drive. Installing an operating system on a machine, without physical access to it, with the target harddrive the only option to boot, is a very hard task. It would have been easy for the guys at the hosting facility, but they refuse to do any custom work, they only reboot or bulldoze over. To complicate matters futher, the default operating system they put on didn't boot either. Combine all 3 problems, and it gets worse.

Anyways, a KVM and some guessed lilo parameters later, the server booted again, and I was able to put the previous installation back on the new drive.

It's back on and opertional (ha ha, otherwise you wouldn't be able to read this).

Was any data lost? I don't know. It wasn't a complete failure, I could copy the contents of the old drive back onto the new one, so basically everything should be back and exactly like before. However, I don't know, if any individual files, or some changes to the database, got lost, either due to the harddrive failure or the reboots. I haven't noticed anything yet. (Sure, I could crawl through logs for hints, but I spent enough time on this.)

Now I just need to try to securely erase the data on the old drive.


Have fun,

Ben
0

#2 User is offline   Jerle 

  • What, me worry?
  • Group: Admin
  • Posts: 3,097
  • Joined: 04-January 01
  • Gender:Not Telling
  • Location:Location! Location!

Posted 26 August 2005 - 01:46 PM

:mower: Who ARE you?

Seriously thanks so much, I know this was a headache for you guys to fix.... way to go! :)
0

#3 User is offline   amon-re 

  • lontahn (discoverer)
  • Group: Team Member
  • Posts: 636
  • Joined: 16-April 03
  • Gender:Male
  • Location:near Ghent, Belgium
  • KI number:01903598

Posted 26 August 2005 - 01:48 PM

Great work bringing the server back, benb! :) :mower:
0

#4 User is offline   J'ohn 

  • oglahnth (ancient one)
  • Group: Member
  • Posts: 10,928
  • Joined: 01-June 01
  • Gender:Male
  • Location:Athens, Greece

Posted 26 August 2005 - 01:59 PM

Thank you very much, Benb and all who worked for the repair. :mower:

SBS :)
0

#5 User is offline   Cactus Wren 

  • shokhootahn (instructor)
  • Group: Veteran Member
  • Posts: 951
  • Joined: 13-November 04
  • Gender:Female
  • Location:A day's drive west of the Cleft
  • KI number:01530661

Posted 26 August 2005 - 02:24 PM

Add my thanks to those already posted.

benb, on Aug 26 2005, 01:42 PM, said:

First, for those who don't know me: I am not a Myst gamer [ ... ]

We could turn you into one! :mower:
0

#6 User is offline   PC Michael 

  • telnahvah (guildmaster)
  • Group: Member
  • Posts: 1,537
  • Joined: 05-June 01
  • Gender:Male
  • Location:~/

Posted 26 August 2005 - 02:29 PM

Thanks so much!! That is really appreciated, I can tell you.
0

#7 User is offline   christopherger 

  • shokhootahn (instructor)
  • Group: Veteran Member
  • Posts: 816
  • Joined: 10-July 05
  • Gender:Male
  • Location:Colorado and somewhere beyond......

Posted 26 August 2005 - 02:50 PM

To the Honourable Benb, Esq. :mower:

Greetings and salutations and most profound thanks and grattitude
for the rescue work performed, and for sharing the details thereof
with us all. Most estemed sir, much obliged greatly :)

No Data lost from the areas that I have posted that I have noted.

thanks again :P ;) ;) :twitch: :twitch:
0

#8 User is offline   linkerjpatrick 

  • oglahnth (ancient one)
  • Group: Member
  • Posts: 6,255
  • Joined: 01-June 01
  • Gender:Male
  • Location:Greenville, SC

Posted 26 August 2005 - 03:26 PM

I like how fresh the new drive fells! Ahhhh!!!! Like a pair of clean underwear! :mower:
0

#9 User is offline   Linklinker 

  • lontahn (discoverer)
  • Group: Member
  • Posts: 685
  • Joined: 01-January 04
  • Gender:Male
  • Location:TARDIS

  Posted 26 August 2005 - 03:53 PM

YES ALRIGHT WERE BACK UP WHEEEEEEEEEEEEEEEEEEEE!!!! :mower:
I started panicking when I couldn't get on.
Oh and Chucker I hope you get to feeling better with the ear.

Quote

I like how fresh the new drive fells! Ahhhh!!!! Like a pair of clean underwear! braces.gif

I hope it didn't fall.
Ahh clean underwear. (I had to put some on when I couldn't get on Mc.)
So Mc is not sick anymore. YEAH!
0

#10 User is offline   benb 

  • glotahn (beginner)
  • Group: Admin
  • Posts: 239
  • Joined: 07-March 92
  • Location:Germany

Posted 26 August 2005 - 04:29 PM

Jerle, on Aug 26 2005, 08:46 PM, said:

:mower: Who ARE you?

The killroy watching you

Posted Image
0

#11 User is offline   J'ohn 

  • oglahnth (ancient one)
  • Group: Member
  • Posts: 10,928
  • Joined: 01-June 01
  • Gender:Male
  • Location:Athens, Greece

Posted 26 August 2005 - 05:37 PM

linklinker, on Aug 26 2005, 11:53 PM, said:

Quote

I like how fresh the new drive fells! Ahhhh!!!! Like a pair of clean underwear! braces.gif

I hope it didn't fall.
Ahh clean underwear. (I had to put some on when I couldn't get on Mc.)
So Mc is not sick anymore. YEAH!

This quote is wrong on so many levels...! :)

SBS :mower:
0

#12 User is offline   David Mouse 

  • telahm (guildsman)
  • Group: Veteran Member
  • Posts: 1,265
  • Joined: 17-April 02
  • Location:Orange County, California

Posted 26 August 2005 - 05:41 PM

I wondered what was going on.

THanks.
0

#13 User is offline   Fireymarbles 

  • Who says you're free?
  • Group: Veteran Member
  • Posts: 5,579
  • Joined: 28-October 03
  • Gender:Male
  • Location:Auckland, New Zealand

Posted 26 August 2005 - 06:19 PM

Heh, I guessed this had something to do with the hard drive weirdness that was going on before... but congratulations on getting everything back so fast, that really sounds like a heck of a job! :mower:
0

#14 User is offline   Carter 

  • glotahn (beginner)
  • Group: Member
  • Posts: 124
  • Joined: 10-June 04

Posted 27 August 2005 - 01:20 AM

securely erase the drive?

you could try....Something like these to securely destroy that hard drive...hehe....
0

#15 User is offline   Wanais 

  • lontahn (discoverer)
  • Group: Member
  • Posts: 559
  • Joined: 31-December 04
  • Location:Uppsala, Sweden

Posted 27 August 2005 - 01:26 AM

carterhawk, on Aug 27 2005, 08:20 AM, said:

securely erase the drive?

you could try....Something like these to securely destroy that hard drive...hehe....

That might be fun, but hard to do without physical access to the hard drive. (On the other hand, collectively we might be able to achieve physical access :mower: )

Thanks for all the hard work! I have similar experiences as you from hosted services - it is a hassle.
0

#16 User is offline   The Lerneaen Hydra 

  • telahm (guildsman)
  • Group: Member
  • Posts: 1,099
  • Joined: 06-April 02
  • Gender:Male
  • Location:Gothenburg, Sweden

Posted 27 August 2005 - 05:50 AM

I'm not sure I understand this, you rent a server from a large company, and when a harddrive b0rked, you had to get a new one and reinstall the OS and get the data from the old drive? Isn't that the responsibility of the hosting service?

(Maybe a day to day backup thingy would be a good thing to setup, prefferably to a physically different machine).
0

#17 User is offline   chucker 

  • oglahnth (ancient one)
  • Group: Admin
  • Posts: 5,330
  • Joined: 04-January 01
  • Gender:Male
  • KI number:196082

Posted 27 August 2005 - 03:02 PM

The hosting company installed the new drive, but they didn't install the operating system, since it is our custom choice.
0

#18 User is offline   Pazuzu 

  • lontahn (discoverer)
  • Group: Member
  • Posts: 519
  • Joined: 11-March 04
  • Gender:Male
  • Location:Colorado, at least at the moment

Posted 27 August 2005 - 09:14 PM

Yes... many thanks, everyone. :mower: Great to see everything back up and running here again so soon!
0

#19 User is offline   River 

  • River
  • Group: Member
  • Posts: 2,607
  • Joined: 22-March 04
  • Gender:Male
  • Location:Oshkosh WI

Posted 30 August 2005 - 11:03 PM

Let us not forget - CHUCKER backed up this board to be able to restore it to it's known sweetness.

BIG :mower: to chucker
0

#20 User is offline   chucker 

  • oglahnth (ancient one)
  • Group: Admin
  • Posts: 5,330
  • Joined: 04-January 01
  • Gender:Male
  • KI number:196082

Posted 31 August 2005 - 04:18 AM

Actually, since benb was able to transfer the contents over from the old hard drive directly, no restoring of a backup was necessary.
0

#21 User is offline   christopherger 

  • shokhootahn (instructor)
  • Group: Veteran Member
  • Posts: 816
  • Joined: 10-July 05
  • Gender:Male
  • Location:Colorado and somewhere beyond......

Posted 31 August 2005 - 11:24 AM

Thank you Chucker et al for the salvation of the forum....much appreiciated
indeed. Glad there are back ups!
0

Share this topic:


Page 1 of 1


Fast Reply

  

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users