File Server Issue
One of our file servers is experiencing difficulties that we are currently working on diagnosing. This will cause load and downtime issues for users on these servers:
abbazaba absinthe achilles adams admiralty aero alameda almalibre alondra altoid alvarado archibald arrow atlantic avalon aviation babyruth badillo balu bandini barranca barry baseline bazooka beaudry bellflower beverly bixel blowpop bonbon bottlecap bounty bourbon boyd broadway brownie bubbleyum bundy burton butler butterfinger cahuenga caramello casamento catalina centinela century cerritos cheeto cheezit chex chiclet cienega cognac colbert compton cookie cordial crackerjack crenshaw crunch cupcake dingdong doheny dorito drumstick dumdum edendale euclid fairfax fc23 federal fig fireballs firestone fletcher flower foothill formosa frito frontseat fruitland fudge fundip garland garth garvey gobstopper goober good gower grand granola grappa gumdrop gummi haven heath heaven highland hillhurst hillmont hoho holt hoover hope hops hyperion jalapa jawbreaker jefferson jerky juniormint jurupa kidsgames kiki kitkat krackel lankershim larchmont laurel lemonhead licorice lifesaver limbo-spunky3 limbo-spunky4 locust look lucas malt manchester mariposa mayo melrose mento milliken mills monroe moorpark mounds munchies nash necco nerds nilla nougat nutrageous oakdale ocean olive olympic ord oreo ouzo overland oxnard paramount payday pernod pez pico planters plenty pocky popcorn poprock powerbar pretzel price pringle rampart razzle redhot redondo reeses rexford ritz rockyroad rodeo rollo roscoe rosecrans rossmore rum runts sake saltine sanborn sawtelle schnapps scotch selma sepulveda shocktart skittle skor skybar slauson smores snicker snocap soto sourpatch spree starburst sugardaddy sunset taffy tampa temple tequila toblerone toffee tootsie topanga torrance truffle twinkie twix twizzler union vanowen vermouth vine virgil warhead washington western whittier whopper wilmington wilshire york zagnut zuma
If you are on these servers, this will may also cause email services to be unavailable. We apologize for this, and we will post more information as it becomes available. Thank you for your patience.
Update 9:50 a.m. PDT: The file server had hung on a thread and decided to throw a kernel panic. Right now it’s saving crash data and then should boot up. After it boots up I’ll need to soothe the servers that are hurting because of the broken mount. We should be back up and running within a half an hour, but I’ll post again if something else goes wrong.
Update 9:56 a.m. PDT: The file server was still not happy and died again. We are still working on resuscitating it.
Update 10:08 a.m. PDT: We found the problem. A bad drive was completely hosing the file server. We replaced the drive and it booted just fine. I’ll be massaging servers that still hurt now.
Update 12:38 p.m. PDT: The rest of the servers that were affected by this have been fixed. If you continue to have issues, please contact support.
.
February 8th, 2008 at 9:23 am
Oh… Crapp….
February 8th, 2008 at 9:24 am
Everything is down, i hope it will be resolved quickly. =O(
February 8th, 2008 at 9:27 am
Please resolve it quickly, all my sites are down!
February 8th, 2008 at 9:27 am
And I thought I broke it! Whew! It wasn’t my super long Blog Carnival.. Annoying let’s hope there’s no permanent damage to the files!
February 8th, 2008 at 9:30 am
Investor, if that’s the case: “In DH backups we trust XD”
February 8th, 2008 at 9:31 am
not just email - whole site is down.
February 8th, 2008 at 9:36 am
I am glad you guys are working it, but I must say I object to the fact that all of my sites, email, and jabbers are down gets classified as only “medium” severity. If EVERYTHING is broken and that is only “medium,” what the heck qualifies as severe?
February 8th, 2008 at 9:36 am
More down time, yippee.
February 8th, 2008 at 9:37 am
As it happened, I was messing around with my .htaccess file at the time of the outage. I was certain it was all my fault.
February 8th, 2008 at 9:40 am
damn sites are down
February 8th, 2008 at 9:40 am
I guess the server can’t handle htaccess, blog carnivals and coffee pots all at the same time:! ;D
February 8th, 2008 at 9:41 am
Will, your site is now loading SLOWLY!
Kenneth
February 8th, 2008 at 9:43 am
This is NOT a MEDIUM severity! I am losing business, all my sites, email, etc. are down! This is SEVERE!
February 8th, 2008 at 9:45 am
Arg! After that whole billing fiasco, now this? I’m seriously loosing my faith on DH…
February 8th, 2008 at 9:46 am
Yeah I agree this should be marked as SEVERE, since everything (http, email, ftp) is down!
February 8th, 2008 at 9:46 am
I agree, this is not a mere inconvenience. This is not a medium issue. All of my work is at a total standstill. My clients are going to be so VERY pleased.
February 8th, 2008 at 9:47 am
To quote DMB:
“….you pay for what you geeeeeeeeet….”
February 8th, 2008 at 9:47 am
Hurry up and fix! Hurry up and fix! Hurry up and fix! Hurry up and fix! Hurry up and fix!
February 8th, 2008 at 9:48 am
I think it’s just “Medium” because we may recover from it unscathed (pissed, but able to just resume normal operations). I think it would be severe if there was data loss, or equipment damage, or an outage lasting a long time.
February 8th, 2008 at 9:51 am
So only 234 or so are down. I guess if dreamhost has about 50,000 servers then this really is a ‘medium severity’ issue >.>
After the whole billing thing recently, other ‘minor’ problems, and now this… with the change in hosting plans we had not so long ago, I think those of us who have been here for awhile and stick around after all this crap should get our space and bandwidth doubled again like they used to do for us at random.
Yes, that is coming from a person who isn’t even using 2% of space and even less of bandwidth, but still paid another 94 cents per month for the upgrade :p
I want to see a picture of Josh with bald spots from ripping his hair out over this!
February 8th, 2008 at 9:54 am
SEVERE!
ps what is the ugly font on this page?
February 8th, 2008 at 9:55 am
Can everybody who says “you get what you pay for” just stop already.
I paid for hosting. no more no less. I didn’t pay for it to be down all the time!
I like DH. They are the best alternative out there but the downtimes are getting just a tad annoying now!
February 8th, 2008 at 9:59 am
You must be a fan of nickel slots…
Do you get this upset when they don’t pay out, too?
February 8th, 2008 at 10:04 am
yes please hurry, this is really annoying
February 8th, 2008 at 10:06 am
My sites are on the rockyroad server. They were down for a bit, but they are now back up. So this probably means that there is a light at the end of the tunnel for the rest of you, too!
February 8th, 2008 at 10:08 am
Holy crap that’s a lot of servers.
Of course, this will be the first time I’ve had any downtime in a about six months. I feel lucky.
February 8th, 2008 at 10:09 am
oh man, timing always seems to work out like this. I just posted the announcement for the official Chiditarod 2008, and everyone will be coming to the site. like, now.
please do your best, dreamhost team. i need you now.
ps - redundancy???
February 8th, 2008 at 10:14 am
some of my sites are coming back….
February 8th, 2008 at 10:16 am
“Update 10:08 a.m. PDT: We found the problem. A bad drive was completely hosing the file server. We replaced the drive and it booted just fine. I’ll be massaging servers that still hurt now.”
this is exactly why i LOVE dreamhost. downtimes happen to every host. but i’d rather read a “real” explanation like this than crappy business-speak. dreamhost, i love you.
February 8th, 2008 at 10:18 am
Funny the way it’s hitting me
1/2 my sites are working 1/2 my sites are down.
February 8th, 2008 at 10:24 am
Anyone on Burton have their sites back yet?
February 8th, 2008 at 10:26 am
I am on pernod and my sites are up.
February 8th, 2008 at 10:30 am
uff
February 8th, 2008 at 10:31 am
Burton is still down for me
February 8th, 2008 at 10:31 am
it is severe. all sites are down. all my future sites will have to be on another host, sadly.
February 8th, 2008 at 10:32 am
melrose still down
February 8th, 2008 at 10:35 am
How the fuck can a single drive take down an entire file server? Unless it was THEE master drive, and in that case, it should have had 2 redundant mirrors. Using a Ti-86 to switch NFS servers? Do you guys know what redundant raid is?
February 8th, 2008 at 10:39 am
How long is “massaging” supposed to take?
February 8th, 2008 at 10:40 am
I’m up! way go! good job!
Thanks for getting that going so fast. Keep up the good work.
February 8th, 2008 at 10:40 am
while admiralty is still down, i am kind of enjoying the phrase “massaging a server”
February 8th, 2008 at 10:41 am
melrose is up, thanks
February 8th, 2008 at 10:42 am
maybe you should hire envygeeks for a day of consultation?
February 8th, 2008 at 10:43 am
My websites are down, but happily my email works
Jabber is dead too…
February 8th, 2008 at 10:45 am
Awesome! I was paying someone to test my new site today. Now I’m paying someone to sit here at the office and watch the connection be refused.
February 8th, 2008 at 10:46 am
İnternetin Sosyallaştığı Site. Bir Pirikette Sen Koy! Site Down
http://www.piriketseverler.tr.gg
February 8th, 2008 at 10:49 am
Well, archibald seems to have finally waken up!
February 8th, 2008 at 10:50 am
… I seem to be back
February 8th, 2008 at 10:54 am
Aviation is still down. Help!
February 8th, 2008 at 10:58 am
The irony here is that my primary dedicated server with SoftLayer died miserably yesterday so I temporarily switched my largest site over to DreamHost. When it rains it pours.
February 8th, 2008 at 10:59 am
+1 ender. Real explanations are best.
… waiting patiently - kinda - not - please hurry and bring aviation back up