Dreamhost firewall error (Resolving)
Due to a typing error on our primary router while trying to block a denial of service attack, DreamHost is currently offline. This includes all email, web hosting, etc. A technician is about 15 minutes out from our datacenter to undo this mistake.
I apologize for this mistake. I was intending to be editing our non-live access-list, and edited the live one by mistake.
Update 9:54 PM Pacific: We have unblocked the firewall and our routing is coming back online. Sites should start working again as soon as our router finishes loading its tables. I use “resolving” because it is still processing the routes.
.
March 7th, 2008 at 9:30 pm
Dear All:
Your blogs suck. You don’t make money off of your dreamhost hosted websites. You are pathetic. STFU.
March 7th, 2008 at 9:30 pm
Wow, talk about Murphy’s Law, I JUST found out the Spam block has for some reason quarantined a LOT of my legitimate email for some reason. Now, here’s the funny part. I JUST this week changed the settings to only send me the quarantine list once a week, on Friday, so that’s why I just now saw it. PLUS, I shortened the time to keep the files too (because until NOW, they had all been crap and I was tired of looking at them, but OF COURSE, now that it is REAL mail that is going to be thrown away very soon, unless I can log in to webmail and white list it, I CAN’T get in to do it!Q@@0 Do you see how funny it is?? I mean funny in an ironic, frustrating, tear your damn hair out kind of way, not Ha Ha.
March 7th, 2008 at 9:30 pm
Posted 41 minutes ago (March 7th, 2008 at 8:47 pm PST) by Kelly
Hey, is there a pilot in the plane?
March 7th, 2008 at 9:30 pm
While you’re waiting, it could be worse:
http://www.theonion.com/content/video/breaking_news_all_online_data
March 7th, 2008 at 9:30 pm
That’s going to be a lot of email.
March 7th, 2008 at 9:30 pm
Error because of a typo …
rm -R /*
???
:p
March 7th, 2008 at 9:30 pm
I think for $10 a month it’s gonna happen every once in a while…the freaks that expect perfection need to open their wallets, not their mouths, and buy themselves a perfect world to go live in…or just drink less.
March 7th, 2008 at 9:31 pm
I don’t understand why someone has to come to the network center because someone made a typo?!? Have the guy call someone on a frickin’ phone and tell him what to type to fix it! How can you possibly be down for this long because of a TYPO!!?? Just hit UNDO or something, CHRIST!
And to the moron posters: It’s LOSE, not LOOSE!!!! Gah!!!
March 7th, 2008 at 9:31 pm
My sites are back up!
LOL, I’m joking!
March 7th, 2008 at 9:31 pm
Okay, I’ve just gotta know - is there an IRC chatroom for this sort of occasion?
March 7th, 2008 at 9:31 pm
dude this sucks :((
March 7th, 2008 at 9:31 pm
Dreamhost is up FINALLY!!!!!!!!!!!!!!
March 7th, 2008 at 9:31 pm
Hey atleast this is not as bad as a typo bringing down a space shuttle. (some genius put a + instead of a -
March 7th, 2008 at 9:31 pm
Looking back through this status blog is depressing. In the first 18 months of use, dh has been the least reliable hosting provider I’ve ever used (since 1995). If my entire job was to keep computers running, you bet your sweet ass they wouldn’t be down more than 12 hours every 1,000 years. Hopefully you’re not counting on hacking fixes for all the other known ISP failure modes as they’re occurring — and going home before testing them.
March 7th, 2008 at 9:31 pm
Thank God Dreamhose isn’t in charge of our nuclear arsenal.
March 7th, 2008 at 9:31 pm
Dreamhost is not working now.
Everyone cries loudly in their large beer.
Bitch better have my money!
March 7th, 2008 at 9:31 pm
DanVanWinkle, putting this site NOT in their hosting makes perfect sense. Same with site monitoring - those pretty much have to be external to be of any use at all. If you’re watching them from inside, and someone backhoes your feed, you’ll not know until the phones start ringing. Measure the system from outside, just as your customers do.
Same thing for a status or support site, very effective to be able to have that on an entirely different network. Can you reach this one? No? How about that one? OK, if you can’t reach both, that tells us (thing), if you can’t reach just one, that tells us (other thing). If the status site wasn’t outside, we’d all be wondering WTF just happened and not know so - I’d say it was a good decision on their part.
That said, as someone else mentioned, I’m losing 0.00000000000000009 cents a minute in adsense revenue here guys, hurry up and all that (bleh…whatever).
March 7th, 2008 at 9:32 pm
Wait, now I have a question, where in the world is the secondary firewall during all of this stuff? Can’t you just fall back to the secondary, or did that typo screw that up also?
March 7th, 2008 at 9:32 pm
Yo Verb - I don’t think Christ works there…back to the perfect world thing…
March 7th, 2008 at 9:32 pm
Monday, 9am Pacific Time - all DH employee have their fingers off, of course.
March 7th, 2008 at 9:32 pm
At least we know the firewall works!!!!!!!!!!!!!!!!!!!!!!! GOOOOOOOOOOOOO FIREWALL!!!!
March 7th, 2008 at 9:32 pm
For the price, DH is great, but downtime comes at the worst times. lol
March 7th, 2008 at 9:32 pm
@ #326 Cal: You’re a moron. Ever think some people might have hundreds of dollars worth of PPC ads running to these sites.
Just because you haven’t figured out how to make real money online, doesn’t mean other people haven’t.
March 7th, 2008 at 9:32 pm
DAMMIT!!! I am updating my site, now it’s not even up!?!?!?!
it’s okay i can wait. Hope they fix it soon!
March 7th, 2008 at 9:32 pm
I really hope someone is getting fired over this. This is completely unacceptable!
March 7th, 2008 at 9:32 pm
“Will the outage last long enough to bring down this blog?”
That’s only happened once to my knowledge. There was one major outage though - I forget the cause of it - where there was so much traffic to dreamhoststatus.com that they converted it to static HTML because the server was being overloaded with PHP requests.
-Kirk
March 7th, 2008 at 9:32 pm
I wonder where ‘DreamhostStatus.com’ is hosted?
March 7th, 2008 at 9:32 pm
I highly recommend “XP Programming” when working on critical systems like this. With two pair of eyes, it is much easier to spot mistakes before they happen.
March 7th, 2008 at 9:32 pm
i am loosing $100 billion/sec
March 7th, 2008 at 9:32 pm
http://www.despair.com/potential.html
March 7th, 2008 at 9:33 pm
Jesus people, some of us pay $100+ for VPS here. We’re not all on the $5 plan
March 7th, 2008 at 9:33 pm
I’m not worried, I know things will get back to normal (sooner than later I hope). On the other hand, I was just thinking that with all the back and forth that goes on in these comments, it’d be really cool if the links weren’t rel=”nofollow”
March 7th, 2008 at 9:33 pm
Update? Please?
March 7th, 2008 at 9:33 pm
@ #337 - The Red Krawler
Hahahhahahaha! Freaking awesome.
March 7th, 2008 at 9:33 pm
Shit happens. Hope you will get it sorted soon.
March 7th, 2008 at 9:33 pm
Taking the scenic route, huh? It’s been a LONG 15 minutes to get there. Stopping to grab dinner, dessert, a few donuts and a coffee?
I probably won’t cancel my subscription, but you’d better believe I’m NOT going to recommend Dreamhost to ANYONE and will definitely NOT be using Dreamhost for any “serious” hosting.
These outages are getting to be ridiculous. Every few weeks I have to email support because PHP5 has been blocked yet again, every few months the entire site becomes inaccessible due to some stupid mistake.
March 7th, 2008 at 9:33 pm
“I wonder where ‘DreamhostStatus.com’ is hosted?”
According to my traceroute, it’s hosted with www.servepath.com.
-Kirk
March 7th, 2008 at 9:34 pm
This is ridiculous. I know people shouldn’t be so upset. But proper precautions weren’t taken. I had been keeping a couple of sites on dreamhost with the idea of moving them all off, but I drug my feet. Bad move. I guess I will be moving them off to my other server. I know it sucks, but one hour of downtime is a bit rough. This is the third time in the last several days that I have suffered enough downtime with Dreamhost to affect me financially.
March 7th, 2008 at 9:34 pm
RonPaul2008.com is down!
March 7th, 2008 at 9:34 pm
UPDATE: The tech guy was caught in the traffic.
March 7th, 2008 at 9:34 pm
Need to use a Goal Objective !!
http://www.freegoalsetting.com
March 7th, 2008 at 9:34 pm
I find this all really funny, actually. Give them a break. It could happen to anyone.
March 7th, 2008 at 9:34 pm
I wish all those jumping up and down about looking for a new host would realize what whiney cheapskates they are making themselves out to be. I’m not a sycophant but I don’t hate Dreamhost either. I just recognize it for what it is: ridiculous amounts of storage and tons of features for next to nothing.
Sometimes things go pear shaped but for the price I’m not going complain a bit. I’ve had other companies promise better support and uptime but they also want to charge $$$ for limited storage, databases, access, etc, etc… For my businesses that requires multiple redundancies and engineers on-site 24/7, I pay out. But for the operations that don’t, I accept the risk with Dreamhost.
March 7th, 2008 at 9:34 pm
“If you are experiencing a problem not reported here, check our web panel for more information.”
*clicks*
*waits*
“Oh, right!”
March 7th, 2008 at 9:34 pm
Carl Lumma, pls send your resume to josh@dreamhost.com
March 7th, 2008 at 9:34 pm
Carl Lumma, your quote of “If my entire job was to keep computers running, you bet your sweet ass they wouldn’t be down more than 12 hours every 1,000 years.”, makes it pretty clear to those of us whose job it IS to keep computers running, that you don’t have the first clue of WTF you’re talking about.
Please, don’t now respond with the usual tactic of “how much does Dreamhost pay you to blah this blah that whatever” tactic. Just don’t. We both know I don’t work for them.
March 7th, 2008 at 9:34 pm
HAHA This is great!!! Block the whole world from DH!!! I love it man, blame it on the typos or the crack or you guys moved to crank by now since you are close to Mexico where they have the big Meth houses :P.
Anyhow as they say sh@# happens! HAHA love this though, look at the responses, seems the $10/mo nitwits are crying again.
Hope to see DH back online, at lease Link is taking a break though!!! Poor linky has had a tough week. Hope you all get it back up because I’ll send out my invoices before long too you guys for the downtime 
Where is Josh when you need the poor guy?
March 7th, 2008 at 9:35 pm
shouldn’t editing a live access-list for hundreds or thousands of site’s be harder than a TYPO?
March 7th, 2008 at 9:35 pm
I’m guessing that the typo’d router is preventing them from remoting into their data center. You can’t undo something you can’t connect back to.
Kelly, mistakes happen. Have a few beers during the weekend and you’ll be right as rain.
March 7th, 2008 at 9:35 pm
“RonPaul2008.com is down!”
No it’s not, it’s working fine on my end, and DreamHost still isn’t.
-Kirk