Dreamhost firewall error (Resolving)
Due to a typing error on our primary router while trying to block a denial of service attack, DreamHost is currently offline. This includes all email, web hosting, etc. A technician is about 15 minutes out from our datacenter to undo this mistake.
I apologize for this mistake. I was intending to be editing our non-live access-list, and edited the live one by mistake.
Update 9:54 PM Pacific: We have unblocked the firewall and our routing is coming back online. Sites should start working again as soon as our router finishes loading its tables. I use “resolving” because it is still processing the routes.
.
March 7th, 2008 at 9:23 pm
It takes the guy 15 minutes to get there, 15 minutes to get down to work and probably another 15 to fix it. I don’t think we’ll be up for a while, just DO SOMETHING ELSE. Me, I’m having fun eating cheese and watching the world flip out :3
March 7th, 2008 at 9:23 pm
#40 Zathman: you owe me one new laptop. I sprayed my beverage, LOL
March 7th, 2008 at 9:24 pm
Meanwhile we’re on our way to DDOSing www.dreamhoststatus.com! BWAHAHAHA
March 7th, 2008 at 9:24 pm
Its still not UP.
March 7th, 2008 at 9:24 pm
Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it
Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it
Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it
March 7th, 2008 at 9:24 pm
GARRETT funcking fuckity GARRETT funcking fuckity GARRETT funcking fuckity GARRETT funcking fuckity GARRETT funcking fuckity GARRETT funcking fuckity GARRETT funcking fuckity GARRETT funcking fuckity
March 7th, 2008 at 9:24 pm
lo, que, representa
la niña esta en mi cabeza, esa niña…
la q esta liando zapatero
March 7th, 2008 at 9:24 pm
My kids are screaming because my sites are down! They can’t play their videos!
March 7th, 2008 at 9:24 pm
D.O.S. attack
Dreamhost warriors shield us all
downed sites are quite safe
March 7th, 2008 at 9:24 pm
maybe the typo is rm -rf /
and all file servers are mounted below /
so……..
March 7th, 2008 at 9:24 pm
lol PWNED by a typo
March 7th, 2008 at 9:24 pm
Obviously the typo was a cover up for Something …
God know what they done again!
Other then break the streek of almost 2 month without down time!
Just another to piss everyone off
March 7th, 2008 at 9:25 pm
“And I want to see one of you stupid and ignorant babies configure one of these bloody things -once- let alone to stop some punk with malicious intent from actively attempting to bring you down. Resolution within the hour is impressive even with larger corporate setups.”
I’m a CCNP so I do have a clue about configuring routers. The first thing you make sure of (especially with a border router or firewall) is that your out of band access is working before making any emergency changes to a production router. Yes, it is very easy to lock yourself out when you goof up an access list. That’s why you always make sure you have a way to back out the changes before you start making them.
March 7th, 2008 at 9:25 pm
an hour already wooooowwwwwwwwwwwwwwwwwwwww
GHETTO SUPERSTARRRR TAHT IS WHAT U AREEEEEEEEEEEEEE
March 7th, 2008 at 9:25 pm
For the record I am still happy with DH, this is the first real issue I have had in the past couple of years.
Anyone started typing up a news bit yet?
March 7th, 2008 at 9:25 pm
I’ve lost $3 million in revenue from this down time.
March 7th, 2008 at 9:25 pm
Gonna go eat some candy cigarettes. Call me when they’re done.
March 7th, 2008 at 9:25 pm
FUCK DREAM HOST I AM DONE WITH IT. I AM GONNA CHANGE MY HOST TOMORROW.
March 7th, 2008 at 9:25 pm
every second this stays down i lose respect for humanity
March 7th, 2008 at 9:26 pm
Dreamhose strikes again.
March 7th, 2008 at 9:26 pm
Why the frik is there not someone there full time who can deal with these sorts of issues, for all the customers and money you must lose each time this happens isn’t it worth it?
March 7th, 2008 at 9:26 pm
Wow, I’m glad that my sites aren’t critical… Seeing as I have made this mistake before, I am not mad at ya, thanks for telling us… Oh, and I am kinda surprised that you all don’t host your “Dreamhoststatus” page, or do you and you just don’t have it firewalled… interesting… well, I guess it’s smart to host this somewhere else if thats what you all do… Hmm, well, hope you get it back up soon!
March 7th, 2008 at 9:26 pm
“Money Blue Book Says (March 7th, 2008 at 9:24 pm):
Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it Fix it”
Only the Professor can fix it!
March 7th, 2008 at 9:26 pm
la la la la la
fat finger!…..
March 7th, 2008 at 9:26 pm
Posted 37 minutes ago (March 7th, 2008 at 8:47 pm PST) by Kelly
This is not serious!
March 7th, 2008 at 9:26 pm
This is funny, I have been building websites for 12 years and the one thing I found was this. The people who bitch and complain the most are the same ones who pay the least. AND they have the least amount of visitors to there website. Its funny how people can say I am loosing thousands of dollars because there website is down when they only get a few visitors and they dont have ecom or any forms / quote requests. just my happy thoughts
March 7th, 2008 at 9:26 pm
Oh shit, barackobama.com is down.
March 7th, 2008 at 9:26 pm
I feel for you - that shit-in-your-blood feeling when you realize that you’ve just screwed up really, really bad, is no fun at all. Anyone who has been in IT for any length of time and had responsibilities, has done something like this. Kudos to you for owning up to it.
That said - cyclades has a great range of products which would be useful in this situation, so your guy doesn’t have to have physical access. Put the console servers on a separate (and I mean, REALLY separate) admin network, VPN in to the console server, and there ya go. Add to that a clueful datacenter tech who can follow direction and you’re golden.
March 7th, 2008 at 9:26 pm
For every second my site is down, I’m losing $.000003 in Google AdWords revenue!?!? Damn you! I was planning to buy a beer in 2009!!!
March 7th, 2008 at 9:26 pm
hurry the fuck up!
March 7th, 2008 at 9:27 pm
Doooooooood…….
You’re telling us that one typo on one server brought down your entire service?
Everything?
What steps is Dreamhost taking so this doesn’t happen again?
March 7th, 2008 at 9:27 pm
I feel your pain, I have been the victim of a typo that took out a major tv provider for about 2 hours… If any one had dropped service durring the superbowl. sorry, it happens
March 7th, 2008 at 9:27 pm
the problem fix in 24 hours. thanks.
March 7th, 2008 at 9:27 pm
Bob, that’s nothing - I’m losing $3 million a minute! Damn you, Dreamhost!
March 7th, 2008 at 9:28 pm
“As usual, all we will get for Dreamhost’s screwup is a “sorry”. I am so glad I moved most of my critical sites to different hosting. Now all I have to do is move my blog (which is where most of my income comes from - thanks a bunch, DH ). And to think I paid in advance for three years of this crap.”
Hilariously enough, I moved all my “critical sites” *to* DreamHost, because they still have better speed, and generally better uptime, than anything else I’ve got.
-Kirk
March 7th, 2008 at 9:28 pm
Hurry up please.. we are all losing business here.
March 7th, 2008 at 9:28 pm
HELLO DREAMHOST STAFF
I AM NOT HAPPY WITH THE FERRARI YOU HAVE SOLD ME FOR POCKET LINT PRICES.
PLEASE TO BE GIVING ME 5 YEARS CREDIT FOR THE 10 MINUTES OF DOWNTIME I SUFFER EVERY 6 MONTHS.
I AM LOSING $11 MILLION DOLLARS FOR EVERY 2 SECONDS MY SITE IS DOWN I WISH FOR YOU TO COMPENSATE WITH ME FOR IT OK
GOOD BYE
March 7th, 2008 at 9:28 pm
Error because of a typo.
I was writing an awesome blog post.
ANd then the screen went crazy.
I was baffled for a sec, thingking what could be wrong.
A friend of mine showed me this.
So I read.
My blog post is gone.
I am sad
:’C
March 7th, 2008 at 9:28 pm
Fire whoever made the typo. This is ridiculous.
March 7th, 2008 at 9:29 pm
Every minute my sites are offline = dollars out of my pocket. :S
March 7th, 2008 at 9:29 pm
Hahahaha…Friday night…a couple of drinks…see what happened? hahahaha…anyway…CTRL-ALT-DEL it´s not a good combination of keys…CHEERS!!!
March 7th, 2008 at 9:29 pm
my site was down earlier in the week - I emailed support and by the time they got back to me it was up again - tech said to write back if it happened again - it happened again - checked dreamhoststatus.com and nothing - so I wrote back that it happened again - came back here and see this - misery loves company, I guess - coming up on 40 min, I think, since I noticed - ah well
March 7th, 2008 at 9:29 pm
These mother fuckers kept deleting my posts, but son of bitches wont go and fix the problem, GO FUCK YOURSELF LOSER AND FIX THE PROBLEM SEE WHO IS FUCKING YOUR MOTHER!!! GOD DAMN I AM SO FUCKING PISSED
March 7th, 2008 at 9:29 pm
Ahh, I’m just kidding.
So you’ve lost $3MM in 30 minutes? What do you sell, Crack?
March 7th, 2008 at 9:29 pm
Stop cryin’ kids!
Any of yawl ever f’d something up?
March 7th, 2008 at 9:29 pm
Why the hell did Dreamhost get rid of their dedicated host support number?
Can’t handle the heat?
March 7th, 2008 at 9:29 pm
“FUCK DREAM HOST I AM DONE WITH IT. I AM GONNA CHANGE MY HOST TOMORROW.”
lol, noes! I think you people just like kicking up dust and seeing what kind of conflict it leads to.
-Kirk
March 7th, 2008 at 9:29 pm
OK YOU ARE SO DONE!
This is just not worth it.
Tomorrow I’m moving my heart monitoring and bank vault security system to a better $10/month service. I might even move my missile control system next month!
March 7th, 2008 at 9:30 pm
Sorry, the assignment was to write a haiku. No credit.
March 7th, 2008 at 9:30 pm
Will the outage last long enough to bring down this blog?