Dreamhost firewall error (Resolving)
Due to a typing error on our primary router while trying to block a denial of service attack, DreamHost is currently offline. This includes all email, web hosting, etc. A technician is about 15 minutes out from our datacenter to undo this mistake.
I apologize for this mistake. I was intending to be editing our non-live access-list, and edited the live one by mistake.
Update 9:54 PM Pacific: We have unblocked the firewall and our routing is coming back online. Sites should start working again as soon as our router finishes loading its tables. I use “resolving” because it is still processing the routes.
.
March 7th, 2008 at 9:42 pm
GO GO GO Fixt the darned problem already!
March 7th, 2008 at 9:42 pm
“Anyone digg this? lol…”
i saw this new via digg
March 7th, 2008 at 9:42 pm
Not only do I host my personal website, but the school I am the technology director for hosts with Dreamhost for free as a non-profit. Free isn’t worth the 4h outage in 3 months.
I wrote to the support team and they basically told “it’s going to go offline f*ck off”.
I just bought my hosting plan today and I’m thinking about canceling. I hosted with Crystal Tech for 5 years and my site was never down once.
March 7th, 2008 at 9:42 pm
I’d like to say that I do understand. Typos happen, and when trying to block a DoS, you tend to get a tad bit flustered and rushed in the heat of the moment. Personally, I’d like to find out which group of people were DoSing a site hosted here and wring their necks.
March 7th, 2008 at 9:42 pm
>.
March 7th, 2008 at 9:43 pm
God, you are all the biggest fucking whiners I’ve ever, ever seen. Internet is broken? Jesus fuck, go outside for a while, what the hell is wrong with all you douchebags?
March 7th, 2008 at 9:43 pm
THIS IS ANNOYING!!!!!!
March 7th, 2008 at 9:43 pm
I want to see a website with big money. Maybe I learn something.
March 7th, 2008 at 9:43 pm
“What I’d like to know is where is their out-of-band management? They should have ways of getting into their network even when all else is hosed. A dial-in modem even, just for emergencies like this.”
Exactly. Changes to a border router or firewall should never be done without OOB management in place.
March 7th, 2008 at 9:43 pm
#418 Wrote:
> Please don’t tell us it’s going to be fixed in 15 minutes when an hour later it’s still down
Nobody said it would be fixed in 15 minutes. That’s how far out the tech was. Fix time wasn’t mentioned or estimated.
March 7th, 2008 at 9:43 pm
The Red Krawler: Go fuck yourself, money is money $10 or $100. If they take $10 it doesn’t mean they have to be down that often. Some people don’t need dedicated servers to run simple websites, which makes them money, alright fucktard?
March 7th, 2008 at 9:43 pm
NERD FURY!!!
March 7th, 2008 at 9:43 pm
Yeah baby, gimmie some of that down time!
March 7th, 2008 at 9:44 pm
Haha! lol. Good job
I’ll label you a terrorist now :-p
Naw, but stuff like that happens. I’ve brought down my own personal home server more than once by screwing up a file or two. Oh, and NEVER try to chmod the boot directory on *nix
We all make mistakes. I think that DreamHost really needs a better sort of backup incase of mistakes like this.
March 7th, 2008 at 9:44 pm
Born Yesterday’s right! IT’S BACK UP!!!!!! Oh, wait…that’s only if you’re INSIDE the hosting facility…
March 7th, 2008 at 9:44 pm
#429 Kirk - like this.
March 7th, 2008 at 9:44 pm
Everybody, I have video of the outage occuring:
http://www.youtube.com/watch?v=eBGIQ7ZuuiU
March 7th, 2008 at 9:44 pm
No one is PERFECT. so errors occur sometimes…
Just solve it ASAP
March 7th, 2008 at 9:44 pm
I am sorry but i have to say this all of you claiming to be loosing Millions are SO FULL OF SHIT!!! if you were really making that kind of money we would not be reading your posts here, we would be talking to your tech guy / girl. ALSO everyone already knows your Lying because to make that kind of money you have to be SMART and no one that is smart would put an important website like that on DREAM HOST, dont get me wron I love dream host for the small low traffic websites.
March 7th, 2008 at 9:45 pm
hey don’t be pricks people make stupid mistakes.. but hey dreahost get this stuff going again wtf
, wtf wt6wfwtlkasdl;kfjasdjf;laskdjf its been much longer than 15 minutes
March 7th, 2008 at 9:45 pm
Seriously… it takes an hour to undo a typo? You guys just can’t seem to do anything right
March 7th, 2008 at 9:45 pm
Woops,
Don’t sweat it, we’ve all made mistakes that affect others.
Ephesians 5:15-16 says: “Then watch how carefully you walk, not as unwise, but as wise ones, redeeming the time, because the days are evil.”
This is one of the “days are evil” moments, when everything goes to crap. So now it’s extra important to remember to redeem the time. Each moment has value, don’t waste it worrying about this mistake. Praise God that we are alive, and don’t sweat what happened.
March 7th, 2008 at 9:45 pm
A little something to keep people entertained during the outage. Here Comes Another Bubble! http://www.youtube.com/watch?v=I6IQ_FOCE6I
March 7th, 2008 at 9:45 pm
I hope it comes back soon.
March 7th, 2008 at 9:45 pm
dreamhoststatus.com is up.
all other sites r down, including blog/panel…..
March 7th, 2008 at 9:46 pm
Thanks for working so hard guys! We appreciate you! I’ve been a member almost ten years, and this is the first serious downtime I can remember in at least the last five years. Dreamhost has an excellent record and does great business. Thanks guys!
March 7th, 2008 at 9:46 pm
Cal, we’re joking - making fun of people who whine that they’re losing important amounts of money, but run there businesses off of dirt cheap webhosting.
March 7th, 2008 at 9:46 pm
DH should give us 1 free year for each 30mins OFF!
So… Take it easy with the fixing…
March 7th, 2008 at 9:46 pm
I kinda hope they don’t get it straightened out quickly. I mean, sleep would be good. I noticed that I’d been getting kinda paranoid lately - just sitting at the PC, changing PHP code to reverse the columns on my site for days on end without sleeping. Well, I wasn’t just changing code columns, I was also making a nifty banner rotator. That kept me up all Wednesday night alone.
So, if the servers stay down for a little while longer I’ll be forced to give up and go to bed.
If they’re down when I wake up, I might even splurge and try to squeeze in a shower.
March 7th, 2008 at 9:46 pm
i want my site hosted where this one is.
March 7th, 2008 at 9:46 pm
Thanks for giving the timely status and for being honest, I’m sure a lot of us here who are also admins have been through similar things as well haha.
March 7th, 2008 at 9:46 pm
It’s past the 15 minutes. Site back up yet? No.
Tick-Tock…Tick-Tock…Tick-Tock
Time’s-A-Wastin’…
Thomas
March 7th, 2008 at 9:46 pm
what the hell is going on…… problem is not resolved yet……. gurrrrrrrrrrrrrrr………………
March 7th, 2008 at 9:46 pm
Cal, we’re joking - making fun of people who whine that they’re losing money, but run their businesses on dirt cheap webhosting.
March 7th, 2008 at 9:46 pm
fail
March 7th, 2008 at 9:47 pm
only this domain is up.
all others r down.
March 7th, 2008 at 9:47 pm
Here you go cry babies:
http://www.vandertramp.com/INTERNETDOWN/
Watch that video and cry some more
March 7th, 2008 at 9:47 pm
Nice. It’s been far longer than 15 minutes now guys. Can we get an update?
March 7th, 2008 at 9:47 pm
Beverly: Who is this? some inside staff of dreamhost? LOL! nice one!
March 7th, 2008 at 9:47 pm
How come this blog is still active???
March 7th, 2008 at 9:47 pm
You know the funny thing? I only had about 6 pages left to code out of over 300 since our last host crashed and we moved to DH. So close to finally being able to have our site back to normal and the new host is down…
twiddling thumbs…..
March 7th, 2008 at 9:47 pm
Anyone can make a typo but there should be safeguards to prevent typos from bringing down an entire NETWORK. I’m a software engineer and to allow a manual typing error to cause this much chaos is incredible incompetence. Don’t blame the guy doing the typing. He just exposed a weakly designed, fly by the seat of your pants system. A similar typo a couple months ago caused massive overbilling, in many cases overdrawing customers bank accounts. Thought maybe you guys would have learned a lesson from that one.
Your programmers (if you have any) should be busy putting checks into your systems to prevent these kind of catastrophic errors.
No wonder you had to cut your rates by half. Another Dream/Nightmare Host mega-blunder. You guys should write a book on how NOT to run a hosting company.
March 7th, 2008 at 9:47 pm
Found this on Google
DreamHost Status » Blog Archive » Dreamhost firewall error
Due to a typing error on our primary router while trying to block a denial of service attack, DreamHost is currently offline. This includes all email, …
-1 - 16 minutes ago - Similar pages
March 7th, 2008 at 9:47 pm
Well, I guess this is one way to “take the night off”!
March 7th, 2008 at 9:48 pm
FOLKS!
Fifteen minutes TO GET TO THE DATACENTER!
Not fifteen minutes to fix the problem!
March 7th, 2008 at 9:48 pm
Just when I was getting motivated to do stuff.
March 7th, 2008 at 9:48 pm
500
March 7th, 2008 at 9:48 pm
There once was an admin at Dreamhost
Who was stuck too late at his post
To fix the firewall
He made a change small
But too late because it was toast
March 7th, 2008 at 9:48 pm
I love how many fucktards are whining about “loosers” who don’t know how to admin servers, or that one server could bring everything down.
Learn what a fucking firewall is, and what it does (hint: they’re not running Windows Firewall here, probably either a Cisco or Juniper). Now you learn how to administer one from the CLI. Too hard for you? SAD, STFU & GTFO.
March 7th, 2008 at 9:49 pm
You know, I yell louder than most when my site goes down and there is no quick response from support with a time table and a solution. When it’s something like this thats quickly reported and every effort is being made to rectify it, I can only chuckle at the perfect folks that whine about the outage. While the down time affects me badly, I have complete faith it will be fixed as soon as they possibly can and not a second later. (Or sooner.)