Major Network Outage.. (resolved)
We just had a drop in our network for about half our servers.. if you can’t access your site or email now, please hold on.. we are looking into it and will update this page as soon as we have more details..
7:15PM PST
We’re still actively trying to diagnose and fix the problem.. it seems to possibly be a physical problem with one of the fiber cables between two of our data centers. We’ll post further updates here as we progress. Thank you very much for your patience and we apologize for the downtime this is causing.
7:20PM PST
Scratch that.. it turned out to be a failure in one of our 10 gigabit network interfaces. We’ve swapped it out for a spare one and thing seem to be working now.. there may be some residual problems where some servers need to be rebooted, however the root cause of the problem is fixed. Please let us know if you are still experiencing any downtime.
Our apologies again..
March 28th, 2007 at 5:36 pm
This was the one day I couldn’t deal with outages like this. I had major clients coming to my site and now I may have lost all hope of building my client base.
March 28th, 2007 at 5:36 pm
Boba server seems to be down and out.
couldn’t ftp, can’t bring up website.
something [major] is going on
March 28th, 2007 at 5:37 pm
ARGHHHHH. Is the entire DNS system down??
I have accounts on 2 servers and both are completely unreachable.
How many times is this going to happen. WTF is wrong with your network!?!?!
March 28th, 2007 at 5:37 pm
well there isn’t a day when dreamhost is not down! we need at least 50% uptime. please you should be able to at least keep the sites up for half the time!
I already moved all my mail elsewhere and will probably move the web also. I will keep my account bc the amount of space and bandwith makes it a good place for off-site backups
March 28th, 2007 at 6:50 pm
all mail and mailing list programs aren’t working on my account.
March 28th, 2007 at 6:51 pm
Dreamhost has been really shaky this year.
And if you have “major clients” coming to your site, move to vps or dedicated..
March 28th, 2007 at 6:52 pm
wow, even dreamhoststatus.com was down (it only showed a one-line message about the mail server)
looks like things are back to normal, thanks DH!
March 28th, 2007 at 6:53 pm
Thanks for fixing this quickly, Dreamhost. A little downtime now and then doesn’t affect the fact that you’re a great company!
March 28th, 2007 at 6:55 pm
Who spilled beer on the server *this* time?
March 28th, 2007 at 6:55 pm
I feel for these guys — we really depend way too much on them. I honestly think that they should lower their offers, and report true SLA.
Thank god we’ve found a Dedicated Host. 200$ a month? Sign me up.
March 28th, 2007 at 6:57 pm
@matt: I think they changed it to a one-liner just to avoid a gazillion comments in the threads. In a situation like this, I probobly would have, too!
March 28th, 2007 at 6:58 pm
“little downtime”? ahgahahah
March 28th, 2007 at 6:58 pm
I’m glad you fixed it quickly and I’m a big supporter, but this is getting a little ridiculous. How often do you guys do, like, preventative maintenance?
March 28th, 2007 at 7:00 pm
I’m just gonna say that this was pretty impressive. The initial problem was reported on dreamhoststatus about an hour ago. 30 minutes later we have another update and 30 more minutes and the problems resolved. Shit happens, and they cleaned it up fairly quickly.
Oh and I second Josh’s comment about “major clients”. If you’re building a business you should shell out for more than shared.
March 28th, 2007 at 7:00 pm
DH, Your services are awesome, but whats with the constant network issues guys? Will these calm down soon?
March 28th, 2007 at 7:03 pm
They can only hope.
March 28th, 2007 at 7:06 pm
Don’t get me wrong… I love DH. But Shared Hosting is paid for in volume. I was ready to shell out 400$ for a DH Dedicated Host, you guys are a _SUPER_ good deal.
Just that dependability comes at a price, and I’m glad you guys are offering services. But your sales material should be updated with real SLA.
March 28th, 2007 at 7:07 pm
One final thing… DH Dedicated Hosts are on a wait list > a month.
Hope to come back to DH.
March 28th, 2007 at 7:08 pm
Ok, I understand, shit happens, but not ONLY shit happens! I started 24/03 and until now there is no 1 day without problems! I use many other free hosting services and I never see anything like this happenning on there… I expected at least this of DH!!!
March 28th, 2007 at 7:12 pm
it’s 23 bucks for the first year and 8 bux a year after that .. you can’t get too angry :]
March 28th, 2007 at 7:15 pm
For people who say they have lost, business, or lots of money, if you lost that much money or that many customers you should invent in dedicated hosting, and not shared hosting. It only makes sense, if your site needs to generate $$$ then you need to make sure its always on. But really if you business is that good, your customers will wait… Trust me.
Be happy Dreamhost communicates with us when they have problems other company’s will not post or admit troubles in a network most hosts don’t, and make you pay for a 30 minute long distance call to support to find out about a system error.
March 28th, 2007 at 7:16 pm
Crazed when you have as many servers as they do its kinda hard to keep up with everything.
First major outtage since ive been with DH and i knew they would have it fixed in no time.
March 28th, 2007 at 7:16 pm
@mike: nice… Now I understand the DH philosophy… Its cheap, don’t bother… Ok, I think thats not a good way to go.. Coke is cheap and I never get disapointted…
March 28th, 2007 at 7:27 pm
@Eu, but coke makes you fat thus causes you to pay for health club fees which are not cheap by any means,
In any case…. I signed up this afternoon and sure it sucked losing connectivity right in the middle of a move (I was stuck in vi server side when the poop hit the fan) but I managed to move 3gb of data and get everything up and running in a matter of 3 hours and that includes the downtime.
March 28th, 2007 at 7:27 pm
@Jason: Perhaps. Realize that I’m as big a fan of Dreamhost as you can get, but if a monthly once-over of things can prevent problems like this, then everyone will be a lot happier.
March 28th, 2007 at 7:28 pm
Just because something doesn’t cost an arm and a leg doesn’t mean that we should expect sub-standard services. Whether I pay $10 a month or $50 a month, I have purchased what’s written on the sales page.
I too rely on my websites for income. And no, I shouldn’t have to fork out hundreds for my needs. All my sites require is dependable uptime! THAT is not too much to ask of any host, shared or not. I don’t host big videos, sucking bandwidth… I don’t go all crazy with huge file uploads and downloads… no load spikes. If I had those things, I would invest in something more robust.
Dreamhost makes promises in their sales materials, yet for a while now, there have been serious issues. Has anyone been offered any kind of compensation for the problems? A free month hosting? $50 on credit? Anything? Seems to me I’d do my best to keep customers happy while they figure out what the hell is going on with their hardware.
Oh - and my sites seem to have returned, but email is still down. Wonderful.
March 28th, 2007 at 7:29 pm
Dreamhost - appreciate the services you guys have, but could you please explain to all of us why uptime has been so difficult to maintain?
If we had an explanation, we could make a choice about whether to stay with you.
Without this, I’m getting pressure from my org to switch.
Please help me convince them things will get better!
March 28th, 2007 at 7:32 pm
@Yvo, I drink coke zero!
March 28th, 2007 at 7:42 pm
Interestingly, with this issue our server was OK, contactable, ssh in and so on, but the database server was not. So since all software we use is DB bases, things didn’t work and got 500 errors.
March 28th, 2007 at 7:44 pm
@eu
Ok, coke zero causes cancer and that isn’t cheap either!
March 28th, 2007 at 7:45 pm
OK - Dreamhost just called back - appreciate that, BUT: the tech support person told me that for the next month they will be moving servers around and that consequently we could continue to have problems.
PLEASE: if this is the case, please inform us ahead of time so we don’t depend on your services during this time.
March 28th, 2007 at 7:48 pm
Jesus, you’re retarded. Be more reliable.
March 28th, 2007 at 8:03 pm
Dreamhost should drop the “strictly business” plan. My personal cheaper server is up way more than the one used for clients. *sigh*
My boss is pissed!
March 28th, 2007 at 8:21 pm
Why should dreamhost make you money if you aren’t making them money? If you are depending on 100% uptime then you can pay for it, otherwise, you can have fun with some downtime when it happens and not complain about losing ‘clients’. Though, really, I don’t know what kind of a retarded client can’t view a website at ANY time of the day and only during a specific hour. Also, why do you not have a local copy if you are, for some reason, showing your client your website in person? Don’t you think it would wow them more if your site was loading with zero lag?
These are some things that people who don’t understand what a massive amount of service they are getting for the low price can think about before they post on here about their made up troubles.
March 28th, 2007 at 8:24 pm
Didn’t even notice the down time and I was checking my site on and off throughout the day.
March 28th, 2007 at 8:46 pm
Thank you DreamHost for dealing with this quickly and efficiently.
Any clues as to why the interface failed though?
March 28th, 2007 at 8:52 pm
Ah!
All of my sites working with fast loading performance.
Thank You Dream Host Team.
March 28th, 2007 at 9:07 pm
I have two sites running. One is on dreamhost and the other is on globat. The yearly payment for shared hosting is comparable between these hosts.
I was pleased with both dreamhost and globat till few months back. Now both suck.
1. Dreamhost - Too many “so called” network issues.
2. Globat - The network is super. No downtime/issues so far. But they charged me $40 for the so called spam shield!!
March 28th, 2007 at 9:10 pm
Thanks for the updates. Your shared hosting is still the best. I have another blog on dreahost and whenever it goes down myphone rings off the hook. I have seen more frequent outages here than when I hosted my web server in my apartment, but overall I’m very pleased with your services (but why no scp or secure ftp?). Cheers
March 28th, 2007 at 9:12 pm
our site is off again, was back for a while there.
March 28th, 2007 at 9:45 pm
Hey, don’t think us dedicated people get better service - in fact we get worse. I wish we never got a dedicated DH server, let alone two. The panel only slightly works with the DH Dedicated solution, and downtime is much higher on those servers than our shared accounts.
March 28th, 2007 at 10:31 pm
@John:
Dreamhost doesn’t make me money, I make my own money.
AND I also make them money - by paying for hosting services and with bringing a multitude of clients along for the joyous ride. (currently almost 300 customers)
I don’t expect 100% uptime, but certainly FAR better than the recent 40% or 50% I’ve been getting.
And as for “retarded clients”, when you deal with REAL business people, they have schedules. You make appointments and drive sometimes hours to meet with them. You show them their sites WORKING, online - all connected to their databases and email and blogs.
And like I said - the price I pay is irrelevant. I purchased a package that included far greater uptime than what we have been getting for quite a while.
Do you really believe things would miraculously be all better if we were charged double what we pay now? I HAVE paid for dedicated service - it’s absolutely no better.
March 28th, 2007 at 10:52 pm
This outage took out half of the servers.
By now, I’m used to seeing the same old exchange of comments:
“I’m losing money!”
“If that’s the case, why not go dedicated?”
However, in this instance, it appears even people who have paid more money for what really ought to be a better service are suffering.
I’ve also noticed that the average server load on the server I’m on (Formosa) is constantly indicating that the CPU is over-burdened.
March 28th, 2007 at 11:36 pm
I think all of us have a reason to be upset over the latest outage. On the other hand, you can’t predict in advance that a network interface will up and fail. Things happen and it was certainly inopportune for me, because I was writing a long update for one of my sites and I had to stop and wait.
I can see where a commerce site would lose money when these things happen, but how do you avoid system breakdowns, involving equipment that isn’t built by your host?
Peace,
Gene
March 29th, 2007 at 12:34 am
My sites are all down. Again. I’m starting to look for an alternative provider. This has been too much this last month.
It is cheap, but it doesn’t deliver. You pay peanuts…
March 29th, 2007 at 12:36 am
My sites are still down, why no update on this?
March 29th, 2007 at 12:46 am
My sites are still very slow.Sometimes,all sites would be down.
I’m losing money now.
March 29th, 2007 at 12:48 am
Uhhh…bad day for this, very bad. Some updated print files never reached their destination and now it is too late - a large job is going to print wrong. Time to start shopping around for a new host.
March 29th, 2007 at 1:02 am
Please, people who’re shouting “if your site’s important, don’t use shared hosting, use dedicated hosting” - stop being assinine. Dedicated hosts at dreamhost are affected by network and power issues at well - the infrastructure is just not great.
As for “what do you expect for $7/month” - well, I expect a better infrastructure than my home DSL modem and residential power; that’s why I shell out ANY money at all, rather than running it off a home server. It’s the whole idea of shared hosting; pool your resources so you can afford decent infrastructure (and some sysadminning).
I haven’t checked if my sites were affected yet (no clients yelling at me, but they only use e-mail anyway) but these apologist arguments are getting really old.
March 29th, 2007 at 1:07 am
My site is off, hurry up please. Is in the morning in Europe and I lose visit for my web.