Power Outage Update
Power has been restored to the building and all servers are back up and running. There currently is an issue with a blade in one of the core routers and it is being replaced, so the servers are still unreachable until this issue is corrected. We are working to get service restored as soon as possible. Our apologies for the inconvenience.
- UPDATE 04:51 PST -
The supervisor in one of our core routers went kaput when the power went off. It was replaced shortly after the original posting. Further problems arose with corrupt configurations left over from the old supervisor. These have been restored and we have re-established connection to seven out of our eight uplinks. We are currently working on the eighth uplink and everything is coming back to normal operation now. We are close to being 100% and thank everyone for their patience. As always, check this space for updates.
- UPDATE 11:21 PST -
The router is up and healthy, and the majority of our network is back up and running. However, a few of our servers are having issues because their file servers are not talking to the rest of the network correctly. There is a link down from one part of the datacenter back to the routers and that is causing the file server issues. If you are on a server whose name is a beverage, this will effect you. We are working very hard to resolve this issue, and we apologize for the inconvenience.
.
February 25th, 2007 at 5:06 am
I don’t think it’s the power outage that’s the issue. Not even the fact I don’t get e-mails. It’s the fact it’s taken 2 hours to repair a blade server.
No, I don’t think it’s deliberate, and I know they’re working their socks off, but I do think it’s poor. There should be a backup core router. It’s nothing to do with the price, it’s to do with common sense.
February 25th, 2007 at 5:07 am
I swapped over to dreamhost 2 months ago and im afraid to say I’ve already decided I will not be renewing my hosting after this year is over.
I have experience more downtime, problems (mail outages, system wide downtime, panel downtime, etc) over the past 2 months that I’ve been with DH then what I’ve ever in the 3 years I was with my previous host. (I had to leave them because my site was going and the plans they offered were not good enough)
February 25th, 2007 at 5:07 am
Why does that happen every time I need to update my site
Hurry up guys.
February 25th, 2007 at 5:07 am
I can totally understand that electricity failures happen, but I have to say a simple email to customers as soon as you found out would have been very helpful.
I run a few forums and can just imagine the hassle I am going to get explaining this, when I could have posted up a notice about it and let people know what was going to happen.
thnx
February 25th, 2007 at 5:08 am
I love dreamhost’s hosting plans, and I’m okay with experiencing some down time, although e-mail notification would’ve been nice, it wasn’t expected. I’m staying with dreamhost, despite any problems, I love having so much space and bandwidth for such a small amount of dosh! ^_^
February 25th, 2007 at 5:10 am
Quote: “omg”
”
ok, time to find a new host
i’m losing a fuckload of money, thank youz dreamhost ”
If you’ve lost so much money in a matter of 4 hours you must be fucking rich from all the other days that dreamhost have been up.
February 25th, 2007 at 5:10 am
You have to pay more for common sense
February 25th, 2007 at 5:11 am
Thanks for the update and working to get everyone back online!
February 25th, 2007 at 5:12 am
HistoriasDelMotor@ Muy buenas tío, me alegro encontrar a un español aquí. Llevo con ellos un poco más de un año, y la verdad que no había tenido un problema con ellos hasta el día de hoy, así que no te desanimes que esto no es la tónica de todas las semanas.
La verdad que para los que tienen contratado un servidor dedicado es una gran putada, tenían que haber pensado en tener servidores descentralizados.
Un saludo.
February 25th, 2007 at 5:13 am
It’s been hours now and it’s getting rather annoying. Is there any chance it is going to be back up anytime soon?
February 25th, 2007 at 5:13 am
@{all DH Users} From an Architects Point of View, you could do *Hedging*. You can, at least what is concerning websites, split the risk up like this:
book a 5MB (or even 1MB), 99,999999% availability plan at some AKAMAI like hoster, which has redundancy across continents. You need only to host the FIRST Page of your Site on it, so you don’t need much space and it won’t cost the world. THis first page will redirect then to the DH Site where the rest is held, or in a case like today give your customers / site visitors a page which explains the downtime, so that the first page of your site AKA your public face is always UP. For mail and DNS there are hedging solutions too. Anyway, this is a small subchapter out of Redundancy 101, as I teach my students.
@{all who think it is night and it won’t bother anyone} dont be so US-centric, many of us clients are in Europe or in ASIA WHERE IT IS ACTUALLY DAY
Cheers,
AlexX
February 25th, 2007 at 5:13 am
The old saying, you get what you pay for applies here. This being said, WTF. Seriously. How can you guys not have redundant backups or whatever? I mean if power to your building goes down, you SHOULD have back ups. Other hosts do.
I have had my site off line or unreachable, or crazy slow more than 20 times in February. I’ve done trace routes, I am on a 25mbs connection so no, it’s not on this end. Poor service *does* catch up with a company.
Either fix the problem, get a new business model or at least be honest and change the name of the company to sometimeshost
February 25th, 2007 at 5:15 am
Still down. Lunchtime here (Reykjavík) now. I HATE these downtimes. Could someone from DH please update the blog with an expected time the websites will be up and running again? This pisses me off, really.
February 25th, 2007 at 5:15 am
Keep up the good work chaps. Shame you didn’t send an email out though as I only moved by core services back last night and its been down all morning.
February 25th, 2007 at 5:15 am
This is Murphy’s Law of Running a Web Hosting Company. No matter what you do, something else always happens making downtime greater.
Then again, it happens even in non-web hosting companies. Happened to me during Thanksgiving. And I wondered why dinner wasn’t ready until 8 that night.
Keep up the terrific work.
February 25th, 2007 at 5:16 am
time is ticking…
February 25th, 2007 at 5:16 am
For the random references to the 12 year olds - I’m 16 (today infact) and this is the first trouble I have had with Dreamhost. I’ve had many web hosts over the years and although Dreamhost aren’t the best for uptime (though this is the first downtime I have experienced) they are by far the best for communication and ethics. I was once with Yahoo’s hosting service, which was _so_ poor I left them within a week, similarly, I was only with GoDaddy for a month. Why? Because they are _too_ corporate, they don’t listen to the individual or care for them - like Dreamhost does.
Dreamhost is good value. If you weigh up what’s important to you, they are good value. And for the price, are you expecting the world? Its a bit like RAM, you can spend £100s on RAM and get something excellent, you can also spend £20 or £30. Do you expect the £20 - £30 RAM to run as well as the £100s RAM?
You have to compromise - if you want great value, you should expect to compromise on something - in Dreamhost’s case, its uptime - but I’d certainly rather compromise that than their immense customer service.
And really, if you do have a critical website, what are doing using shared hosting anyway and paying $10 - $15 /month?
Thanks Dreamhost.
February 25th, 2007 at 5:17 am
Quarter past midday on Sunday here in the UK. Everything still down. No sites, no email, nothing.
Heh! Likewise - I’d planned to upgrade to WP 2.1.1. during today…
February 25th, 2007 at 5:19 am
Some people seem to think their unknown websites are important o.O I can guarantee my site gets more traffic and I’m not too worried about a bit of downtime.
Internet. Serious Business.
February 25th, 2007 at 5:19 am
down down down…
for many hours
February 25th, 2007 at 5:19 am
yeh this is not acceptable…its just too long to be without email for me..
damn, time to start looking for a new host
February 25th, 2007 at 5:21 am
@cordelia, HERE in LONDON, UK, we’re all a bit anxious, relax dear - we all have projects on our heads. Btw. I keep a just-in-case-dh-goes-down backup offline, don’t you? You are a pro now, are you.
February 25th, 2007 at 5:21 am
Email notification of this would have been nice???
February 25th, 2007 at 5:21 am
Why this is taking so long to bring it up? There are many angry users out there!! Did you guys read the disaster recovery chapter on System Administration book yet? Redundancy, fail over or High Availablity chapters? This is highly unacceptable!
February 25th, 2007 at 5:23 am
Didn’t want to go to bed until the site went back up. Time to pour more coffee….
February 25th, 2007 at 5:23 am
does this mean deam host downtime for the year is already like 99.9429223 %
looks like my $10 is looking safe lol
February 25th, 2007 at 5:23 am
I guess that is what you should expect from such low prices. Not whining here, its good value for money, some companies offer a reliable service and charge accordingly, others charge low prices and provide a service accordingly!
February 25th, 2007 at 5:23 am
DALE CHE!!!!!!
Prendé el router y dejá de rascarte que ya estoy re podrido…. hace 4 horas que estoy dándole al F5………………..
February 25th, 2007 at 5:24 am
Hi, my web site is still down:
http://basket.nunolevy.org/
Hurry up please
February 25th, 2007 at 5:24 am
Please hurry up, we all need our servers up and running, and this time it is very long
Please change that router as fast as you can!!!!!!!!!!!!!!!!!!
February 25th, 2007 at 5:25 am
How about an update on the works guys? Takes quite a while now, is’nt it!
February 25th, 2007 at 5:25 am
For being 16, Hamish Robertson sure has more sense than most of the other people freaking out here! Seriously, who would host an important site at a place like this? I heard the complaints before I cam here, but I didn’t care. I wanted a host for my personal sites, that’s it. If it was big business, I wouldn’t have come here. If I had a lot of money, I might have chose something different.
But this is the first time that I have ever noticed anything wrong with my site, and if I would have been asleep when this happened (which I should have been), I wouldn’t have even noticed anything when I woke up. The point is… exactly what everyone else with sense has been saying.
February 25th, 2007 at 5:26 am
Looks like dreamhost.com is back up so I am thinking sites will be back up momentarily.
February 25th, 2007 at 5:26 am
Redondo is down. Panel.dreamhost.com is down. Any info?
February 25th, 2007 at 5:26 am
Mate do you need to make the router? it doesnt take 3 hours to replace a blade. People are killing themselves over this outage.
February 25th, 2007 at 5:27 am
Disgruntled .. ns1.mediatemple.net
February 25th, 2007 at 5:27 am
opinions are like assholes;everyone has 1.
178 comments and only a handful are valid
@ TOTALY DISGUSTED:
i agree with your passion as far as them not contacting all their clients through a simple email.ialso agree that me as a client should not have to endure visiting this part of dreamhosts site to know whats going on with the service i have purchased.it is definetly rediculous bymore than 110%.i am a client and why should i have to go through this childish board which is comparable to a 15 year old aim chatroom.(minus the pornbots)
to the others:
not everyone pays 10 dollars for service.there are clients who pay for dedicated service and they have everyreason to be pissed.a person who is aying for a dedicated box needs his or her site up 24/7. yes no one can say they can keep your site up 24/7,but since this was planned and let me say it again SINCE THIS DOWNTIME WAS PLANNED dreamhost had ample time to notify all its clients through a simple email and not force its clients to have to go through this childish board.im lucky enough that i caught this as soon as it happened because i use a web monitoring service so i didnt suffer as much and luckily dns updates quickly for me. a serious note should any staff monitor this board ( which i highly doubt because it seams they treat all of their clients as if were 10 year olds ) next time just email your clients.
February 25th, 2007 at 5:27 am
I just want to say about the comments I see in here about how “one shouldn’t complain since their prices are so low”.
That type of argument is just bad. No matter the price, this type of outage is unacceptable. It’s year 2007 and Dreamhost can’t even send a mail about an outage like this? Redundant systems, heard of it? Anyone?
This is bad for the company and for its otherwise very good reputation. I for one will look elsewhere now.
February 25th, 2007 at 5:28 am
@Keith - They’ve had it posted on here forvdays atleast about the power outage.
February 25th, 2007 at 5:28 am
I hope this is resolved soon.
February 25th, 2007 at 5:29 am
Alcoholic1: I guess you won your bet. Double or nothing, I say $20 it’s up within an hour.
February 25th, 2007 at 5:29 am
# Keith Says:
February 25th, 2007 at 5:21 am
Email notification of this would have been nice???
————-
Yeah, email is down so let’s notify you all on your email you can’t check at this time, great idea
like said before, if you make a living with your site, don’t use shared hosting for it but go to dedicated of colo.
I don’t care for the downtime because dreamhost is way to cheap to be annoyed about that.
keep up the good (cheap) work and take your time because hurry leads to mistakes
just my 2 cents
February 25th, 2007 at 5:30 am
Hi, my name is Cindy. May I post some nude pics?:)
February 25th, 2007 at 5:31 am
Even if my sites stay down all day, the value for money at DH is still about 10 times better than with any european host I’ve come accros.
Shit happens at almost every host with price ranges like dreamhost.
At least DH is honest about what is going on. (You wouldn’t believe how many times I’ve heard: “It might be your computer sir” with other hosts.
That being said, I hope for an update in the post too about the expected further downtime.
February 25th, 2007 at 5:31 am
@ Hamish Robertson: Happy Birthday
@ Team Dreamhost: I know that you’re trying your best to get it working again so thanks.
I have nothing to say to people who whine….
February 25th, 2007 at 5:32 am
This is very not cool. I had a live event sceduled for this moringing and i cant do it if my site is down! very not cool!!!
February 25th, 2007 at 5:32 am
Just want the Dreamhost folks to know, I feel your pain. I am watching the sun come up from an all-nighter I still don’t see an end to. Damn Oracle upgrades that don’t go as planned.
The comments from some of my fellow users are quite entertaining.
Thanks Dreamhost folk for the hard work.
February 25th, 2007 at 5:33 am
The DreamHost service is really bad. I’m considering to move my sites to a more professional provider.
February 25th, 2007 at 5:34 am
Try dealing with a datacenters power…it never seems to work out. I don’t dout that their servers are up and ready, they are just having a router issue after the power outage, give them a break.
February 25th, 2007 at 5:35 am
Control panel is back.