Spunky and Central Move
We’re about to begin to move the spunky cluster and our central machines. We’ll be taking down servers soon, but our status site, of course, will remain up and running.
Again, we’ll try to keep this move quick and painless as we possibly can, and we apologize for the inconvenience.
For more details on this, see the previous post here:
http://www.dreamhoststatus.com/2008/03/14/central-services-and-spunky-cluster-move/
UPDATE — Sat Mar 22 00:12:05 PDT 2008 –
Most central functions are back up. Dreamhost.com and panel.dreamhost.com are working again. We are currently working on webmail. The spunky cluster move is still underway. We’ll post more here later.
UPDATE — Sat Mar 22 02:45:26 PDT 2008 –
Webmail is up! 99% of all central services are functioning now. Spunky is 80% racked, more info on that to follow.
UPDATE — Sat Mar 22 03:40:27 PDT 2008 –
Spunky is coming up. We’re spotchecking and fixing troubled file, web, and mail servers. We’ll sound an all when all is clear… We will likely be firefighting for the next few hours.
UPDATE — Sat Mar 22 06:01:28 PDT 2008 –
Spunky is still coming up, we’re dealing with a couple of machines that didn’t survive the move and some more cleanup stuff. We’ll keep you posted as things progress!
UPDATE — Sat Mar 22 07:04:35 PDT 2008 –
We have noticed a problem with some of our networks. They seemingly did not make the jump to the new router and we are looking into that. Any websites with IP addresses that look like either 64.111.xxx.xxx or 208.113.xxx.xxx are down to the outside world right now. The machines themselves are up so as soon as the networks are working again they’ll begin working immediately.
UPDATE — Sat Mar 22 09:55:54 PDT 2008 –
The networking problem has been resolved with our network provider so as of right now all services should be up and running. If anything is not working, please open a new support message from our web panel.
.
March 22nd, 2008 at 4:26 am
DH needs to kick their DNS in the balls. The change in IPs still hasn’t been made yet.
March 22nd, 2008 at 4:27 am
Can you send out notification emails before you do anything like this?
My sub domain (shop.ourdrl.com) is down for about half an hour already…
March 22nd, 2008 at 4:27 am
I’m not sure what it holds though, since we’re talking about 10 of possibly a few hundred thousand customers, all of which will be affected by something at one point or another.
It’s not even limited to this incident. Any of the things that have gone down, at any time of the day, get the same whiny crap. It’s always a tiny group making a lot of noise and overestimating their numbers.
Most people (I mean most overall–not here) are smart enough to stay up to date and either just accept the problems, or have a plan in place where they’re not affected. The slow crowd that can’t grasp RSS, which is more reliable than email anyway, are an insignificant minority.
March 22nd, 2008 at 4:27 am
My sites still seem to be down along with webmail. I wouldn’t have minded so much in the past except I now run my business from one of the urls.
Hopefully you guys will sort it out real soon. In the past when I’ve filled a support ticket on such issues the response has been swift.
March 22nd, 2008 at 4:29 am
carlo, any sort of scheduled downtime or maintenance by a hosting company is always timed in respect to the company’s local timezone. It’s the only fair thing to do, as most customers are fairly local (within the country); and since most servers house 500+ accounts, it’s impossible for a host to say “Well, carlo here is half a country away, we’ll time his migration accordingly”, because most everyone else on the server are stationed much closer to the host. So again, the only fair thing to do for everyone involved is base their schedule on their local timezone. It’s never recommended to use a hosting company more than a few hours off from your local time for this very reason.
March 22nd, 2008 at 4:31 am
“Not the point. It’s about letting your customer know that the service they are paying you for is about to be temporarily discontinued.
Posting it on a blog one week before is not good enough. Direct email notification is the very least one could have expected.”
This is utterly ridiculous. With shared hosting, occasional downtime *is* what you’re paying for.
Again, if you want no downtime, ever, you have to be paying upwards of a hundred times what you’re paying Dreamhost. Redundant auto-boot backups are extremely expensive.
March 22nd, 2008 at 4:33 am
I’m still waiting - and I REALLY have something to comlaing about because I READ THE FRIGGING BLOGS (I learned not to expect a straight email Long ago!!!) Great thing about DH’s down time is that half the time its NOT PLANNED - We lost 1/2 day *EXPECTING* to loose NONE, having reviewed the status blog. There are some great people in DH, but this perpetual downtime is driving me nuts - enough to give up on them once & for all. I found a services called linode from another post here that looks awesome at only $19 - so reasonable I have have 2 ‘nodes’ in different states with failover I’ll take a while but I’m planning the move tonight!
March 22nd, 2008 at 4:34 am
Hello ? please repair you server… my all site is down, my bussines dead… caman… rapid reapid server plaesee….
and visit www.hoteles-bariloche.com down….
Repair hosting plaseaaaa
March 22nd, 2008 at 4:35 am
Stop fucking with the servers! This is getting a bit ridiculous! My site is down every fucking week!
March 22nd, 2008 at 4:39 am
I’m on daedalus (janky and tweety) and my site has been down for nine hours - anyone else on those??
March 22nd, 2008 at 4:40 am
lyudi pobistree ispravte sistemu. U nas 50 saytov ne rabotayet. Mi sekundami teryayem kliyentov!
PLEASE
March 22nd, 2008 at 4:40 am
One has absolutely nothing to do with the other.
You’re paying for a service you’re not happy with and think they’re the idiots?!
Keep sending those payments. You’ll show them! Maybe throw in an extra $50 on next month’s bill to show them you really mean business.
March 22nd, 2008 at 4:43 am
ftp is wproking, but thr rest is down.
Its time to bring up this fucking servers!
Each day other shit here.
March 22nd, 2008 at 4:46 am
my email (squirrel mail error: “Error connecting to IMAP server: localhost. 111 : Connection refused”) has been out since sometime around 8pm last night…related or not?
March 22nd, 2008 at 4:48 am
Can this please get fixed? Now?
March 22nd, 2008 at 4:48 am
Well still waiting.
Ok some things take time. i really hope when it goes live again everything will work.
You are so lucky you do not have any ISO certification on quality management though.
From what I see here you would have lost it within a day due to customer complains.
March 22nd, 2008 at 4:49 am
I got a new custom theme designed and was about to publicize and here you are…
its down???
what is happening????
March 22nd, 2008 at 4:50 am
I believe so.
March 22nd, 2008 at 4:51 am
FFS hurry up and bring the ‘tiny’ SQL server back from beyond the grave!
March 22nd, 2008 at 4:54 am
aww man, i had some meshes to post, my first ever too
SOLVE IT QUICK, PLEEZE!!!!
March 22nd, 2008 at 4:54 am
Would have been good to get notification this was going to happen, as I just promised to get a clients site upgraded within the next 24 hours, seeing as I can’t access the account I doubt I can make that happen now. A email notification would have been nice so I could have known in advance. Not unhappy but a bit disappointed that I can’t get anything else done tonight until the sites and ftp’s are back up.
We have 2 accounts with Dreamhost on different clusters, for safety sake. So it acts as a backup in case one cluster is down, that way we can continue work without interruption.However both clusters are totally inactive atm, leaving us stranded until it comes back online.
PLEASE send a notification email 12 hours or more in advance next time a major move / upgrade is going to be started, will make things a lot easier on us rather than getting ready to do something and realising everything is down then having to put everything web based on hold,
March 22nd, 2008 at 4:55 am
>:(
March 22nd, 2008 at 4:56 am
@Vinic
once again… I don’t mind the downtime per se. It’s about being DIRECTLY NOTIFIED OF IT so that I can take damage limitation measures.
do I have to say that again or do you get it yet?
March 22nd, 2008 at 4:56 am
Animo boys !!!!
March 22nd, 2008 at 5:00 am
Wooo! I can access webmail! Does this mean you guys are finished?
March 22nd, 2008 at 5:02 am
Look Guys,
Some notice would have been helpful. This is wrong. I have clients using this site to distribute their Red Sox season tickets, and opening day is a few days away. They are paying top $ for this service. Timing could not have been worse. Having a heads up would have at leased given me time to let people know that this weekend they would not have access.
This is not about a BLOG this is about business, and I am losing money when my users cannot access the site as I will have to rectify the situation with my clients, financially.
Because of you I look UNPROFESSIONAL.
March 22nd, 2008 at 5:03 am
What about PAN ? Still cant access my ftp.
March 22nd, 2008 at 5:04 am
Most of my sites are up, but mail seems to be down. Also SSH is down, so I cannot check email directly.
March 22nd, 2008 at 5:05 am
Shawn,
I am not one of those people that regularly read blogs, and I have never used a feed reader. Many people still do not use feedreaders. That is why that emails were necessary in this situation.
Why the hell don’t I use a feedreader? For the same reason I do not follow the news either. I’d rather not spend my days spamming myself with information. Truly important information will always come to me via other means. You could think of it as an information diet. If you can not understand such a thing, try reading ‘Four Hour Work Week’ by Tim Ferris.
March 22nd, 2008 at 5:08 am
James, the fact that you did not receive an email in this situation rather implies that in today’s world, truly important information will NOT always come to you via other means. Welcome to the 21st century, where I’m afraid you’re going to have to get yourself a feed reader, and manage your information diet by only subscribing to important feeds. If you can not understand such a thing, try reading “The World Does Not Revolve Around You And Sometimes YOU Will Have To Do The Work To Keep Yourself Informed”, by me.
March 22nd, 2008 at 5:09 am
1 of my sites is up. LOL
March 22nd, 2008 at 5:10 am
I’m through with DH. The last time this shit happened, by servers weren’t supposed to be affected, and they were - and they were down for 12 hours. This time, they posted a notification of the impending outage here (a week’s notice) but did NOT send an email notice. I get my renewal emails, my billing emails for my domains - I’ve never had a problem getting ANY emails from DH whatsoever, so don’t tell me the problem is on my end. They should have sent MULTIPLE emails beginning three weeks out. If the decision to do this wasn’t made until a week ago, they should have been sending notifications DAILY.
To everyone who posts a snarky comment about bloggers bitching about this: fuck you. Not everyone who’s complaining here are pointless bloggers. Some of people are able to pay their bills with blogging, some have thousands of readers. Other people, like me, use our sites to *gasp* MAKE MONEY - and 12 hours of downtime… oh, wait, make that TWENTY GODDAMN FOUR HOURS of downtime this year is unacceptable.
And don’t tell me that because this is shared hosting that it’s okay or that if I want true consistent downtime that I should be on dedicated hosting. That’s not an excuse. The industry standard is 99.9% uptime - my sites have been down for 24 hours this year (so far), which blows that right out of the damn water - and we’re only three months into the year.
So, good-bye DH. I’ve been a customer for over two years, but the past three months have seen more outage than the previous 23 months combined. I’m moving on.
To everyone blindly supporting DH for this, all you DH fanboys - open your eyes and look at the situation critically for a change and stop toeing the company line.
March 22nd, 2008 at 5:12 am
The latest post 10 minutes ago said a couple of machines didn’t survive the move. Here’s hoping York is not one of them…
March 22nd, 2008 at 5:16 am
Hi Artie,
Where did you see that a couple of machines did not survive the move?
It seems i am missing some posts.
March 22nd, 2008 at 5:17 am
Root for your favorite server…
Cummon Virgil! VIRGIL! VIRGIL! VIRGIL! VIRGIL! VIRGIL! VIRGIL! VIRGIL! VIRGIL!
March 22nd, 2008 at 5:18 am
Daniel: do you know any better hosts? I’m not saying anything, just asking as the most advertised hosting companies are shit, they’re extremely expensive and don’t stop offering you spam through the mail. What host are you going to switch to?
March 22nd, 2008 at 5:21 am
DH suckers, are you working?
My sites are for making money. Do you pay my loss?
March 22nd, 2008 at 5:21 am
I agree: Daniel: Better service?
Oooh! Maybe hostinf with Windows!!! eeew, or better still: GODADDY! Wheeeeeee
March 22nd, 2008 at 5:24 am
dreamhoststatus.com –
UPDATE — Sat Mar 22 06:01:28 PDT 2008 –
Spunky is still coming up, we’re dealing with a couple of machines that didn’t survive the move and some more cleanup stuff. We’ll keep you posted as things progress
March 22nd, 2008 at 5:26 am
I like DreamHost.
I agree with: “With shared hosting, occasional downtime *is* what you’re paying for.” Maybe if you need to be in total control of your server and plan your downtime, you should buy your own web server and host your stuff at home. Of course, when the thing suddenly goes haywire, you’re the one staying up all night pulling your hair out about it — or calling Dell and suddenly wishing you’d sprung for the more expensive service agreement. I’d rather just leave that stuff to Dreamhost. They administer the server 298,763,487 times better than I could.
I don’t make money with the stuff I host on DreamHost, but I would definitely not be buying shared hosting if it were my income on the line.
But my site is back up just as DreamHost had planned. I’m happy. Now I think I’ll go have some cereal with my daughter and find excuses to delay working on our taxes.
March 22nd, 2008 at 5:26 am
Shawn.. I’ve been checking back here over the last few hours just to see if there is anything worth reading.. and there isn’t really… but it really makes me laugh at how your sad little day has almost completely been taken up by quoting and replying to a lot of these worthless posts in the sarcastic manner with which you obviously are getting some kind of kick out of. In my eyes and I’m sure many others reading through this you are no better if not worse than the ignorant posters here.
I really think it should be you that needs to get a life.
I’ll check back in a couple of hours to see if anything has changed.. haha, you make me laugh.
March 22nd, 2008 at 5:28 am
123 test
March 22nd, 2008 at 5:28 am
this is just GREAT!
Posted 9 hours, 33 minutes ago
Severity: High Resolved: No
March 22nd, 2008 at 5:29 am
What a waste of bandwidth and database storage: allowing people to bicker in comments, to exemplifying their stupidity. Go to bed or go and whine about all the wrong doings else where in the world. STOP USING ELECTRICITY if you want to make the world a better place.. or not.
(this comment is equally as useless… delete it when you have the chance, please)
March 22nd, 2008 at 5:29 am
Damn you Lucky. I clicked your name and got Rick-Rolled!
Hilarious, by the way.
March 22nd, 2008 at 5:30 am
I have to say I was a little surprised there wasn’t an email for scheduled downtime, considering that that wouldn’t be soo difficult to do (after all we get newsletters, why not use same system to email about scheduled downtime!)
Shawn, people have a right to vent their frustration if its affecting their business, no need to get personal.
March 22nd, 2008 at 5:33 am
Haha! Little know fact Rick Roll is also hosted by dream host ;D
March 22nd, 2008 at 5:35 am
0833 CT Saturday 22 March. Most of the rest of my sites are accessible now, but cannot maintain my databases via phpMySQL (”grunt:asakura” MySQL Server). Just got a load of updates. Waiting…
f3
March 22nd, 2008 at 5:36 am
You have to tell us before shut down. I have seen too many disappointing and unrespectful behaviours last two months. I will quit soon if you behave like this
March 22nd, 2008 at 5:44 am
I see one of my sites http://www.TheOrangePaper.com accessible now (incl. emails) … waiting for my other sites to follow