Spacey Cluster Move!

Dude, you pay a low price you have to expect downtime, and they are warning you ahead of time.

Just forgot this: How much of the few $ he paid for it per month, he would like back?

Can’t imagine all the work that is gonna take. I hope everything works out ok.

Just got the e-mail. Funny thing, when the pop for the e-mail came, another window was blocking it and all I saw was spacey, so I was actually a bit confused as to what e-mail I had just gotten.

Will we get any better performance in the other data center? Or is this so the cluster can be expanded?

good luck

“I’m transferring my account to Bluehost ASAP.”

Do it already for G**’s sake. Quit talking about it and just do it. Take your negative mojo elsewhere and pollute another host. Thanks.

Appreciate the head’s up DH. 2 Weeks is enough fair warning for even those who have businesses on a shared hosting account to plan for the downtime , notify their customers of maintenance, or make alternate plans.

I use DH as my primary host, but I have failover to a several other hosts that I will not mention here…I don’t get payed anything for advertising their names…plus it has no relevance to DH. I am not a host fanboy, there’s just too many other things to worry about besides promoting someone else’s business for free. They only pick up slack when my DH server is down…which isn’t often for me. Ever since moving to failover to run my business, I have had the luxury of no downtime. I still love DH the most…I have been with them from the beginning and I greatly appreciate their transparent status policy. I wouldn’t trust my business to run on only 1 system, even my own closet-mounted server that people like to boast about being more stable :)…using 1 host for business is inviting downtime.

Remember people, downtime is GUARANTEED no matter who your host is!

“so 10.95 a month for shared hosting is considered a low price?”

Well, 500GB of storage and 5TB of bandwidth to start that grows weekly for a monthly rate of $10.95 month-to-month with no annual pre-pay is pretty darn good in my opinion.

Are you ever gonna use that space? Who knows. Can you get cheaper elsewhere? Sure…and get less space and bandwidth. You can go for 350GB and 3TB bandwidth for $4.95 elsewhere…but do you get all of the options DH has to offer? Probably not. Are they transparent? Nope. Month to month? Nope.

“Take your negative mojo elsewhere and pollute another host. Thanks.”

Why is everyone so punchy here? People are complaining because their experience has not been good so far. I wouldn’t complain about the scheduled downtime, but it seems my site has been down QUITE often lately. So this is just one more thing.

DreamHost is a cool company, they just might not be the best host. I pay the same price at another host for almost (not exactly) the same space and bandwidth. My site has gone down once and it was at 4AM PST. I wouldn’t have even have known it except that they wrote me to let me know and apologize.

DreamHost isn’t the only host that lets you know what is happening. Every host I’ve ever had does that. But they go down the most by far.

I hope the move goes well, I know things like this are a hassle!

@Red - Any pointers on how you hooked up for failover?

Rofl Bluehost sucked. I started with them and yes they were 99.9% uptime but max download from a site was 200k. With dreamhost I can get 2500k!

I predict failure on an epic scale!

- Roach

@Chris - that might be worth asking support directly. It might also be worthy of an update from DH staff here.

* Response from DreamHost regarding email sent to the server:

Any emails sent to domains hosted in the spacey cluster will either
bounce or be deferred on the sending server. Which of these will happen
is something that the sending server will decide when it attempts to
contact the receiving server and finds it can’t, it’s out of our control.

I did not notice much downtime on my account recently. Anyway this is planned and we are being given notice way ahead.

So seems pretty fine to me.

Umm, i dont know if this issue is affecting me but my blog is down. (im in kratos server) …

Its funny, your going to move to a new host, but your “business” is still on a shared server… How is that smart? I have said it once and I will say it again… “If your going to run a business get a dedicated server, and run it your self”. If you want no down time, you do that, and pay:

~ $150.00/month to your ISP
~ $1,000.00 for the server
~ $50.00 for the DNS
~ $10.00/month for the name

aka: $1,810/yr. ($3620/2yrs.)
and an extra $1,050 for everything else.

so that means in 2 years you will have to pay: $4,670
or you can use dreamhost and pay $214 in 2 years.

With that being said, will your site gain you money or set you back with $1,810/yr?

QUIT ACTING LIKE DREAMHOST IS RETARTED THAT IS ONLY ONE SERVER TRY PAYING FOR 500+ AND MANAGING THEM.

This one cluster has over 100 servers, and I don’t know how many clusters they have but I do know the wiki says 500 serves.

@Ryan/Just - That’s why I’m here. I don’t have time to babysit systems, I need to get work done. I do the work, they do the hosting. Once in a while I get bit, but so what - MS pushes fixes on Tuesdays and knock my desktop(s) over “Your system has been rebooted…” “But I was WORKING!!!” (I’ve now turned off the automatic application so I don’t get surprised) - BUT it’s still downtime.

Let’s see… 8 hours is 1/3rd of a day, in turn about 1/90th of a month. All the services I pay DreamHost per month amount to about $12. $12/90 = $0.1333… $0.14 if you round up. I expect to see that credit in my account immediately. Waaah! How else am I going to make 1/10th of my bus fare? How much is your monthly steven? Are you saving up for candy?

My whole presidential campaign is riding on Dreamhost’s shared server uptime.

MySql on Spacey is gone… down..

Error: Can’t create a new thread (errno 11); if you are not out of available memory, you can consult the manual for a possible OS-dependent bug

si el sitio sigue cortado mi jefe me va a cortar las pelotas por favor solucionen esto !!

If the site continues down, my boss is going to cut my balls please fix this!

I don’t think the issue is that they are having scheduled downtime. I don’t think it’s an issue of cost or reliability.

I think this issue is competence. Let’s just think about the number of ways this particular problem could be perhaps better handled:

1) Don’t put clusters in a full environment. If you were moving all your servers because your data center was shutting down, or because you were getting your own data center, maybe those are moves that “have to be done” and “could not be foreseen”. But a single cluster needing to be moved from one environment to another because of “over-crowding” of your own equipment sounds like someone didn’t run their numbers correctly before it was put in.

2) Dreamhost should be taking every consideration to try to limit the damage of the downtime to it’s customers. Ways to help limit the harm of the downtime:

- Setup a backup MX server (or leave the one you have) running at the old location to have it trap the email until you get to the new data center. Dreamhost should NOT have to rely on other people’s server configurations to insure deliverability. Now, given this won’t solve the problem for the people who maintain their own DNS enteries, it will for all of the customers who are using the DH nameservers. The only trouble to make it happen: a backup mail server with a slightly different configuration from your normal ones.

- Put up a “this site is moving page” on all of the affected IP addresses. I’m assuming that the IP’s will not be moving with cluster to the new location, but even if they are, this is still doable. This will make sure that customers who are running into the pages are getting error messages that don’t make any sense to the majority of the public.

- Assuming the IP addresses again aren’t moving, it would be a very good idea to switch the DNS records before “hitting the road” than after… Fewer delays in getting the site back up (You may already be doing this, but your exact plan of attack isn’t overly public).

Hopefully someone reads this and it helps out.

Someone asked how to implement failover. Here is the “jist” without mentioning companies’ names (they don’t pay me to advertise / not a fanboy - just want to keep my business running proactively).
This is not an easy task, it requires planning and learning. Don’t hold me responsible if you mess anything up…this requires knowledge of how to control your DNS settings. These are not step-by-step instructions, they are the idea of what you should be doing to set it up.
No host will recommend this method because it gives the perception their systems are not reliable, plus it pushes business to the competition.
Also, I will not support any questions if you get stuck here, I just don’t have the time. If you get stuck, research it, Google is very valuable as a research tool.
Use common sense if you implement this. Plan it out first before you make the first settings change or you could be sorry. get the right hosting accounts for you…price, services, make sure they have all of the same programs you run at dreamhost or you may have to manually install…that said, make sure your alternate hosts allow you to install if you need to! If you are just doing straight, non-interactivewebsite, it’ll be easy. mail is not failed over unless you fully research this and plan for it. I’d recommend hosting mail wherever you feel is the most reliable…your own server, another host…whatever you decide. Being a business, I host my own with several layers of backup/redundancy in place. If my server goes down, I can still get new mail and send mail due to my failover plan as well as keep all my existing email accessible.

————

Instead of paying around $100/mo or more for dedicated hosting, failover to several shared hosts is cheaper and should work just fine. (just make sure they are on different ISP backbones if you can in case they have nothing in place to protect against an ISP outage).

-You must setup multiple hosting accounts. The failover company I use allows up to 5 failover hosts…it’s your choice how much failover you want.
-You must setup an account at a DNS failover company. Use Google to find one that suits you. You’re looking at approx $15/yr to $60/yr depending on how many hosted domains you want to control.
-You must setup your domains and all of your DNS settings at the DNS failover company prior to switching over so you don’t have downtime when you switch.
-Turn on the failover option for each domain (dns name) you want failover for and point to the IP of your new hosted account. Setup the failover IP address order to however many failover hosts you have purchased (my failover allows up to 5 hosts). Do this per domain you are controlling.
-You must make sure all of your new hosting accounts mirror your primary, so that when failover occurs, your sites are the same. This means backing up your databases on a regular basis and transferring your databases to the new hosts…either do it manually or figure out an automated way to do this. If you run forums or whatever requires a database, you will want the most up to date versions all the time. Basically, when your site goes down, you will be notified via email addresses you program in…I have text messages sent to me also. Unless you host your mail on a different server, don’t use the email address from the domain you are failing over…setup a free account somewhere or use an alternate email you already have. When you get the alert, upload the latest version of your database you have. I won’t get into that anymore, research how if you do not know how.
-Make sure all htaccess passwords are set and the same, you don’t want to give unfettered access to protected areas of your primary site, and you want to keep passwords and everything working in case you have users who use passwords.
-***To reiterate, use common sense…whatever modifications you have made on your original site must be reproduced on your alternate hosts when you upload your sites.
-Upload all site contents to each alternate host. Make sure everything is setup and in place prior to changing your nameservers over.
-Then, when all your DNS settings are setup and all your sites are uploaded and working as they should with the same data, you must pass DNS control over your domain name to this DNS failover company (you will have full control over everything from there). If your DNS is hosted at Dreamhost, you must pass DNS control from there…same with Network Solutions or whoever you have (meaning, point your nameservers to the DNS Failover company’s nameservers).
-Wait for DNS propagation…1-72 hours for complete propagation across the internet.
-Use dnsstuff.com or dnsgoodies.com…or whoever you like for checking on your DNS settings taking effect.
-Probably good idea to wait at least 24 hours befor testing failover.
-You can test failover by changing the monitored website to an html page that does not exist…wait for the alert and check the site.
-Failover scans cycle every few minutes, so you could have a couple of minutes downtime until the next scan cycle completes and detects your outage. Failover will automatically route traffic to the new host you setup…and as long as all your DNS settings are in place and databases are up to date, you should experience downtime only during the failover time period. I notice a maximum of about 6 minutes and a minimum of 2 mins downtime during failover.

-Again, use common sense, setup all your failover options to the way you want it to work…they have automatic failover back to original site or not, so you can copy your latest database back to the original site prior to switching back.

-reasearch, plan, and research…then plan some more prior to implementing.

Good luck!

Sweet post Red. Exactly what I was looking for and the right level of detail. If you can’t figure it out from that post you PROBABLY shouldn’t be trying it.

Thanks!

@omg,

You are right on the money. There are to many options out there to do a step by step, but you need a certain level of knowledge on how to operate your web site fully before attempting.

I love it, works great. I use 2 alternate hosts myself…the changes would be astronimical that all 3 are down simultaneously…unless they use the same backbone. Choose wisely!

Also, you can add to the mix by setting up several free web site monitoring services that alert you when an outage is detected…that way if minutes count, one of the several monitors you use will have detected the outage before the failover kicks in and you can get to work on uploading your current database.

good practice is to always make changes to all hosting accounts at the same time when you update…so you don’t get sloppy.

@Hilary Clinton
In that case, Dreamhost…please destroy the shared server she is on!

btw, That isn’t really Hilary if you haven’t figured out already. She doesn’t even have hosting though Dreamhost.

@previous poster: If you earn $10/hour to have your site online, why are you being cheap and using a $10/month shared hosting plan?

I use DH for stuff that is not vital to my business, like a personal photo album. For my business, we pay around $25k per year for colocation in a Tier-1 datacenter (Equinix).

Eight hours is a bit long -hopefully it gets done quicker and does not take longer. And as the poster above says - something should have been done to ensure everything kept working until the actual switch over.

this is scary. i wouldve thought there’d have been mirrors at least somewhere for times like these let alone real emergencies??
why do i have this bad bad bad bad feeling that our sites are going to be down for days and days? i hope they prove me wrong but i gotta bad feeling about this one and it getting set back up properly and fast. fingers crossed
please dreamhost try to be as fast as possible,which i know you will. good luck

Anyone having issues with servers -not- part of that cluster at the moment? My mysql is on “courage” (which isn’t listed) and unreachable. Not sure if it’s related to this, but the timing makes me think it’s likely…

Yeap, that’s when mine when out too.

I’ve been hear a long time through the many hiccups that growth has caused to DH and its crew and I have one thing to say.

GET OVER IT YOU WERE WARNED WEEKS IN ADVANCE

I see the severity of this outage as being medium in that it is a simple hardware move, not an outage or a failure where no one knows what happened or what was affected. Power down cluster of servers , move equipment 40+ Miles, reconnect everything, power on, seems fairly simple to me and well within the skills of our gracious hosts.

Keep up the good work guys.

My site is back. Thanks.

Thanksss..

For all those people who want to be compensated, the downtime works out to be about 8.5 cents. 10 cents if you round up! Be sure to submit all your requests to support to get your account credited! morons.