File Server Upgrades 2008-08-28
Posted (August 28th, 2008 at 9:29 pm PST) by PatrickHome stretch for this week! Starting at 22:30 is fozzie, followed by thog, then gladys if time permits.
Update @01:05
Fozzie is three quarters of the way through the upgrade process. Thog is at 10%. Both should be back up by 02:30. After which, we’ll start on gladys. For those on the frisky cluster, once thog is done, your upgrades will be complete.
Update @01:33
Fozzie is back up. Thog is chugging along and is 70% through the update process.
Update @02:08
Thog is back up and running! Last on the list for tonight is gladys, which will begin around 02:30.
Update @04:55
Gladys is serving files again! Good night!
13 Responses to “File Server Upgrades 2008-08-28”
Janice had an issue with the solaris upgrader. It will be revisited either next week or the week after.
All my sites on postac are down again…
Are you upgrading on production servers? Don’t you use labs to test these upgrades first? Why is it that everything you do on your network always ends up in tears? Haven’t you heard of zero downtime? This is the proverbial straw I’m afraid. We’ve had to deal with these amateur hour maintenance windows for over a year and it’s just not good enough. I guess the saying is true that you get what you pay for in this world.
@Jeff:
Postac is not affected by this round of upgrades and is up and running properly. Please contact support with the errors you are receiving trying to connect.
@Moif:
The upgrades we are performing are to be able to patch against bugs we’ve been hitting with the Sunfire X4500 file servers. I’m not sure what testing you expect us to do. The upgrades are working, the patches are being applied, and the sata errors are being fixed. We could take two weeks to offload all of these over to other machines that are pre-patched, then another two weeks to move them back, and continue the cycle for a year. On the flipside, we can take each one down for roughly two hours in the middle of the night to protect your data.
I am sorry if our methods are causing you tears, but we feel this is the best way forward to resolve the problem that has presented itself.
Hi,
Our site is completely down. Does anyone know what is going on and when full service will be resumed?
Thanks.
@Thinkdust
You are directly affected by these upgrades. Your sites should be back up and fully functional in about an hour. Sorry for the inconvenience!
That’s great, thanks for your help!
My sites have been up and down intermittently since Wednesday afternoon. Is there some other problem going on that’s being addressed? I have a support ticket open but no way to updated it/send another email back to dreamhost, because my email is down. I’m normally very patient with respect to these issues, but it’s getting a little out of hand now.
Our entire site is down. Nothing loads.
“Haven’t you heard of zero downtime?”
Zero downtime is a myth, it exists nowhere, even 99.999 is basically unheard of, mentioned a lot in marketing material but rarely seen in practice.
Honestly I can’t complain about DH, you see, because they are FIXING A PROBLEM, wow, what a thought, temporary loss for longterm gain. There’s more than a few hosts out there who wouldn’t even recognize there IS a problem, much less try to fix it.
And why the hell are you people complaining your site is down here, STILL, what part of “Comments posted here may not be viewed by DreamHost staff at all. This is not a way to contact DreamHost.” is so damned hard to understand?
Captain Obvious, we’re well aware of the “comments may not be viewed by Dreamhost” disclaimer. We can read. But, as you’ll see, Patrick DID respond to someone above, so it can happen. And when email is down and tickets have gone unanswered for two days, I figured, why not, no harm in posting a comment here. If they don’t see it, then oh well.
I recognize that Dreamhost is fixing problems, but based on their announcements on this page, my server(s) shouldn’t be affected as far as I’m able to see. So yeah, I’m kind of wondering what’s up. And I’m extremely forgiving regarding downtime, etc., but I’ve had a persistent problem for days now and I think it’s starting to extend past normal downtime expectations.
Hey, Patrick:
Please answer my support tickets. Thanks.
What about janice it still show it has not been done.