30 min Blingy Downtime Tonight
We may almost be in the clear with the problems we’ve been having with Blingy.
We now have a critical patch from our file server vendor which they believe will fix the crashing problems we’ve had. It takes 20 minutes to install, and unfortunately requires a reboot. We’ve decided to schedule this upgrade for 11:30pm PDT tonight, Sunday April 6th, 2008. That is pretty much the lowest usage time across all our users in a week, and we’ve decided we can’t wait another week before applying this patch in order to give more notice.
Our apologies for the very short notice! We received notice that the patch was ready this morning. Hopefully by midnight (PDT) tonight, Blingy will finally be happy again…
UPDATE 12:15AM PDT:
Still haven’t started the upgrade yet.. it took them a little longer to get the patch than expected, however, we should have it in the next 2 minutes, and then the upgrade will begin, probably around 12:30 now, and be done before 1am Monday morning PDT. Hang in there!
UPDATE 12:54AM PDT:
A few more bumps! Still no changes.. and now we’re afraid we’ll have to push off the upgrade to about 1:45AM (9:45AM GMT) it looks like.. hopefully all finished by 2:15AM PDT (10:15AM GMT), our apologies to blingy customers in Europe. 
UPDATE 02:10AM PDT:
All done! The actual reboot only took about 3 minutes, and then there were some residual effects from the reboot on various web and mail servers for up to ten minutes after that. But things seem settled down now and the upgrade “took”. We all hope it did the trick. BIG TIME.
.
April 6th, 2008 at 1:55 pm
What? when
April 6th, 2008 at 2:29 pm
God I hope so.
April 6th, 2008 at 2:36 pm
ROFL
April 6th, 2008 at 2:42 pm
I’m dying to know who that server vendor is …. C;mon, Josh! Tell us!
April 6th, 2008 at 2:44 pm
What???? You mean I can’t access my site from 2:30-3:00????? That’s when, like, ALL of my visitors come! I needed more notice to tell them to come at 3:01!!!!!!!!!!!!!!!!!!!!!!!! You fools!!!!!!!11!
lol.
April 6th, 2008 at 3:11 pm
So if I’m past my initial 97 days and want to cancel my account due to these problems with Blingy, can I get a refund on my unused time rather than simply a Dreamhost credit?
April 6th, 2008 at 3:12 pm
I’ve had like 17 days of blingy downtime… so, is this 30 min notice like a mean joke or something?
You guys are cruel…
April 6th, 2008 at 3:13 pm
ROFL.
April 6th, 2008 at 3:26 pm
ZOMGWTFBBQKITTENS 30 MINUTES OF DOWNTIME ON BLINGY THE WORLD IS GONNA END!!!!1
Seriously, why do they even bother posting this crap; if Blingy functioned for an entire half hour, now that would be a major event. This is exactly why the RSS feed is useless, from 10 posts a week you might get one scrap of real information per year and you’ll have long since given up and started ignoring it by then.
April 6th, 2008 at 3:40 pm
11:30pm PDT or PST?
April 6th, 2008 at 3:48 pm
LMAO.
April 6th, 2008 at 4:34 pm
I hope Longlan is not adversely affected by this outage.
I love you Longlan.
April 6th, 2008 at 4:36 pm
Thank goodness, I’ll be glad to put this behind me, and I’m sure the Dreamhost guys will as well. It was going on three weeks there, but if this works I’ll be glad I didn’t cash out on that refund. Assuming this stops all of the major problems that have been happening with this cluster for months.
Come to think of it, the site hasn’t went “Down” on me today, it loaded a slowly earlier, but that was about it.
I’m going to cross my fingers.
April 6th, 2008 at 5:11 pm
Hopefully the patch works.
April 6th, 2008 at 5:24 pm
In honor of this event, and to foster a festive atmosphere while the world waits anxiously for Blingy’s resurrection, RightousIndignation and http://fuckdreamhost.com will begin hosting a “Raising Blingy From The Dead” marathon celebration beginning now and lasting through the night!
Come party with other believers, skeptics, heretics,lunatics, counts, viscounts, no accounts, poor dumb bastards that can’t count, crusaders, persuaders, dissuaders, ranters, lovers, haters, sycophants, fanboys, twits, toads, tools, trolls, saints, sinners, gays, straights, lesbians, runts, cunts, fuckers, fuckees, fuckups, the fucked over, the fucked up, lions, and tigers, and bears, and goats, and other denizens of the internet too numerous to describe here .. Good Golly Miss Molly everyone will be there!
Tonight! http://fuckdreamhost.com - “Be there or be square!”
April 6th, 2008 at 6:20 pm
Hope the patch goes well. I want the downtime to be done already.
April 6th, 2008 at 6:24 pm
Good Luck! I’ve seen really marked improvement over the past four days so I’m hoping this is the last fix to 100%
April 6th, 2008 at 6:32 pm
LMAO.
April 6th, 2008 at 7:15 pm
I really hope this does it. For the hostees and the hoster. This has been unpleasant for everyone.
Please provide a detailed status - for example, does this mean:
- The Blingy rebuild is complete?
- The data is moved off that is going to be moved off?
- We should open new tickets if we have further problems like a slow site, etc.
Also Dreamhost, a few suggestions which I hope you take to heart.
1) Post updates twice a day for this sort of outage. Even if nothing has changed. Being in the dark is worse than knowing the status has not changed. At least we would know you are keeping us informed.
2) Rework the Status Via RSS feature - It shows resolved for everything. Pretty useless and it degrades confidence when the RSS status updates do not even agree with one another (RSS headlines vs Status page)
3) For at least a day - I was getting a 404 on the status link when I clicked on it from the unresolved part of the status site.
4) If there’s a critical problem, posting the time without a date is pretty silly - see the Update 3:00am entry on the Blingy status.
5) If there is a critical problem don’t let it scroll down the status page when lesser problems happen after it. Critical problems should be at the top. Now I know to check the Unresolved link on the right (which see number 3 above) to see unresolved issues.
Thanks
April 6th, 2008 at 7:25 pm
ROFL.
You need to find a hobby if you feel the need to see “Nothing has changed” as an update.
April 6th, 2008 at 8:01 pm
Get the shirt! “Blingy Spring Break 2008″ 100% cotton, Silkscreen.
I simply must move these shirts! We are completely out of space for any more shirts! (I knew we were running out of space, but it kinda got away from me. Space is very tight; we are now in the position where we have to sell TWO shirts for every new shirt we print.) These must ship out quickly! Get yours today!!
http://www.wholewheat.com/bm/T-Shirts/
Thanks,
Hedge
April 6th, 2008 at 9:14 pm
How sad, here I was hoping for a rickroll…
April 6th, 2008 at 10:18 pm
I sure hope so. Good luck to everyone. I’m keeping my fingers crossed.
April 6th, 2008 at 10:34 pm
Hope this stops all my headeches!!!
April 6th, 2008 at 11:00 pm
longlan lover!
I do not know what you said.
However, I feel that I visit the website has basically returned to normal!
Who are you?
Who are you?
Who are you?
Who are you?
Who are you?
Who are you?
Who are you?
Who are you?
Who are you?Who are you?Who are you?
April 6th, 2008 at 11:09 pm
11:30pm PDT tonight?
11:30pm PDT tonight?
OMG!
Blingy will finally be happy again
Blingy will finally be happy again
Blingy will finally be happy again
Blingy will finally be happy again
April 6th, 2008 at 11:13 pm
GoodLuck!!!!! DH
April 6th, 2008 at 11:25 pm
GoodLuck!!!!! DH
GoodLuck!!!!! DH
GoodLuck!!!!! DH
April 6th, 2008 at 11:44 pm
GoodLuck!!!!! DH and GoogLuck!!!!! Langlan!
April 7th, 2008 at 12:09 am
My site’s up again.
Even slower than before. Not even funny…
April 7th, 2008 at 12:11 am
Installation has already been completed?
DH!!!
April 7th, 2008 at 12:12 am
Who are you?!LongLan lover?!
April 7th, 2008 at 12:14 am
Installation has already been completed?
April 7th, 2008 at 12:16 am
ROFL.onglan
April 7th, 2008 at 12:25 am
… … …
April 7th, 2008 at 12:46 am
why wait?
http://www.dreamhoststatus.com/2008/03/27/filer-problems-with-blingy-cluster/comment-page-27/#comment-156803
1 COPY/PASTA
2 hit Enter
3 ????
4 PROFIT!!
April 7th, 2008 at 1:08 am
L
M
A
O
April 7th, 2008 at 1:18 am
I come from China.It is afternoon, then how should I do? DH?!
I hope you can accelerate.
April 7th, 2008 at 1:46 am
I’d like DH to host my site in the exact same cluster where dreamhoststatus.com is located.
It handles the load pretty well
April 7th, 2008 at 2:11 am
[quote]1) Post updates twice a day for this sort of outage. Even if nothing has changed. Being in the dark is worse than knowing the status has not changed. At least we would know you are keeping us informed.[/quote]
@danonymous: Nothing has changed
April 7th, 2008 at 2:12 am
They don’t host the status site here, so it’s up when other stuff is down. The Dreamhost status site is hosted at astonishinghost.com on their 2nd largest hosting plan.
April 7th, 2008 at 2:35 am
@39 - Bob
LOL. Seconded!
April 7th, 2008 at 3:44 am
DH,
Your screwing us here in Asia. It’s 6:45 pm on 4/7, and my sites are down completely. Hey man, haven’t you heard? The China market is where it’s at. Why are you doing us this way?
April 7th, 2008 at 3:45 am
OK, they’re back up now, but a static html page took a minute to load, and no it isn’t my ISP. Everything else on the web is normal. 加油！
April 7th, 2008 at 5:22 am
Well, I have to say this didn’t seem to work either Dreamhost. Site still loading slow as hell.
April 7th, 2008 at 8:23 am
whats the difference between this category : “30 min Blingy Downtime Tonight” being resolved ….. and “Filer problems with blingy cluster” still not resolved?
April 7th, 2008 at 10:03 am
new user: the downtime is behind us and the patch is installed… that’s why this one is resolved.
Presumably they aren’t yet -confident- that the overall troubles with Blingy are solved - which is why the other issue isn’t marked as resolved…
My site seems much more responsive today… That’s good news, I was starting to get tired of posting links to my site and having to tell people “but it’s not working too well right now”… Hopefully it stays well-behaved…
April 7th, 2008 at 2:30 pm
Soo, ~3 weeks of problems with Blingy cluster and all it took was for DH to give up, call their vendor for ideas and spend 20min installing a patch? …. aside from my year for $20, remind me again why I stay here?
April 7th, 2008 at 3:35 pm
LOLZ
April 7th, 2008 at 5:21 pm
“aside from my year for $20, remind me again why I stay here?”
Not sure, to me it just looks like you’re really cheap.