File Server Issue Causing a Few Web Service Interruptions
We have a file server that is filling up at an extraordinary rate and we are working currently to find the cause. During the search some web services may be interrupted. We are working to get this issue resolved as soon as possible.
- UPDATE -
The issue has been resolved and any servers that might have had web services temporarily turned off are being put back into action.
- Post Mortem -
This is what actually happened, to answer the various speculations below. Apache2.0 fixed the “bug” which meant that log files could not exceed 2.0G in size. We recently have been doing a lot of roll outs on Apache2.0, versus our previous default of Apache1.3. This meant, if a log file was growing at an enormous rate, it would cap out at 2.0gigs, crash the apache, and we could nearly instantly find and fix it.
Now, if a file is filling up at a rate of say, a gigabyte/minute, it can happily fill up a file server. We first noticed when a file server went from no warning on our monitoring software, to 12gigs free. We lowered the reserve space for our backups to give us breathing room, but still only had about 60 minutes to figure out what was causing this problem, and what to do about it. Running a quota report on the file server showed that root was using about 463gigs of space, well more than normal, so we assumed it was a log file growth issue. (All of our apache logs are stored by “root” for a few days so that we have accurate accounts of what happened on our servers.)
What this meant, was on one of the handful of servers that mounts the file server, in one of the thousands of home directories, there was a HUGE log file for us to delete. After some fruitless ideas of attepting to turn off Apache2.0 on servers which appeared to be “busier” than normal turned up nothing, we started running per-server “du” commands on all users log directories. (”du” stands for “disk usage”, it tally’s up how much space is being used by a file or directory regardless of user.) This went faster than expected, and on a hunch I skipped ahead to the server “flanders”, which turned out to be the culprit. Found the site, deleted the file, and went on our happy way restoring service.
[HomerSimpson]Stupid Flanders…[/HomerSimpson]
I apologise for the stopping some services to try and find the culprit. The idea behind it is if we can find the sevice which is stopping the growth, or even the server, we can prevent the file server from filling up, which would have put thousands more out of service. We are in the process of writing something to troll our file servers looking for these problems, however they tend to crop up very quickly.
Looks like we need to see if:
http://httpd.apache.org/docs/2.0/programs/rotatelogs.html
Is a viable solution for our logging needs.
.
September 22nd, 2006 at 8:44 am
Is this causing our site not to open up?
Thanks
Uday
September 22nd, 2006 at 8:45 am
It probably is - my domain is down as well.
But since this wasn’t done too long ago, I suppose it’ll all come back up… eventually.
September 22nd, 2006 at 8:51 am
Mine is down as well.
September 22nd, 2006 at 8:53 am
This is my 2nd day with an operational website on this new webhost, does trouble follow me everwhere I go?
September 22nd, 2006 at 8:53 am
Yeah…its not doing to hot lately >.
September 22nd, 2006 at 8:55 am
How long do you think it will be down for?
September 22nd, 2006 at 9:00 am
hey Russ, why’d you have to bring the dark cloud with you, man? it’s been months and months with no service interruptions!
September 22nd, 2006 at 9:03 am
I suggest that those who have a filer issue, fill-in the “Is there a system-wide problem?” form in the “Contact Support” tab.
September 22nd, 2006 at 9:04 am
Well I have 2 website parked with dreamhost… on same account…
domain1.com is working fine with data base application…
domain2.com is down, how ever I have 2 custome DNS set as, sub1.domain2.com and sub2.domain.com, both are working as they are parked some where else… but the main domain, thats domain2.com which is in dreamhost is down for a while now…
September 22nd, 2006 at 9:18 am
I’ll tell you what’s filling up the filer… it’s BAZOOKA’s swap file!
top - 09:14:31 up 16:39, 6 users, load average: 335.24, 326.79, 236.28
Tasks: 3 total, 1 running, 2 sleeping, 0 stopped, 0 zombie
Cpu(s): 8.2% user, 8.8% system, 0.0% nice, 83.0% idle
Mem: 4075060k total, 3960816k used, 114244k free, 1100k buffers
Swap: 6313512k total, 5015680k used, 1297832k free, 29668k cached
HA!
Seriously. Someone is doing something on BAZOOKA that’s way, way, way I/O bound. Look at that load, yet it’s 83% idle? I’m betting that whatever’s hammering on the filer is coming from BAZOOKA. This has been happening on BAZOOKA on and off for three days at least.
I’ve been running “top” non-stop because I’m waiting for it to break a 1000 load. 982 is my record so far but it was at zero swap, so probably rejecting new processes.
September 22nd, 2006 at 9:46 am
OK… after more than a 5 hours of downtime… finally my domain2.com is up and running…
September 22nd, 2006 at 11:15 am
Is everyone’s site working now? Mine is still down. Hmmm…
September 22nd, 2006 at 11:25 am
My two active sites at DreamHost are down: http://www.jokecatalog.com/ and http://www.guide2christmas.com/
September 22nd, 2006 at 11:28 am
One of my domains is still down.
Does anyone else find that this is happening a LOT lately? I have been with DH for almost 2 years and for the first 1.5 things were glorious. I could count the amount of downtime on one hand. And ever since the big LA power outage last year, things have been up - down - up - down. I really just cant afford to be down so much.
September 22nd, 2006 at 11:29 am
mine still down
need to fix this asap pliz
September 22nd, 2006 at 11:29 am
My mail (which I do have the Spam filter turned ON) is very very slow at best.
Yesterday I noticed that some e-mails came in before others.
Frustrating!
September 22nd, 2006 at 11:35 am
What time was that update note above? When is Paperboy due back up? Why am I posting this here? When will I get a life and chill out? Who left the cake out in the rain?
Friday is a day for unanswerable questions.
September 22nd, 2006 at 11:37 am
Man this is such a bummer,
Why can’t they just kill the offending processes.. I don’t understand why my ftp/www/mail/ssh are all down. Don’t they have any kind of redundancy or backups? I paid for the Level 3 Code Monster PLAN, and I never expected to have such unreliable service as this.
I am really pissed because I was in the middle of a presentation for the launch of a new website.
AHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH!
September 22nd, 2006 at 11:39 am
FWIW one of the accounts I run was down this morning, three others were up and running the whole time (but that seems to happen a lot). It came back up for a few hours, but now all four are down, so whatever happened, happened recently and is very widespread (all four of my accounts are on different servers).
September 22nd, 2006 at 11:42 am
And the control panel seems to be down as well now
September 22nd, 2006 at 11:44 am
Curly seems to be down at the moment as well; no web or ftp access. Mail still seems to be okay, but the control panel is really sluggish (taking a couple minutes to fully refresh). This always seems to be great timing too, as I’m in the middle of posting files for a client’s project. Sigh….
September 22nd, 2006 at 11:44 am
My site is down as well. And the control panel is as slow as an old lady crossing the street at rush hour.
What’s going on?
September 22nd, 2006 at 11:46 am
Again. If I got a dime for everytime my domain didnt work I’d be rich. 2 terabyes of bandwidth mean NOTHING when my website is unreliable. I understand issues but why does it always result to hours and hours of downtime for the customers?
September 22nd, 2006 at 11:50 am
Yep… down again. I was up most of the day… only been down about 20 minutes.
September 22nd, 2006 at 11:51 am
My site’s still down. Been a rough couple of weeks.
September 22nd, 2006 at 11:52 am
Having lots of problems with Jabber winking out on us…are Jabberd ’s impacted by this? Several of our domains are suffering jabber blackouts
September 22nd, 2006 at 11:53 am
My is down as well. This is the third time in as many months (that I know of!) where my site has been down during primetime for hours on end. Something wrong with this picture…
September 22nd, 2006 at 11:54 am
I’d have to agree with Hooky above. It seems that in the last year there’s been more downtime with DH than in the past 4 years. I’ve been with these guys that long and never noticed much of this until lately. Even if we have Gigs an Gigs of space/bandwith it aint gonna help us out if it doesn’t work. Come on dreamhost, pick it back up.
September 22nd, 2006 at 11:56 am
down down down down down!
I need to get to my files!!
September 22nd, 2006 at 11:57 am
Mine went down at around 2:20 PM Eastern and is still down.
September 22nd, 2006 at 11:57 am
We’re down too. BUMMER!
September 22nd, 2006 at 11:58 am
Down here too :(. Support just changed my support ticket from “Unverified Web Outage: Reported” to “Resolved, No Problem Found”. Everything is still down. SO frustrating!!!
September 22nd, 2006 at 11:58 am
Down too…even though they say things are fixed.
September 22nd, 2006 at 11:58 am
The issue has definitely NOT been resolved contrary to what the EDIT is showing. FTP and HTTP are both down for my one account hosted here.
September 22nd, 2006 at 11:59 am
Down here too.
September 22nd, 2006 at 11:59 am
I’m so very confused here. I have a CMS that manages multiple domains. The domain the CMS is installed on is dead. Other domains that are powered by this CMS and explicitly rely on the domain it’s installed on are working fine.
I only stay with Dreamhost because I like to read the wacky monthly newsletters
September 22nd, 2006 at 12:02 pm
down down downnnnnnnnnn down down every dayyyyyyyyy downnnnnnnnnnn
every dayyyyyyyyyy in the townnnnnnnnnnnnnnn
i have one teraaaaaaaaaaaaaaaaaaaa of bandwidthhhhh
but i never go to use ittttttttttttttttttttttt
down down downnnnnnnnnn my web is alwayyyyyyyy dowwwwwwwwwwnnnnnnnn
thanks
September 22nd, 2006 at 12:03 pm
First time replying on this. I have been with you guys for two years and now it’s time to move on. I have never been with a company that has had so much downtime as you guys do, it’s pretty out-of-control. I understand you’re still recovering from that power outage and I’m not mad at anyone as your customer service has been top-notch amazing. Doesn’t matter who’s problem it is but I can’t wait any longer for things to get better. Thanks for the ride, it was awesome up until the end. Good luck to you guys!
September 22nd, 2006 at 12:05 pm
My sites were working this morning, all down now.
http://lotusmedia.org
http://orangepolitics.org
http://brianandruby.org
etc…
September 22nd, 2006 at 12:05 pm
What’s getting frsutrating is the site shave been going down, we file support requests that never get answered, and nothing gets explained. We’ve had a support request in for a couple of hours on this, which has been ignored. I’ve got my boss breathing down my neck about it, and I can’t tell her anything. Makes me regret referring another client a few days ago, but *that* site is at least coming up.
September 22nd, 2006 at 12:06 pm
yup, still down for me too. It’s obviously NOT fixed. In fact, it was workign fine earlier aroudn the tiem that accordign to this there were problems. Not that it’s “fixed” I’m down.
AN dI just wrote a post defending Dreamhost a day or two ago…..
September 22nd, 2006 at 12:07 pm
Agree with the most comments above. it’s crazy what’s going on lately with DH. I need seriously to think about swithing to another provider. Imagine this, my sales guy was on the phone with a very important new client and he asked the client to check our website. Well, the site was down and the sales call went to hell…
September 22nd, 2006 at 12:07 pm
according to the last update all sites should be up. but i’m down again.
my husband just switched his site. i’m going to have to switch mine as well.
this is unfortunate, cause until a year ago, we were never down!
September 22nd, 2006 at 12:07 pm
2006 service is very bad. We’re thinking of moving our domains. Ftp problems and downtime. And lies about these problems
September 22nd, 2006 at 12:10 pm
My site http://www.nsssa.ca/ is also down.
September 22nd, 2006 at 12:14 pm
Dude, the Bighappyfunhouse is neither Big Happy or Fun.
Help, I’m drowning here…..
September 22nd, 2006 at 12:15 pm
They just changed status on my shell login to resolved, even though it’s not. My machine is lifesaver. Anyone else on it?
Site is still out. Not a deal breaker, but I’m wondering what happened.
September 22nd, 2006 at 12:15 pm
Go to the support page and use the form, the one for system-wide problems.
If your sites are still having problems, use it! Mine are still showing as unconfirmed because someone out there is not using it.
September 22nd, 2006 at 12:16 pm
do other providers really never have downtime? I have been with DH for a few months, and the only times I have noticed downtime it has bee fixed pretty quickly. My sites are all down right now, and have been for over an hour, but that is the longest I have ever gone without service. I hate to see it go down at all, but realistically isn’t that a problem everywhere?
September 22nd, 2006 at 12:16 pm
I’m glad I’m not the only one. I think it might be time for a change too. Twice now with clients checking my site, I’ve had downtime. Not to mention I seem to get hit when my traffic for the day is pretty good…UGH
September 22nd, 2006 at 12:16 pm
My site is still down.. it’s 2:13 Central - filedump.dreamhosters.com
September 22nd, 2006 at 12:17 pm
It’s probably a ddos attack by some discruntled ex user-geeky-type that decided to get back at dreamhost.
I have a site down also, but it does say “The issue has been resolved and any servers that might have had web services temporarily turned off are being put back into action.”
note the ‘are being put back into action’
which to me means one at a time.
Keep up the good work dealing with this dreamhost.
September 22nd, 2006 at 12:17 pm
I love it when all my clients, in unison, come complaining to me about their website / email being down. Dreamhost, I love you, but you’re definitely in the dog house tonight.
September 22nd, 2006 at 12:18 pm
I’m thinking maybe there’s a separate issue. Seems like everything was fine and the problem was listed as “fixed” and then after that, an hour ago, everybody’s sites went down.
September 22nd, 2006 at 12:20 pm
Looks like PHP is falling over for me. Plain HTML pages load, but anything dynamic generates a 500 server error.
September 22nd, 2006 at 12:24 pm
Comstock:
That’s one of my beefs, too… several times I’ve had issues in the panel get closed out on me when the problem wasn’t fixed yet, it’s like whenever an overarching issue gets resolved the assumption is that it fixed all the reported problems, rather than taking the time to check.
September 22nd, 2006 at 12:24 pm
Alexander is running high too.
12:23:57 up 13 days, 8:24, 6 users, load average: 771.41, 867.97, 402.
Mem: 4075060k total, 3961624k used, 113436k free, 1072k buffers
Swap: 6313512k total, 156956k used, 6156556k free, 646168k cached
September 22nd, 2006 at 12:25 pm
Dreamhost, is the dream ending? :/
September 22nd, 2006 at 12:26 pm
I just got this…
I’m extremely sorry, but we are experiencing extreme network failure right now. All of our Admins are working extremely hard to find the root of the problem. This problem will be fixed as soon as possible.
September 22nd, 2006 at 12:26 pm
i’m gone. i just sent out an email introducing my new company today - and this happens.
September 22nd, 2006 at 12:26 pm
Our sites are down here. I hate this down time. There’s been too much of it lately.
September 22nd, 2006 at 12:26 pm
Why the hell is dreamhoststatus.com alive, yet everyone else is dying?!?!
September 22nd, 2006 at 12:27 pm
Hello Dreamhost. I just want you to know that i have lost 2 clients today because of this beautifull system outage. A i know ithat i am not alone in this i beg you to move and solve the problem because we need to eat too. Thank you very much. Greetings from Republic of Panama.
September 22nd, 2006 at 12:28 pm
We just moved our site here recently because of all the down-time we experienced with another host company. But in the few months we’ve been with DreamHost, we have experienced numerous unexpected outages. This has us muttering bitterly: Maybe the last host company wasn’t so bad…
September 22nd, 2006 at 12:29 pm
Both of my sites are now down… why one problem had to affect so many people is most boggling. There better be some kind of restitution for this downtime because I’m seriously considering moving elsewhere.
September 22nd, 2006 at 12:29 pm
My site just came back up at 3:30 PM eastern
September 22nd, 2006 at 12:30 pm
Did anyone else have a problem with their MySQL prior to this. My original problem was connection to MySQL. Support told me that theyhad rebooted the MySQL server. Now, nothing will load.
This is really bad for me. I monitor political activity in the state and tomorrow is Primary Election day. Can anybody say, disaster?
September 22nd, 2006 at 12:30 pm
Been with DH for about 2 years. Currently using 4 hosts and downtime in DH is constant. My sites are still down. Can’t wait till all domains from DH are switched.
September 22nd, 2006 at 12:31 pm
Ah, got shell status (briefly) on lifesaver:
Over a thirty second time frame:
[lifesaver:~] $ w
12:27:34 up 8:13, 6 users, load average: 493.50, 97.04, 74.90
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
[lifesaver:~] $ w
12:27:36 up 8:13, 6 users, load average: 551.44, 115.63, 81.03
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
[lifesaver:~] $ w
12:27:41 up 8:13, 6 users, load average: 589.92, 130.84, 86.14
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
[lifesaver:~] $ w
12:27:46 up 8:13, 6 users, load average: 626.84, 146.12, 91.32
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
[lifesaver:~] $ w
12:27:55 up 8:13, 6 users, load average: 649.44, 158.79, 95.71
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
[lifesaver:~] $ w
12:28:10 up 8:13, 6 users, load average: 783.03, 220.57, 117.36
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
Something is dying. Wow.
(While I’m writing this:)
Load averages now dropping. And my site is up.
September 22nd, 2006 at 12:31 pm
This is just stupid. If my sites can’t be up for 3 days straight, I don’t care if the hosting costs $1. If there is no real downtime resolution, I’m outa here.
September 22nd, 2006 at 12:32 pm
I’m back!
Thanks for the posts it gave me something to read while waiting.
Kick it up DH!
September 22nd, 2006 at 12:33 pm
“Why the hell is dreamhoststatus.com alive, yet everyone else is dying?!?!”
Presumably it’s hosted apart from the other machines. It wouldn’t be useful if it only worked when your systems worked too.
Still - I wish I could host my site on the status or ordering machines.
September 22nd, 2006 at 12:33 pm
This is terrible .. just sent out my site to prospective/current clients and now this!?!?!?
Totally unacceptable.
September 22nd, 2006 at 12:34 pm
Website is back up now and seems to be working quickly. Let’s hope for the best. I now have turned on a monitoring service on the website hosted here to drill down a little further as to downtime percentages.
September 22nd, 2006 at 12:34 pm
We’re back up!!
September 22nd, 2006 at 12:34 pm
I think we’ve been down more than up for the last month. This is literally the sixth time I’ve been down recently. I understand you are having problems. But once more this month, and I’m gone.
September 22nd, 2006 at 12:35 pm
when was this update posted? it would be useful if updates would be timestamped separately from the original posts.
September 22nd, 2006 at 12:35 pm
I’m partly back up. Can’t get mail yet.
September 22nd, 2006 at 12:36 pm
Not up yet… rassum frassum… well, you gets what you pays fer.
September 22nd, 2006 at 12:38 pm
But now, a new monitoring system has appeared for submissions to this blog. To post a message, I had to enter a code, and then my message was “approved” !!! Mmmmmmm
September 22nd, 2006 at 12:38 pm
Well, our site was up 20 minutes ago and now it’s down. The number of issues getting posted to the Dreamhost RSS feed on a daily basis over the last several weeks is troubling. I can’t recommend Dreamhost to anyone at this point; moreover I am going to have to reconsider renewing our hosting with Dreamhost in January.
September 22nd, 2006 at 12:38 pm
OK - now this is unacceptable. Dreamhost just responded to my site down post by saying no system wide error was found. 77 posts here would suggest otherwise. I’m pretty tollerant and luckily my sites are far from mission critical. But, come on!
Definitely time to shop around.
September 22nd, 2006 at 12:39 pm
My sites, mail and jabber STILL don’t work!!!
September 22nd, 2006 at 12:40 pm
Nothing up for me, and that’s four domains. Shouldn’t there be something to acknowledge this on the status board. Like Neil, I have run into this much more this month than I expected and will also have to go elsewhere. bummer
September 22nd, 2006 at 12:41 pm
Brian:
Same here, and that’s an insult to me as a customer… it should be clear that a LOT of us have sites down at the moment.
September 22nd, 2006 at 12:42 pm
Of all the downtime that I have experienced, not once have DH made any restitution. Sites still down!!!
September 22nd, 2006 at 12:43 pm
I’m down EST at 3:45
September 22nd, 2006 at 12:43 pm
Not only are all my sites down, but the support panel refuses to even verify my outages. Not exactly confidence inspiring.
I’ve been with DH for over 6 years and I’m sad to say I’m considering seeing other hosts…
September 22nd, 2006 at 12:45 pm
zebes.dreamhost.com is down. Server is working cause it accepts pings but it does not accept http or ssh or ftp requests. whas going on…
September 22nd, 2006 at 12:45 pm
Hello,
none of our emails addresses are receiving mails. We can send but nothing else is happening.
During the time your server had problems, could it be that some emails were lost?
Thanks,
Elina
September 22nd, 2006 at 12:46 pm
sourpatch appears to be down as well.
this is getting a bit absurd. we have quite a bit of downtime over the last month or so.
give us a fix, and if you can’t do that please give us some answers.
September 22nd, 2006 at 12:46 pm
Also, no access by http://FTP. Now that’s a problem. At least there should be an update so we know what is going on. No excuse for not having one. And pretending that all is well with 90 posts to the contrary….well….
September 22nd, 2006 at 12:46 pm
Good ol’ medina still down!
September 22nd, 2006 at 12:47 pm
Soooo… still down huh?
September 22nd, 2006 at 12:47 pm
Yah, my site is down as well, all of them actually. 4 Different domains.
September 22nd, 2006 at 12:47 pm
im down 3:47PM EST
September 22nd, 2006 at 12:48 pm
My sites are all down no go for lucky
September 22nd, 2006 at 12:49 pm
Dreamhost should be called nightmarehost. I’ve stopped counting the number of times my sites have been down. Time to go i guess and find a reliable host.
September 22nd, 2006 at 12:49 pm
Damn we’ve been on a bad streak this last little while…not great timing for me
September 22nd, 2006 at 12:50 pm
This blog has certainly grown in popularity in the last month
September 22nd, 2006 at 12:50 pm
yep :[
September 22nd, 2006 at 12:50 pm
This is pretty disapointing, especially as I’ve recommended dreamhost to so many people.. makes me look bad.
September 22nd, 2006 at 12:51 pm
My domains are down now (they weren’t earlier), especially the WordPress Admin for my blog: http://www.mostlymuppet.com/
September 22nd, 2006 at 12:51 pm
peso is down
September 22nd, 2006 at 12:52 pm
I think Dreamhost knows there are issues, and you probably don’t see them lurking in here because the big guys are in the data center. Not all sites at dreamhost are down. I have over 100 sites, just checked them, but I do have about 8 that are down. Although, email is working fine at the moment.
Sit tight, or go look for a new host. Yes, Dreamhost is getting it from both ends right now and most of you have every reason to leave. It’s not fun sitting around wondering when. I doubt Dreamhost is having a party at the datacenter and ignoring us. But if they are, send some piña colada’s my way. It is after noon here!
September 22nd, 2006 at 12:53 pm
Oh, more downtime? Whoda thunk it? I sure am glad I’m shelling out $100’s a year for services that are down an absurd amount of time!
September 22nd, 2006 at 12:53 pm
mine is down… STILL. ugh, was working not that long ago.
UGH
September 22nd, 2006 at 12:53 pm
Multiple sites down as well. I echo the frutrations of others. If this was an isolated incident or these types of issues were fixed quicker, I could accept it. I can’t accept or recommend the service as it is. Recently, I have had emails from support saying that the issue is resolved, but it’s not.
September 22nd, 2006 at 12:54 pm
Angry mob building up here…
But I noticed some sites are partially up, despise that it seems that this “network failure” covered a pretty big area . hopefully they will go through, hope they fix it soon
September 22nd, 2006 at 12:54 pm
Dear Dreamhost;
WTF
Sincerely,
Ron
September 22nd, 2006 at 12:54 pm
My blog has been down since 2:27 pm (central time).
September 22nd, 2006 at 12:54 pm
I can’t log into my shell account on xevious, andmy site is quite down just now.
September 22nd, 2006 at 12:55 pm
Downtime is the new uptime.
September 22nd, 2006 at 12:55 pm
This is crap - been w/ DH for couple of great years but something has changed - did you guys lose the brain wizz kid?????
If this continues another day, I’mm switching hosting service - I’d suggest you get your act together and fix your business.
September 22nd, 2006 at 12:55 pm
my site is down!!!! look
Hello,
The following url is down:
http://www.arquivoamador.com
the error detected is:
Http error:Http_client.No_reply
Error was detected at 2006-09-22 16:30:49
Your login:arquivoamador
–
Best regards,
http://host-tracker.com/ support team
September 22nd, 2006 at 12:57 pm
on another host?
September 22nd, 2006 at 12:58 pm
3 of my sites are down, ftp is not accessible, mail is toast…sigh.
September 22nd, 2006 at 12:58 pm
WTF dreamhost. Are you going to communicate with us, offer some info… or just leave out to dry. This is getting beyond ridiculous. I love what dreamhost has to offer service-wise but I just dont get how this can happen day after day.
September 22nd, 2006 at 12:59 pm
Yes, since sometime ~11:30 - 11:45am PDT today, my domains, sites, ftp, and Dreamhost’s Control Panel have been down.
September 22nd, 2006 at 1:02 pm
what is going on, I have over 50 clients that I’ve reccommended to dreamhost that are now wanting to jump ship. PLEASE HURRY AND FIX THESE ISSUES. I have never encountered such problems in all my years of dealing with hosing companies.
Very Dissapointing
September 22nd, 2006 at 1:02 pm
I dont get it, how is it that I cant view my site but I can ftp to it and access everything else just fine. I also find it funny that there site http://www.dreamhost.com is always up but our sites are not this tells me that they are not using there own service to host there own site or they have it isolated from the rest.
I hope they fix it soon. It looks like its been down all day (hmm must be friday)
DREAMHOST Im loosing potential customers with downtime (I hope you fix it soon).
September 22nd, 2006 at 1:03 pm
I’m with SJBG. This string of outages has been unusual for DreamHost. Everything ran very smoothly in the past years.
September 22nd, 2006 at 1:03 pm
Looks like the problem is still happening. zebes doesn’t seem to be working, and I can’t log in to my shell account either.
September 22nd, 2006 at 1:03 pm
I just moved a client over from those idiots over at 1planhost, and we’re up for one day on dreamhost and this happens. I’m
embarassed.
September 22nd, 2006 at 1:04 pm
By the way Put me on the server this blog is on it doesn’t seem to be having any issues.
How about that!!!!!!!!!!1111111
September 22nd, 2006 at 1:07 pm
My domains are down, paysprite.com and g13can.com
September 22nd, 2006 at 1:08 pm
All my sites are down, which tells me that this is another example of claiming something is fixed when it’s not.
I don’t mean to be harsh, but is it so hard to just hold off on declaring something fixed until it’s actually fixed? You have no idea how confusing this is. I propose that all system problems have to resolved for 24 hours before they’re declared “fixed”. That way you can catch any problems without disappointing people.
September 22nd, 2006 at 1:08 pm
Customer Service 101
When there’s an outage, communicate with your customers to let them know what’s going on (especially when you have a page like dreamhoststatus.com.)
September 22nd, 2006 at 1:10 pm
It would suck if you just bought a bunch of traffic.
September 22nd, 2006 at 1:10 pm
This is pretty much the icing on the cake. My sites have been done more than up lately. I will not be renewing my service at the end of the year, and will most likely pull all of my sites before then. I don’t know what happened to Dreamhost, but I have a pretty good idea of what’s going to happen.
September 22nd, 2006 at 1:11 pm
wtf
September 22nd, 2006 at 1:14 pm
I’m real pissed! I just notified 600 people that they can buy tickets to an event I’m promoting at http://www.gainesvilleimprov.com and now I find out that the server is down. This is not good guys!!!!
September 22nd, 2006 at 1:14 pm
You jus reported my sites are up, but they are not. Did you actually check?
September 22nd, 2006 at 1:15 pm
Just to let you know; I’m now officially looking at other webhosting plans. Most of the last 2 years have been great, but the last month or two has been terrible.
September 22nd, 2006 at 1:17 pm
Oh come the hell on Dreamhost! Why are my sites down again?
September 22nd, 2006 at 1:32 pm
If the FTP is working as some say it is (I haven’t checked mine - why bother and get even more upset?), it’s not a server problem, it’s something else.
September 22nd, 2006 at 1:40 pm
I’m down again, wow what a suprise!
http://www.3sails.com
Oh well I guess I get to put a bunch more stuff into my app before I even start to promote the prelaunch. Who wants to launch any way?
September 22nd, 2006 at 1:46 pm
my sites are still down too.
September 22nd, 2006 at 1:59 pm
Please guys - work on the servers at night so we can experience some uptime during the day.
I would like to move more clients to Dreamhost but I’ve got to know they will be happy and get their email without downtime.
I think an explaination is in order?
September 22nd, 2006 at 9:39 pm
Sad to report that as of of this moment, about 12 hours after this was seemingly fixed, it looks like it’s back again.
Filesystem Size Used Avail Use% Mounted on
/dev/hda1 2.2G 1.5G 578M 72% /
/dev/hda3 7.4G 5.2G 2.0G 73% /usr/local
10.3.19.68:/vol/boot/postal/dhs
60G -10.0Z 206G 101% /home/.dhs
10.3.100.105:/vol/boot/postal/fuzznut
1.2T 1.1T 114G 91% /home/.fuzznut
10.3.100.104:/vol/boot/postal/jinsen
1.2T 1.1T 114G 91% /home/.jinsen
10.3.100.105:/vol/boot/postal/jocasta
1.2T 1.1T 114G 91% /home/.jocasta
10.3.100.104:/vol/boot/postal/jockey
1.2T 1.1T 114G 91% /home/.jockey
10.3.100.105:/vol/boot/postal/johannesburg
1.2T 1.1T 114G 91% /home/.johannesburg
10.3.100.104:/vol/boot/postal/keerokee
1.2T 1.1T 114G 91% /home/.keerokee
10.3.100.104:/vol/boot/postal/keets
1.2T 1.1T 114G 91% /home/.keets
10.3.100.105:/vol/boot/postal/kekkonen
1.2T 1.1T 114G 91% /home/.kekkonen
10.3.100.104:/vol/boot/postal/leland
1.2T 1.1T 114G 91% /home/.leland
10.3.100.105:/vol/boot/postal/lemming
1.2T 1.1T 114G 91% /home/.lemming
10.3.100.104:/vol/boot/postal/mallymal
1.2T 1.1T 114G 91% /home/.mallymal
10.3.100.105:/vol/boot/postal/malted
1.2T 1.1T 114G 91% /home/.malted
10.3.100.104:/vol/boot/postal/ness
1.2T 1.1T 114G 91% /home/.ness
10.3.100.105:/vol/boot/postal/nessu
1.2T 1.1T 114G 91% /home/.nessu
10.3.100.104:/vol/boot/postal/opium
1.2T 1.1T 114G 91% /home/.opium
10.3.100.104:/vol/boot/postal/oprah
1.2T 1.1T 114G 91% /home/.oprah
10.3.100.104:/vol/boot/postal/patch
1.2T 1.1T 114G 91% /home/.patch
10.3.100.104:/vol/boot/postal/patchouli
1.2T 1.1T 114G 91% /home/.patchouli
10.3.19.117:/vol/boot/postal.localbackups/bart
1.2T 1.1T 61G 95% /mnt/filerbackup
10.3.19.117:/vol/boot/postal.crontabs/bart
1.2T 1.1T 61G 95% /usr/local/var/spool/cron/crontabs
September 23rd, 2006 at 2:21 pm
The thing w/ rotatelogs is that it can end up opening an extra filehandle if you run it per virtual host — easier to do one mongo log file that includes the virtual host field (%v) and split it up via post-processing (which it sounds like you may do). Other than that gotcha, rotatelogs is OK. cronolog is far nicer (http://cronolog.org/) — same caveats apply.
September 24th, 2006 at 2:28 am
Thanks for fixing it guys
March 27th, 2007 at 7:24 pm
take it easy! I hope dreamhost will update their quality.Don’t let my website d to many times~
If u let my website d too many times I will fuck u ~Aha~
March 28th, 2007 at 1:43 am
My FTP service is giving me a no-socket error…