File storage server issues

One of our many file storage servers suffered an outage which has caused high loads and slowdowns on some servers (only affected one cluster/group of servers). The storage machine is back online, no data was lost. You may experience reboots, high loads and slowdowns on your machine for a short while longer, but everything is on its way back to normal.

Very sorry about the inconvenience. Please contact support if you have any questions.

.

40 Responses to “File storage server issues”

  1. 1
    banglogic Says:

    You know, I have like twenty different domains and about fifteen web sites running for different clients, all running through DreamHost. When all of my sites go down at once makes it me sad. It makes me look bad to my clients because they think I didn’t select a good vendor for hosting their web site. Just when I think DreamHost has gotten it’s act together, my phone start ringing with my client’s saying “It’s down again.”

    Please DreamHost, get better soon. Please.

  2. 2
    Rodrigo Says:

    My site isn’t working yet… Fucking 3rd world service

  3. 3
    dne Says:

    my site is down too. ive NEVER had problems with dreamhost so im only too happy to wait for you guys to fix whatever the problem is. just wish my site could be seen!

  4. 4
    JLP at AllFinancialMatters Says:

    My blog has been down for well over 1.5 hours now. It is quite annoying!

  5. 5
    bdillahu Says:

    Not that I like downtime, but I’ve been fairly happy with them.

    And just to point out it happens to everybody, have you looked at the Blogger world this week? And they certainly have resources.

    http://buzz.blogger.com/2006/10/blogger-outages-novel.html
    http://status.blogger.com/

    http://www.craftacraft.com/

  6. 6
    Roach Says:

    It’s very simple, really. Their definitions of the words “fixed”, “normal”, and “soon” aren’t the same as everyone else’s.

  7. 7
    JLP at AllFinancialMatters Says:

    bdillahu,

    Last I checked, blogger was FREE! Dreamhost is not.

    I realize that downtime is a natural part of the business. But, whenever the downtime gets to be more than 20 minutes, I start gettin’ real pissed!

  8. 8
    JLP at AllFinancialMatters Says:

    bdillahu,

    Last I checked, blogger was FREE!

    I understand that downtime is natural. However, anytime it gets to be longer than 20 minutes, I get pissed! This is going on 1 hour and 45 minutes, which is crazy!

  9. 9
    Swoggy Says:

    Awwww…. just when everything seemed to be running relatively quickly and stably for about a week :(

    Hope it comes up soon (said the Bishop to the actress)

  10. 10
    OG Says:

    This has happened twice in this week :( very annoying.
    It’s the mySQL server, isn’t it?

  11. 11
    Swoggy Says:

    No, mySQL servers are Fridays… file servers are Mondays :p

  12. 12
    banglogic Says:

    Looks like it’s working again…

  13. 13
    Winn Says:

    The file server is still down for me. I can’t get a reliable connection over http://FTP. It is very frustrating. There seems to be a large underlying problem over at dreamhost. I don’t know whether it’s old equipment, poor management, not enough cooling, or too many customers crammed onto one server, but something major has to change or they will start losing customers at alarming rates.

  14. 14
    Edt Says:

    Getting pretty frustrating. I know most guys claim 99% uptime, but this is about the 5th time this year the sites have been down for an extended period of time. This is getting pretty old.

  15. 15
    Winn Says:

    Server uptime percentages are very misleading. 99% server up time sounds high until you think about what that actually means. A server can be down for 87 hours per year or 100 minutes per week, and still claim 99% server up time.

    My best guess is that our servers on Dreamhost are down for about 1 hour per week, which means that Dreamhost can probably claim 99.5% server up time, and still be truthful.

  16. 16
    Angie Says:

    I’ve been with Dreamhost for going on 5 years but this - these last several months - unbelievably ridiculous. Time to swallow the checkbook and actually buy a hosting account that, while it may cost four times more, will have four times’ better uptime. How are we supposed to run a business on this?

  17. 17
    Robert Says:

    Hear, hear, Angie!

    Dreamhost was fabulous for the 5 years I’ve been with them, but it’s been a frickin’ train wreck since this summer. It’s incredibly frustrating because I keep wanting to believe they’ll get better. I mean, judging by the responses we get from blogs and customer service, it really does sound like they’re trying to address the problem, but it’s just not getting better. :-(

  18. 18
    Dan tdaxp Says:

    I am having users complain about images that have just disappeared. JPGs I uploaded both yesterday and today no longer exist.

    Andrea, when you said “no data was lost,” did you mean that the data will be restored, or you were not aware of the problem?

  19. 19
    the ninja Says:

    i only just started using my account yesterday and its down already. That is not good.

  20. 20
    polimast Says:

    Where are all our data on plan code-warior? And where is this plan??
    Please , activate it

  21. 21
    dvg Says:

    I have to go with Robert (above). I’ve been a happy DreamHoster — no, an ECSTATIC DreamHoster — for at least 4 years now. But since sometime around July it’s just seemed like everything is going to hell in a handbasket. I love DreamHost so much, and I hate the idea of having to move my 20+ sites …

    Please, please make it all better, guys.

  22. 22
    Bill Doern Says:

    Unbelievable. And, unprofessional to boot.

    Dreamhost might want to consider being “happy dreamhost professionals” for a change. Like others that have posted here, we manage several sites for our clients. I agree - this makes us look bad. Robert couldn’t be more right. Dreamhost used to have excellent responses to technical issues, now they just seem clouded with ambiguity and non-commital responses.

    Dreamhost - if you read your comments, perhaps you could dignify us with a response and possibly an action plan for addressing our concerns in the future rather than hiding behind a shroud of technobabble and irregular updates - we have businesses to run.

    Happily,
    Pissed of Dreamhost sucker

  23. 23
    bany Says:

    FTP server down again. This is the server at Dreamhost that are customers use for upload and download of data for support. Please, get it working and keep it working.

  24. 24
    The Zero Boss Says:

    mySql just went down for me. Sigh.

  25. 25
    The DH Optimist Says:

    SQL, File server.. My sites on Apok have been on the DH roller coaster since last week.

    …But, lemme guess. We are now going to get 4 billion times the amount of space, 40 billions times the bandwidth and 400 billion times the downtime to make up for it? ;-)

  26. 26
    Ivar Says:

    hmm.. My site works just fine. I´m getting a little worried though reading all these comments. I just purchased my site yesterday
    I must say though that you (Dreamhost) were damn quick. After i had payed, 2 hours later i was uploading files via FTP and viewing my site :D
    Sweet

    I can understand that you might be undermanned or something.

    I just hope that you can work things out for the best

    Best Regards

    Ivar Radmanesh

  27. 27
    Asian Persuasion Says:

    DH Says: ALL YOUR SITE ARE BELONG TO US!

  28. 28
    chip Says:

    I am running our school’s website and webmail through Dreamhost. I am in the position of having to explain and defend all the downtime we are having. I moved this stuff off site so I wouldn’t have to do server maintainence but I have been pricing servers all day.

  29. 29
    Curtis Says:

    Mine has been running slow for a few days, last night and today they are unreachable! It realy sucks when your in the middle of a site update and mods :(

  30. 30
    Thomas Says:

    Come on, give them a break. They have 750+ servers, Murphy’s law suggests that at least one of these servers is bound to be down!

    Happily,
    Happy Dreamhoster

  31. 31
    Dreamhost Sales Says:

    OK - We are really, really sorry about being such colossal fuck ups.

    We’re California, we’re happy, we’re employee-owned. C’mon, guys - we’re a web hosting Kibbutz. But, Gawdalmighty, we’re making a ridiculous amount of money!!!

    Oh, yeah — the outages. We are so, so, sorry. I mean - It’s all good, but no. this isn’t good, is it?

    C’mon! Happy Dreamhost Customers! Put on a smilee brave face!

    Laugh with us! We’re all happy and fun, right?

    Right?

    Alright. You win:

    3 PetaBytes of Storage for EVERYONE! And a Hundred Bazillion ExaBytes of Bandwidth!

    Yeah! That’s it!

    And - 400 ftp users and a million shell account users. No, wait! 2 Million shell account users!

    Happy now?

    Of course, the server up time will still suck ass.

    But, you get a bunch of storage… so that’s OK, right?

  32. 32
    Textheavy Says:

    I love reading all of these whiny comments.

    If you want 100% uptime, and particularly if you are using the Web for business, you should look elsewhere. Unless you are on a Strictly Business plan, don’t expect business-level performance. You should understand that you are getting a hell of a lot more from Dreamhost than you would get anywhere else.

    I have a business Web site, and guess what…I co-location host it with Rackspace, because it’s business. Dreamhost plans are not sold as business solutions. My personal project sites are all hosted here on my Code Monster account, and although I’ve seen my share of downtime, I still love Dreamhost and recommend them to anyone looking for a hosting service for there personal projects.

    Seriously, show me a host out there that will give me 400+GB of storage and 5TB of transfer, with both growing weekly.

    And if you really get so bent out of shape because your blog is offline for a few hours…Welcome to the Web, stuff like that happens. If you can’t shell out a few hundred a month for business hosting, I guess your blog isn’t really that big of a deal anyway.

  33. 33
    Brandi Says:

    Hmm. Well, at least it’s not just me. I hope. Since it’s now October 31 at 10am Eastern and I *still* can’t get into the http://FTP... but at least everything else on my site seems to load. Slowly.

  34. 34
    zee Says:

    Fellow Dreamhosters,

    I recently purchased a dreamhost account. They are the only hosting company that allow you to host multiple domain names under one account.

    If you need non intensive(basic html and image websites) hosting and host about 300 domain names under one accountI go with dreamhost.

    If you’re in this business(developer, webmaster) I strongly recommend to shell out 160 a month and get a dedicated server for demanding projects or demanding clients.

    Companies I have used and recommend:
    Softlayer.com - I love their control panel
    servermatrix.com - Try their super celeron servers - very affordable

    I haven’t tried dreamhost dedicated hosting but since they outsource it it has to be good.

    -zee

  35. 35
    zee Says:

    I have to add.

    Dont’ do dedicated windows hosting over at servermatrix.com - you will regret it. Servermatrix does a better job with linux servers then windows.

  36. 36
    Raoul Says:

    I purchased a code monster package in the summer & found it to be to so slow & unresponsive that I abandoned all hope of running a fast busy site on dreamhost & use my account here for file storage only. You get what you pay for in life & once my years worth of hostings up I wont be resubscribing.

  37. 37
    matt Says:

    you know, i used to let the downtime slide off me like water on a duck’s back. i’ve been a DH customer since 2002, and i’ve experienced downtime since 2002, LIKE I WOULD ANYWHERE probably. i know this.

    however, this downtime since july IS REALLY STARTING TO SUCK. it’s constant. it’s irritating. it’s like CHINESE F*CKING WATER TORTURE. and yes, IT’S MORE OFTEN than would be experienced elsewhere for the same basic hosting service. i know this because if this happened to all cheap hosts, thousands of people would be pulling their hair out online — on fora, blogs, in mail lists — every few weeks. and that isn’t happening.

    sorry for the caps.

    they credited me a month. hmm. i don’t want the money, i want my sites to be up and to serve pages quickly.

    i love the DH features, but the downtime is really. kicking. my. users’. asses.

  38. 38
    Alfred_JP Says:

    my files are gone!?!

  39. 39
    nukelol Says:

    I’m basically using my plan as a file host, and for hosting a few sites for clans and what not. The downtime doesn’t get to me as badly, but of course, it seems like my file server (arrow), and MySQL server are fairly reliable. I haven’t had any (non-maintenance) downtime since the bout with bad luck in the summer.

    I really wish DH would make a post just explaining that, “Hey! We need to work on our infrastructure!” or something. I mean, It’s better that they just be honest about what the problems are.

    My guess is that some of the servers they’re using are using cheap hardware, and that the data center they’re using has very little space to grow/improve. So, more than likely, they need a complete overhaul of the systems they use, which would cause some major downtime for a day or so, but would be worth the consequential uptime that would follow.

    My thoughts……

  40. 40
    Marc Melvin Says:

    Guys… this is pretty darn scary… this is the result of me running a “df” command. That second number on the first entry… that’s the disk space usage… why is it -9223372036853337412? And 101% usage percentage? That’s great.

    I don’t really care about amigo, geena, mahimahi, or waller (I don’t think… might have to look at what my databases are running on)… but keep reading, and you’ll find a 100% usage on almost every partition. Geesh. I am slightly amused… but fairly annoyed at the same time. What is causing this craziness? My subversion commits are failing! Aurgh!

    10.3.100.153:/vol/boot/spunky/amigo
    116583712 -9223372036853337412 118022108 101% /home/.amigo
    10.3.100.125:/vol/boot2/spunky/amnesia
    168580632 168580632 0 100% /home/.amnesia
    10.3.100.125:/vol/boot2/spunky/basel
    168580632 168580632 0 100% /home/.basel
    10.3.100.125:/vol/boot2/spunky/caracal
    168580632 168580632 0 100% /home/.caracal
    10.3.100.124:/vol/boot2/spunky/debbie
    168580632 168580632 0 100% /home/.debbie
    10.3.100.125:/vol/boot2/spunky/emil
    168580632 168580632 0 100% /home/.emil
    10.3.100.125:/vol/boot2/spunky/fenne
    168580632 168580632 0 100% /home/.fenne
    10.3.100.153:/vol/boot/spunky/geena
    116583712 -9223372036853337428 118022092 101% /home/.geena
    10.3.100.161:/vol/boot/spunky/harrington
    1361606416 896886824 464719592 66% /home/.harrington
    10.3.100.151:/vol/boot/spunky/ingot
    1712515420 1430926064 281589356 84% /home/.ingot
    10.3.100.13:/vol/boot/spunky/jazzycat
    1161096480 843898944 317197536 73% /home/.jazzycat
    10.3.100.6:/vol/boot/spunky/kaska
    1361606416 1033427792 328178624 76% /home/.kaska
    10.3.100.165:/vol/boot/spunky/laser
    196152828 142132444 54020384 73% /home/.laser
    10.3.100.175:/vol/boot/spunky/mahimahi
    58291856 -9223372036788842788 124224876 101% /home/.mahimahi
    10.3.100.125:/vol/boot2/spunky/nebuchadnezzar
    168580632 168580632 0 100% /home/.nebuchadnezzar
    10.3.100.183:/vol/boot/spunky/olesko
    282016876 181955036 100061840 65% /home/.olesko
    10.3.100.125:/vol/boot2/spunky/papuchni
    168580632 168580632 0 100% /home/.papuchni
    10.3.100.124:/vol/boot2/spunky/quoddy
    168580632 168580632 0 100% /home/.quoddy
    10.3.100.124:/vol/boot2/spunky/ramona
    168580632 168580632 0 100% /home/.ramona
    10.3.100.125:/vol/boot2/spunky/samarra
    168580632 168580632 0 100% /home/.samarra
    10.3.100.183:/vol/boot/spunky/talita
    282016876 181955040 100061836 65% /home/.talita
    10.3.100.125:/vol/boot2/spunky/urbana
    168580632 168580632 0 100% /home/.urbana
    10.3.100.112:/vol/boot/spunky/waller
    282016876 -9223372036710313060 426479624 101% /home/.waller
    10.3.100.108:/vol/boot/spunky.localbackups/pico

Leave a Reply