Database server problems, riffraff (Update on backups)
Posted (January 25th, 2007 at 10:25 pm PST) by mikehThe database server “riffraff” is having hardware problems. We’re working to get them figured out and resolved and get it back online as quickly as we can.
Watch this space for updates.
Edit 10:15pm pst by Jamesh: I’ve just finished putting a new raid card in Riffraff and it’s up and working now, if you’re having any problems with your databases please contact support.
Edit 10:15am pst by Kelly: It seems there was some data corruption on the live disks due to the funky raid card. I moved the backup drive out of the old server and into a spare machine. The backups are copying over and should be available shortly via the web panel for restores! The data has been moving for about 20 minutes now, it should finish in about 15.
84 Responses to “Database server problems, riffraff (Update on backups)”
Now my Wordpress is telling me it can’t connect to the DB. Have all issues been resolved?
All my sites are down. This is not acceptable. Is not mysql server on a cluster? Please be hurry. I started to lose my customers.
Taner you are an idiot, how could you set up a database cluster without running the risk of losing data. And if you are losing “customers” then why don’t you get your self a dedicated box in several locations.
how to know my database server name?
@Brad - You might want to tone down your responses a bit. It is perectly easy to set up a database cluster without losing data. You simply have shared storage and have the servers fail over to each other - all the major UNIX vendors allow this sort of system.
But there’s no way a low-cost provider like DH is going to do this, because it involves extra cost.
@SILENCE - Look under “MySQL Status” on the web panel. The server names are listed there.
At this moment I cannot access even the Dreamhost homepage.
I cannot access my site and the Dreamhost’s sites too.
I’ve been asking around and it seems Brazil has been shut out of Dreamhost. Friends in Europe have no problem.
If you use a proxy you can access everything.
Jesus… everyone has an idea, and no clue how to impliment it.
Do your thing, guys. I know how much of a pain this is. And this is from someone who’s forums are down.
My site is offline, and I dont get access to http://www.dreamhost.com also….is it a problem with Dreamhost or a Brazilian Network problem?
wft
i’m trying to access my website from costa rica, so it seems like the problem is a bit more widespread.
I cannot access even the Dreamhost page too.
PS: I from Brazil too. =/
I believe it is a problem with Embratel Network.
In my discussions lists I ask whos is having problem with Dreamhost access and everybody that is having problem is below Embratel Network.
(GVT is also below Embratel Network)
Mas Douglas, (vou usar português, ninguém do suporte vai ler mesmo)… O meu provedor tem ligação com a Telemar. Será que a Cicarelli tem dedinho nisso? uhashudsahusa, tudo é culpa dela agora e a Dreamhost é um ninho de blogs danado. =P
Enfim, e agora o que a gente faz? Eu tentei acessar por um proxy (proxydream.com) e o site carregou na hora. =/
Douglas, probably it’s not only with Embratel Network. Telemar Network (Velox) also have this problem.
I’m using it and can not access even site from Dreamhost.
Mas o problema não parece ser com a Dreamhost já que com proxy consigo acessar.
I´m not getting to access anything.
FROM BRAZIL TOO =/
What can I do?
this needs to hurry up.
Tb não estou conseguindo acessar nenhum site da Dreamhost… não sei como consegui acessar esse aqui. Deve ser algum problema no link aki do Brasil… tomara que volte logo
Ontem houve um problema com um link da Intelig…acredito que seja algo com isso
It’s a private party or everyone can come in? =)
Incrível, eu achei que seria o primeiro brasileiro a tentar comentar sobre os servidores da dreamhost no blog de status e já tem uns 8 fazendo bagunça aqui. Uau. =P
Acredito que seja um problema de propagação de DNS, mas enfim, palpites são palpites.
My server’s name is Sobe and it’s offline too.
I’m from Brazil too, same problem. Traceroute below:
[ademar@optimus ~]$ traceroute dreamhost.com
traceroute to dreamhost.com (66.33.201.141), 30 hops max, 38 byte packets
1 tlm23.ras.dialuol.com.br (200.221.25.96) 624.090 ms 627.865 ms 623.994 ms
2 assinantes-gw.ix.uol.com.br (200.221.25.1) 620.102 ms 619.933 ms 619.993
ms
3 ir4-border2.ix.uol.com.br (200.221.30.22) 672.001 ms 676.445 ms 615.638 ms
4 200.187.142.21 (200.187.142.21) 795.890 ms 795.974 ms 759.996 ms
5 200.223.254.138 (200.223.254.138) 797.404 ms 796.585 ms 791.510 ms
6 PO12-0.ARC-RJ-ROTD-03.telemar.net.br (200.223.131.138) 931.893 ms 927.981 ms 928.013 ms
7 so-6-2-0.ar2.MIA1.gblx.net (208.51.142.233) 943.991 ms 939.994 ms 923.973 ms
8 * * *
9 * * *
10 * * *
…
I get a whois of 208.51.142.233, last step in traceroute…
Search results for: 208.51.142.233
OrgName: Global Crossing
OrgID: GBLX
Address: 14605 South 50th Street
City: Phoenix
StateProv: AZ
PostalCode: 85044-6471
Country: US
ReferralServer: rwhois://rwhois.gblx.net:4321
NetRange: 208.50.192.0 - 208.51.255.255
CIDR: 208.50.192.0/18, 208.51.0.0/16
NetName: GBLX-6D
NetHandle: NET-208-50-192-0-1
Parent: NET-208-0-0-0-0
NetType: Direct Allocation
NameServer: NAME.ROC.GBLX.NET
NameServer: NAME.PHX.GBLX.NET
NameServer: NAME.SNV.GBLX.NET
NameServer: NAME.JFK1.GBLX.NET
Comment: THESE ADDRESSES ARE NON-PORTABLE
RegDate:
Updated: 2002-10-14
RTechHandle: IA12-ORG-ARIN
RTechName: GBLX-IPADMIN
RTechPhone: +1-800-404-7714
RTechEmail: ipadmin@gblx.net
OrgAbuseHandle: GBLXA-ARIN
OrgAbuseName: GBLX-Abuse
OrgAbusePhone: +1-800-404-7714
OrgAbuseEmail: abuse@gblx.net
OrgNOCHandle: GBLXN-ARIN
OrgNOCName: GBLX-NOC
OrgNOCPhone: +1-800-404-7714
OrgNOCEmail: gc-noc@gblx.net
OrgTechHandle: IA12-ORG-ARIN
OrgTechName: GBLX-IPADMIN
OrgTechPhone: +1-800-404-7714
OrgTechEmail: ipadmin@gblx.net
# ARIN WHOIS database, last updated 2007-01-25 20:10
# Enter ? for additional hints on searching ARIN’s WHOIS database.
Hi folks, I am from Brazil too and I can´t access http://www.dreamhost.com / achilles.dreamhost.com / panel.dreamhost.com using Velox(from Telemar) but I can access it from a dedicated link from Embratel.
I´m getting crazy.
If someone could tell something to me my contact is:
einetinternet at gmail dot com
Thanks all.
Ralph Liebessohn
A friend of mine can access dreamhost servers w/out problems from the Brazil Telecom Network (Curitiba - PR). I can confirm from his traceroute that his traffic doesn’t pass through the gblx network, so the problem appears to be isolated at this point.
This is my traceroute from Embratel that works.
716 ms
3 ebt-g7-0-dist04.bhe.embratel.net.br (200.244.164.10) 4.879 ms 3.796 ms 3.750 ms
4 ebt-p9-1-core01.spo.embratel.net.br (200.244.140.242) 12.296 ms 12.229 ms 40.618 ms
5 ebt-p3-0-intl01.spo.embratel.net.br (200.230.0.141) 11.651 ms 11.798 ms 11.959 ms
6 ebt-so-1-0-1-intl01.mia6.embratel.net.br (200.230.3.30) 122.615 ms 129.952 ms 118.674 ms
7 POS2-0.IG5.MIA4.ALTER.NET (63.65.188.221) 118.889 ms peer-so-2-1-0-intl01.mia6.embratel.net.br (200.167.0.10) 124.251 ms 125.301 ms
8 0.so-2-0-0.xl1.mia4.alter.net (152.63.84.170) 122.002 ms 124.852 ms 125.933 ms
9 0.so-5-0-0.XL1.ATL4.ALTER.NET (152.63.86.113) 135.623 ms 138.582 ms 139.227 ms
10 0.so-6-0-0.BR1.ATL4.ALTER.NET (152.63.86.169) 135.642 ms 135.711 ms 139.642 ms
11 so7-1-0-2488m.ar4.atl1.gblx.net (64.208.110.89) 191.078 ms 164.778 ms 156.058 ms
12 pos3-1-2488m.ar2.lax2.gblx.net (67.17.95.158) 213.010 ms 209.271 ms 212.876 ms
13 ge1-gx.dreamhost.com (67.17.162.162) 193.240 ms 193.451 ms 194.135 ms
14 achilles.dreamhost.com (64.111.122.18) 207.635 ms 208.149 ms 208.879 ms
Even the dreamhost main website isn´t working. I´m from Brazil too.
Alguém já tentou contato com a Anatel? Estou com o mesmo problema, não acesso sequer a página da Dreamhost.
Somebody already tried contact with the Anatel? I am with the same problem, not access at least the site of the Dreamhost.
FROM BRAZIL TOO
Ok, now it’s officially a party. Bring your traceroute’s and let’s dance! =D
Hey, I am from Brazil too, and my sites doesn’t work too, when I access in a Telemar link.
In CTBC, i don’t have problems in access any things.
Embratel, this works fine.
Sergio,
Anatel doesn´t care about that.What internet provider do you use?.
I think this is a problem in a server inside USA not in Brazil.
But I cannot find which server/router is having problems.
If somebody could access dreamhost panel and star a support ticket for our problems that will be nice. You can also send the link for these posts too.
Thank you.
Let´s try to solve it, all my sites are down and people are dying for it!.
Ralph Liebessohn
Pessoal, acabo de mandar a mensagem abaixo para a DreamHost pelo painel de controle.
—8
Ralph, desculpe discordar, a Anatel precisa resolver, é órgão público e TEM que funcionar, nós pagamos impostos para isso. Eu liguei e eles só poderão tomar qualquer atitude após protocolo na Telemar. Peço a todos os usuários que reforcem a reclamação, liguem para a Telemar e peçam protocolo para o problema, posteriormente falaremos com a Anatel.
Ralph, forgives to disagree, the necessary Anatel to decide, is public agency and HAS that to function, we pay taxes for this. I bound and they only will be able to take any attitude after protocol in the Telemar. I ask for to all the users who strengthen the claim, bind for the Telemar and ask for protocol for the problem, later we will speak with the Anatel.
Hi people.
Do you recommend Dreamhost? I’ve had dreamhost but only for minor things… now I’m about to launch a DB extensive website… is their service worth it ? (thinking about getting the cheapest plan…)
What the heck is wrong now? Everything is down! Even the dreamhost webpage. LOL!
Yep, can’t even access the Dreamhost webpanel. Firm my MySQL system went down, and now this…*sigh*
aqui voltou =]
Now my site is a bit slow but now I can access it. Idem to others sites from dreamhost.
It’s fixed to me.
I never knew there were so many people from Brazil here!
Global crossing is having problems, another ISP has posted a status update about this.
Tom -
Apparently you can see that Brazil is having some issues getting through Global Crossing to DH network. But if you’d like to ask specific questions about dreamhost, you can sign into the discussion forums and hear both the good and the bad. http://discussion.dreamhost.com We’re all users there and will gladly tell you what we really think.
I hope Brazil’s connectivity issues are solved soon.
Why…
I had 2 minutes of downtime today. OMG I AM LOSING CUSTOMERS
I HAD MISSION CRITICAL MEDICAL RECORZ HOSTED AND TEH SERVER IS DOWN
joke
POxa, pensei que ia ficar off o mês todo essa B**** 
e ae galera do brasil
aqui já está tudo beleza
pra quem curtir existe um fórum (não oficial) brasileiro
http://www.dreamhost-brasil.com
vamos fazer parte lah ;]
abraços
I lost my youthful innocence yesterday. I do believe some downtime was involved. But I’m smiling.
It explains why yesterday 9am to 10am (11 am to 1pm UTC), Google Analytics reported zero visitors. I was wondering what happened but here is the answer.
Isso explica pq ontem das 9h às 10h da manhã (hora de Brasília), o Google Anlaytics não computou uma visita sequer. Estava imaginando o que tinha acontecido e agora encontrei a resposta!
I just keep wondering why pacman keeps having sympathy reactions with all this downed equipment…
ED affects millions of men worldwide.
why: So does VD
selma server is down? i cannot acces my domains.anyone with same problem?
True!
teh servar
I just setup new email accounts and when I try to login through webmail.domain.com I get the following message:
ERROR
Error connecting to IMAP server: mail.lklaw.net.
0 : Success
Anybody know how to fix this? (Also, the emails I send to those accounts are immediately blocked with:
This is the Postfix program at host spunkymail-a5.g.dreamhost.com.
I’m sorry to have to inform you that your message could not be be delivered to one or more recipients. It’s attached below.
For further assistance, please send mail to
If you do so, please include this problem report. You can delete your own text from the attached returned message.
The Postfix program
: host lklaw.net[208.97.185.13] said: 554 : Relay
access denied (in reply to RCPT TO command) )
———– anybody, please help me if you know how to . thanks! -Johnny
Are you running your own DNS? Did you contact support if you use their DNS?
so basically there is a problem with something at their servers at the moment?
and I was wondering why none of my newly setup pages work
Want to improve your blood pressure?
Take MySQL Outages + Internal Server Errors daily
—-
Yesterday was a day with (leaving for a while the MySQL Outages) the fewest number of visits at my website since I joined Dreamhost.com
There are two reasons, the first, it’s holidays here, and the second, crawlers are degrading my site due to several internal server errors (ggl PR decreased by 1)
Yesterday, less than 500 visits and less than 1000 pages. Also, please, take into consideration that most of this pages were HTML cached to avoid DH’s CPUs think (this is another juggling I’ve recently implemented to keep my website up and running most of thetime). Then, we can consider the effort to generate the half (less than 500 dynamic PHP pages.
Nevertheless, internal server errors still happening.
What’s going on in DH servers? I can´t believe DH shared services can´t manage less than 500 dynamic pages in a day without giving internal server errors.
I’ve heard from support that a new resources control software was recently installed (more strict). I guess some tunning in it’s setup should be made.
I’ve also received a recomendation from DH about taking a look to my app, in order to improve resources. I did. Caching the half of dynamic pages into static HTML for a whole day for anon users, dropping minor important functional options, dropping dynamic blocks (due to HTML cached pages this blocks has no meaning), simplifying menues, dropping pictures, etc, etc.
Nevertheless, internal server errors still happening.
Another recomendation, moving to a dedicated (¿¿ for less than 500 dynamic pages daily??). This is $350 divided by 15000 pages = US$ 0.02 per page. Crazy! Insane!
I think that memory and CPU usage restrictions should be in the DH offering before sign.
Does someone know which IP corresponds to each server frindly name, rifraf, mickey pluto, minnie, batman…. etc?
Hi, around 10:20am EST I started having extremely slow performance from my installation of MediaWiki and then MediaWiki completely lost contact with mysql (whichever server is at 208.113.131.227) about 5 minutes later. Seems to be back up now, but just adding my notes in case the issue isn’t completely resolved.
Hmmmmm…. This morning I was able to get to my email, but now I’m not. None of them (I have two separate plans with DH and about five different domains).
Sometimes the routers are the problem, not the servers. DH is a small outfit with big offerings. A lot of times I suspect the dregs of the internet abuse DH because it’s so cheap, and so problems like this are constantly making DH look bad when all they’re trying to do is their best.
Now if I were to do something on the internet with mission critical data or absolute need for more than 95% uptime, then I’d…. shoot myself. Doesn’t matter which “host” provider you use, if you want to put mission critical data on the internet be prepared to not be able to access it from time to time. As for uptime, then I’d have to say the only way to get better uptime than a host is to learn how to host myself and then I’d see what kind of hassles are involved. All of them, not just the “me me me” stuff from clients expecting me to solve their internet problems.
Having problems to log on my email accounts.
I am using SquirrelMail v1.4.9a
The error is:
Error connecting to IMAP server: mail.orientemediovivo.com.br.
110 : Connection timed out
Can someone check it, please?
yep, mail seems dead for me… panel is SLOOOOOW must be all the support calls going in. Hey Dreamhost - Please open up an outage on this website when stuff breaks!
Also… if dreamhoststatus.com never goes down; please host my site on this server
splendid idea bubba
I am on clamato and some websites work,
others dont. kind of random :7
@Gustavo either do a nslookup on servername.dreamhost.com or go to http://nwtools.com and enter your hostname or servername there.
my mysql is working off and on and everything is really slow…
1.-In the last 2 weeks my panel was very slowly..
2.-All my databases are down.. When you gonna fix this problem??
3.-Why don’t you send us a mail telling us ’bout your technical problems? Hey!, We pay for your services…
tomoe sql server is down… ughhh…
wiki is down as well. DH starting to suck…
just restarted my DNS system. and mail and database…
And dreamhost just move servers and office and people.
Hope you get up to speed. and not repeat 2006 outages all over again.
Hi, everyone!
My websites worked. They are WordPress-powered. Suddenly, they stopped working without my intervention at all:(
What I see now on top of the websites is:
WordPress database error: [Got error 28 from storage engine]
SELECT DISTINCT * FROM wpen_posts WHERE 1=1 AND post_date_gmt
How do I know my MySQL server?
What is this ERROR 28?
Yesterday my website was a bit slow, maybe.
Today, the MySQL doesn’t work 
How do I know my MySQL server?…
I contacted DreamHost support, now waiting…
Let’s hope the 2006 outages won’t be repeated…
UPDATE:
MySQL is back again… What happened?…
(How do I know my MySQL server name? I didn’t find it anywhere in my Dreamhost control panel…)
Thx
yep, mail seems dead for me… panel is SLOOOOOW must be all the support calls going in. Hey Dreamhost - Please open up an outage on this website when stuff breaks!
Wow, this is distressing to see, is it resolved?
how long will it take?
what is the problem with hardware?
Nice work dreamhost! Great Hosting!
Thanks
Now if I were to do something on the internet with mission critical data or absolute need for more than 95% uptime, then I’d…. shoot myself. Doesn’t matter which “host” provider you use, if you want to put mission critical data on the internet be prepared to not be able to access it from time to time. As for uptime, then I’d have to say the only way to get better uptime than a host is to learn how to host myself and then I’d see what kind of hassles are involved. All of them, not just the “me me me” stuff from clients expecting me to solve their internet problems.
thank you
hmm not database problem
tomoe sql server is down
thanks ypu
Leave a Reply
Comments posted here may not be viewed by DreamHost staff at all. Please note that this is not a way to contact DreamHost.
the site is just slowing down… takes a long time to load. external websites are fast in comparison.