Whopper getting DOS’d (Not anymore!)
It seems whopper is getting dos’d (possibly ddos’d). It is currently doing about 120k packets per second and totally saturating the uplink on the switch. I am working on isolating the source so I can drop it at our router and upstream if needed. More on this as it develops.
These servers are affected:
grizabella snowdrop lincoln fletcher gaz fairfax firestone paramount holt lifesaver whopper
–
All is better (mostly!) We have isolated the destination, however the source is pretty well distributed. Whopper got a whopper for a ddos. (I couldn’t resist…) For now there are a handful of domains which are offline while we investigate which one was the target, however the rest of whopper and the other servers named above should be OK now!
.
November 15th, 2006 at 1:55 am
Whopper is up again! Thanks!
November 15th, 2006 at 9:12 am
If you want to catch untargeted attacks and such, you can check out http://labrea.sf.net. Heh, most IPs that get stuck in it are usually from RIPE, it works rather nicely. For targeted attacks, I usually keep one port on a switch open and ready to mirror so I can analyze it with ethereal and use its nifty stats tools to figure out who is being naughty, then I block them.
November 15th, 2006 at 11:36 am
I actually just ran tcpdump on the affected server and then let ethereal have its way with it. We have some other tricks up our sleaves, such as sampling netflow off of our core routers. It turned out to be fairly well distributed (through RIPE…) so we just dumped the single target IP. As far as we know, the switches we use on the edge to connect our web servers don’t support snooping. Thanks for the suggestions though.
November 15th, 2006 at 9:27 pm
Would this have anything to do with the ridiculously massive amount of comment spam I started getting today on my blog? I usually don’t get any but today I left my computer for an hour and when I came back I had 80 notifications of comments.
November 15th, 2006 at 9:44 pm
You run dreamhoststatus.com?
November 16th, 2006 at 5:13 am
What? No, I’m just saying that this entry talks about getting a flood of incoming data, so I’m wondering if some of that data is in the form of comment spam, which would explain a problem I’m having with my blog.
November 16th, 2006 at 7:40 am
That, Sir, was a joke.
November 16th, 2006 at 8:46 am
My site is running slow as anything, whats going on?
People are having trouble even connecting now!
November 16th, 2006 at 10:04 am
My site has gone through periods of inaccessibility off and on for days; right now is one of those times.
November 16th, 2006 at 2:28 pm
I wonder what being dos’d means. Okay, I looked it up. I wonder why I would need to, that is, why it isn’t spelled out here for those of us who do not spend all of our days in server land, but just want a reliable ISP.
November 16th, 2006 at 3:32 pm
I’ve read that DOS attacks now carry a jail sentence in England now. Good.
November 16th, 2006 at 3:50 pm
I don’t work for Dreamhost, I’m just a customer.
I know availability of our websites are a priority, but the fact is that denial-of-service attacks are here to stay. DOS attacks are a thriving enterprise now, albeit a criminal one. At best, consider them the unavoidable natural disasters of the internet. At worst, consider them the equivalent to violent crimes.
I won’t move out of my house for the occasional bad weather or random criminal act, and as long as Dreamhost continues to actively combat network attacks, I’ll stick with ‘em.
Wanna see a worst-case bot-attack business scenario? Read this chilling news article (”How one company fought the new Internet mafia – and lost”) : http://www.wired.com/wired/archive/14.11/botnet.html
November 16th, 2006 at 7:14 pm
My site, http://www.university-bookstore.org was messed around with. I know it was working last night, and when I checked on it a few minutes ago, I encountered a configuration error. So now, I’m having to rebuild my site.
I’m doing a full backup of all my domains right now.
I’m on Cerritos.
November 16th, 2006 at 7:36 pm
Will someone(s) please confirm that my websites are either slow or do not render:
http://www.permian-mall.midessa.net
http://www.university-bookstore.org
http://forum.midessa.net
November 16th, 2006 at 9:38 pm
“I wonder what being dos’d means. Okay, I looked it up. I wonder why I would need to, that is, why it isn’t spelled out here for those of us who do not spend all of our days in server land, but just want a reliable ISP.”
Are you kidding me? Any other words you’re too lazy to look up that you need help with? I’m surprised you had enough energy to even post that.
November 16th, 2006 at 10:14 pm
Midessa - http://www.university-bookstore.org looks to be working well now - Yeah, I’m hours behind but no one else piped up.
November 16th, 2006 at 10:22 pm
why: ….oooh. Yeah, I suck.
November 17th, 2006 at 8:24 am
No, comment spam sucks. THAT should be a crime - along with DDOS attacks and stupid drivers.
November 17th, 2006 at 9:13 am
Is the grog also being smashed by dos? — My site is acting like a dead
November 17th, 2006 at 9:19 am
ALL of my Dreamhost sites are inaccessible right now. This doesn’t make me happy, esp because I JUST switched to them from another host!!!! ARGH!!!
November 17th, 2006 at 9:23 am
DC, what are your sites? server?
November 17th, 2006 at 10:02 am
I am on Dr. Pepper and it seems to be down as well. Just timing out on my site.
November 17th, 2006 at 10:09 am
Ditto on Dr. Pepper. The kicker is that I’m hosting a project that’s holding a vote on some project matters today…rawr.
November 17th, 2006 at 6:15 pm
I know “Fanta” isn’t listed but is anyone else having FTP issues? It gets to the password stage and then times out.
November 17th, 2006 at 11:04 pm
Well my website is on lifesaver and my site was hacked today.
November 18th, 2006 at 5:31 am
My site is on slice and its not working atm
November 18th, 2006 at 4:06 pm
glad its fixed
November 20th, 2006 at 9:20 am
Guys, are you still getting Dos’d here?
This is the fifth weekday out of the last seven that the server’s bugged out at around the same time of day - about 4pm over here (UK), which means about 8am PST.
This is all looking a bit uncoincidental to say the least and it’s all getting a bit frustrating as I’ve got a new project due to launch in about a week’s time and need to be sure that everything’s not going to be falling over every 24 hours before I go live.
November 21st, 2006 at 9:16 am
Whoper is down again?
Now I can’t access to my blog…
November 21st, 2006 at 11:34 pm
Since we have some techies here that seem familiar with DOS attacks, I’m wondering, what’s the most “foolproof” method of stopping them at the source? Seems to me in my completely perfect world that there should be some way that a server can spot an attack and dump a single targeted IP - if that’s the case - automatically. What do companies that no doubt get many more attacks - Microsoft or Yahoo or something tempting - do to counter the attacks, and can Dreamhost implement something similar, or is that super high-tech and not cost efficient. I’m asking because it seems like Dreamhost could save some money (and some headaches) by implementing something like this, if possible.
November 22nd, 2006 at 2:03 am
The problem is that this is a Distributed attack, so it’s much more difficult to block. Instead of all the requests emanating from a few nefarious IP addresses, they come from many many seemingly random IPs. Even though there are of course repeats, it’s not as simple as a single line in the htaccess file.
Check and see if your logs are bigger than normal. If so, you’ll probably be able to see at least a few of these in your error log.
October 25th, 2007 at 10:42 pm
Thanks you dreamhost for the great work done. Dreamhost Rocks!