Problems caused by apache service updates
Posted (May 6th, 2008 at 11:18 am PST) by andreaWe have noticed that any apache service updates initiated from the control panel are breaking the service. This includes anything that has to do with the domain’s web service, like adding a new domain, changing FTP users on domain, etc. This doesn’t just break the actual domain that initiated the change, but all domains on that apache service.
We are fixing this as we find them, and trying to catch up with the ones that broke a little while ago. Very sorry about the web downtime this has caused you, and please check back here for updates.
Uptdate 5/6/08 5pm Pacific: This issue is now resolved. Sorry about the delay, we have reconfigured all apache services for which the firs fix didn’t take. If you’re still having problems, then it’s probably unrelated. Please write to support with some details, and we’ll be happy to check into it for you.
79 Responses to “Problems caused by apache service updates”
Same as with Gabriel. I haven’t made any updates. 500 Internal Server Error.
I made changes, but I didn’t want to start all this, I swear!
@1:
This pattern is exactly the failure I am experiencing. This appears to have started at about 9:30AM PST according to the error logs. I first got 404 errors and now it’s throwing 500 errors. The logs are now showing this when I attempt to hit any websites:
[Tue May 06 11:18:50 2008] [error] script ‘/home///web/internal_error.html’ not found or unable to stat
[Tue May 06 11:20:58 2008] [error] failed to open log file
[Tue May 06 11:20:58 2008] [error] fopen: Permission denied
[Tue May 06 11:20:58 2008] [error] [ Premature end of script headers: php5.cgi
Aside from zero response for the ticket I put in an hour and a half ago, the part that’s most frustrating is that we have a dedicated account. Like you, we made no changes to our dreamhost configuration today, so how did this just break?
Something I’ll have to talk to support about I suppose.
yay right when I buy a 30 dollar ad campaign on a few sites for the day….
My sites broke too… I expect them be back soon.
By the way… Trying to access by FTP I got “Incorrect password” on all the sites on my account I’ve tried.
Me too. All my databases are down. So now I am just wondering how long this is going to take for them to get it fixed.
Haven’t faced any probs yet. No problems in accessing my http://FTP.
MySQL works slow, but fine, same as http://FTP.
Also didn’t change anything. 404.
The requested URL /cgi-system/php5.cgi/index.php was not found on this server.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
Keep getting a broken pipe while trying to upload to http://ftp. Related?
Another oddity. There is a site we registered when we setup our DH account. That site still works. This is the same account and server that has failing sites on it. It might be on it’s own unique IP, but I’m not sure.
While the initial post didn’t seem to affect my problem at all, reading the comments shows that we are indeed experiencing this, it seems.
“Not Found
The requested URL /cgi-system/php5.cgi/index.php was not found on this server.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.”
This comes up whenever attempting to access the forum I run. The website runs fine, however.
Ahh, actually it appears it’s any site that uses PHP that fails. Static HTML sites seem to work fine.
Doesn’t anyone read the announcement before they post here?
“This doesn’t just break the actual domain that initiated the change, but all domains on that apache service. ”
So all it takes is for someone whose site is running off the same Apache service to change something, and it will break your site too.
I don’t think that you can relate the Apache service to anything else, like the account name or the server name, so this is going to seem random at best.
Look ma’! DownHost is down again!
Ah, thanks for the notice. I’m guessing this is why my site is suddenly not working. I thought it was my connection being iffy.
Hopefully it will be fixed soon~
Great stuff, sent out an email for banner advertising last week, got two paying banner ads now and allready they see a website down and then the adserver isn’t working anymore when the site is back up.
Please get my site back up asap. It’s been down for too long! This happens too often.
Ok my site is restored and running again, but cant login to my http://ftp...
Also having password issues. Is this part of the problem???
THIS IS RIDICULOUS.
I cannot believe this is happening. WHEN will my service be online again????
FIX THIS OR I AM MOVING AND DEMANDING A REFUND
I’M A LOUDMOUTHED ASSHOLE THAT HAS SHIT FOR BRAINS.
Does this mean it’s safe to create a new domain if your server appears to be working? Or not? If not, when will it be safe?
I’m back up. yeah.
Lets hope they fixed the root cause so someone doesn’t go into apache and cause everything to stop working again.
Christ, spoke too soon *JINX*.
Seems they are restoring files, but forgot permissions…
I havent touched the panel and I am getting a 500 error.
no updates on my end, but still getting 500 error on php files. html files accessed just fine.
password error on ftp, too.
i, too, am beginning to question the quality of hosting here at dreamhost. it seems our email or website experiences issues on a weekly basis. anyone know where to find the average uptime?
waiting the fix
Password error on http://FTP. PHP Sessions can’t be instanciated because there’s is no write permission on /tmp/ folder.
The few files I serve here are fine, but FTP is down (which is what caused me to come here, to see if this was a known problem, which indeed it appears to be!). Not quite sure how apache crashing is causing FTP login errors, but such is life.
Ugh, what’s the deal? We have ads running like crazy and no one can get to us. Please fix this SOON! We’re losing money by the minute!
the same like all
400 and 500 errors, I didn’t make any update from some time ago, my error log show ” Invalid query name 1″ message around 30 times
Apache slowness also effects speed of my sites. Anyone interested in a little personal history can check the following forum post over at our forums: http://www.terminatorfiles.net/forum/viewtopic.php?t=436
Hope to see this resolved asap… as always!
Same as everybody else, all domains offline, no ftp, 500 Internal Server Error. Help!!
my site is down and i haven’t touched anything on the panel. this problem comes only a month after we were down for about three days because of the blingy cluster issue. i’ve never had a host that had so much downtime before and it’s a real pain the ass.
GET IT FIXED NOW!
No http://FTP. Typical. Please fix before I get back from fridge.
COME ON guys, it’s been several hours. You’ve replaced the Apache config error problem with a permissions error problem. Now users on my site are greeted with :
Warning: require_once(/home/.nikademus/belli/ianbell.com/wp-config.php) [function.require-once]: failed to open stream: Permission denied in /home/.nikademus/belli/ianbell.com/wp-blog-header.php on line 16
Fatal error: require_once() [function.require]: Failed opening required ‘/home/.nikademus/belli/ianbell.com/wp-config.php’ (include_path=’.:/usr/local/php5/lib/php:/usr/local/lib/php’) in /home/.nikademus/belli/ianbell.com/wp-blog-header.php on line 16
Me too. Same as everybody else, 500 internal server error. Please help.
Dear Dreamhost: Here’s the mail that you refuse to receive, and tell
me that I have to send thru the web panel:
You totally blew it on your apache updates that you just announced you
started.
You have messed up the php apache interaction/paths.
$ w3m -dump http://radioscanningtw.jidanni.org/
Warning: require_once(./LocalSettings.php) [function.require-once]: failed to
open stream: Permission denied in /home/.yoshie/jidanni/mediawiki-1.12.0/
includes/WebStart.php on line 86
Fatal error: require_once() [function.require]: Failed opening required ‘./
LocalSettings.php’ (include_path=’.:/usr/local/php5/lib/php:/usr/local/lib/
php’) in /home/.yoshie/jidanni/mediawiki-1.12.0/includes/WebStart.php
on line 86
Same for my other two wikis.
So…I would say that for $10 a month we shuld expect issues like global outages every 30 days…and then I read the following on the dreamhost.com “About Us” page…THIS IS SOME FUNNY SHIT!!!!
The Bottom Line
We love it here and we’re all focused on hosting. It’s what we do. It’s what we’re good at.
BWAHAHAHA!!!!!! HARHARHARHARHAR!!!!!!! OH HAHAHAHA!!!!!!!!!!!!!!
You guys ROCK at DOWNTIME!!!! Your metrics look like the inside of a fucking cave - everything is pointing down…
Plus I wish to note again that Dreamhost turns HTTP 500 (error) into
HTTP 200 OK
$ HEAD http://radioscanningtw.jidanni.org/
200 OK
In the logs the only clue that something would be wrong is the smaller
byte transfered numbers, as error messages use less bytes.
Too bad Dreamhost does not read these comments.
All my sites are down…
Warning: require_once(/home/.muppet/rbarros/ci9.com.br/blog/wp-config.php) [function.require-once]: failed to open stream: Permission denied in /home/.muppet/rbarros/ci9.com.br/blog/wp-blog-header.php on line 16
Fatal error: require_once() [function.require]: Failed opening required ‘/home/.muppet/rbarros/ci9.com.br/blog/wp-config.php’ (include_path=’.:/usr/local/php5/lib/php:/usr/local/lib/php’) in /home/.muppet/rbarros/ci9.com.br/blog/wp-blog-header.php on line 16
Warning: require_once(/home/.moxie/eversite/everlinux.com/blog/wp-config.php) [function.require-once]: failed to open stream: Permission denied in /home/.moxie/eversite/everlinux.com/blog/wp-blog-header.php on line 14
Fatal error: require_once() [function.require]: Failed opening required ‘/home/.moxie/eversite/everlinux.com/blog/wp-config.php’ (include_path=’.:/usr/local/php5/lib/php:/usr/local/lib/php’) in /home/.moxie/eversite/everlinux.com/blog/wp-blog-header.php on line 14
cant access my sites of sftp..
Can someone at Dreamhost please update us as to what’s going on? It’s been nearly FOUR HOURS!
this is not funny anymore..
arghhh
i’ll use this time to look for a better hosting
Only one of my websites is experiencing this issue (went down 10 minutes ago) and I haven’t touched the control panel today.
So permissions are currently goofy… but it appears the actual service is working, non?
—
Warning: require_once(/home/.witness/browne/andrewbrowne.ca/wp-config.php) [function.require-once]: failed to open stream: Permission denied in /home/.witness/browne/andrewbrowne.ca/wp-blog-header.php on line 16
Fatal error: require_once() [function.require]: Failed opening required ‘/home/.witness/browne/andrewbrowne.ca/wp-config.php’ (include_path=’.:/usr/local/php5/lib/php:/usr/local/lib/php’) in /home/.witness/browne/andrewbrowne.ca/wp-blog-header.php on line 16
All sites down that are with dreamhost. Don’t know how long its been. But all 500 errors, and I didn’t touch anything today. I guess this brings down the downtime averages…..waaaaa
From my Apache logs:
[Tue May 06 14:13:23 2008] [error] [client 84.92.201.33] Premature end of script headers: php5.cgi
[Tue May 06 14:13:23 2008] [error] [client 84.92.201.33] (13)Permission denied: exec of ‘/home/knightnet/knightnet.org.uk/cgi-bin/php5.cgi’ failed
[Tue May 06 14:13:23 2008] [error] [client 84.92.201.33] Warning: SuexecUserGroup directive requires SUEXEC wrapper.
…. REPEATED LOTS ….
[Tue May 06 14:13:23 2008] [error] [client 84.92.201.33] Premature end of script headers: php5.cgi
This is repeated several times
Last successful request was at 09:25:08 PST
I have a sub-folder of my main site configured to use PHP as an Apache module (PHP4) - THAT WORKS OK!!!!
I can’t tell whether this is indeed the problem as my main sites all rely on PHP5.
Regards, J.
Ah, yes I do have a site running PHP4 as a module complete with MySQL access AND IT WORKS!!
So it looks like a problem with running PHP via the CGI interface.
I’m on dawber (ftp/http) and fluffy (sql) running dynamic PHP4 pages and get no errors. I had some “flakyness” with ftp two days ago, contacted support and they replied next day that the machine is up and running smoothly, so the problem isn’t at the source it’s in-between, so they had me send them traceroute results. Haven’t heard back.
Not sure if all of this is related…
databases are down on a few sites. I ended up putting up a note replacing the index.php file with a Scheduled Maintenance note for our users. Just so they don’t hammer me with questions about the two errors that show up. : ) yes, i did copy the index.php file that was initially there prior to replacing : )
Email is wonky. slowed down quite a bit. Email works great for all sites with MX records pointing elsewhere. my basic html pages load fast with no issues.
I also will look for better hosting or use the CustomDNS from DynDNS.ORG for a dedicated home server.
Hey! They canceled all my Rewards this week! I published my promocodes at a brazilian forum and they considered it illegal by a operator that cannot read brazilian portuguese. hehehe.
fyi, my mysql server is chimchim
my email servers affected are randy and postal
php 5 with extra web security and another site running php 4 with extra web security (both blogs get the same error)
also ftp works fine on all my sites
postal mail server is offline.
Looks like my site is a back up.
Now, I can neither read a simple email.
Congratulations, DH. Since my first day, problems and more problems.
This is a very bad service. I’m losing money!!!
I lose some dolars every hour you are down. Who is gonna refund this? Are you going to refund at least for the time you are not offering the service and we are paying you??
I’ll change to another hosting.
fuck you
so? any update?
you’ll keep us without new information for how many more hours?
DH Uptime = 72,2% maybe?
Just slowness on my end.
I have been with these guys for 5 years, and it seems like over the last year or so there have been an increasing number of problems. Downtime, slow response, the billing debacle earlier this year. Now this.
You guys are really dropping the ball here. I run an e-commerce site and am losing money by the minute. Get your act together.
my mysql databases are working now. email still wonky. but did notice there is another post on email load balancers. thanks for such fast response dreamhost : )
Now what are you doing, downgrading everybody to PHP 4?
$ HEAD http://radioscanningtw.jidanni.org/
Server: Apache/2.0.61 (Unix) PHP/4.4.7 …
I see, it is an eclectic mix of 4 and 5.
$ set http://radioscanningtw.jidanni.org/index.php?title=Special:Version
$ { HEAD $@; w3m -dump $@;}|grep PHP
Server: Apache/2.0.61 (Unix) PHP/4.4.7 …
X-Powered-By: PHP/5.2.3
PHP 5.2.3 (cgi-fcgi)
Well at least it works, but slow.
Says “RESOLVED”
Isn’t.
WTF?
hurry up guys, I can not create posts. it is payment day, there are so many queue jobs to write 
Eventhough this is marked as “Resolved,” it’s obviously not. It’s always one thing or another, and the whole “be patient, while we find out what the deal is” is getting old. There’s so much downtime that it flat out annoying to even bother with. After the whole billing twice thing, and the server upgrade in March (that caused my site to be unstable to the point where you can’t even view it for more than a few hours), I thought that Dreamhost has finally got their act together, but seems like more problems are pilling up.
Will considered to find a different host if this keeps up. Btw, if sites are still not responding so well to whatever you guys did to fix it, don’t mark it as resolved, since people are still complaining about downtimes.
My sites are up, but FTP is sure still broken. Really damned annoying.
Here’s what resolved it for me:
Go into the control panel, go to “manage domains”, for each affected domain click on “edit” in the “web hosting” column. Don’t change anything on the next page, but do “save changes”. After about 5 minutes, the problem goes away.
Worked for me, anyway.
My site is *still* down. 500 error on wordpress/index.php .
And all of a sudden a major images dir has become completely unacessible, making my ebook store look like a skeleton in dr Frankenstein’s dungeon.
Today I won’t be selling any ebook this way. Yet I have to pay for something that has been “resolved” but apparently still sucking like a hoover.
Things still appear to be messed as none of my hosted slide shows are running.
Also the database supporting my blog at http://www.timkbrown.com seemed totally messed — even after the so called resolution. Restoring the database ’seemed’ to have resolved it.
In the meantime, I’m majorly fussed about my brand new slideshow website being dead in the water.
One of my users has been getting the Internal Sever error upon logging in
FYI, I didn’t make any changes in my panel, and experienced the 404, and now a 500 error. So I’d say that any panel updates from any user (maybe limited to my server) cause this problem.