Failover on Archibald to new hardware was a success. Sites are back up and stable. If your site is hosted on archibald and you see any errors / problems please submit a support ticket with as much detailed info as possible.
The web server Archibald is being failed over to new hardware. It had been crashy throughout the day with high loads . This shouldn’t take longer than 30 minutes, and it won’t affect email and mysql at all.
Sorry about the trouble.
After more tweaking and reorganizing today we’re finally seeing some positive results from the router upgrade last night. Load averages everywhere are dropping at the moment but we’re watching and waiting to see how things progress. This is just an update to let you all know that we’re still working on things and that [...]
Our router upgrade seems to have improved performance for some of you but we are still seeing a lot of complaints of little or no improvement at all. Now that we are confident the routers are not presenting bottlenecks performance-wise we will be focusing our attention on the specific areas of the network that [...]
It appears that the automatic update scheduler service wasn’t turned back on last night after finishing with the network upgrades. We turned them back on about half and hour ago, but it may take a while for them to catch up. This affects all new additions and changes made through the control panel, such as [...]
All up!
There are still some stragglers but at this point 99% of your sites should be up. We still have admins and tech support staff on hand to watch over things and fix problems as they arise. If your site is still down or you’re experiencing some other problem go ahead and submit a [...]
While probably 90% of all webservers and mail servers are up, there are quite a few straggling Apaches that are having to be restarted by hand. It looks like they didn’t play nicely with our runlevel changes earlier. There are only one or two on each server but for a couple dozen servers. We’re currently [...]
Core networking functionality has been restored. Web and mail servers are on their way up right now if not so already.
We’re working on cleanup of stray services and then we’ll post an “all up” message.
In addition to the recent router hardware replacement, we will be performing multiple upgrades to our primary core router at 10pm PDT tonight. This will entail the same amount of downtime, roughly 30-60 minutes, to the same services (HTTP, Mail, webpanel, SSH, etc).
For those of you interested, we will be installing 2 2500W power supplies, [...]
It seems our core01 router lost its BGP sessions this morning which caused a very brief networking issue for about half of our network. It already looks like everything is back online already and we’re not sure what happened exactly but we’ll post more information as we have it.
Comments: