Archive for the 'General Outages' Category

Delay in quarantined junk mail delivery

Posted 1 week, 2 days ago (May 3rd, 2008 at 11:47 pm PST)

We have noticed that messages that were quarantined by our junk filter are not getting delivered to the Junk Folder. We checked on the messages, and they are still on the servers. The reason they weren’t getting delivered was due to a connection problem to the mysql database that controls the Junk Folder. The problem […]

redhot getting new hardware

Posted 1 week, 3 days ago (May 2nd, 2008 at 12:06 pm PST)

Redhot has had a hardware failure and needs to be replaced. We are currently working on moving it over to new hardware.
This should take about 30 minutes, and we will keep you posted as things progress!
Update: 1:06pm pacific: the failover has completed and services are up and running. If you experience any issues, […]

Mysql Maintenance

Posted 1 week, 4 days ago (May 1st, 2008 at 9:47 pm PST)

These mysql servers will be experiencing some downtime tonight for maintenance:
jake
leo
midnight
snarf
Downtime is expected to be less than 30 minutes. It should happen shortly after midnight PDT.
–Update
The maintenance was preformed and the servers were back up within 15 minutes.

Dr pepper moving to new hardware.

Posted 1 week, 4 days ago (May 1st, 2008 at 7:27 pm PST)

There’s an issue with the hardware on dr pepper. It is being moved to new hardware now.
Update–
Things should now be working properly.

Loony Mail issues.

Posted 1 week, 4 days ago (May 1st, 2008 at 11:06 am PST)

I am opening a new post for you guys - we did install an additional file server for your cluster but the hardware proved to be faulty and that’s why you haven’t seen the improvement that we had anticipated. We have our storage specialist at the data center today working on this and we are […]

Planned outage for mysql’s toto rover dogbert dino beethoven bootsy camille jaws

Posted 1 week, 5 days ago (April 30th, 2008 at 9:42 am PST)

On the night of Tuesday May 6th we’ll be replacing one of the network switches that serves the following mysql servers:
toto
rover
dogbert
dino
beethoven
jaws
camille
bootsy
spacey-vsql01
frisky-vsql01
We hope to be able to do this one server at a time with only seconds of downtime for each, however we’re posting this in case that’s not possible and we need to bring all […]

Mysql server Sanspoof having problems

Posted 2 weeks ago (April 28th, 2008 at 11:17 pm PST)

We are currently experiencing problems with the mysql server “sanspoof” crashing. The issue is being looked into at this time, we apologize for any trouble and will update you as soon as there is more information regrading this problem.
UPDATE 23:22:43 PDT
It appears that the memory in this server is causing it to […]

Spunky cluster mail issues

Posted 2 weeks ago (April 28th, 2008 at 2:49 pm PST)

If you’re on the spunky cluster (check the name of your mail cluster under Account Status in the upper right corner of the control panel), you may have been experiencing delays in sending and receiving mail for the past couple of hours. The mail database for the cluster is having issues, and it’s being worked […]

Webserver flamenco being moved to new hardware

Posted 2 weeks ago (April 28th, 2008 at 12:15 pm PST)

The web server flamenco didn’t come back after a multitude of reboots, so we are migrating it to new hardware. It should be back up within an hour. Sorry about the inconvenience this is causing you, and please check back here for updates.
Update: Some networking issues caused flamenco to be down for longer […]

Blingy File Server Issue

Posted 2 weeks ago (April 28th, 2008 at 9:21 am PST)

We are currently experiencing issues with the main file server for the blingy cluster. Our file server admins are on their way to the data center to look into the issue, and we are doing our best to get this fixed up and working again as soon as possible. This issue will cause websites and […]