Archive for October, 2006

Tank Puckered Out (Resolved)

Posted 1 year, 6 months ago (October 25th, 2006 at 2:09 am PST)

It put up a good fight, but in the long run, tank lost the war. It’s being moved over to new hardware now to fight another day. Check this space for updates.
- UPDATE Wed Oct 25 02:59:56 PDT 2006 -
Tank is back up on new hardware and is ready for battle! If you are still […]

Go Hardware Failure (Resolved)

Posted 1 year, 6 months ago (October 24th, 2006 at 2:51 pm PST)

A network interface card has failed on machine go, and we are working right now to switch everyone over to new hardware. We apologize for the inconvenience. We will have everyone back up and running within an hour.
Edit: Oct. 24 3:42 PM PST. Go and all its services are back up and running.

More Server Relocating (Complete)

Posted 1 year, 6 months ago (October 24th, 2006 at 2:17 am PST)

The following servers are going to joing their comrades in Alchemy (the datacenter across the hall):
kiki, limbo-postal, gap, medina, lira, yerba, dali, spree
Again, expected downtime is around 20 minutes. As always, check this space for updates.
-UPDATE Tue Oct 24 03:04:06 PDT 2006-
The servers have been moved and are all back […]

Server Relocation (Complete)

Posted 1 year, 6 months ago (October 23rd, 2006 at 11:54 pm PST)

The following six servers are being relocated:
ruble, io, zuul, slash, squirm, rhod
They are moving across the hall to our other datacenter. Total downtime will be around 20 minutes. Check this space for updates.
-UPDATE Tue Oct 24 00:49:30 PDT 2006-
Everything is back up and running. If you are having any problems […]

Wanna Fanta!

Posted 1 year, 6 months ago (October 21st, 2006 at 3:14 pm PST)

It looks like Fanta’s network interface fizzled out, so, we decided to restore it to new hardware.
All of the network configuration scripts are furiously working to restore it’s … pop … so we should have it back up and working soon! Sorry about that!
Happy-not-soo-bubbly-server-restore-team!

Fileserver switch troubles (Resolved!)

Posted 1 year, 7 months ago (October 19th, 2006 at 4:40 pm PST)

One of the switches which connects some of our fileservers is having unexplained high cpu load. This is causing slower than normal loading of some sites. We’re looking at the issues and hope to have it fixed up shortly. Updates to follow.

We think we have this one resolved. It seems the CPU […]

RAM Upgrades! (Completed)

Posted 1 year, 7 months ago (October 18th, 2006 at 9:49 pm PST)

Mayo, wilshire, and cognac are all going down for RAM upgrades now. Unfortunately they need to have their BIOS flashed as well, so downtime will be a tad bit longer than normal (around 20 minutes). Check this space for updates.
- UPDATE Wed Oct 18 22:10:59 PDT 2006 -
All three servers are flashed and back up […]

File server failure (Resolved)

Posted 1 year, 7 months ago (October 17th, 2006 at 3:11 pm PST)

One of our file servers failed about 30 minutes ago causing high loads and site inaccessibility for some of you. We are currently swapping in new hardware so everything should be online shortly. The disk shelf is fine so there won’t be any lost data.
We’ll post an update as soon as everything is back online.

We […]

Nelson is Being Moved to New Hardware

Posted 1 year, 7 months ago (October 17th, 2006 at 1:26 pm PST)

One of the network cards in the nelson server has failed. The server is being moved to new hardware now and will be back up shortly. Our apologies for the downtime.

File Server Moving!

Posted 1 year, 7 months ago (October 17th, 2006 at 9:24 am PST)

More File server moving. This is for Wednesday night, October 18, 2006 starting at 10pm sharp! Again, we have no easy way of saying who will be affected, however less than 10% of our total customer base.
I have an idea for how to make this go twice as fast, so hopefully it will be done […]