Posted (October 28th, 2008 at 9:17 pm PST) by Jason C
It would appear that the server “bugsy” had a few issues that required the loving attention of a member of our admin team. As it stands, the server is going thru a deep file system check that requires some close monitoring due to possible intervention. Once it is done with that, everything should hopefully come back up and return to normal. If that doesn’t happen, we’ll be sure to post an update here.
UPDATE: We have been working on Bugsy throughout the evening and we are at a point where we are working on repairing the disks. If that does not work, we’ll restore from backups and then repair the disks. Unfortunately, we have no ETA on this process, but we will update here as soon as we know more.
UPDATE 10/29 3:10p Pacific: Unfortunately we still do not have a solid ETA on a resolution for bugsy. We are however attacking the issue with both solutions listed above. We have a new bugsy up and we are copying data from our backups to this server. The new bugsy is the live server so you may notice that your files are missing. Do not fear, they are being copied from backups. We are also still trying to repair the disks on old bugsy. If that is successful then old bugsy will go back live. So we’re basically in a situation where the first solution to finish will be the one we go with.
Update 10/30 11:00a Pacific: The data being copied from backups to the new Bugsy is roughly 50% done. This unfortunately means that we are likely looking at more than 24 hours for the moves to be complete. As your files are moved your sites will be working, but we unfortunately do not have a way to prioritize the moves by account at this time. The disk repair on old Bugsy was not 100% successful unfortunately. The plan is to let the copy of the backups finish, then run another copy of data on old Bugsy to the new one. The backups we have are unfortunately 2 weeks old, so after we get that data up we will run the second copy in order to get your most up to date files. Sorry about the problems this is causing you. A free month of hosting will be added for customers on the Bugsy server.
Update 10/31 3:30p Pacific: The data moves from the 2 week old backups to the new server have finished. We are now starting the data move from old Bugsy to the new Bugsy. Those of you mentioning you out like to opt-out on the second restore have been set to skip. We are also not going to overwrite any files currently on the server if they have a newer date than the ones copying over. So your sites should be up now. Sorry about the problems this has caused you.
Comments: