The site was down for almost twenty hours yesterday. Let’s start with the pathetic excuses: Our ISP had a broken cable that took a long time to repair.
Obviously one needs a backup ISP for this kind of situation. I recently asked a question about ISP failover on serverfault.com. There doesn’t seem to exist a foolproof and cheap way to do it. I’ll have to re-read the answers and try to learn some more. But I guess any kind of ISP failover is better than none.