Technology - the "never have only one of anything" principle

So I wanted to take a look at our website before I went off to bed last night and discovered that the PeerFX website was down.

Weird, I thought. The contract guaranteed 99.9999% of the time the site would be up (this is a reminder for all of us to look DEEP into the fine print, I'm sure somewhere in there is a provision that says they will base this on a "best efforts" scenario). So I thought, oh well, it must be that there's this short lapse in our server hosting; woke up this morning and went online to check again 6:45am. Still not up.

I'm definitely concerned now - since our business comes in usually during the morning hours. My biz dev guy should be even more concerned because he's on the hook for a sales quota and burning up his entire day with a down website.

I send a couple emails and call the owner, letting them know that this is essentially a whole business day gone down the drain for us. Of course I get the usual apology and the "whole team is on it" explanation but it's been another 2 hours now and still nothing. Well, I guess not nothing, since we know that the interruption is caused by hardware breakdown. BAD. Just form a customer perspective, a hardware breakdown seems really bad because it takes much more time to move our stuff in the box over to another one and get the system back up again. Nothing really has been done to reassure me of an actual progress timeline as to when the site will be up again, besides the guarantee that it will be back up as soon as humanly possible.

Hopefully that's in the next couple minutes.

This is a point in time where I wish I had a backup server ready to go for the front-end site at a snap of my fingers. Another lesson learned on the "never have only one of anything" principle.