It seems like only yesterday I was writing an article about the merits of cloud computing, storing your precious irreplaceable data and photos online. At the time it was topical as the Victorian Black Saturday fires had devastated parts of the state. Many treasured memories were lost forever. I thought about online storage and backups once again when the floods and then cyclones ripped through Queensland recently.
Then the fires in Perth, closely followed by flooding in New South Wales, Tasmania and parts of South Australia. As I write this now, Darwin has a cyclone on it’s doorstep and there is another hovering off the Western Australian coast.
In times like this you can’t avoid thinking about your own data and then your clients data.
As a reseller and solution provider, you and I are responsible for giving our clients the best possible advice and giving them the piece of mind that their data and hence businesses are safe. If your client looses access to their server and accounting data for one day, what does it cost? Do they send home their staff on paid leave (with workcover levy, superannuation, entitlements)? Does the client loose their incoming money stream and outgoing billing systems? Will the business survive?
This is why we put in hardware and software redundancy solutions. We would be insane not to recommend some form of redundancy in solutions and then push the topic as hard as possible. There is no such thing as too much redundancy. If the client can’t afford it, you need to find something else to remove or substitute into their solution, to make it more affordable. The client can not be allowed to choose a solution with less redundancy, at least not until you have explained all the options. A cheaper solution might be needed but a less redundant one is dangerous. The customer must go into this discussion and select the solution after they completely understand the options.
Recently, I was placed in the unfortunate situation where a clients high end server failed. They had an uninterruptible power supply, hardware RAID, redundant fans, daily image backups to USB drives, a secondary backup off to a network shared USB drive, Volume Shadow copy and a firewall backed up by a good industry antivirus. On top of this they had separate backups of their accounting databases and the server was a Hewlett Packard reliable server. We had tried to protect this client from everything short of an earthquake, fire or a thief with a fork lift.
Here is where it all went wrong. The client did not want us to monitor his backups, opting for self-monitoring. He had a gut feeling a week earlier that something was wrong with the backups but proceeded to use the same hard disks for the following five days. These drives were being blanked as the backup started, and then the servers image was not being copied to the disks. Then two capacitors bulged and leaked. The electrolyte dripped down onto the 64 bit PCI slot and then blew the tracks from the network card. The server then powered down. Looking at the hard disks in the raid, they no longer detect. The capacitor fault seems to have caused an internal power surge and has damaged the hard drives. The backup to the network attached drive was not a complete server backup, just some key files.
I am left with a failed server, failed backups, failed hard disks and very little means of recovery. The client has staff that cannot work and now has the added delay from me, as I need to order and receive parts for a new server. When this server arrives, I have to start building it from scratch. I can’t do a bare metal restore. The recovery process is not going to be simple nor complete.
Truly, you can never have too much redundancy. You can try to allow for everything that can possibly go wrong, still something will be left out of your risk analysis and come at you from an unexpected angle. As resellers and solution providers we need to sell our clients on the merits of monitoring. The merits of taking backups offsite and the merits of adding additional weekly/monthly backups to devices that are then not overwritten for long amounts of time. We need to convince them that we need to put in the redundant power supplies, fans, hard disks and procedures. I have always presented extended warranty options and supply trusted and tested equipment.
The best you and I can do is prepare for the 95 per cent of what we know can go wrong and assure the client we have covered all the angles to our best ability. We are the professionals and they trust us.
If my business is flooded or catches fire, I have trust in the systems we have in place. I trust the rotated drives (with one offsite at any time) and I have been through the testing of our backup software to make sure I can rebuild a server from the files it produces. I monitor the backups daily and have my staff also review them. If my server fails, I have contingencies in place to extract out the accounting database and also the email stores. If I need to wait for a replacement server, I have backup office processes in place. Why should my clients be any different?
Michael Jenkin is a director of Business Technology Partners. You can contact him by emailing firstname.lastname@example.org