In the second of a series of articles regarding backing up your organisation’s data and systems, we discuss the recommended best practices for backing up your organisation’s data and what makes good management discipline for ensuring that, if you ever need to rely upon them, your backups are always up-to-date and ready.
Backup best practices
As the core of any successfully managed IT infrastructure, backups need to be done right. Until needed they are nearly invisible and receive minimal budgetary and management attention. However, when needed, they can become the most important thing in computing.
Firstly - Backup your backups
Ironically, one of the most important things to back up is your backup software itself and the ‘catalogue’, the database your backup software generates of what it has backed up and when. You can still restore your data without this, but it becomes very much harder.
You can for example run through one of your backup tapes and inventory what is backed up on it and to see if what you want to restore is there, but this can take a very long time and time may not be on your side in an emergency. With the catalogue, you can browse the history of what you have backed up and identify and select the correct tape or drive etc to access to reinstate the resources and the version you require. In the event that you need to reinstall your backup server elsewhere to ‘start from scratch’, having a backup of your library of backed up data will be invaluable to seamlessly move operations to the new site.
Generally, it’s also a good idea to keep a virtual copy of the server that your backup software is installed on for archival purposes – you may not always use the same software package to carry out your backups, and you may need native access to the old system if you need to pull some data out of long term archive.
Getting started Dos and Don’ts
Starting out or replacing a backup system is a good time to take stock and ensure that you avoid some of the common pitfalls associated with setting up a reliable and future proof backup regime.
- Plan carefully what backup regime is most appropriate for your organisation’s needs. Determine the budget, level of acceptable complexity and the greatest acceptable window of disruption and data loss. This is data that is generated between one backup completing and the next backup running that will be lost. In a simple scenario where the backup and business continuity plan only include periodic copying of data and not continuously replicating data between resilient systems you will experience some data loss in the event of system failure or accidental data loss/destruction – you need to understand how much of this your business can tolerate
- Buy the best backup system you can afford, with the largest capacity considering the anticipated growth of your data. You need to think ahead over the life of the backup system to make sure it will have the required capacity to do the job it is being asked to do now and in several years’ time. Also, buying high quality mainstream equipment ensures that your backup system and your backup archive will be well supported over the timeframe during which you may be required to restore data from the backup media. E.g. buying a high-quality tape drive is important to ensure, for instance, that if you need to access these records in 5 years’ time there will still be hardware and software support to enable you to do this
- Buy high quality media for physical backups. The backup systems you choose will be used extensively throughout their lifespan, and you want to ensure that when you need to access the backups they have made that they can be relied upon to be intact and accessible even after some time has passed
- Make sure that your approach to backing up your systems and data is deliberate and considered and get advice and help as appropriate. Document the plan and communicate to all key non-technical stakeholders
- Clearly and simply document what is being backed up and the schedule you are using, and ensure that any relevant stakeholders are informed and that they inform you of any significant changes
- Do appoint a ‘backup’ backup person, to take over media management and log checking in your absence
- Split up larger backup jobs so that they run within a workable time-frame both now and as data grows
- Have the system email or notify you of any problems as well as successfully completing a backup job.
- Don’t make your backup schedules and routines overly complicated, and don’t allow your support team or support partner do this either. Proper backups require management oversight and involvement from the non-IT side of the business, so if you don’t understand what is going on you won’t be able to tell when it’s going wrong
- Don’t scrimp on your backup systems - they represent the investment that protects all of your systems, your data and your reputation. A good quality tape drive backup system with appropriate software, configured and managed properly can cost thousands or even tens of thousands, money which many see as wasted, especially as ‘it will probably never be needed’ - until it is. Then having a solid, dependable, well managed backup system is invaluable, at any price
- Don’t rely on anyone else to ensure your data is properly backed up. If the worst does happen, or a prior version of data is required, there little point in blaming others if you didn’t keep an eye on your backups to ensure everything was backed up regularly and correctly. Again, backups are as much a management as a technical issue.
Backup management Dos and Don’ts
Once you have a solid backup solution in place this is where the works begins, not ends. Backup management is an ongoing and largely thankless task, however, if or when they are needed you’ll be glad that you followed some sensible advice.
- Regularly audit what data should be backed up, and review this against changes on your network to ensure that no new items materialise which are missed from being backed up
- Document and communicate to all stakeholders what is being backed up and how - the system used to back up the data, who manages it and what is the greatest window of anticipated data loss in the event of a critical system failure
- Ensure that someone is responsible for ensuring that the backup process completes successfully every night, and that they log any anomalies and that these anomalies (such as backups not completing properly, not completing inside a reasonable time window or errors with the backup system) are reported upon and fixed. A backup system that isn’t monitored isn’t worth much
- Log the outcome of each backup, each night, and document any items of note. If a backup didn’t complete make sure you log the reason and what wasn't backed up. As soon as possible, re-run the backup job once you’ve addressed the issue preventing its completion to capture the missed data
- Ensure that the backups are regularly reviewed and tested to ensure that when required, they work and that data can be restored. This will involve doing periodic dry runs and test-restoring some real data (being careful not to overwrite the original source data in the process – restore to a test area)
- Consider the axiom that you don’t really have any backup at all until you have at least two copies of your data, and at least one of them is stored off-site well away from the physical location of the data and systems they represent
- Make sure your backup media (tapes, hard drives or whatever you store your backups onto) are stored securely off-site in an appropriate environment. There are many companies which specialise in the regular secure collection and storage of backup media. Cloud backups are by default already stored ‘off-site’ in that the data moves straight out of your office and into your service providers data-centre
- Consider all the data protection options as a complementary suite of protection, not as either/or choices: tape, cloud, replication, mirroring, clustering, redundant systems, distributed systems, co-location etc. The size and shape of your backup solution ultimately depends on your budget and criticality of access to data and sensitivity to downtime, but some companies we’ve worked with thought nothing of having several backup strategies in place as well as mirroring key systems off-site for instant fail-over. It doesn’t have to break the bank, but when considering the cost of downtime or lost data many professional organisations make the investment in cast-iron supporting systems
- Take for granted the skills or diligence of your IT Support team or IT Support partner - too often I’ve heard so called IT professionals use phrases like ‘set it and forget it’ and ‘simple-setup zero-admin’ when it comes to things like setting up and running backup systems. This is usually to influence purchase decisions and dumb down what is a critical system in order to make life easier for themselves. Often, they may not actually have the knowledge required to do it right. If you aren't comfortable that one of the most important systems in your arsenal is being given the care and attention that deserves, then find people who will
- Forget to familiarise yourself with the backup software – don’t let the first time you use it be in anger. Carrying out test restores is the ideal way to find your way around the package - most backup software systems have a lot in common, and none are that scary. If they are overly complicated or unreliable it’s time for a change
- Forget to review and audit the backed-up data whenever there is a change to your systems - new hardware, administrative changes, changes in permissions on folders, adding systems or shared network folders can all introduce changes to your network and data which may be missed by your backup routines unless ‘backup thinking’ is incorporated into every change you make to your systems
When dealing with a system that interfaces with all of your data at once, great care needs to be taken to ensure its integrity and safety.
- Find a reliable data destruction company to destroy your media properly when it is time to retire them and ensure they certify its destruction. If you think about it, putting your old backup tapes in the bin is essentially giving away all your businesses data in one neat little package, when you have tried so hard to protect and secure it in other areas
- Do encrypt your backup tapes such that even if they are lost or stolen, they are all but useless to anyone who acquires them that shouldn’t have them. This is usually fairly straightforward to configure in any standard backup software but incredibly isn’t enabled usually by default. This may entail a slight performance hit and backups may take slightly longer to complete, which may need to be factored into your backup plans if you are backing up a lot of data and timeframes are tight, but unless there is a good practical reason why it is not turned on then you should enable encryption
- Create a policy document outlining what your compliance and GDPR/data protection responsibilities are, what data is backed up and what happens to the media. Detail what happens in the event of the data expiring, and who needs to sign off and log its destruction. Ensure all this is done in cooperation with your GDPR/Data Protection officer
- Ensure the backup media are physically secured from unauthorised access.
- Use the default ‘Administrator’ account when setting up the backup system to access the companies network and resources. Use specially created ‘service accounts’ instead. When the backup software needs to access the data and resources on your network to make a copy to back them up, it will require an unfettered level of access to read this information. A common mistake is to use either the main ‘Administrator’ account - the most powerful account on the network - or to use the user account of the person in charge of backups or whoever set up the software. This can lead to several problems most notably the account having more security access than is necessary which is a serious security risk and a risk should the software ‘go rogue’ and decide to start deleting data. Also, should, as good password policy dictates, the password for the account change then unless it is updated in all the places it has been used then it will lose access to the network resources it is trying to back up. The solution is to use a dedicated user account created just for this purpose and no other, enabling fine grained control of the permissions available to the account and reassurance that the password won’t be inadvertently changed. Of course, you will be reviewing the backup jobs every night to ensure they complete successfully, so even if this were to occur you’d catch it straight away
- Give people access to the backup software that don’t need it. Ensure that the computers it runs on and the drives it uses to create backups are physically secured.
Good backup discipline can be hard to keep up, especially if your primary systems are well maintained and complacency has set in. The things that can go wrong are often out of your control however, and disaster rarely announces itself in advance. If you get into the good habits of backup now it will pay enormous dividends in the future.
One way or another, whether it be accidentally deleting an important file or a huge water leak in the office that ruins your servers, at some point you will need to rely on your backups.
We can assist with the setup, configuration, auditing and management of all good backup systems and work with on-premises backup solutions, Cloud based backup systems and hybrid options offering the best of both.