This Knowledgebase article describes best practices for integrating Neverfail Heartbeat V4.6 or later with Symantec Backup Exec for Windows Servers 10d and contains advice specific to Topology #2.
If you are running an older version of Neverfail Heartbeat, please refer to Knowledgebase article #308 - 'Best Practices for Integrating Neverfail Heartbeat with Backup Solutions'.
Please ensure you have read the following Knowledgebase article before proceeding:
- Knowledgebase article #1172 - 'Best Practices for Integrating Neverfail Heartbeat V4.6 or Later with Backup Solutions'.
Deployment: Topology #2
Neverfail recommends installing and configuring backup software before installing Heartbeat, since this will eliminate the need to repeat the installation on the Secondary server.
- With a local SQL Server instance
- With a remote SQL Server instance
- With its own MSDE instance
Please read the “NOTES” in the appropriate sections below BEFORE proceeding with the installation.
Neverfail recommends using a remote SQL Server instance for Backup Exec to store its databases. This will reduce the complexity of the deployment and ensure that the environment is robust and easy to configure.
If a local SQL Server instance is used instead, Neverfail recommends that this instance is protected by Neverfail for SQL Server.
If there is not a protected SQL Server instance on the server pair, installing Backup Exec’s MSDE is recommended.
If Backup Exec is configured to use a local database engine (either SQL Server or MSDE) which is NOT protected by Neverfail for SQL Server, this database engine must be considered as an integral part of the backup software, rather than as a separate entity. All advice below about installation, configuration and service control applies to the database engine as well as to the backup tool.
Installing backup software on a server pair after installing Neverfail Heartbeat.
ASSUMPTIONS: This procedure assumes that the Primary server is ACTIVE, and the Secondary server is PASSIVE. If the Secondary server is active, please omit step 2 below.
- On the active or passive server, or a remote client, open the Neverfail Heartbeat Management Client.
- Perform a Switchover and wait for the Secondary server to become active.
- Shutdown Heartbeat leaving protected applications running. Please note that any changes in the protected data done in the time between steps 3 and 10 will be discarded!
- Install backup software on the Secondary server. If the installation process requires it, reboot the Secondary server.
- Configure backup software settings: devices, folders, etc.
- You must now set all backup software services to manual and include them in the start/stop scripts on the Secondary server. Please see section 'C' below for advice on how to do this.
- Shutdown Heartbeat; choose to stop protected applications.
- Set the “Neverfail Server R2” service to manual on the Secondary server. Change the role of the Secondary server to Passive, then change the role of the Primary server to active.
- Reboot both servers. The Primary server should now be active, and the Secondary server should be passive. Neither Heartbeat nor the backup software should start on the Secondary (passive) server.
- On the Primary server, shutdown Heartbeat (leave protected applications running).
- Install backup software on the Primary server. If the installation process requires it, reboot the Primary server.
- Configure backup software on the Primary server: backup devices, folders, etc. The settings on the two servers must be identical as far as possible.
- You must now set all backup software services to manual and include them in the start/stop scripts on the Primary server. Please see section 'C' below for advice on how to do this.
- Shutdown Heartbeat on the Primary server; leave protected applications running.
- Set the “Neverfail Server R2” service to automatic on the Secondary server. Reboot the Primary server and restart Heartbeat on the Secondary server.
- On the active or passive server, or on a remote client, open the Neverfail Heartbeat Management Client. On the System > Status & Control tab, check that the 'Application Status' is 'Started'.
- On the Primary-active server, open the Windows Service Control Manager (SCM) and check that all backup software services are set to manual / started
- On the Secondary-passive server, open the Windows SCM and check that all backup software services are set to manual / stopped
Note: if you are using Backup Exec’s own MSDE instance for Backup Exec to store its databases, and Neverfail for SQL Server is also installed on the server pair, you must set the MSDE services (MSSQL$BKUPEXEC and SQLAGENT$BKUPEXEC) to manual and add them to the SQLServer start/stop scripts.
If you are using a remote SQL Server instance for Backup Exec to store its database, you must now delete the BEDB database from the remote SQL Server, before proceeding with the Backup Exec installation on the Primary server. BEDB is the database where Backup Exec stores some of its data; this database will already exist, having been created during the installation on the Secondary server. Failure to delete the BEDB database from the remote SQL Server will cause the Backup Exec installation on the Primary server (step 12) to fail.
If you are using Backup Exec’s own MSDE instance for Backup Exec to store its databases, and Neverfail for SQL Server is also installed on the server pair, you must set the MSDE services (MSSQL$BKUPEXEC and SQLAGENT$BKUPEXEC) to manual and add them to the SQLServer start/stop scripts.
Installing backup software before installing Neverfail Heartbeat - the recommended deployment strategy.
Once you have installed and configured the backup software on the production server, please follow these steps:
- Set all the backup software services to manual /stopped.
- Install Heartbeat as per installation guide. Make sure Heartbeat and replication are started on both servers
- You must now include backup software services in the start/stop scripts on both servers. Advice on doing this can be found in section 'C' below.
- On the active or passive server, or on a remote client, open the Neverfail Heartbeat Management Client. Stop replicating (you may leave protected applications running) and then start replicating.
- On the System- > Status & Control tab, check that the 'Application Status' is 'Started'.
- On the Primary-active server, open the Windows Service Control Manager (SCM) and check that all backup software services are set to manual / started.
- On the Secondary-passive server, open the Windows SCM and check that all backup software services are set to manual / stopped
If you are planning to use Backup Exec’s own MSDE for Backup Exec to store its databases, and Neverfail for SQL Server will be installed on the server pair, you must set the MSDE services (MSSQL$BKUPEXEC and SQLAGENT$BKUPEXEC) to automatic/started before installing Neverfail Heartbeat. Neverfail will detect these services and protect/control them (so they will be automatically included in the start/stop scripts of the SQL Server AM).
Since these services will be added to the monitored services set, manually stopping Backup Exec’s services will trigger any actions defined within the Configure Monitoring application, ranging from simple service restarts (default), to switchovers. It is recommended that the default actions be left unchanged for Backup Exec’s services.
- Editing the Neverfail Heartbeat start/stop scripts.
This section describes the operation of editing the start/stop scripts. When editing the scripts, double check all changes since they will have a direct impact on the status of backup software and protected applications.
The start (start.bat) and stop (stop.bat) scripts for each application module can be found in the %installdir%\neverfail\r2\scripts\%application%\ folder, where %application% is the name of any protected application (e.g. FileServer, SQLServer).
The scripts must be edited to include start or stop commands for all the backup services which you want started or stopped. You must use the short name when referring to a service; commands controlling backup services must be included after all other commands controlling protected services.
The Start script should contain lines of the form:
NfNet Start "BackupService1" /R || set FAILED=1
The Stop script should contain lines of the form:
NfNet Stop "BackupService" /R || set FAILED=1
Deciding which scripts to modify to include backup software and the correct order of the services is based upon the following criteria:
- If there are service dependencies (i.e. backup software services depend upon protected services already included in the start/stop scripts), then each of these services has to be included in the same script (start.bat), after the parent service(s). A dependant service does not have to be added to the stop script (stop.bat) as it will be stopped when the parent service is stopped.
- Backup software services that do not depend on any protected services should be included in the start and stop scripts, after the protected services. It does not matter which start or stop script will include these services (i.e. you can add them to the start/stop scripts for SQL Server AM, or FileServer AM, or Exchange AM, etc.), unless there are logical dependencies (e.g. your backup tool will backup SQL Server data, or it uses SQL Server to store its databases, so it should be started after SQL Server is started), in which case you should take them into account. The order in which they should be listed depends on their own interdependencies: if there are two backup software services A and B, and B depends on A, then B should be included in the start script after A, but should not be included in the stop script.
- As a general rule, whenever possible, try to group all the backup software services in the same scripts.
Note: The Backup Exec settings (e.g. jobs, schedule, options, job history, etc.) which are stored in protected SQL Server databases (either MSDE or SQL2000) will be replicated to the Secondary server. This will ease the work necessary to keep the two servers identical, but will make the setting of independent backup options on each server more difficult (e.g. once a job is scheduled, it will be executed on either server, when active). Because of this, care must be taken when configuring scheduled tasks to run on one server: the other server must also meet all the conditions for the tasks to run, or the jobs will not be performed properly.
- Backup jobs should be configured so that they do not overlap with Neverfail Heartbeat tasks (for example, DRM shadow creation, Rollback operations, or Full System Check).
- Please ensure that you exclude the following directory from any backup job: %installdir%\neverfail\r2\log including subdirectories
- Please avoid scheduling a backup job during a full system check. Competition between file access between Neverfail Heartbeat and backup software will significantly increase the durations of both the backup job and the full system check task. This is especially important in environments with large databases (Exchange or SQL Server) where the time required to backup data is considerable.
- Any restore should be performed under careful supervision of a system administrator.
Restores must take place on the active server only. Neverfail Heartbeat should be stopped before performing any restore.
If prompted, reboot the machine in order to complete the restore. DO NOT start Heartbeat on the passive server yet. You now have the opportunity to examine the restored data (on the active server) without it being replicated:
- If, for any reason, you don’t want to keep the restored data, you have the option to shutdown Heartbeat on the active server (if it is running), manually switch the roles of the servers (using the Configure Server wizard), and then start Heartbeat and replication on both servers. This will bring the data on the server pair back to the state it was before the restore. Once the servers are synchronized, you can switchover to the pre-restore server pair configuration.
- If you want to continue using the restored data, simply start Heartbeat on the active (if required) and passive servers and allow the pair to get in synch.
- When restoring system state, service state, registry data, or any other selection containing the Neverfail installation folder, or folders which are vital to the operating system (for example, the Windows installation directory), a reboot is usually required. Following the reboot, Neverfail may detect an unclean shutdown, set the server role to passive, and not start – there will be two passive servers. Please refer to Knowledgebase article #984 - 'Resolve Two Passive Servers', for advice on how to resolve this situation. Also, there must be no cross-restores of such selections. A cross-restore is the restoration of data, which was backed up on one server in the server pair, and is restored on the other server.
Uninstalling backup software
When uninstalling backup software, if Neverfail Heartbeat is still installed on the server pair, you must perform the following steps:
- Shutdown Heartbeat (you may leave protected applications running).
- Uninstall the backup software on both servers and reboot the servers if required. If a reboot is not required, simply restart Heartbeat on both servers.
On both servers, edit the start/stop scripts so that any reference to services belonging to backup software must be removed before the uninstall process.
Both Neverfail Heartbeat task times and backup jobs times will increase significantly when they overlap; also, the overall performance of the server pair may be degraded.
Shadow creation tasks may fail when performed during certain phases of backup jobs, depending on the settings used. Even though this will not cause any significant problems, care should be taken when scheduling tasks on a server pair running Heartbeat.
Using Symantec Volume Snapshot Provider from Backup Exec 10d with Neverfail Heartbeat V4.6 and later may cause the backup to be marked as completed with exceptions resulting from Backup Exec failing to obtain the minimum required quiet time.
Neverfail Heartbeat V4.6 or Later