When the Active Server Fails, Under Certain Situations Split-brain Detection may Prevent Failover

Follow

Summary

This Knowledgebase article provides information about how under certain situations, split-brain detection may prevent failover when the active server fails.


More Information

In the following environment, split-brain detection will by design prevent failover to the passive server and protect data integrity when the channel fails. This will prevent the passive server from becoming active with an already active Primary server.

Scenario

Primary Server
192.168.1.1

Secondary Server
192.168.2.1

Split-brain detection on the Secondary is set to monitor the following IP Address 192.168.1.1 and the Primary is configured to monitor 192.168.2.1.

Exception

If a BSOD, or other situation occurs on the active server where the principal (Public) network adapter is still responding but Windows has 'crashed', a failover will not occur.

Impact: Clients cannot access the protected application, the Secondary server is still passive, and the Primary is displaying the BSOD.

Neverfail Heartbeat Split-brain detection can still ping the Primary Network IP address, as the NIC is still contactable on the network despite the BSOD. Failover will not occur because the passive server cannot use the active servers IP address as the IP address is still in use on the network. This is by design and manual intervention is required.

Recovery

To recover from a Windows server “Blue Screen” where:

  1. The network IP address is still visible on the network.
  2. Neverfail Heartbeat is configured to check the principal (public) IP for Split-brain avoidance.
  3. Failover has NOT occurred.

The following manual steps are required to restore client connectivity to the application. Data integrity has not been compromised by a Split-brain syndrome.

  1. Shutdown the Primary-active failed server. Do not restart, as the Primary server will return as passive and Neverfail will shutdown if there is no active-passive configured servers present.
  2. The main network IP address will no longer be visible on the Network and Neverfail Heartbeat will initiate a failover to the Secondary passive server.
  3. Observe the Secondary server as it becomes active on the network. The protected application should start normally if all other dependant services are available.
  4. Network clients should now have access.
  5. Unplug the Primary server from network and start the server.
  6. Confirm that Primary is now passive. Neverfail Heartbeat by default starts as passive following a system failure.
  7. Re-Connect the network cable and allow Neverfail servers to Verify and Synchronize.
  8. Initiate a switchback to the original Primary-active and Secondary-passive mode once Primary server has completed Verify and Synchronize and confirmed as operational.

Safe Guards

To help eliminate the downtime in this scenario, the alerting system should be configured to warn administrators that the channel has disconnected.


Applies To

All Versions


Related Information

None

KBID-240

0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.