This Knowledge Base article provides information about EVault Remote Disaster Recovery Out of Disk behavior.
Active Server CommsMgr Logs
The send queue on the active server is used to store intercepted data prior to sending across the channel to the passive server. Send queue build-ups here indicate communication problems with the Secondary server, or insufficient bandwidth for the data being replicated. The queue stats are displayed in the EVault Remote Disaster Recovery Management Client on the Data: Traffic/Queues page.
These updates are stored in memory or to disk in the default location ( C:\Program Files\EVault\Remote Disaster Recovery\r2\log ). The maximum size on disk is configurable, MaxDiskUsage, is by default 10GB. Both these settings can be configured via the Configure Server wizard when EVault Remote Disaster Recovery has been stopped.
The queue will be written out to disk if the active server is replicating and:
- The passive server was never connected.
- The channel suddenly disconnects and the configured number of heartbeats is very large.
In either case, the following could happen:
- The MaxDiskUsage will be reached and NFChannelExceededMaxDiskUsageException will be logged.
Exception in CommsMgr [L9] Exceeded the maximum disk usage(NFChannelExceeded MaxDiskUsageException)
- If available space on the drive is less than the MaxDiskUsage then NFChannelIOException will be logged.
Exception in CommsMgr [M4] Cannot open log file 2004-09-07-203.log(NFChannelCannotOpenIOException) because there is not enough space on the disk (IOException)
In both these situations, EVault Remote Disaster Recovery will:
- Cease to log updates to the data.
- Discard all existing logs.
- Upon channel reconnection, a SCAChannelLostMessageEvent will be generated.
- A Full System Check will be initiated to get the system back in sync.
Passive Server CommsMgr Logs
When data is received on the passive server, it is stored in the receive (safe) queue of the passive server until Apply is ready to update the protected file. Depending upon system load, this will either be in memory or to disk. Under normal operating conditions, this queue should remain small. The receive queue stats are displayed in the EVault Remote Disaster Recovery Management Client on the Data: Traffic/Queues page.
A build up in the queue may indicate a problem applying updates to the protected files. Common causes are:
- Hardware / software problems with the disk subsystem.
- Under spec'd equipment, for example disk drives on the passive server are far slower than the active server disks.
- Applications running on the passive server blocking updates.
When this queue gets very large, the protected application will begin to slow down to avoid overloading the passive server with updates. If the configured limit is reached, EVault Remote Disaster Recovery will raise a NFChannelExceededMaxDiskUsageException on the passive server. The EVault Remote Disaster Recovery server should be shutdown opting to leave the application running on the active server. The application's performance will return to normal.
The passive server's hardware should be investigated for problems.
- Start by checking the Windows Application and System logs for Errors and Warnings regarding impending hardware failure, or other problems.
- Device Manager may show problems with drivers or RAID controllers malfunctioning.
- Alternatively, run system diagnostic checks that are supplied with the hardware.
Passive Server Lacks Protected Disk Space
This occurs when the active server has more disk space than the passive server. Protected data cannot be written to the passive server, so updates will fail. Apply will raise a Disk Full or Quota Exceeded exception in the log saying that it cannot create files. The system will attempt to stop.
"Error","Disk Full Or Quota Exceeded","[N27]Failed to write information for the file: D:\protected\some file.txt to the disk. Either the disk is full or the quota (for the SYSTEM account) has been exceeded ."
Employ the following to prevent Out of Disk Behavior:
- Ensure that the available disk space on a drive exceeds MaxDiskUsage. The default configuration requires 10GB of disk space on the EVault Remote Disaster Recovery installation drive.
- The passive server disk space should equal that of the active server.
Consider hosting the EVault Remote Disaster Recovery log directory on its own disk, or one that does not host:
- Protected Application data.
- The Windows System folder.
EVault Remote Disaster Recovery All Versions