This Knowledgebase article provides information about Windows Perfmon.
Neverfail Engine provides integration with Perfmon at three junctures (or components) and these are reflected in three separate Perfmon "Performance Objects". These performance objects and their corresponding components are:
- Heartbeat Server Performance - NFServerR2.exe
- Heartbeat File Interceptor - nfdrv.sys
- Heartbeat Packet Filter - nfpktfltr.sys
It is highly recommended that you do not start Perfmon for any purpose until after Engine is started and has begun replication. In fact, you should not run Perfmon during a Engine installation. If this advice is not followed, you may experience some difficulty obtaining performance counters, which are retrieved from the kernel-based performance objects, namely 'Heartbeat File Interceptor' and 'Heartbeat Packet Filter'. Furthermore, kernel-based performance objects are only available on Windows 2003 and above.
When you start Perfmon, you will notice a "plus" sign in the middle of the taskbar just above the performance graph. This is used to add 'counters', which are selected from discrete performance objects. Thus, to access Heartbeat-specific performance counters, you must use this button. After pressing it, you will be presented with an 'Add Counters' dialog box. Within this dialog, use the 'Performance Object' dropdown list to select the particular Heartbeat performance object you require. As discussed below, not all Heartbeat performance objects will be available at all times and some performance counters are only available on either the active or passive server.
To see the 'Heartbeat Server Performance' performance object when you start Perfmon, you must provide the "/wmi" switch on the command line:
When used in this fashion, Perfmon will not present the 'Heartbeat File Interceptor' or 'Heartbeat Packet Filter' performance objects. To see these two particular performance objects, start Perfmon normally after Engine has started and has begun replication.
If you have difficulty finding a particular performance object see the FAQ section at the end of this note.
Heartbeat Server Performance
The 'Heartbeat Server Performance' performance object displays counters exported from the Neverfail Server R2 service (NFServerR2.exe). Some of the counters are specific to an active server, some are specific to a passive server, and some are common to both. The 32-bit counters currently available include:
- MegaBytes Sent - Number of Megabytes sent across all channels
- MegaBytes Received - Number of Megabytes received across all channels
- Current Throughput - Current throughput via the CommsMgr in MBits/sec
- Maximum Throughput - Maximum throughput via the CommsMgr in MBits/sec
- Messages Sent - Individual messages sent by CommsMgr
- Messages Received - Individual messages received by CommsMgr
- Safe Update Queue Size - Number of bytes in CommsMgr Safe Update Queue
- Unsafe Update Queue Size - Number of bytes in CommsMgr Unsafe Update Queue
- Safe Update Queue Drain Rate - Number of bytes drained per millisecond from CommsMgr Safe Update Queue
- Unsafe Update Queue Drain Rate - Number of bytes drained per millisecond from CommsMgr Unsafe Update Queue
- Oldest Safe Update Queue Entry - The age of the oldest Safe Update Queue entry in milliseconds
- Oldest Unsafe Update Queue Entry - The age of the oldest Unsafe Update Queue entry in milliseconds
- Stubborn Files - CommsMgr implementation specific counter
- Referenced Files - CommsMgr implementation specific counter
Unsafe Update Queue Size – In normal operation, a low value for this counter is good. If this value continues to grow in size it may indicate, a problem applying updates to the passive server. The highest value for this queue size can be configured and will depend on the environment Heartbeat is intended to protect.
Megabytes Received – This counter can be used to determine how many Megabytes have been replicated to the passive server. If this number is low or remains at zero it could indicate a problem receiving replicated data on the passive server.
Heartbeat File Interceptor
The 'Heartbeat File Interceptor' performance object displays counters exported from the Heartbeat file interceptor driver (nfdrv.sys). The file interceptor driver is only operational on the active Engine server, thus its counters will not be available on the passive side of a server pair. This performance object, available on Windows 2003 and above, provides the following two counters, which can be used to track the memory resource that is being dynamically consumed by the file interceptor driver. The 32-bit counters currently available include:
- BufferNonPagedBytes - the number of bytes of non paged pool in use in the interceptor buffer
- BufferPagedBytes - the number of bytes of paged pool in use in the interceptor buffer
Heartbeat Packet Filter
The 'Heartbeat Packet Filter' performance object displays counters exported from the Heartbeat packet filter driver (nfpktfltr.sys). The packet filter driver is operational at all times on both the active and passive server so its counters are always available (on Windows 2003 and above). This performance object is mainly used to look at fined-grained performance details of the Engine channel traffic. In order to do this, the packet filter driver must be enabled on the channel NIC(s). This entails a certain performance hit so it is advisable to disable the packet filter driver on these NIC(s) when your Perfmon session does not require these counters. There is some overlap with the counters provided by the 'Heartbeat Server Performance' performance object. This mitigates the fact that 'Heartbeat Packet Filter' performance object is only available on Windows 2003 and above. Also, if the user’s interest lies in the overlapped region, that performance object can be used in preference to enabling the packet filter driver on the channel NIC(s). However, the kernel-based source of these counters can be extended to provide instrumentation not amenable to NFServerR2.exe. The 32-bit counters currently available include:
- Interceptor Bytes Sent - Count of bytes sent by file system filter driver. This includes verify, synchronize and interception traffic
- Interceptor Bytes Received - Count of bytes received by file system filter driver. Same qualification as above
- Proxy Immediate Bytes Sent - Bytes sent by ProxyManager in immediate mode
- Proxy Immediate Bytes Received - Bytes received by ProxyManager in immediate mode
- Proxy Delayed Bytes Sent - Bytes sent by ProxyManager in delayed mode
- Proxy Delayed Bytes Received - Bytes received by ProxyManager in delayed mode
- Control Bytes Sent - Bytes sent by Controller
- Control Bytes Received - Bytes received by Controller
- Heartbeat Bytes Sent - Bytes sent by heartbeat mechanism
- Heartbeat Bytes Received - Bytes received by heartbeat mechanism
- FSM Bytes Sent - Bytes sent by FileStateManager
- FSM Bytes Received - Bytes received by FileStateManager
- Total Bytes Sent - Bytes sent by all components\
- Total Bytes Received - Bytes received by all components
- Total Throughput - Throughput in MBits/second
- NFMsgs Received - Messages received over the NF Channel\
- NFMsgs Sent - Messages sent over the NF Channel
Heartbeat bytes received – This counter could be used to determine if Heartbeats are being received. If this value continues to be 0 beyond the preconfigured time for sending Heartbeats, then they may not be reaching the passive server and further troubleshooting is then required.
Total Throughput – This counter can be used to determine if the channel connection is meeting current expectations. This value will vary depending on the speed of the physical connection and the amount of data being replicated at the time. Typically, a high value indicates a high throughput of information.
FAQ - Why can't I see Heartbeat Performance Objects? If you can't see performance object 'Heartbeat Server Performance', you most likely have started Perfmon without the "/wmi" switch. If you can't see performance objects 'Heartbeat Packet Filter' and 'Heartbeat File Interceptor', you may have accidentally started Perfmon with the "/wmi" switch.
Alternatively, you might have forgotten that these counters are only available on Windows 2003 and above. Another reason for not seeing these two performance objects is due to a Microsoft bug. To get around this problem, you must close the Perfmon 'Add Counters' dialog and re-enter it a second time after which these performance objects ought to be available. If you still can't see these counters, follow the procedure described below (as recommended by Microsoft):
- Exit Perfmon and stop the 'wmiapsrv' service by typing "net stop wmiapsrv" at the Windows command prompt.
- Go into the registry and delete the value HKLM\Software\Microsoft\WBEM\Providers\Performance Data.
- Restart the 'wmiapsrv' service by typing “net start wmiapsrv”. The above registry value should then be repopulated with the necessary data. The first time you click the 'Add Counters' (+) button in Perfmon you will not see the WMI counters in the list. At this point you should open Task Manager (by running taskmgr.exe) and wait until the winmgmt.exe process returns to 0% CPU utilization. Now click the 'Add Counters' (+) button again and you will see the required counters in the list.