This Knowledgebase article provides information about Microsoft Exchange consuming all available physical memory to perform its tasks even though memory limitations have been configured.
Knowledgebase article #1439 - ‘How to Limit Exchange 2007 Physical Memory Utilization' provides a procedure to set an upper threshold to the Exchange 2007 memory consumption. Care should be taken because after setting the maximum value, free memory will be available to other processes, which may use the new free memory, leaving the server without free physical memory. If this is the case, Exchange memory utilization should be decreased to a lower value. After setting the threshold all the processes that reside on the active server, the processes should then be monitored to ensure they all have sufficient resources.
Use Windows perfmon.exe utility to measure each process. Using Knowledgebase article #879 - ‘How to Set Up PERFMON Counters’, set up the following counters for the Process Object and monitor them to assure all processes have sufficient memory for proper operation:
Performance Object: Process
Counters: Private Bytes, Virtual Bytes, Working Set
Instances: All instances
Neverfail Heartbeat V5.1 and Later