Hey Checkyourlogs Fans,
This post explores how we used PoolMon.exe, part of Microsoft’s Windows Driver Kit (WDK), to identify and remediate a weekly memory leak in a production Windows Server 2025 environment. This leak occurred across physical and virtual servers, gradually consuming memory over 5–7 days and forcing weekly reboots.
Our investigation pointed toward a kernel memory leak—possibly caused by an agent like antivirus software or a patch management tool like ManageEngine Update Manager. Using PoolMon, we isolated the issue to a memory tag, which led us to the culprit driver.
Symptoms of the Leak
| Symptom | Description |
| High RAM usage over time | Memory usage grows without an associated high-usage process |
| Task Manager shows low process use | But the server still consumes 90–100% RAM |
| Nonpaged pool usage > 1–2 GB | (Normal range is 200–500 MB) |
| The server needs weekly reboots | Freeing memory temporarily resolves the issue |
| Multiple servers affected | Common agent suspected across all systems |
Step 1: Installing PoolMon.exe
To get started with PoolMon:
| Step | Action |
|---|---|
| 1 | Download and install the Windows Driver Kit (WDK) for Windows 10/11 or Server 2025 |
| 2 | Locate poolmon.exe – usually found at:
C:\Program Files (x86)\Windows Kits\10\Tools\x64 |
| 3 | Copy poolmon.exe to the affected server |
| 4 | Run from elevated Command Prompt or PowerShell |
Note: Pool tagging is enabled by default in Server 2025. If PoolMon doesn’t show data, ensure the registry value below exists:
HKLM\SYSTEM\CurrentControlSet\Control\Session Manager
DWORD: GlobalFlag = 0x400
Reboot after applying this setting.
Step 2: Capturing Pool Usage During a Leak
Run PoolMon to capture live pool allocation data:
poolmon /b
Inside PoolMon:
| Key Press | Action |
| P | Toggle between Paged/Nonpaged pool |
| B | Sort list by bytes used |
Focus on Nonpaged Pool usage first. It’s common for drivers and kernel agents to leak here.
Sample PoolMon Output (Filtered by Nonpaged Pool and Sorted by Bytes)
| Tag | Type | Allocs | Frees | Diff | Bytes | Avg Bytes/Alloc |
| MEUP | Nonp | 150,000 | 5,000 | 145,000 | 800,000,000 | ~5.5 KB |
| Ntfx | Nonp | 45,000 | 44,950 | 50 | 12,000,000 | ~240 KB |
| ConT | Nonp | 33,000 | 32,500 | 500 | 8,000,000 | ~16 KB |
The MEUP tag is the outlier—consuming 800 MB of kernel memory and showing 145,000 unfreed allocations.
Step 3: Interpreting the Results
To determine if a tag is leaking:
| Indicator | Description |
| High Diff | Indicates allocations that haven’t been freed |
| Increasing Bytes | Memory usage by that tag is growing over time |
| High Avg Bytes/Alloc | Each allocation is large (proper for assessing impact) |
| Tag unknown in system docs | Suggests a custom or 3rd-party driver (investigate further) |
Take snapshots of PoolMon hourly. If a tag keeps climbing, it’s likely the culprit.
Step 4: Mapping Pool Tags to Drivers
We need to determine which driver is responsible for the MEUP tag.
Using findstr (Recommended)
cd C:\Windows\System32\drivers
findstr /m /s /l “MEUP” *.sys
Output Example:
MEAgent.sys
This confirms that the tag MEUP exists inside the MEAgent.sys binary—likely part of ManageEngine Update Manager.
Optional: PowerShell Alternative
Get-ChildItem *.sys | Select-String “MEUP” | Select-Object -Unique Path
Confirm Driver Details
You can validate the driver details with:
Get-Item “C:\Windows\System32\drivers\MEAgent.sys” | Select-Object VersionInfo
Step 5: Triaging and Isolating the Culprit
| Step | Purpose |
| Confirm driver belongs to agent | Cross-check with installed apps or services |
| Check other servers | Is the same tag present on other leaking systems? |
| Disable agent on test system | See if the leak behavior stops after disabling the suspected agent |
| Search forums/vendor KB | Others may have reported this issue; fixes may already exist |
Step 6: Remediation Options
| Option | Description |
| Update the agent/driver | Always check for the latest version – many leaks are fixed in updates |
| Contact vendor support | Provide tag name, driver file, and PoolMon stats |
| Temporarily disable agent | If safe, disable or stop the agent to confirm the leak source |
| Schedule reboots | Not ideal, but it may be necessary until a fix is in place |
| Adjust task scheduling | If scans or updates trigger the leak, stagger or reduce frequency |
| Implement monitoring | Use PerfMon or other tools to alert when Nonpaged Pool exceeds safe limits |
Summary: The PoolMon Method
| Step | Tool/Command | Goal |
| Capture memory usage | poolmon /b | Identify high pool tags |
| Filter by pool type | Press P in PoolMon | Focus on Nonpaged or Paged |
| Find the offending tag | Sort by Bytes (press B) | Spot leaks growing over time |
| Map tag to driver | findstr /m /s /l “TAG” *.sys | Identify the associated .sys file |
| Confirm source | File version check/service listing | Match driver to installed software |
| Take action | Update/contact vendor / disable | Stop the leak and stabilize the system |
Pro Tip: Monitor Memory Pools Automatically
Consider setting up monitoring using PerfMon counters:
- Memory\Pool Nonpaged Bytes
- Memory\Pool Paged Bytes
Set alert thresholds (e.g., >1.5 GB Nonpaged Pool), and automate service restarts or send alerts when hit.
Conclusion
Memory leaks are sneaky, especially in kernel space. But with the right tools and a bit of patience, you can:
- Isolate the issue
- Find the leaking component
- Take proactive steps before it causes outages
In our case, the weekly server reboots were traced to a ManageEngine agent leaking memory under the MEUP tag. Using PoolMon and standard Windows tools, we saved countless hours of guesswork and got our servers back to a healthy state.
Stay sharp, sysadmins—and don’t let a 4-letter pool tag take down your infrastructure.
Thanks,
Dave
