场景：公司用户VM，老出现IP异常为169的问题，找网络专家定位发现该问题是VM的OS问题，于是开一个高级case给微软并协助抓取日志，花费一个多月，初步的定位结论： 应该是audiodg.exe（声卡）在创建的时候继承了父进程（DHCP Client Service）句柄，导致一个EndPoint没有释放， 所以此EndPoint收到ip续租的回包内存不足（与前面的分析对应），导致续租失败。
附其分析日志，但存在一些疑问，微软工程师答复他们是借助他们源代码调试分析的，我们自己拿到dump，很非常困难调试分析的，但其中有些疑问在微软那里得不到明确的答复，于是特来请张老师指点：从AFD日志： datagram dropped: 2: Process 0x8876e658, Endpoint 0x86cc5ee0, Buffer 0x87fc44d2, Length 312, Address 10.66.104.1:67, Seq 10001, Reason Insufficient local buffer space 所以我们可以确定，出问题的endpoint地址是0x86cc5ee0，DHCP client service进程是0x8876e658 从dump中我们发现下面的TCP Endpoint 绑定在端口68 0x89748150 对应的AFD Endpoint就是0x86cc5ee0 在一台干净的机器上，这些endpoints应该在DHCP client service 关闭socket的时候清除。对应的调用堆栈是 fffff880`0367f818 fffff880`024b3c90 tcpip!UdpCloseEndpoint fffff880`0367f820 fffff880`024b4122 afd!AfdCleanupCore+0x410 fffff880`0367f9a0 fffff800`017d968f afd!AfdDispatch+0x42 fffff880`0367f9f0 fffff800`017bf304 nt!IopCloseFile+0x11f fffff880`0367fa80 fffff800`017d9181 nt!ObpDecrementHandleCount+0xb4 fffff880`0367fb00 fffff800`017d9094 nt!ObpCloseHandleTableEntry+0xb1 fffff880`0367fb90 fffff800`014c5153 nt!ObpCloseHandle+0x94 fffff880`0367fbe0 00000000`77bfffaa nt!KiSystemServiceCopyEnd+0x13 00000000`014ef8f8 000007fe`fd3419ca ntdll!ZwClose+0xa 从dump中看，对应的AfdCleanupCore函数没有被调用过，根据之前svchost.exe (dhcp client service) dump中可以确定，用户模式的socket handle已经被关闭了。 这里唯一的解释就是有程序或者驱动打开了对应的afd endpoint的文件对象(file_object) 从dump中，我一共找到了130+ AFD ENDPOINT的文件对象，最后找到了0x86b1c480 ----如何从核心dump日志中，可以通过其他方法得到endpoint下的所有文件对象？？？？2: kd> !object 86b1c480 Object: 86b1c480 Type: (86960f78) File ObjectHeader: 86b1c468 (new version) HandleCount: 1 PointerCount: 1 Directory Object: 00000000 Name: \Endpoint {Afd} 应该还有句柄打开这。 接下来我又遍历了系统的句柄表，最后发现audiodg.exe ---------如何通过文件对象来查相应的进程名？04b8: Object: 86b1c480 GrantedAccess: 0016019f (Inherit) Inherit表示继承，也就是说这个句柄是从父进程继承下来的。那么audiodg.exe 父进程是谁呢？ 从dump来看就是dhcp client service 所在的svchost进程。 从MSDN的解释来看
A child process can inherit handles from its parent process. An inherited handle is valid only in the context of the child process. To enable a child process to inherit open handles from its parent process, use the following steps.
1. Create the handle with the bInheritHandle member of the SECURITY_ATTRIBUTES structure set to TRUE.
2. Create the child process using the CreateProcess function, with the bInheritHandles parameter set to TRUE.
应该是audiodg.exe在创建的时候继承了父进程句柄。从进程创建时间来看： 2: kd> !process 86c7aa58 1 PROCESS 86c7aa58 SessionId: 0 Cid: 0528 Peb: 7ffde000 ParentCid: 03b4 DirBase: eeaf6bc0 ObjectTable: be3423b0 HandleCount: 209. Image: audiodg.exe VadRoot 88de3a00 Vads 83 Clone 0 Private 2545. Modified 8805. Locked 0. DeviceMap b31c6d48 Token be258668 ElapsedTime 7 Days 08:14:05.302 UserTime 00:00:06.328 KernelTime 00:00:09.390 Dump创建时间 Mon Oct 22 17:05:19.201 2012 (UTC + 8:00) UDP_ENDPOINT创建时间：Mon Oct 15 08:51:13.023 2012 (UTC + 8:00) 所以也可以确认audiodg.exe 正是那个时候创建的。17:05:19 – 08:14:05 = 08:51:14 从今天你收集的smc –stop smc –start 之后问题消失点的dump来看：最后创建的几个进程是： PROCESS 88696278 SessionId: 0 Cid: 1fc8 Peb: 7ffdc000 ParentCid: 02a4 DirBase: eeddf240 ObjectTable: ac4e1698 HandleCount: 727. Image: Smc.exe PROCESS 88068bf8 SessionId: 1 Cid: 2084 Peb: 7ffd6000 ParentCid: 1fc8 DirBase: eeddfec0 ObjectTable: 900d3d20 HandleCount: 345. Image: SmcGui.exe PROCESS 89b02d40 SessionId: 0 Cid: 1e70 Peb: 7ffd4000 ParentCid: 0eb8 DirBase: eeddfe20 ObjectTable: b28e6968 HandleCount: 172. Image: w3wp.exe PROCESS 883ff030 SessionId: 0 Cid: 1c8c Peb: 7ffd5000 ParentCid: 03bc DirBase: eeddfca0 ObjectTable: b23740a0 HandleCount: 127. Image: audiodg.exe PROCESS 88fc4778 SessionId: 1 Cid: 18c4 Peb: 7ffdf000 ParentCid: 0314 DirBase: eeddfd00 ObjectTable: a5eae7c0 HandleCount: 124. Image: dllhost.exe PROCESS 87d36030 SessionId: 1 Cid: 1ba8 Peb: 7ffde000 ParentCid: 1580 DirBase: eeddfd80 ObjectTable: b1b5a228 HandleCount: 96. Image: NotMyfault.exe 这也说明问题消失前，audiodg.exe 被重起了。 我会继续察看audiodg.exe的创建过程。
问题解决了没？
对于列文件对象的问题，至少可以用以下方法之一：
1）如果Global Flag中事先设置了0x4000，那么可以用!object 0 File这样的命令列出所有文件对象
2）写一个简单的脚本文件，针对每个进程执行!handle命令
抓取dump没有设置global flag，所以很难通过！object o file的命令列出所有文件对象
由于该dump很大（压缩后），无法提供给你亲自调试，请您指导看看那可以从那些思路来定位找出对于有问题的object，然后根据该object来定位该进程的方法？？？？
贴一个脚本供参考
$$*****************************************************************$$ Script by Raymond to enumerate all processes and$$ show details of each one$$ Execute by $$><lp.txt$$ All rights reserved. Sep. 11, 2012$$*****************************************************************
r $t0 = nt!PsActiveProcessHead
$$ Iterate over all processes in list..for (r $t1 = poi(@$t0); (@$t1 != 0) & (@$t1 != @$t0); r $t1 = poi(@$t1)){ r? $t2 = #CONTAINING_RECORD(@$t1, nt!_EPROCESS, ActiveProcessLinks)
as /x ${/v:$Procc} @$t2
$$ Get image name into $ImageName. as /ma $ImageName @@c++(&@$t2->ImageFileName[0])
.block { .echo ${$ImageName} at ${$Procc} }
$$ switch to the process
.process @$t2 $$ $Procc
$$.reload /user
.if(@@c++(@$t2->UniqueProcessId!=4)) { .echo seaching in ${$ImageName}
s -a 10000 L?8000000 "taskmgr" } .else { .echo skip ${$ImageName} } .echo **********end of one process **********************
ad $ImageName ad ${/v:$Procc}}
如下是从核心内存转储后，加载您提供的脚步，运行得到的结果，请见：
Microsoft (R) Windows Debugger Version 6.6.0003.5Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [F:\IP change169\luzhonglun 00186776 -- 21121022\20121022-17点取了一个问题机器的核心内存转储\MEMORY1.DMP]Kernel Summary Dump File: Only kernel address space is available
Symbol search path is: SRV*f:\dump*http://msdl.microsoft.com/download/symbols
Executable search path is: Windows Vista Kernel Version 7600 MP (4 procs) Free x86 compatibleProduct: WinNt, suite: TerminalServer SingleUserTSBuilt by: 7600.16988.x86fre.win7_gdr.120401-1505Kernel base = 0x83c0c000 PsLoadedModuleList = 0x83d54810Debug session time: Mon Oct 22 17:05:19.201 2012 (GMT+8)System Uptime: 7 days 8:15:44.736Loading Kernel Symbols..................................................................................................................................Page 72db2 not present in the dump file. Type ".hh dbgerr004" for details.....................................................Loading User SymbolsPEB is paged out (Peb.Ldr = 7ffdf00c). Type ".hh dbgerr001" for detailsLoading unloaded module list..............................2: kd> $$><script.txtSystem at 0xffffffff8688ccc0Implicit process is now 8688ccc0skip System**********end of one process **********************smss.exe at 0xffffffff87eed838Process 87eed838 has invalid page directoriesseaching in smss.exe**********end of one process **********************csrss.exe at 0xffffffff8842ba90Process 8842ba90 has invalid page directoriesseaching in csrss.exe**********end of one process **********************csrss.exe at 0xffffffff885c3030Process 885c3030 has invalid page directoriesseaching in csrss.exe**********end of one process **********************wininit.exe at 0xffffffff885c5d40Process 885c5d40 has invalid page directoriesseaching in wininit.exe**********end of one process **********************winlogon.exe at 0xffffffff885d48e0Process 885d48e0 has invalid page directoriesseaching in winlogon.exe**********end of one process **********************services.exe at 0xffffffff88607030Process 88607030 has invalid page directoriesseaching in services.exe**********end of one process **********************lsass.exe at 0xffffffff8861c2a0Implicit process is now 8861c2a0seaching in lsass.exe**********end of one process **********************lsm.exe at 0xffffffff88626ad8Process 88626ad8 has invalid page directoriesseaching in lsm.exe**********end of one process **********************svchost.exe at 0xffffffff8868d518Process 8868d518 has invalid page directoriesseaching in svchost.exe**********end of one process **********************svchost.exe at 0xffffffff883b2d40Process 883b2d40 has invalid page directoriesseaching in svchost.exe**********end of one process **********************svchost.exe at 0xffffffff8876e658Process 8876e658 has invalid page directoriesseaching in svchost.exe**********end of one process **********************svchost.exe at 0xffffffff8879f1c0Process 8879f1c0 has invalid page directoriesseaching in svchost.exe**********end of one process **********************svchost.exe at 0xffffffff887bd030Process 887bd030 has invalid page directoriesseaching in svchost.exe**********end of one process **********************svchost.exe at 0xffffffff887c2ad0Process 887c2ad0 has invalid page directoriesseaching in svchost.exe**********end of one process **********************svchost.exe at 0xffffffff887d5d40Process 887d5d40 has invalid page directoriesseaching in svchost.exe**********end of one process **********************Smc.exe at 0xffffffff887ec420Process 887ec420 has invalid page directoriesseaching in Smc.exe**********end of one process **********************PicaSessionMgr at 0xffffffff87f77b00Process 87f77b00 has invalid page directoriesseaching in PicaSessionMgr**********end of one process **********************picaDispMgr.ex at 0xffffffff888663e8Process 888663e8 has invalid page directoriesseaching in picaDispMgr.ex**********end of one process **********************picaTWIHost.ex at 0xffffffff88867500Process 88867500 has invalid page directoriesseaching in picaTWIHost.ex**********end of one process **********************svchost.exe at 0xffffffff888f9d40Process 888f9d40 has invalid page directoriesseaching in svchost.exe**********end of one process **********************ccSvcHst.exe at 0xffffffff88971660Process 88971660 has invalid page directoriesseaching in ccSvcHst.exe**********end of one process **********************spoolsv.exe at 0xffffffff889f9130Process 889f9130 has invalid page directoriesseaching in spoolsv.exe**********end of one process **********************svchost.exe at 0xffffffff88a1e030Process 88a1e030 has invalid page directoriesseaching in svchost.exe**********end of one process **********************armsvc.exe at 0xffffffff88a7c2f8Process 88a7c2f8 has invalid page directoriesseaching in armsvc.exe**********end of one process **********************svchost.exe at 0xffffffff88aa8660Process 88aa8660 has invalid page directoriesseaching in svchost.exe**********end of one process **********************arr_srvs.exe at 0xffffffff88af63f8Process 88af63f8 has invalid page directoriesseaching in arr_srvs.exe**********end of one process **********************arr_isrv.exe at 0xffffffff88b5eac8Process 88b5eac8 has invalid page directoriesseaching in arr_isrv.exe**********end of one process **********************residentAgent. at 0xffffffff88b6f140Process 88b6f140 has invalid page directoriesseaching in residentAgent.**********end of one process **********************CitrixCseEngin at 0xffffffff88dd8638Process 88dd8638 has invalid page directoriesseaching in CitrixCseEngin**********end of one process **********************CtxAudioServic at 0xffffffff88e11d40Process 88e11d40 has invalid page directoriesseaching in CtxAudioServic**********end of one process **********************CtxSvcHost.exe at 0xffffffff88e24d40Process 88e24d40 has invalid page directoriesseaching in CtxSvcHost.exe**********end of one process **********************CtxSvcHost.exe at 0xffffffff88e215c0Process 88e215c0 has invalid page directoriesseaching in CtxSvcHost.exe**********end of one process **********************AccCheckerServ at 0xffffffff88e66d40Process 88e66d40 has invalid page directoriesseaching in AccCheckerServ**********end of one process **********************collector.exe at 0xffffffff88e38a10Process 88e38a10 has invalid page directoriesseaching in collector.exe**********end of one process **********************conhost.exe at 0xffffffff88e7e380Process 88e7e380 has invalid page directoriesseaching in conhost.exe**********end of one process **********************hservice.exe at 0xffffffff87d7fd40Process 87d7fd40 has invalid page directoriesseaching in hservice.exe**********end of one process **********************IMEDICTUPDATE. at 0xffffffff89a15a58Process 89a15a58 has invalid page directoriesseaching in IMEDICTUPDATE.**********end of one process **********************LocalSch.EXE at 0xffffffff88ab1cd0Process 88ab1cd0 has invalid page directoriesseaching in LocalSch.EXE**********end of one process **********************pds.exe at 0xffffffff87e82458Process 87e82458 has invalid page directoriesseaching in pds.exe**********end of one process **********************issuser.exe at 0xffffffff89a96770Process 89a96770 has invalid page directoriesseaching in issuser.exe**********end of one process **********************kxescore.exe at 0xffffffff89a9d4d0Process 89a9d4d0 has invalid page directoriesseaching in kxescore.exe**********end of one process **********************policy.client. at 0xffffffff89abfd40Process 89abfd40 has invalid page directoriesseaching in policy.client.**********end of one process **********************tmcsvc.exe at 0xffffffff882a2030Process 882a2030 has invalid page directoriesseaching in tmcsvc.exe**********end of one process **********************svchost.exe at 0xffffffff883f2378Process 883f2378 has invalid page directoriesseaching in svchost.exe**********end of one process **********************nsd.exe at 0xffffffff88406d40Process 88406d40 has invalid page directoriesseaching in nsd.exe**********end of one process **********************hagent.exe at 0xffffffff896c5b08Process 896c5b08 has invalid page directoriesseaching in hagent.exe**********end of one process **********************svchost.exe at 0xffffffff885cbd40Process 885cbd40 has invalid page directoriesseaching in svchost.exe**********end of one process **********************ntmulti.exe at 0xffffffff885ca678Process 885ca678 has invalid page directoriesseaching in ntmulti.exe**********end of one process **********************SMSvcHost.exe at 0xffffffff8987a8f0Process 8987a8f0 has invalid page directoriesseaching in SMSvcHost.exe**********end of one process **********************SoftMon.exe at 0xffffffff899b3030Process 899b3030 has invalid page directoriesseaching in SoftMon.exe**********end of one process **********************SpesService.ex at 0xffffffff89a07268Process 89a07268 has invalid page directoriesseaching in SpesService.ex**********end of one process **********************Rtvscan.exe at 0xffffffff898f7d40Process 898f7d40 has invalid page directoriesseaching in Rtvscan.exe**********end of one process **********************UVPUpgradeServ at 0xffffffff89aec8b0Process 89aec8b0 has invalid page directoriesseaching in UVPUpgradeServ**********end of one process **********************uvpmonitor.exe at 0xffffffff89b0a030Process 89b0a030 has invalid page directoriesseaching in uvpmonitor.exe**********end of one process **********************svchost.exe at 0xffffffff89b0d770Process 89b0d770 has invalid page directoriesseaching in svchost.exe**********end of one process **********************CtxSvcHost.exe at 0xffffffff899b5350Process 899b5350 has invalid page directoriesseaching in CtxSvcHost.exe**********end of one process **********************CtxSvcHost.exe at 0xffffffff89b8ed40Process 89b8ed40 has invalid page directoriesseaching in CtxSvcHost.exe**********end of one process **********************encsvc.exe at 0xffffffff89c21a98Process 89c21a98 has invalid page directoriesseaching in encsvc.exe**********end of one process **********************svchost.exe at 0xffffffff88b3f568Process 88b3f568 has invalid page directoriesseaching in svchost.exe**********end of one process **********************taskhost.exe at 0xffffffff89894030Process 89894030 has invalid page directoriesseaching in taskhost.exe**********end of one process **********************dwm.exe at 0xffffffff88e40030Process 88e40030 has invalid page directoriesseaching in dwm.exe**********end of one process **********************explorer.exe at 0xffffffff89b96870Implicit process is now 89b96870seaching in explorer.exe**********end of one process **********************SmcGui.exe at 0xffffffff88e32030Process 88e32030 has invalid page directoriesseaching in SmcGui.exe**********end of one process **********************HwUVPUpgrade.e at 0xffffffff87c27850Process 87c27850 has invalid page directoriesseaching in HwUVPUpgrade.e**********end of one process **********************KSafeTray.exe at 0xffffffff8868b030Process 8868b030 has invalid page directoriesseaching in KSafeTray.exe**********end of one process **********************CpmTray.exe at 0xffffffff87dcf9e0Process 87dcf9e0 has invalid page directoriesseaching in CpmTray.exe**********end of one process **********************SpesConsole.ex at 0xffffffff87dd8588Process 87dd8588 has invalid page directoriesseaching in SpesConsole.ex**********end of one process **********************ccApp.exe at 0xffffffff87db1a58Process 87db1a58 has invalid page directoriesseaching in ccApp.exe**********end of one process **********************ServicesMonito at 0xffffffff87deca58Process 87deca58 has invalid page directoriesseaching in ServicesMonito**********end of one process **********************FingerConf.exe at 0xffffffff89757d40Process 89757d40 has invalid page directoriesseaching in FingerConf.exe**********end of one process **********************Snape.exe at 0xffffffff8a004d40Process 8a004d40 has invalid page directoriesseaching in Snape.exe**********end of one process **********************AccAgent.exe at 0xffffffff89e3a6e0Process 89e3a6e0 has invalid page directoriesseaching in AccAgent.exe**********end of one process **********************svchost.exe at 0xffffffff869f0748Process 869f0748 has invalid page directoriesseaching in svchost.exe**********end of one process **********************SPES5.exe at 0xffffffff887f0a70Process 887f0a70 has invalid page directoriesseaching in SPES5.exe**********end of one process **********************BSPServer.exe at 0xffffffff8883e338Process 8883e338 has invalid page directoriesseaching in BSPServer.exe**********end of one process **********************mmvdhost.exe at 0xffffffff86a84548Process 86a84548 has invalid page directoriesseaching in mmvdhost.exe**********end of one process **********************NotesInfoSearc at 0xffffffff8a24fd40Process 8a24fd40 has invalid page directoriesseaching in NotesInfoSearc**********end of one process **********************eSpace.exe at 0xffffffff86c9ba58Process 86c9ba58 has invalid page directoriesseaching in eSpace.exe**********end of one process **********************audiodg.exe at 0xffffffff86c7aa58Process 86c7aa58 has invalid page directoriesseaching in audiodg.exe**********end of one process **********************rcgui.exe at 0xffffffff88de2030Process 88de2030 has invalid page directoriesseaching in rcgui.exe**********end of one process **********************FaultReport.ex at 0xffffffff869f5030Process 869f5030 has invalid page directoriesseaching in FaultReport.ex**********end of one process **********************SearchIndexer. at 0xffffffff88a97118Process 88a97118 has invalid page directoriesseaching in SearchIndexer.**********end of one process **********************nlnotes.exe at 0xffffffff86a3aab8Process 86a3aab8 has invalid page directoriesseaching in nlnotes.exe**********end of one process **********************ntaskldr.exe at 0xffffffff86a86250Process 86a86250 has invalid page directoriesseaching in ntaskldr.exe**********end of one process **********************wuauclt.exe at 0xffffffff86d53ac0Process 86d53ac0 has invalid page directoriesseaching in wuauclt.exe**********end of one process **********************Stormtray.exe at 0xffffffff89daca78Process 89daca78 has invalid page directoriesseaching in Stormtray.exe**********end of one process **********************KanKan.exe at 0xffffffff897a1d40Process 897a1d40 has invalid page directoriesseaching in KanKan.exe**********end of one process **********************w3wp.exe at 0xffffffff87112030Process 87112030 has invalid page directoriesseaching in w3wp.exe**********end of one process **********************VDARedirector. at 0xffffffff87a0eac8Process 87a0eac8 has invalid page directoriesseaching in VDARedirector.**********end of one process **********************cmd.exe at 0xffffffff87012a38Process 87012a38 has invalid page directoriesseaching in cmd.exe**********end of one process **********************AcroRd32.exe at 0xffffffff86aefbd0Process 86aefbd0 has invalid page directoriesseaching in AcroRd32.exe**********end of one process **********************svchost.exe at 0xffffffff86992030Process 86992030 has invalid page directoriesseaching in svchost.exe**********end of one process **********************CdfSvc.exe at 0xffffffff8991e790Process 8991e790 has invalid page directoriesseaching in CdfSvc.exe**********end of one process **********************POWERPNT.EXE at 0xffffffff88ab2030Process 88ab2030 has invalid page directoriesseaching in POWERPNT.EXE**********end of one process **********************SndVol.exe at 0xffffffff88e947d0Process 88e947d0 has invalid page directoriesseaching in SndVol.exe**********end of one process **********************SogouCloud.exe at 0xffffffff8893c8f8Process 8893c8f8 has invalid page directoriesseaching in SogouCloud.exe**********end of one process **********************SogouCloud.exe at 0xffffffff87b21708Process 87b21708 has invalid page directoriesseaching in SogouCloud.exe**********end of one process **********************picaSvc.exe at 0xffffffff87ec4d40Process 87ec4d40 has invalid page directoriesseaching in picaSvc.exe**********end of one process **********************CitrixUSB.exe at 0xffffffff88399d40Process 88399d40 has invalid page directoriesseaching in CitrixUSB.exe**********end of one process **********************CpSvc.exe at 0xffffffff87e8da40Process 87e8da40 has invalid page directoriesseaching in CpSvc.exe**********end of one process **********************WorkstationAge at 0xffffffff8855a590Process 8855a590 has invalid page directoriesseaching in WorkstationAge**********end of one process **********************w3wp.exe at 0xffffffff89190228Process 89190228 has invalid page directoriesseaching in w3wp.exe**********end of one process **********************SogouCloud.exe at 0xffffffff88549030Process 88549030 has invalid page directoriesseaching in SogouCloud.exe**********end of one process **********************LogonUI.exe at 0xffffffff8708e4f8Process 8708e4f8 has invalid page directoriesseaching in LogonUI.exe**********end of one process **********************cmd.exe at 0xffffffff899c9918Process 899c9918 has invalid page directoriesseaching in cmd.exe**********end of one process **********************conhost.exe at 0xffffffff89622be0Process 89622be0 has invalid page directoriesseaching in conhost.exe**********end of one process **********************cmd.exe at 0xffffffff88511770Process 88511770 has invalid page directoriesseaching in cmd.exe**********end of one process **********************conhost.exe at 0xffffffff89586100Process 89586100 has invalid page directoriesseaching in conhost.exe**********end of one process **********************NotMyfault.exe at 0xffffffff89f34798Process 89f34798 has invalid page directoriesseaching in NotMyfault.exe**********end of one process **********************