Re: 关于一个dump的分析

Windows内核调试

关于一个dump的分析

zmsx 2010-02-01, 11:50 上午
我把相关信息列在这，由于这个程序加了vmp，所以没办法双机调。希望高人指点思路。

1: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: c700000c, memory referenced
Arg2: 00000002, IRQL
Arg3: 00000000, bitfield :
bit 0 : value 0 = read operation, 1 = write operation
bit 3 : value 0 = not an execute operation, 1 = execute operation (only on chips which support this level of status)
Arg4: 8051174d, address which referenced memory

Debugging Details:
------------------

PEB is paged out (Peb.Ldr = 7ffd300c). Type ".hh dbgerr001" for details
PEB is paged out (Peb.Ldr = 7ffd300c). Type ".hh dbgerr001" for details

READ_ADDRESS: c700000c

CURRENT_IRQL: 2

FAULTING_IP:
nt!MiDeleteValidAddress+51
8051174d 8b460c mov eax,dword ptr [esi+0Ch]

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

PROCESS_NAME: daemon.exe

TRAP_FRAME: a5f73b84 -- (.trap 0xffffffffa5f73b84)
ErrCode = 00000000
eax=00000002 ebx=03ffffff ecx=00000020 edx=02800000 esi=c7000000 edi=c0002000
eip=8051174d esp=a5f73bf8 ebp=a5f73c14 iopl=0 nv up ei pl nz na po cy
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010203
nt!MiDeleteValidAddress+0x51:
8051174d 8b460c mov eax,dword ptr [esi+0Ch] ds:0023:c700000c=????????
Resetting default scope

LAST_CONTROL_TRANSFER: from 8051174d to 805444c0

STACK_TEXT:
a5f73b84 8051174d badb0d00 02800000 00000002 nt!KiTrap0E+0x238
a5f73c14 805af081 00400221 8879f2a0 007ffff8 nt!MiDeleteValidAddress+0x51
a5f73c34 80513315 8879f2a0 8879f3d8 8879f2a0 nt!MiDeleteAddressesInWorkingSet+0x65
a5f73c68 805d13e7 0079f2a0 8860b728 40010004 nt!MmCleanProcessAddressSpace+0x193
a5f73cf0 805d14cb 40010004 a5f73d4c 804ff91d nt!PspExitThread+0x621
a5f73cfc 804ff91d 8860b728 a5f73d48 a5f73d3c nt!PsExitSpecialApc+0x23
a5f73d4c 80541467 00000001 00000000 a5f73d64 nt!KiDeliverApc+0x1af
a5f73d4c 7c90e514 00000001 00000000 a5f73d64 nt!KiServiceExit+0x59
WARNING: Frame IP not in any known module. Following frames may be wrong.
02b8ff80 00000000 00000000 00000000 00000000 0x7c90e514

STACK_COMMAND: kb

FOLLOWUP_IP:
nt!MiDeleteValidAddress+51
8051174d 8b460c mov eax,dword ptr [esi+0Ch]

SYMBOL_STACK_INDEX: 1

SYMBOL_NAME: nt!MiDeleteValidAddress+51

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: nt

DEBUG_FLR_IMAGE_TIMESTAMP: 4a78232a

IMAGE_NAME: memory_corruption

FAILURE_BUCKET_ID: 0xA_VRF_nt!MiDeleteValidAddress+51

BUCKET_ID: 0xA_VRF_nt!MiDeleteValidAddress+51

Followup: MachineOwner
---------

1: kd> !process
PROCESS 8879f2a0 SessionId: 0 Cid: 0c5c Peb: 7ffd3000 ParentCid: 16f0
DirBase: 0a5c0580 ObjectTable: 00000000 HandleCount: 0.
Image: daemon.exe
VadRoot 88be6a00 Vads 182 Clone 0 Private 1153. Modified 51024. Locked 0.
DeviceMap e3bf3d38
Token e2cd65e8
ElapsedTime 09:34:39.812
UserTime 00:00:00.453
KernelTime 00:00:00.343
QuotaPoolUsage[PagedPool] 102080
QuotaPoolUsage[NonPagedPool] 7280
Working Set Sizes (now,min,max) (2980, 50, 345) (11920KB, 200KB, 1380KB)
PeakWorkingSetSize 3412
VirtualSize 59 Mb
PeakVirtualSize 80 Mb
PageFaultCount 4552
MemoryPriority BACKGROUND
BasePriority 8
CommitCharge 1709

THREAD 88aeba48 Cid 0c5c.0bac Teb: 00000000 Win32Thread: 00000000 RUNNING on processor 1

1: kd> !pte 00400221
VA 00400221
PDE at 00000000C0600010 PTE at 00000000C0002000
contains 00000000791C2867 contains 00002EA800000020
pfn 791c2 ---DA--UWEV not valid
PageFile: 0
Offset: 2ea8
Protect: 1 - Readonly

1: kd> ub nt!MiDeleteValidAddress+0x51
nt!MiDeleteValidAddress+0x36:
80511732 8bd1 mov edx,ecx
80511734 0facc20c shrd edx,eax,0Ch
80511738 bbffffff03 mov ebx,3FFFFFFh
8051173d 23d3 and edx,ebx
8051173f 8bf2 mov esi,edx
80511741 6bf61c imul esi,esi,1Ch
80511744 0335c8105680 add esi,dword ptr [nt!MmPfnDatabase (805610c8)]
8051174a c1e80c shr eax,0Ch

1: kd> u nt!MiDeleteValidAddress+0x51
nt!MiDeleteValidAddress+0x51:
8051174d 8b460c mov eax,dword ptr [esi+0Ch]
80511750 a808 test al,8
80511752 8955f0 mov dword ptr [ebp-10h],edx
80511755 0f84c4000000 je nt!MiDeleteValidAddress+0x123 (8051181f)
8051175b a801 test al,1
8051175d 8b5604 mov edx,dword ptr [esi+4]
80511760 8955f4 mov dword ptr [ebp-0Ch],edx
80511763 7531 jne nt!MiDeleteValidAddress+0x9a (80511796)

Re: 关于一个dump的分析

格蠹老雷 2010-02-01, 15:40 下午
看起来像是PFN数据库被破坏掉了。
dd nt!MmPfnDatabase
把上面命令的结果贴上来看看。
另外执行下version，想知道是哪个版本的NT

Re: 关于一个dump的分析

zmsx 2010-02-01, 15:56 下午
目前发现出现问题的机器都是开了 PAE的机器。

1: kd> dd nt!MmPfnDatabase
805610c8 81000000 fffffffe 00000006 0000003f
805610d8 0003489f 000672d2 03c54a88 0004b9fb
805610e8 01cd39e4 00000000 01890f74 00394216
805610f8 0013d281 00000000 00000000 000c17b0
80561108 0000c17b 0002180e 00002eef 0000001e
80561118 000000fa 0004c17e 0007d3fe 0007d3fe
80561128 0007d38a 00000040 00000000 7fff0000
80561138 80000000 7ffeffff 00000000 00000000

version
Windows XP Kernel Version 2600 (Service Pack 2) MP (2 procs) Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 2600.xpsp_sp2_qfe.090804-1435
Machine Name:
Kernel base = 0x804d7000 PsLoadedModuleList = 0x8055c720
Debug session time: Fri Jan 29 18:55:08.406 2010 (GMT+8)
System Uptime: 0 days 9:38:12.074
32-bit Kernel summary dump: \\nj-fs\home\tdme\Arthur_Zuo\dump\NJ-STEVEN-DU\Build 2.5.0.1042 (20100201_093816).DMP

Microsoft (R) Windows Debugger Version 6.11.0001.404 X86
Copyright (c) Microsoft Corporation. All rights reserved.

command line: '"C:\Program Files\Debugging Tools for Windows (x86)\windbg.exe" -k com:port=\\.\pipe\com_1,baud=11520,pipe' Debugger Process 0xF7C
dbgeng: image 6.11.0001.404, built Thu Feb 26 09:55:43 2009
[path: C:\Program Files\Debugging Tools for Windows (x86)\dbgeng.dll]
dbghelp: image 6.11.0001.404, built Thu Feb 26 09:55:30 2009
[path: C:\Program Files\Debugging Tools for Windows (x86)\dbghelp.dll]
DIA version: 11212
Extension DLL search Path:
C:\Program Files\Debugging Tools for Windows (x86)\WINXP;C:\Program Files\Debugging Tools for Windows (x86)\winext;C:\Program Files\Debugging Tools for Windows (x86)\winext\arcade;C:\Program Files\Debugging Tools for Windows (x86)\pri;C:\Program Files\Debugging Tools for Windows (x86);C:\Program Files\Debugging Tools for Windows (x86)\winext\arcade;D:\Program Files\Windows Resource Kits\Tools\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;D:\Program Files\Perforce;C:\Program Files\Microsoft SQL Server\90\Tools\binn\;C:\Program Files\Microsoft Driver Test Manager\Controller\;D:\Program Files\Microsoft Platform SDK for Windows XP SP2\Bin\.;D:\Program Files\Microsoft Platform SDK for Windows XP SP2\Bin\WinNT\.;C:\Program Files\Common Files\Thunder Network\KanKan\Codecs;D:\Program Files\Support Tools\;d:\Program Files\010 Editor v3;D:\Program Files\Microsoft Platform SDK for Windows XP SP2\Bin\.;D:\Program Files\Microsoft Platform SDK for Windows XP SP2\Bin\WinNT\.
Extension DLL chain:
dbghelp: image 6.11.0001.404, API 6.1.6, built Thu Feb 26 09:55:30 2009
[path: C:\Program Files\Debugging Tools for Windows (x86)\dbghelp.dll]
ext: image 6.11.0001.404, API 1.0.0, built Thu Feb 26 09:55:30 2009
[path: C:\Program Files\Debugging Tools for Windows (x86)\winext\ext.dll]
exts: image 6.11.0001.404, API 1.0.0, built Thu Feb 26 09:55:24 2009
[path: C:\Program Files\Debugging Tools for Windows (x86)\WINXP\exts.dll]
kext: image 6.11.0001.404, API 1.0.0, built Thu Feb 26 09:55:24 2009
[path: C:\Program Files\Debugging Tools for Windows (x86)\winext\kext.dll]
kdexts: image 6.1.7015.0, API 1.0.0, built Thu Feb 26 09:54:56 2009
[path: C:\Program Files\Debugging Tools for Windows (x86)\WINXP\kdexts.dll]
Driver verifier enabled, level 0xff

Re: 关于一个dump的分析

zmsx 2010-02-01, 16:01 下午
出问题的进程是虚拟光驱软件 daemon tool lite ，关机的时候会出现。我的driver 会修改PE头，我做的操作是先改PTE 设 Copy-on-write . 然后写，写完后设为只读. 如果写完后不还原，也会出现相同问题。已经处理了pae 和 pse的情况。并且开了verifier 不开 verifier 也不会有问题。verifier 只验证自己的driver.

Special pool: Enabled
Force IRQL checking: Enabled
Low resources simulation: Disabled
Pool tracking: Enabled
I/O verification: Enabled
Deadlock detection: Enabled
Enhanced I/O verification: Disabled
DMA checking: Enabled

Re: 关于一个dump的分析

格蠹老雷 2010-02-01, 20:53 下午
DUMP文件发到我的信箱里或者放到一个FTP，我看一下

Re: 关于一个dump的分析

zmsx 2010-02-02, 14:19 下午
我已经发你邮箱了。yinkui.zhang@gmail.com
谢谢张老师

Re: 关于一个dump的分析

格蠹老雷 2010-02-04, 22:22 下午

直接的原因是向MiDeleteValidAddress传递了一个无效的地址——00400221。如这个函数的函数名所说的，这个函数是用来删除一个有效的地址，也就是位于物理内存中的地址。每个有效地址在PFN数据库中有一个MMPFN结构与之对应。删除这个地址时需要找到这个结构然后对其进行更新。

但因为00400221是个无效的地址，所以用它计算出的PFN索引来寻找MMPFN结构的字段时非法访问了。

仔细观察出问题的那一刻：

eax=00000002 ebx=03ffffff ecx=00000020 edx=02800000 esi=c7000000 edi=c0002000
eip=8051174d esp=a5f73bf8 ebp=a5f73c14 iopl=0 nv up ei pl nz na po cy
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010203
nt!MiDeleteValidAddress+0x51:
8051174d 8b460c mov eax,dword ptr [esi+0Ch] ds:0023:c700000c=????????

其中edx的值应该是要删除地址（00400221）的pfn，在PFN数据库中每个表项的长度是0x1C：

1: kd> ?? sizeof(_MMPFN)
unsigned int 0x1c

所以要把这个值*1C得到在PFN数据库中的偏移：

8051173f 8bf2 mov esi,edx
80511741 6bf61c imul esi,esi,1Ch

然后加上PFN数据库的起始地址：

80511744 0335c8105680 add esi,dword ptr [nt!MmPfnDatabase (805610c8)]

我们手工算一下：

1: kd> ? 02800000*1c+poi(MmPfnDatabase)
Evaluate expression: -956301312 = c7000000

也就是计算出的要删除地址的MMPFN结构位于c7000000。

然后取偏移0xc处的_MMPFNENTRY：

8051174d 8b460c mov eax,dword ptr [esi+0Ch]

而这个地方根本不是一个有效地MMPFNENTRY，所以就异常了。

那么为什么要把一个无效的地址交给MiDeleteValidAddress去删除呢？

这要问上一级调用函数了，即MiDeleteAddressesInWorkingSet。

MiDeleteAddressesInWorkingSet负责删除指定进程的工作集。使用!wsle命令可以观察工作集的信息：

1: kd> !wsle

Working Set @ c0883000
    FirstFree      103 FirstDynamic       11
    LastEntry     108d NextSlot            7 LastInitialized     10c0
    NonDirect      56c HashTable           0 HashTableSize       a00

如果要列出当前进程工作集中的所有表项（内存页），那么可以执行!wsle 7命令：

1: kd> !wsle 7

Reading the WSLE data .....................................

Virtual Address           Age Locked ReferenceCount
        c0600203          0        1        1
        c0601203          0        1        1
        c0602203          0        1        1
        c0603203          0        1        1
        c0604203          0        1        1
        c0882203          0        1        1
        c0883203          0        1        1

...

所以现在看来应该是这个进程的工作集数据有错误，这恰好与驱动程序曾经修改PTE导致问题相符...

Re: 关于一个dump的分析

nightxie 2010-02-05, 00:22 上午
看张老师的这个分析，感觉就像读教程一样~~~张老师真该出本实例分析的书了！！！

Re: 关于一个dump的分析

single 2010-02-05, 09:17 上午
张老师真该出本实例分析的书了！！！同意希望张老师可以考虑：）

Re: 关于一个dump的分析

zmsx 2010-02-05, 10:48 上午
谢谢张老师的分析。不知有没有什么修改的建议。由于加了vmp，没办法双机调。所以也就不能对pte所在的地址下断点。不知道有没有其他的办法。或者说我修改的方式本身有问题？

地址 400000
我做的修改只是置 copy-on-write 位，然后修改内存。

修改前 PTE 的标志是 read-only Usermode WSLE 中为 400201

修改后 PTE 的标志是 write Usermode accessed dirty 11bit 为1 WSLE中为400209

后来有一发现。即修改后将 pte的标志改为和修改前一样。到目前为止还没出现蓝屏。