Re: windows 2003 系统挂死，用NMI生产dump请指点，万分感谢

WinDbg

windows 2003 系统挂死，用NMI生产dump请指点，万分感谢

---

Gaoquan 2009-12-08, 14:28 下午

描述：

今天在跑备份的时候(Symantec veritas NUB6.0),机器挂死。Symantec公司师无解，无奈只好自己分析dump，分析出来的结果是好像vsp。sys这个驱动，但我不敢确定，所以请大家帮忙看看(这个现象出现了一个出现了4次，都没有解决)

minidup文件分析如下

oading Dump File [C:\Documents and Settings\e000234\桌面\backup issue\Mini120809-02.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

Symbol search path is: SRV*c:\localsymbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows Server 2003 Kernel Version 3790 (Service Pack 2) MP (2 procs) Free x86 compatible
Product: Server, suite: TerminalServer SingleUserTS
Built by: 3790.srv03_sp2_gdr.080813-1204
Machine Name:
Kernel base = 0x80800000 PsLoadedModuleList = 0x808a6ea8
Debug session time: Tue Dec  8 11:16:59.984 2009 (GMT+8)
System Uptime: 0 days 0:28:28.765
Loading Kernel Symbols
...............................................................
..................................................
Loading User Symbols
Loading unloaded module list
.....
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 80, {4f4454, 0, 0, 0}

Probably caused by : ntkrpamp.exe ( nt!KiTrap02+136 )

Followup: MachineOwner
---------

1: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

NMI_HARDWARE_FAILURE (80)
This is typically due to a hardware malfunction.  The hardware supplier should
be called.
Arguments:
Arg1: 004f4454
Arg2: 00000000
Arg3: 00000000
Arg4: 00000000

Debugging Details:
------------------

CUSTOMER_CRASH_COUNT:  2

DEFAULT_BUCKET_ID:  DRIVER_FAULT_SERVER_MINIDUMP

BUGCHECK_STR:  0x80

PROCESS_NAME:  bpbkar32.exe

CURRENT_IRQL:  2

LAST_CONTROL_TRANSFER:  from 80a60df7 to 80827c83

STACK_TEXT: 
f772d2cc 80a60df7 00000080 004f4454 00000000 nt!KeBugCheckEx+0x1b
f772d31c 8088a74e 00000000 80897818 8089a9ca hal!HalHandleNMI+0x1a5
f772d31c 80a5e3f0 00000000 80897818 8089a9ca nt!KiTrap02+0x136
f772d3a0 f78b2d40 ff180010 00000000 00000000 hal!KeAcquireSpinLockRaiseToSynch+0x20
WARNING: Frame IP not in any known module. Following frames may be wrong.
f772d3a4 ff180010 00000000 00000000 00000000 0xf78b2d40
f772d3a8 00000000 00000000 00000000 00000000 0xff180010

STACK_COMMAND:  kb

FOLLOWUP_IP:
nt!KiTrap02+136
8088a74e ff0dd0a98980    dec     dword ptr [nt!KiAbiosPresent+0x4c (8089a9d0)]

SYMBOL_STACK_INDEX:  2

SYMBOL_NAME:  nt!KiTrap02+136

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: nt

IMAGE_NAME:  ntkrpamp.exe

DEBUG_FLR_IMAGE_TIMESTAMP:  48a2ac75

FAILURE_BUCKET_ID:  0x80_nt!KiTrap02+136

BUCKET_ID:  0x80_nt!KiTrap02+136

Followup: MachineOwner
---------

请帮忙分析一下

谢谢

Re: windows 2003 系统挂死，用NMI生产dump请指点，万分感谢

---

格蠹老雷 2009-12-08, 23:08 下午

关键是找到NMI中断前CPU所处的状态。不妨尝试执行!pcr看一下CPU的状态；

f772d3a4 ff180010 00000000 00000000 00000000 0xf78b2d40

f772d3a8 00000000 00000000 00000000 00000000 0xff180010

最后两个返回地址（0xf78b2d40、0xff180010）对应不到任何模块么？

尝试.reload; ln 0xf78b2d40

Re: windows 2003 系统挂死，用NMI生产dump请指点，万分感谢

---

Gaoquan 2009-12-09, 18:08 下午
谢谢张老师
!pcr 结果如下，用ln 0xf78b2d40找不到任何模块
1: kd> !pcr
KPCR for Processor 1 at f7727000:
Major 1 Minor 1
NtTib.ExceptionList: ffffffff
NtTib.StackBase: 00000000
NtTib.StackLimit: 00000000
NtTib.SubSystemTib: f7727fe0
NtTib.Version: 000b740b
NtTib.UserPointer: 00000002
NtTib.SelfTib: 7ffdd000

SelfPcr: f7727000
Prcb: f7727120
Irql: 0000001f
IRR: 00000000
IDR: ffffffff
InterruptMode: 00000000
IDT: f772d800
GDT: f772d400
TSS: f772a350

CurrentThread: 88e14db0
NextThread: 89e873f0
IdleThread: f772a090

DpcQueue: 0x89d2be54 KPCR for Processor 1 at f7727000:
Major 1 Minor 1
NtTib.ExceptionList: ffffffff
NtTib.StackBase: 00000000
NtTib.StackLimit: 00000000
NtTib.SubSystemTib: f7727fe0
NtTib.Version: 000b740b
NtTib.UserPointer: 00000002
NtTib.SelfTib: 7ffdd000

SelfPcr: f7727000
Prcb: f7727120
Irql: 0000001f
IRR: 00000000
IDR: ffffffff
InterruptMode: 00000000
IDT: f772d800
GDT: f772d400
TSS: f772a350

CurrentThread: 88e14db0
NextThread: 89e873f0
IdleThread: f772a090

DpcQueue: 0x89d2be54 *** ERROR: Module load completed but symbols could not be loaded for bxvbdx.sys
*** ERROR: Module load completed but symbols could not be loaded for bxvbdx.sys
0xf7862bea [Normal] bxvbdx
0x8a08e72c 0xf7862bea [Normal] bxvbdx
0x8a08e72c *** ERROR: Module load completed but symbols could not be loaded for percsas.sys
*** ERROR: Module load completed but symbols could not be loaded for percsas.sys
0xf74d9f7a [Normal] percsas
0xb814b7a0 0xb80e275c [Normal] tcpip!TCBTimeoutdpc
0x8a0c709c 0xf72aea90 [Normal] atapi!IdePortCompletionDpc
0x8a0a2ecc 0xf735e4e6 [Normal] ACPI!ACPIInterruptServiceRoutineDPC
0x89d2fe54 0xf7862bea [Normal] bxvbdx

0xf74d9f7a [Normal] percsas
0xb814b7a0 0xb80e275c [Normal] tcpip!TCBTimeoutdpc
0x8a0c709c 0xf72aea90 [Normal] atapi!IdePortCompletionDpc
0x8a0a2ecc 0xf735e4e6 [Normal] ACPI!ACPIInterruptServiceRoutineDPC
0x89d2fe54 0xf7862bea [Normal] bxvbdx

1: kd> ln 0xf78b2d40
1: kd> ln 0xf78b2d40

Re: windows 2003 系统挂死，用NMI生产dump请指点，万分感谢

---

Gaoquan 2009-12-09, 22:47 下午


莫非cpu没有办法处理dpc队列里面的请求？

Re: windows 2003 系统挂死，用NMI生产dump请指点，万分感谢

---

Gaoquan 2009-12-09, 23:21 下午
张老师
我刚才用kvn，04号桢栈有问题，但是又列不出模块名字
1: kd> kvn
# ChildEBP RetAddr Args to Child
00 f772d2cc 80a60df7 00000080 004f4454 00000000 nt!KeBugCheckEx+0x1b (FPO: [5,0,0])
01 f772d31c 8088a74e 00000000 80897818 8089a9ca hal!HalHandleNMI+0x1a5 (FPO: [1,10,0])
02 f772d31c 80a5e3f0 00000000 80897818 8089a9ca nt!KiTrap02+0x136 (FPO: [0,0] TrapFrame @ f772d330)
03 f772d3a0 f78b2d40 ff180010 00000000 00000000 hal!KeAcquireSpinLockRaiseToSynch+0x20 (FPO: [0,0,0])
# ChildEBP RetAddr Args to Child
00 f772d2cc 80a60df7 00000080 004f4454 00000000 nt!KeBugCheckEx+0x1b (FPO: [5,0,0])
01 f772d31c 8088a74e 00000000 80897818 8089a9ca hal!HalHandleNMI+0x1a5 (FPO: [1,10,0])
02 f772d31c 80a5e3f0 00000000 80897818 8089a9ca nt!KiTrap02+0x136 (FPO: [0,0] TrapFrame @ f772d330)
03 f772d3a0 f78b2d40 ff180010 00000000 00000000 hal!KeAcquireSpinLockRaiseToSynch+0x20 (FPO: [0,0,0])
*** ERROR: Module load completed but symbols could not be loaded for vsp.sys
WARNING: Frame IP not in any known module. Following frames may be wrong.
*** ERROR: Module load completed but symbols could not be loaded for vsp.sys
WARNING: Frame IP not in any known module. Following frames may be wrong.
04 f78b2d4c f74fb483 8a248eb8 f78b2e20 f78b2dbc 0xf78b2d40
05 f78b2d98 809b523e 00000000 8a248e02 87850bb8 vsp+0x4483
06 f78b2dbc 8081e123 00000000 8a248eb8 f78b2e20 nt!IovpLocalCompletionRoutine+0xb4 (FPO: [3,1,4])
07 f78b2dec 809b577e 89c971c0 00000000 89e8f848 nt!IopfCompleteRequest+0xcd (FPO: [0,4,4])
08 f78b2e58 f7276829 f78b2e88 f72763ec 89e93738 nt!IovCompleteRequest+0x9a (FPO: [0,22,4])
09 f78b2e60 f72763ec 89e93738 8a248eb8 00000001 CLASSPNP!ClassCompleteRequest+0x11 (FPO: [3,0,0])
0a f78b2e88 809b523e 00000000 8a20af48 89c971c0 CLASSPNP!TransferPktComplete+0x1fd (FPO: [3,2,4])
0b f78b2eac 8081e123 00000000 8a20af48 f78b2f10 nt!IovpLocalCompletionRoutine+0xb4 (FPO: [3,1,4])
0c f78b2edc 809b577e 00000000 8a20af48 89e950e8 nt!IopfCompleteRequest+0xcd (FPO: [0,4,4])
0d f78b2f48 f72892c2 89c9726c f78b2f8c f72949e5 nt!IovCompleteRequest+0x9a (FPO: [0,22,4])
0e f78b2f54 f72949e5 8a20af48 00000001 00000000 storport!RaidCompleteRequestEx+0x1c (FPO: [3,0,4])
0f f78b2f8c f7289768 89edb938 8a0c6a8c f78b2ff4 04 f78b2d4c f74fb483 8a248eb8 f78b2e20 f78b2dbc 0xf78b2d40
05 f78b2d98 809b523e 00000000 8a248e02 87850bb8 vsp+0x4483
06 f78b2dbc 8081e123 00000000 8a248eb8 f78b2e20 nt!IovpLocalCompletionRoutine+0xb4 (FPO: [3,1,4])
07 f78b2dec 809b577e 89c971c0 00000000 89e8f848 nt!IopfCompleteRequest+0xcd (FPO: [0,4,4])
08 f78b2e58 f7276829 f78b2e88 f72763ec 89e93738 nt!IovCompleteRequest+0x9a (FPO: [0,22,4])
09 f78b2e60 f72763ec 89e93738 8a248eb8 00000001 CLASSPNP!ClassCompleteRequest+0x11 (FPO: [3,0,0])
0a f78b2e88 809b523e 00000000 8a20af48 89c971c0 CLASSPNP!TransferPktComplete+0x1fd (FPO: [3,2,4])
0b f78b2eac 8081e123 00000000 8a20af48 f78b2f10 nt!IovpLocalCompletionRoutine+0xb4 (FPO: [3,1,4])
0c f78b2edc 809b577e 00000000 8a20af48 89e950e8 nt!IopfCompleteRequest+0xcd (FPO: [0,4,4])
0d f78b2f48 f72892c2 89c9726c f78b2f8c f72949e5 nt!IovCompleteRequest+0x9a (FPO: [0,22,4])
0e f78b2f54 f72949e5 8a20af48 00000001 00000000 storport!RaidCompleteRequestEx+0x1c (FPO: [3,0,4])
0f f78b2f8c f7289768 89edb938 8a0c6a8c f78b2ff4 storport!RaidUnitCompleteRequest+0x8f (FPO: [1,6,4])
10 f78b2f9c 80832110 8a0c6a8c 8a0c6a18 00000000 storport!RaidpAdapterDpcRoutine+0x28 (FPO: [4,0,4])
11 f78b2ff4 8088db57 b5c577bc 00000000 00000000 nt!KiRetireDpcList+0xca (FPO: [0,13,4])
12 f78b2ff8 b5c577bc 00000000 00000000 00000000 nt!KiDispatchInterrupt+0x37 (FPO: [Uses EBP] [0,0,1])
13 8088db57 00000000 0000000a 0083850f bb830000 0xb5c577bc
storport!RaidUnitCompleteRequest+0x8f (FPO: [1,6,4])
10 f78b2f9c 80832110 8a0c6a8c 8a0c6a18 00000000 storport!RaidpAdapterDpcRoutine+0x28 (FPO: [4,0,4])
11 f78b2ff4 8088db57 b5c577bc 00000000 00000000 nt!KiRetireDpcList+0xca (FPO: [0,13,4])
12 f78b2ff8 b5c577bc 00000000 00000000 00000000 nt!KiDispatchInterrupt+0x37 (FPO: [Uses EBP] [0,0,1])
13 8088db57 00000000 0000000a 0083850f bb830000 0xb5c577bc
1: kd> lm m 0xf78b2d40
1: kd> lm m 0xf78b2d40
start end module name
start end module name

Re: windows 2003 系统挂死，用NMI生产dump请指点，万分感谢

---

Gaoquan 2009-12-09, 23:28 下午
ub 04号桢栈结果如下，这样是否可以判断引起故障的模块就是vsp呢？

1: kd> ub f74fb483
1: kd> ub f74fb483
vsp+0x4462:
f74fb462 83c60c add esi,0Ch
f74fb465 c70600000000 mov dword ptr [esi],0
f74fb46b 6a01 push 1
f74fb46d 56 push esi
f74fb46e ff1578734ff7 call dword ptr [vsp+0x378 (f74f7378)]
f74fb474 8b4b28 mov ecx,dword ptr [ebx+28h]
f74fb477 81c154010000 add ecx,154h
f74fb47d ff15a0724ff7 call dword ptr [vsp+0x2a0 (f74f72a0)]

Re: windows 2003 系统挂死，用NMI生产dump请指点，万分感谢

---

格蠹老雷 2009-12-10, 10:13 上午

如果需要我帮忙，那么请把转储文件发到我的信箱，需要亲自看一下...