代码之家 › 专栏 › 技术社区 › Diego Jancic

.NET应用程序挂起。奇怪的垃圾堆

crash-dumps memory .net

Diego Jancic · 技术社区 · 14 年前

生产服务器有一个全天候运行的应用程序,有时它开始消耗接近50%的CPU。我无法在本地复制它,但我用adplus做了内存转储。服务器有64位的Windows2008服务器,.NET3.5。应用程序监听msmq并有一个线程池来执行操作,当我进行内存转储时,它应该什么也不运行,但它仍然在消耗cpu。

下面是垃圾场。你知道会发生什么吗??奇怪的是,慢线程还说gc已禁用!那是什么?

奇怪的是,我在同一台服务器上有一个asp.net应用程序,有时会发生同样的事情。

0:000> !threads
ThreadCount: 23
UnstartedThread: 0
BackgroundThread: 6
PendingThread: 0
DeadThread: 5
Hosted Runtime: no
                                              PreEmptive                                                Lock
       ID OSID        ThreadOBJ     State   GC     GC Alloc Context                  Domain           Count APT Exception
   0    1 1ec0 0000000000249c10      a020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
   2    2  84c 0000000000253470      b220 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA (Finalizer)
   3    3 10b8 0000000019d0f900    80a220 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA (Threadpool Completion Port)
   5    4 2184 0000000019d4a550   880b220 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA (Threadpool Completion Port)
   6    6 14a0 0000000019d602a0   180b220 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA (Threadpool Worker)
   8    7  490 0000000019d7db70   200b020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
   9    8 2164 0000000019dae680   200b020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
  10    9  ac0 0000000019db4740   200b020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
  11    a 24cc 0000000019db51f0   200b020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
  12    b 1fb4 0000000019db5ca0   200b020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
  13    c 2408 0000000019db1520   200b020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
  14    d 1b44 0000000019db1fd0   200b020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
  15    e 1280 0000000019db2a80   200b020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
  16    f 206c 0000000019db8780   200b020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
  17   10 1ff8 0000000019db9230   200b020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
  18   11 2548 0000000019dc2120   200b020 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA
  20   18 2588 0000000019dc49d0   180b220 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 MTA (Threadpool Worker)
  21   1a 20a0 0000000019dc5570      b220 Disabled 0000000001653d88:0000000001655978 00000000002410b0     1 MTA
XXXX   26    0 0000000019dc89c0      9820 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 Ukn
XXXX   2c    0 0000000019dc5b40      9820 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 Ukn
XXXX   2b    0 0000000019dc3290      9820 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 Ukn
XXXX    5    0 0000000019dc8f90      9820 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 Ukn
XXXX   27    0 0000000019dc7850      9820 Enabled  0000000000000000:0000000000000000 00000000002410b0     0 Ukn

 0:000> !runaway
 User Mode Time
  Thread       Time
  **21:20a0      0 days 0:27:21.718**
   6:14a0      0 days 0:00:01.921
  18:2548      0 days 0:00:01.015
   2:84c       0 days 0:00:00.890
   (18 more....)

0:000> ~21s
*** WARNING: Unable to verify checksum for System.ni.dll
mscorwks!JIT_WriteBarrier_Fast+0x3:
000007fe`f88de5c3 4881fa607f4701  cmp     rdx,1477F60h

0:021> !clrstack
OS Thread Id: 0x20a0 (21)
*** WARNING: Unable to verify checksum for mscorlib.ni.dll
Child-SP         RetAddr          Call Site
000000001b05f090 000007fef651b1ba System.Net.TimerThread+TimerNode..ctor(Callback, System.Object, Int32, System.Object)
000000001b05f0d0 000007fef69a0c3f System.Net.TimerThread+TimerQueue.CreateTimer(Callback, System.Object)
000000001b05f180 000007fef651b73e System.Net.ConnectionPool.CleanupCallbackWrapper(Timer, Int32, System.Object)
000000001b05f1d0 000007fef651b560 System.Net.TimerThread+TimerNode.Fire()
000000001b05f2a0 000007fef64e6388 System.Net.TimerThread+TimerQueue.Fire(Int32 ByRef)
000000001b05f330 000007fef271175b System.Net.TimerThread.ThreadProc()
000000001b05f400 000007fef27a95fd System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
000000001b05f450 000007fef88e1552 System.Threading.ThreadHelper.ThreadStart()
0:021> kb
RetAddr           : Args to Child      : Call Site
000007fe`f6a1b87e : 00000000 (Edited) : mscorwks!JIT_WriteBarrier_Fast+0x3
000007fe`f651b1ba : 00000000 (Edited) : System_ni+0x70b87e
000007fe`f69a0c3f : 00000000 (Edited) : System_ni+0x20b1ba
000007fe`f651b73e : 000007fe (Edited) : System_ni+0x690c3f
000007fe`f651b560 : 00000000 (Edited) : System_ni+0x20b73e
000007fe`f64e6388 : 00000000 (Edited) : System_ni+0x20b560
000007fe`f271175b : 00000000 (Edited) : System_ni+0x1d6388
000007fe`f27a95fd : 00000000 (Edited) : mscorlib_ni+0x2f175b
000007fe`f88e1552 : 00000000 (Edited) : mscorlib_ni+0x3895fd
000007fe`f877e5e3 : 00000000 (Edited) : mscorwks!CallDescrWorker+0x82
000007fe`f878c83f : 00000000 (Edited) : mscorwks!CallDescrWorkerWithHandler+0xd3
000007fe`f887ae8d : 00000000 (Edited) : mscorwks!MethodDesc::CallDescr+0x24f
000007fe`f8839374 : 00000000 (Edited) : mscorwks!ThreadNative::KickOffThread_Worker+0x191
000007fe`f8732045 : 00000000 (Edited) : mscorwks!TypeHandle::GetParent+0x5c
000007fe`f8846139 : 00000000 (Edited) : mscorwks!SVR::gc_heap::make_heap_segment+0x155
000007fe`f8700e15 : 00000000 (Edited) : mscorwks!ZapStubPrecode::GetType+0x39
000007fe`f8700ae7 : 00000000 (Edited) : mscorwks!ThreadNative::KickOffThread+0x401
000007fe`f88614fc : 00000002 (Edited) : mscorwks!ThreadNative::KickOffThread+0xd3
00000000`76bdbe3d : 00000000 (Edited) : mscorwks!Thread::intermediateThreadProc+0x78
00000000`76d16a51 : 00000000 (Edited) : kernel32!BaseThreadInitThunk+0xd

1 回复 | 直到 14 年前

Community Reversed Engineer 7 年前

我在这里还不是专家,但还有一些有用的信息:

根据 this ,gc线程是在clr启动时创建的,至少对于服务器gc是这样,所以没有足够的线程来运行gc可能是不可能的;-)
的“gc”列中的“disabled” 21号线意味着它决定不被最终GC抢先操作。当线程上的代码决定做一个关键的手术不应受到GC的干扰(如装载和组装,因此融合)。
从“kb”命令输出我猜你实际上在用服务器GC(StackFrame “祝你好运!svr::gc_heap::make_heap_segment“; 工作站GC会有一些东西使用类/命名空间“wks”)。这并不出乎意料是“服务器”上的默认值操作系统。你应该用“!eeversion“命令。此外,您还应该了解您拥有多少内核,因为如果服务器gc运行,
它将使用尽可能多的线程(每个逻辑/物理核心一个)。

可能是计时器经常触发,或者比前一个计时器完成得更快。您可以使用“!threadpool“命令。

另外,您可能需要检查方法和局部变量的实际参数(!clrstack-a)和/或堆栈上的当前对象(!DSO)。也许这能给我们更多的启示。

不出所料,一些搜索“system.net.connectionpool.cleanupcallbackwrapper”会产生以下链接,这可能是你的问题吧?