Green Dragon's Lair

The root cause for application performance problems might seem to be totally unrelated

I just read a post on If broken it is, fix it you should, where he's debugging a performance problem in an ASP.Net application.

The most recent one was one where the ASP.NET site would respond slowly at regular intervals (read: it grinded to a halt every few hours). 

First shot at gathering data:

We started off by getting a memory dump with debug diag when the process was responding slowly.  Unfortunately at the time that the dump was taken no requests were executing (i.e. ~* e !clrstack showed no managed stacks and ~* kb showed all the threads in their idle states), so we didn't get much of a clue as to why the process was responding so slowly. 

It's an interesting read, but the conclusion was even more surprising:

Why did the appdomain restart?

After adding logging per the "ASP.NET Case Study: Lost session variables and appdomain recycles" article the following showed up in the eventlog

_shutDownStack:    at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
   at System.Environment.get_StackTrace()
   at System.Web.Hosting.HostingEnvironment.InitiateShutdownInternal()
   at System.Web.Hosting.HostingEnvironment.InitiateShutdown()
   at System.Web.HttpRuntime.ShutdownAppDomain(String stackTrace)
   at System.Web.HttpRuntime.OnCriticalDirectoryChange(Object sender, FileChangeEvent e)
   at System.Web.FileChangesMonitor.OnCriticaldirChange(Object sender, FileChangeEvent e)
   at System.Web.DirectoryMonitor.FireNotifications()
   at System.Web.Util.WorkItem.CallCallbackWithAssert(WorkItemCallback callback)
   at System.Web.Util.WorkItem.OnQueueUserWorkItemCompletion(Object state)
   at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)
_shutDownMessage: Change Notification for critical directories.
bin dir change or directory rename
HostingEnvironment initiated shutdown
HostingEnvironment caused shutdown
Change in C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\root\5fe0fce6\53087c60\hash\hash.web

The next step was to figure out who was touching the hash.web by monitoring the file with process monitor, and It turned out, it was antivirus scanning the file, which in turn triggered the filechangesmonitor to recycle the appdomain.


Posted May 24 2008, 01:11 PM by admin

Add a Comment

(required)  
(optional)
(required)  
Remember Me?
(C) Miki Watts
Powered by Community Server (Non-Commercial Edition), by Telligent Systems