Troubleshooting high CPU utilization in ASP.NET worker process

High CPU analysis in ASP.NET worker process

Server I was asked to troubleshoot had high CPU utilization in w3wp.exe process. There were in fact 2 w3wp.exe processes for 2 application pools existing on system. Below are steps taken to troubleshoot the issue.

1. You need to identify which application pool is causing high utilization, you can do it in a number of ways. I usually launch process explorer (download and put it under /system32 directory on all production machines).
Launch process explorer and find w3wp.exe process which takes high CPU, double click on that process and on "Image" scroll to end of "Command Line" field, application pool name is passed as -ap parameter.
See ScreenShot below.

Identifying application pool will allow first recycle it if need (this will not get rid of high CPU utilization but will allow your website to continue running after you kill this runaway pool).


2. Create FULL memory dump of process in question. If you are on Windows 2008, you can just go to task manager, right click on process and choose "Create Dump File". This will generate dump file with size of virtual memory for that process.

3. Copy dump file to your workstation where you will perform dump file analysis and launch windbg. Make sure symbols are configured already.

• load sos module for debugging managed code

0:014> .loadby sos mscorwks

• Find which threads are occupying CPU

0:014> !runaway
 User Mode Time
  Thread       Time
  14:f8c       0 days 0:25:57.953
  17:738       0 days 0:21:42.312
  25:1d8       0 days 0:00:03.156
   4:1600      0 days 0:00:00.609
   3:1294      0 days 0:00:00.421
  11:e24       0 days 0:00:00.109
  22:c14       0 days 0:00:00.031
  19:150c      0 days 0:00:00.031
   0:a78       0 days 0:00:00.031
  15:123c      0 days 0:00:00.015

We can see threads 14 and 17 are monopolizing CPU

• Switch to thread in question and see managed stack

0:014> ~14s

ntdll!ZwGetContextThread+0xa:

00000000`77c6783a c3              ret

0:014> !clrstack

OS Thread Id: 0xf8c (14)

Child-SP         RetAddr          Call Site

000000001c51d5b0 000007fef764753d Company.SiebelExtract.Services.BusinessRules.ServiceLookups.b__a(System.Data.DataRow, System.Data.DataRow)

000000001c51d5f0 000007fef7645f73 System.Linq.Enumerable+d__31`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].MoveNext()

000000001c51d660 000007fef764a023 System.Linq.Enumerable+WhereSelectEnumerableIterator`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].MoveNext()

000000001c51d6b0 000007ff001f40e7 System.Linq.Enumerable+d__7a`1[[System.__Canon, mscorlib]].MoveNext()

000000001c51d720 000007ff001ee868 Company.SiebelExtract.Services.BusinessRules.ServiceLookups.getAccountsList(System.String)

000000001c51d960 000007fefa7913b2 Company.SiebelExtract.Services.SiebelExtractLookup.lookupAssetAndEntitlementHeirarchyByGrantNumber(System.String)

000000001c51e310 000007fef8a69b46 System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo, Boolean)

000000001c51e4b0 000007fef2f44197 System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)

000000001c51e500 000007fef2fc3c5f System.Web.Services.Protocols.LogicalMethodInfo.Invoke(System.Object, System.Object[])

000000001c51e580 000007fef2fc412b System.Web.Services.Protocols.WebServiceHandler.Invoke()

000000001c51e610 000007fef2fc4c90 System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest()

000000001c51e680 000007fef35cf047 System.Web.Services.Protocols.SyncSessionlessHandler.ProcessRequest(System.Web.HttpContext)

000000001c51e6d0 000007fef0560347 System.Web.Script.Services.ScriptHandlerFactory+HandlerWrapper.ProcessRequest(System.Web.HttpContext)

000000001c51e700 000007fef05245cb System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()

000000001c51e7b0 000007fef0c0caa1 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)

000000001c51e850 000007fef0bfde22 System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)

000000001c51e9e0 000007fef0bdf8f9 System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)

000000001c51ea30 000007fef0d06904 System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)

000000001c51eb50 000007fef0d05abb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)

000000001c51ecc0 000007fef0d05874 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)

000000001c51ed20 000007fefa78ef2a DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)

000000001c51f550 000007fef0d069e4 DomainNeutralILStubClass.IL_STUB(IntPtr, System.Web.RequestNotificationStatus ByRef)

000000001c51f630 000007fef0d05abb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)

000000001c51f7a0 000007fef0d05874 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)

000000001c51f800 000007fefa78f17b DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)


From stack it appears call which started high CPU originated at Company.SiebelExtract.Services.SiebelExtractLookup.lookupAssetAndEntitlementHeirarchyByGrantNumber(System.String)

• Find which page was being executed

0:014> !clrstack -p

OS Thread Id: 0xf8c (14)

Child-SP         RetAddr          Call Site

000000001c51d5b0 000007fef764753d McAfee.SiebelExtract.Services.BusinessRules.ServiceLookups.b__a(System.Data.DataRow, System.Data.DataRow)

    PARAMETERS:

        row =

        rowLink =

000000001c51d5f0 000007fef7645f73 System.Linq.Enumerable+d__31`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].MoveNext()

    PARAMETERS:

        this = 0x0000000004857198

000000001c51d660 000007fef764a023 System.Linq.Enumerable+WhereSelectEnumerableIterator`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].MoveNext()

    PARAMETERS:

        this =

000000001c51d6b0 000007ff001f40e7 System.Linq.Enumerable+d__7a`1[[System.__Canon, mscorlib]].MoveNext()

    PARAMETERS:

        this = 0x0000000004857288

000000001c51d720 000007ff001ee868 McAfee.SiebelExtract.Services.BusinessRules.ServiceLookups.getAccountsList(System.String)

    PARAMETERS:

        GrantNumber =

000000001c51d960 000007fefa7913b2 McAfee.SiebelExtract.Services.SiebelExtractLookup.lookupAssetAndEntitlementHeirarchyByGrantNumber(System.String)

    PARAMETERS:

        this =

        GrantNumber = 0x0000000002c75a90

000000001c51e310 000007fef8a69b46 System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo, Boolean)

    PARAMETERS:

        this =

        obj =

        invokeAttr =

        binder =

        parameters =

        culture =

        skipVisibilityChecks =

000000001c51e4b0 000007fef2f44197 System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)

    PARAMETERS:

        this =

        obj =

        invokeAttr =

        binder =

        parameters =

        culture =

000000001c51e500 000007fef2fc3c5f System.Web.Services.Protocols.LogicalMethodInfo.Invoke(System.Object, System.Object[])

    PARAMETERS:

        this =

        target =

        values =

000000001c51e580 000007fef2fc412b System.Web.Services.Protocols.WebServiceHandler.Invoke()

    PARAMETERS:

        this = 0x0000000002c733b8

000000001c51e610 000007fef2fc4c90 System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest()

    PARAMETERS:

        this = 0x0000000002c733b8

000000001c51e680 000007fef35cf047 System.Web.Services.Protocols.SyncSessionlessHandler.ProcessRequest(System.Web.HttpContext)

    PARAMETERS:

        this =

        context =

000000001c51e6d0 000007fef0560347 System.Web.Script.Services.ScriptHandlerFactory+HandlerWrapper.ProcessRequest(System.Web.HttpContext)

    PARAMETERS:

        this =

        context =

000000001c51e700 000007fef05245cb System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()

    PARAMETERS:

        this =

000000001c51e7b0 000007fef0c0caa1 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)

    PARAMETERS:

        this = 0x0000000002c272a0

        step = 0x0000000002c2a678

        completedSynchronously = 0x000000001c51e908

000000001c51e850 000007fef0bfde22 System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)

    PARAMETERS:

        this = 0x0000000002c2a4b0

        error =

000000001c51e9e0 000007fef0bdf8f9 System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)

    PARAMETERS:

        this =

        context =

        cb =

000000001c51ea30 000007fef0d06904 System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)

    PARAMETERS:

        this = 0x0000000002a45fc0

        wr = 0x0000000002c25080

        context = 0x0000000002c25400

000000001c51eb50 000007fef0d05abb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)

    PARAMETERS:

        managedHttpContext =

        nativeRequestContext = 0x0000000002358f98

        moduleData =

        flags =

000000001c51ecc0 000007fef0d05874 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)

    PARAMETERS:

        managedHttpContext =

        nativeRequestContext =

        moduleData =

        flags =

000000001c51ed20 000007fefa78ef2a DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)

    PARAMETERS:

       

       

       

       

000000001c51f550 000007fef0d069e4 DomainNeutralILStubClass.IL_STUB(IntPtr, System.Web.RequestNotificationStatus ByRef)

    PARAMETERS:

       

000000001c51f630 000007fef0d05abb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)

    PARAMETERS:

        managedHttpContext =

        nativeRequestContext =

        moduleData =

        flags =

000000001c51f7a0 000007fef0d05874 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)

    PARAMETERS:

        managedHttpContext =

        nativeRequestContext =

        moduleData =

        flags =

000000001c51f800 000007fefa78f17b DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)

    PARAMETERS:

       

       

       

       

We need to find "context" object address and dump that object


0:014> !do 0x0000000002c25400
Name: System.Web.HttpContext
MethodTable: 000007fef06b0f28
EEClass: 000007fef0312398
Size: 336(0x150) bytes
 (C:\Windows\assembly\GAC_64\System.Web\2.0.0.0__b03f5f7f11d50a3a\System.Web.dll)
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
000007fef06b3a50  4000fe6        8 ...IHttpAsyncHandler  0 instance 0000000000000000 _asyncAppHandler
000007fef06b3580  4000fe7       10 ...b.HttpApplication  0 instance 0000000002c272a0 _appInstance
000007fef06b3ac0  4000fe8       18 ....Web.IHttpHandler  0 instance 0000000002c733f8 _handler
000007fef06b3f00  4000fe9       20 ...m.Web.HttpRequest  0 instance 0000000002c25550 _request
000007fef06b4280  4000fea       28 ....Web.HttpResponse  0 instance 0000000002c256a0 _response
000007fef06b4798  4000feb       30 ...HttpServerUtility  0 instance 0000000000000000 _server
000007fef8b2fcf8  4000fec       38 ...Collections.Stack  0 instance 0000000000000000 _traceContextStack
000007fef06b9ad0  4000fed       40 ....Web.TraceContext  0 instance 0000000000000000 _topTraceContext
000007fef8b3e0b0  4000fee       48 ...ections.Hashtable  0 instance 0000000002c397d8 _items
000007fef8b3d640  4000fef       50 ...ections.ArrayList  0 instance 0000000000000000 _errors
000007fef8b36a20  4000ff0       58     System.Exception  0 instance 0000000000000000 _tempError
000007fef8b35928  4000ff1      118       System.Boolean  1 instance                0 _errorCleared
000007fef8b516b0  4000ff2       60 ...ncipal.IPrincipal  0 instance 0000000002c2bc60 _user
000007fef8b39160  4000ff3      100        System.IntPtr  1 instance                0 _pManagedPrincipal
000007fef06b31f8  4000ff4       68 ...ofile.ProfileBase  0 instance 0000000000000000 _Profile
000007fef8b76c90  4000ff5      128      System.DateTime  1 instance 0000000002c25528 _utcTimestamp
000007fef06b8bf0  4000ff6       70 ...HttpWorkerRequest  0 instance 0000000002c25080 _wr
000007fef8b32528  4000ff7      130 ...Services.GCHandle  1 instance 0000000002c25530 _root
000007fef8b39160  4000ff8      108        System.IntPtr  1 instance          21c18e0 _ctxPtr
000007fef06ab060  4000ff9       78 ...m.Web.VirtualPath  0 instance 0000000000000000 _configurationPath
000007fef8b35928  4000ffa      119       System.Boolean  1 instance                0 _skipAuthorization
000007fef8b3f0d8  4000ffb       80 ...ation.CultureInfo  0 instance 0000000002a40270 _dynamicCulture
000007fef8b3f0d8  4000ffc       88 ...ation.CultureInfo  0 instance 0000000002a40270 _dynamicUICulture
000007fef8b3d9c8  4000ffd      110         System.Int32  1 instance                0 _serverExecuteDepth
000007fef8b2fcf8  4000ffe       90 ...Collections.Stack  0 instance 0000000000000000 _handlerStack
000007fef8b35928  4000fff      11a       System.Boolean  1 instance                0 _preventPostback
000007fef8b35928  4001000      11b       System.Boolean  1 instance                0 _runtimeErrorReported
000007fef8b35928  4001001      11c       System.Boolean  1 instance                1 _firstNotificationInitCalled
000007fef8b76c90  4001002      138      System.DateTime  1 instance 0000000002c25538 _timeoutStartTime
000007fef8b35928  4001003      11d       System.Boolean  1 instance                1 _timeoutSet
000007fef8b76b90  4001004      140      System.TimeSpan  1 instance 0000000002c25540 _timeout
000007fef8b3d9c8  4001005      114         System.Int32  1 instance                1 _timeoutState
000007fef06c0fa8  4001006       98 ...b.Util.DoubleLink  0 instance 0000000002c2bbb8 _timeoutLink
000007fef8b371f8  4001007       a0 ....Threading.Thread  0 instance 0000000002b0ac58 _thread
000007fef06ad480  4001008       a8 ...eb.CachedPathData  0 instance 0000000000000000 _configurationPathData
000007fef06ad480  4001009       b0 ...eb.CachedPathData  0 instance 0000000002c2b558 _filePathData
000007fef8b36758  400100a       b8        System.String  0 instance 0000000000000000 _sqlDependencyCookie
000007fef06bd218  400100b       c0 ...essionStateModule  0 instance 0000000000000000 _sessionStateModule
000007fef0623cf0  400100c       c8 ...I.TemplateControl  0 instance 0000000000000000 _templateControl
000007fef0684ad0  400100d       d0 ...tificationContext  0 instance 0000000002c734f0 _notificationContext
000007fef8b35928  400100e      11e       System.Boolean  1 instance                1 _isAppInitialized
000007fef8b35928  400100f      11f       System.Boolean  1 instance                1 _isIntegratedPipeline
000007fef8b35928  4001010      120       System.Boolean  1 instance                0 _finishPipelineRequestCalled
000007fef8b35928  4001011      121       System.Boolean  1 instance                0 _impersonationEnabled
000007fef8b35928  4001012      122       System.Boolean  1 instance                0 HideRequestResponse
000007fef8b35928  4001013      123       System.Boolean  1 instance                1 InIndicateCompletion
000007fef06b4d78  4001014       d8 ...ion+ThreadContext  0 instance 0000000002c2bac0 IndicateCompletionContext
000007fef06c0e78  4001015       e0 ...ronizationContext  0 instance 0000000002c2ba88 _syncContext
000007fef8b35928  4001016      124       System.Boolean  1 instance                0 RequiresSessionState
000007fef8b35928  4001017      125       System.Boolean  1 instance                0 ReadOnlySessionState
000007fef8b35928  4001018      126       System.Boolean  1 instance                0 InAspCompatMode
000007fef06b3ac0  4001019       e8 ....Web.IHttpHandler  0 instance 0000000000000000 _remapHandler
000007fef06b3ac0  400101a       f0 ....Web.IHttpHandler  0 instance 0000000000000000 _currentHandler
000007fef8b35928  400101b      127       System.Boolean  1 instance                1 _ProfileDelayLoad
000007fef06b96d8  400101c       f8 ...ielessHelperClass  0 instance 0000000002c257e0 _CookielessHelper
000007fef8b39218  4000fe5      1a0 ...flection.Assembly  0   shared           static SystemWebAssembly


Get request object and dump it



0:014> !do 0000000002c25550
Name: System.Web.HttpRequest
MethodTable: 000007fef06b3f00
EEClass: 000007fef0313c90
Size: 336(0x150) bytes
 (C:\Windows\assembly\GAC_64\System.Web\2.0.0.0__b03f5f7f11d50a3a\System.Web.dll)
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
000007fef06b8bf0  400107a        8 ...HttpWorkerRequest  0 instance 0000000002c25080 _wr
000007fef06b0f28  400107b       10 ...m.Web.HttpContext  0 instance 0000000002c25400 _context
000007fef8b36758  400107c       18        System.String  0 instance 0000000002c27278 _httpMethod
000007fef066d450  400107d      130         System.Int32  1 instance                5 _httpVerb
000007fef8b36758  400107e       20        System.String  0 instance 0000000000000000 _requestType
000007fef06ab060  400107f       28 ...m.Web.VirtualPath  0 instance 0000000002c2ee60 _path
000007fef8b36758  4001080       30        System.String  0 instance 0000000000000000 _rewrittenUrl
000007fef8b35928  4001081      138       System.Boolean  1 instance                0 _computePathInfo
000007fef06ab060  4001082       38 ...m.Web.VirtualPath  0 instance 0000000002c25808 _filePath
000007fef06ab060  4001083       40 ...m.Web.VirtualPath  0 instance 0000000000000000 _currentExecutionFilePath
000007fef06ab060  4001084       48 ...m.Web.VirtualPath  0 instance 0000000000000000 _pathInfo
000007fef8b36758  4001085       50        System.String  0 instance 0000000000000000 _queryStringText
000007fef8b35928  4001086      139       System.Boolean  1 instance                0 _queryStringOverriden
000007fef8b3e798  4001087       58        System.Byte[]  0 instance 0000000000000000 _queryStringBytes
000007fef8b36758  4001088       60        System.String  0 instance 0000000002c25170 _pathTranslated
000007fef8b36758  4001089       68        System.String  0 instance 0000000002c391c0 _contentType
000007fef8b3d9c8  400108a      134         System.Int32  1 instance             1394 _contentLength
000007fef8b36758  400108b       70        System.String  0 instance 0000000000000000 _clientTarget
000007fef8b24548  400108c       78      System.Object[]  0 instance 0000000000000000 _acceptTypes
000007fef8b24548  400108d       80      System.Object[]  0 instance 0000000000000000 _userLanguages
000007fef0620148  400108e       88 ...owserCapabilities  0 instance 0000000000000000 _browsercaps
000007fef7fd19a8  400108f       90           System.Uri  0 instance 0000000000000000 _url
000007fef7fd19a8  4001090       98           System.Uri  0 instance 0000000000000000 _referrer
000007fef06c2000  4001091       a0 ...b.HttpInputStream  0 instance 0000000002c73370 _inputStream
000007fef0671860  4001092       a8 ...ClientCertificate  0 instance 0000000000000000 _clientCertificate
000007fef8b25b30  4001093       b0 ...l.WindowsIdentity  0 instance 0000000000000000 _logonUserIdentity
000007fef061fb30  4001094       b8 ...tpValueCollection  0 instance 0000000000000000 _params
000007fef061fb30  4001095       c0 ...tpValueCollection  0 instance 0000000000000000 _queryString
000007fef061fb30  4001096       c8 ...tpValueCollection  0 instance 0000000000000000 _form
000007fef06c1048  4001097       d0 ...pHeaderCollection  0 instance 0000000002c38d70 _headers
000007fef061fcc0  4001098       d8 ...verVarsCollection  0 instance 0000000000000000 _serverVariables
000007fef06be4f8  4001099       e0 ...pCookieCollection  0 instance 0000000000000000 _cookies
000007fef0653b10  400109a       e8 ...ttpFileCollection  0 instance 0000000000000000 _files
000007fef06c15e0  400109b       f0 ...awUploadedContent  0 instance 0000000002c70d90 _rawContent
000007fef8b35928  400109c      13a       System.Boolean  1 instance                1 _readEntityBody
000007fef8b24548  400109d       f8      System.Object[]  0 instance 0000000000000000 _multipartContentElements
000007fef8b3e5a8  400109e      100 System.Text.Encoding  0 instance 0000000000000000 _encoding
000007fef0648ae0  400109f      108 ...treamFilterSource  0 instance 0000000000000000 _filterSource
000007fef8b39708  40010a0      110     System.IO.Stream  0 instance 0000000000000000 _installedFilter
000007fef061f530  40010a1      140 ...SimpleBitVector32  1 instance 0000000002c25690 _flags
000007fef8b36758  40010a4      118        System.String  0 instance 0000000000000000 _AnonymousId
000007fef06ab060  40010a5      120 ...m.Web.VirtualPath  0 instance 0000000002c25808 _clientFilePath
000007fef06ab060  40010a6      128 ...m.Web.VirtualPath  0 instance 0000000000000000 _clientBaseDir
000007fef8b36048  40010a2      1b0        System.Object  0   shared           static s_browserLock

Get _pathTranslated object and this will provide URL which is being executed

0:014> !do 0000000002c25170

Name: System.String

MethodTable: 000007fef8b36758

EEClass: 000007fef873e520

Size: 204(0xcc) bytes

 (C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)

String: E:\Inetpub\wwwroot\webservices\SiebelExtractInterface\2011\04\01\SiebelExtractLookup.asmx

Fields:

              MT    Field   Offset                 Type VT     Attr            Value Name

000007fef8b3d9c8  4000096        8         System.Int32  1 instance               90 m_arrayLength

000007fef8b3d9c8  4000097        c         System.Int32  1 instance               89 m_stringLength

000007fef8b381a0  4000098       10          System.Char  1 instance               45 m_firstChar

000007fef8b36758  4000099       20        System.String  0   shared           static Empty

                                 >> Domain:Value  00000000023f1fb0:0000000002981308 000000000246c1c0:0000000002981308 00000000024cea50:0000000002981308 00000000024e8b60:0000000002981308 000000001c578a20:0000000002981308 000000001c58f590:0000000002981308 0000000026905040:0000000002981308 0000000026941740:0000000002981308 000000001c56fe10:0000000002981308 0000000026b59100:0000000002981308 000000002684a790:0000000002981308 <<

000007fef8b38050  400009a       28        System.Char[]  0   shared           static WhitespaceChars

• Now we need to know what is being passed to method in question which end up in high CPU utilization. From output of !clrstack -p you we can see address for that object 

0:014> !do 0x0000000002c75a90

Name: System.String

MethodTable: 000007fef8b36758

EEClass: 000007fef873e520

Size: 48(0x30) bytes

 (C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)

String: 4236762-COM

Fields:

              MT    Field   Offset                 Type VT     Attr            Value Name

000007fef8b3d9c8  4000096        8         System.Int32  1 instance               12 m_arrayLength

000007fef8b3d9c8  4000097        c         System.Int32  1 instance               11 m_stringLength

000007fef8b381a0  4000098       10          System.Char  1 instance               34 m_firstChar

000007fef8b36758  4000099       20        System.String  0   shared           static Empty

                                 >> Domain:Value  00000000023f1fb0:0000000002981308 000000000246c1c0:0000000002981308 00000000024cea50:0000000002981308 00000000024e8b60:0000000002981308 000000001c578a20:0000000002981308 000000001c58f590:0000000002981308 0000000026905040:0000000002981308 0000000026941740:0000000002981308 000000001c56fe10:0000000002981308 0000000026b59100:0000000002981308 000000002684a790:0000000002981308 <<

000007fef8b38050  400009a       28        System.Char[]  0   shared           static WhitespaceChars

So when this 4236762-COM being passed to Company.SiebelExtract.Services.SiebelExtractLookup.lookupAssetAndEntitlementHeirarchyByGrantNumber it's causing high CPU utiliation.

This is sufficient information for developers to debug the code on their own at this point. 

Using URLRewrite to curcumvent limitation of anonymous WebDAV in IIS7

IIS7 removed possibility to use anonymous WebDAV and one of the solution proposed to curcumvent this (Here) was using custom HTTP Module which would inject Basic Authorization header into the stream.

This works but pretty difficult to implement. First you have to compile assembly, secondly you have to hardcode username and password into DLL without possibility of changing it once deployed. There are some drawbacks to that solution as well since that module is being executed on every incoming request regardless wether it's WebDAV or not.
Instead of that you can just simple use URLRewrite module to do exactly the thing mentioned with no additional complexities mentioned. You need to create rewrite configuration below. You encode desired username password using base64 encoding (for example here Here) in format "username:password" and append that value after "Basic" value below.

Managed Stack Explorer compatible with x64 and IIS 7 environment

Hi,

This is just recompiled version of Managed Stack Explorer which will run in x64 environment and IIS7.
File is here for download
Also if you have .NET 4.0 installed you can use following tool instead of depreciated MSE CLR stack Explorer

Virtual CPUs and performance of virtual machine

Performance comparison of multi threaded application in relevance to multiple virtual CPUs in virtual machine

I was tasked (actually wanted to know myself) if common understanding going around that adding virtual CPUs to virtual machine does not affect performance of such. Idea behind this thought is that since CPU is virtual then hypervisor will be spreading load across physical CPUs anyway and hence additional CPUs will not increase performance of your application.
I wanted specifically to test multi threaded application since this is what majority of web servers are.
So to test raw CPU performance I acquired 4 CPU IBM box. I wanted to make a test against VM guest running inside ESX 3.5 with different number of virtual CPUs and 1 another test - against physical hardware (no hypervisor).
For test itself I wanted something which is very CPU intensive and has no IO dependencies. I figured calculating PI to certain digit will be a good choice in this case.
Entire Visual Studio 2008 solution can be downloaded below. It consists of web service which accepts only 1 parameter which is "Number of Digits of PI to be calculated to" and returns back total time taken to calculate that number.
Solution also contains sample webpage which will call web service, that page was called by 10 virtual clients simultaneously. Total number of pages served were calculated and presented in table and graph below.

Conclusion: Adding more virtual CPUs for multithreaded application does it in fact increase performance of application.

Number of logical CPUs	Number of requests served
1 CPU	24
2 CPU	47
4 CPU	65
Physical box with 4 CPUs	70

Attachment:Download

How to use .NET profiler to troubleshoot ASP.NET page perfomance issues

How to use .NET profiler to troubleshoot slow ASP.NET page performance

 

I'm frequently tasked to figure out why certain pages or parts of websites are slow (assuming site is ASP.NET based). In this specific article I'll show how to use free .NET profiler from Equatec http://www.eqatec.com/tools/profiler to troubleshoot this issues.

 

My common troubleshooting steps (before diving into ASP.NET website performance troubleshooting are below).

 

1. 
   
   Use Fiddler tool (http://www.fiddler2.com/fiddler2/) to actually measure response time of web page. This works with AJAX based pages as well and will actually provide you information which you usually would not be able to see in browser window (simple GET/POST requests are so 90s in modern technologies).
   
   
2. 
   
   Once you get statistics and confirmed that you do in fact have backend problem other then networking issues I usually see if there are any known dependencies for the page (like SQL backend). This is pretty easy to figure out by looking through web.config file. I start SQL profiler on SQL server afterwards and resubmit request from browser again watching profiler statements coming through. Sometimes troubleshooting ends here (if you see that Execution time of stored procedure or ad-hoc query corresponds to total page response time.
   
   
3. 
   
   
   

   At this point if you ruled out backend slowness you are probably looking at some issues within ASP.NET website itself. I usually enable ASP.NET tracing at this point by going to web.config and adding following statement there
   

    

   
   

   <trace enabled="true" localOnly="false" pageOutput="true"/>). This will output tracing information (most importanly times to takes to execute specific methods in pipeline). Sometimes it's enough to to see which part of the page pipeline fails.
   

   

   
   

    
   
   
4. 
   
   
   

   Please note that page execution times listed trace output are not including times which are spends outside of the page (like authenticating request or processing events in global.asax). So you might see fast execution times in trace output but page will still be slow to render and hence we come to real purpose of this article, i.e. to use .NET profiler to see what is causing page to stall.

 

 

I wrote a sample website where I simulated different stalling conditions. Stalling in pre-page execution pipeline (in global.asax) and stalling inside page itself (SQL call or Web Services call or something of that nature). To simulate long running method I inserted following statement in relevant methods (System.Threading.Thread.Sleep(5000);). This makes current thread sleep for specified amount of time. I put Tracing statements through the code so you can actually see in screenshot below where latency are being introduced. Lines in red indicate Trace statements in the code. (First one in global.asax Application_BeginRequest event and second one inside Page_Load() statement of default.aspx page). In real life scenario you most likely would not have any Tracing statements in code you'll be troubleshooting though.

 

 

I compiled my test website and at this point all I have is what you probably would have in your case. You can download it here (ZIP). Once you installed it, navigate to default.aspx page and see how much it takes to load a page (shall be around 7s or so).

 

Next step is to install Equatec .NET profiler from here (Here).

 

• 
  
  On first run it'll display default screen below

 

• 
  
  Click browse button and navigate to installation directory of your website (BIN folder) and open precompiled binary for your application (profiler.dll in this case)

 

• 
  
  Click "Build" button on bottom. Profiler will inject profiling information into your assembly and output results to the "Bin-Profiled" folder under the same root where your original "Bin" folder is. You'll see that you original DLL file grew up in size because of that.


• 
  
  Copy all files from "Bin-Profiled" folder to "Bin" folder (replacing files)


• 
  
  Switch to "Run" tab and navigate to your problem page. You'll notice that Control area will be updated with information "Profiled app Profiler started"

• 
  
  Click on "Take snapshot" button. It'll generate report about execution time and put report under "View Snapshot reports" area.

• 
  
  Click "View" button now with latest report selected. Result below is exactly what you'd expect to see from test file. Which is first 5 seconds were spent in BeginRequest event and next 2 in Page_Load calling "DoSQLQuery" method.

 

• 
  
  Now since you know which specifics methods take that much time you need to take a look at what is being done inside those methods. In this case (and probably yours) you'll just have assembly (profile.dll). You need to use tool like .NET Reflector (Download here) to see contents in readable form.

 

Now we can see exactly why it took so long to execute this specific method. In your case it might be SQL call or web services or call or something of that nature.

Instructions how to enable SSTP VPN s...

Instructions how to enable SSTP VPN server in Windows 2008 simplified

In this brief 10 minutes procedure below you'll learn how to create VPN connection using SSTP (SSL tunnel) to your work/home location from anywhere where HTTPS is allowed.

Guide below assumes you don't want to use your VPN server as Internet Router and you have some other devices doing routing. Steps below will allow you to VPN in your remote location and route everything (including Internet traffic) through that VPN connection. 

Difference from other guides on Internet:

• Don't need domain controller or certificate services for this to work
  
• Don't need 2 network adapters on your server to work
  
• Don't need to publish SSL CRL (certificate revocation list)

What is needed:

• Server SSL certificate (self-signed)
  
• Windows 2008 on your local network (single NIC)
  
• Vista SP1 or later

Steps:  

1. Create self-signed certificate and import it into certificate store on client and server
   
   
   
   • Create self-signed certificate with domain name which you'll be using for your VPN server. I used this excellent utility (http://www.pluralsight.com/community/blogs/keith/archive/2009/01/22/create-self-signed-x-509-certificates-in-a-flash-with-self-cert.aspx). What you want to do is to put CN name as external name which you'll be using for your VPN server (like sstp.mydomain.com) and choose to export as PFX file. This will create PFX file with your certificate and designated location. In steps below it would be imported into client and server as Trusted CA.
     
     
     
     • Import certificate on client computer by following this steps: Launch MMC and choose Add Snap-in, choose "Certificates", on next screen choose "Local Computer" and then choose "Trusted Root Certification Authorities" and import your certificate into that store.
       
     • Import certificate on server computer exactly the same ways it was done on client.
   
   
2. Enabled RRAS role on Windows 2008 and configure SSTP.
   
   
   
   • Install RRAS service on your server
     
     
     
     • Add Server Role called "Network Policy and Access Services". Choose only "Remote Access Service" for service since we don't need routing. Go through installation process.
       
     • Once server is setup, launch Routing and Remote Access MMC and choose to "Configure Routing and Remote Access server".
       
     • Choose "Custom Configuration" and choose "VPN access".
       
     • For your user go to user property pages. Choose "Dial-in" tab and choose "Allow access" under "Network Access Permission".
     
     
   • Configure your external router to port forward TCP 443 to your internal IP of your VPN server.
   
   
3. Configure your Vista VPN client.
   
   
   
   • Go to Network And Sharing Center/Setup Connection or network/Connect to a workplace. Choose "Use Internet Connection", put your hostname for connection and go through the rest of the steps. Use "skip" since connection will probably fail since we need to configure connection to use SSTP. Once connection is created, go to properties of connection/Networking and choose as type of VPN - SSTP, uncheck IpV6 since you don't need at this point.

 

At this point you shall be able to VPN into your remote home/office.

 

This is test I did in school when I was 10 years old (3d grade). See if your 10-11 old child will be able to make it. (Test for 7 year olds below this post)

Test 1.

Airplane needs to fly between 2 cities, distance between cities is 4150 km. First 2 hours airplane was flying 676 kmh, then 3 hours was flying with speed 760 kmh.

Question: How many km is left to fly?

Test 2.

Swimmer had swam 448 laps in first day. On second day he was swimming 2 hours less then first day and swam total of 336 laps.

Question: How many total hours swimmer was swimming on first day and on second day?

Test 3.

Your yard is 45 m in length and 24 m in width. What is perimeter of your yard?

Below answers from original paperwork filed in 1981

This is test which I did at the end of 1st grade in Moscow School on May 17th, 1978 when I was 7 years old.

See if your 7-8 year old child can answer questions below.

Test 1.

There are 27 airplanes and 16 helicopters at airport. 18 aircrafts took off. Question: How many aircrafts left at airport?

Test 2.

• 64 + 23 = ?
• 14 - 8 = ?
• 48 + 5 = ?
• 53 + 27 = ?
• 9 + 6 = ?
• 90 - 26 = ?
• Put <,>,= between 20-6 and 20-9