SharePoint Is Slow On Isolated Environments

15 בדצמבר 2010

תגיות: ,
2 תגובות

This is an interesting case that was solved thanks to the help of Nitsan Raz – an all mighty system administrator and SharePoint admin.

Problem

So I was asked to investigate a performance problem at one of my regular clients.

To demonstrate the performance problem, the client’s developers showed me a simple WinForm application (on the server), that created an instance of an SPSite object. the code had 1 line of interest, which is:

SPSite site = new SPSite("http://mySite");

That line alone took 30 seconds to execute! What’s more interesting, was that subsequent calls was very fast to execute. It was just the first one that took so long.

Cause

First thing we though was to isolate the location of the hang. Was it on the application side, or DB side? We ran SQL profiler and saw that SQL queries took like a millisecond to execute. That test eliminated the option of DB problem.

Having figured that out, we decided to run some tests on the application server (SharePoint Web Front End). After hours of time measures and watching endless performance counters go up and down the screen, we decided to monitor network activity.

We found 2 interesting lines in the log. These were DNS queries for crl.microsoft.com. How is that has to do with our application?

It turns out, that SharePoint’s assemblies are signed using a Authenticode, and that every time they are loaded into an app domain, the system tries to verify their validity by asking Microsoft for Certificate Revocation List (CRL). This is a list that contains certificates that are no longer valid.

Our environment was disconnected from the internet, so obviously we can’t reach Microsoft’s servers to get the CRL. That process failed on the very beginning , when trying to lookup Microsoft.com on DNS. The DNS didn’t respond on time, so we were waiting for the lookup to time out… guessing what’s the  time out for DNS lookups? 15 seconds. This explains why we waited 30 seconds (for 2 assemblies to load).

Solution

There are several ways we can deal with this: (You should choose only one of the following)

1. Add the following configuration to the configuration file for the application that is loading the files: (This could be MyApp.config, web.config, machine.config, etc… – depending on your application)

<configuration> 
<runtime>
<generatePublisherEvidence enabled="false"/>
</runtime>
</configuration>

This solution requires a hotfix for .NET 2.0, and is already built into later versions. More info:

http://support.microsoft.com/kb/936707

2. Turn off CRL checks (the the user level). Internet Options->Advanced tab->Security section under the Settings area->Check for publisher’s certificate revocation->turn that off (uncheck). 
image

3. Add a fake DNS record that points to yourself (127.0.0.1), that will shorten the time to fail.

4. Pre-fetch CRL files, so that it will be pulled locally and not from the internet.

Download these:
http://crl.microsoft.com/pki/crl/products/CodeSignPCA.crl
http://crl.microsoft.com/pki/crl/products/CodeSignPCA2.crl
Add to certificate store:
certutil -addstore CA CodeSignPCA.crl
certutil -addstore CA CodeSignPCA2.crl

5. Allow internet access, obviously (if only to crl.microsoft.com)

Credits

Dirk Van den Berghe

Nitsan Raz

For more info on assembly signing:

http://blogs.msdn.com/b/junfeng/archive/2006/03/11/549355.aspx

הוסף תגובה
facebook linkedin twitter email

כתיבת תגובה

האימייל לא יוצג באתר. שדות החובה מסומנים *

2 תגובות

  1. Dirk Van den Berghe15 בדצמבר 2010 ב 15:34

    Thanks for the credit, don't forget Jeroen Ritmeijer 🙂

    However I wasn't aware it was also the case when instantiating an SPSite object as I only had the issue with stsadm.

    best regards

    Dirk Van den Berghe

    הגב
  2. Itay Shakury15 בדצמבר 2010 ב 17:05

    you deserve it 🙂

    As I see it, it is the case for everything that loads an assembly. Because we were using a winform app, it loaded the assembly to it's app domain regardless of ASP.net. as a matter of fact, I could have written anything inside that app (not particularly new SPSite) and that would have had the same consequences i guess.

    הגב