The Cathedral, the Bazaar, and Microsoft’s presence on both sides of the Vietnam War of Computer Science

Let's start this post with a couple of warnings / disclaimers:

1. Be advised that this post mixes metaphors all over the place! J
2. Regardless of the fact that I am a Microsoft employee, this post represents my opinions only, and should not be mistaken as representing Microsoft's position on the matter discussed, or the opinion of any other Microsoft employee.

The Cathedral's Bazaar

One of the most interesting and influential papers I ever read about practicing software development is Eric Raymond's "The Cathedral and the Bazaar". Very readable, entertaining, illuminating, and most importantly – extremely influential! I have yet to see a sociological study of the subject, but it is said that this paper was one of the most influential in popularizing the Open Source movement.

The idea of developers offering they're open-sourced and frequently freeware products in an open market (i.e. Bazaar) as opposed to platform vendors creating strictly commercial, shrink-wrapped software (i.e. Cathedral) has indeed gained a lot of ground since 1997 (the year "The Cathedral and the Bazaar" was first published). So much so, that even one of the most well-known examples of a cathedral oriented company – Microsoft – has adopted an Open Source strategy and is actively encouraging an Open Source community.

This is similar to the concept of a bazaar (IMHO) but not quite, since it is, after all, a cathedral-sponsored initiative.

A real bazaar community is currently in the process of coalescing in the form of ALT.NET. These guys have been around for quite a long time, coding vigorously. See NHibernate, RhinoMocks, SharpDevelop and more. The all-encompassing umbrella nickname "ALT.NET" was originally proposed by David Laribee and was adopted as its official name. Since then, quite a few of the best-and-brightest of .NET open source developers joined ALT.NET (or more correctly, since it is a very loosely-coupled group, chose to associate themselves with the group. I don't think they became "fully-indoctrinated card-carrying members of the ALT.NET Party"... J)

However, not everything is milk-and-honey in the land of ALT.NET. There are "some" aspects of "some" self-described members of ALT.NET that may be described as "problematic".

Allow me to quote one of my favorite bloggers – Sam Gentile – on the subject:

"ALT.NET is a divisive thing. No matter what they tell you, they are full of negative energy, they sneer at others that don't buy into their view and sneer at the "enterprisey" folks." (October 6^th, 2007)

And another one, since it is very relevant to the discussion (I'm getting there… please bare with me J)

"For a one sentence statement of my position… "ALT.NET is perceived as being anti-Microsoft because of a few bloggers' overzealous nature in describing their disdain for Microsoft products and all things anti-Agile." (October 7^th 2007)

Aside from that, I must express my admiration for ALT.NET as an idea, and for many of its members' activity as expressed in invaluable software products, intellectual content and in their passion for writing software at the highest level.

The War

Here's a small insiders' secret: During the past couple of TechEd events in Israel, the Architects Panel session (hosted by MCS Israel's Guy Ron) was repeatedly asked about its opinions regarding the Relational vs. ORM debate. Although repetitive and somewhat cliche, this question always aroused the most interest among the audience, and the most heated debate among the panel's participants.

Ted Neward described the topic as "The Vietnam War of Computer Science". He certainly has a flair for the dramatic, but he has a point. This war is especially visible in the .NET world since, as Martin Fowler says in PoEEA, the tools support for the Relational approach in .NET is much more advanced, available and novice-friendly than the ORM tools.

For a long time, the "war" was fought with Microsoft exclusively on the Relational side, and open sourced community efforts on the ORM side (the most visible - and IMHO mature - opponent being NHibernate).

Enter ADO.NET Entity Framework.

It isn't the first attempt by Microsoft to enter the ORM space (did I hear somebody say "ObjectSpaces"???!!!) but it is the strongest effort yet. And it has some very impressive features.

However, It does have its limitations, as does every piece of software written by mere-mortals.

It is also a piece of software that is located in a virtual war-zone, and as such it should (and perhaps ought to) draw fire.

And fire it has drawn. The passion I so admire can be somewhat painful (see Ayende Rahien's apology which was elaborated on in .NetRocks' panel on "The Future of .NET") but I still think it is an invaluable tool on the way for better software.

On another level altogether, is the issue of the "ADO .NET Entity Framework Vote of No Confidence". The concerns raised in the letter are legitimate, and are discussed in length by Tim Mallalieu from the EF development team.

In a nutshell:

The letter's signatories express "concern for the welfare of software projects… that will make use of the forthcoming ADO .NET Entity Framework" due to "technical misgivings with the Entity Framework's current design and implementation". They go on and enumerate a number of architectural and technical issues, but the heart of the matter, IMO, is written between the lines of the starting paragraphs: The fact that "Microsoft and the Entity Framework team have received a tremendous amount of feedback from experts in entity-based applications and software architectures on the .NET platform" did not lead to Microsoft (and the EF Team's) accepting and implementing that feedback. The Machiavellian essence of the letter (as opposed to its technological essence) is a failure to influence the current direction EF is taking.

On the other corner we find Tim Mallalieu, a Program Manager from the EF team. His response points to some basic facts in software development: "The unfortunate reality is that these are scenarios [i.e. the scenarios outlined as problematic in the letter] that we care deeply about but do not fully support in V1.0... the choices on these features were heavily considered, but we had to deal with the tension between trying to add more features vs. trying to stay true to our initial goal which was to lay the core foundation for a multiple-release strategy for building out a broader data platform offering". He goes on to respond in detail to every item raised in the letter, but the point is that they are planning on an evolving product, using prioritized backlog, in which some of the items raised by the letters' signatories did not make it to release 1.0… Every Agile developer should feel at home with this approach.

I would encourage you to read both the "No Confidence" letter and Tim's blog post response before reading the following InfoQ commentary on the subject, especially the "Conclusions" section.

My 2 cents worth on the subject

Having surveyed the battle zone (quite shallowly, since an in-depth survey will consume the length of a career or two…) I would like to share some thoughts:

I would argue that the "No Confidence" letter marks a potential watershed in the relationship between the open source .NET community and Microsoft's development teams.

I am not saying that there will be a strain on future relationships – far from it!

I do believe, however, that this letter marks the community's most vehement (so far…) will and demand for a much more effective involvement in the decision making process (or backlog prioritization process) of select Microsoft development teams.

In hindsight, it should have been predictable that such a controversial subject would cause such an uproar in the community, especially since the topic of ORM best practices has been consuming so much time and intellectual effort from so many of .NET's best and brightest (the Vietnam War indeed…).

How will Microsoft teams react to such criticism? I believe that favorably and with an open mind. In my experience, both as an MS employee and as a Microsoft-oriented-developer before that, Microsoft is very open to criticism, and it encourages critical thinking from both inside and outside the company. There's no way the company could have reached its current market position without this feature built into its DNA, and I do not believe this will change anytime soon (and hopefully - ever).

This openness to criticism and community influence I am counting on and hoping for will also benefit ALT.NET and the open source community. A more influential community will create a partnership between equals in the development of the developer platform, and will create a true bazaar both inside and outside the cathedral.

It will of course create some very interesting (J) management challenges to Microsoft program managers, but hey… that's what they get paid for!!!

התפצלויות

Technical Readiness

When it is recommended to use temporary tables in Stored procedures?

שלום לכולם,

שמי אסף פרנקל, ואני ארכיטקט ויועץ בתחום מסדי נתונים, בזמני הפנוי אני עוסק גם בניהול פרויקטים.

בפוסט קצר זה אעסוק בשאלה נפוצה, מתי יש להשתמש בטבלאות זמניות בתוך Stored Procedures.

מצד אחד, כמובן, לכל  יצירה של טבלא זמנית יש מחיר, היא נרשמת ב TempDb, ובסך הכל זו פעולה כבדה. לכן אם יש לנו שאילתת סינון מומלץ בדרך כלל לבצע Join מורכב בין שאילתת הסינון לשאר השאילתא ולא להשתמש בטבלא זמניות.

מנסיוני יש מספר מקרים בהם בכל זאת השימוש מומלץ:

שימוש חוזר - עדיף לשמור בטבלא זמנית את תוצאת שאילתת הסינון ואז לבצע Join 3 פעמים עם הטבלא הזמנית הקטנה, מאשר להוסיף את ה Join לשלושת השאילתות

מספר לא ידוע של רשומות - לעיתים קרובות קשה לאופטימייזר להעריך את מספר הרשומות שיחזרו משאילתת הסינון. פיצול לשניים יאפשר לאופטי מייזר לדעת במדויק את מספר הרשומות, ולפי מספר זה לייצר תכנית מתאימה ל Join של הטבלא הזמנית עם שאר השאילתא

ישנם מקרים נוספים, ואשמח לראות אותם בתגובות שלכם.

בנוסף, אתגר, אם הסיבה שבחרנו הנה שימוש חוזר, מתי כדאי להשתמש דווקא בטבלא זמנית מסוג temporary table ולא מסוג table variable?

קריאה מהנה,

אסף

The limitations of WCF sessions - part 1: Transport-level dependence

I feel that having started a blog post with the word “limitations” in its title I should first express my feelings towards WCF.

I love it. I think it’s the greatest thing since sliced bread.

OK, now that I got that off my chest, I can get to the point.

Lately I have had a couple of conversations with customers and colleagues about session support in WCF (session in the sense of having a way of correlating any number of client requests identified by the server’s service as related to one another, and by doing so allowing the service developer to apply some application specific functionality – such as state management, activity tracking etc.)

WCF supports session-full services through the SessionMode ServiceContractAttribute. I won’t go into the extremely well documented details of its functionality and syntax (instead, I will point you to the “Using Sessions” article in MSDN).

The first thing to note about session in WCF, is that they rely heavily on the underlying transport mechanism.

When using TCP or NamedPipes transports (using NetTcpBinding and NetNamedPipeBinding, respectively) the service and proxy (i.e. client) correlation are managed through the underlying transport-level continuous connection. As a result, closing or aborting the transport-level connection also terminates the proxy application-level session.

As for HTTP based bindings, the transport-level HTTP connection cannot be relied upon to be continuous (even when using HTTP 1.1’s Keep-Alive option) so BasicHttpBinding does not support session at all, while when using WsHttpBinding, it is implemented as a logical session identifier using the message headers as a mechanism.

Why is it a limitation?

For a start, it falls smack in the middle of the number 1 fallacy of distributed computing: “The network is reliable”.

I created a simple WCF service (based on the ‘WCF Service Library’ template in Visual Studio 2008) and added the SessionMode.Required attribute to the service interface:

[ServiceContract(SessionMode=SessionMode.Required)]
public interface IService1

Provided that A few other requirements were met (which I will discuss in future posts), I will be able to access the WCF supplied session identifier using the SessionId property, using the client proxy’s InnerChannel object:

string s = proxy.InnerChannel.SessionId;

Or the service’s current OperationContext object:

string s = OperationContext.Current.SessionId;

I added a connection abortion simulation to the service by adding the following method to the contract:

        void AbortChannel();

(the definition of the method as ‘IsOneWay=true’ is intended to avoid having a connection failure exception being raised immediately by the AbortChannel method itself, as a result of not being able to return the operation’s result message)

I implemented it in the service as follows:

public void AbortChannel()                 {           
//emulate a transport-level
//connection abortion            OperationContext.Current.Channel.Abort();
}

After calling the AbortChannel() operation from the client’s proxy, I try to call another operation on the same proxy. I should not be surprised that I get an exception, but it is (mildly) surprising that I get different exceptions, depending on the binding used.

Let’s start with NetTcpBinding. The Exception I get is of type “System.ServiceModel.CommunicationException” with the following message:

'The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:00:59.9640000'.'

When using NetNamedPipeBinding I get the same exception type with the message:

'There was an error reading from the pipe: Unrecognized error 109 (0x6d).'

This isn’t so bad. Although these are different messages, it’s the same exception type, and the messages are indicative (IMHO).

However, when using WsHttpBinding, the seams under WCF’s transport-level abstraction layer start to crack. The exception is of type “System.ServiceModel.Security.MessageSecurityException” and the message is:

'An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail.'

‘The message could not be processed. This is most likely because the action 'http://tempuri.org/IService1/SetSessionValue' is incorrect or because the message contains an invalid or expired security context token or because there is a mismatch between bindings. The security context token would be invalid if the service aborted the channel due to inactivity. To prevent the service from aborting idle sessions prematurely increase the Receive timeout on the service endpoint's binding.’

You can download the sample code as a Visual Studio 2008 solution from the following location: http://blogs.microsoft.co.il/blogs/mcs/WcfSessionSample.zip

Have fun,

Danny. 

איך להחזיר את החיוך לפניהם של `עובדי המידע’?

Loading Unity Configuration From Alternative Source

In my previous post , I’ve showed you how to use unity schema in web.config for mapping interface types against concrete types.

If you have multiples mappings or you just don’t want to store unity configuration in your web.config , than you can load unity configuration information into container from a specific named configuration file by using System.Configuration.Configuration

protected override void InitializeRuntime(){
  ExeConfigurationFileMap map = new ExeConfigurationFileMap();
  map.ExeConfigFilename = @"c:\OtherPlace\UnityDemoConfiguration.config";
  Configuration config = ConfigurationManager.OpenMappedExeConfiguration
                         (map, ConfigurationUserLevel.None);
  IUnityContainer container = new UnityContainer();
  UnityConfigurationSection section = 
 (UnityConfigurationSection)config.GetSection("unity");
  section.Containers.Default.Configure(container);
  Extensions.Add(new UnityExtension(container));
  base.InitializeRuntime();
}

The map.ExeConfigFilename property point to the alternative source file.

Generating Time Dimension with SSIS

Hi Everyone,

Time dimension is the most popular and common in data warehouse systems.

We always create scripts to generate time intelligence in our data warehouse and analysis services.

The SSIS package has two parameters (Variables)  , "StartDate" , "EndDate".

There is a SQL script in the project to create the destination table (in TempDB).

Script component is the source of data flow and its generating a standard time dimension.

It also prepares "TimeKey" with the most popular expression, For Example:

Dim oStartDate As Date = CDate(Me.Variables.StartDate.ToString())
        Dim oEndDate As Date = CDate(Me.Variables.EndDate.ToString())

        Do While oStartDate <= oEndDate
            With Me.MyAddressOutputBufferBufferBuffer
                .AddRow()
                .TimeKey = (oStartDate.Year * 10000) + (oStartDate.Month * 100) + oStartDate.Day
                .Day = oStartDate.Day.ToString()
                .DayName = DateAndTime.DatePart(DateInterval.Day, oStartDate, FirstDayOfWeek.Sunday).ToString()
                .Month = oStartDate.Month.ToString()
                .MonthName = MonthName(oStartDate.Month)
                .Quarter = DateAndTime.DatePart(DateInterval.Quarter, oStartDate, FirstDayOfWeek.Sunday).ToString()
                .QuarterName = "Q" & DateAndTime.DatePart(DateInterval.Quarter, oStartDate, FirstDayOfWeek.Sunday).ToString()
                .QuarterFullName = "Q" & DateAndTime.DatePart(DateInterval.Quarter, oStartDate, FirstDayOfWeek.Sunday).ToString() & _
                "/" & oStartDate.Year.ToString()
                .Year = oStartDate.Year.ToString()
            End With
            oStartDate = DateAndTime.DateAdd(DateInterval.Day, 1, oStartDate)
        Loop

The Package

 

The Result

Bye and Enjoy

.NET and the basics of Math

Hi,

My name is Memi Lavi, and I'm too a member of the MCS group in Microsoft Israel.

I would like to share with you something which looks weird on first glance:

Look at the following code snippet (in C#):

Integrating Unity Application Block with WCF - Part 1

The Unity Application Block is a lightweight, extensible dependency injection container with support for constructor, property and method call injection. In Layered Architecture, Unity can help us to achieve decoupled or very loosely coupled design between the different tiers (more details about the layered architecture with Unity in the next posts.. ).

There are three ways we can set up and prepare a Unity container by populating it with registered mappings. We can:

• Provide an XML format configuration file that adheres to Unity schema.
• Use the container API to add specific registration entries.
• Use the container configuration API to provide custom configuration for the container.

The steps below demonstrate one way of setting up Unity container within IIS using the web.config technique [most credits go to Jorg Joose)  . The steps are:

1. Create WCF Extension class to hold the Unity container.
2. Create WCF ServiceHost class and override the InitializeRuntime method to set up the Unity container.
3. Create WCF ServiceHostFactory class to initialize the custom WCF ServiceHost.
4. Update the .svc file to load the custom ServiceHostFactory class.
5. Update WCF web.config to contains the Unity mappings.

Create WCF Extension Class

class UnityExtension : IExtension<ServiceHostBase>, IDisposable
{
    private static readonly object containerLock = new object();
    private static IUnityContainer containerSingleton;

    internal UnityExtension(IUnityContainer container)
    {
        lock (containerLock)
        {
            if (containerSingleton == null)
            {
                containerSingleton = container;
            }
        }
    }

    internal IUnityContainer Container
    {
        get { return containerSingleton; }
    }

    #region IExtension<ServiceHostBase> Members

    public void Attach(ServiceHostBase owner)
    {
        // Not used.
    }

    public void Detach(ServiceHostBase owner)
    {
        // Not used.
    }

    #endregion

    #region IDisposable Members

    public void Dispose()
    {
        lock (containerLock)
        {
            if (containerSingleton == null)
            {
                containerSingleton.Dispose();
                containerSingleton = null;
            }
        }
    }

Create WCF ServiceHost Class

public class UnityServiceHost : ServiceHost
{
    public UnityServiceHost(Type serviceType, Uri[] baseAddresses)
        : base(serviceType, baseAddresses)
    {
    }

    public UnityServiceHost(object singletonInstance, Uri[] baseAddresses)
        : base(singletonInstance, baseAddresses)
    {
    }

    protected override void InitializeRuntime(){
     VirtualPathExtension extension = Extensions.Find<VirtualPathExtension>();
     Configuration config = WebConfigurationManager.OpenWebConfiguration
     (extension.VirtualPath);
     IUnityContainer container = new UnityContainer();
     UnityConfigurationSection section = (UnityConfigurationSection)config.GetSection("unity");
     section.Containers.Default.Configure(container);
     Extensions.Add(new UnityExtension(container));
     base.InitializeRuntime();
    }
}

Create WCF ServiceHostFactory Class

public class UnityServiceHostFactory : ServiceHostFactory{
 protected override ServiceHost CreateServiceHost
(Type serviceType, Uri[] baseAddresses){
    return new UnityServiceHost(serviceType, baseAddresses);
 }
}

Update svc file

The svc file should declare the custom ServiceHostFactory class

<%@ ServiceHost Language="C#" Debug="true" Factory="UnityServiceHostFactory"%>

Update WCF web.config

For example (mapping between ICustomerOrchestration interface type to CustomerOrchestrationImplementation concrete type)

 <unity>
   <typeAliases>
     <!-- Lifetime manager types -->
     <typeAlias alias="singleton" 
     type="Microsoft.Practices.Unity.ContainerControlledLifetimeManager, 
     Microsoft.Practices.Unity" />
     <typeAlias alias="external" 
     type="Microsoft.Practices.Unity.ExternallyControlledLifetimeManager, 
     Microsoft.Practices.Unity" />
     <!-- User-defined type aliases -->
    <typeAlias alias="ICustomersOrchestration" 
               type="CustomerOrchestrationInterface" />
   </typeAliases>
   <containers>
     <container>
       <types>
           <type type="ICustomerOrchestration" 
                 mapTo="CustomersOrchestrationImplementation">
           </type>
      </types>
     </container>
   </containers>
 </unity>

Reporting Services Querying with WQL

Welcome to Israel MCS blog , this is the first technical post and I belive you're going to see them a lot :)

 Today I want to share a solution I had last week.

From time to time we need to "ask" our Operation system questions such as:

1.       How are you today?

2.       How was your night?

3.       Do you need anything?

4.       Did somebody do something to you?  Who?

Well, the best way to do it is querying the system with WMI queries by WQL language.

Few days ago my customer needed to decide what the best way is.

So, how we can do it? , the following solutions are some examples of ways to do it:

1.       Working with the magnificent task in SSIS "WMI Data Reader Task", you can run it against the system and load it to a DB destination and querying it with a Report tool.

2.       Design a new .NET data extension for Reporting services and querying the system.

3.       Using a CLR integration in SQL Server 2005/2008 and develop a stored procedure to integrate with WMI. You have to load the system.managment.dll to SQL server.

4.       "Use it at home..." J , using SSIS as data source in Reporting Services.

And the list can grow....

My customer needed for a generic source without any Report tool dependency.

We decide to develop a .NET stored procedure with CLR integration which gives us a generic WMI source for any report tool that can querying SQL server.

The solution has two projects: (Download the WMIConnector Project)

1.       .NET stored procedure with a dedicated class with WMI support.(Scripts under "Test Scripts" Directory).

2.       Reporting Services Project with sample WMI report.

The report has three parameters:

1.       Server name

2.       WMI namespace

3.       WMI query.

Installation:

1.       Create a SQL Server database with a name: "WMIProxyDB"

2.       Load the system.managment namespace to SQL server - Run the Step I.sql script in the Test Scripts directory.

3.       Open and deploy the .Net stored procedure project. (WMIConnector.sln).

4.       Run the Sample script against it. (Step II.sql).

Open the sample report project and test it.

Enjoy , bye