AppliSec

Where is WF in the .Net Services July CTP

One of the most frequent feedbacks the .Net services got about WF in the .Net services was why do you use WF 3.5 and not WF 4.0?

So they won't.

The .Net services will use WF 4.0 but they cannot do that until WF ships (22 March 2010) so until then WF was simply removed. Here is the formal announcement.

Let us wait.

There is no doubt that this was the right thing to do.

WCF Advantages

It is time for the big customers to move to WCF.

WCF is 5 years in the market. We are moving into the third version (WCF 4.0)  and there is no doubt that WCF is mature enough for the big customers (who hate to take risks).

The managers ask me: Why WCF ? So I wrote this short summary for executives to read:

WCF is a unified framework for rapidly building service oriented applications.WCF is the most powerful of all distributed applications frameworks, yet it is the simplest to work with. WCF introduce the following advantages:

·         Simple programming model

o   Low learning curve

o   Short development time.

o   Similar object model for all communication technologies.

o   Simple debugging and maintenance

·         WCF support WS* standards

o   With WCF it is possible to enjoy the capabilities introduced by WS* Standards like:

§  Security

§  Federation

§  Transactions (simple and distributed )

§  Reliable messaging

§  Routing (WS Addressing)

§  Policy (WS Policy)

§  Metadata exchange

§  Service Discovery

o   Because WCF implements known standards it interoperable with other frameworks (for example all J2EE frameworks)

·         Performance

o   WCF has the best performance in comparison to other services connectivity frameworks.

·         Extensibility

o   WCF has numerous extension points where you change its behavior or make it implement your protocols and constraints.

o   WCF can enable communication between SOA applications in any communication protocols including proprietary ones.

·         Communication patterns

o   Support for many communication patterns like:

§  Request response

§  One Way (Datagram or asynchronous calls)

§  Duplex

§  Pub-sub

§  Streaming

·         Support for advanced topologies

o   Content based routing

o   Broker

o   Service façade

o   Custom

·         Support for REST

o   WCF is a full RESTful framework

o   It is possible to create syndication Feeds in several formats.

o   Intimate HTTP and URI support.

·         Known and leading technology

o   WCF Microsoft's leading service oriented technology thus all SOA integration and maintenance tools today support WCF.

o   Microsoft SOA technologies are based on WCF.

·         Maintenance and governance

o   Powerful configuration.

o   Built in tracing including ETW support.

o   Built in performance counters

o   WAS on IIS7.0 and Dublin (released in the near future) are hosts for WCF services with built in advanced governance capabilities.

·         Tools

o   Full Visual studio support

o   Tracing and configuration tools.

o   Many community and commercials tools

Hope this will help.

Critical Changes from WF 4.0 Beta 1 to Beta 2

There are very important changes between the two beta versions as was just published by the WF and WCF team blog.

I would like to summarize the most critical updates. (There more changes I will describe in future posts)

1. The activity class hierarchy changed dramatically.

From :

To:

• WorkflowElement goes away, and is replaced with Activity.  Activity is the root type for all units of execution within the world of WF
• Addition of ActivityWithResult which is the base for the Expression activities
• Addition of AsyncCodeActivity.  We got a lot of feedback that people liked what we were doing integrating asynchronous programming within activities but that there was still a fair amount of work in order to hook those up.  AsyncCodeActivity is some nice sugar that makes it pretty easy to write an activity that takes advantage APIs that surface a Begin/End pair using the asynchronous programming model.

2. WorkflowInstance was renamed to WorkflowApplication

3. Correlation

• XPath can now be generated from ParametersContent
• CorrelationQuery and AdditionalCorrelations have been merged into a collection called CorrelationInitializers.   Also, we’ve reduced the need for CorrelationHandles all over the place by improving the CorrelationScope and having an implicit correlation handle.

4. Parameters support : The Parameters activities were removed and merged into the Send and Receive activities.  You can now use the Content property to support MessageContent (primarily for untyped Message or MessageContract ) and ParametersContent (which is for the more RPC style list of name value pairs).

Enjoy.

Live services resource model

Live services as you all knows allows you to access share and synchronize your data.

Once you add your devices to the mesh you can synchronize your data and application between your devices. share your data with your friends and family and access the data from anywhere using the live desktop.

This is remarkable. To enable that there must be a shared data model between all your devices and the cloud.

In this post I want to describe the Live services resource model.

The following famous diagram describes the model:

Let us explain.

The Live Framework Resource Model describes various resources and their relationships to other resources. It is based on a simple Property, Entity, Collection based information model. Each resource is an entity which contains various properties and a set of entities make up a collection. Every collection and entity is uniquely addressable and is identified by a URL. The relationships between entries are represented through hyperlinks. This allows for navigation from one resource to the related resource (entity) or collection of resources (collection). Note that such an entity/collection based model can be transformed trivially to an item/feed based information model of RSS or ATOM.

At the heart of the resource model is Data Entry. Data Entry is the representation of a single unit of data such as photo, a document, a blog post etc. Each data entry contains textual information which describes the data and the link to the binary information - the media (video, photo, etc).

Media related to the data entry is linked via the “EditMediaLink” property and is stored under Media Resources.

A Data Feed is a collection of data entries. In most cases a data feed will be a logical collection of similar / related data entries, however, the content and type of data entries present as part of a data feed is whatever you want it to be.

A collection of data feeds can be represented by a container which we call as a Mesh Object. Usually data feeds grouped under a Mesh Object will be logically related but there is no such binding.
Mesh objects are the basic unit of extensibility, data synchronization and sharing in the Live Framework Resource Model.

All of a user’s data is represented by a collection of such mesh objects and they can be referenced via the user’s Mesh.

Each mesh object not only exposes the collection of data feeds which contains user data (MeshObjects) but also some other important collections :

1.Mapping:
Every mesh object can be synchronized to multiple devices in a user’s mesh. Information on devices on which a given mesh object is synchronized is available via the mappings feed where each entry provides information on a single device where this mesh object is present.

2.Members:
Every mesh object can be shared with multiple people in various capacities.
Every entry in the members feed provide information on every user who has access to the given mesh object
and the type of access they have.

3.News:
Whenever a data entry is added, updated or deleted under a given mesh object, the system automatically generates news events.
These are also generated whenever there is an addition / deletion to the members feed or mappings feed for the mesh object.
Each such news event forms an entry in the news feed for the given mesh object.

4.Subscriptions:
Users can subscribe to changes to mesh objects such that they can be notified and can potentially take appropriate actions.

Other important resources exposed via Mesh in addition to Mesh Objects are

1.Devices:
Information on all the devices on which all or some of the mesh objects are synchronized is available via the devices feed where each entry in the feed provides information on a single device.

2.Applications:
Collective information on all the Mesh enabled applications deployed on the mesh for that particular user.

3.News:
Collective information on all the news entries for all the mesh objects and global news related to the user is available via the news feed where every entry represents a single news event.

4.Notification Queues:
A collection of queues that receives notifications about changes to a certain resources. When registering for notification to a particular resource a notification queue is created. It is possible to create a queue for notifications about multiple resources.

Reading ETW tracing using Event Viewer

I was looking for a tool to read ETW tracing data. It turned out that the tool I need is right under my nose. Event Viewer.

The problem was that ETW (using the logman tool) produces etl files that are not readable by Event Viewer. Fortunately there is a trick. I tried to load the etl file by Event Viewer using open saved log and it failed to load.

Event Viewer

I saved the etl file as an evtx file and now I could see the tracing data.

For WCF and WF you do not need the etl file. All you have to do is follow the following procedure:

Activate a WCF service with tracing enabled and all the tracing data will be there.

ListenUriMode.Unique

It basically solves the challenge of generating a "unique" address for an endpoint when you don't know what's already in use. Here's how it works:

Using this configuration with TCP endpoints causes WCF to automatically choose a "free" port number -- it also appends a unique GUID to the end of the address to ensure uniqueness. For HTTP and named pipes it only appends the GUID to the address, it doesn't generate a unique port number.

When I run this configuration, the listener address for this endpoint is:

net.tcp://localhost:39619/math/908ae8dc-7427-407a-9ea4-8aeecdfa62be

When the value of the ListenUriMode is set to Unique the transport is responsible for creating a unique URI. Different transports used by Windows Communication Foundation (WCF) generate this unique URI differently:

• For TCP in exclusive mode (PortSharingEnabled is false) this means binding to a uniquely available port number.
• For TCP in port sharing mode (PortSharingEnabled is true) and for all of the other existing WCF transports, this means appending a unique path (a GUID) to the end of the ListenUri.

When using ListenUri.Unique the discovery capabilities of WCF 4.0 are a perfect match.

On the server side the ListenURI will be set automatically to be unique and the client will find about the chosen ListenURI using WS discovery.

After the client had learned about the unique URI from the discovery information
(var viaUri = discoveredEndpoint.ListenUris[0]) all it has to do is to point the proxy to the right spot using : clientProxy.Endpoint.Behaviors.Add(new ClientViaBehavior(viaUri)

Simple and clean.

Data Contract topology in WCF 4.0 - DataContractResolver

In WCF 3.x, type resolution was done using the “known types” mechanism. During deserialization, when the serializer encounters an instance that isn’t of the same type as the declared type, it inspects the list of declared “known types” to figure out what type to use.

As the author of the service, you could annotate your types/methods with the [KnownType] or [ServiceKnownType] attributes to define the list of possible substitutions. (in code or config)

Unfortunately, WCF 3.x doesn’t provide an easy way to override the type-mapping algorithm used by DataContractSerializer when performing this type of dynamic type resolution at runtime. This means that it was not possible to implement true polymorphism. It was not easy to make sure the serializer will serialize the correct type in the inheritance tree.

The result was flat data contracts.

Personally I think that flat data contracts are good. It makes services simpler and interoperable but I am not happy that a polymorphic data contract is not possible or very difficult to implement.

I think that the service should be just a facade to the business logic. The actual business entities should not exposed. The data contracts are just flat containers from which the business entities are created.

There are scenarios where the actual business entities should be exposed. In those scenarios we would like to serialize a polymorphic hierarchy of data contracts.

In WCF it is possible and easy using the DataContractResolver.

WCF in .NET 4 provides the abstract DataContractResolver class that you can derive from to implement your own custom type resolution algorithm. Your resolver implementation is provided to the DataContractSerializer instance :

var serializer = new DataContractSerializer(typeof(Object), null, int.MaxValue, false, true, null, new MyDataContractResolver());

The new serializer can be bound to the WCF infra using a custom behavior and you are done.

The new WCF_WF SDK samples has a simple example how to implement such resolver.

Event Tracing for Windows (ETW)

In WF and WCF 4.0 there are new logging (tracking) mechanisms that uses the ETW.

What is ETW?

Event Tracing for Windows (ETW) is a general-purpose, high-speed tracing facility provided by the operating system. Using a buffering and logging mechanism implemented in the kernel, ETW provides a tracing mechanism for events raised by both user-mode applications and kernel-mode device drivers. Additionally, ETW gives you the ability to enable and disable logging dynamically, making it easy to perform detailed tracing in production environments without requiring reboots or application restarts. The logging mechanism uses per-processor buffers that are written to disk by an asynchronous writer thread. This allows large-scale server applications to write events with minimum disturbance.

ETW was first introduced on Windows 2000. Since then, various core OS and server components have adopted ETW to instrument their activities, and it's now one of the key instrumentation technologies on Windows platforms. A growing number of third-party applications are using ETW for instrumentation as well, and some take advantage of the events provided by Windows itself. ETW has also been abstracted into the Windows preprocessor (WPP) software tracing technology, which provides a set of easy-to-use macros for tracing "printf" style messages for debugging during development.

On Windows Vista, ETW has gone through a major upgrade, and one of the most significant changes is the introduction of the unified event provider model and APIs. In short, the new unified APIs combine logging traces and writing to the Event Viewer into one consistent, easy-to-use mechanism for event providers. At the same time, several new features have been added to improve developer and end user experience.

ETW architecture can be described like so:

There are four main types of components in ETW: event providers, controllers, consumers, and event trace sessions. Buffering and logging take place in event tracing sessions, which accept events and create a trace file. There are a number of logging modes available for ETW sessions. For instance, a session can be configured to deliver events directly to consumer applications or to overwrite old events in a file by wrapping around when a certain size is reached. A separate writer thread created for each session flushes them to a file or to real-time consumer applications. To enable high-performance, per-processor buffers are used to eliminate the need for a lock in the logging path.

An event provider is a logical entity that writes events to ETW sessions. Any recordable activity of significance can be an event, and each is represented by an event logged to ETW. An event provider can be a user-mode application, a managed application, a driver, or any other software entity. The only requirement is that the event provider must register a provider ID with ETW through the registration API. A provider first registers with ETW and writes events from various points in the code by invoking the ETW logging API. When a provider is enabled dynamically by the ETW controller application, calls to the logging API send events to a specific trace session designated by the controller. Each event sent by the event provider to the trace session consists of a fixed header that includes event metadata and additional variable user-context data. Due to the growing event instrumentation in many OS components, even a simple application for Windows Vista will already contain several components that are event providers.

When an event is logged to a session, ETW adds a few extra data items along with the user-provided data. They include timestamp, process and thread ID, processor number, and CPU usage data of the logging thread. These data items are recorded in the ETW event header and passed on to event consumers along with the variable event content given by the provider. Many trace consumers find these data fields to be essential in their analysis.

A controller starts and stops ETW sessions and enables providers to them. In some scenarios, such as debugging and diagnosis, a controller tool is invoked as needed to collect in-depth traces.A controller must have ETW permission on Windows Vista to control sessions, which is given only to a small group of privileged users by default.

a consumer is an application that reads log files or listens to a session for real time events and processes them. Event consumption is callback-based; a consumer registers an event callback, which ETW calls with one event at a time. Events are delivered to the ETW consumer in chronological order. There are general-purpose event consumer tools that dump the events into various formats.

On my next post I will show how to use ETW in WF and WCF 4.0

Reporting and SOA

SOA is an architectural style that promises agility, scalability and reuse. The bottom line is SOA can save you money. There are many definitions to SOA. The best definition I found states that a service is something that exposes functionality by accepting data and maybe returning data. Service is simple. Data in - Data out.

To achieve all the SOA advantages a service must be autonomous. 

The problem starts with reporting and ETL.

The traditional two tier or three tier architecture had the reporting engine connected strait to the database. Simple and efficient but the service is not autonomous. There is a strong dependency between the service and the reporting engine. This dependency must be broken.

What is the SOA recommended architecture concerning reporting?

The answer is replication. The services data will be replicated to a central storage where the reporting and ETL will be executed. To do that there are few technologies that can be taken into consideration.

1. Synchronization framework

2. Expose the data as a feed using WCF syndication

3. Restful services (REST in WCF) or ADO.Net Data Services 

replication is not easy and might introduce complicated challenges but this is the only way reporting can be done in the SOA world.

PersistenceParticipant

I am looking at WF 4.0 lately and I found out that there is no clear documentation on PersistenceParticipants.

Well The idea is simple:
If you want to add some state to your workflow persistence data or in other words save some more data with your workflow, you can use a PersistenceParticipant.

The PersistenceParticipant is a class that holds your data but also implements IPersistenceParticipant interface. This interface makes it possible to load and unload the data.

PersistenceParticipant is a great extensibility point with which you can hook extra work to the persistence events.

The PersistenceParticipant has to be bound to the workflow instance using:
void IActivityExtensionProvider.EnsureExtensions(IExtensionHost host)
{
        if (host.GetExtension<StepCountExtension>() == null)
        {
            host.Extensions.Add(new YourPersistenceParticipant ());
        }
}

To get hold of the participant from inside or outside the workflow use: the GetExtension<YourPersistenceParticipant>() method of the workflow instance or the CodeActivityContext.

There is a great example in the wf_wcf 4.0 samples

Velocity crashes on windows 7

I installed velocity – The new distributed cache infrastructure by Microsoft on win XP machine and had no problems. When I tried to do the same on windows 7 (as an administrator) the installation failed to configure the firewall and the power shell tool crashed. This prevents the activation of the distributed cache service.

The new velocity CTP 4.0 is about to be released very soon. I hope it will support windows 7 better than today.