May 2008 - Posts
A few weeks ago I posted a preliminary version of a WCF error handling behavior which facilitated automatic translation of server-side exceptions to faults.
Since then, I have reconsidered parts of my approach. It occurred to me that the most proper way of mapping exceptions to faults is using attributes - the same way that is used to declare fault contracts in the first place.
Therefore, I reworked the solution to use an attribute on the operation level - a [MapExceptionToFault] attribute. The attribute can be used to map an exception type to a fault detail type on a specific operation. This is a significantly more intuitive and useful approach than having a catch-all event handler that would be responsible for detecting the current operation and translating the fault accordingly.
Additionally, the attribute-based approach allows for validation. As part of the IServiceBehavior implementation, I can now use the Validate method to make sure that the operation on which the attribute has been placed indeed exposes an appropriate fault contract.
I submitted the final version of this approach to CodeProject, including code and significantly more detailed introduction and explanations. Read it there: WCF Error Handling and Fault Conversion.
(I've also used the opportunity to rework the code that detects the current operation. There's in fact a clean way of doing it using the WCF operation context API: Find the current endpoint, ask it for the dispatch operations and find the one matching the current action, extract the operation description from the contract on the endpoint.)
I've discussed dump analysis on several occasions, including detailed walkthroughs of native and managed issues - crashes, hangs, deadlocks and the rest of the bug zoo.
(Alon has just recently been to this bug zoo and promised me pictures.)
Oftentimes it's useful to generate a dump programmatically. For example, if your application fails and you want to record the data and send it over the network or persist it to some durable storage, you can do so without relying on Windows Error Reporting or other mechanisms.
[DllImportAttribute("dbghelp.dll")]
[return: MarshalAsAttribute(UnmanagedType.Bool)]
private static extern bool MiniDumpWriteDump(
[In] IntPtr hProcess,
uint ProcessId,
SafeFileHandle hFile,
DumpType DumpType,
[In] IntPtr ExceptionParam,
[In] IntPtr UserStreamParam,
[In] IntPtr CallbackParam);
The way to go is the MiniDumpWriteDump function, exposed from dbghelp.dll. I've written a simple helper class for using this API from C#, which can be downloaded from here. Sample usage would be:
DumpHelper.WriteTinyDumpForThisProcess(
@"D:\temp\mini.dmp");
Process process = Process.Start("notepad.exe");
process.WaitForInputIdle();
DumpHelper.WriteFullDumpForProcess(
process, @"D:\temp\full.dmp");
process.Kill();
In the previous posts in this series, we have observed a couple of weird behaviors around Workflow Services, specifically:
Both of these behaviors have been confirmed.
The suggested workaround for the one-way issue is to release the WCF thread on the service side (by using ThreadPool.QueueUserWorkItem or any similar API).
The race condition scenario is more complicated that I thought. The reason for the problem is that when the WCF message reaches the workflow, the WorkflowOperationInvoker ultimately uses WorkflowInstance.EnqueueItemOnIdle to enqueue the pending work. This works perfectly if there's no idle point between the send activity and the receive activity:
However, if a delay activity is introduced, then the delay potentially causes the workflow to idle before the receive activity has created a queue for the incoming message. As a result, the message is swallowed.
The workaround, as I suggested in the original post, is to ensure the receive activity is scheduled before the workflow goes idle. The parallel paradigm with the receive activity on the left-most branch and the rest of the work in the other branches will ensure this happens at runtime:
I love showing people the power of declarative, designer-driven programming when I first introduce them to Workflow Foundation. The workflow designer is a feast for the eyes:
One of the first questions people always ask me is whether the workflow can be debugged just as easily as a bunch of cooperating components. The answer, as usual, is: It Depends.
Theoretically, you can always place a breakpoint on an activity and when the workflow executes then execution will stop at the breakpoint:
However, this doesn't always work. Specifically, it works in a simple project but then stops working when you really need the debugging support. Specifically, when you're loading the workflow assembly using Assembly.LoadFrom (it's not one of the most friendly APIs out there, by the way).
Workaround? Well, if it's a CodeActivity, put a breakpoint in the method being called. If it's a SendActivity, put a breakpoint in the BeforeSend event handler. You get the idea.
After a long period of anticipation, all session recordings from this year's TechEd Israel are now online! You can choose between a streamed online WMV and a zipped download for viewing offline.
My Next Generation Production Debugging session is there, and I remind you that if you've missed anything in the slides or the live recording, then you should take a look at the post series, featuring a complete walkthrough of what I've shown during the session:
- Demo 1 - What can be done without a debugger
- Demos 2-3 - Production debugging basics and memory leaks
- Demo 4 - Deadlock analysis
- Demo 5 - Disentangling invalid handles
- Demo 6 - A performance surprise
I intend to watch most of the sessions in the Development Track, but here are some of the sessions I already had a chance to see that I strongly recommend:
- Alon Fliess on ORPC and Services - this session is a must for architects and developers looking at integrating complex systems together, and constructing applications that will survive more than just a couple of years in our ever-emerging ecosystem.
- Tomer Shamam on WPF Data Binding - frankly, I'm very far from being a WPF expert, but I liked this lecture a lot: it shows how easy it is to enable amazing things with WPF, and it shows that the underlying framework has been well-designed, and it serves an example for framework developers.
- Matt Winkler on Workflow Services and Durable Services - unfortunately, the session recording is still unavailable, but you can take a look at the slides; this was a great introduction session to Workflow Services.
- Dave Probert on Windows Server 2008 Kernel Architecture - an excellent session highlighting the interesting software and hardware advances in the new OS; some of these things (like specific optimizations for NUMA hardware) haven't been mentioned before.
This is also a great opportunity to say that I am very impressed with the quality of the recordings. It might appear that I am biased because the company producing these videos is Sela Group, the company I work for. However, I strongly recommend that you be the judge and take a look at the recordings. The video and audio quality is incredible, even over a low-bandwidth connection. Considering the technological challenge of producing over 150 sessions and recording over 10 sessions concurrently, the result is impressive in global standards. (And I've watched enough screencasts and session recordings from all kinds of events around the world.)
Just a quick post to let you know that I've published two articles outside the blog, so if you're interested:
I've written an article (in Hebrew) that serves as an introduction to Workflow Services (article direct link). It has been published in the May 2008 edition of the MSDN Pulse, a Microsoft Israel online digest for developers. In the article I describe what a workflow, a service and a workflow service is, mention a few design patterns and limitations, offer an overview of future directions, and wrap it up with a few pointers for you to start exploring Workflow Services in your code.
I've also written an article on the acute subject of reading unmanaged data into managed structures (article direct link). It has been published on the Code Project website. This is the kind of problem we are all doomed to stumble upon at some point, whether it's for reading a network packet into a C# structure, or trying to parse some weird binary-encoded pattern into something accessible from code. In the article we'll examine multiple approaches to the problem from the performance perspective, as we attempt to come up with a solution that is both generic and fast. The BinaryReader class, pointers in C#, the Marshal class and C++/CLI are some of the things we'll be looking at.
Windows Workflow Foundation has rich support for rule-driven work. The built-in PolicyActivity, for example, features the ability to take a set of rules and execute them. Other activities can make use of the RuleSet, Rule, RuleAction and RuleExpressionCondition types to leverage policy-making facilities inside or outside a workflow. In a nutshell:
- A rule condition is a condition that can be evaluated to a boolean value. It features validation and cloning facilities, as well as a dependency management mechanism.
- A rule action represents an action to be executed.
- A rule is a collection of conditions that is grouped into an IF...THEN...ELSE flow mechanism. The IF clause is associated with the conditions, and the THEN and ELSE clauses are associated with rule actions.
- A rule set is a collection of rules that can be executed and validated as a group.
The Existing Design-Time Experience
As far as the design-time experience is concerned, the primary means of defining rules is the RuleSetDialog and RuleConditionDialog classes. Wherever a rule or a rule condition is involved as a property in an activity or a workflow, the design-time editors for the rule or rule condition use these dialogs. For example, here's the PolicyActivity design-time experience. First, you are prompted to choose a rule set reference or to create a new one; next, prompted to configure a rule set by adding rules to it:
And finally, prompted to configure the rule condition, the THEN actions and the ELSE actions:
If all you need is a single rule condition in your code, that you will use for making some kind of runtime decision, then invoking the whole RuleSet to Rule to RuleCondition design-time chain is a serious overkill. Instead, you're probably looking for something like the following dialog:
In order to actually have a choice, you will need to provide an editor capable of displaying the UI of your choice. The built-in editors (such as the RuleSetDefinitionEditor) are sealed, and therefore a different way must be pursued. Let's take a look at how this would be accomplished.
Customizing Your Design-Time Experience
Assume that I have an activity that defines a rule condition. This rule condition then has to be passed to an entity that is external to my workflow, so that it can be evaluated against some mediating context object. This sounds very general, but it's an enabler for very useful scenarios:
- Define a credit authorization condition and pass it to a credit authorization external service;
- Define a filter for messages that you are interested in and pass it to a routing service;
- Define a subscription predicate for events you want to register for and pass it to a publish/subscribe component . . .
What we need, then, is the ability to store a rule expression condition, and we need to provide an editor that will invoke the rule condition dialog with the context object of our choice. Since the context object could be anything, we're looking at a generic solution. First, we need a wrapper for a RuleExpressionCondition:
[Serializable]
[DesignerSerializer(typeof(MyCodeDomSerializer),
typeof(CodeDomSerializer))]
public abstract class EditableRuleCondition
{
public override string ToString() . . .
public RuleExpressionCondition Condition
{
get;
set;
}
}
Next, we need a generic editor for that wrapper class: (Some implementation omitted for brevity)
public abstract class EditableRuleConditionEditor<
TContext, TCondition> : UITypeEditor
where TCondition : EditableRuleCondition, new()
{
public override object EditValue(...)
{
CodeExpression expression = . . .;
RuleConditionDialog dlg =
new RuleConditionDialog(typeof(TContext),
null, expression);
service.ShowDialog(dlg);
RuleExpressionCondition condition =
new RuleExpressionCondition(dlg.Expression);
return new TCondition {
Condition = condition };
}
}
The beauty of this solution is that now all we need to do is derive from EditableRuleCondition and EditableRuleConditionEditor (which is pretty straightforward) and we have a specialization for our own context type which we can directly use in any activity:
public class CreditCardAuthorization
{
public string CardId { get; set; }
public string Owner { get; set; }
}
[Serializable]
[Browsable(true)]
[Editor(typeof(SampleConditionEditor),
typeof(UITypeEditor))]
public class SampleCondition :
EditableRuleCondition
{
}
public class SampleConditionEditor :
EditableRuleConditionEditor<
CreditCardAuthorization, SampleCondition>
{
}
So let's write a credit card authorization activity that takes this kind of condition and does something external with it. It will expose a simple field that can then be assigned to in the designer (we'll skip validation etc. for today):
public class CreditCardActivity : Activity
{
public SampleCondition CreditCardCondition
{
get;
set;
}
protected override ActivityExecutionStatus Execute(ActivityExecutionContext executionContext)
{
if (CreditCardCondition == null ||
CreditCardCondition.Condition == null)
return ActivityExecutionStatus.Closed;
Console.WriteLine("Condition: " +
CreditCardCondition.ToString());
CreditCardAuthorization creditCardInfo =
new CreditCardAuthorization
{
CardId = "123456789",
Owner = "Sasha"
};
RuleValidation validation =
new RuleValidation(
typeof(CreditCardAuthorization), null);
RuleExecution execution =
new RuleExecution(
validation, creditCardInfo);
if (!CreditCardCondition.Condition.Validate(
validation))
{
throw new ArgumentException(
"Invalid condition.");
}
bool result =
CreditCardCondition.Condition.Evaluate(execution);
Console.WriteLine(
"Condition evaluated to: " + result);
return ActivityExecutionStatus.Closed;
}
}
Note that this activity does all the work internally, but nothing prevents us from taking the CreditCardCondition, serializing it and passing it to an external entity. The workflow instance isn't necessary because the context object isn't the workflow - it's an instance of the CreditCardAuthorization class.
Now, let's put this activity in a workflow:
And let's configure the condition (note the intellisense we get for the context object - the context type here is CreditCardAuthorization):
Our workflow is ready to run, and the condition will evaluate to true because we have set up the credit card number to begin with 1234.
Repeating the process for a rule set editor is quite similar, and is left as an exercise for the reader.
The sample code above can be downloaded as a Visual Studio 2008 solution. Bear in mind that it's sample code, and lots of validation logic has been omitted.
In the previous part of this open-ended series, we have looked at a strange case where asynchronous calls do not always behave asynchronously.
This time, we will look into an inherent race condition that can occur in a workflow service which communicates with the outside world in a duplex fashion.
If your workflow wants to communicate with an external WCF service in a duplex fashion, you'll very quickly find that you need a SendActivity to send a message to the outside world, and a ReceiveActivity to receive the reply. Arranging these activities in your workflow, you might come up with the following result:
Everything works perfectly, and you are convinced that this is the way to go about implementing a duplex workflow service.
Dramatic suspense.
Now imagine that between the send and the receive operations there is some additional processing that you might require. For example, you might add a code activity that emulates some processing:
Everything still works perfectly. Even if the external service wants to talk to the workflow before the code activity completes, the message is queued and processed when the receive activity executes.
Another dramatic suspense.
Unless, of course, you introduce a DelayActivity into the workflow:
What does he want, you might ask. What's the difference between a delay activity and just another code activity that does some processing? Well, apparently there is a difference.
What happens now is that if the external service sends a message to the workflow while the workflow is inside the delay activity, the message is transparently swallowed and lost. It never reaches the ReceiveActivity. Yes, never. And there is no exception to let you know this happened.
You can reproduce this curious scenario yourself by downloading the sample Visual Studio 2008 solution, enabling the DelayActivity and seeing for yourself that the workflow never completes if the delay activity is running while the external message is being enqueued.
Workaround? Let's consider what we need to happen. We need the receive activity to be executing already when the workflow is going into the delay activity. How can we do that? We need a ParallelActivity. The left branch (executed first) will be the ReceiveActivity, and the right branch (executed next) will be the SendActivity and all the rest:
And now all of the sudden everything falls in its place, and the receive activity receives the message after the delay activity completes.
Since you can never be sure (unless you're writing a very very simple workflow) that you won't end up having a DelayActivity before the receive operation, you should always use this paradigm to write a duplex workflow service.
In the previous part of this open-ended series, we have discussed the issue of smuggling external transactions into a workflow.
Today, we will look into a strange scenario where allegedly asynchronous calls are not actually executed asynchronously.
The workflow services samples distributed in the MSDN and as part of the Visual Studio 2008 training kit almost always use synchronous communication. However, it is often the case that you want fire-and-forget behavior within a workflow service calling an external operation. In that case, you will merrily put IsOneWay=true in your [OperationContract] attribute, and move on.
However, the operation will only be executed asynchronously if the external service uses the BasicHttpBinding. With all other bindings I tried*, the operation is not actually invoked asynchronously. What happens is that only when the work on the external service's end is completed, the SendActivity returns and the workflow can continue executing.
To demonstrate this, I used the following simple workflow that calls out to a WCF service:
The first code activity sets up the parameters to the service call. The SendActivity (called DuplexOperation here) calls the WCF service. The second code activity outputs a string to the console to indicate that the SendActivity has completed. The WCF service on the other end issues a Thread.Sleep, and has the following (one-way) interface:
[ServiceContract]
public interface IMyService
{
[OperationContract(IsOneWay = true)]
void AsyncOperation(TimeSpan sleepSpan);
}
Unless the WCF service is using the BasicHttpBinding, the SendActivity doesn't return and the code activity doesn't get a chance to execute until the operation on the service side completes the Thread.Sleep call and returns. This is easily observable by running the workflow and checking your wrist watch for the time it takes for the second code activity to output something to the console.
The sample code used in this post can be downloaded as a Visual Studio 2008 solution (10KB).
* To be fair, I didn't try the bindings that do not work with the context channel, such as the MSMQ binding. But hey, they do not work at all, synchronously or asynchronously :-)
In this open-ended series of posts, I would like to outline some limitations of the current Workflow Services implementation (.NET 3.5). The information presented here is the result of several months' work designing, prototyping and implementing an application framework on top of Workflow Services.
Please note: Some of these issues have been non-officially confirmed by Microsoft, some others are pending - so you should treat all of this with a grain of salt. None of this substitutes the existing Microsoft documentation.
If you're looking for a general overview of what Workflow Services are about, the following resources might be useful:
Workflows and External Transactions
In our architecture, there is a distributed transaction that starts at a node outside the workflow. This node then talks to the workflow, which has a ReceiveActivity to handle the request.
We need to flow the transaction from the remote node to the workflow instance. WCF gives us the necessary infrastructure for flowing the transaction, but the ReceiveActivity does not. The code of the activity and the parameter bindings are executed outside the transaction, without any transaction context, regardless of the transaction that flows with the WCF service.
This is in sync with the MSDN documentation, which clearly states that it’s impossible for a workflow to participate in a transaction that has started outside the workflow. For example, the default implementation for the WorkflowCommitWorkBatchService explicitly removes any ambient transaction that does not originate from within the workflow.
We have tried several ways to work around this:
- Provide an IDispatchMessageInspector extension which checks incoming messages for the TransactionMessageProperty and sets the ambient transaction if a transaction is present. This doesn’t work because the ambient transaction created in this step doesn’t flow into the workflow. (It’s either explicitly removed or simply not flowed by the scheduler service.)
- Provide a custom workflow scheduler service which schedules a thread for activity execution but first sets the ambient transaction if necessary. This doesn’t work because the workflow runtime created by the WorkflowServiceHost has an explicit validation behavior in place (in the WorkflowRuntimeBehavior) which makes sure that the scheduling service is an internal class called SynchronizationContextWorkflowSchedulerService.
This has been an interesting exercise in extending WCF and workflow, but these attempts failed. The workflow runtime itself uses a TransactionScope to suppress any external transactions.
What's Possible?
The only supported way of flowing a transaction into the workflow instance is during the unload (persistence point) of the instance. Here is the sequence of steps necessary to perform it:
- Within a transaction, suspend the workflow instance.
- Within a transaction, enqueue a message on the workflow instance.
- Within a transaction, call Unload on the workflow instance. The workflow runtime will use the ambient transaction to persist and unload the workflow.
- Commit the transaction.
- Resume the workflow instance. (This happens outside the transaction.)
This guarantees that the persistence and unloading of the workflow instance occur within the external transaction. This is the perfect solution for durable messaging, because it can be used to transactionally enqueue an external message onto the workflow and persist the workflow, so that the message becomes durable.
Workaround and What's Impossible
However, this does not provide the mechanism necessary for performing the work inside the workflow within the external transaction. From the moment the workflow has been persisted, the transaction is irrelevant. The next time the workflow is loaded and continues execution, it no longer participates in the external transaction.
In view of this limitation, the only applicable workaround is to have the workflow instance initiate the transaction (reversing the transaction root and planting it inside the workflow instance).
For example, instead of having the external node open the transaction and flow it to the workflow instance, the external node can signal the workflow that work is pending, outside a transaction. The workflow will then initiate a transaction and contact the external node within that transaction to pull whatever work is pending and execute the necessary operations within a transaction. This solution is associated with an extra roundtrip necessary to signal the fact that work is pending.
This workaround still doesn't alleviate the scenario where two distinct workflow instances need to communicate within a transaction. When the first workflow calls the second workflow, the transaction will be removed; if the second workflow initiates work within the transaction and calls the first workflow, the transaction will be removed nonetheless.