January 2010 - Posts
I thought I would share a nice small tweak I made to a LINQ to SQL model.
Assume I have the following table structure for ‘Categories’:
I faced with the need to select categories while ordering it by the combination of Description1 and Description2 (Note: nullable fields).
I entered some dummy data to the table to illustrate the point:
The goal is to retrieve the order as follows: 1, 3, 5, 2, 4
After explaining that, lets forward to the LINQ statements then.
Take 1 – Simple Aggregation
var query = from c in Categories
orderby c.Description1 + c.Description2
select c;
query.Dump();
Result:
That is not the desired result and that’s ok, it was expected.
Lets look at the generated SQL statement:
SELECT [t0].[Id], [t0].[Name], [t0].[Description1], [t0].[Description2]
FROM [Categories] AS [t0]
ORDER BY [t0].[Description1] + [t0].[Description2]
The way SQL aggregation works is in case it encounters NULL in one of the aggregated fields, the entire aggregated expression is NULL.
That is why we get this order.
To make things short, we need this kind of SQL statement:
select * from categories
order by IsNull(description1, '') + IsNull(description2, '')
That will give us the right result, which is:
So let’s get that result already..
Take 2 – Null Consideration
var query = from c in Categories
orderby (c.Description1 == null ? string.Empty : c.Description1) + (c.Description2 == null ? string.Empty : c.Description2)
select c;
query.Dump();
Good news – that yields the results I intended for.
However, the generated SQL isn’t quite what I aimed for -
-- Region Parameters
DECLARE @p0 NVarChar(1) SET @p0 = ''
DECLARE @p1 NVarChar(1) SET @p1 = ''
-- EndRegion
SELECT [t0].[Id], [t0].[Name], [t0].[Description1], [t0].[Description2]
FROM [Categories] AS [t0]
ORDER BY (
(CASE
WHEN [t0].[Description1] IS NULL THEN CONVERT(NVarChar(500),@p0)
ELSE [t0].[Description1]
END)) + (
(CASE
WHEN [t0].[Description2] IS NULL THEN CONVERT(NVarChar(500),@p1)
ELSE [t0].[Description2]
END))
You can see that LINQ to SQL translated it into the piece of order-by statement which is acceptable from the LINQ query I wrote, it did get me the results I needed too.
Still, I was looking for using the built-in SQL IsNull function as seen in take 1 as my final goal.
Take 3 – Using IsNull Function
In order to use the “IsNull” function, I need to define it.
I added the method to the generated Data Context class through the partial class paradigm -
public partial class LocalTestDataContext
{
[Function(Name = "IsNull", IsComposable = true)]
[return: Parameter(DbType = "NVarChar(MAX)")]
public string IsNull(
[Parameter(Name = "field", DbType = "NVarChar(MAX)")] string field,
[Parameter(Name = "output", DbType = "NVarChar(MAX)")] string output)
{
return ((string)(this.ExecuteMethodCall(this,
((MethodInfo)(MethodInfo.GetCurrentMethod())),
field, output).ReturnValue));
}
}
Then I can go on and use that function as part of my LINQ queries -
var ctx = new LocalTest.LocalTestDataContext();
var query = from c in ctx.Categories
orderby ctx.IsNull(c.Description1, "") + ctx.IsNull(c.Description2, "")
select c;
query.Dump();
..And that’s the way the cookie crumbles, got my ordered results using the built-in IsNull function.
The routing service that ships with WCF4 beta 2 is a very useful layer for implementing basic standard router facade which provides virtualization, versioning, protocol bridging, windows identity impersonation, transaction propagation and routing capabilities for your back-end services.
Unfortunately, I’m afraid that it has a lot to advance in order to be usable for advanced more-sophisticated layer as far as it goes for implementing an actual management layer for message brokering and a policy enforcement point.
Following are links to my previous posts concerning the routing service:
WCF 4 – Routing Service, The Beginning
WCF 4 Routing Service – Going Basic
WCF 4 Routing Service – Supported Bindings
WCF 4 Routing Service – O/W Operation and Corresponding Channel Shapes
WCF 4 Routing Service – Resolve Endpoint in Runtime
WCF 4 Routing Service – Runtime Governance
WCF 4 Routing Service as a PEP (Policy Enforcement Point)
Concluding failure points discussed in previous articles:
- WebHttp isn’t supported as much as I could tell
- I didn’t succeed with calling a O/W operation on a back-end service exposed with WsHttpBinding and Reliable Session enabled
- I didn’t find a way to control the contract of the channel shape used by the router service down the chain when calling to the back-end service. That prevented me from controlling the invocation pattern if I have more information regarding the target back-end service and operation.
- I didn’t find a way to get a hold of the reply messages in duplex communication mode in order to apply my custom logic such as logging the messages passed through it in runtime.
- The router service is difficult to extend, having control over its invocation process is extremely limited, making it impossible to utilize it as a PEP.
This will be the last feature requirement I plan to check with the router service of WCF 4.
One of the core purposes for developing the router facade here at my workplace was to have our own layer where we could implement and ensure any kinds of standards/policies.
Few examples include encryption, SLA, authorization, logging, notifications. The list goes on.
These things require me to have control over the request and reply messages as well as the invocation process flow.
Let’s take a simple authorization for example -
Let’s assume we have a simplified authorization module we would like to implement in our router facade.
The router will determine if the client’s identity can call the desired operation (according to the message action) against some IDM and Permissions management layer. If the client is authorized, the router should flow the message to the back-end service. Otherwise, it should return an “access denied” security fault message to the client and never invoke the back-end service.
First thing first, intercepting the request message can be done using a message inspector.
Though what then?
If I validate the request and find that it is unauthorized, how can I cancel the invocation process and return a fault?
This is problematic in the current design of the routing service, which is the fifth point of failure.
Throwing a security exception during the “AfterReceiveRequest” in the authorization inspector doesn’t map to the appropriate access-denied fault when returned to the client. (it is discovered as a general Fault Exception)
Moreover, when dealing with duplex contracts, it is resolved as a different kind of exception (CommunicationException rather than FaultException)
The different behavior is due to the fact that duplex channels behave differently.
Exceptions should be created as fault messages and as any other message, needs to be sent over to the client through the duplex channel. This is why simple exception throwing doesn’t yield the same result, it raises a communication exception in this case.
Oddly enough, I did find that if you throw a SecurityException from a custom filter table that you implement it does map to the appropriate access-denied security fault back to the client, not sure it would work with duplex contracts though, haven’t tried that.
Concluding failure points so far:
- WebHttp isn’t supported as much as I could tell
- I didn’t succeed with calling a O/W operation on a back-end service exposed with WsHttpBinding and Reliable Session enabled
- I didn’t find a way to control the contract of the channel shape used by the router service down the chain when calling to the back-end service. That prevented me from controlling the invocation pattern if I have more information regarding the target back-end service and operation.
- I didn’t find a way to get a hold of the reply messages in duplex communication mode in order to apply my custom logic such as logging the messages passed through it in runtime.
- The router service is difficult to extend, having control over its invocation process is extremely limited, making it impossible to utilize it as a PEP.
One of the goals for building the router facade at my workplace was to make it an essential runtime agent for registering runtime data to the governance realm I implemented.
The router sends meta-data of all the runtime messages that goes through it over to the governance registration service queue.
Basically, this means that I need a way to intercept every request and reply.
I couldn’t find a complete solution for that in the current WCF 4 Routing Service, which is the fourth point of failure.
The best way that I found to go about it was attaching my custom IDispatchMessageInspector to get a hold of every request, that works fine.
Unfortunately, not every reply goes through that layer.
For the duplex contract, which you should use for Net TCP binding and duplex communications, the reply message never goes through it.
This is due to the fact that in duplex communications, the reply messages are pushed directly to the communication channel of the caller, this is done by getting a hold of the callback contract and pumping the messages down to the client.
The problem is that there is no way to extend this behavior of the routing service to perform your own custom logic there, making it impossible to use it as a runtime governance agent with duplex communications as my need is.
Concluding failure points so far:
- WebHttp isn’t supported as much as I could tell
- I didn’t succeed with calling a O/W operation on a back-end service exposed with WsHttpBinding and Reliable Session enabled
- I didn’t find a way to control the contract of the channel shape used by the router service down the chain when calling to the back-end service. That prevented me from controlling the invocation pattern if I have more information regarding the target back-end service and operation.
- I didn’t find a way to get a hold of the reply messages in duplex communication mode in order to apply my custom logic such as logging the messages passed through it in runtime.
Entity Framework has made great progress towards being a popular ORM framework.
EF 4 is much more practical and usable version than its previous one.
There’s a nice post which debates about the comparison between NHibernate and EF4:
Debate: Comparing NHibernate and EF 4
“The general consensus appeared to be that NHibernate has a number of features which are missing from Entity Framework 4.0 like read/write batching, "extra" laziness, collection filters, tweaking, and others, however Entity Framework has a better LINQ provider, documentation and it is supported by Microsoft”
The WCF Routing Service comes with a nice set of configuration for defining the client endpoints and the filters to map them for a single request.
You can define filters, be that on the message headers or content (using XPath) in order to select the appropriate endpoint to the back-end service.
I was looking for a way to determine this in runtime. The router facade I built uses information from the service registry to select the endpoint information for communicating with the back-end service.
This means that I can’t follow the standard way of defining this in configuration since this information is dynamic and resolved in runtime.
I didn’t find a way to plug-in my own behavior to affect the routing service endpoint selection in the manner of an extensibility point for returning an actual service endpoint from the current context.
That’s a shame but there’s still a different way to do that.
WCF 4 samples include a sample showing how to reconfigure the routing in runtime.
In general, the way to go about it is reconfiguring the routing behavior whenever needed, you may want to look into implementing your own custom message filter as well.
To summarize this post, although the routing service isn’t built in a very extensible manner, resolving endpoint information in runtime is still possible.
O/W operations require to be invoked through the ISimplex contracts which represent O/W channel shapes.
When I say O/W operations, I mean that the channel shape reflects that too.
E.g. TCP protocol is by nature a duplex channel, you should invoke O/W operations through IDuplexSessionRouter contract.
I couldn’t manage to call a O/W operation for a back-end service which is exposed with WsHttpBinding with Reliable Session enabled! That is the second point of failure.
In addition, At my router facade I have rich meta data regarding the target back-end service and operation by managing information from the service registry in an optimized manner.
This means that according to the ‘To’ header and the message ‘Action’, I can determine if the target operation is one-way or not in runtime.
Unfortunately, I didn’t find a way to intervene and control the channel shape that the router service uses to call the back-end service with.
This means that I still have to expose the relevant contract for each endpoint in the router service without having the ability to control it down the calling chain, that is the third point of failure.
That’s just too bad, this means that the client needs to talk to different endpoints if the operation itself requires O/W or not. if there’s a service implemented with a contract that has both R/R and O/W operations, the client can’t call it using the same proxy. (I guess you can manage the address of the proxy each call to direct it to a different address but that brings up a whole other set of issues)
Concluding failure points so far:
- WebHttp isn’t supported as much as I could tell
- I didn’t succeed with calling a O/W operation on a back-end service exposed with WsHttpBinding and Reliable Session enabled
- I didn’t find a way to control the contract of the channel shape used by the router service down the chain when calling to the back-end service. That prevented me from controlling the invocation pattern if I have more information regarding the target back-end service and operation.
I will continue with the series of posts where I will discuss what I believe to be missing furthermore. However, I do know for sure that I can’t be using this as the base implementation for our base router facade instead of the one I had written which does support all of these things.
For starters, I had to examine if the router service supports all the common bindings we’ve been using in our enterprise.
The following worked just fine:
- BasicHttp
- Http Protocol, Text Encoding and Soap12 Message Version (part of the WS-I Basic Profile)
- WsHttp
- Standard
- Reliable Session (instead for O/W – more at the next post)
- NetTcp
- Standard
- Reliable Session
For the relevant bindings above, sessionful contracts were preserved and transactions were propagated – as long as I configured everything correctly.
Unfortunately, I didn’t get WebHttp (Implementing RESTful services) to work through the router service, I didn’t dive into all the bits and bytes so perhaps there is a way to somehow make it work but I didn’t find a way to do that so far. This is the first point of failure.
Concluding failure points:
- WebHttp isn’t supported as much as I could tell
The routing service that comes with WCF 4 enables you to make a single up-front layer for accessing back-end services.
The basic concept is that you define endpoints for your router service to be accessible by consumers as well as the internal client endpoints the router service will call the back-end services with.
This shields clients from having to know about the physical findings of the back-end services, allowing you to implement useful things such as versioning, virtualization, protocol bridging, routing, PEP and back up lists (call a different service if the main one is down)
In addition, the routing service supports preserving sessions, impersonating a client's windows token to delegate the client credentials and transaction propagation.
The endpoints that you expose in your router service are defined with the channel shape that you desire.
- ISimplexDatagramRouter / ISimplexSessionRouter – Represents one-way operation invocations. (the session router requires a sessionful channel)
- IRequestReplyRouter – Represents R/R operations
- IDuplexSessionRouter – Represents duplex communication channel. The channel defines a callback contract for pushing response messages to the channel of the caller
Soap Processing Mode:
The routing service can be configured to have soap processing enabled which makes the protocol bridging possible.
This approach makes the router service as a processing service rather than a simple pass-through service which simply forwards the incoming message to the physical service.
If you wish to disable soap processing mode, make sure the communication pattern is compatible with this mode.
For example, standard NetTcp binding is supported but NetTcp with reliable session enabled is not.
I have decided I had enough time to “rest” while being swamped with work and personal agenda over the last few months, it’s time to get back on the saddle.
I mentioned it some time ago, but one of the core projects I have been working on is building a smart router facade, a sort of a message broker, providing versioning, virtualization, PEP and so on.
WCF 4 introduces us with a nicely wrapped router service OOTB.
I was on my way to try to determine if I can use that service as the base implementation for the router facade that I had written.
The built-in Routing Service sounds cool on paper, with useful features. (protocol bridging, filters, routing, back-up list) However, the router I had written does have necessary things that I’m not sure the built-in new routing service can provide.
For starters:
1) Is that usable in terms of non-simple-static-config-plain-physical services? (E.g. load client endpoints from service registry/discovered endpoints)
2) Can the router service handle things dynamically and support most common communication patterns?
3) Is the router service extensible and pluggable enough for you to tweak stuff down the chain?
..
I plan to write a series of posts that demonstrate if and how the built-in router can provide technical answers to the questions above.
I had a small glimpse yesterday and I’m afraid it’s not a perfect "Yes it can”, I will get to that.