Debugging Distributed Transactions: Additional Examples

July 29, 2010

no comments

In this final installment we’ll take a look at some additional examples of problems caused by distributed transactions.

The following execution history results in a deadlock. What would you suggest to address it?

Time Transaction 1 (Inventory Service) Transaction 2 (Shipping Service)
0 Select all orders by customer A  
1 Calling Shipping Service, passing to it each order Receive order from Inventory service
2   Select the customer’s shipping address
3   Update the shipping date of the order
4 Receive execution result Return from call
5 Commit Commit

The deadlock occurs at T=3 when Transaction 2 attempts to acquire an exclusive lock on a row that is under a shared lock by Transaction 2. To alleviate the deadlock, the Inventory service might want to commit the transaction before calling the Shipping service (why does this help?), or it could flow the transaction into the Shipping service, effectively unifying the two transactions into one.

Let’s take a look at a more complicated example. In the following execution history, the Inventory WCF service is configured with ConcurrencyMode.Single. The execution history results in a deadlock.

Time Transaction 1 (Order Service) Transaction 2 (Shipping Service) Inventory Service (within caller’s TX)
0 Request in-stock information for an item from the Inventory Service   Receive request from caller
1     Select in-stock information for the item
2 Receive reply from service   Return to caller
3   Select all orders  
4   Update in-stock information in Inventory Service Receive request from caller
5     Update in-stock information for the items in the order
7 Update in-stock information in Inventory Service    

Note that what happens is the following: at T=1, the Inventory service acquired a shared lock on a certain row on Transaction 1’s behalf. At t=4, the Inventory service starts processing a request on Transaction 2’s behalf, but it requires an exclusive lock on the same row that’s currently under a shared lock by Transaction 1. Therefore, the Inventory service blocks. At T=7, the Order service needs to access the Inventory service again, but because of the use of ConcurrencyMode.Single, it can’t get into the service—the Shipping service is already inside. This forms again a deadlock.

The Suspend-Enqueue-Unload-Resume pattern for smuggling transactional work into a WF instance introduces a similar transactional deadlock. [I won’t repeat myself—this has been discussed in an earlier post.]

Finally, a fairly common error when working with distributed transactions occurs when you inadvertently attempt to use a transaction that has already been aborted for some reason.

One place where this can happen is if you call two WCF service operations within the same distributed transaction, and the first operation faults (aborting the transaction). Trying to flow the transaction into the second operation will cause an exception.

Another cause for this error could be a timeout. Your thread might have blocked for a long time, and after waking up it would attempt to use the ambient transaction. If it were already aborted (due to the transaction timeout), an error will occur.

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>