Tpl Dataflow (ISourceBlock) – Part 3

2011/12/10

Tpl Dataflow (ISourceBlock) – Part 3

the previous post discus the concept ITargetBlock
which is the TDF consumer contract.

this post will focus on the source block which is the producer contract.

as mention in previous post, sources and targets engage in a protocol for transferring messages between them.

manufacture_small

Source Block:

the source block main responsibility is to produce (or manipulate) data which will be consume by the target.

as we learn in previous post the target may consume the data either directly (push) or indirectly (pull).

a target can be attached to source using the ISourceBlock<T> LinkTo method :

LinkTo:

link to provide the ability to link one or more targets to a source, this is how we can build a Dataflow network.
when target is linked to sources the sources automatically propagate any data they contain to targets (asynchronously upon the source distribution strategy).

Detaching linked target from a source:

the LinkTo method returns a IDisposable which can be use to detach the target from the source (at nay time).

Dataflow network:

actually we can dynamically change the Dataflow network at runtime while attaching and detaching blocks.

Pull:

When source push data to a target, the protocol employed may allow the target to simply accept and take ownership on the offered data, or it may require the target to communicate back to the source (pull).

see the previous post for more details.

Race condition:

when a target is pulling data out of one or more sources it may have to ensure that the data does not simultaneously consumed by other targets.
it is getting even more complicate when a target try to atomically consume messages from multiple sources (which mean either succeeding to consume all messages on none).

Two-phase commit:

by using the two-phase commit protocol the target can atomically consume multiple messages (from one or more sources).

the ISourceBlock<T> define the following methods in order to enable two-phase commit protocol:

  • ConsumeMeassage (commit)
  • ReserveMessage (prepare)
  • ReleaseMessage (roll-back)
Code Snippet
  1. interface ISourceBlock<out TOutput> : IDataflowBlock
  2. {
  3.     IDisposable LinkTo(
  4.         ITargetBlock<TOutput> target,
  5.         bool unlinkAfterOne);
  6.  
  7.     TOutput ConsumeMessage(
  8.         DataflowMessageHeader messageHeader,
  9.         ITargetBlock<TOutput> target,
  10.         out bool messageConsumed);
  11.     bool ReserveMessage(
  12.         DataflowMessageHeader messageHeader,
  13.         ITargetBlock<TOutput> target);
  14.     void ReleaseReservation(
  15.         DataflowMessageHeader messageHeader,
  16.         ITargetBlock<TOutput> target);
  17. }

Two-phase commit in action:

when target want to atomically consume messages from one or more sources:

  1. it try to reserve the messages on each source (the message header has the message identifier).
  2. if it failed to reserve one of the messages, it should release other reservation using ReleaseResarvation (roll back).
  3. if all reservation succeed it can safely consume the messages, using ConsumeMessage (commit).
Consume Message:

even without two-phase commit, the ConsumeMessage design to handle race condition.
when ever we call the consume message, the source block assign the consume message’s messageConsumed out parameter to indicate whether the consume operation has succeed. this way the source can ensure that the message will be consume by a single target and avoid race conditions.

Summary

using the TDF source and target contract we can apply push and pulling strategy while avoiding race condition.

in the following posts I will speak about the blocks house keeping contract and propagating.

Shout it kick it on DotNetKicks.com

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>