Tpl Dataflow - User Group
Today I was lecturing at Microsoft User Group
about the new TPL Dataflow (TDF) library.
first I want to thanks all attendants.
I will post about more aspects of the Tpl Dataflow in future posts.
the presentation and code samples,
available for download from here and here.
download pdf.
download code samples.
Rx Contrib - new release
finally I got the time to update the Rx Contrib with the Rx release bits.
the release contain couple of new features and refactoring.
This a snapshot of what's in the release:
- The ReactiveQueue was refactor to QueueSubject.
- The static Create method overload has changed:
- now it is having a flags which can define it’s behaviors, a-sync and whether to
publish performance counter (currently ETW counter are not enabled),
It also can be configure to run synchronously (for debug purpose).
- New dll (System.Reactive.Contrib.Profile.dll) is adding real-time profiling capabilities.
It comes with TraceViewer (System.Reactive.Contrib.TraceViewer.exe) which will present the profiling information.
Currently the profiling data transferred its datum over WCF queue binding (in the future it will use ETW).
You can check it out by running the TraceViewr and then run the
System.Reactive.Contrib.TestProfile.exe (under the test folder).
- Buffer with time and count were renamed to BufferSafe.
It is a thread safe buffer (the out of the box buffer is not thread-safe and you
should use the Synchronize before the Buffer if the buffer input come from multiple threads),
for example:
Observable.Interval(…).ObserveOn(Scheduleer.TaskPool).Synchronize().Buffer(10); - Finally the motivation to use QueueSubject instead of the standard Subjects is performance.
On intensive data flow scenarios QueueSubject perform up to 30 times faster than
the standard one (this is because it doesn't over subscribe the system and reuse the tasks).
you can check it by running the System.Reactive.Contrib.ConsoleStress.exe
when it finish all of the test it will summarize the result (be aware that when it write Byte[] this mean it has used the standard Subject all other information represent different mode of the QueueSubject).
Summary
you can use parts of the contrib project (like the profiler or buffer).
you can use it with both RX release version v0.1.10621 and v1.1.10621.
download available here.
you can download the code and and see the tests for better understanding of how to use it.
on future post I will discuss the profiler in greater details.
Tpl Dataflow - Part 1
this post is the first of post series which will focus
on the new Tpl Dataflow library (TDF).
TDF goal is to address high-throughput / low-latency
flow scenario of complex computing / IO intensive / immense traffic.
The library is using a few basic buffering and message base pattern in order to
enable basic block which can be compose together into full scale scenario.
each building block construct as agent which have internal buffer and execution management.
one of the most simple block is the ActionBlock:
each time item is send into the ActionBlock it will be buffer in
the internal buffer, than the Block Task will take
one item at a-time and will process it.
by default the ActionBlock schedule only single Task at a-time,
but latter on this series we will see that this default can be change.
actually the Block release Its Task whenever the buffer become empty.
Code Snippet
- var ab = new ActionBlock<int>(i => Console.Write(i));
- Parallel.For(0, 1000, i => ab.Post(i));
as we can see the ActionBlock syntax can be very simple.
it define an action (in the above case Console.Write) and it can fed with data, ab.Post(i).
Summary
ActionBlock is one of the simplest block.
in the future post we will survey other blocks, TDF interfaces, blocks overloads and concepts,
we will see how to chain blocks in a row, and we will
discuss some of the architecture decisions.
Point Of Interest:
My lecture on this topic at Microsoft User group has change to 2011-08-17.
you can register using this link.
